Skip to contentSkip to conversation

builddeb: make deb building more flexible

PATCH Superseded [email protected] 2 participants 12 messages Apr 11, 2021 – Apr 19, 2021 Kbuild

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover builddeb: make deb building more flexible

From: Bastian Germann <[email protected]>

Building Debian packages via the [bin]deb-pkg make targets has some shortcomings. These targets do not allow for packing a Debian source package independently from actually building the binary package.

This series improves that process by making it possible to rebuild the packages without errors via dpkg-buildpackage, by separating source package creation from binary package building, and by making the dbg package build optional.

Bastian Germann (6):
builddeb: diff-ignore unexported top-level files
builddeb: set CC on cross build to prefixed gcc
builddeb: clean generated package content
builddeb: use standard format for copyright file
builddeb: introduce profile excluding the dbg pkg
kbuild: introduce srcdeb-pkg target

 scripts/Makefile.package |  8 ++++++--
 scripts/package/mkdebian | 40 ++++++++++++++++++++++++++--------------
 2 files changed, 32 insertions(+), 16 deletions(-)
Signature · 1 line
2.30.2
#1 lore raw

PATCH 1/6 builddeb: diff-ignore unexported top-level files

From: Bastian Germann <[email protected]>

scripts/Makefile.package's TAR_CONTENT lists the files to include in orig tarballs while the deb-pkg make target only ignores .git. This results in the other top-level files ending up in the .diff.gz.

Let dpkg-source ignore .git with the default diff-ignore and list the other top-level files in extend-diff-ignore. Use the debian/source/options to always have those available on building the package.

---
 scripts/Makefile.package | 2 +-
 scripts/package/mkdebian | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
scripts/Makefile.package +1 -1
index f952fb64789d..280f3a2fa334 100644
@@ -75,7 +75,7 @@ deb-pkg:
 	$(call cmd,src_tar,$(KDEB_SOURCENAME))
 	origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
 		mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
-	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -i.git -us -uc
+	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -us -uc
 
 PHONY += bindeb-pkg
 bindeb-pkg:
scripts/package/mkdebian +2 -0
index 60a2a63a5e90..3d2d4b033e44 100755
@@ -134,6 +134,8 @@ fi
 
 mkdir -p debian/source/
 echo "1.0" > debian/source/format
+echo diff-ignore > debian/source/options
+echo 'extend-diff-ignore = ".clang-format|.cocciconfig|.config.old|.*ignore|.mailmap|.version|CREDITS|MAINTAINERS|README"' >> debian/source/options
 
 echo $debarch > debian/arch
 extra_build_depends=", $(if_enabled_echo CONFIG_UNWINDER_ORC libelf-dev:native)"
Signature · 1 line
2.30.2
#2 lore raw

Re: [PATCH 1/6] builddeb: diff-ignore unexported top-level files

Quoted 35 lines · On Sun, Apr 11, 2021 at 7:14 PM <[email protected]> wrote:

From: Bastian Germann <[email protected]>

scripts/Makefile.package's TAR_CONTENT lists the files to include in orig tarballs while the deb-pkg make target only ignores .git. This results in the other top-level files ending up in the .diff.gz.

Let dpkg-source ignore .git with the default diff-ignore and list the other top-level files in extend-diff-ignore. Use the debian/source/options to always have those available on building the package.

---
 scripts/Makefile.package | 2 +-
 scripts/package/mkdebian | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
scripts/Makefile.package +1 -1
index f952fb64789d..280f3a2fa334 100644
@@ -75,7 +75,7 @@ deb-pkg:
        $(call cmd,src_tar,$(KDEB_SOURCENAME))
        origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
                mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
-       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -i.git -us -uc
+       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -us -uc
 
 PHONY += bindeb-pkg
 bindeb-pkg:
scripts/package/mkdebian +0 -0
index 60a2a63a5e90..3d2d4b033e44 100755

scripts/package/mkdebian fi · line 138

Quoted 5 lines
scripts/package/mkdebian +2 -0
@@ -134,6 +134,8 @@ fi
 
 mkdir -p debian/source/
 echo "1.0" > debian/source/format
+echo diff-ignore > debian/source/options
+echo 'extend-diff-ignore = ".clang-format|.cocciconfig|.config.old|.*ignore|.mailmap|.version|CREDITS|MAINTAINERS|README"' >> debian/source/options

Please add
.clang-format, .cocciconfig, .mailmap, CREDITS, MAINTAINERS, README to TAR_CONTENTS.

These are real source files.

Quoted 6 lines
scripts/package/mkdebian +0 -1
 echo $debarch > debian/arch
 extra_build_depends=", $(if_enabled_echo CONFIG_UNWINDER_ORC libelf-dev:native)"
--

2.30.2

Signature · 2 lines
Best Regards
Masahiro Yamada
#3 lore raw

Re: [PATCH 1/6] builddeb: diff-ignore unexported top-level files

Quoted 47 lines · Masahiro Yamada
Quoted 35 lines · On Sun, Apr 11, 2021 at 7:14 PM <[email protected]> wrote:

From: Bastian Germann <[email protected]>

scripts/Makefile.package's TAR_CONTENT lists the files to include in orig tarballs while the deb-pkg make target only ignores .git. This results in the other top-level files ending up in the .diff.gz.

Let dpkg-source ignore .git with the default diff-ignore and list the other top-level files in extend-diff-ignore. Use the debian/source/options to always have those available on building the package.

---
 scripts/Makefile.package | 2 +-
 scripts/package/mkdebian | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
scripts/Makefile.package +1 -1
index f952fb64789d..280f3a2fa334 100644
@@ -75,7 +75,7 @@ deb-pkg:
        $(call cmd,src_tar,$(KDEB_SOURCENAME))
        origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
                mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
-       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -i.git -us -uc
+       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -us -uc
 
 PHONY += bindeb-pkg
 bindeb-pkg:
scripts/package/mkdebian +0 -0
index 60a2a63a5e90..3d2d4b033e44 100755

scripts/package/mkdebian fi · line 138

Quoted 5 lines
scripts/package/mkdebian +2 -0
@@ -134,6 +134,8 @@ fi
 
 mkdir -p debian/source/
 echo "1.0" > debian/source/format
+echo diff-ignore > debian/source/options
+echo 'extend-diff-ignore = ".clang-format|.cocciconfig|.config.old|.*ignore|.mailmap|.version|CREDITS|MAINTAINERS|README"' >> debian/source/options

Please add
.clang-format, .cocciconfig, .mailmap, CREDITS, MAINTAINERS, README to TAR_CONTENTS.

Quoted 2 lines

These are real source files.

.*ignore

is difficult to understand.

Please add .gitignore to TAR_CONTENTS
and .git to diff-ignore.

Signature · 2 lines
Best Regards
Masahiro Yamada
#4 lore raw

PATCH 2/6 builddeb: set CC on cross build to prefixed gcc

In reply to [email protected]

From: Bastian Germann <[email protected]>

Building the generated package with CC unset will fail for cross compilation. Detect that and set CC to a sane default prefixed by dpkg-architecture's DEB_HOST_GNU_TYPE variable.

---
 scripts/package/mkdebian | 8 ++++++++
 1 file changed, 8 insertions(+)
scripts/package/mkdebian +8 -0
index 3d2d4b033e44..2fa149796791 100755
@@ -223,6 +223,14 @@ fi
 cat <<EOF > debian/rules
 #!$(command -v $MAKE) -f
 
+include /usr/share/dpkg/architecture.mk
+
+ifneq (\$(DEB_BUILD_GNU_TYPE),\$(DEB_HOST_GNU_TYPE))
+ifeq (\$(origin CC),default)
+CC := \$(DEB_HOST_GNU_TYPE)-gcc
+endif
+endif
+
 srctree ?= .
 
 build-indep:
Signature · 1 line
2.30.2
#5 lore raw

PATCH 3/6 builddeb: clean generated package content

In reply to [email protected]

From: Bastian Germann <[email protected]>

For each binary Debian package, a directory with the package name is created in the debian directory. Include the generated files in the package's clean target.

---
 scripts/package/mkdebian | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/package/mkdebian +1 -1
index 2fa149796791..717223ef6b03 100755
@@ -246,7 +246,7 @@ binary-arch: build-arch
 	KBUILD_BUILD_VERSION=${revision} -f \$(srctree)/Makefile intdeb-pkg
 
 clean:
-	rm -rf debian/*tmp debian/files
+	rm -rf debian/*tmp debian/files debian/linux-*
 	\$(MAKE) clean
 
 binary: binary-arch
Signature · 1 line
2.30.2
#6 lore raw

Re: [PATCH 3/6] builddeb: clean generated package content

Quoted 26 lines · On Sun, Apr 11, 2021 at 7:14 PM <[email protected]> wrote:

From: Bastian Germann <[email protected]>

For each binary Debian package, a directory with the package name is created in the debian directory. Include the generated files in the package's clean target.

---
 scripts/package/mkdebian | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/package/mkdebian +1 -2
index 2fa149796791..717223ef6b03 100755
@@ -246,7 +246,7 @@ binary-arch: build-arch
        KBUILD_BUILD_VERSION=${revision} -f \$(srctree)/Makefile intdeb-pkg
 
 clean:
-       rm -rf debian/*tmp debian/files
+       rm -rf debian/*tmp debian/files debian/linux-*
        \$(MAKE) clean
 
 binary: binary-arch
--
Quoted 2 lines

2.30.2

Please also remove debian/*tmp.

Add this tag:

Fixes 1694e94e4f46 ("builddeb: match temporary directory name to the

package name")

Signature · 2 lines
Best Regards
Masahiro Yamada
#7 lore raw

PATCH 4/6 builddeb: use standard format for copyright file

In reply to [email protected]

From: Bastian Germann <[email protected]>

Convert the minimal copyright file to the DEP-5 standard format that is commonly used in Debian.

---
 scripts/package/mkdebian | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)
scripts/package/mkdebian +14 -13
index 717223ef6b03..bb5c19735345 100755
@@ -152,22 +152,23 @@ EOF
 
 # Generate copyright file
 cat <<EOF > debian/copyright
-This is a packacked upstream version of the Linux kernel.
-
-The sources may be found at most Linux archive sites, including:
-https://www.kernel.org/pub/linux/kernel
-
-Copyright: 1991 - 2018 Linus Torvalds and others.
-
-The git repository for mainline kernel development is at:
-git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
-
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+Upstream-Name: Linux
+Source: https://www.kernel.org/pub/linux/kernel
+Comment: This is a packaged upstream version of the Linux kernel.
+
+Files: *
+Copyright: 1991 - 2021 Linus Torvalds and others.
+License: GPL-2
+ The git repository for mainline kernel development is at:
+ git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+ .
     This program is free software; you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
     the Free Software Foundation; version 2 dated June, 1991.
-
-On Debian GNU/Linux systems, the complete text of the GNU General Public
-License version 2 can be found in \`/usr/share/common-licenses/GPL-2'.
+ .
+ On Debian GNU/Linux systems, the complete text of the GNU General Public
+ License version 2 can be found in \`/usr/share/common-licenses/GPL-2'.
 EOF
 
 # Generate a control file
Signature · 1 line
2.30.2
#8 lore raw

Re: [PATCH 4/6] builddeb: use standard format for copyright file

Quoted 8 lines · On Sun, Apr 11, 2021 at 7:14 PM <[email protected]> wrote:

From: Bastian Germann <[email protected]>

Convert the minimal copyright file to the DEP-5 standard format that is commonly used in Debian.

This is equivalent to this patch:
https://lore.kernel.org/linux-kbuild/[email protected]/

Quoted 20 lines
---
 scripts/package/mkdebian | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)
scripts/package/mkdebian +0 -8
index 717223ef6b03..bb5c19735345 100755
@@ -152,22 +152,23 @@ EOF
 
 # Generate copyright file
 cat <<EOF > debian/copyright
-This is a packacked upstream version of the Linux kernel.
-
-The sources may be found at most Linux archive sites, including:
-https://www.kernel.org/pub/linux/kernel
-
-Copyright: 1991 - 2018 Linus Torvalds and others.
-
-The git repository for mainline kernel development is at:

scripts/package/mkdebian EOF · line 162

Quoted 10 lines
scripts/package/mkdebian +8 -2
-git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
-
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+Upstream-Name: Linux
+Source: https://www.kernel.org/pub/linux/kernel
+Comment: This is a packaged upstream version of the Linux kernel.
+
+Files: *
+Copyright: 1991 - 2021 Linus Torvalds and others.
+License: GPL-2

My concern is this license info is wrong
since not all files are GPL-2.

Is it better to make apparently wrong license info machine-interpretable?

That is a question.

Signature · 2 lines
Best Regards
Masahiro Yamada
#9 lore raw

PATCH 5/6 builddeb: introduce profile excluding the dbg pkg

In reply to [email protected]

From: Bastian Germann <[email protected]>

Enabling CONFIG_DEBUG_INFO implies building the binary linux-image-*-dbg. As this increases package build time significantly, one might want to exclude it from being built.

Add a pkg.$sourcename.nodbg build profile for that package so it can be excluded via e.g.: make DPKG_FLAGS="-P=pkg.linux-5.11.0.nodbg" deb-pkg

---
 scripts/package/mkdebian | 1 +
 1 file changed, 1 insertion(+)
scripts/package/mkdebian +1 -0
index bb5c19735345..369a989499ef 100755
@@ -213,6 +213,7 @@ if is_enabled CONFIG_DEBUG_INFO; then
 cat <<EOF >> debian/control
 
 Package: linux-image-$version-dbg
+Build-Profiles: <!pkg.$sourcename.nodbg>
 Section: debug
 Architecture: $debarch
 Description: Linux kernel debugging symbols for $version
Signature · 1 line
2.30.2
#10 lore raw

PATCH 6/6 kbuild: introduce srcdeb-pkg target

In reply to [email protected]

From: Bastian Germann <[email protected]>

A Debian source package can be generated only in combination with building it afterwards. Introduce a target srcdeb-pkg that generates the source package without building it (adding dpkg-buildpackage's -S flag).

Make the former deb-pkg depend on both srcdeb-pkg and bindeb-pkg to retain its behavior.

---
 scripts/Makefile.package | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
scripts/Makefile.package +6 -2
index 280f3a2fa334..78a363623c3a 100644
@@ -69,13 +69,16 @@ binrpm-pkg:
 		$(UTS_MACHINE) -bb $(objtree)/binkernel.spec
 
 PHONY += deb-pkg
-deb-pkg:
+deb-pkg: srcdeb-pkg bindeb-pkg
+
+PHONY += srcdeb-pkg
+srcdeb-pkg:
 	$(MAKE) clean
 	$(CONFIG_SHELL) $(srctree)/scripts/package/mkdebian
 	$(call cmd,src_tar,$(KDEB_SOURCENAME))
 	origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
 		mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
-	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -us -uc
+	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -S -us -uc
 
 PHONY += bindeb-pkg
 bindeb-pkg:
@@ -145,6 +148,7 @@ help:
 	@echo '  rpm-pkg             - Build both source and binary RPM kernel packages'
 	@echo '  binrpm-pkg          - Build only the binary kernel RPM package'
 	@echo '  deb-pkg             - Build both source and binary deb kernel packages'
+	@echo '  srcdeb-pkg          - Build only the source kernel deb package'
 	@echo '  bindeb-pkg          - Build only the binary kernel deb package'
 	@echo '  snap-pkg            - Build only the binary kernel snap package'
 	@echo '                        (will connect to external hosts)'
Signature · 1 line
2.30.2
#11 lore raw

Re: [PATCH 6/6] kbuild: introduce srcdeb-pkg target

Quoted 20 lines · On Sun, Apr 11, 2021 at 7:14 PM <[email protected]> wrote:

From: Bastian Germann <[email protected]>

A Debian source package can be generated only in combination with building it afterwards. Introduce a target srcdeb-pkg that generates the source package without building it (adding dpkg-buildpackage's -S flag).

Make the former deb-pkg depend on both srcdeb-pkg and bindeb-pkg to retain its behavior.

---
 scripts/Makefile.package | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
scripts/Makefile.package +0 -0
index 280f3a2fa334..78a363623c3a 100644

scripts/Makefile.package binrpm-pkg: · line 72

Quoted 5 lines
scripts/Makefile.package +1 -1
@@ -69,13 +69,16 @@ binrpm-pkg:
                $(UTS_MACHINE) -bb $(objtree)/binkernel.spec
 
 PHONY += deb-pkg
-deb-pkg:
+deb-pkg: srcdeb-pkg bindeb-pkg

Does this work reliably with the parallel build option?

While srcdeb-pkg is cleaning the tree, bindeb-pkg simultaneously builds objects in the tree.

Quoted 24 lines
scripts/Makefile.package +5 -2
+
+PHONY += srcdeb-pkg
+srcdeb-pkg:
        $(MAKE) clean
        $(CONFIG_SHELL) $(srctree)/scripts/package/mkdebian
        $(call cmd,src_tar,$(KDEB_SOURCENAME))
        origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
                mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
-       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -us -uc
+       +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -S -us -uc
 
 PHONY += bindeb-pkg
 bindeb-pkg:
@@ -145,6 +148,7 @@ help:
        @echo '  rpm-pkg             - Build both source and binary RPM kernel packages'
        @echo '  binrpm-pkg          - Build only the binary kernel RPM package'
        @echo '  deb-pkg             - Build both source and binary deb kernel packages'
+       @echo '  srcdeb-pkg          - Build only the source kernel deb package'
        @echo '  bindeb-pkg          - Build only the binary kernel deb package'
        @echo '  snap-pkg            - Build only the binary kernel snap package'
        @echo '                        (will connect to external hosts)'
--

2.30.2

Signature · 2 lines
Best Regards
Masahiro Yamada