Skip to contentSkip to conversation

Debian build polishing

PATCH In review Enrico Weigelt, metux IT consult 5 participants 12 messages Mar 8, 2019 – Mar 11, 2019

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi folks,

here're some patches for polishing up the Debian packaging stuff, so it can be directly used w/ usual Debian machinery like pbuilder, git-buildpackage, dck-buildpackage, etc.

These expect debian/rules to exist in the unpacked/patched tree and drive the whole build. Currently 'make deb-pkg' does it in the opposite direction - it creates debian/rules and fills in some data, that's derived from .config etc.

My goal is building the kernel package in exactly the same way as any other Debian package - so there must be a debian/rules as the primary entry point. To do that, w/ minimal change and w/o breaking the existing machinery, I'm going in several steps:

#1: add Makefile rules for retrieving missing makefile-internal
variables kernel config system .config (eg. kernel arch).

this could be used for other build systems, too. just call: make kernelarch or make kernellocalversion

#2: add an env variable for changing the name of the rules file
generated by mkdebian. When coming from an existing rules file, we can prevent this from being overwritten.

#3: add a generic debian/rules file, that calls mkdebian to
create the remaining debian control files (w/ rules redirected into nirvana)

The existing make deb-pkg is bypassed and remains ontouched.

One point still puzzling me: once the debian/rules is applied and somebody calls make deb-pkg, he'll end up w/ unclean tree, as now a git-tracked file is changed.

Perhaps I just change deb-pkg to call debian/rules then, but I'd like to hear your oppinions about this, before.

What do you think about that ?

--mtx

#1 lore raw

PATCH 1/5 Makefile: rules for printing kernel architecture and localversion

trivial rule to print out the kernel arch and localversion, so external tools, like distro packagers, can easily get it.

---
 Makefile | 6 ++++++
 1 file changed, 6 insertions(+)
Makefile +6 -0
index f070e0d..95b50f8 100644
@@ -1664,6 +1664,12 @@ kernelrelease:
 kernelversion:
 	@echo $(KERNELVERSION)
 
+kernellocalversion:
+	@$(CONFIG_SHELL) $(srctree)/scripts/setlocalversion $(srctree) | sed -e 's~^\-~~'
+
+kernelarch:
+	@echo $(ARCH)
+
 image_name:
 	@echo $(KBUILD_IMAGE)
 
Signature · 1 line
1.9.1
#2 lore raw

PATCH 2/5 scripts: mkdebian: allow renaming generated debian/rules via env

In reply to Enrico Weigelt, metux IT consult

Add new environment variable KDEB_RULES for controlling where the generated debian rules are written to. By defaults, it's debian/rules, but packagers might override it for providing their own rules file.

---
 scripts/package/mkdebian | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
scripts/package/mkdebian +4 -1
index edcad61..ff226be 100755
@@ -99,6 +99,9 @@ kernel_headers_packagename=linux-headers-$version
 dbg_packagename=$packagename-dbg
 debarch=
 set_debarch
+if [ -z "$KDEB_RULES" ]; then
+    KDEB_RULES=debian/rules
+fi
 
 if [ "$ARCH" = "um" ] ; then
 	packagename=user-mode-linux-$version
@@ -202,7 +205,7 @@ Description: Linux kernel debugging symbols for $version
  all the necessary debug symbols for the kernel and its modules.
 EOF
 
-cat <<EOF > debian/rules
+cat <<EOF > $KDEB_RULES
 #!$(command -v $MAKE) -f
 
 build:
Signature · 1 line
1.9.1
#3 lore raw

PATCH 3/5 scripts: mkdebian: fix missing dependencies

In reply to Enrico Weigelt, metux IT consult

We missed some build dependencies in the generated debian/control file.

---
 scripts/package/mkdebian | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/package/mkdebian +1 -1
index ff226be..1caaf0b 100755
@@ -173,7 +173,7 @@ Source: $sourcename
 Section: kernel
 Priority: optional
 Maintainer: $maintainer
-Build-Depends: bc, kmod, cpio
+Build-Depends: bc, kmod, cpio, bison, libelf-dev, libssl-dev, flex
 Homepage: http://www.kernel.org/
 
 Package: $packagename
Signature · 1 line
1.9.1
#4 lore raw

PATCH 4/5 scripts: checkpatch.pl: don't complain that debian/rules is executable

In reply to Enrico Weigelt, metux IT consult

checkpatch.pl complains when adding executable "debian/rules", obviously a false alarm. Therefore add an exception for that.

---
 scripts/checkpatch.pl | 1 +
 1 file changed, 1 insertion(+)
scripts/checkpatch.pl +1 -0
index b737ca9..7bcc70a 100755
@@ -2576,6 +2576,7 @@ sub process {
 		if ($line =~ /^new (file )?mode.*[7531]\d{0,2}$/) {
 			my $permhere = $here . "FILE: $realfile\n";
 			if ($realfile !~ m@scripts/@ &&
+			    $realfile !~ "debian/rules" &&
 			    $realfile !~ /\.(py|pl|awk|sh)$/) {
 				ERROR("EXECUTE_PERMISSIONS",
 				      "do not set execute permissions for source files\n" . $permhere);
Signature · 1 line
1.9.1
maintainer
#5 lore raw

Re: [PATCH v1 4/5] scripts: checkpatch.pl: don't complain that debian/rules is executable

Quoted 2 lines · metux IT consult

checkpatch.pl complains when adding executable "debian/rules", obviously a false alarm. Therefore add an exception for that.

This is predicated on a debian/ directory actually being added to the generic kernel tree.

I'm not sure that's a good idea.

As debian/ does not currently exist in the generic kernel source, I think this can be kept locally in their tree if they choose.

scripts/checkpatch.pl

Quoted 1 line
scripts/checkpatch.pl +0 -0

[]

Quoted 8 lines
scripts/checkpatch.pl +1 -0
@@ -2576,6 +2576,7 @@ sub process {
 		if ($line =~ /^new (file )?mode.*[7531]\d{0,2}$/) {
 			my $permhere = $here . "FILE: $realfile\n";
 			if ($realfile !~ m@scripts/@ &&
+			    $realfile !~ "debian/rules" &&
 			    $realfile !~ /\.(py|pl|awk|sh)$/) {
 				ERROR("EXECUTE_PERMISSIONS",
 				      "do not set execute permissions for source files\n" . $permhere);
#6 lore raw

PATCH 5/5 debian: add generic rule file

In reply to Enrico Weigelt, metux IT consult

Adding a generic debian rule file, so we can build the directly via usual Debian package build tools (eg. git-buildpackage, dck-buildpackage, etc). It expects the .config file already placed in the source tree.

The rule file contains a rule for creating debian/control and other metadata - this is done similar to the 'deb-pkg' make rule, scripts/packaging/mkdebian.

---
 .gitignore   |  1 +
 MAINTAINERS  |  6 ++++++
 debian/rules | 27 +++++++++++++++++++++++++++
 3 files changed, 34 insertions(+)
 create mode 100755 debian/rules
.gitignore +1 -0
index a20ac26..2d8081f 100644
@@ -68,6 +68,7 @@ modules.builtin
 # Debian directory (make deb-pkg)
 #
 /debian/
+!/debian/rules
 
 #
 # Snap directory (make snap-pkg)
MAINTAINERS +6 -0
index 09d47cd..1edc169 100644
@@ -4363,6 +4363,12 @@ F:	include/uapi/linux/dccp.h
 F:	include/linux/tfrc.h
 F:	net/dccp/
 
+DEBIAN PACKAGING FILES
+M:	Enrico Weigelt <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	debian/
+
 DECnet NETWORK LAYER
 W:	http://linux-decnet.sourceforge.net
 L:	[email protected]
debian/rules new file +27 -0
new file mode 100755
index 0000000..c2f0319
@@ -0,0 +1,27 @@
+#!/usr/bin/make -f
+# SPDX-License-Identifier: GPL-2.0
+
+export MAKE
+export KERNELARCH           = $(shell $(MAKE) kernelarch)
+export KERNELRELEASE        = $(shell $(MAKE) kernelrelease)
+export KBUILD_DEBARCH       = $(shell dpkg-architecture -qDEB_HOST_ARCH)
+export KBUILD_BUILD_VERSION = $(shell $(MAKE) kernellocalversion)
+export KDEB_RULES           = debian/rules.auto
+export KDEB_SOURCENAME      = linux-source
+export ARCH                 = $(KERNELARCH)
+
+debian/control debian/changelong debian/arch debian/copyright:
+debian/control:
+	./scripts/package/mkdebian
+
+build:	debian/control
+	$(MAKE) KERNELRELEASE=$(KERNELRELEASE) ARCH=$(KERNELARCH) KBUILD_BUILD_VERSION=$(KBUILD_BUILD_VERSION) KBUILD_SRC=
+
+binary-arch:	debian/control
+	$(MAKE) KERNELRELEASE=$(KERNELRELEASE) ARCH=$(KERNELARCH) KBUILD_BUILD_VERSION=$(KBUILD_BUILD_VERSION) KBUILD_SRC= intdeb-pkg
+
+clean:
+	rm -rf debian/*tmp debian/files debian/changelog debian/control debian/copyright debian/rules.auto debian/arch
+	$(MAKE) clean
+
+binary: binary-arch
Signature · 1 line
1.9.1
#7 lore raw
In reply to Enrico Weigelt, metux IT consult
Quoted 3 lines · metux IT consult

One point still puzzling me: once the debian/rules is applied and somebody calls make deb-pkg, he'll end up w/ unclean tree, as now a git-tracked file is changed.

This is not something I've noticed, but I build my Debian packages like this:

make O=/build/linux-build bindeb-pkg

This works really well for me, since all of the build-artificats land in /build, and I can use a HDD (or a PD-HDD when building using Google Compute Engine) for /build, while I use a SSD for my source tree. I find that using a HDD for a target of a build doesn't really slow things down, and this allows me to save $$$ (when using a Cloud VM) and reduce flash wearout and capital cost (on my personal machines).

So I really hope your patches don't break this. Also, are there any changes the performance of building the Debian packages before and after your changes? And are there any differences in the packages in terms of any pre-or-post install/removal scripts?

There are a lot of things I really dislike about the "official" Debian kernel build processes (they're optimzied for distribution release engineers, not kernel developers), so I'm really hoping that making things more like the "official Debian way" doesn't break some of the things I really like about the existing "make bindeb-pkg" build system.

Cheers,

- Ted

#8 lore raw
Quoted 9 lines · Theodore Ts'o
Quoted 3 lines · metux IT consult

One point still puzzling me: once the debian/rules is applied and somebody calls make deb-pkg, he'll end up w/ unclean tree, as now a git-tracked file is changed.

This is not something I've noticed, but I build my Debian packages like this:

make O=/build/linux-build bindeb-pkg

This doesn't work for my environment:

The requirement is that every debian source tree has a ./debian/rules file (w/ the standard rules like binary, clean, ...). This is called by upper level build machinery (git-buildpackage, dpkg-buildpackage, ...)

The idea behind is building all packages exactly the same way - one standard workflow, that just takes a source tree (from git) and spits out some .deb files.

Quoted 1 line

So I really hope your patches don't break this.

It shouldn't (at least I dont intend to). Haven't tried where exactly the generated debian/rules file lands in your case.

Quoted 3 lines

Also, are there any changes the performance of building the Debian packages before and after your changes? And are there any differences in the packages in terms of any pre-or-post install/removal scripts?

I don't think so. In essence I just replace the generated rules file by a fixed one. Okay, it will take *a little* bit longer, as the rules file now calls make kernelarch, make kernelrelease and make kernellocalversion, before starting actual build. But that shouldn't be the big deal, IMHO.

Quoted 3 lines

There are a lot of things I really dislike about the "official" Debian kernel build processes (they're optimzied for distribution release engineers, not kernel developers),

I try to bring these two world a bit closer together.

Quoted 4 lines

so I'm really hoping that making
things more like the "official Debian way" doesn't break some of the things I really like about the existing "make bindeb-pkg" build system.

I intend not to.

Actually, the official Debian way for Kernel build is entirely different, and that's really complicated and hard to really understand (that's why I'm doing this here).

But here I'm just coping w/ the first entry point into the build process, which now becomes ./debian/rules binary instead of make bindeb-pkg. The final result (in the .deb files) should be the time (unless I did something wrong).

--mtx

Signature · 3 lines
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[email protected] -- +49-151-27565287
#9 lore raw
Quoted 4 lines · metux IT consult
Quoted 1 line

So I really hope your patches don't break this.

It shouldn't (at least I dont intend to). Haven't tried where exactly the generated debian/rules file lands in your case.

The full debian directory lands in the O= directory. So presumably you could make it work for use case if there was a "make O=/build/dir debian-prepare" which copies the necessary debian/ files into the build directory and then someone could cd into /build/dir and run "dpkg-buildpackage"?

- Ted

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#10 lore raw
Quoted 5 lines · Theodore Ts'o

The full debian directory lands in the O= directory. So presumably you could make it work for use case if there was a "make O=/build/dir debian-prepare" which copies the necessary debian/ files into the build directory and then someone could cd into /build/dir and run "dpkg-buildpackage"?

Didn't touch these rules at all, so it should still work as it used to.

I thought about changing the old deb rules to just call the new debian/rules file, but haven't done so yet.

Maybe you could find some time to test whether my approach also works for you ?

--mtx

Signature · 3 lines
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[email protected] -- +49-151-27565287
#11 lore raw
In reply to Enrico Weigelt, metux IT consult

(+CC more debian folks)

Quoted 32 lines · metux IT consult

Hi folks,

here're some patches for polishing up the Debian packaging stuff, so it can be directly used w/ usual Debian machinery like pbuilder, git-buildpackage, dck-buildpackage, etc.

These expect debian/rules to exist in the unpacked/patched tree and drive the whole build. Currently 'make deb-pkg' does it in the opposite direction - it creates debian/rules and fills in some data, that's derived from .config etc.

My goal is building the kernel package in exactly the same way as any other Debian package - so there must be a debian/rules as the primary entry point. To do that, w/ minimal change and w/o breaking the existing machinery, I'm going in several steps:

#1: add Makefile rules for retrieving missing makefile-internal
variables kernel config system .config (eg. kernel arch).

this could be used for other build systems, too. just call: make kernelarch or make kernellocalversion

#2: add an env variable for changing the name of the rules file
generated by mkdebian. When coming from an existing rules file, we can prevent this from being overwritten.

#3: add a generic debian/rules file, that calls mkdebian to
create the remaining debian control files (w/ rules redirected into nirvana)

Quoted 5 lines

The existing make deb-pkg is bypassed and remains ontouched.

One point still puzzling me: once the debian/rules is applied and somebody calls make deb-pkg, he'll end up w/ unclean tree, as now a git-tracked file is changed.

Then, setlocalversion will set -dirty flag.

Committing debian/rules looks questionable to me.

Quoted 10 lines

Perhaps I just change deb-pkg to call debian/rules then, but I'd like to hear your oppinions about this, before.

What do you think about that ?

--mtx

Signature · 2 lines
Best Regards
Masahiro Yamada