Skip to contentSkip to conversation

scripts/package/Makefile: put proper config in source tarball

PATCH In review Arseny Maslennikov 2 participants 6 messages Mar 9, 2019 – Mar 17, 2019 Kbuild

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/3 scripts/package/Makefile: put proper config in source tarball

It is widely known that one can build a kernel without a .config in the source tree by setting KCONFIG_CONFIG equal to the actual configuration file path.

When making a *-pkg target, make(1) prepares a source tarball and tries to pack `.config' in there regardless of the value of KCONFIG_CONFIG, failing spectacularly if .config is absent and packing the wrong config if it exists.
Let's fix that by including whatever KCONFIG_CONFIG points to in the archive under the traditional name `.config'.

We have to pass --absolute-names to tar, since it seems to trim the paths of archive members first and only then consider --transform, which breaks KCONFIG_CONFIG=../../some.config, as well as absolute paths.

---
 scripts/package/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
scripts/package/Makefile +2 -1
index 2c6de21e5152..134cefee068d 100644
@@ -27,7 +27,7 @@ KDEB_SOURCENAME ?= linux-$(KERNELRELEASE)
 KBUILD_PKG_ROOTCMD ?="fakeroot -u"
 export KDEB_SOURCENAME
 # Include only those top-level files that are needed by make, plus the GPL copy
-TAR_CONTENT := $(KBUILD_ALLDIRS) .config .scmversion Makefile \
+TAR_CONTENT := $(KBUILD_ALLDIRS) $(KCONFIG_CONFIG) .scmversion Makefile \
                Kbuild Kconfig COPYING $(wildcard localversion*)
 MKSPEC     := $(srctree)/scripts/package/mkspec
 
@@ -44,6 +44,7 @@ if test "$(objtree)" != "$(srctree)"; then \
 fi ; \
 $(srctree)/scripts/setlocalversion --save-scmversion; \
 tar -cz $(RCS_TAR_IGNORE) -f $(2).tar.gz \
+	--absolute-names --transform 's:^$(KCONFIG_CONFIG)$$:.config:S' \
 	--transform 's:^:$(2)/:S' $(TAR_CONTENT) $(3); \
 rm -f $(objtree)/.scmversion
 
Signature · 1 line
2.20.1
#1 lore raw

PATCH v2 2/3 scripts/package/mkdebian: avoid implicit effects

* The man page for dpkg-source(1) notes:

Quoted 4 lines

-b, --build directory [format-specific-parameters]
Build a source package (--build since dpkg 1.17.14). <...>

Quoted 9 lines

dpkg-source will build the source package with the first format found in this ordered list: the format indicated with the --format command line option, the format indicated in debian/source/format, “1.0”. The fallback to “1.0” is deprecated and will be removed at some point in the future, you should always document the desired
source format in debian/source/format. See section
SOURCE PACKAGE FORMATS for an extensive description of the various source package formats.

Thus it would be more foolproof to explicitly use 1.0 (as we always did) than to rely on dpkg-source's defaults.

* In a similar vein, debian/rules is not made executable by mkdebian, and dpkg-source warns about that but still silently fixes the file. Let's be explicit once again.

---
 scripts/package/mkdebian | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
scripts/package/mkdebian +4 -1
index f030961c5165..d276eb671a27 100755
@@ -132,7 +132,9 @@ else
         echo >&2 "Install lsb-release or set \$KDEB_CHANGELOG_DIST explicitly"
 fi
 
-mkdir -p debian/
+mkdir -p debian/source/
+echo "1.0" > debian/source/format
+
 echo $debarch > debian/arch
 
 # Generate a simple changelog template
@@ -221,5 +223,6 @@ clean:
 
 binary: binary-arch
 EOF
+chmod +x debian/rules
 
 exit 0
Signature · 1 line
2.20.1
#2 lore raw

Re: [PATCH v2 2/3] scripts/package/mkdebian: avoid implicit effects

Quoted 20 lines · Arseny Maslennikov

* The man page for dpkg-source(1) notes:

Quoted 4 lines

-b, --build directory [format-specific-parameters]
Build a source package (--build since dpkg 1.17.14). <...>

Quoted 9 lines

dpkg-source will build the source package with the first format found in this ordered list: the format indicated with the --format command line option, the format indicated in debian/source/format, “1.0”. The fallback to “1.0” is deprecated and will be removed at some point in the future, you should always document the desired
source format in debian/source/format. See section
SOURCE PACKAGE FORMATS for an extensive description of the various source package formats.

Thus it would be more foolproof to explicitly use 1.0 (as we always did) than to rely on dpkg-source's defaults.

Quoted 6 lines

* In a similar vein, debian/rules is not made executable by mkdebian, and dpkg-source warns about that but still silently fixes the file. Let's be explicit once again.

---

Applied to linux-kbuild. Thanks.

Quoted 28 lines
 scripts/package/mkdebian | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
scripts/package/mkdebian +4 -2
index f030961c5165..d276eb671a27 100755
@@ -132,7 +132,9 @@ else
         echo >&2 "Install lsb-release or set \$KDEB_CHANGELOG_DIST explicitly"
 fi
 
-mkdir -p debian/
+mkdir -p debian/source/
+echo "1.0" > debian/source/format
+
 echo $debarch > debian/arch
 
 # Generate a simple changelog template
@@ -221,5 +223,6 @@ clean:
 
 binary: binary-arch
 EOF
+chmod +x debian/rules
 
 exit 0
--

2.20.1

Signature · 2 lines
Best Regards
Masahiro Yamada
#3 lore raw

PATCH v2 3/3 package: {mkdebian,Makefile}: ignore .config in dpkg-source diffs

In reply to Arseny Maslennikov

* We'd like to make dpkg-source ignore certain filepath patterns while producing the .diff.gz as part of a Debian source package. The relationship between dpkg-source's -i and --extend-diff-ignore options is quite complicated[1]. It's thus not very convenient to specify file path patterns to ignore on dpkg-buildpackage's command line (they do not end up in the source package as well), so we turn `-i.git' into a record in debian/source/options.
Besides, the dot in `-i.git' is a metacharacter and matches a tad too much.

* In cases where a .config exists in the source tree, but the .config in the source tarball is provided by wherever KCONFIG_CONFIG pointed to, dpkg-source would attempt to replace the second, proper one with the first one, which is undesirable and nullifies the effort made to put the proper config in the source tarball.
To prevent this, force dpkg-source to ignore /^[.]config$/ as well.

This makes it possible to build working Debian source packages with a redefined KCONFIG_CONFIG:
- that file is put in the source tarball at its traditional place, and the diff.gz does not dare touch it;
- when the source package is rebuilt, debian/rules initiates the build process with the desired config already in the standard location.

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=625789#22

---
 scripts/package/Makefile | 2 +-
 scripts/package/mkdebian | 6 ++++++
 2 files changed, 7 insertions(+), 1 deletion(-)
scripts/package/Makefile +1 -1
index 134cefee068d..f2fca7e7d9fa 100644
@@ -73,7 +73,7 @@ deb-pkg: FORCE
 	$(call cmd,src_tar,$(KDEB_SOURCENAME))
 	origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
 		mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
-	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -i.git -us -uc
+	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -us -uc
 
 bindeb-pkg: FORCE
 	$(CONFIG_SHELL) $(srctree)/scripts/package/mkdebian
scripts/package/mkdebian +6 -0
index d276eb671a27..74158c7f28ea 100755
@@ -135,6 +135,12 @@ fi
 mkdir -p debian/source/
 echo "1.0" > debian/source/format
 
+cat <<EOF > debian/source/options
+# The regexes are matched against paths from the source tree.
+diff-ignore = "[.]git"
+extend-diff-ignore = "^[.]config$"
+EOF
+
 echo $debarch > debian/arch
 
 # Generate a simple changelog template
Signature · 1 line
2.20.1
#4 lore raw
In reply to Arseny Maslennikov
Quoted 11 lines · Arseny Maslennikov

It is widely known that one can build a kernel without a .config in the source tree by setting KCONFIG_CONFIG equal to the actual configuration file path.

When making a *-pkg target, make(1) prepares a source tarball and tries to pack `.config' in there regardless of the value of KCONFIG_CONFIG, failing spectacularly if .config is absent and packing the wrong config if it exists.
Let's fix that by including whatever KCONFIG_CONFIG points to in the archive under the traditional name `.config'.

Quoted 3 lines

We have to pass --absolute-names to tar, since it seems to trim the paths of archive members first and only then consider --transform, which breaks KCONFIG_CONFIG=../../some.config, as well as absolute paths.

One way to avoid --absolute-names here is to do the dance below:

Quoted 20 lines
---
 scripts/package/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
scripts/package/Makefile +1 -1
index 2c6de21e5152..134cefee068d 100644
@@ -27,7 +27,7 @@ KDEB_SOURCENAME ?= linux-$(KERNELRELEASE)
 KBUILD_PKG_ROOTCMD ?="fakeroot -u"
 export KDEB_SOURCENAME
 # Include only those top-level files that are needed by make, plus the GPL copy
-TAR_CONTENT := $(KBUILD_ALLDIRS) .config .scmversion Makefile \
+TAR_CONTENT := $(KBUILD_ALLDIRS) $(KCONFIG_CONFIG) .scmversion Makefile \
                Kbuild Kconfig COPYING $(wildcard localversion*)
 MKSPEC     := $(srctree)/scripts/package/mkspec
 

scripts/package/Makefile if test "$(objtree)" != "$(srctree)"; then \ · line 45

Quoted 2 lines
scripts/package/Makefile +0 -0
@@ -44,6 +44,7 @@ if test "$(objtree)" != "$(srctree)"; then \
 fi ; \
 $(srctree)/scripts/setlocalversion --save-scmversion; \

+cp $(KCONFIG_CONFIG) .tmp.config-tarball

scripts/package/Makefile if test "$(objtree)" != "$(srctree)"; then \ · line 46

Quoted 1 line
scripts/package/Makefile +0 -0
 tar -cz $(RCS_TAR_IGNORE) -f $(2).tar.gz \

+ --absolute-names --transform 's:^.tmp.config-tarball$$:.config:S' \

scripts/package/Makefile if test "$(objtree)" != "$(srctree)"; then \ · line 48

Quoted 1 line
scripts/package/Makefile +0 -0
 	--transform 's:^:$(2)/:S' $(TAR_CONTENT) $(3); \

+rm -f .tmp.config-tarball

Quoted 5 lines
scripts/package/Makefile +0 -0
 rm -f $(objtree)/.scmversion
 
Signature · 1 line
2.20.1

I'm not sure if this is strictly better.

#5 lore raw
In reply to Arseny Maslennikov
Quoted 12 lines · Arseny Maslennikov

It is widely known that one can build a kernel without a .config in the source tree by setting KCONFIG_CONFIG equal to the actual configuration file path.

When making a *-pkg target, make(1) prepares a source tarball and tries to pack `.config' in there regardless of the value of KCONFIG_CONFIG, failing spectacularly if .config is absent and packing the wrong config if it exists.
Let's fix that by including whatever KCONFIG_CONFIG points to in the archive under the traditional name `.config'.

Quoted 5 lines

We have to pass --absolute-names to tar, since it seems to trim the paths of archive members first and only then consider --transform, which breaks KCONFIG_CONFIG=../../some.config, as well as absolute paths.

rpm-pkg will work in a strange way.
(probably snap-pkg too).

rpm package is compiled in $(HOME)/rpmbuild/BUILD.
The source package will contain the .config in the standard location, but KCONFIG_CONFIG will point to a different path.

Rather, I'd like to stop the build.

quiet_cmd_src_tar = TAR $(2).tar.gz

      cmd_src_tar = \
if test "$(objtree)" != "$(srctree)"; then \
        echo >&2; \
        echo >&2 "  ERROR:"; \
        echo >&2 "  Building source tarball is not possible outside the"; \
        echo >&2 "  kernel source tree. Don't set KBUILD_OUTPUT, or use the"; \
        echo >&2 "  binrpm-pkg or bindeb-pkg target instead."; \
        echo >&2; \
        false; \
fi ; \
if test "$(KCONFIG_CONFIG)" != .config; then \
        echo >&2; \
        echo >&2 "  ERROR:"; \
        echo >&2 "  Please do not override KCONFIG_CONFIG"; \
        echo >&2 "  for source package build"; \
        echo >&2; \
        false; \

fi ; \
$(srctree)/scripts/setlocalversion --save-scmversion; \
tar -cz $(RCS_TAR_IGNORE) -f $(2).tar.gz \
--transform 's:^:$(2)/:S' $(TAR_CONTENT) $(3); \
rm -f $(objtree)/.scmversion

I am included to do the opposite;
replace KCONFIG_CONFIG with the hard-coded .config

KCONFIG_CONFIG does not work.
It is half-baked at best.

Quoted 28 lines
---
 scripts/package/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
scripts/package/Makefile +2 -2
index 2c6de21e5152..134cefee068d 100644
@@ -27,7 +27,7 @@ KDEB_SOURCENAME ?= linux-$(KERNELRELEASE)
 KBUILD_PKG_ROOTCMD ?="fakeroot -u"
 export KDEB_SOURCENAME
 # Include only those top-level files that are needed by make, plus the GPL copy
-TAR_CONTENT := $(KBUILD_ALLDIRS) .config .scmversion Makefile \
+TAR_CONTENT := $(KBUILD_ALLDIRS) $(KCONFIG_CONFIG) .scmversion Makefile \
                Kbuild Kconfig COPYING $(wildcard localversion*)
 MKSPEC     := $(srctree)/scripts/package/mkspec
 
@@ -44,6 +44,7 @@ if test "$(objtree)" != "$(srctree)"; then \
 fi ; \
 $(srctree)/scripts/setlocalversion --save-scmversion; \
 tar -cz $(RCS_TAR_IGNORE) -f $(2).tar.gz \
+       --absolute-names --transform 's:^$(KCONFIG_CONFIG)$$:.config:S' \
        --transform 's:^:$(2)/:S' $(TAR_CONTENT) $(3); \
 rm -f $(objtree)/.scmversion
 
--

2.20.1

Signature · 2 lines
Best Regards
Masahiro Yamada