Skip to contentSkip to conversation

scripts/package/Makefile: put proper config in source tarball

PATCH Superseded Arseny Maslennikov 4 participants 11 messages Feb 21, 2019 – Mar 9, 2019

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 scripts/package/Makefile: put proper config in source tarball

It is widely known that one can build a kernel without a .config in the source tree by setting KCONFIG_CONFIG equal to the actual configuration file path.

When making a *-pkg target, make(1) prepares a source tarball and tries to pack `.config' in there regardless of the value of KCONFIG_CONFIG, failing spectacularly if .config is absent and packing the wrong config if it exists.
Let's fix that.

---
 scripts/package/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/package/Makefile +1 -1
index 453fecee62f0..100a0d28ad25 100644
@@ -27,7 +27,7 @@ KDEB_SOURCENAME ?= linux-$(KERNELRELEASE)
 KBUILD_PKG_ROOTCMD ?="fakeroot -u"
 export KDEB_SOURCENAME
 # Include only those top-level files that are needed by make, plus the GPL copy
-TAR_CONTENT := $(KBUILD_ALLDIRS) .config .scmversion Makefile \
+TAR_CONTENT := $(KBUILD_ALLDIRS) $(KCONFIG_CONFIG) .scmversion Makefile \
                Kbuild Kconfig COPYING $(wildcard localversion*)
 MKSPEC     := $(srctree)/scripts/package/mkspec
 
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/1 scripts/package/mkdebian: expose KCONFIG_CONFIG to debian/rules

If KCONFIG_CONFIG is passed to make deb-pkg as a makeflag or environment variable, the source tarball produced contains whatever specified by that variable instead of the usual `.config'.

While `make deb-pkg' yields a correct Debian binary package (at least on amd64), the generated source package's debian/rules does not pass KCONFIG_CONFIG to make(1), thus rendering the source package unable to be rebuilt.

Steps to reproduce the kind of failure being fixed:
1) Produce a source and binary package with:
% KCONFIG_CONFIG=my.config make deb-pkg
2) In the parent directory:
% dpkg-source -x linux-*.dsc
3) In the unpacked source directory:
% debuild -i -us -uc

Here is an excerpt from an example output of `debuild -i -us -uc':

make[2]: *** No rule to make target '.config', needed by 'kernel/config_data.gz'. Stop.
make[1]: *** [Makefile:1043: kernel] Error 2
make[1]: Leaving directory '/home/ar/linux-kernel/deb/linux-5.0.0-rc5-tks-gitd6ff78479ec5-57-5.0.0-rc5-tks-gitd6ff78479ec5-57'
make: *** [debian/rules:4: build] Error 2
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
debuild: fatal error at line 1182:
dpkg-buildpackage -us -uc -ui -i failed

---
 scripts/package/mkdebian | 2 ++
 1 file changed, 2 insertions(+)
scripts/package/mkdebian +2 -0
index edcad61fe3cd..130d3f6696eb 100755
@@ -207,10 +207,12 @@ cat <<EOF > debian/rules
 
 build:
 	\$(MAKE) KERNELRELEASE=${version} ARCH=${ARCH} \
+	KCONFIG_CONFIG=${KCONFIG_CONFIG} \
 	KBUILD_BUILD_VERSION=${revision} KBUILD_SRC=
 
 binary-arch:
 	\$(MAKE) KERNELRELEASE=${version} ARCH=${ARCH} \
+	KCONFIG_CONFIG=${KCONFIG_CONFIG} \
 	KBUILD_BUILD_VERSION=${revision} KBUILD_SRC= intdeb-pkg
 
 clean:
Signature · 1 line
2.20.1
Reviewed-by on v1 1/1
#2 lore raw

Re: [PATCH] scripts/package/mkdebian: expose KCONFIG_CONFIG to debian/rules

Hi,

Quoted 19 lines

If KCONFIG_CONFIG is passed to make deb-pkg as a makeflag or environment variable, the source tarball produced contains whatever specified by that variable instead of the usual `.config'.

While `make deb-pkg' yields a correct Debian binary package (at least on amd64), the generated source package's debian/rules does not pass KCONFIG_CONFIG to make(1), thus rendering the source package unable to be rebuilt.

Steps to reproduce the kind of failure being fixed:
1) Produce a source and binary package with:
% KCONFIG_CONFIG=my.config make deb-pkg
2) In the parent directory:
% dpkg-source -x linux-*.dsc
3) In the unpacked source directory:
% debuild -i -us -uc

Here is an excerpt from an example output of `debuild -i -us -uc':

Quoted 7 lines

make[2]: *** No rule to make target '.config', needed by 'kernel/config_data.gz'. Stop.
make[1]: *** [Makefile:1043: kernel] Error 2
make[1]: Leaving directory '/home/ar/linux-kernel/deb/linux-5.0.0-rc5-tks-gitd6ff78479ec5-57-5.0.0-rc5-tks-gitd6ff78479ec5-57'
make: *** [debian/rules:4: build] Error 2
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
debuild: fatal error at line 1182:
dpkg-buildpackage -us -uc -ui -i failed

Kind regards,
Petr

#3 lore raw

Re: [PATCH] scripts/package/mkdebian: expose KCONFIG_CONFIG to debian/rules

In reply to Arseny Maslennikov
Quoted 20 lines · Arseny Maslennikov

If KCONFIG_CONFIG is passed to make deb-pkg as a makeflag or environment variable, the source tarball produced contains whatever specified by that variable instead of the usual `.config'.

While `make deb-pkg' yields a correct Debian binary package (at least on amd64), the generated source package's debian/rules does not pass KCONFIG_CONFIG to make(1), thus rendering the source package unable to be rebuilt.

Steps to reproduce the kind of failure being fixed:
1) Produce a source and binary package with:
% KCONFIG_CONFIG=my.config make deb-pkg
2) In the parent directory:
% dpkg-source -x linux-*.dsc
3) In the unpacked source directory:
% debuild -i -us -uc

Here is an excerpt from an example output of `debuild -i -us -uc':

Quoted 7 lines

make[2]: *** No rule to make target '.config', needed by 'kernel/config_data.gz'. Stop.
make[1]: *** [Makefile:1043: kernel] Error 2
make[1]: Leaving directory '/home/ar/linux-kernel/deb/linux-5.0.0-rc5-tks-gitd6ff78479ec5-57-5.0.0-rc5-tks-gitd6ff78479ec5-57'
make: *** [debian/rules:4: build] Error 2
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
debuild: fatal error at line 1182:
dpkg-buildpackage -us -uc -ui -i failed

I am not convinced with this patch.

People would be upset if they got a source package and they did not find the .config in the standard location.

It is difficult to notice the .config has been moved to somewhere until they check debian/rules.

Why do we need to create such a strange package?

Quoted 23 lines
---
 scripts/package/mkdebian | 2 ++
 1 file changed, 2 insertions(+)
scripts/package/mkdebian +2 -1
index edcad61fe3cd..130d3f6696eb 100755
@@ -207,10 +207,12 @@ cat <<EOF > debian/rules
 
 build:
        \$(MAKE) KERNELRELEASE=${version} ARCH=${ARCH} \
+       KCONFIG_CONFIG=${KCONFIG_CONFIG} \
        KBUILD_BUILD_VERSION=${revision} KBUILD_SRC=
 
 binary-arch:
        \$(MAKE) KERNELRELEASE=${version} ARCH=${ARCH} \
+       KCONFIG_CONFIG=${KCONFIG_CONFIG} \
        KBUILD_BUILD_VERSION=${revision} KBUILD_SRC= intdeb-pkg
 
 clean:
--
Quoted 2 lines

2.20.1

--
Best Regards
Masahiro Yamada

#4 lore raw

PATCH 1/1 scripts/kconfig/Makefile: use KCONFIG_CONFIG if set

In reply to Arseny Maslennikov

In a kernel source tree:

% make mrproper
% make KCONFIG_CONFIG=.myconfig kvmconfig'

  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTCC  scripts/kconfig/preprocess.o
  LEX     scripts/kconfig/zconf.lex.c
  YACC    scripts/kconfig/zconf.tab.h
  HOSTCC  scripts/kconfig/zconf.lex.o
  YACC    scripts/kconfig/zconf.tab.c
  HOSTCC  scripts/kconfig/zconf.tab.o
  HOSTLD  scripts/kconfig/conf

The base file '.config' does not exist. Exit.
make[1]: *** [scripts/kconfig/Makefile:96: kvm_guest.config] Error 1
make: *** [Makefile:538: kvmconfig] Error 2

Essentially `make kvmconfig' (and probably other similar targets) did ignore KCONFIG_CONFIG set to a different filename.

If KCONFIG_CONFIG is set, use it; otherwise behave as if it was set to `.config'.

---
 scripts/kconfig/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/kconfig/Makefile +1 -1
index 181973509a05..1c7ac07def4d 100644
@@ -93,7 +93,7 @@ configfiles=$(wildcard $(srctree)/kernel/configs/$@ $(srctree)/arch/$(SRCARCH)/c
 
 %.config: $(obj)/conf
 	$(if $(call configfiles),, $(error No configuration exists for this target on this architecture))
-	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m .config $(configfiles)
+	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m $(KCONFIG_CONFIG) $(configfiles)
 	+$(Q)yes "" | $(MAKE) -f $(srctree)/Makefile oldconfig
 
 PHONY += kvmconfig
Signature · 1 line
2.20.1
Reviewed-by Tested-by on v1 1/1
#5 lore raw

Re: [PATCH] scripts/kconfig/Makefile: use KCONFIG_CONFIG if set

Hi Arseny,

Quoted 20 lines

In a kernel source tree:

% make mrproper
% make KCONFIG_CONFIG=.myconfig kvmconfig'

  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTCC  scripts/kconfig/preprocess.o
  LEX     scripts/kconfig/zconf.lex.c
  YACC    scripts/kconfig/zconf.tab.h
  HOSTCC  scripts/kconfig/zconf.lex.o
  YACC    scripts/kconfig/zconf.tab.c
  HOSTCC  scripts/kconfig/zconf.tab.o
  HOSTLD  scripts/kconfig/conf

The base file '.config' does not exist. Exit.
make[1]: *** [scripts/kconfig/Makefile:96: kvm_guest.config] Error 1
make: *** [Makefile:538: kvmconfig] Error 2

Quoted 7 lines

Essentially `make kvmconfig' (and probably other similar targets) did ignore KCONFIG_CONFIG set to a different filename.

If KCONFIG_CONFIG is set, use it; otherwise behave as if it was set to `.config'.

Good catch :).
Yes, many targets are affected to it: xenconfig, tinyconfig + archs:
rm -f .config; make ARCH=arm KCONFIG_CONFIG=.myconfig dram_0xd0000000.config

Kind regards,
Petr

Quoted 17 lines
---
 scripts/kconfig/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/kconfig/Makefile +1 -1
index 181973509a05..1c7ac07def4d 100644
@@ -93,7 +93,7 @@ configfiles=$(wildcard $(srctree)/kernel/configs/$@ $(srctree)/arch/$(SRCARCH)/c
 
 %.config: $(obj)/conf
 	$(if $(call configfiles),, $(error No configuration exists for this target on this architecture))
-	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m .config $(configfiles)
+	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m $(KCONFIG_CONFIG) $(configfiles)
 	+$(Q)yes "" | $(MAKE) -f $(srctree)/Makefile oldconfig
 
 PHONY += kvmconfig
#6 lore raw

Re: [PATCH] scripts/kconfig/Makefile: use KCONFIG_CONFIG if set

In reply to Arseny Maslennikov
Quoted 20 lines · Arseny Maslennikov

In a kernel source tree:

% make mrproper
% make KCONFIG_CONFIG=.myconfig kvmconfig

  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTCC  scripts/kconfig/preprocess.o
  LEX     scripts/kconfig/zconf.lex.c
  YACC    scripts/kconfig/zconf.tab.h
  HOSTCC  scripts/kconfig/zconf.lex.o
  YACC    scripts/kconfig/zconf.tab.c
  HOSTCC  scripts/kconfig/zconf.tab.o
  HOSTLD  scripts/kconfig/conf

The base file '.config' does not exist. Exit.
make[1]: *** [scripts/kconfig/Makefile:96: kvm_guest.config] Error 1
make: *** [Makefile:538: kvmconfig] Error 2

Quoted 7 lines

Essentially `make kvmconfig' (and probably other similar targets) did ignore KCONFIG_CONFIG set to a different filename.

If KCONFIG_CONFIG is set, use it; otherwise behave as if it was set to `.config'.

Since this patch has nothing to do with debian packaging and thus with the rest of the thread, I'll resend it separately.

Quoted 20 lines
---
 scripts/kconfig/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/kconfig/Makefile +1 -1
index 181973509a05..1c7ac07def4d 100644
@@ -93,7 +93,7 @@ configfiles=$(wildcard $(srctree)/kernel/configs/$@ $(srctree)/arch/$(SRCARCH)/c
 
 %.config: $(obj)/conf
 	$(if $(call configfiles),, $(error No configuration exists for this target on this architecture))
-	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m .config $(configfiles)
+	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m $(KCONFIG_CONFIG) $(configfiles)
 	+$(Q)yes "" | $(MAKE) -f $(srctree)/Makefile oldconfig
 
 PHONY += kvmconfig
Signature · 1 line
2.20.1
Reviewed-by on v1 1/1
#7 lore raw
In reply to Arseny Maslennikov

Hi,

Quoted 11 lines

It is widely known that one can build a kernel without a .config in the source tree by setting KCONFIG_CONFIG equal to the actual configuration file path.

When making a *-pkg target, make(1) prepares a source tarball and tries to pack `.config' in there regardless of the value of KCONFIG_CONFIG, failing spectacularly if .config is absent and packing the wrong config if it exists.
Let's fix that.

Kind regards,
Petr

#8 lore raw
In reply to Arseny Maslennikov
Quoted 3 lines · Arseny Maslennikov

It is widely known that one can build a kernel without a .config in the source tree

"without a .config in the source file" means KCONFIG_CONFIG will probably contain '..' or absolute path.

ex1) KCONFIG_CONFIG=../my_dir/.config

ex2) KCONFIG_CONFIG=/absolute/pass/to/my/.config

Quoted 8 lines

by setting KCONFIG_CONFIG equal to the actual configuration file path.

When making a *-pkg target, make(1) prepares a source tarball and tries to pack `.config' in there regardless of the value of KCONFIG_CONFIG, failing spectacularly if .config is absent and packing the wrong config if it exists.
Let's fix that.

Did you notice the log 'Removing leading ...' from tar for case ex1), ex2) ?

masahiro@grover:~/ref/linux$ make -j8  KCONFIG_CONFIG=../.config  deb-pkg
make clean
/bin/bash ./scripts/package/mkdebian
  TAR     linux-5.0.0+.tar.gz
tar: Removing leading `../' from member names
   ...

Even if I apply 3/3, it will produce a broken source package since the tar will rip off the leading directory path, then it mismatches to KCONFIG_CONFIG recorded in debian/rules.

A simple case like KCONFIG_CONFIG=my.config will work, but I am not convinced with 3/3.

Quoted 19 lines
---
 scripts/package/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/package/Makefile +1 -2
index 453fecee62f0..100a0d28ad25 100644
@@ -27,7 +27,7 @@ KDEB_SOURCENAME ?= linux-$(KERNELRELEASE)
 KBUILD_PKG_ROOTCMD ?="fakeroot -u"
 export KDEB_SOURCENAME
 # Include only those top-level files that are needed by make, plus the GPL copy
-TAR_CONTENT := $(KBUILD_ALLDIRS) .config .scmversion Makefile \
+TAR_CONTENT := $(KBUILD_ALLDIRS) $(KCONFIG_CONFIG) .scmversion Makefile \
                Kbuild Kconfig COPYING $(wildcard localversion*)
 MKSPEC     := $(srctree)/scripts/package/mkspec
 
--
Quoted 2 lines

2.20.1

--
Best Regards
Masahiro Yamada

#9 lore raw
Quoted 5 lines · Masahiro Yamada
Quoted 3 lines · Arseny Maslennikov

It is widely known that one can build a kernel without a .config in the source tree

Quoted 9 lines

"without a .config in the source file" means KCONFIG_CONFIG will probably contain '..' or absolute path.

ex1) KCONFIG_CONFIG=../my_dir/.config

ex2) KCONFIG_CONFIG=/absolute/pass/to/my/.config

Hmm, I did not consider that; my bad, sorry.

Quoted 18 lines
Quoted 4 lines
Quoted 8 lines

by setting KCONFIG_CONFIG equal to the actual configuration file path.

When making a *-pkg target, make(1) prepares a source tarball and tries to pack `.config' in there regardless of the value of KCONFIG_CONFIG, failing spectacularly if .config is absent and packing the wrong config if it exists.
Let's fix that.

Did you notice the log 'Removing leading ...' from tar for case ex1), ex2) ?

Quoted 5 lines
masahiro@grover:~/ref/linux$ make -j8  KCONFIG_CONFIG=../.config  deb-pkg
make clean
/bin/bash ./scripts/package/mkdebian
  TAR     linux-5.0.0+.tar.gz
tar: Removing leading `../' from member names

Yes, I know tar does that. I mostly use KCONFIG_CONFIG with a bunch of config files in the source tree, so did not experience this while making sure the change makes sense.

Quoted 10 lines
   ...

Even if I apply 3/3, it will produce a broken source package since the tar will rip off the leading directory path, then it mismatches to KCONFIG_CONFIG recorded in debian/rules.

A simple case like KCONFIG_CONFIG=my.config will work, but I am not convinced with 3/3.

So instead of this one and 3/3 we could ensure the source tarball contains a .config equal to the contents of KCONFIG_CONFIG provided to make(1), so the source package is fixed and its users are not confused.

If the following is ok, I'll send a v2.
A quick look at the generated orig tarball tells me it ought to work well.

Something like this:

scripts/package/Makefile +1 -0
index 100a0d28ad25..8530fe267a76 100644
@@ -44,6 +44,7 @@ if test "$(objtree)" != "$(srctree)"; then \
 fi ; \
 $(srctree)/scripts/setlocalversion --save-scmversion; \
 tar -cz $(RCS_TAR_IGNORE) -f $(2).tar.gz \
+	--absolute-names --transform 's:^$(KCONFIG_CONFIG):.config:S' \
 	--transform 's:^:$(2)/:S' $(TAR_CONTENT) $(3); \
 rm -f $(objtree)/.scmversion
 
#10 lore raw
Quoted 72 lines · Arseny Maslennikov
Quoted 5 lines · Masahiro Yamada
Quoted 3 lines · Arseny Maslennikov

It is widely known that one can build a kernel without a .config in the source tree

Quoted 9 lines

"without a .config in the source file" means KCONFIG_CONFIG will probably contain '..' or absolute path.

ex1) KCONFIG_CONFIG=../my_dir/.config

ex2) KCONFIG_CONFIG=/absolute/pass/to/my/.config

Hmm, I did not consider that; my bad, sorry.

Quoted 18 lines
Quoted 4 lines
Quoted 8 lines

by setting KCONFIG_CONFIG equal to the actual configuration file path.

When making a *-pkg target, make(1) prepares a source tarball and tries to pack `.config' in there regardless of the value of KCONFIG_CONFIG, failing spectacularly if .config is absent and packing the wrong config if it exists.
Let's fix that.

Did you notice the log 'Removing leading ...' from tar for case ex1), ex2) ?

Quoted 5 lines
masahiro@grover:~/ref/linux$ make -j8  KCONFIG_CONFIG=../.config  deb-pkg
make clean
/bin/bash ./scripts/package/mkdebian
  TAR     linux-5.0.0+.tar.gz
tar: Removing leading `../' from member names

Yes, I know tar does that. I mostly use KCONFIG_CONFIG with a bunch of config files in the source tree, so did not experience this while making sure the change makes sense.

Quoted 10 lines
   ...

Even if I apply 3/3, it will produce a broken source package since the tar will rip off the leading directory path, then it mismatches to KCONFIG_CONFIG recorded in debian/rules.

A simple case like KCONFIG_CONFIG=my.config will work, but I am not convinced with 3/3.

So instead of this one and 3/3 we could ensure the source tarball contains a .config equal to the contents of KCONFIG_CONFIG provided to make(1), so the source package is fixed and its users are not confused.

If the following is ok, I'll send a v2.
A quick look at the generated orig tarball tells me it ought to work well.

Something like this:

scripts/package/Makefile +0 -0
index 100a0d28ad25..8530fe267a76 100644

scripts/package/Makefile if test "$(objtree)" != "$(srctree)"; then \ · line 47

Quoted 4 lines
scripts/package/Makefile +1 -0
@@ -44,6 +44,7 @@ if test "$(objtree)" != "$(srctree)"; then \
 fi ; \
 $(srctree)/scripts/setlocalversion --save-scmversion; \
 tar -cz $(RCS_TAR_IGNORE) -f $(2).tar.gz \
+	--absolute-names --transform 's:^$(KCONFIG_CONFIG):.config:S' \

GNU tar seems to trim the paths of archive members first and only then consider the --transform rules, so it does not work with KCONFIG_CONFIG=../../my.config and the likes if -P/--absolute-names is not provided.

Quoted 3 lines
scripts/package/Makefile +0 -0
 	--transform 's:^:$(2)/:S' $(TAR_CONTENT) $(3); \
 rm -f $(objtree)/.scmversion