Skip to contentSkip to conversation

core/pkg-kconfig: Use olddefconfig when available

PATCH Superseded Romain Izard 3 participants 5 messages Apr 20, 2016 – Apr 22, 2016

v6 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

PATCH v6 1/1 core/pkg-kconfig: Use olddefconfig when available

Using 'yes "" | make oldconfig' to regenerate a .config file from an existing defconfig does not work reliably in all cases. Specifically, it does not work well with tristate choice entries.

The correct way to do it is to use 'make olddefconfig', but this target is not supported in projects that use an old version of kconfig. As most projects do not use tristate entries, there is no need for them to upgrade.

'oldnoconfig' is an alias for 'olddefconfig', but it also is the only name for this target between Linux versions 2.6.36 and 3.6.

Check which targets are supported by the current project, and use the appropriate method to regenerate the .config file.

---
Changelog:
v1: New patch
v2: Use oldnoconfig instead of olddefconfig
Fix silent mode
Change the pattern in both places where it is used Use a macro to avoid redundancy
v3: Do not use $(call ...)

    Use the $(2)_KCONFIG_MAKE variable

v4: No change
v5: Detect available methods and use the most recent one
v6: Remove colon from matched rule names

 package/pkg-kconfig.mk | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)
package/pkg-kconfig.mk +21 -4
index 4033915..478f5bb 100644
@@ -60,18 +60,35 @@ $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES): | $(1)-patch
 $(2)_KCONFIG_MAKE = \
 	$$($(2)_MAKE_ENV) $$(MAKE) -C $$($(2)_DIR) $$($(2)_KCONFIG_OPTS)
 
+$(2)_KCONFIG_RULES = \
+	$$(shell $$($(2)_KCONFIG_MAKE) -pn config 2>/dev/null | \
+		sed 's/^\([_0-9a-zA-Z]*config\):.*/\1/ p; d')
+
+# The correct way to regenerate a .config file is to use 'make olddefconfig'.
+# For historical reasons, the target name is 'oldnoconfig' between Linux kernel
+# versions 2.6.36 and 3.6, and remains as an alias in later versions.
+# In older versions, and in some other projects that use kconfig, the target is
+# not supported at all, and we use 'yes "" | make oldconfig' as a fallback
+# only, as this can fail in complex cases.
+define $(2)_REGEN_DOT_CONFIG
+	$$(if $$(filter olddefconfig,$$($(2)_KCONFIG_RULES)),
+		$$(Q)$$($(2)_KCONFIG_MAKE) olddefconfig,
+		$$(if $$(filter oldnoconfig,$$($(2)_KCONFIG_RULES)),
+			$$(Q)$$($(2)_KCONFIG_MAKE) oldnoconfig,
+			$$(Q)(yes "" | $$($(2)_KCONFIG_MAKE) oldconfig)))
+endef
+
 # The specified source configuration file and any additional configuration file
 # fragments are merged together to .config, after the package has been patched.
 # Since the file could be a defconfig file it needs to be expanded to a
-# full .config first. We use 'make oldconfig' because this can be safely
-# done even when the package does not support defconfigs.
+# full .config first.
 $$($(2)_DIR)/.config: $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES)
 	$$(Q)$$(if $$($(2)_KCONFIG_DEFCONFIG), \
 		$$($(2)_KCONFIG_MAKE) $$($(2)_KCONFIG_DEFCONFIG), \
 		cp $$($(2)_KCONFIG_FILE) $$(@))
 	$$(Q)support/kconfig/merge_config.sh -m -O $$(@D) \
 		$$(@) $$($(2)_KCONFIG_FRAGMENT_FILES)
-	$$(Q)yes "" | $$($(2)_KCONFIG_MAKE) oldconfig
+	$$($(2)_REGEN_DOT_CONFIG)
 
 # If _KCONFIG_FILE or _KCONFIG_FRAGMENT_FILES exists, this dependency is
 # already implied, but if we only have a _KCONFIG_DEFCONFIG we have to add
@@ -82,7 +99,7 @@ $$($(2)_DIR)/.config: | $(1)-patch
 # The exact rules are specified by the package .mk file.
 define $(2)_FIXUP_DOT_CONFIG
 	$$($(2)_KCONFIG_FIXUP_CMDS)
-	$$(Q)yes "" | $$($(2)_KCONFIG_MAKE) oldconfig
+	$$($(2)_REGEN_DOT_CONFIG)
 	$$(Q)touch $$($(2)_DIR)/.stamp_kconfig_fixup_done
 endef
 
Signature · 1 line
2.5.0
#1 lore raw

PATCH v6 1/1 core/pkg-kconfig: Use olddefconfig when available

Romain, All,

On 2016-04-20 11:54 +0200, Romain Izard spake thusly:

Quoted 38 lines

Using 'yes "" | make oldconfig' to regenerate a .config file from an existing defconfig does not work reliably in all cases. Specifically, it does not work well with tristate choice entries.

The correct way to do it is to use 'make olddefconfig', but this target is not supported in projects that use an old version of kconfig. As most projects do not use tristate entries, there is no need for them to upgrade.

'oldnoconfig' is an alias for 'olddefconfig', but it also is the only name for this target between Linux versions 2.6.36 and 3.6.

Check which targets are supported by the current project, and use the appropriate method to regenerate the .config file.

---
Changelog:
v1: New patch
v2: Use oldnoconfig instead of olddefconfig
Fix silent mode
Change the pattern in both places where it is used Use a macro to avoid redundancy
v3: Do not use $(call ...)

    Use the $(2)_KCONFIG_MAKE variable

v4: No change
v5: Detect available methods and use the most recent one
v6: Remove colon from matched rule names

 package/pkg-kconfig.mk | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)
package/pkg-kconfig.mk +0 -0
index 4033915..478f5bb 100644

package/pkg-kconfig.mk $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES): | $(1)-patch · line 65

Quoted 6 lines
package/pkg-kconfig.mk +3 -0
@@ -60,18 +60,35 @@ $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES): | $(1)-patch
 $(2)_KCONFIG_MAKE = \
 	$$($(2)_MAKE_ENV) $$(MAKE) -C $$($(2)_DIR) $$($(2)_KCONFIG_OPTS)
 
+$(2)_KCONFIG_RULES = \
+	$$(shell $$($(2)_KCONFIG_MAKE) -pn config 2>/dev/null | \
+		sed 's/^\([_0-9a-zA-Z]*config\):.*/\1/ p; d')

Ok, so... Well... Err...

Could you respin v4 of the patch, the one that used something like:

    make oldnoconfig || (yes "" |make oldconfig)

please? ;-)

OK, what's wrong with my proposal? It works in an unconfigured tree. But as soon as the tree is configured, it breaks, notably with older kernels, like 3.0.

    $ cd linux-3.0
    $ make -qpn oldefconfig 2>/dev/null |grep olddefconfig
    [nothing]
    $ touch .config
    $ make -qpn oldefconfig 2>/dev/null |grep olddefconfig
    olddefconfig: scripts_basic outputmakefile FORCE

But 3.0 does not have olddefconfig. It's jsut the %config generic rule kicking in. So we'd believe there would be support for olddefconfig (or oldnoconfig) when there isn't.

So, I have to apologise for the delusion. Your v4 is the best we can do.

Thank you! :-)

Regards,
Yann E. MORIN.

Quoted 43 lines
package/pkg-kconfig.mk +17 -4
+# The correct way to regenerate a .config file is to use 'make olddefconfig'.
+# For historical reasons, the target name is 'oldnoconfig' between Linux kernel
+# versions 2.6.36 and 3.6, and remains as an alias in later versions.
+# In older versions, and in some other projects that use kconfig, the target is
+# not supported at all, and we use 'yes "" | make oldconfig' as a fallback
+# only, as this can fail in complex cases.
+define $(2)_REGEN_DOT_CONFIG
+	$$(if $$(filter olddefconfig,$$($(2)_KCONFIG_RULES)),
+		$$(Q)$$($(2)_KCONFIG_MAKE) olddefconfig,
+		$$(if $$(filter oldnoconfig,$$($(2)_KCONFIG_RULES)),
+			$$(Q)$$($(2)_KCONFIG_MAKE) oldnoconfig,
+			$$(Q)(yes "" | $$($(2)_KCONFIG_MAKE) oldconfig)))
+endef
+
 # The specified source configuration file and any additional configuration file
 # fragments are merged together to .config, after the package has been patched.
 # Since the file could be a defconfig file it needs to be expanded to a
-# full .config first. We use 'make oldconfig' because this can be safely
-# done even when the package does not support defconfigs.
+# full .config first.
 $$($(2)_DIR)/.config: $$($(2)_KCONFIG_FILE) $$($(2)_KCONFIG_FRAGMENT_FILES)
 	$$(Q)$$(if $$($(2)_KCONFIG_DEFCONFIG), \
 		$$($(2)_KCONFIG_MAKE) $$($(2)_KCONFIG_DEFCONFIG), \
 		cp $$($(2)_KCONFIG_FILE) $$(@))
 	$$(Q)support/kconfig/merge_config.sh -m -O $$(@D) \
 		$$(@) $$($(2)_KCONFIG_FRAGMENT_FILES)
-	$$(Q)yes "" | $$($(2)_KCONFIG_MAKE) oldconfig
+	$$($(2)_REGEN_DOT_CONFIG)
 
 # If _KCONFIG_FILE or _KCONFIG_FRAGMENT_FILES exists, this dependency is
 # already implied, but if we only have a _KCONFIG_DEFCONFIG we have to add
@@ -82,7 +99,7 @@ $$($(2)_DIR)/.config: | $(1)-patch
 # The exact rules are specified by the package .mk file.
 define $(2)_FIXUP_DOT_CONFIG
 	$$($(2)_KCONFIG_FIXUP_CMDS)
-	$$(Q)yes "" | $$($(2)_KCONFIG_MAKE) oldconfig
+	$$($(2)_REGEN_DOT_CONFIG)
 	$$(Q)touch $$($(2)_DIR)/.stamp_kconfig_fixup_done
 endef
 
Signature · 1 line
2.5.0
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#2 lore raw

PATCH v6 1/1 core/pkg-kconfig: Use olddefconfig when available

Yann, all,

2016-04-22 0:47 GMT+02:00 Yann E. MORIN <[email protected]>:

Quoted 20 lines

Ok, so... Well... Err...

Could you respin v4 of the patch, the one that used something like:

    make oldnoconfig || (yes "" |make oldconfig)

please? ;-)

OK, what's wrong with my proposal? It works in an unconfigured tree. But as soon as the tree is configured, it breaks, notably with older kernels, like 3.0.

    $ cd linux-3.0
    $ make -qpn oldefconfig 2>/dev/null |grep olddefconfig
    [nothing]
    $ touch .config
    $ make -qpn oldefconfig 2>/dev/null |grep olddefconfig
    olddefconfig: scripts_basic outputmakefile FORCE
Quoted 6 lines

But 3.0 does not have olddefconfig. It's jsut the %config generic rule kicking in. So we'd believe there would be support for olddefconfig (or oldnoconfig) when there isn't.

So, I have to apologise for the delusion. Your v4 is the best we can do.

After some more testing, I believe it should work. I've tested the following command on the latest commit for all stable branches since 2.6.11, and I can reliably detect which targets are supported. Moreover, I have the same output whether I touched .config or not.

make -pn O=dir config 2>/dev/null | grep -oE '^old(no|def)config:'

I believe the problem with your tests is that you try to use both '-q' and '-n', while GNU Make's manual states (Chapter 9.3, Instead of executing recipes) that:

"It is an error to use more than one of these three flags [-t -q -n] in the same invocation of 'make'."

Best regards,

Signature · 1 line
Romain Izard
#3 lore raw

PATCH v6 1/1 core/pkg-kconfig: Use olddefconfig when available

Romain, All,

On 2016-04-22 15:51 +0200, Romain Izard spake thusly:

Quoted 33 lines

2016-04-22 0:47 GMT+02:00 Yann E. MORIN <[email protected]>:

Quoted 20 lines

Ok, so... Well... Err...

Could you respin v4 of the patch, the one that used something like:

    make oldnoconfig || (yes "" |make oldconfig)

please? ;-)

OK, what's wrong with my proposal? It works in an unconfigured tree. But as soon as the tree is configured, it breaks, notably with older kernels, like 3.0.

    $ cd linux-3.0
    $ make -qpn oldefconfig 2>/dev/null |grep olddefconfig
    [nothing]
    $ touch .config
    $ make -qpn oldefconfig 2>/dev/null |grep olddefconfig
    olddefconfig: scripts_basic outputmakefile FORCE
Quoted 6 lines

But 3.0 does not have olddefconfig. It's jsut the %config generic rule kicking in. So we'd believe there would be support for olddefconfig (or oldnoconfig) when there isn't.

So, I have to apologise for the delusion. Your v4 is the best we can do.

After some more testing, I believe it should work. I've tested the following command on the latest commit for all stable branches since 2.6.11, and I can reliably detect which targets are supported. Moreover, I have the same output whether I touched .config or not.

Quoted 8 lines

make -pn O=dir config 2>/dev/null | grep -oE '^old(no|def)config:'

I believe the problem with your tests is that you try to use both '-q' and '-n', while GNU Make's manual states (Chapter 9.3, Instead of executing recipes) that:

"It is an error to use more than one of these three flags [-t -q -n] in the same invocation of 'make'."

Weird, my make manpage does not state anything about that...

Anyway, I'll do some more testing following your input. Thanks for your patience! ;-)

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#4 lore raw

PATCH v6 1/1 core/pkg-kconfig: Use olddefconfig when available

In reply to Romain Izard
Quoted 30 lines · Romain Izard

Yann, all,

2016-04-22 0:47 GMT+02:00 Yann E. MORIN <[email protected]>:

Quoted 20 lines

Ok, so... Well... Err...

Could you respin v4 of the patch, the one that used something like:

     make oldnoconfig || (yes "" |make oldconfig)

please? ;-)

OK, what's wrong with my proposal? It works in an unconfigured tree. But as soon as the tree is configured, it breaks, notably with older kernels, like 3.0.

     $ cd linux-3.0
     $ make -qpn oldefconfig 2>/dev/null |grep olddefconfig
     [nothing]
     $ touch .config
     $ make -qpn oldefconfig 2>/dev/null |grep olddefconfig
     olddefconfig: scripts_basic outputmakefile FORCE
Quoted 6 lines

But 3.0 does not have olddefconfig. It's jsut the %config generic rule kicking in. So we'd believe there would be support for olddefconfig (or oldnoconfig) when there isn't.

So, I have to apologise for the delusion. Your v4 is the best we can do.

Quoted 6 lines

After some more testing, I believe it should work. I've tested the following command on the latest commit for all stable branches since 2.6.11, and I can reliably detect which targets are supported. Moreover, I have the same output whether I touched .config or not.

make -pn O=dir config 2>/dev/null | grep -oE '^old(no|def)config:'

This indeed seems to work reliably.

Yann discovered that when you run exactly the same command twice you get
different results [1]. But that's because he uses .DEFAULT as the target instead of config. The problem in that case is that oldno/defconfig don't appear in the top-level Makefile, so they won't be printed. Only the first time it is printed, because of include/config/auto.conf: this is include'd in the Makefile so it will be regenerated (because of the include/config/%.conf: $(KCONFIG_CONFIG) include/config/auto.conf.cmd dependency), which calls the sub-make in scripts/kconfig, which does have the olddefconfig target so it is printed. The second time, auto.conf is newer than .config so it is not regenerated, so the olddefconfig target is not printed.

Using config as the target elegantly solves that because it forces calling the
submake in scripts/kconfig.

So yes, you v6 would work I think.

However, I think we (= Yann and I) mentioned at some point that we preferred
everything in one command, using grep instead of filter. Or was there a counter-argument against that?

Quoted 7 lines

I believe the problem with your tests is that you try to use both '-q' and '-n', while GNU Make's manual states (Chapter 9.3, Instead of executing recipes) that:

"It is an error to use more than one of these three flags [-t -q -n] in the same invocation of 'make'."

I think the issue is that -q doesn't cause calling sub-make's, ever. That's
because it's not needed: if a sub-make is going to be called, something is going to be run so it can return 1 immediately without actually calling the sub-make. So with -q it is in fact guaranteed never to print an olddefconfig.

Regards,
Arnout

[1] http://code.bulix.org/qwcju4-97419?raw

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF