kconfig: allow pre-seending randpackageconfig

RFC PATCH New Yann E. MORIN 3 participants 8 messages Jul 19, 2014 – Oct 14, 2014 Kbuild

lore

Share & tools

lore

Yann E. MORIN

author

2014-07-19 23:25 UTC #0 lore raw

RFC PATCH 1/1 kconfig: allow pre-seending randpackageconfig

NOTE: this is only a proof of concept, for initial feedback.

kconfig is not able to properly randomise choices when a base .config file is provided.

Sicne we do have quite a few choices that selects one package or another, and we do want to sometime build them, we need to be able to preseed those packages.

Add a new environment variable that contains the path to a file containing options to be preseeded to randpackageconfig.

Signed-off-by "Yann E. MORIN" <[email protected]>

Cc Thomas Petazzoni <[email protected]>

---
 Makefile | 3 +++
 1 file changed, 3 insertions(+)

Makefile +3 -0

index 1af51de..eeaf7d3 100644
@@ -751,6 +751,9 @@ randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@grep '^config BR2_PACKAGE_' Config.in.legacy | \
 		while read config pkg; do \
 		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
+ifneq ($(BR2_RAND_PRESEED_CONFIG),)
+	@cat $(BR2_RAND_PRESEED_CONFIG) >> $(CONFIG_DIR)/.config.nopkg
+endif
 	@$(COMMON_CONFIG_ENV) \
 		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
 		$< --randconfig $(CONFIG_CONFIG_IN)

Signature · 1 line

1.9.1

Arnout Vandecappelle (Essensium/Mind)

2014-10-12 17:26 UTC #2 lore raw

PATCH 1/1 randconfig: seed with BR2_RAND_PRESEED_CONFIG

In reply to Yann E. MORIN

Currently, randconfig is completely unusable. Instead, we use randpackageconfig, with two use cases: in the autobuilders, and when doing manual testing of the validity of depends/select chains. In both cases, we actually want the possibility to explicitly select some packages or to set a string value or to randomise between a choice or to test bootloaders and kernels - all of this is currently impossible.

With this patch, we can use randconfig instead of randpackageconfig, and provide it with a seed file specified with BR2_RAND_PRESEED_CONFIG.

Signed-off-by Arnout Vandecappelle (Essensium/Mind) <[email protected]>

Cc "Yann E. MORIN" <[email protected]>

Cc Thomas Petazzoni <[email protected]>

---
 Makefile | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

Makefile +12 -2

index c0dcec8..79b12c6 100644
@@ -727,7 +727,16 @@ oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 
 randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
-	@$(COMMON_CONFIG_ENV) $< --randconfig $(CONFIG_CONFIG_IN)
+ifneq ($(BR2_RAND_PRESEED_CONFIG),)
+	@cp $(BR2_RAND_PRESEED_CONFIG) $(CONFIG_DIR)/.config.nopkg
+endif
+	@grep '^config BR2_PACKAGE_' Config.in.legacy | \
+		while read config pkg; do \
+		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
+	@$(COMMON_CONFIG_ENV) \
+		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
+		$< --randconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
 
 allyesconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config

@@ -852,7 +861,8 @@ help:
 	@echo '  oldconfig              - resolve any unresolved symbols in .config'
 	@echo '  silentoldconfig        - Same as oldconfig, but quietly, additionally update deps'
 	@echo '  olddefconfig           - Same as silentoldconfig but sets new symbols to their default value'
-	@echo '  randconfig             - New config with random answer to all options'
+	@echo '  randconfig             - New config with random answer to all options -'
+	@echo '                             options can be forced by setting BR2_RAND_PRESEED_CONFIG'
 	@echo '  defconfig              - New config with default answer to all options'
 	@echo '                             BR2_DEFCONFIG, if set, is used as input'
 	@echo '  savedefconfig          - Save current config as ./defconfig (minimal config)'

Signature · 1 line

2.1.1

Arnout Vandecappelle (Essensium/Mind)

2014-10-12 18:21 UTC #3 lore raw

PATCH v2 1/2 randconfig, allyesconfig: fix handling of legacy options

Randconfig and allyesconfig should never select legacy options, so use the same trick as allpackageyesconfig and randpackageconfig to remove them.

Signed-off-by Arnout Vandecappelle (Essensium/Mind) <[email protected]>

---
 Makefile | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

Makefile +16 -2

index 907a0fc..5bcaa77 100644
@@ -731,11 +731,25 @@ oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 
 randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
-	@$(COMMON_CONFIG_ENV) $< --randconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
+	@grep '^config ' Config.in.legacy | \
+		while read config pkg; do \
+		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
+	@$(COMMON_CONFIG_ENV) \
+		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
+		$< --randconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
 
 allyesconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
-	@$(COMMON_CONFIG_ENV) $< --allyesconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
+	@grep '^config ' Config.in.legacy | \
+		while read config pkg; do \
+		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
+	@$(COMMON_CONFIG_ENV) \
+		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
+		$< --allyesconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
 
 allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config

Signature · 1 line

2.1.1

Arnout Vandecappelle (Essensium/Mind)

2014-10-12 18:21 UTC #4 lore raw

PATCH v2 2/2 randconfig: seed with BR2_RAND_PRESEED_CONFIG_FILES

This patch adds a BR2_RAND_PRESEED_CONFIG_FILES environment variable that can be set to a (set of) files with config options that should be set or unset. randconfig and randpackageconfig will force these values.

Signed-off-by Arnout Vandecappelle (Essensium/Mind) <[email protected]>

Cc "Yann E. MORIN" <[email protected]>

Cc Thomas Petazzoni <[email protected]>

---
v2:
- split off the fix for legacy
- apply to both randconfig and randpackageconfig
- rename to BR2_RAND_PRESEED_CONFIG_FILES because it supports multiple files

Signed-off-by Arnout Vandecappelle (Essensium/Mind) <[email protected]>

---
 Makefile | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

Makefile +10 -2

index 5bcaa77..fb99de1 100644
@@ -732,6 +732,9 @@ oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@rm -f $(CONFIG_DIR)/.config.nopkg
+ifneq ($(BR2_RAND_PRESEED_CONFIG_FILES),)
+	@cp $(BR2_RAND_PRESEED_CONFIG_FILES) $(CONFIG_DIR)/.config.nopkg
+endif
 	@grep '^config ' Config.in.legacy | \
 		while read config pkg; do \
 		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done

@@ -758,6 +761,9 @@ allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@grep -v BR2_PACKAGE_ $(BR2_CONFIG) > $(CONFIG_DIR)/.config.nopkg
+ifneq ($(BR2_RAND_PRESEED_CONFIG_FILES),)
+	@grep BR2_PACKAGE_ $(BR2_RAND_PRESEED_CONFIG_FILES) >> $(CONFIG_DIR)/.config.nopkg
+endif
 	@grep '^config BR2_PACKAGE_' Config.in.legacy | \
 		while read config pkg; do \
 		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done

@@ -870,13 +876,15 @@ help:
 	@echo '  oldconfig              - resolve any unresolved symbols in .config'
 	@echo '  silentoldconfig        - Same as oldconfig, but quietly, additionally update deps'
 	@echo '  olddefconfig           - Same as silentoldconfig but sets new symbols to their default value'
-	@echo '  randconfig             - New config with random answer to all options'
+	@echo '  randconfig             - New config with random answer to all options -'
+	@echo '                             options can be forced by setting BR2_RAND_PRESEED_CONFIG_FILES'
 	@echo '  defconfig              - New config with default answer to all options'
 	@echo '                             BR2_DEFCONFIG, if set, is used as input'
 	@echo '  savedefconfig          - Save current config as ./defconfig (minimal config)'
 	@echo '  allyesconfig           - New config where all options are accepted with yes'
 	@echo '  allnoconfig            - New config where all options are answered with no'
-	@echo '  randpackageconfig      - New config with random answer to package options'
+	@echo '  randpackageconfig      - New config with random answer to package options -'
+	@echo '                             options can be forced by setting BR2_RAND_PRESEED_CONFIG_FILES'
 	@echo '  allyespackageconfig    - New config where pkg options are accepted with yes'
 	@echo '  allnopackageconfig     - New config where package options are answered with no'
 ifeq ($(BR2_PACKAGE_BUSYBOX),y)

Signature · 1 line

2.1.1

Yann E. MORIN

author

2014-10-14 07:32 UTC #5 lore raw

PATCH v2 2/2 randconfig: seed with BR2_RAND_PRESEED_CONFIG_FILES

Arnout, All,

On 2014-10-12 20:21 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:

Quoted 32 lines

Currently, randconfig is completely unusable. Instead, we use randpackageconfig, with two use cases: in the autobuilders, and when doing manual testing of the validity of depends/select chains. In both cases, we actually want the possibility to explicitly select some packages or to set a string value or to randomise between a choice or to test bootloaders and kernels - all of this is currently impossible.
This patch adds a BR2_RAND_PRESEED_CONFIG_FILES environment variable that can be set to a (set of) files with config options that should be set or unset. randconfig and randpackageconfig will force these values.
Signed-off-by Arnout Vandecappelle (Essensium/Mind) <[email protected]>
Cc "Yann E. MORIN" <[email protected]>
Cc Thomas Petazzoni <[email protected]>
---
v2:
- split off the fix for legacy
- apply to both randconfig and randpackageconfig
- rename to BR2_RAND_PRESEED_CONFIG_FILES because it supports multiple files
Signed-off-by Arnout Vandecappelle (Essensium/Mind) <[email protected]>
---
 Makefile | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
Makefile +0 -0
index 5bcaa77..fb99de1 100644

Makefile oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile · line 737

Quoted 6 lines

Makefile +3 -0

@@ -732,6 +732,9 @@ oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@rm -f $(CONFIG_DIR)/.config.nopkg
+ifneq ($(BR2_RAND_PRESEED_CONFIG_FILES),)
+	@cp $(BR2_RAND_PRESEED_CONFIG_FILES) $(CONFIG_DIR)/.config.nopkg
+endif

I guess you meant something like:

    @cat $(BR2_RAND_PRESEED_CONFIG_FILES) $>(CONFIG_DIR)/.config.nopkg

Here's what I get with cp:

    $ cp foo bar buz
    cp: target ?buz? is not a directory

Makefile allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile · line 766

Quoted 10 lines

Makefile +3 -0

 	@grep '^config ' Config.in.legacy | \
 		while read config pkg; do \
 		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done

@@ -758,6 +761,9 @@ allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
 	@grep -v BR2_PACKAGE_ $(BR2_CONFIG) > $(CONFIG_DIR)/.config.nopkg
+ifneq ($(BR2_RAND_PRESEED_CONFIG_FILES),)
+	@grep BR2_PACKAGE_ $(BR2_RAND_PRESEED_CONFIG_FILES) >> $(CONFIG_DIR)/.config.nopkg
+endif

Why do you explicitly grep for packages here?

Regards,
Yann E. MORIN.

Signature · 6 lines

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Arnout Vandecappelle

2014-10-14 09:08 UTC #6 lore raw

PATCH v2 2/2 randconfig: seed with BR2_RAND_PRESEED_CONFIG_FILES

Quoted 3 lines · Yann E. MORIN

Arnout, All,
On 2014-10-12 20:21 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:

[snip]

Quoted 13 lines

Makefile oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile · line 737

Quoted 11 lines

Makefile +3 -0

index 5bcaa77..fb99de1 100644
@@ -732,6 +732,9 @@ oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
     @mkdir -p $(BUILD_DIR)/buildroot-config
     @rm -f $(CONFIG_DIR)/.config.nopkg
+ifneq ($(BR2_RAND_PRESEED_CONFIG_FILES),)
+    @cp $(BR2_RAND_PRESEED_CONFIG_FILES) $(CONFIG_DIR)/.config.nopkg
+endif

I guess you meant something like:

Quoted 5 lines

    @cat $(BR2_RAND_PRESEED_CONFIG_FILES) $>(CONFIG_DIR)/.config.nopkg

Here's what I get with cp:

    $ cp foo bar buz
    cp: target ?buz? is not a directory

Well spotted, thanks. I changed _FILE into _FILES at the last moment
but never tested it.

Quoted 11 lines

Makefile allnoconfig: $(BUILD_DIR)/buildroot-config/conf

Quoted 5 lines

Makefile +0 -0

     @grep '^config ' Config.in.legacy | \
         while read config pkg; do \
         echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done

@@ -758,6 +761,9 @@ allnoconfig: $(BUILD_DIR)/buildroot-config/conf

outputmakefile

Makefile allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile

Quoted 5 lines

Makefile +2 -0

 randpackageconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
     @mkdir -p $(BUILD_DIR)/buildroot-config
     @grep -v BR2_PACKAGE_ $(BR2_CONFIG) > $(CONFIG_DIR)/.config.nopkg
+ifneq ($(BR2_RAND_PRESEED_CONFIG_FILES),)
+    @grep BR2_PACKAGE_ $(BR2_RAND_PRESEED_CONFIG_FILES) >>

Quoted 4 lines

$(CONFIG_DIR)/.config.nopkg
Makefile allnoconfig: $(BUILD_DIR)/buildroot-config/conf · line 766
Quoted 1 line
Makefile +1 -0
+endif
Why do you explicitly grep for packages here?

.config.nopkg will already contain the config of everything else (cfr.
the line above). So if you keep the full preseed file, you'll get duplicate definitions in the .config.nopkg.

Of course, if you do

cp foo .config
make randpackageconfig BR2_RAND_PRESEED_CONFIG_FILES=bar

then the .config only contains whatever you had in foo, which is presumably a toolchain config fragment. But that use case is better handled with

make randconfig BR2_RAND_PRESEED_CONFIG_FILES="foo bar"

Regards,
Arnout

Signature · 6 lines

Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

Yann E. MORIN

author

2014-10-14 07:18 UTC #7 lore raw

PATCH v2 1/2 randconfig, allyesconfig: fix handling of legacy options

In reply to Arnout Vandecappelle (Essensium/Mind)

Arnout, All,

On 2014-10-12 20:21 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:

Quoted 5 lines

Randconfig and allyesconfig should never select legacy options, so use the same trick as allpackageyesconfig and randpackageconfig to remove them.
Signed-off-by Arnout Vandecappelle (Essensium/Mind) <[email protected]>

Reviewed-by "Yann E. MORIN" <[email protected]>

Regards,
Yann E. MORIN.

Quoted 43 lines

---
 Makefile | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

Makefile +16 -2

index 907a0fc..5bcaa77 100644
@@ -731,11 +731,25 @@ oldconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 
 randconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
-	@$(COMMON_CONFIG_ENV) $< --randconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
+	@grep '^config ' Config.in.legacy | \
+		while read config pkg; do \
+		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
+	@$(COMMON_CONFIG_ENV) \
+		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
+		$< --randconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
 
 allyesconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config
-	@$(COMMON_CONFIG_ENV) $< --allyesconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
+	@grep '^config ' Config.in.legacy | \
+		while read config pkg; do \
+		echo "# $$pkg is not set" >> $(CONFIG_DIR)/.config.nopkg; done
+	@$(COMMON_CONFIG_ENV) \
+		KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
+		$< --allyesconfig $(CONFIG_CONFIG_IN)
+	@rm -f $(CONFIG_DIR)/.config.nopkg
 
 allnoconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@mkdir -p $(BUILD_DIR)/buildroot-config

Signature · 6 lines

2.1.1

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Signature · 6 lines

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'