consolidate in kernel configuration

It appears that when we enable CONFIG_IKCONFIG and CONFIG_IKCONFIG_PROC that we actually include two different copies of the .config file. One plain text and one compressed. The plain text one used when extracting from the binary is also not recovered cleanly. The patch below removes this duplication using the same compressed version for both purposes. Hopefully this will also make it more reasonable to default this option to on.

-apw

=== 8< ====
Being able to recover the configuration from a kernel is very useful and it would be nice to default this option to Yes. Currently, to have the config available both from the image (using extract-ikconfig) and via /proc we keep two copies of the original .config in the kernel. One in plain text and one gzip compressed. This is not optimal.

This patch removes the plain text version of the configuration and updates the extraction tools to locate and use the gzip'd version of the file. This has the added bonus of providing us with the exact same results in both cases, the original .config; including the comments.

---

diff -X /home/apw/brief/lib/vdiff.excl -rupN reference/kernel/Makefile current/kernel/Makefile
@@ -34,23 +34,7 @@ ifneq ($(CONFIG_IA64),y)
 CFLAGS_sched.o := $(PROFILING) -fno-omit-frame-pointer
 endif
 
-# configs.o uses generated files - dependecies must be listed explicitly
-$(obj)/configs.o: $(obj)/ikconfig.h
-
-ifdef CONFIG_IKCONFIG_PROC
 $(obj)/configs.o: $(obj)/config_data.h
-endif
-
-# ikconfig.h contains all the selected config entries - generated
-# from top-level Makefile and .config. Info from ikconfig.h can
-# be extracted from the kernel binary.
-
-quiet_cmd_ikconfig = IKCFG   $@
-      cmd_ikconfig = $(CONFIG_SHELL) $< .config $(srctree)/Makefile > $@
-
-targets += ikconfig.h
-$(obj)/ikconfig.h: scripts/mkconfigs .config Makefile FORCE
-	$(call if_changed,ikconfig)
 
 # config_data.h contains the same information as ikconfig.h but gzipped.
 # Info from config_data can be extracted from /proc/config*

@@ -59,7 +43,7 @@ $(obj)/config_data.gz: .config FORCE
 	$(call if_changed,gzip)
 
 quiet_cmd_ikconfiggz = IKCFG   $@
-      cmd_ikconfiggz = cat $< | scripts/bin2c kernel_config_data > $@
+      cmd_ikconfiggz = (echo "const char kernel_config_data[] = MAGIC_START"; cat $< | scripts/bin2c; echo "MAGIC_END;") > $@
 targets += config_data.h
 $(obj)/config_data.h: $(obj)/config_data.gz FORCE
 	$(call if_changed,ikconfiggz)

diff -X /home/apw/brief/lib/vdiff.excl -rupN reference/kernel/configs.c current/kernel/configs.c
@@ -34,13 +34,26 @@
 /**************************************************/
 /* the actual current config file                 */
 
-/* This one is for extraction from the kernel binary file image. */
-#include "ikconfig.h"
+/*
+ * Define kernel_config_data and kernel_config_data_size, which contains the
+ * wrapped and compressed configuration file.  The file is first compressed
+ * with gzip and then bounded by two eight byte magic numbers to allow
+ * extraction from a binary kernel image:
+ *
+ *   IKCFG_ST
+ *   <image>
+ *   IKCFG_ED
+ */
+#define MAGIC_START	"IKCFG_ST"
+#define MAGIC_END	"IKCFG_ED"
+#include "config_data.h"
 
-#ifdef CONFIG_IKCONFIG_PROC
 
-/* This is the data that can be read from /proc/config.gz. */
-#include "config_data.h"
+#define MAGIC_SIZE (sizeof(MAGIC_START) - 1)
+#define kernel_config_data_size \
+	(sizeof(kernel_config_data) - MAGIC_SIZE * 2)
+
+#ifdef CONFIG_IKCONFIG_PROC
 
 /**************************************************/
 /* globals and useful constants                   */

@@ -58,7 +71,7 @@ ikconfig_read_current(struct file *file,
 		return 0;
 
 	count = min(len, (size_t)(kernel_config_data_size - pos));
-	if(copy_to_user(buf, kernel_config_data + pos, count))
+	if (copy_to_user(buf, kernel_config_data + MAGIC_SIZE + pos, count))
 		return -EFAULT;
 
 	*offset += count;

Andy, this needs rework for top-level-Makefile changes which Linus recently merged.

Apologies should have checked -bk before posting. Here is an updated patch against linux-2.6.7-bk6.

-apw

=== 8< ===
Being able to recover the configuration from a kernel is very useful and it would be nice to default this option to Yes. Currently, to have the config available both from the image (using extract-ikconfig) and via /proc we keep two copies of the original .config in the kernel. One in plain text and one gzip compressed. This is not optimal.

This patch removes the plain text version of the configuration and updates the extraction tools to locate and use the gzip'd version of the file. This has the added bonus of providing us with the exact same results in both cases, the original .config; including the comments.

Revision: $Rev: 274 $

---

diff -upN reference/kernel/Makefile current/kernel/Makefile
@@ -33,23 +33,7 @@ ifneq ($(CONFIG_IA64),y)
 CFLAGS_sched.o := $(PROFILING) -fno-omit-frame-pointer
 endif
 
-# configs.o uses generated files - dependecies must be listed explicitly
-$(obj)/configs.o: $(obj)/ikconfig.h
-
-ifdef CONFIG_IKCONFIG_PROC
 $(obj)/configs.o: $(obj)/config_data.h
-endif
-
-# ikconfig.h contains all the selected config entries - generated
-# from top-level Makefile and .config. Info from ikconfig.h can
-# be extracted from the kernel binary.
-
-quiet_cmd_ikconfig = IKCFG   $@
-      cmd_ikconfig = $(CONFIG_SHELL) $< .config $(srctree)/Makefile > $@
-
-targets += ikconfig.h
-$(obj)/ikconfig.h: scripts/mkconfigs .config $(srctree)/Makefile FORCE
-	$(call if_changed,ikconfig)
 
 # config_data.h contains the same information as ikconfig.h but gzipped.
 # Info from config_data can be extracted from /proc/config*

@@ -58,7 +42,7 @@ $(obj)/config_data.gz: .config FORCE
 	$(call if_changed,gzip)
 
 quiet_cmd_ikconfiggz = IKCFG   $@
-      cmd_ikconfiggz = cat $< | scripts/bin2c kernel_config_data > $@
+      cmd_ikconfiggz = (echo "const char kernel_config_data[] = MAGIC_START"; cat $< | scripts/bin2c; echo "MAGIC_END;") > $@
 targets += config_data.h
 $(obj)/config_data.h: $(obj)/config_data.gz FORCE
 	$(call if_changed,ikconfiggz)

diff -upN reference/kernel/configs.c current/kernel/configs.c
@@ -34,13 +34,26 @@
 /**************************************************/
 /* the actual current config file                 */
 
-/* This one is for extraction from the kernel binary file image. */
-#include "ikconfig.h"
+/*
+ * Define kernel_config_data and kernel_config_data_size, which contains the
+ * wrapped and compressed configuration file.  The file is first compressed
+ * with gzip and then bounded by two eight byte magic numbers to allow
+ * extraction from a binary kernel image:
+ *
+ *   IKCFG_ST
+ *   <image>
+ *   IKCFG_ED
+ */
+#define MAGIC_START	"IKCFG_ST"
+#define MAGIC_END	"IKCFG_ED"
+#include "config_data.h"
 
-#ifdef CONFIG_IKCONFIG_PROC
 
-/* This is the data that can be read from /proc/config.gz. */
-#include "config_data.h"
+#define MAGIC_SIZE (sizeof(MAGIC_START) - 1)
+#define kernel_config_data_size \
+	(sizeof(kernel_config_data) - 1 - MAGIC_SIZE * 2)
+
+#ifdef CONFIG_IKCONFIG_PROC
 
 /**************************************************/
 /* globals and useful constants                   */

@@ -58,7 +71,7 @@ ikconfig_read_current(struct file *file,
 		return 0;
 
 	count = min(len, (size_t)(kernel_config_data_size - pos));
-	if(copy_to_user(buf, kernel_config_data + pos, count))
+	if (copy_to_user(buf, kernel_config_data + MAGIC_SIZE + pos, count))
 		return -EFAULT;
 
 	*offset += count;

On Wed, 23 Jun 2004 15:28:56 +0100 Andy Whitcroft wrote:

| === 8< ===
| Being able to recover the configuration from a kernel is very
| useful and it would be nice to default this option to Yes.
| Currently, to have the config available both from the image (using
| extract-ikconfig) and via /proc we keep two copies of the original
| .config in the kernel.  One in plain text and one gzip compressed.
| This is not optimal.
| 
| This patch removes the plain text version of the configuration and
| updates the extraction tools to locate and use the gzip'd version
| of the file.  This has the added bonus of providing us with the
| exact same results in both cases, the original .config; including
| the comments.
| 
| Revision: $Rev: 274 $
| 
| Signed-off-by: Andy Whitcroft <[email protected]>
| 
| ---

Yes, looks good and works for me.

We also need a decent way to run scripts/extract-ikconfig and build scripts/binoffset. I recall that SuSE has/had a 'make cloneconfig' once upon a time (in 2.4.x). I've begun on a 'make getconfig' (but I can rename it to 'cloneconfig' if that's more appropriate), but it needs some more work (i.e., not yet working). [patch below]

Thanks,
--
~Randy

(beginning, not working...)
add 'getconfig' target to extract kernel config from the kernel image file

diffstat:=

 Makefile         |    6 ++++++
 scripts/Makefile |    3 ++-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff -Naurp ./scripts/Makefile~getconfig ./scripts/Makefile
@@ -5,7 +5,8 @@
 # docproc: 	 Preprocess .tmpl file in order to generate .sgml docs
 # conmakehash:	 Create arrays for initializing the kernel console tables
 
-host-progs	:= conmakehash kallsyms modpost mk_elfconfig pnmtologo bin2c
+host-progs	:= conmakehash kallsyms modpost mk_elfconfig pnmtologo \
+		   bin2c binoffset
 always		:= $(host-progs) empty.o
 
 modpost-objs	:= modpost.o file2alias.o sumversion.o

diff -Naurp ./Makefile~getconfig ./Makefile
@@ -675,6 +675,11 @@ include/config/MARKER: include/linux/aut
 	@scripts/basic/split-include include/linux/autoconf.h include/config
 	@touch $@
 
+.PHONY: getconfig
+getconfig: $(srctree)/scripts/binoffset
+	scripts/extract-ikconfig $@
+
+
 # Generate some files
 # ---------------------------------------------------------------------------
 

@@ -879,6 +884,7 @@ help:
 	@echo  ''
 	@echo  'Configuration targets:'
 	@$(MAKE) -f $(srctree)/scripts/kconfig/Makefile help
+	@echo  '  getconfig kernel_filename - extract kernel config to stdout'
 	@echo  ''
 	@echo  'Other generic targets:'
 	@echo  '  all		  - Build all targets marked with [*]'

On Wed, 23 Jun 2004 23:32:21 +0200 Sam Ravnborg wrote:

| On Wed, Jun 23, 2004 at 10:53:46AM -0700, Randy.Dunlap wrote:
| > 
| > We also need a decent way to run scripts/extract-ikconfig and build
| > scripts/binoffset.  I recall that SuSE has/had a 'make cloneconfig'
| > once upon a time (in 2.4.x).  I've begun on a 'make getconfig' (but
| > I can rename it to 'cloneconfig' if that's more appropriate), but
| > it needs some more work (i.e., not yet working).  [patch below]
| > 
| > 
| > Thanks,
| > --
| > ~Randy
| > 
| > 
| > 
| > (beginning, not working...)
| > add 'getconfig' target to extract kernel config from the kernel image file
| > 
| > Signed-off-by: Randy Dunlap <[email protected]>
| > 
| > diffstat:=
| >  Makefile         |    6 ++++++
| >  scripts/Makefile |    3 ++-
| >  2 files changed, 8 insertions(+), 1 deletion(-)
| > 
| > diff -Naurp ./scripts/Makefile~getconfig ./scripts/Makefile
| > --- ./scripts/Makefile~getconfig	2004-06-23 07:21:28.000000000 -0700
| > +++ ./scripts/Makefile	2004-06-23 09:16:06.266845368 -0700
| > @@ -5,7 +5,8 @@
| >  # docproc: 	 Preprocess .tmpl file in order to generate .sgml docs
| >  # conmakehash:	 Create arrays for initializing the kernel console tables
| >  
| > -host-progs	:= conmakehash kallsyms modpost mk_elfconfig pnmtologo bin2c
| > +host-progs	:= conmakehash kallsyms modpost mk_elfconfig pnmtologo \
| > +		   bin2c binoffset
| >  always		:= $(host-progs) empty.o
| >  
| >  modpost-objs	:= modpost.o file2alias.o sumversion.o
| > diff -Naurp ./Makefile~getconfig ./Makefile
| > --- ./Makefile~getconfig	2004-06-23 07:21:23.000000000 -0700
| > +++ ./Makefile	2004-06-23 09:22:46.571989736 -0700
| > @@ -675,6 +675,11 @@ include/config/MARKER: include/linux/aut
| >  	@scripts/basic/split-include include/linux/autoconf.h include/config
| >  	@touch $@
| >  
| > +.PHONY: getconfig
| > +getconfig: $(srctree)/scripts/binoffset
| > +	scripts/extract-ikconfig $@
| 
| 
| There is a conflict with what rule to select here since all *config targets are sent to
| scripts/kconfig.
| Since what is done here is handling the kernel config maybe everything should be
| moved there?

OK, I don't mind and don't think it will matter.
Can you help? :)

--
~Randy

Should we really compress it inside the kernel image? If you care for space you compress the kernel image anyway.

In this way it is compressed even when the kernel is uncompressed.

Sam