Skip to contentSkip to conversation

What about make mergeconfig ?

DISCUSSION Benjamin Herrenschmidt 8 participants 15 messages Oct 25, 2006 – Oct 27, 2006 Kconfig

lore
Share & tools
lore
#0 lore raw

Hi folks !

I'm not good enough at make and friends to do that myself without spending a lot more time than I have at hand, but I figured it might be something doable in a blink for whoever knows Kconfig guts :)

What about something like:

make mergeconfig <path_to_file>

That would merge all entries in the specified file with the current .config. By mergeing, that basically means that rule:

N + N = N
m + N = m
Y + N = Y
m + Y = Y

(that is, we basically take for each entry max(.config, merge file)

The idea here is that on archs like powerpc, we have the ability to build kernels that can boot several platforms. However, the defconfigs we ship (g5_defconfig, pseries_defconfig, maple_defconfig, cell... ) are tailored for one platform.

Now, if (for testing typically) I want to build a kernel that boots (and has all the necessary drivers) for both a g5 and a cell, I need to start from one of the defconfigs (the g5 one) and basically add in manually all the bits necessary from the other one (the cell one).

Thus it might be useful to have a mecanism to automate that...

Thanks in advance for whichever guru wants to have a look at this :)

Cheers,
Ben.

#1 lore raw
Quoted 10 lines · Benjamin Herrenschmidt

make mergeconfig <path_to_file>

That would merge all entries in the specified file with the current .config. By mergeing, that basically means that rule:

N + N = N
m + N = m
Y + N = Y
m + Y = Y

We have something vaguely similar in the Fedora package (I think it's only in the CVS tree rather than in the SRPM itself, but the CVS tree is public too).

http://cvs.fedora.redhat.com/viewcvs/*checkout*/rpms/kernel/devel/merge.pl?rev=1.9

It doesn't do quite what you asked for -- it works with 'incremental' configuration. So any options specified in _any_ form in the second config are overridden in the output. It lets us start with a generic config, then apply options (turning stuff both on and off) for PowerPC in general, and then for specific builds on top of that.

Signature · 1 line
dwmw2
#2 lore raw
In reply to Benjamin Herrenschmidt
Quoted 25 lines · Benjamin Herrenschmidt

Hi folks !

I'm not good enough at make and friends to do that myself without spending a lot more time than I have at hand, but I figured it might be something doable in a blink for whoever knows Kconfig guts :)

What about something like:

make mergeconfig <path_to_file>

That would merge all entries in the specified file with the current .config. By mergeing, that basically means that rule:

N + N = N
m + N = m
Y + N = Y
m + Y = Y

(that is, we basically take for each entry max(.config, merge file)

The idea here is that on archs like powerpc, we have the ability to build kernels that can boot several platforms. However, the defconfigs we ship (g5_defconfig, pseries_defconfig, maple_defconfig, cell... ) are tailored for one platform.

Quoted 7 lines

Now, if (for testing typically) I want to build a kernel that boots (and has all the necessary drivers) for both a g5 and a cell, I need to start from one of the defconfigs (the g5 one) and basically add in manually all the bits necessary from the other one (the cell one).

Thus it might be useful to have a mecanism to automate that...

Hmm, wouldn't you only have to build that config once and then in the future just use "make oldconfig" to keep it up-to-date with newer kernels ?

Signature · 3 lines
Jesper Juhl <[email protected]>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html
#4 lore raw
In reply to Benjamin Herrenschmidt

Benjamin Herrenschmidt <[email protected]> wrote:

| I'm not good enough at make and friends to do that myself without
| spending a lot more time than I have at hand, but I figured it might be
| something doable in a blink for whoever knows Kconfig guts :)
| 
| What about something like:
| 
| make mergeconfig <path_to_file>
| 
| That would merge all entries in the specified file with the
| current .config. By mergeing, that basically means that rule:
| 
| N + N = N
| m + N = m
| Y + N = Y
| m + Y = Y
| 
| (that is, we basically take for each entry max(.config, merge file)

Can't you do that with just a sort command?

sort .config other.config > new.config

Signature · 3 lines
Dick Streefland                      ////                      Altium BV
[email protected]           (@ @)          http://www.altium.com
--------------------------------oOO--(_)--OOo---------------------------
#5 lore raw
Quoted 4 lines

Can't you do that with just a sort command?

sort .config other.config > new.config

That does not work where .config and other.config have the same symbol listed, kconfig will bark and use the first value encountered. Because I do have exactly that problem with my patch series (changes some Ys to Ms), I am in need of the following patch to Kconfig TDTRT.

This is probably also what the OP is looking for, except that it does not require a special 'mergeconfig', but works with all 'oldconfig', 'menuconfig', xconfig, etc.

kconfig_override.diff

scripts/kconfig/confdata.c +5 -9
Index: linux-2.6.18_rc4/scripts/kconfig/confdata.c
===================================================================
@@ -170,8 +170,7 @@ load:
 					sym->type = S_BOOLEAN;
 			}
 			if (sym->flags & def_flags) {
-				conf_warning("trying to reassign symbol %s", sym->name);
-				break;
+				conf_warning("override: reassigning to symbol %s", sym->name);
 			}
 			switch (sym->type) {
 			case S_BOOLEAN:
@@ -207,8 +206,7 @@ load:
 					sym->type = S_OTHER;
 			}
 			if (sym->flags & def_flags) {
-				conf_warning("trying to reassign symbol %s", sym->name);
-				break;
+				conf_warning("override: reassigning to symbol %s", sym->name);
 			}
 			switch (sym->type) {
 			case S_TRISTATE:
@@ -284,11 +282,9 @@ load:
 				}
 				break;
 			case yes:
-				if (cs->def[def].tri != no) {
-					conf_warning("%s creates inconsistent choice state", sym->name);
-					cs->flags &= ~def_flags;
-				} else
-					cs->def[def].val = sym;
+				if(cs->def[def].tri != no)
+					conf_warning("override: %s turns state choice", sym->name);
+				cs->def[def].val = sym;
 				break;
 			}
 			cs->def[def].tri = E_OR(cs->def[def].tri, sym->def[def].tri);

#<EOF>

	-`J'
#6 lore raw

Jan Engelhardt <[email protected]> wrote:

| >Can't you do that with just a sort command?
| >
| >  sort .config other.config > new.config
| 
| That does not work where .config and other.config have the same symbol 
| listed, kconfig will bark and use the first value encountered. Because I 
| do have exactly that problem with my patch series (changes some Ys to 
| Ms), I am in need of the following patch to Kconfig TDTRT.

Or you can use the following hack:

(sort .config other.config; echo set) | sh | grep ^CONFIG_ > new.config

Signature · 3 lines
Dick Streefland                      ////                      Altium BV
[email protected]           (@ @)          http://www.altium.com
--------------------------------oOO--(_)--OOo---------------------------
#3 lore raw
In reply to Jesper Juhl
Quoted 4 lines

Hmm, wouldn't you only have to build that config once and then in the future just use "make oldconfig" to keep it up-to-date with newer kernels ?

Well, it's handy to be able to pick up a fresh new upstream tree and build a config for pmac+pseries for example.

Ben.

#13 lore raw
In reply to Benjamin Herrenschmidt
Quoted 4 lines · Benjamin Herrenschmidt

The idea here is that on archs like powerpc, we have the ability to build kernels that can boot several platforms. However, the defconfigs we ship (g5_defconfig, pseries_defconfig, maple_defconfig, cell... ) are tailored for one platform.

Same issue on IA64. Would be good to have.

#7 lore raw
In reply to Dick Streefland
Quoted 10 lines
| >Can't you do that with just a sort command?
| >
| >  sort .config other.config > new.config
| 
| That does not work where .config and other.config have the same symbol 
| listed, kconfig will bark and use the first value encountered. Because I 
| do have exactly that problem with my patch series (changes some Ys to 
| Ms), I am in need of the following patch to Kconfig TDTRT.
Quoted 3 lines

Or you can use the following hack:

(sort .config other.config; echo set) | sh | grep ^CONFIG_ > new.config

That does not properly deal with "# CONFIG_XYZ is not set" lines in other.config.

	-`J'
#8 lore raw
Quoted 9 lines · Jan Engelhardt
Quoted 9 lines
| >Can't you do that with just a sort command?
| >
| >  sort .config other.config > new.config
| 
| That does not work where .config and other.config have the same symbol 
| listed, kconfig will bark and use the first value encountered. Because I 
| do have exactly that problem with my patch series (changes some Ys to 
| Ms), I am in need of the following patch to Kconfig TDTRT.
Quoted 6 lines
Quoted 3 lines

Or you can use the following hack:

(sort .config other.config; echo set) | sh | grep ^CONFIG_ > new.config

That does not properly deal with "# CONFIG_XYZ is not set" lines in other.config.

What about RCS merge?

merge -p other.config .config.old .config > other.config.new

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#14 lore raw
In reply to Benjamin Herrenschmidt
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Benjamin" == Benjamin Herrenschmidt <[email protected]> writes:

Benjamin> That would merge all entries in the specified file with the Benjamin> current .config.

You don't need it to be a Makefile target, it can be an external script. Would this one (untested!) do what you want?

Sam

Signature · 62 lines
Samuel Tardieu -- [email protected] -- http://www.rfc1149.net/

#! /usr/bin/python
#
# (c) 2006 Samuel Tardieu <[email protected]>
#
# This software may be used and distributed according to the terms
# of the GNU General Public License, incorporated herein by reference.
#
# Usage: mergeconfig config1 config2 ... > newconfig
#
# To get the formatting back, it is advised to run "make oldconfig" on the
# result.
#
# Be careful in not using the same file as input and output

import sre, sys

_not_set = sre.compile('# (CONFIG_\S+) is not set')
_set = sre.compile('(CONFIG_[^=]+)=(.*)')

def read_config(fn):
    """Read a kernel configuration file and return a dictionary with
    all the options present in the file. If an option is commented out,
    set its value as None."""
    d = {}
    for l in file(fn):
        l = l.rstrip('\r\n')
        x = _not_set.match(l)
        if x: d[x.group(1)] = None
        x = _set.match(l)
        if x: d[x.group(1)] = x.group(2)
    return d

def merge_option(o, v1, v2):
    """Merge option value v1 and v2."""
    if 'y' in [v1, v2]: return 'y'
    if 'm' in [v1, v2]: return 'm'
    if v1 != v2:
        sys.stderr.write('Option %s has two incompatible values: %s and %s' %
                         (o, v1, v2))
        sys.exit(1)
    return v1

def merge_config_into(a, b):
    """Merge configuration dictionary a into configuration dictionary b.
    In addition, this function returns b after the merge."""
    for k, v in a.items():
        try:
            b[v] = merge_option(k, v, b[v])
        except KeyError:
            b[k] = v
    return b

def output_config(d):
    for k, v in d.items():
        if v is None: print '# %s is not set' % k
        else: print '%s=%s' % (k, v)

if __name__ == '__main__':
    output_config (reduce(lambda x, y: merge_config_into(x, read_config(y)),
                          sys.argv[1:], {}))
#9 lore raw
In reply to Geert Uytterhoeven
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Or you can use the following hack:

(sort .config other.config; echo set) | sh | grep ^CONFIG_ > new.config

That does not properly deal with "# CONFIG_XYZ is not set" lines in other.config.

What about RCS merge?

I take it we do not want to depend on too many tools (remember the kconfig implementation language debate).

Quoted 1 line

merge -p other.config .config.old .config > other.config.new

This also does not seem conflict-safe.

	-`J'
#10 lore raw
Quoted 11 lines · Jan Engelhardt
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Or you can use the following hack:

(sort .config other.config; echo set) | sh | grep ^CONFIG_ > new.config

That does not properly deal with "# CONFIG_XYZ is not set" lines in other.config.

What about RCS merge?

I take it we do not want to depend on too many tools (remember the kconfig implementation language debate).

If you have CVS installed, you have RCS merge.

Quoted 3 lines
Quoted 1 line

merge -p other.config .config.old .config > other.config.new

This also does not seem conflict-safe.

Indeed, you can still have conflicts, which you have to resolve manually.

But it depends on what you want to achieve: do you want to set each config option in the destination config to max(config1.option, config2.option), or do you want to apply the recent changes for one config (which may include disabling options) to another config?

For the latter, merge should work fine.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#11 lore raw
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

What about RCS merge?

I take it we do not want to depend on too many tools (remember the kconfig implementation language debate).

If you have CVS installed, you have RCS merge.

11:54 ichi:~ > rpm -q cvs rcs
cvs-1.12.12-19
package rcs is not installed

11:54 ichi:~ > gzip -cd /ARCHIVES.gz | grep "/merge$" ./CD1/suse/i586/rcs-5.7-879.i586.rpm:

    -rwxr-xr-x    1 root    root 45252 May  2 09:42 /usr/bin/merge

CVS does not need RCS.

Quoted 10 lines
Quoted 3 lines
Quoted 1 line

merge -p other.config .config.old .config > other.config.new

This also does not seem conflict-safe.

Indeed, you can still have conflicts, which you have to resolve manually.

But it depends on what you want to achieve: do you want to set each config option in the destination config to max(config1.option, config2.option), or do you want to apply the recent changes for one config (which may include disabling options) to another config?

In my case, the latter.

Quoted 1 line

For the latter, merge should work fine.

Is merge a lot different from what patch is doing?

	-`J'
#12 lore raw
Quoted 11 lines · Jan Engelhardt
Quoted 6 lines
Quoted 4 lines
Quoted 1 line

What about RCS merge?

I take it we do not want to depend on too many tools (remember the kconfig implementation language debate).

If you have CVS installed, you have RCS merge.

11:54 ichi:~ > rpm -q cvs rcs
cvs-1.12.12-19
package rcs is not installed

Quoted 5 lines

11:54 ichi:~ > gzip -cd /ARCHIVES.gz | grep "/merge$" ./CD1/suse/i586/rcs-5.7-879.i586.rpm:

    -rwxr-xr-x    1 root    root 45252 May  2 09:42 /usr/bin/merge

CVS does not need RCS.

OK, so CVS assimilated RCS merge internally.

Quoted 11 lines
Quoted 10 lines
Quoted 3 lines
Quoted 1 line

merge -p other.config .config.old .config > other.config.new

This also does not seem conflict-safe.

Indeed, you can still have conflicts, which you have to resolve manually.

But it depends on what you want to achieve: do you want to set each config option in the destination config to max(config1.option, config2.option), or do you want to apply the recent changes for one config (which may include disabling options) to another config?

Quoted 5 lines

In my case, the latter.

Quoted 1 line

For the latter, merge should work fine.

Is merge a lot different from what patch is doing?

Yes, it does 3-way merges (merge my' version with your' version using a common ancestor). I.e. it's better in resolving conflicts than patch.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds