Skip to contentSkip to conversation

streamline_config.pl: add LMC_KEEP to preserve some kconfigs

PATCH Superseded Changbin Du 3 participants 6 messages May 3, 2020 – May 9, 2020

Subsystems (3)

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 streamline_config.pl: add LMC_KEEP to preserve some kconfigs

Sometimes it is useful to preserve batches of configs when making localmodconfig. For example, I usually don't want any usb and fs modules to be disabled. Now we can do it by:

 $ make LMC_KEEP="drivers/usb;fs" localmodconfig

---
v4: fix typo.
v3: rename LOCALMODCONFIG_PRESERVE to shorter LMC_KEEP.
v2: fix typo in documentation. (Randy Dunlap)

---
 Documentation/admin-guide/README.rst |  8 +++++++-
 scripts/kconfig/Makefile             |  1 +
 scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
 3 files changed, 31 insertions(+), 1 deletion(-)
Documentation/admin-guide/README.rst +7 -1
index cc6151fc0845..1371deab8bc7 100644
@@ -209,10 +209,16 @@ Configuring the kernel
                            store the lsmod of that machine into a file
                            and pass it in as a LSMOD parameter.
 
+                           Also, you can preserve modules in certain folders
+                           or kconfig files by specifying their paths in
+                           parameter LMC_KEEP.
+
                    target$ lsmod > /tmp/mylsmod
                    target$ scp /tmp/mylsmod host:/tmp
 
-                   host$ make LSMOD=/tmp/mylsmod localmodconfig
+                   host$ make LSMOD=/tmp/mylsmod \
+                           LMC_KEEP="drivers/usb;drivers/gpu;fs" \
+                           localmodconfig
 
                            The above also works when cross compiling.
 
scripts/kconfig/Makefile +1 -0
index c9d0a4a8efb3..e0abbf5805f5 100644
@@ -123,6 +123,7 @@ help:
 	@echo  '  gconfig	  - Update current config utilising a GTK+ based front-end'
 	@echo  '  oldconfig	  - Update current config utilising a provided .config as base'
 	@echo  '  localmodconfig  - Update current config disabling modules not loaded'
+	@echo  '                    except those preserved by LMC_KEEP environment variable'
 	@echo  '  localyesconfig  - Update current config converting local mods to core'
 	@echo  '  defconfig	  - New config with default from ARCH supplied defconfig'
 	@echo  '  savedefconfig   - Save current config as ./defconfig (minimal config)'
scripts/kconfig/streamline_config.pl +23 -0
index e2f8504f5a2d..d26543a807c9 100755
@@ -143,6 +143,7 @@ my %depends;
 my %selects;
 my %prompts;
 my %objects;
+my %config2kfile;
 my $var;
 my $iflevel = 0;
 my @ifdeps;
@@ -201,6 +202,7 @@ sub read_kconfig {
 	if (/^\s*(menu)?config\s+(\S+)\s*$/) {
 	    $state = "NEW";
 	    $config = $2;
+	    $config2kfile{"CONFIG_$config"} = $kconfig;
 
 	    # Add depends for 'if' nesting
 	    for (my $i = 0; $i < $iflevel; $i++) {
@@ -592,6 +594,22 @@ while ($repeat) {
 
 my %setconfigs;
 
+my @preserved_kconfigs;
+@preserved_kconfigs = split(/;/,$ENV{LMC_KEEP}) if (defined($ENV{LMC_KEEP}));
+
+sub in_preserved_kconfigs {
+    my $kconfig = $config2kfile{$_[0]};
+    if (!defined($kconfig)) {
+        return 0;
+    }
+    foreach my $excl (@preserved_kconfigs) {
+        if($kconfig =~ /^$excl/) {
+            return 1;
+        }
+    }
+    return 0;
+}
+
 # Finally, read the .config file and turn off any module enabled that
 # we could not find a reason to keep enabled.
 foreach my $line (@config_file) {
@@ -644,6 +662,11 @@ foreach my $line (@config_file) {
     }
 
     if (/^(CONFIG.*)=(m|y)/) {
+        if (in_preserved_kconfigs($1)) {
+            dprint "Preserve config $1";
+            print;
+            next;
+        }
 	if (defined($configs{$1})) {
 	    if ($localyesconfig) {
 	        $setconfigs{$1} = 'y';
Signature · 1 line
2.25.1
#1 lore raw
Quoted 7 lines · Changbin Du

Sometimes it is useful to preserve batches of configs when making localmodconfig. For example, I usually don't want any usb and fs modules to be disabled. Now we can do it by:

 $ make LMC_KEEP="drivers/usb;fs" localmodconfig
Quoted 7 lines

---
v4: fix typo.
v3: rename LOCALMODCONFIG_PRESERVE to shorter LMC_KEEP.
v2: fix typo in documentation. (Randy Dunlap)
---

Personally, I do not mind the long LOCALMODCONFIG_PRESERVE, but this tends to be bike-sheding.
I do not have a strong opinion.

Quoted 13 lines
 Documentation/admin-guide/README.rst |  8 +++++++-
 scripts/kconfig/Makefile             |  1 +
 scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
 3 files changed, 31 insertions(+), 1 deletion(-)
Documentation/admin-guide/README.rst +0 -0
index cc6151fc0845..1371deab8bc7 100644
@@ -209,10 +209,16 @@ Configuring the kernel
                            store the lsmod of that machine into a file
                            and pass it in as a LSMOD parameter.
 

Documentation/admin-guide/README.rst Configuring the kernel · line 220

Quoted 10 lines
Documentation/admin-guide/README.rst +6 -1
+                           Also, you can preserve modules in certain folders
+                           or kconfig files by specifying their paths in
+                           parameter LMC_KEEP.
+
                    target$ lsmod > /tmp/mylsmod
                    target$ scp /tmp/mylsmod host:/tmp
 
-                   host$ make LSMOD=/tmp/mylsmod localmodconfig
+                   host$ make LSMOD=/tmp/mylsmod \
+                           LMC_KEEP="drivers/usb;drivers/gpu;fs" \

This might be another bike-sheding item, but
can you use a space for the delimiter?

LMC_KEEP="drivers/usb drivers/gpu fs"

If you pass multiple directories,
you will need to surround them with double-quotes.

Quoted 37 lines
Documentation/admin-guide/README.rst +1 -0
+                           localmodconfig
 
                            The above also works when cross compiling.
 
scripts/kconfig/Makefile +1 -0
index c9d0a4a8efb3..e0abbf5805f5 100644
@@ -123,6 +123,7 @@ help:
        @echo  '  gconfig         - Update current config utilising a GTK+ based front-end'
        @echo  '  oldconfig       - Update current config utilising a provided .config as base'
        @echo  '  localmodconfig  - Update current config disabling modules not loaded'
+       @echo  '                    except those preserved by LMC_KEEP environment variable'
        @echo  '  localyesconfig  - Update current config converting local mods to core'
        @echo  '  defconfig       - New config with default from ARCH supplied defconfig'
        @echo  '  savedefconfig   - Save current config as ./defconfig (minimal config)'
scripts/kconfig/streamline_config.pl +2 -0
index e2f8504f5a2d..d26543a807c9 100755
@@ -143,6 +143,7 @@ my %depends;
 my %selects;
 my %prompts;
 my %objects;
+my %config2kfile;
 my $var;
 my $iflevel = 0;
 my @ifdeps;
@@ -201,6 +202,7 @@ sub read_kconfig {
        if (/^\s*(menu)?config\s+(\S+)\s*$/) {
            $state = "NEW";
            $config = $2;
+           $config2kfile{"CONFIG_$config"} = $kconfig;
 
            # Add depends for 'if' nesting
            for (my $i = 0; $i < $iflevel; $i++) {

scripts/kconfig/streamline_config.pl while ($repeat) { · line 598

Quoted 5 lines
scripts/kconfig/streamline_config.pl +2 -0
@@ -592,6 +594,22 @@ while ($repeat) {
 
 my %setconfigs;
 
+my @preserved_kconfigs;
+@preserved_kconfigs = split(/;/,$ENV{LMC_KEEP}) if (defined($ENV{LMC_KEEP}));

Maybe, you can do 'my' declaration and the assignment in a single line?

Can you drop the 'if ...' conditional?

Does this work for you?

my @preserved_kconfigs = split(/;/,$ENV{LMC_KEEP});

Quoted 32 lines
scripts/kconfig/streamline_config.pl +19 -1
+
+sub in_preserved_kconfigs {
+    my $kconfig = $config2kfile{$_[0]};
+    if (!defined($kconfig)) {
+        return 0;
+    }
+    foreach my $excl (@preserved_kconfigs) {
+        if($kconfig =~ /^$excl/) {
+            return 1;
+        }
+    }
+    return 0;
+}
+
 # Finally, read the .config file and turn off any module enabled that
 # we could not find a reason to keep enabled.
 foreach my $line (@config_file) {
@@ -644,6 +662,11 @@ foreach my $line (@config_file) {
     }
 
     if (/^(CONFIG.*)=(m|y)/) {
+        if (in_preserved_kconfigs($1)) {
+            dprint "Preserve config $1";
+            print;
+            next;
+        }
        if (defined($configs{$1})) {
            if ($localyesconfig) {
                $setconfigs{$1} = 'y';
--

2.25.1

Signature · 2 lines
Best Regards
Masahiro Yamada
#2 lore raw
Quoted 10 lines · On Thu, 7 May 2020 12:19:57 +0900 Masahiro Yamada <[email protected]> wrote:
Quoted 9 lines · Changbin Du

Sometimes it is useful to preserve batches of configs when making localmodconfig. For example, I usually don't want any usb and fs modules to be disabled. Now we can do it by:

 $ make LMC_KEEP="drivers/usb;fs" localmodconfig
Quoted 8 lines
Quoted 5 lines

---
v4: fix typo.
v3: rename LOCALMODCONFIG_PRESERVE to shorter LMC_KEEP.
v2: fix typo in documentation. (Randy Dunlap)
---

Personally, I do not mind the long LOCALMODCONFIG_PRESERVE,

Perhasp we allow both? ;-)

And just have one set to the other.

Quoted 33 lines

but this tends to be bike-sheding.
I do not have a strong opinion.

Quoted 13 lines
 Documentation/admin-guide/README.rst |  8 +++++++-
 scripts/kconfig/Makefile             |  1 +
 scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
 3 files changed, 31 insertions(+), 1 deletion(-)
Documentation/admin-guide/README.rst +0 -0
index cc6151fc0845..1371deab8bc7 100644
@@ -209,10 +209,16 @@ Configuring the kernel
                            store the lsmod of that machine into a file
                            and pass it in as a LSMOD parameter.
 

Documentation/admin-guide/README.rst Configuring the kernel · line 220

Quoted 10 lines
Documentation/admin-guide/README.rst +6 -1
+                           Also, you can preserve modules in certain folders
+                           or kconfig files by specifying their paths in
+                           parameter LMC_KEEP.
+
                    target$ lsmod > /tmp/mylsmod
                    target$ scp /tmp/mylsmod host:/tmp
 
-                   host$ make LSMOD=/tmp/mylsmod localmodconfig
+                   host$ make LSMOD=/tmp/mylsmod \
+                           LMC_KEEP="drivers/usb;drivers/gpu;fs" \  

This might be another bike-sheding item, but
can you use a space for the delimiter?

Quoted 4 lines

LMC_KEEP="drivers/usb drivers/gpu fs"

If you pass multiple directories,
you will need to surround them with double-quotes.

I agree that spaces look better.

Quoted 50 lines
Quoted 41 lines
Documentation/admin-guide/README.rst +1 -0
+                           localmodconfig
 
                            The above also works when cross compiling.
 
scripts/kconfig/Makefile +1 -0
index c9d0a4a8efb3..e0abbf5805f5 100644
@@ -123,6 +123,7 @@ help:
        @echo  '  gconfig         - Update current config utilising a GTK+ based front-end'
        @echo  '  oldconfig       - Update current config utilising a provided .config as base'
        @echo  '  localmodconfig  - Update current config disabling modules not loaded'
+       @echo  '                    except those preserved by LMC_KEEP environment variable'
        @echo  '  localyesconfig  - Update current config converting local mods to core'
        @echo  '  defconfig       - New config with default from ARCH supplied defconfig'
        @echo  '  savedefconfig   - Save current config as ./defconfig (minimal config)'
scripts/kconfig/streamline_config.pl +2 -0
index e2f8504f5a2d..d26543a807c9 100755
@@ -143,6 +143,7 @@ my %depends;
 my %selects;
 my %prompts;
 my %objects;
+my %config2kfile;
 my $var;
 my $iflevel = 0;
 my @ifdeps;
@@ -201,6 +202,7 @@ sub read_kconfig {
        if (/^\s*(menu)?config\s+(\S+)\s*$/) {
            $state = "NEW";
            $config = $2;
+           $config2kfile{"CONFIG_$config"} = $kconfig;
 
            # Add depends for 'if' nesting
            for (my $i = 0; $i < $iflevel; $i++) {

scripts/kconfig/streamline_config.pl while ($repeat) { · line 598

Quoted 5 lines
scripts/kconfig/streamline_config.pl +2 -0
@@ -592,6 +594,22 @@ while ($repeat) {
 
 my %setconfigs;
 
+my @preserved_kconfigs;
+@preserved_kconfigs = split(/;/,$ENV{LMC_KEEP}) if (defined($ENV{LMC_KEEP}));  

Maybe, you can do 'my' declaration and the assignment in a single line?

Quoted 8 lines

Can you drop the 'if ...' conditional?

Does this work for you?

my @preserved_kconfigs = split(/;/,$ENV{LMC_KEEP});

Will an undefined warning happen if LMC_KEEP isn't defined?

-- Steve

#3 lore raw
Quoted 59 lines · Steven Rostedt
Quoted 10 lines · On Thu, 7 May 2020 12:19:57 +0900 Masahiro Yamada <[email protected]> wrote:
Quoted 9 lines · Changbin Du

Sometimes it is useful to preserve batches of configs when making localmodconfig. For example, I usually don't want any usb and fs modules to be disabled. Now we can do it by:

 $ make LMC_KEEP="drivers/usb;fs" localmodconfig
Quoted 8 lines
Quoted 5 lines

---
v4: fix typo.
v3: rename LOCALMODCONFIG_PRESERVE to shorter LMC_KEEP.
v2: fix typo in documentation. (Randy Dunlap)
---

Personally, I do not mind the long LOCALMODCONFIG_PRESERVE,

Perhasp we allow both? ;-)

And just have one set to the other.

Quoted 33 lines

but this tends to be bike-sheding.
I do not have a strong opinion.

Quoted 13 lines
 Documentation/admin-guide/README.rst |  8 +++++++-
 scripts/kconfig/Makefile             |  1 +
 scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
 3 files changed, 31 insertions(+), 1 deletion(-)
Documentation/admin-guide/README.rst +0 -0
index cc6151fc0845..1371deab8bc7 100644
@@ -209,10 +209,16 @@ Configuring the kernel
                            store the lsmod of that machine into a file
                            and pass it in as a LSMOD parameter.
 

Documentation/admin-guide/README.rst Configuring the kernel · line 220

Quoted 10 lines
Documentation/admin-guide/README.rst +6 -1
+                           Also, you can preserve modules in certain folders
+                           or kconfig files by specifying their paths in
+                           parameter LMC_KEEP.
+
                    target$ lsmod > /tmp/mylsmod
                    target$ scp /tmp/mylsmod host:/tmp
 
-                   host$ make LSMOD=/tmp/mylsmod localmodconfig
+                   host$ make LSMOD=/tmp/mylsmod \
+                           LMC_KEEP="drivers/usb;drivers/gpu;fs" \  

This might be another bike-sheding item, but
can you use a space for the delimiter?

Quoted 7 lines
Quoted 4 lines

LMC_KEEP="drivers/usb drivers/gpu fs"

If you pass multiple directories,
you will need to surround them with double-quotes.

I agree that spaces look better.

I am okay with space. But what about ":" which is used by shell PATH and many others?

Quoted 53 lines
Quoted 53 lines
Quoted 41 lines
Documentation/admin-guide/README.rst +1 -0
+                           localmodconfig
 
                            The above also works when cross compiling.
 
scripts/kconfig/Makefile +1 -0
index c9d0a4a8efb3..e0abbf5805f5 100644
@@ -123,6 +123,7 @@ help:
        @echo  '  gconfig         - Update current config utilising a GTK+ based front-end'
        @echo  '  oldconfig       - Update current config utilising a provided .config as base'
        @echo  '  localmodconfig  - Update current config disabling modules not loaded'
+       @echo  '                    except those preserved by LMC_KEEP environment variable'
        @echo  '  localyesconfig  - Update current config converting local mods to core'
        @echo  '  defconfig       - New config with default from ARCH supplied defconfig'
        @echo  '  savedefconfig   - Save current config as ./defconfig (minimal config)'
scripts/kconfig/streamline_config.pl +2 -0
index e2f8504f5a2d..d26543a807c9 100755
@@ -143,6 +143,7 @@ my %depends;
 my %selects;
 my %prompts;
 my %objects;
+my %config2kfile;
 my $var;
 my $iflevel = 0;
 my @ifdeps;
@@ -201,6 +202,7 @@ sub read_kconfig {
        if (/^\s*(menu)?config\s+(\S+)\s*$/) {
            $state = "NEW";
            $config = $2;
+           $config2kfile{"CONFIG_$config"} = $kconfig;
 
            # Add depends for 'if' nesting
            for (my $i = 0; $i < $iflevel; $i++) {

scripts/kconfig/streamline_config.pl while ($repeat) { · line 598

Quoted 5 lines
scripts/kconfig/streamline_config.pl +2 -0
@@ -592,6 +594,22 @@ while ($repeat) {
 
 my %setconfigs;
 
+my @preserved_kconfigs;
+@preserved_kconfigs = split(/;/,$ENV{LMC_KEEP}) if (defined($ENV{LMC_KEEP}));  

Maybe, you can do 'my' declaration and the assignment in a single line?

Can you drop the 'if ...' conditional?

Quoted 8 lines
Quoted 5 lines

Does this work for you?

my @preserved_kconfigs = split(/;/,$ENV{LMC_KEEP});

Will an undefined warning happen if LMC_KEEP isn't defined?

I just verifed there is no warning raised.

Quoted 1 line

-- Steve

Signature · 2 lines
Cheers,
Changbin Du
#4 lore raw
Quoted 4 lines · On Fri, 8 May 2020 22:00:47 +0800 Changbin Du <[email protected]> wrote:
Quoted 2 lines

I agree that spaces look better.

I am okay with space. But what about ":" which is used by shell PATH and many others?

I'm OK with a ':'

-- Steve

#5 lore raw
In reply to Changbin Du
Quoted 63 lines · Changbin Du
Quoted 61 lines · Steven Rostedt
Quoted 10 lines · On Thu, 7 May 2020 12:19:57 +0900 Masahiro Yamada <[email protected]> wrote:
Quoted 9 lines · Changbin Du

Sometimes it is useful to preserve batches of configs when making localmodconfig. For example, I usually don't want any usb and fs modules to be disabled. Now we can do it by:

 $ make LMC_KEEP="drivers/usb;fs" localmodconfig
Quoted 8 lines
Quoted 5 lines

---
v4: fix typo.
v3: rename LOCALMODCONFIG_PRESERVE to shorter LMC_KEEP.
v2: fix typo in documentation. (Randy Dunlap)
---

Personally, I do not mind the long LOCALMODCONFIG_PRESERVE,

Perhasp we allow both? ;-)

And just have one set to the other.

Quoted 35 lines

but this tends to be bike-sheding.
I do not have a strong opinion.

Quoted 13 lines
 Documentation/admin-guide/README.rst |  8 +++++++-
 scripts/kconfig/Makefile             |  1 +
 scripts/kconfig/streamline_config.pl | 23 +++++++++++++++++++++++
 3 files changed, 31 insertions(+), 1 deletion(-)
Documentation/admin-guide/README.rst +0 -0
index cc6151fc0845..1371deab8bc7 100644
@@ -209,10 +209,16 @@ Configuring the kernel
                            store the lsmod of that machine into a file
                            and pass it in as a LSMOD parameter.
 

Documentation/admin-guide/README.rst Configuring the kernel · line 220

Quoted 10 lines
Documentation/admin-guide/README.rst +6 -1
+                           Also, you can preserve modules in certain folders
+                           or kconfig files by specifying their paths in
+                           parameter LMC_KEEP.
+
                    target$ lsmod > /tmp/mylsmod
                    target$ scp /tmp/mylsmod host:/tmp
 
-                   host$ make LSMOD=/tmp/mylsmod localmodconfig
+                   host$ make LSMOD=/tmp/mylsmod \
+                           LMC_KEEP="drivers/usb;drivers/gpu;fs" \

This might be another bike-sheding item, but can you use a space for the delimiter?

LMC_KEEP="drivers/usb drivers/gpu fs"

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

If you pass multiple directories,
you will need to surround them with double-quotes.

I agree that spaces look better.

I am okay with space. But what about ":" which is used by shell PATH and many others?

I am not sure 'many others',
but you can pick what you like.

Signature · 2 lines
Best Regards
Masahiro Yamada