Skip to contentSkip to conversation

streamline_config.pl: fix: process configs set to "y"

PATCH In review David Hunter 3 participants 30 messages Oct 14, 2024 – Nov 30, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v3 cover streamline_config.pl: fix: process configs set to "y"

An assumption made in this script is that the config options do not need to be processed because they will simply be in the new config file. This assumption is incorrect.

Process the config entries set to "y" because those config entries might have dependencies set to "m". If a config entry is set to "m" and is not loaded directly into the machine, the script will currently turn off that config entry; however, if that turned off config entry is a dependency for a "y" option. that means the config entry set to "y" will also be turned off later when the conf executive file is called.

Here is a model of the problem (arrows show dependency):

Original config file
Config_1 (m) <-- Config_2 (y)

Config_1 is not loaded in this example, so it is turned off. After scripts/kconfig/streamline_config.pl, but before scripts/kconfig/conf Config_1 (n) <-- Config_2 (y)

After scripts/kconfig/conf
Config_1 (n) <-- Config_2 (n)

It should also be noted that any module in the dependency chain will also be turned off, even if that module is loaded directly onto the computer. Here is an example:

Original config file
Config_1 (m) <-- Config_2 (y) <-- Config_3 (m)

Config_3 will be loaded in this example.
After scripts/kconfig/streamline_config.pl, but before scripts/kconfig/conf Config_1 (n) <-- Config_2 (y) <-- Config_3 (m)

After scripts/kconfig/conf
Config_1 (n) <-- Config_2 (n) <-- Config_3 (n)

I discovered this problem when I ran "make localmodconfig" on a generic Ubuntu config file. Many hardware devices were not recognized once the kernel was installed and booted. Another way to reproduced the error I had is to run "make localmodconfig" twice. The standard error might display warnings that certain modules should be selected but no config files are turned on that select that module.

With the changes in this series patch, all modules are loaded properly and all of the hardware is loaded when the kernel is installed and booted.

David Hunter (7):
streamline_config.pl: fix missing variable operator in debug print streamline_config.pl: ensure all defaults are tracked streamline_config.pl: remove prompt warnings for configs with defaults streamline_config.pl: include tool to learn about a config option streamline_config.pl: fix: implement choice for kconfigs streamline_config.pl: process config options set to "y" streamline_config.pl: check prompt for bool

---
V1: https://lore.kernel.org/all/[email protected]/

V2: https://lore.kernel.org/all/[email protected]/
- Put in subject.

V3:
- changed the order of patches
- removed a patch that was unneccessary
- added a patch for a debugging tool
---

 scripts/kconfig/streamline_config.pl | 130 ++++++++++++++++++++++++---
 1 file changed, 119 insertions(+), 11 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH v2 1/7 streamline_config.pl: fix missing variable operator in debug print

Put in the dollar sign for the variable '$config'. That way, the debug message has more meaning.

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- changed the subject

---
 scripts/kconfig/streamline_config.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/kconfig/streamline_config.pl +1 -1
index d51cd7ac15d2..a85d6a3108a1 100755
@@ -503,7 +503,7 @@ sub parse_config_selects
 
 	    # Check if something other than a module selects this config
 	    if (defined($orig_configs{$conf}) && $orig_configs{$conf} ne "m") {
-		dprint "$conf (non module) selects config, we are good\n";
+		dprint "$conf (non module) selects $config, we are good\n";
 		# we are good with this
 		return;
 	    }
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH v2 1/7] streamline_config.pl: fix missing variable operator in debug print

Quoted 10 lines · David Hunter

Put in the dollar sign for the variable '$config'. That way, the debug message has more meaning.

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- changed the subject

Applied to linux-kbuild. Thanks!

Quoted 18 lines
---
 scripts/kconfig/streamline_config.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/kconfig/streamline_config.pl +1 -2
index d51cd7ac15d2..a85d6a3108a1 100755
@@ -503,7 +503,7 @@ sub parse_config_selects
 
            # Check if something other than a module selects this config
            if (defined($orig_configs{$conf}) && $orig_configs{$conf} ne "m") {
-               dprint "$conf (non module) selects config, we are good\n";
+               dprint "$conf (non module) selects $config, we are good\n";
                # we are good with this
                return;
            }
--
Quoted 2 lines

2.43.0

--
Best Regards
Masahiro Yamada

#3 lore raw

PATCH v2 2/7 streamline_config.pl: ensure all defaults are tracked

In reply to David Hunter

Track default options on the second line. On the second line of some config entries, default and depndency options sometimes appear. In those instances, the state will be "NEW" and not "DEP".

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- changed the subject
- changed the condition to be more in line with the script style

---
 scripts/kconfig/streamline_config.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/kconfig/streamline_config.pl +1 -1
index a85d6a3108a1..85f4712e2bf3 100755
@@ -220,7 +220,7 @@ sub read_kconfig {
 	    $depends{$config} = $1;
 	} elsif ($state eq "DEP" && /^\s*depends\s+on\s+(.*)$/) {
 	    $depends{$config} .= " " . $1;
-	} elsif ($state eq "DEP" && /^\s*def(_(bool|tristate)|ault)\s+(\S.*)$/) {
+	} elsif ($state ne "NONE" && /^\s*def(_(bool|tristate)|ault)\s+(\S.*)$/) {
 	    my $dep = $3;
 	    if ($dep !~ /^\s*(y|m|n)\s*$/) {
 		$dep =~ s/.*\sif\s+//;
Signature · 1 line
2.43.0
#4 lore raw

Re: [PATCH v2 2/7] streamline_config.pl: ensure all defaults are tracked

On Mon, 14 Oct 2024 10:13:32 -0400
David Hunter <[email protected]> wrote:

Hi David,

Thanks for sending these.

Quoted 2 lines

Track default options on the second line. On the second line of some config entries, default and depndency options sometimes appear. In those

"dependency"

Quoted 1 line

instances, the state will be "NEW" and not "DEP".

Can you list some examples of where this happens?

Thanks,

-- Steve

Quoted 25 lines

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- changed the subject
- changed the condition to be more in line with the script style

---
 scripts/kconfig/streamline_config.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/kconfig/streamline_config.pl +1 -1
index a85d6a3108a1..85f4712e2bf3 100755
@@ -220,7 +220,7 @@ sub read_kconfig {
 	    $depends{$config} = $1;
 	} elsif ($state eq "DEP" && /^\s*depends\s+on\s+(.*)$/) {
 	    $depends{$config} .= " " . $1;
-	} elsif ($state eq "DEP" && /^\s*def(_(bool|tristate)|ault)\s+(\S.*)$/) {
+	} elsif ($state ne "NONE" && /^\s*def(_(bool|tristate)|ault)\s+(\S.*)$/) {
 	    my $dep = $3;
 	    if ($dep !~ /^\s*(y|m|n)\s*$/) {
 		$dep =~ s/.*\sif\s+//;
#5 lore raw

Re: [PATCH v2 2/7] streamline_config.pl: ensure all defaults are tracked

Quoted 8 lines · Steven Rostedt

On Mon, 14 Oct 2024 10:13:32 -0400
David Hunter <[email protected]> wrote:

Hi David,

Thanks for sending these.

Quoted 7 lines
Quoted 2 lines

Track default options on the second line. On the second line of some config entries, default and depndency options sometimes appear. In those

"dependency"

Quoted 1 line

instances, the state will be "NEW" and not "DEP".

Can you list some examples of where this happens?

As described in the commit log, the def_bool, def_trristate, and default can appear in the second line.

config FOO
def_tristate BAR

Then, FOO has dependency on BAR.

Quoted 16 lines

Thanks,

-- Steve

Quoted 12 lines

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- changed the subject
- changed the condition to be more in line with the script style

---
 scripts/kconfig/streamline_config.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 14 lines

scripts/kconfig/streamline_config.pl sub read_kconfig { · line 226

Quoted 13 lines
scripts/kconfig/streamline_config.pl +1 -1
index a85d6a3108a1..85f4712e2bf3 100755
@@ -220,7 +220,7 @@ sub read_kconfig {
          $depends{$config} = $1;
      } elsif ($state eq "DEP" && /^\s*depends\s+on\s+(.*)$/) {
          $depends{$config} .= " " . $1;
-     } elsif ($state eq "DEP" && /^\s*def(_(bool|tristate)|ault)\s+(\S.*)$/) {
+     } elsif ($state ne "NONE" && /^\s*def(_(bool|tristate)|ault)\s+(\S.*)$/) {
          my $dep = $3;
          if ($dep !~ /^\s*(y|m|n)\s*$/) {
              $dep =~ s/.*\sif\s+//;

--
Best Regards
Masahiro Yamada

#6 lore raw

Re: [PATCH v2 2/7] streamline_config.pl: ensure all defaults are tracked

From: [email protected]

There are many instances of the second line of a config entry containing infomation about defaults. Here is a summary:

def_bools found: 746
def_tristates found: 59
defaults found: 169

Quoted 1 line

Can you list some examples of where this happens?

I created a file that finds all the times that a default information is given in the line following a CONFIG entry is created. Here is a link for the output, which will show the file and the line numbers:

https://github.com/dshunter107/linux-tools/blob/main/localmodconfig/defaults.output

If you would like to see the shell script I made to find the locations, you can view it at this link:

https://github.com/dshunter107/linux-tools/blob/main/localmodconfig/check_define.sh

#7 lore raw

Re: [PATCH v2 2/7] streamline_config.pl: ensure all defaults are tracked

Quoted 10 lines · On Tue, 26 Nov 2024 18:48:13 -0500 David Hunter <[email protected]>…

From: [email protected]

There are many instances of the second line of a config entry containing infomation about defaults. Here is a summary:

def_bools found: 746
def_tristates found: 59
defaults found: 169

Quoted 1 line

Can you list some examples of where this happens?

Quoted 8 lines

I created a file that finds all the times that a default information is given in the line following a CONFIG entry is created. Here is a link for the output, which will show the file and the line numbers:

https://github.com/dshunter107/linux-tools/blob/main/localmodconfig/defaults.output

If you would like to see the shell script I made to find the locations, you can view it at this link:

https://github.com/dshunter107/linux-tools/blob/main/localmodconfig/check_define.sh

Thanks, can you ping me again next week. I'm still finishing up this merge window and have a lot of tasks that need to get done before Thanksgiving, and will be taking a break from computing during the Holiday.

I'm not going to have time to look at this this week, and will likely forget about it.

-- Steve

#8 lore raw

Re: [PATCH v2 2/7] streamline_config.pl: ensure all defaults are tracked

In reply to David Hunter
Quoted 5 lines · David Hunter

Track default options on the second line. On the second line of some config entries, default and depndency options sometimes appear. In those instances, the state will be "NEW" and not "DEP".

Quoted 8 lines

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- changed the subject
- changed the condition to be more in line with the script style
---

Applied to kbuild with the typo fixed.
Thanks.

Quoted 17 lines
 scripts/kconfig/streamline_config.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/kconfig/streamline_config.pl +1 -2
index a85d6a3108a1..85f4712e2bf3 100755
@@ -220,7 +220,7 @@ sub read_kconfig {
            $depends{$config} = $1;
        } elsif ($state eq "DEP" && /^\s*depends\s+on\s+(.*)$/) {
            $depends{$config} .= " " . $1;
-       } elsif ($state eq "DEP" && /^\s*def(_(bool|tristate)|ault)\s+(\S.*)$/) {
+       } elsif ($state ne "NONE" && /^\s*def(_(bool|tristate)|ault)\s+(\S.*)$/) {
            my $dep = $3;
            if ($dep !~ /^\s*(y|m|n)\s*$/) {
                $dep =~ s/.*\sif\s+//;
--
Quoted 2 lines

2.43.0

--
Best Regards
Masahiro Yamada

#9 lore raw

PATCH v2 3/7 streamline_config.pl: remove prompt warnings for configs with defaults

In reply to David Hunter

Ignore process select warnings for config entries that have a default option. Some config entries have no prompt, and nothing selects them, but these config options are okay because they have a default option.

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- changed subject
- put a space between "if" and "("

---
 scripts/kconfig/streamline_config.pl | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
scripts/kconfig/streamline_config.pl +12 -2
index 85f4712e2bf3..8e23faab5d22 100755
@@ -144,6 +144,7 @@ my %selects;
 my %prompts;
 my %objects;
 my %config2kfile;
+my %defaults;
 my $var;
 my $iflevel = 0;
 my @ifdeps;
@@ -222,6 +223,7 @@ sub read_kconfig {
 	    $depends{$config} .= " " . $1;
 	} elsif ($state ne "NONE" && /^\s*def(_(bool|tristate)|ault)\s+(\S.*)$/) {
 	    my $dep = $3;
+            $defaults{$config} = 1;
 	    if ($dep !~ /^\s*(y|m|n)\s*$/) {
 		$dep =~ s/.*\sif\s+//;
 		$depends{$config} .= " " . $dep;
@@ -523,8 +525,16 @@ sub parse_config_selects
 
     # If no possible config selected this, then something happened.
     if (!defined($next_config)) {
-	print STDERR "WARNING: $config is required, but nothing in the\n";
-	print STDERR "  current config selects it.\n";
+
+	# Some config options have no prompt, and nothing selects them, but
+	# they stay turned on once the final checks for the configs
+	# are done. These configs have a default option, so turn off the
+	# warnings for configs with default options.
+	if (!defined($defaults{$config})) {
+	    print STDERR "WARNING: $config is required, but nothing in the\n";
+	    print STDERR "  current config selects it.\n";
+	}
+
 	return;
     }
 
Signature · 1 line
2.43.0
#10 lore raw

Re: [PATCH v2 3/7] streamline_config.pl: remove prompt warnings for configs with defaults

Quoted 7 lines · David Hunter

Ignore process select warnings for config entries that have a default option. Some config entries have no prompt, and nothing selects them, but these config options are okay because they have a default option.

---

Applied to linux-kbuild.
Thanks.

Quoted 51 lines

V1 https://lore.kernel.org/all/[email protected]/

V2
- changed subject
- put a space between "if" and "("

---
 scripts/kconfig/streamline_config.pl | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
scripts/kconfig/streamline_config.pl +12 -3
index 85f4712e2bf3..8e23faab5d22 100755
@@ -144,6 +144,7 @@ my %selects;
 my %prompts;
 my %objects;
 my %config2kfile;
+my %defaults;
 my $var;
 my $iflevel = 0;
 my @ifdeps;
@@ -222,6 +223,7 @@ sub read_kconfig {
            $depends{$config} .= " " . $1;
        } elsif ($state ne "NONE" && /^\s*def(_(bool|tristate)|ault)\s+(\S.*)$/) {
            my $dep = $3;
+            $defaults{$config} = 1;
            if ($dep !~ /^\s*(y|m|n)\s*$/) {
                $dep =~ s/.*\sif\s+//;
                $depends{$config} .= " " . $dep;
@@ -523,8 +525,16 @@ sub parse_config_selects
 
     # If no possible config selected this, then something happened.
     if (!defined($next_config)) {
-       print STDERR "WARNING: $config is required, but nothing in the\n";
-       print STDERR "  current config selects it.\n";
+
+       # Some config options have no prompt, and nothing selects them, but
+       # they stay turned on once the final checks for the configs
+       # are done. These configs have a default option, so turn off the
+       # warnings for configs with default options.
+       if (!defined($defaults{$config})) {
+           print STDERR "WARNING: $config is required, but nothing in the\n";
+           print STDERR "  current config selects it.\n";
+       }
+
        return;
     }
 
--

2.43.0

Signature · 2 lines
Best Regards
Masahiro Yamada
#11 lore raw

PATCH 4/7 streamline_config.pl: include tool to learn about a config option

In reply to David Hunter

Include a debugging tool that can help developers find out about a particular config option.

It can be challenging to find out what is going on behind the scenes of the script due to the vast amount of config options and the ways the config options interact with each other.

---
 scripts/kconfig/streamline_config.pl | 49 ++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
scripts/kconfig/streamline_config.pl +49 -0
index f5a07b3a1922..bbcd5a6e9726 100755
@@ -713,4 +713,53 @@ foreach my $module (keys(%modules)) {
     }
 }
 
+# the following tool can be used to check a particular config option.
+# You can read the output if the variable "debugprint" is set to "1."
+# Here is a sample way to use the tool:
+# dprintvar("INET_ESP");
+#
+sub dprintvar {
+    if ($debugprint) {
+	my $dconfig = $_[0];
+	my $fconfig = "CONFIG_$dconfig";
+	my $con_found = "";
+	my $dep_found = "";
+	my $def_found = "";
+	my $sel_found = "";
+	my $prom_found = "";
+	my $ori_found = "";
+	my $proc_found = "";
+
+        if (defined($configs{$fconfig})) {
+            $con_found = $configs{$fconfig};
+        }
+        if (defined($depends{$dconfig})) {
+            $dep_found = $depends{$dconfig};
+        }
+        if (defined($defaults{$dconfig})) {
+            $def_found = $defaults{$dconfig};
+        }
+        if (defined($selects{$dconfig})) {
+            $sel_found = $selects{$dconfig};
+        }
+        if (defined($prompts{$dconfig})) {
+            $prom_found = $prompts{$dconfig};
+        }
+        if (defined($orig_configs{$fconfig})) {
+            $ori_found = $orig_configs{$fconfig};
+        }
+        if (defined($process_selects{$dconfig})) {
+            $proc_found = $process_selects{$dconfig};
+        }
+
+        dprint "config: $dconfig\n";
+        dprint "config found: $con_found\n";
+        dprint "depends: $dep_found\n";
+        dprint "defaults: $def_found\n";
+        dprint "selects: $sel_found\n";
+        dprint "prompts: $prom_found\n";
+        dprint "in original config: $ori_found\n";
+	dprint "process selected: $proc_found\n";
+    }
+}
 # vim: softtabstop=4
Signature · 1 line
2.43.0
#12 lore raw

Re: [PATCH v1 4/7] streamline_config.pl: include tool to learn about a config option

Quoted 10 lines · David Hunter

Include a debugging tool that can help developers find out about a particular config option.

It can be challenging to find out what is going on behind the scenes of the script due to the vast amount of config options and the ways the config options interact with each other.

---

This is not suitable for upstream.

Indeed, this tool is broken, and hard to figure out what is happening, but you can add print-debug when needed.

Quoted 64 lines
 scripts/kconfig/streamline_config.pl | 49 ++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
scripts/kconfig/streamline_config.pl +49 -1
index f5a07b3a1922..bbcd5a6e9726 100755
@@ -713,4 +713,53 @@ foreach my $module (keys(%modules)) {
     }
 }
 
+# the following tool can be used to check a particular config option.
+# You can read the output if the variable "debugprint" is set to "1."
+# Here is a sample way to use the tool:
+# dprintvar("INET_ESP");
+#
+sub dprintvar {
+    if ($debugprint) {
+       my $dconfig = $_[0];
+       my $fconfig = "CONFIG_$dconfig";
+       my $con_found = "";
+       my $dep_found = "";
+       my $def_found = "";
+       my $sel_found = "";
+       my $prom_found = "";
+       my $ori_found = "";
+       my $proc_found = "";
+
+        if (defined($configs{$fconfig})) {
+            $con_found = $configs{$fconfig};
+        }
+        if (defined($depends{$dconfig})) {
+            $dep_found = $depends{$dconfig};
+        }
+        if (defined($defaults{$dconfig})) {
+            $def_found = $defaults{$dconfig};
+        }
+        if (defined($selects{$dconfig})) {
+            $sel_found = $selects{$dconfig};
+        }
+        if (defined($prompts{$dconfig})) {
+            $prom_found = $prompts{$dconfig};
+        }
+        if (defined($orig_configs{$fconfig})) {
+            $ori_found = $orig_configs{$fconfig};
+        }
+        if (defined($process_selects{$dconfig})) {
+            $proc_found = $process_selects{$dconfig};
+        }
+
+        dprint "config: $dconfig\n";
+        dprint "config found: $con_found\n";
+        dprint "depends: $dep_found\n";
+        dprint "defaults: $def_found\n";
+        dprint "selects: $sel_found\n";
+        dprint "prompts: $prom_found\n";
+        dprint "in original config: $ori_found\n";
+       dprint "process selected: $proc_found\n";
+    }
+}
 # vim: softtabstop=4
--

2.43.0

Signature · 2 lines
Best Regards
Masahiro Yamada
#13 lore raw

PATCH v2 5/7 streamline_config.pl: fix: implement choice for kconfigs

In reply to David Hunter

Properly implement the config entries that are within the choice keyword for kconfig. Currently, the script only stops the previous config entry when a choice keyword is encountered.

When the keyword "choice" is encountered, do the following:
- distribute the lines immediately following the "choice" keyword to each config entry inside the "choice" section.
- process the config entries with the distributed lines.

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- changed the subject prefix
- changed the method for storing and distributing the choice block. No longer using temp file.

---
 scripts/kconfig/streamline_config.pl | 47 ++++++++++++++++++++++++++--
 1 file changed, 45 insertions(+), 2 deletions(-)
scripts/kconfig/streamline_config.pl +45 -2
index b7ed79c5e070..4149c4b344db 100755
@@ -149,6 +149,34 @@ my $var;
 my $iflevel = 0;
 my @ifdeps;
 
+# distributes choice entries to different config options
+sub set_hash_value {
+    my %htable = %{$_[0]};
+    my $tmp_config = $_[1];
+    my $current_config = $_[2];
+    if (defined($htable{$tmp_config})) {
+	${$_[0]}{$current_config} = $htable{$tmp_config};
+    }
+}
+
+# distribute choice config entries
+sub copy_configs {
+    my $tmp_config = "TMP_CONFIG";
+    my $choice_config = $_[0];
+    set_hash_value (\%depends, $tmp_config, $choice_config);
+    set_hash_value (\%selects, $tmp_config, $choice_config);
+    set_hash_value (\%prompts, $tmp_config, $choice_config);
+    set_hash_value (\%defaults, $tmp_config, $choice_config);
+}
+
+sub delete_temp_config {
+    my $tmp_config = "TMP_CONFIG";
+    $depends{$tmp_config} = undef;
+    $selects{$tmp_config} = undef;
+    $prompts{$tmp_config} = undef;
+    $defaults{$tmp_config} = undef;
+}
+
 # prevent recursion
 my %read_kconfigs;
 
@@ -163,6 +191,7 @@ sub read_kconfig {
 
     my $source = "$ksource/$kconfig";
     my $last_source = "";
+    my $choice_activated = 0;
 
     # Check for any environment variables used
     while ($source =~ /\$\((\w+)\)/ && $last_source ne $source) {
@@ -205,9 +234,13 @@ sub read_kconfig {
 	    $config = $2;
 	    $config2kfile{"CONFIG_$config"} = $kconfig;
 
+	    if ($choice_activated) {
+		copy_configs $config;
+	    }
+
 	    # Add depends for 'if' nesting
 	    for (my $i = 0; $i < $iflevel; $i++) {
-		if ($i) {
+		if (defined($depends{$config})) {
 		    $depends{$config} .= " " . $ifdeps[$i];
 		} else {
 		    $depends{$config} = $ifdeps[$i];
@@ -260,8 +293,18 @@ sub read_kconfig {
 	    $iflevel-- if ($iflevel);
 
 	# stop on "help" and keywords that end a menu entry
-	} elsif (/^\s*(---)?help(---)?\s*$/ || /^(comment|choice|menu)\b/) {
+	} elsif (/^\s*(---)?help(---)?\s*$/ || /^(comment|menu)\b/) {
 	    $state = "NONE";
+
+	# for choice, distribute the lines before each config entry to
+	# each config entry
+	} elsif (/^\s*choice\b/) {
+	    $choice_activated = 1;
+	    $config = "TMP_CONFIG";
+	    $state = "NEW";
+	} elsif (/^\s*endchoice/) {
+	    delete_temp_config;
+	    $choice_activated = 0;
 	}
     }
     close($kinfile);
Signature · 1 line
2.43.0
#14 lore raw

Re: [PATCH v2 5/7] streamline_config.pl: fix: implement choice for kconfigs

Quoted 26 lines · On Mon, 14 Oct 2024 10:13:35 -0400 David Hunter <[email protected]>…

Properly implement the config entries that are within the choice keyword for kconfig. Currently, the script only stops the previous config entry when a choice keyword is encountered.

When the keyword "choice" is encountered, do the following:
- distribute the lines immediately following the "choice" keyword to each config entry inside the "choice" section.
- process the config entries with the distributed lines.

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- changed the subject prefix
- changed the method for storing and distributing the choice block. No longer using temp file.

---
 scripts/kconfig/streamline_config.pl | 47 ++++++++++++++++++++++++++--
 1 file changed, 45 insertions(+), 2 deletions(-)
scripts/kconfig/streamline_config.pl +0 -0
index b7ed79c5e070..4149c4b344db 100755

scripts/kconfig/streamline_config.pl my $var; · line 156

Quoted 8 lines
scripts/kconfig/streamline_config.pl +5 -0
@@ -149,6 +149,34 @@ my $var;
 my $iflevel = 0;
 my @ifdeps;
 
+# distributes choice entries to different config options
+sub set_hash_value {
+    my %htable = %{$_[0]};
+    my $tmp_config = $_[1];
+    my $current_config = $_[2];

The other functions usually make this a single line. Also, we should give names for all parameters.

  sub set_hash_value {
      my ($phtable, $tmp_config, $current_config) = @_;
      my %htable = %{$phtable};
      if (defined($htable{$tmp_config})) {
          ${$phtable}{$current_config} = $htable{$tmp_config};
      }
  }

Names make it a bit easier to read. Perl is already know as being very cryptic, we don't need to make it worse.

scripts/kconfig/streamline_config.pl my $var; · line 165

Quoted 9 lines
scripts/kconfig/streamline_config.pl +9 -0
+    if (defined($htable{$tmp_config})) {
+	${$_[0]}{$current_config} = $htable{$tmp_config};
+    }
+}
+
+# distribute choice config entries
+sub copy_configs {
+    my $tmp_config = "TMP_CONFIG";
+    my $choice_config = $_[0];

And this as:

  sub copy_configs {
     my ($choice_config) = $@;
     my $tmp_config = "TMP_CONFIG"

As parameters should always be first, which is how the rest of the file does it. Consistency would be good.

-- Steve

Quoted 61 lines
scripts/kconfig/streamline_config.pl +31 -2
+    set_hash_value (\%depends, $tmp_config, $choice_config);
+    set_hash_value (\%selects, $tmp_config, $choice_config);
+    set_hash_value (\%prompts, $tmp_config, $choice_config);
+    set_hash_value (\%defaults, $tmp_config, $choice_config);
+}
+
+sub delete_temp_config {
+    my $tmp_config = "TMP_CONFIG";
+    $depends{$tmp_config} = undef;
+    $selects{$tmp_config} = undef;
+    $prompts{$tmp_config} = undef;
+    $defaults{$tmp_config} = undef;
+}
+
 # prevent recursion
 my %read_kconfigs;
 
@@ -163,6 +191,7 @@ sub read_kconfig {
 
     my $source = "$ksource/$kconfig";
     my $last_source = "";
+    my $choice_activated = 0;
 
     # Check for any environment variables used
     while ($source =~ /\$\((\w+)\)/ && $last_source ne $source) {
@@ -205,9 +234,13 @@ sub read_kconfig {
 	    $config = $2;
 	    $config2kfile{"CONFIG_$config"} = $kconfig;
 
+	    if ($choice_activated) {
+		copy_configs $config;
+	    }
+
 	    # Add depends for 'if' nesting
 	    for (my $i = 0; $i < $iflevel; $i++) {
-		if ($i) {
+		if (defined($depends{$config})) {
 		    $depends{$config} .= " " . $ifdeps[$i];
 		} else {
 		    $depends{$config} = $ifdeps[$i];
@@ -260,8 +293,18 @@ sub read_kconfig {
 	    $iflevel-- if ($iflevel);
 
 	# stop on "help" and keywords that end a menu entry
-	} elsif (/^\s*(---)?help(---)?\s*$/ ||
 /^(comment|choice|menu)\b/) {
+	} elsif (/^\s*(---)?help(---)?\s*$/ || /^(comment|menu)\b/) {
 	    $state = "NONE";
+
+	# for choice, distribute the lines before each config entry to
+	# each config entry
+	} elsif (/^\s*choice\b/) {
+	    $choice_activated = 1;
+	    $config = "TMP_CONFIG";
+	    $state = "NEW";
+	} elsif (/^\s*endchoice/) {
+	    delete_temp_config;
+	    $choice_activated = 0;
 	}
     }
     close($kinfile);
#15 lore raw

Re: [PATCH v2 5/7] streamline_config.pl: fix: implement choice for kconfigs

In reply to David Hunter
Quoted 48 lines · David Hunter

Properly implement the config entries that are within the choice keyword for kconfig. Currently, the script only stops the previous config entry when a choice keyword is encountered.

When the keyword "choice" is encountered, do the following:
- distribute the lines immediately following the "choice" keyword to each config entry inside the "choice" section.
- process the config entries with the distributed lines.

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- changed the subject prefix
- changed the method for storing and distributing the choice block. No longer using temp file.

---
 scripts/kconfig/streamline_config.pl | 47 ++++++++++++++++++++++++++--
 1 file changed, 45 insertions(+), 2 deletions(-)
scripts/kconfig/streamline_config.pl +18 -0
index b7ed79c5e070..4149c4b344db 100755
@@ -149,6 +149,34 @@ my $var;
 my $iflevel = 0;
 my @ifdeps;
 
+# distributes choice entries to different config options
+sub set_hash_value {
+    my %htable = %{$_[0]};
+    my $tmp_config = $_[1];
+    my $current_config = $_[2];
+    if (defined($htable{$tmp_config})) {
+       ${$_[0]}{$current_config} = $htable{$tmp_config};
+    }
+}
+
+# distribute choice config entries
+sub copy_configs {
+    my $tmp_config = "TMP_CONFIG";
+    my $choice_config = $_[0];
+    set_hash_value (\%depends, $tmp_config, $choice_config);
+    set_hash_value (\%selects, $tmp_config, $choice_config);
+    set_hash_value (\%prompts, $tmp_config, $choice_config);
+    set_hash_value (\%defaults, $tmp_config, $choice_config);

scripts/kconfig/streamline_config.pl my $var; · line 179

Quoted 10 lines
scripts/kconfig/streamline_config.pl +10 -0
+}
+
+sub delete_temp_config {
+    my $tmp_config = "TMP_CONFIG";
+    $depends{$tmp_config} = undef;
+    $selects{$tmp_config} = undef;
+    $prompts{$tmp_config} = undef;
+    $defaults{$tmp_config} = undef;
+}
+

I previously suggested checking how the 'if' statement is handled.
https://lore.kernel.org/lkml/CAK7LNAQ8D4OVT81iTVs8jjrBXX6Zgwc+VJ_vb7hb4J-vCZZN=g@mail.gmail.com/

These two behave in a similar way
regarding the dependency propagation.

if FOO
config A
bool "A"

config B
bool "B"
endif

choice
prompt "choose"
depends on FOO
config A
bool "A"
config B
bool "B"
endchoice

The inner A and B inherit the dependency on FOO.

I attached a completely-untested patch.
I think 'if' and 'choice' can share the code.

BTW, 'menu' also can have 'depends on'.

menu "menu"
depends on FOO
config A
bool "A"
config B
bool "B"
endmenu

This is not implemented, either.

I am not sure how much effort should be invested in this script, though.

Signature · 2 lines
Best Regards
Masahiro Yamada
scripts/kconfig/streamline_config.pl +14 -5
index 8e23faab5d22..28faf8cd0e2a 100755
@@ -146,7 +146,7 @@ my %objects;
 my %config2kfile;
 my %defaults;
 my $var;
-my $iflevel = 0;
+my $iflevel = -1;
 my @ifdeps;
 
 # prevent recursion
@@ -206,7 +206,7 @@ sub read_kconfig {
 	    $config2kfile{"CONFIG_$config"} = $kconfig;
 
 	    # Add depends for 'if' nesting
-	    for (my $i = 0; $i < $iflevel; $i++) {
+	    for (my $i = 0; $i <= $iflevel; $i++) {
 		if ($i) {
 		    $depends{$config} .= " " . $ifdeps[$i];
 		} else {
@@ -221,6 +221,10 @@ sub read_kconfig {
 	    $depends{$config} = $1;
 	} elsif ($state eq "DEP" && /^\s*depends\s+on\s+(.*)$/) {
 	    $depends{$config} .= " " . $1;
+	} elsif ($state eq "NONE" && /^\s*depends\s+on\s+(.*)$/) {
+	    # "depends on" in the "NONE" state is from a choice block.
+	    # Propagate the dependency to its member config options.
+	    $ifdeps[$iflevel] .= ':' . $ifdeps[$iflevel];
 	} elsif ($state ne "NONE" && /^\s*def(_(bool|tristate)|ault)\s+(\S.*)$/) {
 	    my $dep = $3;
             $defaults{$config} = 1;
@@ -253,11 +258,16 @@ sub read_kconfig {
 
 	    my @deps = split /[^a-zA-Z0-9_]+/, $deps;
 
-	    $ifdeps[$iflevel++] = join ':', @deps;
+	    $ifdeps[++$iflevel] = join ':', @deps;
 
-	} elsif (/^endif/) {
+	} elsif (/^\s*choice\s*$/) {
 
-	    $iflevel-- if ($iflevel);
+	    $state = "NONE";
+	    $ifdeps[++$iflevel] = '';
+
+	} elsif (/^(endif|endchoice)/) {
+
+	    --$iflevel if ($iflevel > -1);
 
 	# stop on "help" and keywords that end a menu entry
 	} elsif (/^\s*(---)?help(---)?\s*$/ || /^(comment|choice|menu)\b/) {
#16 lore raw

Re: [PATCH v2 5/7] streamline_config.pl: fix: implement choice for kconfigs

Quoted 3 lines · Masahiro Yamada

I previously suggested checking how the 'if' statement is handled.
https://lore.kernel.org/lkml/CAK7LNAQ8D4OVT81iTVs8jjrBXX6Zgwc+VJ_vb7hb4J-vCZZN=g@mail.gmail.com/

I think I understand now what you were saying. I misunderstood what you were saying because I thought that you were saying that the "if" blocks were not implemented.

To paraphrase, I believe that you are saying that the "choice" blocks should have a similar style to the "if" blocks.

I will take a look at the patch that you sent and figure out how it would work. I would like some clarification on the information in the choice blocks that are not "depends." Should those also have the same style as the "if" block?

I am not sure if you saw this email:
https://lore.kernel.org/all/[email protected]/

There are lots of information, specifically "prompts" and "defaults" that are distributed to each of the config options in the "choice" blocks.

Quoted 13 lines

BTW, 'menu' also can have 'depends on'.

menu "menu"
depends on FOO
config A
bool "A"
config B
bool "B"
endmenu

Quoted 5 lines

This is not implemented, either.

I am not sure how much effort should be invested in this script, though.

I will look into distributing the "menu" information.

#17 lore raw

Re: [PATCH v2 5/7] streamline_config.pl: fix: implement choice for kconfigs

Quoted 9 lines · David Hunter
Quoted 3 lines · Masahiro Yamada

I previously suggested checking how the 'if' statement is handled.
https://lore.kernel.org/lkml/CAK7LNAQ8D4OVT81iTVs8jjrBXX6Zgwc+VJ_vb7hb4J-vCZZN=g@mail.gmail.com/

I think I understand now what you were saying. I misunderstood what you were saying because I thought that you were saying that the "if" blocks were not implemented.

Quoted 8 lines

To paraphrase, I believe that you are saying that the "choice" blocks should have a similar style to the "if" blocks.

I will take a look at the patch that you sent and figure out how it would work. I would like some clarification on the information in the choice blocks that are not "depends." Should those also have the same style as the "if" block?

I am not sure if I understood your question correctly, but I guess you are referring to the similarity between the following two constructs:

[1]
choice
prompt "choice"
depends on X

[ code block]

endchoice

[2]
if X

[code block]

endif

choice ... endchoice
if ... endif
are similar in the sense that they both define a code block.

The if ... endif construct is always associated with a dependency.

The choice ... endchoice construct can optionally have a 'depends on' statement. If it does, the dependency is applied to the entire code block within choice ... endchoice.

Quoted 5 lines

I am not sure if you saw this email:
https://lore.kernel.org/all/[email protected]/

There are lots of information, specifically "prompts" and "defaults" that are distributed to each of the config options in the "choice" blocks.

- A choice entry cannot have a 'select' property.

- A choice entry should always have 'prompt', otherwise the choice statement is non-sense.

- A choice entry can optionally have 'default'.
If 'default' is not explicitly specified, the first CONFIG option within
the choice block is the default.

- A choice entry can optionally have 'depends on'.

Quoted 19 lines
Quoted 13 lines

BTW, 'menu' also can have 'depends on'.

menu "menu"
depends on FOO
config A
bool "A"
config B
bool "B"
endmenu

Quoted 5 lines

This is not implemented, either.

I am not sure how much effort should be invested in this script, though.

I will look into distributing the "menu" information.

Signature · 2 lines
Best Regards
Masahiro Yamada
#18 lore raw

PATCH v2 6/7 streamline_config.pl: process config options set to "y"

In reply to David Hunter

An assumption made in this script is that the config options do not need to be processed because they will simply be in the new config file. This assumption is incorrect.

Process the config entries set to "y" because those config entries might have dependencies set to "m". If a config entry is set to "m" and is not loaded directly into the machine, the script will currently turn off that config entry; however, if that turned off config entry is a dependency for a "y" option. that means the config entry set to "y" will also be turned off later when the conf executive file is called.

Here is a model of the problem (arrows show dependency):

Original config file
Config_1 (m) <-- Config_2 (y)

Config_1 is not loaded in this example, so it is turned off. After scripts/kconfig/streamline_config.pl, but before scripts/kconfig/conf Config_1 (n) <-- Config_2 (y)

After scripts/kconfig/conf
Config_1 (n) <-- Config_2 (n)

It should also be noted that any module in the dependency chain will also be turned off, even if that module is loaded directly onto the computer. Here is an example:

Original config file
Config_1 (m) <-- Config_2 (y) <-- Config_3 (m)

Config_3 will be loaded in this example.
After scripts/kconfig/streamline_config.pl, but before scripts/kconfig/conf Config_1 (n) <-- Config_2 (y) <-- Config_3 (m)

After scripts/kconfig/conf
Config_1 (n) <-- Config_2 (n) <-- Config_3 (n)

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- change subject
- changed part of the code that only processed config options set to "m" so that config options set to "y" are processed. I forgot to put this in v1 when making the series patch.
- removed an unneccessary condition that would have skipped a dependency loop.

---
 scripts/kconfig/streamline_config.pl | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)
scripts/kconfig/streamline_config.pl +7 -8
index 4149c4b344db..c3b434220c9f 100755
@@ -473,6 +473,11 @@ foreach my $line (@config_file) {
 
     if (/(CONFIG_[$valid]*)=(m|y)/) {
 	$orig_configs{$1} = $2;
+	# all configs options set to 'y' need to be processed
+	if ($2 eq "y") {
+            $configs{$1}= $2;
+        }
+
     }
 }
 
@@ -499,8 +504,8 @@ sub parse_config_depends
 
 	    $p =~ s/^[^$valid]*[$valid]+//;
 
-	    # We only need to process if the depend config is a module
-	    if (!defined($orig_configs{$conf}) || $orig_configs{$conf} eq "y") {
+	    # Make sure that this config exists in the current .config file
+	    if (!defined($orig_configs{$conf})) {
 		next;
 	    }
 
@@ -600,12 +605,6 @@ sub loop_depend {
 
       forloop:
 	foreach my $config (keys %configs) {
-
-	    # If this config is not a module, we do not need to process it
-	    if (defined($orig_configs{$config}) && $orig_configs{$config} ne "m") {
-		next forloop;
-	    }
-
 	    $config =~ s/^CONFIG_//;
 	    $depconfig = $config;
 
Signature · 1 line
2.43.0
#19 lore raw

Re: [PATCH v2 6/7] streamline_config.pl: process config options set to "y"

Quoted 16 lines · On Mon, 14 Oct 2024 10:13:36 -0400 David Hunter <[email protected]>…

An assumption made in this script is that the config options do not need to be processed because they will simply be in the new config file. This assumption is incorrect.

Process the config entries set to "y" because those config entries might have dependencies set to "m". If a config entry is set to "m" and is not loaded directly into the machine, the script will currently turn off that config entry; however, if that turned off config entry is a dependency for a "y" option. that means the config entry set to "y" will also be turned off later when the conf executive file is called.

Here is a model of the problem (arrows show dependency):

Original config file
Config_1 (m) <-- Config_2 (y)

Quoted 6 lines

Config_1 is not loaded in this example, so it is turned off. After scripts/kconfig/streamline_config.pl, but before scripts/kconfig/conf Config_1 (n) <-- Config_2 (y)

After scripts/kconfig/conf
Config_1 (n) <-- Config_2 (n)

Hmm, can you show the example of this. My worry here is that there are a lot of cases that have:

Config_1 (m) <-- Config_2 (y)

Where Config_2 is just some binary option to modify the module set by Config_1. If we keep Config_1 because there's some 'y' that selects it, we are going to keep a lot of unused modules around.

Quoted 7 lines

It should also be noted that any module in the dependency chain will also be turned off, even if that module is loaded directly onto the computer. Here is an example:

Original config file
Config_1 (m) <-- Config_2 (y) <-- Config_3 (m)

If Config_3 is used, then it should add Config_2 as a dependency. I guess that's what this patch does.

Quoted 51 lines

Config_3 will be loaded in this example.
After scripts/kconfig/streamline_config.pl, but before scripts/kconfig/conf Config_1 (n) <-- Config_2 (y) <-- Config_3 (m)

After scripts/kconfig/conf
Config_1 (n) <-- Config_2 (n) <-- Config_3 (n)

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- change subject
- changed part of the code that only processed config options set to "m" so that config options set to "y" are processed. I forgot to put this in v1 when making the series patch.
- removed an unneccessary condition that would have skipped a dependency loop.

---
 scripts/kconfig/streamline_config.pl | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)
scripts/kconfig/streamline_config.pl +7 -2
index 4149c4b344db..c3b434220c9f 100755
@@ -473,6 +473,11 @@ foreach my $line (@config_file) {
 
     if (/(CONFIG_[$valid]*)=(m|y)/) {
 	$orig_configs{$1} = $2;
+	# all configs options set to 'y' need to be processed
+	if ($2 eq "y") {
+            $configs{$1}= $2;
+        }
+
     }
 }
 
@@ -499,8 +504,8 @@ sub parse_config_depends
 
 	    $p =~ s/^[^$valid]*[$valid]+//;
 
-	    # We only need to process if the depend config is a module
-	    if (!defined($orig_configs{$conf}) || $orig_configs{$conf} eq "y") {
+	    # Make sure that this config exists in the current .config file
+	    if (!defined($orig_configs{$conf})) {
 		next;
 	    }
 

scripts/kconfig/streamline_config.pl sub loop_depend { · line 608

Quoted 9 lines
scripts/kconfig/streamline_config.pl +0 -6
@@ -600,12 +605,6 @@ sub loop_depend {
 
       forloop:
 	foreach my $config (keys %configs) {
-
-	    # If this config is not a module, we do not need to process it
-	    if (defined($orig_configs{$config}) && $orig_configs{$config} ne "m") {
-		next forloop;
-	    }
-

OK, and I guess you tests show this doesn't have any other issues. I'll check it out.

Thanks,

-- Steve

Quoted 3 lines
scripts/kconfig/streamline_config.pl +0 -0
 	    $config =~ s/^CONFIG_//;
 	    $depconfig = $config;
 
#20 lore raw

Re: [PATCH v2 6/7] streamline_config.pl: process config options set to "y"

Quoted 27 lines · Steven Rostedt
Quoted 16 lines · On Mon, 14 Oct 2024 10:13:36 -0400 David Hunter <[email protected]>…

An assumption made in this script is that the config options do not need to be processed because they will simply be in the new config file. This assumption is incorrect.

Process the config entries set to "y" because those config entries might have dependencies set to "m". If a config entry is set to "m" and is not loaded directly into the machine, the script will currently turn off that config entry; however, if that turned off config entry is a dependency for a "y" option. that means the config entry set to "y" will also be turned off later when the conf executive file is called.

Here is a model of the problem (arrows show dependency):

Original config file
Config_1 (m) <-- Config_2 (y)

Quoted 6 lines

Config_1 is not loaded in this example, so it is turned off. After scripts/kconfig/streamline_config.pl, but before scripts/kconfig/conf Config_1 (n) <-- Config_2 (y)

After scripts/kconfig/conf
Config_1 (n) <-- Config_2 (n)

Quoted 8 lines

Hmm, can you show the example of this. My worry here is that there are a lot of cases that have:

Config_1 (m) <-- Config_2 (y)

Where Config_2 is just some binary option to modify the module set by Config_1. If we keep Config_1 because there's some 'y' that selects it, we are going to keep a lot of unused modules around.

Basically, there is no good way to distinguish in-module option (just CONFIG_BCACHEFS_SIX_OPTIMISTIC_SPIN=y, which is a property for CONFIG_BCACHEFS=m) from real dependency.

Quoted 10 lines
Quoted 7 lines

It should also be noted that any module in the dependency chain will also be turned off, even if that module is loaded directly onto the computer. Here is an example:

Original config file
Config_1 (m) <-- Config_2 (y) <-- Config_3 (m)

If Config_3 is used, then it should add Config_2 as a dependency. I guess that's what this patch does.

Agree, but I am not sure how to pick up only this case.

Signature · 2 lines
Best Regards
Masahiro Yamada
#21 lore raw

PATCH v2 7/7 streamline_config.pl: check prompt for bool options

In reply to David Hunter

Select configs that do not have a prompt. Config options can be bool or tristate. Ensure that bool options are also selected.

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- changed patch subject
- changed the order of this patch in the series patch

---
 scripts/kconfig/streamline_config.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/kconfig/streamline_config.pl +1 -1
index 4038afed2926..8c75ab888f87 100755
@@ -273,7 +273,7 @@ sub read_kconfig {
 	    }
 
 	# configs without prompts must be selected
-	} elsif ($state ne "NONE" && /^\s*(tristate\s+\S|prompt\b)/) {
+	} elsif ($state ne "NONE" && /^\s*((bool|tristate)\s+\S|prompt\b)/) {
 	    # note if the config has a prompt
 	    $prompts{$config} = 1;
 
Signature · 1 line
2.43.0
#22 lore raw

Re: [PATCH v2 7/7] streamline_config.pl: check prompt for bool options

Quoted 4 lines · David Hunter

Select configs that do not have a prompt. Config options can be bool or tristate. Ensure that bool options are also selected.

Quoted 8 lines

---
V1 https://lore.kernel.org/all/[email protected]/

V2
- changed patch subject
- changed the order of this patch in the series patch
---

As I reviewed in v1, this patch depends on 6/7.

Signature · 2 lines
Best Regards
Masahiro Yamada
#23 lore raw

Re: [PATCH v2 7/7] streamline_config.pl: check prompt for bool options

Quoted 2 lines

As I reviewed in v1, this patch depends on 6/7.

Hello,

I am a little confused by this part. I originally understood this to say that I should put it after the 6/7 patch because it depends on it. Should I have combined the two patches into one?

Sorry for not understanding.

#24 lore raw

Re: [PATCH v2 7/7] streamline_config.pl: check prompt for bool options

Quoted 11 lines · David Hunter
Quoted 3 lines

As I reviewed in v1, this patch depends on 6/7.

Hello,

I am a little confused by this part. I originally understood this to say that I should put it after the 6/7 patch because it depends on it. Should I have combined the two patches into one?

Sorry for not understanding.

No. What I understood from Steven's feedback is,
6/7 is skeptical.

6/7 enables many unnecessary modules because of them depending on 'y', but most of the cases, the 'y' value is module-internal switch.

While 6/7 may save some false-negatives, it introduces many false-positives.
So, I do not know whether it should get in or not.

If we do not apply 6/7, there is not reason to apply 7/7 either.

--
Best Regards
Masahiro Yamada

#25 lore raw

Re: [PATCH v2 7/7] streamline_config.pl: check prompt for bool options

Understood.

#26 lore raw
In reply to David Hunter
Quoted 46 lines · On Mon, 14 Oct 2024 10:13:30 -0400 David Hunter <[email protected]>…

An assumption made in this script is that the config options do not need to be processed because they will simply be in the new config file. This assumption is incorrect.

Process the config entries set to "y" because those config entries might have dependencies set to "m". If a config entry is set to "m" and is not loaded directly into the machine, the script will currently turn off that config entry; however, if that turned off config entry is a dependency for a "y" option. that means the config entry set to "y" will also be turned off later when the conf executive file is called.

Here is a model of the problem (arrows show dependency):

Original config file
Config_1 (m) <-- Config_2 (y)

Config_1 is not loaded in this example, so it is turned off. After scripts/kconfig/streamline_config.pl, but before scripts/kconfig/conf Config_1 (n) <-- Config_2 (y)

After scripts/kconfig/conf
Config_1 (n) <-- Config_2 (n)

It should also be noted that any module in the dependency chain will also be turned off, even if that module is loaded directly onto the computer. Here is an example:

Original config file
Config_1 (m) <-- Config_2 (y) <-- Config_3 (m)

Config_3 will be loaded in this example.
After scripts/kconfig/streamline_config.pl, but before scripts/kconfig/conf Config_1 (n) <-- Config_2 (y) <-- Config_3 (m)

After scripts/kconfig/conf
Config_1 (n) <-- Config_2 (n) <-- Config_3 (n)

I discovered this problem when I ran "make localmodconfig" on a generic Ubuntu config file. Many hardware devices were not recognized once the kernel was installed and booted. Another way to reproduced the error I had is to run "make localmodconfig" twice. The standard error might display warnings that certain modules should be selected but no config files are turned on that select that module.

Quoted 4 lines

With the changes in this series patch, all modules are loaded properly and all of the hardware is loaded when the kernel is installed and booted.

So I tried this out and compared it to what it use to do, and it was quite a big difference! Attached is the diffconfig.

I copied over my special config, ran localmodconfig, saved it, added your patches, copied over the same config, and then did the diffconfig between the two results.

There's a lot I know I don't use that was added!

For example:

BCACHEFS_FS n -> m

Why is that needed?

I can give you the starting config as well as my lsmod. But right now, it's adding too much that is not needed.

-- Steve

#27 lore raw
Quoted 2 lines · On Thu, 17 Oct 2024 18:37:11 -0400 Steven Rostedt <[email protected]> wrote:

I can give you the starting config as well as my lsmod. But right now, it's adding too much that is not needed.

Attached is the config I started with and the lsmod used. You could copy the config to .config and then run:

make LSMOD=lsmod localmodconfig

to test with.

-- Steve

#28 lore raw

From: [email protected]

I think that there are 2 distinct reasons that a module should not be turned off when using the 'localmodconfig' command.

1) the module is used directly
2) the module is needed for another module that has been included in the kernel with a "y".

I think we both agree on the first reason, but we may be having a disagreement/miscommunication for reason 2.

Quoted 5 lines

For example:

BCACHEFS_FS n -> m

Why is that needed?

With regards to your question, I see that 'CLOSURES' is set to 'y' in your original .config file. 'BCACHEFS_FS' selects 'CLOSURES'. I suspect that if 'BCACHEFS_FS' is set to 'n', then your config would have no way of keeping 'CLOSURES' as 'y'.

I understand that the following patch is not suitable for upstream; however, the following patch might help you to understand a little bit more about each config options like 'BCACHEFS_FS'.

https://lore.kernel.org/all/[email protected]/

If you put in

dprintvar("BCACHEFS_FS);

you would be able to see what selects 'BCACHEFS_FS' and what is selected by 'BCACHEFS_FS'. I suspect that if you were to use it for each of the config options that you have questions about, you would likely see that each config option set to "m" has a "y" option that it selects. The question then would be is the "y" option actually required for your particular machine.

This brings us to the root cause of the issue that we are having: there is no way to know if a config option set to 'y' is actually required. If there was a way to tell if 'CLOSURES' is needed, we can easily determine whether BCACHEFS_FS is actually necessary.

Without knowing whether the 'y' options are needed, we then have to make a determination whether it is better to have the (potentially necessary) 'y' options with the extra (not directly used) 'm' options or to drop those 'y' options along with those 'm' options.

My argument is that the 'y' options are important, even if we cannot determine whether they are used or not. The problem I had that made me aware of the issues with localmodconfig was that my new computer would consistently do an emergency shutdown whenever I would try to compile the kernel. I eventually realized that the fan was not being recognized, so my workaround was to put the BIOS in control of the fan (instead of the kernel).

It was not until I realized that I had a few hardware devices that were not being recognized that I was able to pinpoint the root cause of the problem. For clarity, some of my hardware devices that were nonfunctional were USBs, the microphone, Bluetooth (and as mentioned earlier, the fan).

I am curious to know if there are any hardware devices that are not recognized on your computer after you use localmodconfig. One way to check is to use use 'ls' on the psuedofilesystem or to use any of the commands (like lspci) that recognize hardware.

I do not know whether any of your hardware requires the 'CLOSURES' config, but because someone's hardware MAY need it, my reasoning is that streamline_config.pl should include 'BCACHEFS_FS' if it is the only thing that selects another config opiton set to 'y'.

On my computer, when all of these configs set to 'm' are added (because of reason 2), all of my hardware works (including the Bluetooth, microphone, and USBs.

If, on the other hand, you still feel like it would be better to not include these particular config options, I would ask that you allow a command line option like '-s' or '--safe' that users could use to make the config file with the added modules.

#29 lore raw
Quoted 11 lines · On Wed, 27 Nov 2024 18:05:47 -0500 David Hunter <[email protected]>…

From: [email protected]

I think that there are 2 distinct reasons that a module should not be turned off when using the 'localmodconfig' command.

1) the module is used directly
2) the module is needed for another module that has been included in the kernel with a "y".

I think we both agree on the first reason, but we may be having a disagreement/miscommunication for reason 2.

Note, Linus uses make localmodconfig a lot. He didn't always, because he complained that it would keep too many options enabled. I don't want to suddenly have Linus's make localmodconfig blow up the options again. I can see him complaining about it. Believe me, you don't want Linus to notice this patch for the wrong reasons.

This is why I'm very concerned about things like BCACHEFS being enabled after localmodconfig when there's obviously no reason for it. Even if it enables CLOSURES. BCACHEFS is an filesystem that I don't use. It should not be enabled on my machines.

Quoted 11 lines
Quoted 6 lines

For example:

BCACHEFS_FS n -> m

Why is that needed?

With regards to your question, I see that 'CLOSURES' is set to 'y' in your original .config file. 'BCACHEFS_FS' selects 'CLOSURES'. I suspect that if 'BCACHEFS_FS' is set to 'n', then your config would have no way of keeping 'CLOSURES' as 'y'.

Which is OK for this example.

Quoted 6 lines

I understand that the following patch is not suitable for upstream; however, the following patch might help you to understand a little bit more about each config options like 'BCACHEFS_FS'.

https://lore.kernel.org/all/[email protected]/

Note, I'll try to remember to look at this after the Thanksgiving break.

Quoted 11 lines

If you put in

dprintvar("BCACHEFS_FS);

you would be able to see what selects 'BCACHEFS_FS' and what is selected by 'BCACHEFS_FS'. I suspect that if you were to use it for each of the config options that you have questions about, you would likely see that each config option set to "m" has a "y" option that it selects. The question then would be is the "y" option actually required for your particular machine.

I would argue that if a 'y' depends on a 'm' then it's not needed unless that 'y' is a dependency for other 'm's that are needed. We should look into seeing if that's the case.

Quoted 5 lines

This brings us to the root cause of the issue that we are having: there is no way to know if a config option set to 'y' is actually required. If there was a way to tell if 'CLOSURES' is needed, we can easily determine whether BCACHEFS_FS is actually necessary.

I would say that CLOSURES is considered needed if a 'm' that is needed depends on it. Otherwise, I would say no.

Quoted 14 lines

Without knowing whether the 'y' options are needed, we then have to make a determination whether it is better to have the (potentially necessary) 'y' options with the extra (not directly used) 'm' options or to drop those 'y' options along with those 'm' options.

My argument is that the 'y' options are important, even if we cannot determine whether they are used or not. The problem I had that made me aware of the issues with localmodconfig was that my new computer would consistently do an emergency shutdown whenever I would try to compile the kernel. I eventually realized that the fan was not being recognized, so my workaround was to put the BIOS in control of the fan (instead of the kernel).

Quoted 4 lines

It was not until I realized that I had a few hardware devices that were not being recognized that I was able to pinpoint the root cause of the problem. For clarity, some of my hardware devices that were nonfunctional were USBs, the microphone, Bluetooth (and as mentioned earlier, the fan).

Can you show the dependency change for those devices. Knowing what exactly was disabled and knowing what was needed would be useful. That way we can fix the actual problem without using a big hammer approach of just keeping any module enabled that enables a 'y'.

Quoted 10 lines

I am curious to know if there are any hardware devices that are not recognized on your computer after you use localmodconfig. One way to check is to use use 'ls' on the psuedofilesystem or to use any of the commands (like lspci) that recognize hardware.

I do not know whether any of your hardware requires the 'CLOSURES' config, but because someone's hardware MAY need it, my reasoning is that streamline_config.pl should include 'BCACHEFS_FS' if it is the only thing that selects another config opiton set to 'y'.

BACHEFS is a new file system. If there's a dependency in my config that requires it, that's a bug in the config.

Quoted 9 lines

On my computer, when all of these configs set to 'm' are added (because of reason 2), all of my hardware works (including the Bluetooth, microphone, and USBs.

If, on the other hand, you still feel like it would be better to not include these particular config options, I would ask that you allow a command line option like '-s' or '--safe' that users could use to make the config file with the added modules.

Let's first find out what the problem is. Remember, this is code that Linus uses for his builds. Rule #1 I was told when I first became a kernel developer was, "Whatever you do. Don't break Linus's workstation!"

-- Steve