fixing the last failures in randconfig builds
Subsystems (4)
cover fixing the last failures in randconfig builds
I've recently done some build testing on 4.14-stable and to my surprise found only a handful of bugfixes were required to build all x86 and arm64 randconfig output without any warnings using gcc-7.2.1. I have submitted the remaining trivial patches now, and this is the non-trivial remaining part, where we turn off some options that are known to cause problems in randconfig builds.
While I've carried the same patches in my randconfig test tree for a long time, I wasn't entirely sure about my approach and never submitted them for inclusion. Now that this is literally all that is remains, I'd like to propose all three for inclusion.
If someone has another idea for how to address these, that's fine with me as well, as long as we can make sure at least COMPILE_TEST is enabled for randconfig builds by default (everything else can be derived from that if necessary).
Note that the 4.15 release has a couple of build-time regressions that I have sent bugfixes for but that so far have not made it in, and 4.16-rc still has a number of other issues without patches.
Arnd
Arnd Bergmann (3):
Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
Kconfig: improve handling for all{rand,yes,no,}.config fragments
Kconfig: add a default allrandom.config
kernel/configs/allrandom.config | 6 ++++++ kernel/trace/Kconfig | 1 + scripts/kconfig/conf.c | 27 +++++++++++++++++++-------- 3 files changed, 26 insertions(+), 8 deletions(-) create mode 100644 kernel/configs/allrandom.config
Signature · 1 line
2.9.0
PATCH 1/3 Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
This can easily double the time for compiling a driver but does not provide any benefit for the compile tester, so it's better left disabled.
In addition, any 'inline' function that is not also 'static' and that contains an 'if' causes a warning like
include/linux/string.h:212:2: note: in expansion of macro 'if'
if (strscpy(p, q, p_size < q_size ? p_size : q_size) < 0)
^~include/linux/compiler.h:162:4: warning: '______f' is static but declared in inline function 'strcpy' which is not static
without this patch, and I could not come up with a nice fix for that. In combination with my patch to always enable 'CONFIG_COMPILE_TEST' during 'randconfig' builds, we can at least hide these warnings for most users.
--- kernel/trace/Kconfig | 1 + 1 file changed, 1 insertion(+)
kernel/trace/Kconfig +1 -0
@@ -345,6 +345,7 @@ config PROFILE_ANNOTATED_BRANCHES config PROFILE_ALL_BRANCHES bool "Profile all if conditionals" select TRACE_BRANCH_PROFILING + depends on !COMPILE_TEST help This tracer profiles all branch conditions. Every if () taken in the kernel is recorded whether it hit or miss.
Signature · 1 line
2.9.0
Steven Rostedt
maintainerRe: [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
Quoted 11 lines · On Fri, 16 Feb 2018 22:41:11 +0100 Arnd Bergmann <[email protected]> wrote:
This can easily double the time for compiling a driver but does not provide any benefit for the compile tester, so it's better left disabled.
In addition, any 'inline' function that is not also 'static' and that contains an 'if' causes a warning like
include/linux/string.h:212:2: note: in expansion of macro 'if'
if (strscpy(p, q, p_size < q_size ? p_size : q_size)< 0)
^~include/linux/compiler.h:162:4: warning: '______f' is static but declared in inline function 'strcpy' which is not static
Quoted 4 lines
without this patch, and I could not come up with a nice fix for that. In combination with my patch to always enable '
CONFIG_COMPILE_TEST' during 'randconfig' builds, we can at least hide these warnings for most users.
This looks like it fixes the same issue that was already fixed and is in Linus's tree.
http://lkml.kernel.org/r/[email protected]
See commit 68e76e034b6b1 ("tracing: Prevent PROFILE_ALL_BRANCHES when FORTIFY_SOURCE=y")
-- Steve
Re: [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
Quoted 19 lines · Steven Rostedt
Quoted 11 lines · On Fri, 16 Feb 2018 22:41:11 +0100 Arnd Bergmann <[email protected]> wrote:
This can easily double the time for compiling a driver but does not provide any benefit for the compile tester, so it's better left disabled.
In addition, any 'inline' function that is not also 'static' and that contains an 'if' causes a warning like
include/linux/string.h:212:2: note: in expansion of macro 'if'
if (strscpy(p, q, p_size < q_size ? p_size : q_size)< 0)
^~include/linux/compiler.h:162:4: warning: '______f' is static but declared in inline function 'strcpy' which is not staticQuoted 4 lines
without this patch, and I could not come up with a nice fix for that. In combination with my patch to always enable '
CONFIG_COMPILE_TEST' during 'randconfig' builds, we can at least hide these warnings for most users.
Quoted 7 lines
This looks like it fixes the same issue that was already fixed and is in Linus's tree.
http://lkml.kernel.org/r/[email protected]
See commit
68e76e034b6b1("tracing: PreventPROFILE_ALL_BRANCHESwhen FORTIFY_SOURCE=y")
Ah, right. I missed that when I wrote the new changelog text for this old patch of mine. It also means I should rebase the patch so it applies on mainline, as I still want PROFILE_ALL_BRANCHES to be disabled in COMPILE_TEST kernels for the build speed aspect.
Greg, could you add the 68e76e034b6b1 commit to 4.14-stable and 4.15-stable in the meantime?
Arnd
Re: [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
Quoted 25 lines · Arnd Bergmann
Quoted 24 lines · Steven Rostedt
Quoted 11 lines · On Fri, 16 Feb 2018 22:41:11 +0100 Arnd Bergmann <[email protected]> wrote:
This can easily double the time for compiling a driver but does not provide any benefit for the compile tester, so it's better left disabled.
In addition, any 'inline' function that is not also 'static' and that contains an 'if' causes a warning like
include/linux/string.h:212:2: note: in expansion of macro 'if'
if (strscpy(p, q, p_size < q_size ? p_size : q_size)< 0)
^~include/linux/compiler.h:162:4: warning: '______f' is static but declared in inline function 'strcpy' which is not staticQuoted 4 lines
without this patch, and I could not come up with a nice fix for that. In combination with my patch to always enable '
CONFIG_COMPILE_TEST' during 'randconfig' builds, we can at least hide these warnings for most users.This looks like it fixes the same issue that was already fixed and is in Linus's tree.
Quoted 7 lines
Quoted 2 lines
See commit
68e76e034b6b1("tracing: PreventPROFILE_ALL_BRANCHESwhen FORTIFY_SOURCE=y")Ah, right. I missed that when I wrote the new changelog text for this old patch of mine. It also means I should rebase the patch so it applies on mainline, as I still want
PROFILE_ALL_BRANCHESto be disabled inCOMPILE_TESTkernels for the build speed aspect.
I retested on top of that patch and found a couple of other warnings show up in an allmodconfig build with PROFILE_ALL_BRANCHES:
lib/zstd/decompress.c: In function 'ZSTD_decompressStream':lib/zstd/decompress.c:416:2: error: argument 1 null where non-null expected [-Werror=nonnull] drivers/crypto/qat/qat_common/qat_algs.c: In function 'qat_alg_do_precomputes':drivers/crypto/qat/qat_common/qat_algs.c:156:7: error: argument 1 range [18446744071562067968, 18446744073709551615] exceeds maximum object size 9223372036854775807 [-Werror=alloc-size-larger-than=] drivers/isdn/hardware/eicon/message.c: In function 'mixer_notify_update':drivers/isdn/hardware/eicon/message.c:11162:54: error: array subscript is above array bounds [-Werror=array-bounds]
((CAPI_MSG *) msg)->info.facility_req.structs[1] =
LI_REQ_SILENT_UPDATE & 0xff;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/isdn/hardware/eicon/message.c:11163:54: error: array subscript
is above array bounds [-Werror=array-bounds]
((CAPI_MSG *) msg)->info.facility_req.structs[2] =
LI_REQ_SILENT_UPDATE >> 8;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/isdn/hardware/eicon/message.c:11164:54: error: array subscript
is above array bounds [-Werror=array-bounds]
((CAPI_MSG *) msg)->info.facility_req.structs[3] = 0;All those are nonsense AFAICT, and we see them only because the "if()" override ends up confusing gcc's value-range tracking in the same way it used to cause lots of -Wmaybe-uninitialized warnings (which we just disable these days with PROFILE_ALL_BRANCHES).
Arnd
Steven Rostedt
maintainerRe: [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
Quoted 12 lines · On Fri, 16 Feb 2018 23:40:22 +0100 Arnd Bergmann <[email protected]> wrote:
((CAPI_MSG *) msg)->info.facility_req.structs[1] =
LI_REQ_SILENT_UPDATE & 0xff;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/isdn/hardware/eicon/message.c:11163:54: error: array subscript
is above array bounds [-Werror=array-bounds]
((CAPI_MSG *) msg)->info.facility_req.structs[2] =
LI_REQ_SILENT_UPDATE >> 8;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/isdn/hardware/eicon/message.c:11164:54: error: array subscript
is above array bounds [-Werror=array-bounds]
((CAPI_MSG *) msg)->info.facility_req.structs[3] = 0;Quoted 4 lines
All those are nonsense AFAICT, and we see them only because the "
if()" override ends up confusing gcc's value-range tracking in the same way it used to cause lots of -Wmaybe-uninitialized warnings (which we just disable these days withPROFILE_ALL_BRANCHES).
I'm fine with your patch then.
-- Steve
Greg Kroah-Hartman
maintainerRe: [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
In reply to Arnd BergmannQuoted 28 lines · Arnd Bergmann
Quoted 19 lines · Steven Rostedt
Quoted 11 lines · On Fri, 16 Feb 2018 22:41:11 +0100 Arnd Bergmann <[email protected]> wrote:
This can easily double the time for compiling a driver but does not provide any benefit for the compile tester, so it's better left disabled.
In addition, any 'inline' function that is not also 'static' and that contains an 'if' causes a warning like
include/linux/string.h:212:2: note: in expansion of macro 'if'
if (strscpy(p, q, p_size < q_size ? p_size : q_size)< 0)
^~include/linux/compiler.h:162:4: warning: '______f' is static but declared in inline function 'strcpy' which is not staticQuoted 4 lines
without this patch, and I could not come up with a nice fix for that. In combination with my patch to always enable '
CONFIG_COMPILE_TEST' during 'randconfig' builds, we can at least hide these warnings for most users.Quoted 7 lines
This looks like it fixes the same issue that was already fixed and is in Linus's tree.
http://lkml.kernel.org/r/[email protected]
See commit
68e76e034b6b1("tracing: PreventPROFILE_ALL_BRANCHESwhen FORTIFY_SOURCE=y")
Quoted 7 lines
Ah, right. I missed that when I wrote the new changelog text for this old patch of mine. It also means I should rebase the patch so it applies on mainline, as I still want
PROFILE_ALL_BRANCHESto be disabled inCOMPILE_TESTkernels for the build speed aspect.Greg, could you add the
68e76e034b6b1commit to 4.14-stable and 4.15-stable in the meantime?
It's already in 4.15, so I'll just queue it up to 4.14.y.
thanks,
greg k-h
Re: [PATCH 1/3] Kconfig: disable PROFILE_ALL_BRANCHES for compile testing
In reply to Arnd Bergmann2018-02-17 6:41 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 11 lines
This can easily double the time for compiling a driver but does not provide any benefit for the compile tester, so it's better left disabled.
In addition, any 'inline' function that is not also 'static' and that contains an 'if' causes a warning like
include/linux/string.h:212:2: note: in expansion of macro 'if'
if (strscpy(p, q, p_size < q_size ? p_size : q_size)< 0)
^~include/linux/compiler.h:162:4: warning: '______f' is static but declared in inline function 'strcpy' which is not static
Quoted 7 lines
without this patch, and I could not come up with a nice fix for that. In combination with my patch to always enable '
CONFIG_COMPILE_TEST' during 'randconfig' builds, we can at least hide these warnings for most users.Cc Steven Rostedt <[email protected]>Signed-off-by Arnd Bergmann <[email protected]>
I took a look at this just in case
because the subject is prefixed with "Kconfig:",
but it is actually "trace:".
I expect "Kconfig:" for core changes of Kconfig.
Using precise patch prefix would get more attention from right people.
Quoted 23 lines
--- kernel/trace/Kconfig | 1 + 1 file changed, 1 insertion(+)kernel/trace/Kconfig +1
-1@@ -345,6 +345,7 @@ config PROFILE_ANNOTATED_BRANCHES config PROFILE_ALL_BRANCHES bool "Profile all if conditionals" select TRACE_BRANCH_PROFILING + depends on !COMPILE_TEST help This tracer profiles all branch conditions. Every if () taken in the kernel is recorded whether it hit or miss. --2.9.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 2 lines
Best Regards Masahiro Yamada
PATCH 2/3 Kconfig: improve handling for all{rand,yes,no,}.config fragments
In reply to Arnd BergmannThe kernel currently supports two methods of dealing with config fragments in the tree:
a) Running "make foo.config" looks for arch/$(ARCH)/configs/foo.config and kernel/configs/foo.config, and applies the defaults from those files on top of the current configuration.
b) Running "KCONFIG_ALLCONFIG=1 make randconfig" (or the equivalent allmodconfig/allnoconfig/allyesconfig/alldefconfig) will look for a "allrandconfig.config" file in the current directory or the top of the $(srctree). These are used as defaults before we generate the remaining options.
This is rather inconsistent, and prevents us from easily shipping good defaults for "randconfig". I'm extending the logic here so that the second case also looks for the hardcoded file names in the standard directories (first arch/$(ARCH)/configs/, then kernel/configs) in the source tree.
--- scripts/kconfig/conf.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-)
scripts/kconfig/conf.c +19 -8
@@ -493,8 +493,9 @@ int main(int ac, char **av) {const char *progname = av[0 ];int opt; -const char *name, *defconfig_file =NULL /* gcc uninit */ ; +const char *arch, *name, *defconfig_file =NULL /* gcc uninit */ ;struct stat tmpstat; +char fullname[PATH_MAX +1 ];setlocale (LC_ALL ,"" );bindtextdomain (PACKAGE ,LOCALEDIR );
@@ -621,14 +622,24 @@ int main(int ac, char **av)case randconfig: name ="allrandom.config" ;break ;default :break ; } -if (conf_read_simple (name,S_DEF_USER ) && -conf_read_simple ("all.config" ,S_DEF_USER )) { -fprintf (stderr, -_ ("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n" ), - name); -exit (1 ); +/* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */ +if (!conf_read_simple (name,S_DEF_USER )) +break ; + arch =getenv ("ARCH" ); +if (arch) { +snprintf (fullname,sizeof (fullname),"arch/%s/configs/%s" , + arch, name); +if (!conf_read_simple (fullname,S_DEF_USER )) +break ; } -break ; +snprintf (fullname,sizeof (fullname),"kernel/configs/%s" , name); +if (!conf_read_simple (fullname,S_DEF_USER )) +break ; + +fprintf (stderr, +_ ("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n" ), + name); +exit (1 );default :break ; }
Signature · 1 line
2.9.0
Re: [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments
2018-02-17 6:41 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 49 lines
The kernel currently supports two methods of dealing with config fragments in the tree:
a) Running "make foo.config" looks for arch/$(ARCH)/configs/foo.config and
kernel/configs/foo.config, and applies the defaults from those files on top of the current configuration.b) Running "KCONFIG_ALLCONFIG=1 make randconfig" (or the equivalent allmodconfig/allnoconfig/allyesconfig/alldefconfig) will look for a "allrandconfig.config" file in the current directory or the top of the $(srctree). These are used as defaults before we generate the remaining options.
This is rather inconsistent, and prevents us from easily shipping good defaults for "randconfig". I'm extending the logic here so that the second case also looks for the hardcoded file names in the standard directories (first arch/$(ARCH)/configs/, then kernel/configs) in the source tree.
Signed-off-by Arnd Bergmann <[email protected]>--- scripts/kconfig/conf.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-)scripts/kconfig/conf.c +2
-7@@ -493,8 +493,9 @@ int main(int ac, char **av) {const char *progname = av[0 ];int opt; -const char *name, *defconfig_file =NULL /* gcc uninit */ ; +const char *arch, *name, *defconfig_file =NULL /* gcc uninit */ ;struct stat tmpstat; +char fullname[PATH_MAX +1 ];setlocale (LC_ALL ,"" );bindtextdomain (PACKAGE ,LOCALEDIR );@@ -621,14 +622,24 @@ int main(int ac, char **av)case randconfig: name ="allrandom.config" ;break ;default :break ; } -if (conf_read_simple (name,S_DEF_USER ) && -conf_read_simple ("all.config" ,S_DEF_USER )) { -fprintf (stderr, -_ ("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n" ), - name); -exit (1 );
scripts/kconfig/conf.c int main(int ac, char **av) · line 634
Quoted 10 lines
scripts/kconfig/conf.c +9
-0+/* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */ +if (!conf_read_simple (name,S_DEF_USER )) +break ; + arch =getenv ("ARCH" ); +if (arch) { +snprintf (fullname,sizeof (fullname),"arch/%s/configs/%s" , + arch, name); +if (!conf_read_simple (fullname,S_DEF_USER )) +break ; }
I am not a big fan of hard-coding the kernel directory structure.
We already do this [1], but I am thinking of kicking this out.
[1] https://github.com/torvalds/linux/blob/v4.16-rc1/scripts/kconfig/confdata.c#L33
BTW, I am trying to compiler capability check to Kconfig. If this work is done, I hope some "depends on !COMPILE_TEST" will go away (but not all?)
https://patchwork.kernel.org/patch/10225375/
Quoted 19 lines
scripts/kconfig/conf.c +8
-2-break ; +snprintf (fullname,sizeof (fullname),"kernel/configs/%s" , name); +if (!conf_read_simple (fullname,S_DEF_USER )) +break ; + +fprintf (stderr, +_ ("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n" ), + name); +exit (1 );default :break ; } --2.9.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 2 lines
Best Regards Masahiro Yamada
Re: [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments
Quoted 26 lines · Masahiro Yamada
2018-02-17 6:41 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 10 lines
scripts/kconfig/conf.c +0
-6@@ -621,14 +622,24 @@ int main(int ac, char **av)case randconfig: name ="allrandom.config" ;break ;default :break ; } -if (conf_read_simple (name,S_DEF_USER ) && -conf_read_simple ("all.config" ,S_DEF_USER )) { -fprintf (stderr, -_ ("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n" ), - name); -exit (1 );scripts/kconfig/conf.c int main(int ac, char **av) · line 634
Quoted 10 lines
scripts/kconfig/conf.c +9
-0+/* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */ +if (!conf_read_simple (name,S_DEF_USER )) +break ; + arch =getenv ("ARCH" ); +if (arch) { +snprintf (fullname,sizeof (fullname),"arch/%s/configs/%s" , + arch, name); +if (!conf_read_simple (fullname,S_DEF_USER )) +break ; }I am not a big fan of hard-coding the kernel directory structure.
Quoted 2 lines
We already do this [1], but I am thinking of kicking this out.
[1] https://github.com/torvalds/linux/blob/v4.16-rc1/scripts/kconfig/confdata.c#L33
Ok, I see. How about adding a way to detect that we are build-testing with randconfig instead?
Quoted 4 lines
BTW, I am trying to compiler capability check to
Kconfig. If this work is done, I hope some "depends on !COMPILE_TEST" will go away (but not all?)
https://patchwork.kernel.org/patch/10225375/
I think most of the 'depends on !COMPILE_TEST' are for cases that impact compile-testing in some way (more false-positive warnings, fewer real warnings, much slower compile speed), rather than a lack of check for a specific compiler version.
We might be able to replace the CONFIG_STANDALONE and CONFIG_PREVENT_FIRMWARE_BUILD with Kconfig $(shell) checks if you think that leads to an improvement.
Arnd
Re: [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments
2018-02-20 18:59 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 28 lines
Quoted 26 lines · Masahiro Yamada
2018-02-17 6:41 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 10 lines
scripts/kconfig/conf.c +0
-6@@ -621,14 +622,24 @@ int main(int ac, char **av)case randconfig: name ="allrandom.config" ;break ;default :break ; } -if (conf_read_simple (name,S_DEF_USER ) && -conf_read_simple ("all.config" ,S_DEF_USER )) { -fprintf (stderr, -_ ("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n" ), - name); -exit (1 );scripts/kconfig/conf.c int main(int ac, char **av) · line 634
Quoted 10 lines
scripts/kconfig/conf.c +9
-0+/* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */ +if (!conf_read_simple (name,S_DEF_USER )) +break ; + arch =getenv ("ARCH" ); +if (arch) { +snprintf (fullname,sizeof (fullname),"arch/%s/configs/%s" , + arch, name); +if (!conf_read_simple (fullname,S_DEF_USER )) +break ; }I am not a big fan of hard-coding the kernel directory structure.
Quoted 5 lines
Quoted 2 lines
We already do this [1], but I am thinking of kicking this out.
[1] https://github.com/torvalds/linux/blob/v4.16-rc1/scripts/kconfig/confdata.c#L33Ok, I see. How about adding a way to detect that we are build-testing with randconfig instead?
How about implementing something in scripts/kconfig/Makefile?
merge_config collects config fragments into 'configfiles'
I was thinking of a similar thing.
If KCONFIG_ALLCONFIG is 1,scripts/kconfig/Makefile searches
'./', '$srctree/', 'arch/$(SRCARCH)/configs/', 'kernel/configs/' and sets the found file in it,
If KCONFIG_ALLCONFIG already contains a file name, it is used as-is.
scripts/kconfig/conf.c will be simplified.
Quoted 5 lines
Quoted 4 lines
BTW, I am trying to compiler capability check to
Kconfig. If this work is done, I hope some "depends on !COMPILE_TEST" will go away (but not all?)
https://patchwork.kernel.org/patch/10225375/
Quoted 9 lines
I think most of the 'depends on !COMPILE_TEST' are for cases that impact compile-testing in some way (more false-positive warnings, fewer real warnings, much slower compile speed), rather than a lack of check for a specific compiler version.
We might be able to replace the
CONFIG_STANDALONEandCONFIG_PREVENT_FIRMWARE_BUILDwithKconfig$(shell) checks if you think that leads to an improvement.
OK, then your kernel/configs/allrandom.config approach will be useful.
Signature · 2 lines
Best Regards Masahiro Yamada
Re: [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments
Quoted 43 lines · Masahiro Yamada
2018-02-20 18:59 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 28 lines
Quoted 26 lines · Masahiro Yamada
2018-02-17 6:41 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 10 lines
scripts/kconfig/conf.c +0
-6@@ -621,14 +622,24 @@ int main(int ac, char **av)case randconfig: name ="allrandom.config" ;break ;default :break ; } -if (conf_read_simple (name,S_DEF_USER ) && -conf_read_simple ("all.config" ,S_DEF_USER )) { -fprintf (stderr, -_ ("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n" ), - name); -exit (1 );scripts/kconfig/conf.c int main(int ac, char **av) · line 634
Quoted 10 lines
scripts/kconfig/conf.c +9
-0+/* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */ +if (!conf_read_simple (name,S_DEF_USER )) +break ; + arch =getenv ("ARCH" ); +if (arch) { +snprintf (fullname,sizeof (fullname),"arch/%s/configs/%s" , + arch, name); +if (!conf_read_simple (fullname,S_DEF_USER )) +break ; }I am not a big fan of hard-coding the kernel directory structure.
Quoted 5 lines
Quoted 2 lines
We already do this [1], but I am thinking of kicking this out.
[1] https://github.com/torvalds/linux/blob/v4.16-rc1/scripts/kconfig/confdata.c#L33Ok, I see. How about adding a way to detect that we are build-testing with randconfig instead?
How about implementing something in
scripts/kconfig/Makefile?
merge_configcollects config fragments into 'configfiles'I was thinking of a similar thing.
Quoted 8 lines
If
KCONFIG_ALLCONFIGis 1,scripts/kconfig/Makefilesearches
'./', '$srctree/', 'arch/$(SRCARCH)/configs/', 'kernel/configs/' and sets the found file in it,If
KCONFIG_ALLCONFIGalready contains a file name, it is used as-is.
scripts/kconfig/conf.cwill be simplified.
Yes, good idea. I'm struggling a bit with the implementation (fitting it into the
simple-tagets rule), can you come up with a prototype?
We also need to decide what happens when e.g. both a ./all.config and arch/${SRCARCH}/configs/allrand.config file exist, should the first path take precedence or the more specific file?
Arnd
Re: [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments
2018-02-22 1:57 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 54 lines
Quoted 43 lines · Masahiro Yamada
2018-02-20 18:59 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 28 lines
Quoted 26 lines · Masahiro Yamada
2018-02-17 6:41 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 10 lines
scripts/kconfig/conf.c +0
-6@@ -621,14 +622,24 @@ int main(int ac, char **av)case randconfig: name ="allrandom.config" ;break ;default :break ; } -if (conf_read_simple (name,S_DEF_USER ) && -conf_read_simple ("all.config" ,S_DEF_USER )) { -fprintf (stderr, -_ ("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n" ), - name); -exit (1 );scripts/kconfig/conf.c int main(int ac, char **av) · line 634
Quoted 10 lines
scripts/kconfig/conf.c +9
-0+/* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */ +if (!conf_read_simple (name,S_DEF_USER )) +break ; + arch =getenv ("ARCH" ); +if (arch) { +snprintf (fullname,sizeof (fullname),"arch/%s/configs/%s" , + arch, name); +if (!conf_read_simple (fullname,S_DEF_USER )) +break ; }I am not a big fan of hard-coding the kernel directory structure.
Quoted 5 lines
Quoted 2 lines
We already do this [1], but I am thinking of kicking this out.
[1] https://github.com/torvalds/linux/blob/v4.16-rc1/scripts/kconfig/confdata.c#L33Ok, I see. How about adding a way to detect that we are build-testing with randconfig instead?
How about implementing something in
scripts/kconfig/Makefile?
merge_configcollects config fragments into 'configfiles'I was thinking of a similar thing.
Quoted 8 lines
If
KCONFIG_ALLCONFIGis 1,scripts/kconfig/Makefilesearches
'./', '$srctree/', 'arch/$(SRCARCH)/configs/', 'kernel/configs/' and sets the found file in it,If
KCONFIG_ALLCONFIGalready contains a file name, it is used as-is.
scripts/kconfig/conf.cwill be simplified.
Quoted 8 lines
Yes, good idea. I'm struggling a bit with the implementation (fitting it into the
simple-tagets rule), can you come up with a prototype?We also need to decide what happens when e.g. both a ./all.config and
arch/${SRCARCH}/configs/allrand.configfile exist, should the first path take precedence or the more specific file?
Hmm, maybe the following order?
[1] ./allrandom.config
[2] $(srctree)/allrandom.config
[3] ./all.config
[4] $(srctree)/all.config
[5] kernel/configs/allrandom.config + arch/$(SRCARCH)/configs/allrandom.config
I think you want to include
both kernel/configs/allrandom.config and
arch/$(SRCARCH)/configs/allrandom.config
if exist.
Some options are common, some are arch-specific.
This is how tinyconfig works.
Signature · 2 lines
Best Regards Masahiro Yamada
Re: [PATCH 2/3] Kconfig: improve handling for all{rand,yes,no,}.config fragments
In reply to Arnd Bergmann2018-02-22 1:57 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 50 lines
Quoted 48 lines · Masahiro Yamada
2018-02-20 18:59 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 28 lines
Quoted 26 lines · Masahiro Yamada
2018-02-17 6:41 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 10 lines
scripts/kconfig/conf.c +0
-6@@ -621,14 +622,24 @@ int main(int ac, char **av)case randconfig: name ="allrandom.config" ;break ;default :break ; } -if (conf_read_simple (name,S_DEF_USER ) && -conf_read_simple ("all.config" ,S_DEF_USER )) { -fprintf (stderr, -_ ("*** KCONFIG_ALLCONFIG set, but no \"%s\" or \"all.config\" file found\n" ), - name); -exit (1 );scripts/kconfig/conf.c int main(int ac, char **av) · line 634
Quoted 10 lines
scripts/kconfig/conf.c +9
-0+/* try ./name, arch/$(ARCH)/configs/name and kernel/config/name */ +if (!conf_read_simple (name,S_DEF_USER )) +break ; + arch =getenv ("ARCH" ); +if (arch) { +snprintf (fullname,sizeof (fullname),"arch/%s/configs/%s" , + arch, name); +if (!conf_read_simple (fullname,S_DEF_USER )) +break ; }I am not a big fan of hard-coding the kernel directory structure.
Quoted 5 lines
Quoted 2 lines
We already do this [1], but I am thinking of kicking this out.
[1] https://github.com/torvalds/linux/blob/v4.16-rc1/scripts/kconfig/confdata.c#L33Ok, I see. How about adding a way to detect that we are build-testing with randconfig instead?
How about implementing something in
scripts/kconfig/Makefile?
merge_configcollects config fragments into 'configfiles'I was thinking of a similar thing.
If
KCONFIG_ALLCONFIGis 1,scripts/kconfig/Makefilesearches
'./', '$srctree/', 'arch/$(SRCARCH)/configs/', 'kernel/configs/' and sets the found file in it,
Quoted 7 lines
Quoted 3 lines
If
KCONFIG_ALLCONFIGalready contains a file name, it is used as-is.
scripts/kconfig/conf.cwill be simplified.Yes, good idea. I'm struggling a bit with the implementation (fitting it into the
simple-tagets rule), can you come up with a prototype?
Just a prototype for the Makefile part.
(We need to agree with the search order specification, though)
------------>8-------------configs_dirs := $(srctree)/kernel/configs/ $(srctree)/arch/$(SRCARCH)/configs/
allconfig_files = $(if $(filter 1, $(KCONFIG_ALLCONFIG)),$(firstword \
$(foreach f, $(1), $(wildcard $(addsuffix $(f), ./ $(srctree)/))) \
$(foreach f, $(1), $(subst $(space),$(comma),$(wildcard
$(addsuffix $(f), $(configs_dirs)))))), \
$(KCONFIG_ALLCONFIG))allyesconfig: override KCONFIG_ALLCONFIG := $(call allconfig_files, allyes.config all.config)
allmodconfig: override KCONFIG_ALLCONFIG := $(call allconfig_files, allmoyes.config all.config)
(likewise for allnoconfig, alldefconfig)
randconfig: override KCONFIG_ALLCONFIG := $(call allconfig_files, allrandom.config all.config)
------------->8-----------
If a user specifies a file path in KCONFIG_ALLCONFIG, it is used as-is.
IF KCONFIG_ALLCONFIG=1, the input file is searched in the following order. ( [1] - [4] is the same as the current specification)
[1] ./allrandom.config
[2] $(srctree)/allrandom.config
[3] ./all.config
[4] $(srctree)/all.config
[5] $(srctree)/kernel/configs/allrandom.config + $(srctree)/arch/$(SRCARCH)/configs/allrandom.config
[6] $(srctree)/kernel/configs/all.config + $(srctree)/arch/$(SRCARCH)/configs/all.config
For [5] and [6], if multiple files are found, KCONFIG_ALLCONFIG is set to comma-separated file paths.
scripts/kconfig/conf.c must be tweaked a little bit if we need to read multiple config fragments.
Currently, tinyconfig for x86 comes from common part and arch-specific part. ./kernel/configs/tiny.config./arch/x86/configs/tiny.config
What do you think?
Quoted 4 lines
We also need to decide what happens when e.g. both a ./all.config and
arch/${SRCARCH}/configs/allrand.configfile exist, should the first path take precedence or the more specific file?
Signature · 2 lines
Best Regards Masahiro Yamada
PATCH 3/3 Kconfig: add a default allrandom.config
In reply to Arnd BergmannBuilding randconfig kernels frequently leads to build errors from drivers that have additional build dependencies, or that we don't want to build for compile-testing for some other reason.
We already have a couple of compile-time options that can deal with this problem, but the complete set is not documented well. We also have a method to override some options during 'make randconfig' and a couple of other make targets. However, we don't yet combine those two.
This adds a new allrandom.config file to turn on CONFIG_COMPILE_TEST, CONFIG_STANDALONE and CONFIG_PREVENT_FIRMWARE_BUILD for all randconfig builds. This in turn disables some other options that we already try to avoid in 'allmodconfig' builds and that make no sense in a general randconfig build.
Building with 'make randconfig KCONFIG_ALLCONFIG=1' should now always succeed without warnings on x86 and arm64, aside from recent regressions. Other architectures probably need additional bugfixes.
--- kernel/configs/allrandom.config | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 kernel/configs/allrandom.config
kernel/configs/allrandom.config new file +6 -0
@@ -0,0 +1,6 @@ +# maximize search space, disable options not worth testing +CONFIG_COMPILE_TEST=y + +# reduce compile-time dependencies +CONFIG_STANDALONE=y +CONFIG_PREVENT_FIRMWARE_BUILD=y
Signature · 1 line
2.9.0
Re: [PATCH 3/3] Kconfig: add a default allrandom.config
2018-02-17 6:41 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 3 lines
Building randconfig kernels frequently leads to build errors from drivers that have additional build dependencies, or that we don't want to build for compile-testing for some other reason.
Can you fix the former case?
The latter seems OK.
Quoted 7 lines
We already have a couple of compile-time options that can deal with this problem, but the complete set is not documented well. We also have a method to override some options during 'make randconfig' and a couple of other make targets. However, we don't yet combine those two.
From this statement,
kernel/configs/compile_test.config
would make more sense?
Quoted 7 lines
This adds a new allrandom.config file to turn on
CONFIG_COMPILE_TEST,CONFIG_STANDALONEandCONFIG_PREVENT_FIRMWARE_BUILDfor all randconfig builds. This in turn disables some other options that we already try to avoid in 'allmodconfig' builds and that make no sense in a general randconfig build.Building with 'make randconfig KCONFIG_ALLCONFIG=1' should now always
Can you type 'make randconfig compile_test.config', instead?
(Or 'make allmodconfig compile_test.config' for full-build test)
The merge_config has a little bit different logic
from KCONFIG_ALLCONFIG=1, but I guess it leads to the almost same result.
Quoted 28 lines
succeed without warnings on x86 and arm64, aside from recent regressions. Other architectures probably need additional bugfixes.
Signed-off-by Arnd Bergmann <[email protected]>--- kernel/configs/allrandom.config | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 kernel/configs/allrandom.configkernel/configs/allrandom.config new file +6
-1@@ -0,0 +1,6 @@ +# maximize search space, disable options not worth testing +CONFIG_COMPILE_TEST=y + +# reduce compile-time dependencies +CONFIG_STANDALONE=y +CONFIG_PREVENT_FIRMWARE_BUILD=y --2.9.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 2 lines
Best Regards Masahiro Yamada
Re: [PATCH 3/3] Kconfig: add a default allrandom.config
Quoted 6 lines · Masahiro Yamada
2018-02-17 6:41 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 3 lines
Building randconfig kernels frequently leads to build errors from drivers that have additional build dependencies, or that we don't want to build for compile-testing for some other reason.
Can you fix the former case?
I'm sure it can be done with runtime detection of the tools, not sure if that's a good idea. What we would need here is
CONFIG_WANXL_BUILD_FIRMWARE: needs to check for as68k and ld68k CONFIG_AIC79XX_BUILD_FIRMWARE: needs flex, bison and berkeley db.
I have all those but it still failed to build for me.
STANDALONE now (after the removal of sound/oss) only controls
ACPI_CUSTOM_DSDT_FILE, we could remove STANDALONE now
I think, it doesn't actually get in the way of anything. For the stable kernels, I'd still need to deal with CONFIG_STANDALONE for the oss drivers.
My feeling right now is that we're better off making both STANDALONE and PREVENT_FIRMWARE_BUILD depend on !COMPILE_TEST,
and then find a way to force that.
Quoted 11 lines
Quoted 4 lines
We already have a couple of compile-time options that can deal with this problem, but the complete set is not documented well. We also have a method to override some options during 'make randconfig' and a couple of other make targets. However, we don't yet combine those two.
From this statement,
kernel/configs/compile_test.configwould make more sense?
My original thought was that the allrandom.config file name is already documented and I just wanted a more consistent behavior, not come up with something new.
Quoted 9 lines
Quoted 7 lines
This adds a new allrandom.config file to turn on
CONFIG_COMPILE_TEST,CONFIG_STANDALONEandCONFIG_PREVENT_FIRMWARE_BUILDfor all randconfig builds. This in turn disables some other options that we already try to avoid in 'allmodconfig' builds and that make no sense in a general randconfig build.Building with 'make randconfig KCONFIG_ALLCONFIG=1' should now always
Quoted 7 lines
Can you type 'make randconfig compile_test.config', instead?
(Or 'make allmodconfig compile_test.config' for full-build test)
The
merge_confighas a little bit different logic
from KCONFIG_ALLCONFIG=1, but I guess it leads to the almost same result.
I suspect that one difference in a randconfig kernel is that all options that depend on CONFIG_COMPILE_TEST have a 50% lower chance of getting enabled that way, but eventually they will get enabled.
Arnd
Re: [PATCH 3/3] Kconfig: add a default allrandom.config
2018-02-20 20:11 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 33 lines
Quoted 6 lines · Masahiro Yamada
2018-02-17 6:41 GMT+09:00 Arnd Bergmann <[email protected]>:
Quoted 3 lines
Building randconfig kernels frequently leads to build errors from drivers that have additional build dependencies, or that we don't want to build for compile-testing for some other reason.
Can you fix the former case?
I'm sure it can be done with runtime detection of the tools, not sure if that's a good idea. What we would need here is
CONFIG_WANXL_BUILD_FIRMWARE: needs to check for as68k and ld68kCONFIG_AIC79XX_BUILD_FIRMWARE: needs flex, bison and berkeley db.
I have all those but it still failed to build for me.
STANDALONE now (after the removal ofsound/oss) only controls
ACPI_CUSTOM_DSDT_FILE, we could remove STANDALONE now
I think, it doesn't actually get in the way of anything. For the stable kernels, I'd still need to deal withCONFIG_STANDALONEfor the oss drivers.My feeling right now is that we're better off making both STANDALONE and
PREVENT_FIRMWARE_BUILDdepend on !COMPILE_TEST,
and then find a way to force that.Quoted 8 lines
Quoted 4 lines
We already have a couple of compile-time options that can deal with this problem, but the complete set is not documented well. We also have a method to override some options during 'make randconfig' and a couple of other make targets. However, we don't yet combine those two.
From this statement,
Quoted 7 lines
Quoted 3 lines
kernel/configs/compile_test.configwould make more sense?
My original thought was that the allrandom.config file name is already documented and I just wanted a more consistent behavior, not come up with something new.
I thought this would be used by merge_config.sh, but I understood allrandom.config would be better.
Quoted 14 lines
Quoted 14 lines
Quoted 7 lines
This adds a new allrandom.config file to turn on
CONFIG_COMPILE_TEST,CONFIG_STANDALONEandCONFIG_PREVENT_FIRMWARE_BUILDfor all randconfig builds. This in turn disables some other options that we already try to avoid in 'allmodconfig' builds and that make no sense in a general randconfig build.Building with 'make randconfig KCONFIG_ALLCONFIG=1' should now always
Can you type 'make randconfig compile_test.config', instead?
(Or 'make allmodconfig compile_test.config' for full-build test)
Quoted 7 lines
Quoted 2 lines
The
merge_confighas a little bit different logic
from KCONFIG_ALLCONFIG=1, but I guess it leads to the almost same result.I suspect that one difference in a randconfig kernel is that all options that depend on
CONFIG_COMPILE_TESThave a 50% lower chance of getting enabled that way, but eventually they will get enabled.
You are right.
The behavior is slightly different.
Signature · 2 lines
Best Regards Masahiro Yamada