kconfig: reject malformed KCONFIG_PROBABILITY values
PATCH 1/1 kconfig: reject malformed KCONFIG_PROBABILITY values
randconfig checks the numeric range of each probability but does not validate where strtol() stops. For example, KCONFIG_PROBABILITY=50% is accepted as 50:0:0: the '%' is parsed twice as zero. This silently sets both tristate y/m probabilities to zero, reducing the coverage of random configuration builds. Empty fields and extra fields are also accepted.
Require each field to contain an integer followed by the end of the string or a colon introducing another field, with at most three fields. Preserve the leading whitespace and optional sign accepted by strtol(). Keep the strtol() result as long until the range check so that narrowing to int cannot turn an out-of-range value into a valid probability.
Add regression tests for malformed and out-of-range values, the supported probability formats, and the documented empty-value default.
---
Validation on kbuild-next, x86_64, GCC 13.3.0:
- make testconfig with HOSTCFLAGS=-Werror: 58 passed. With the original conf binary: 18 failed, 40 passed; all failures are new regression tests.
- ASan/UBSan at -O1, with leak detection disabled: the same 58 tests passed.
- Both builds passed 1,635 input cases and 440 comparisons with the original conf using valid inputs and fixed seeds. Malformed inputs preserved an existing .config, including when KCONFIG_ALLCONFIG was set.
- make defconfig, allnoconfig, allmodconfig and valid randconfig passed. KCONFIG_PROBABILITY=50% changed from success to the expected error.
- No vmlinux build or boot test; this changes the host configuration tool. A 32-bit host build was unavailable because multilib headers were missing.
An AI coding assistant helped find the issue, prepare the fix, description and tests, and run validation. The requested task was to find and fix a useful, reproducible Linux bug suitable for a first contribution.
scripts/kconfig/conf.c | 9 +++- .../tests/randconfig_probability/Kconfig | 12 +++++ .../tests/randconfig_probability/__init__.py | 54 +++++++++++++++++++ 3 files changed, 74 insertions(+), 1 deletion(-) create mode 100644 scripts/kconfig/tests/randconfig_probability/Kconfig create mode 100644 scripts/kconfig/tests/randconfig_probability/__init__.py
scripts/kconfig/conf.c +8 -1
@@ -191,7 +191,14 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode) n =0 ;while (env && *env) {char *endp; -int tmp =strtol (env, &endp,10 ); +long tmp =strtol (env, &endp,10 ); + +if (endp == env || (*endp && *endp !=':' ) || + (*endp ==':' && (!endp[1 ] || n ==2 ))) { + errno =EINVAL ; +perror ("KCONFIG_PROBABILITY" ); +exit (1 ); + }if (tmp >=0 && tmp <=100 ) { p[n++] = tmp;
scripts/kconfig/tests/randconfig_probability/Kconfig new file +12 -0
@@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config MODULES + bool + default y + modules + +config BOOL + bool "Bool" + +config TRI + tristate "Tristate"
scripts/kconfig/tests/randconfig_probability/__init__.py new file +54 -0
54 lines not on this page · raw message
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
Julian Braha
reviewerHi Dmitrii,
I see that this is your first patch submission. Welcome! And thank you for reporting this.
Quoted 5 lines · Dmitrii Tulnov
randconfig checks the numeric range of each probability but does not validate where
strtol()stops. For example, KCONFIG_PROBABILITY=50% is accepted as 50:0:0: the '%' is parsed twice as zero. This silently sets both tristate y/m probabilities to zero, reducing the coverage of random configuration builds.
It would be good to note here that 'KCONFIG_PROBABILITY=50' (without the '%') should be 25:25:50 instead.
Quoted 1 line
Empty fields and extra fields are also accepted.
Right, in these cases, I think it's reasonable to assume the user made a typo or other oversight.
Quoted 4 lines
Require each field to contain an integer followed by the end of the string or a colon introducing another field, with at most three fields. Preserve the leading whitespace and optional sign accepted by
strtol().
I disagree with allowing the leading whitespace and sign. The fact that it's allowed input for strtol() is incidental to us. The intended input format for Kconfig is:
XX:XX:XX
not:
XX: +XX: -XX
Quoted 39 lines
Keep the
strtol()result as long until the range check so that narrowing to int cannot turn an out-of-range value into a valid probability.Add regression tests for malformed and out-of-range values, the supported probability formats, and the documented empty-value default.
Fixes e43956e60769 ("kconfig: implement KCONFIG_PROBABILITY for randconfig")Assisted-by LLMSigned-off-by Dmitrii Tulnov <[email protected]>---
Validation on kbuild-next,x86_64, GCC 13.3.0:
- make testconfig with HOSTCFLAGS=-Werror: 58 passed. With the original conf binary: 18 failed, 40 passed; all failures are new regression tests.
- ASan/UBSan at -O1, with leak detection disabled: the same 58 tests passed.
- Both builds passed 1,635 input cases and 440 comparisons with the original conf using valid inputs and fixed seeds. Malformed inputs preserved an existing .config, including whenKCONFIG_ALLCONFIGwas set.
- make defconfig, allnoconfig, allmodconfig and valid randconfig passed. KCONFIG_PROBABILITY=50% changed from success to the expected error.
- No vmlinux build or boot test; this changes the host configuration tool. A 32-bit host build was unavailable because multilib headers were missing.An AI coding assistant helped find the issue, prepare the fix, description and tests, and run validation. The requested task was to find and fix a useful, reproducible Linux bug suitable for a first contribution.
scripts/kconfig/conf.c | 9 +++- .../tests/randconfig_probability/Kconfig | 12 +++++ .../tests/randconfig_probability/__init__.py | 54 +++++++++++++++++++ 3 files changed, 74 insertions(+), 1 deletion(-) create mode 100644 scripts/kconfig/tests/randconfig_probability/Kconfig create mode 100644 scripts/kconfig/tests/randconfig_probability/__init__.pyscripts/kconfig/conf.c +0
-0@@ -191,7 +191,14 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode) n =0 ;
scripts/kconfig/conf.c static void conf_set_all_new_symbols(enum conf_def_mode mode) · line 200
Quoted 10 lines
scripts/kconfig/conf.c +7
-1while (env && *env) {char *endp; -int tmp =strtol (env, &endp,10 ); +long tmp =strtol (env, &endp,10 ); + +if (endp == env || (*endp && *endp !=':' ) || + (*endp ==':' && (!endp[1 ] || n ==2 ))) { + errno =EINVAL ; +perror ("KCONFIG_PROBABILITY" ); +exit (1 );
Unfortunately, sometimes users depend on unintended behavior. So I would prefer to make this a warning for now, and then promote this to an error later assuming nobody complains.
- Julian Braha
Hi Julian,
Thank you for the review and for the welcome.
Quoted 2 lines
It would be good to note here that 'KCONFIG_PROBABILITY=50' (without the '%') should be 25:25:50 instead.
You're right. The v2 commit message now states explicitly that KCONFIG_PROBABILITY=50 gives tristate y/m/n probabilities of 25%/25%/50%. The regression test compares the equivalent inputs 50 and 50:25:25 across 20 fixed seeds.
Quoted 1 line
I disagree with allowing the leading whitespace and sign.
Agreed. Leading whitespace and signs are now warning-only because they are accepted by strtol() but outside Kconfig's documented XX:XX:XX format.
Quoted 2 lines
I would prefer to make this a warning for now, and then promote this to an error later assuming nobody complains.
The v2 implementation warns about malformed syntax while retaining the existing parsing behavior for compatibility. Numeric range violations remain errors.
Thank you again for taking the time to review this.
Best regards,
Dmitrii
PATCH v2 1/1 kconfig: warn about malformed KCONFIG_PROBABILITY values
In reply to Julian Braharandconfig checks the numeric range of each probability but does not validate where strtol() stops. For example, KCONFIG_PROBABILITY=50% is accepted as 50:0:0: the '%' is parsed repeatedly as zero. This silently sets both tristate y/m probabilities to zero, reducing the coverage of random configuration builds. Empty fields and extra fields are also accepted.
Warn when the value does not follow the documented decimal format. Keep the existing parsing behavior for now so that users depending on these inputs are not broken; the warning can be promoted to an error later. The documented one-field value 50 gives tristate y/m/n probabilities of 25%/25%/50%. Leading whitespace and signs are accepted by strtol(), but are also warned about because they are outside the documented format. Keep the strtol() result as long until the range check so that narrowing to int cannot turn an out-of-range value into a valid probability.
Add regression tests for malformed warnings and out-of-range values, the supported probability formats, and the documented empty-value default.
---
Changes since v1:
- Warn about malformed values while retaining their previous parsing behavior.
- Treat leading whitespace and signs as warning-only undocumented input.
- Clarify that 50 gives tristate y/m/n probabilities of 25%/25%/50%, and compare it with the equivalent input 50:25:25 across 20 fixed seeds.
Validation on kbuild-next, x86_64, GCC 13.3.0:
- make testconfig with HOSTCFLAGS=-Werror: 78 passed. Malformed values now produce warnings while preserving the previous parsing behavior; numeric range errors remain fatal.
- ASan/UBSan at -O1, with leak detection disabled: the same 78 tests passed.
- The fixed build accepted 916 of 1,635 generated inputs (including warning-only malformed values) and rejected 719 numeric range errors. It matched the original on 380 documented-input comparisons with fixed seeds. Range errors preserved an existing .config, including with KCONFIG_ALLCONFIG set.
- make defconfig, allnoconfig, allmodconfig and valid randconfig passed. KCONFIG_PROBABILITY=50% now produces a warning and remains compatible.
- No vmlinux build or boot test; this changes the host configuration tool. A 32-bit host build was unavailable because multilib headers were missing.
scripts/kconfig/conf.c | 14 +++- .../tests/randconfig_probability/Kconfig | 12 +++ .../tests/randconfig_probability/__init__.py | 79 +++++++++++++++++++ 3 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 scripts/kconfig/tests/randconfig_probability/Kconfig create mode 100644 scripts/kconfig/tests/randconfig_probability/__init__.py
scripts/kconfig/conf.c +13 -1
@@ -186,12 +186,24 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)if (mode == def_random) {int n, p[3 ]; +bool warned = false;char *env =getenv ("KCONFIG_PROBABILITY" ); n =0 ;while (env && *env) {char *endp; -int tmp =strtol (env, &endp,10 ); +long tmp =strtol (env, &endp,10 ); + +if (endp == env ||isspace ((unsigned char )*env) || + *env =='+' || *env =='-' || + (*endp && *endp !=':' ) || + (*endp ==':' && (!endp[1 ] || n ==2 ))) { +if (!warned) { +fprintf (stderr, +"warning: KCONFIG_PROBABILITY has malformed format\n" ); + warned = true; + } + }if (tmp >=0 && tmp <=100 ) { p[n++] = tmp;
scripts/kconfig/tests/randconfig_probability/Kconfig new file +12 -0
@@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config MODULES + bool + default y + modules + +config BOOL + bool "Bool" + +config TRI + tristate "Tristate"
scripts/kconfig/tests/randconfig_probability/__init__.py new file +79 -0
79 lines not on this page · raw message
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
Julian Braha
reviewerRe: [PATCH v2] kconfig: warn about malformed KCONFIG_PROBABILITY values
Hi Dmitrii,
Quoted 7 lines · Dmitrii Tulnov
randconfig checks the numeric range of each probability but does not validate where
strtol()stops. For example, KCONFIG_PROBABILITY=50% is accepted as 50:0:0: the '%' is parsed repeatedly as zero. This silently sets both tristate y/m probabilities to zero, reducing the coverage of random configuration builds. Empty fields and extra fields are also accepted.
Quoted 5 lines
Warn when the value does not follow the documented decimal format. Keep the existing parsing behavior for now so that users depending on these inputs are not broken; the warning can be promoted to an error later. The documented one-field value 50 gives tristate y/m/n probabilities of 25%/25%/50%. Leading whitespace and signs are accepted by
strtol(), but are
This "25:25:50" explanation belongs in the first paragraph where you give the background and state that "KCONFIG_PROBABILITY=50% is accepted as 50:0:0". Your second paragraph is about what you're changing.
Quoted 48 lines
also warned about because they are outside the documented format. Keep the
strtol()result as long until the range check so that narrowing to int cannot turn an out-of-range value into a valid probability.Add regression tests for malformed warnings and out-of-range values, the supported probability formats, and the documented empty-value default.
Fixes e43956e60769 ("kconfig: implement KCONFIG_PROBABILITY for randconfig")Assisted-by LLMSigned-off-by Dmitrii Tulnov <[email protected]>---
Changes since v1:
- Warn about malformed values while retaining their previous parsing behavior.
- Treat leading whitespace and signs as warning-only undocumented input.
- Clarify that 50 gives tristate y/m/n probabilities of 25%/25%/50%, and compare it with the equivalent input 50:25:25 across 20 fixed seeds.Validation on kbuild-next,
x86_64, GCC 13.3.0:
- make testconfig with HOSTCFLAGS=-Werror: 78 passed. Malformed values now produce warnings while preserving the previous parsing behavior; numeric range errors remain fatal.
- ASan/UBSan at -O1, with leak detection disabled: the same 78 tests passed.
- The fixed build accepted 916 of 1,635 generated inputs (including warning-only malformed values) and rejected 719 numeric range errors. It matched the original on 380 documented-input comparisons with fixed seeds. Range errors preserved an existing .config, including withKCONFIG_ALLCONFIGset.
- make defconfig, allnoconfig, allmodconfig and valid randconfig passed. KCONFIG_PROBABILITY=50% now produces a warning and remains compatible.
- No vmlinux build or boot test; this changes the host configuration tool. A 32-bit host build was unavailable because multilib headers were missing.scripts/kconfig/conf.c | 14 +++- .../tests/randconfig_probability/Kconfig | 12 +++ .../tests/randconfig_probability/__init__.py | 79 +++++++++++++++++++ 3 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 scripts/kconfig/tests/randconfig_probability/Kconfig create mode 100644 scripts/kconfig/tests/randconfig_probability/__init__.pyscripts/kconfig/conf.c +1
-0@@ -186,12 +186,24 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)if (mode == def_random) {int n, p[3 ]; +bool warned = false;
scripts/kconfig/conf.c static void conf_set_all_new_symbols(enum conf_def_mode mode) · line 198
Quoted 10 lines
scripts/kconfig/conf.c +4
-1char *env =getenv ("KCONFIG_PROBABILITY" ); n =0 ;while (env && *env) {char *endp; -int tmp =strtol (env, &endp,10 ); +long tmp =strtol (env, &endp,10 ); + + if (endp == env || isspace((unsigned char)*env) || + *env == '+' || *env == '-' ||
This condition expression is pretty long. You can simplify this:
endp == env || isspace((unsigned char)*env) || *env == '+' || *env == '-'
into:
!isdigit((unsigned char)*env)
Quoted 39 lines
scripts/kconfig/conf.c +8
-0+ (*endp && *endp != ':') || + (*endp == ':' && (!endp[1] || n == 2))) { +if (!warned) { +fprintf (stderr, +"warning: KCONFIG_PROBABILITY has malformed format\n" ); + warned = true; + } + }if (tmp >=0 && tmp <=100 ) { p[n++] = tmp;scripts/kconfig/tests/randconfig_probability/Kconfig new file +12
-0@@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config MODULES + bool + default y + modules + +config BOOL + bool "Bool" + +config TRI + tristate "Tristate"scripts/kconfig/tests/randconfig_probability/__init__.py new file +4
-0@@ -0,0 +1,79 @@ +# SPDX-License-Identifier: GPL-2.0-only +"""Validate KCONFIG_PROBABILITY without changing the supported distributions.""" + +import pytest
scripts/kconfig/tests/randconfig_probability/__init__.py line 14
Quoted 10 lines
scripts/kconfig/tests/randconfig_probability/__init__.py new file +10
-0+ + [email protected]('probability', [ + 'invalid', ' ', '50%', '50 ', '0x32', '10 20', '10:20x', + '10:20:invalid', '10:20:30x', + ':50', '50:', '10::20', '10:20:', '10:20:30:', '10:20:30:40', + '+100:0', ' \t100:0', '0: \t100:0', +]) +def test_malformed_warns(conf, monkeypatch, probability): + monkeypatch.setenv('KCONFIG_PROBABILITY', probability)
You should set the environment variables using 'conf._run_conf' instead of 'monkeypatch.setenv'.
For an example, see the test in:scripts/kconfig/tests/warn_changed_input/
and how it uses 'extra_env'.
scripts/kconfig/tests/randconfig_probability/__init__.py line 24
Quoted 10 lines
scripts/kconfig/tests/randconfig_probability/__init__.py new file +10
-0+ + assert conf.randconfig(seed=0) == 0 + assert 'warning: KCONFIG_PROBABILITY has malformed format' in conf.stderr + assert conf.config is not None + + [email protected]('probability', [ + '-1', '101', '0:101', '0:0:101', '60:41', '0:60:41', + '4294967296', '-4294967296', + '999999999999999999999999', '-999999999999999999999999',
It would be good to include '-0', which is the only negative input accepted before this patch.
Also you should include an instance with '+'.
- Julian Braha
Re: [PATCH v2] kconfig: warn about malformed KCONFIG_PROBABILITY values
Hi Julian,
Thanks for the review. I've addressed your comments locally for v3. Before sending it, I'd like to clarify how the new warning should interact with KCONFIG_WERROR.
With KCONFIG_PROBABILITY=50% and KCONFIG_WERROR=1, v2 and my current v3 still exit successfully and write .config. The warning is printed directly and does not go through the existing warning/error handling.
Would you prefer it to become an error when KCONFIG_WERROR is set, while remaining a warning otherwise, or to stay non-fatal even in that mode during the compatibility period? I'd lean toward honoring the explicit strict mode.
For the compatibility tests, I propose comparing the generated configurations for 50% versus 50:0:0 and -0 versus 0 with the same seed, and checking that the malformed input produces exactly one warning. That would put checks for the preserved interpretation in the submitted tests, in addition to the local comparisons I've run. Does that sound appropriate for this patch?
Best regards,
Dmitrii
Julian Braha
reviewerRe: [PATCH v2] kconfig: warn about malformed KCONFIG_PROBABILITY values
Quoted 6 lines · Dmitrii Tulnov
Hi Julian,
Thanks for the review. I've addressed your comments locally for v3. Before sending it, I'd like to clarify how the new warning should interact with
KCONFIG_WERROR.
Quoted 8 lines
With KCONFIG_PROBABILITY=50% and KCONFIG_WERROR=1, v2 and my current v3 still exit successfully and write .config. The warning is printed directly and does not go through the existing warning/error handling.
Would you prefer it to become an error when
KCONFIG_WERRORis set, while remaining a warning otherwise, or to stay non-fatal even in that mode during the compatibility period? I'd lean toward honoring the explicit strict mode.
Yes, better to support it. Thanks!
Quoted 8 lines
For the compatibility tests, I propose comparing the generated configurations for 50% versus 50:0:0 and -0 versus 0 with the same seed, and checking that the malformed input produces exactly one warning. That would put checks for the preserved interpretation in the submitted tests, in addition to the local comparisons I've run. Does that sound appropriate for this patch?
Yeah that's fine.
Quoted 2 lines
Best regards,
Dmitrii
PATCH v3 1/1 kconfig: warn about malformed KCONFIG_PROBABILITY values
randconfig checks the numeric range of each probability but does not validate where strtol() stops. For example, KCONFIG_PROBABILITY=50% is accepted as 50:0:0: the '%' is parsed repeatedly as zero. For the documented value 50, tristate y/m/n probabilities are 25%/25%/50%. With 50%, both y/m probabilities silently become zero, reducing the coverage of random configuration builds. Empty fields and extra fields are also accepted.
Warn when the value does not follow the documented decimal format. For malformed inputs whose parsed probabilities are in range, preserve the existing interpretation unless KCONFIG_WERROR is set. In that case, exit with an error before writing the configuration. Leading whitespace and signs are accepted by strtol(), but also trigger the warning because they are outside the documented format.
Keep the strtol() result as long until the range check. This rejects out-of-range values that narrowing to int previously made valid, such as 4294967296 becoming zero on a 64-bit host, regardless of KCONFIG_WERROR.
Add regression tests for one warning per malformed input, preserved configurations, KCONFIG_WERROR, out-of-range values, the supported probability formats, and the documented empty-value default.
---
Changes since v2:
- Move the expected tristate distribution into the problem description.
- Simplify the initial format check to !isdigit((unsigned char)*env).
- Pass probability values and seeds through conf._run_conf(extra_env=...). Use monkeypatch.delenv only to remove inherited variables.
- Add -0, +0, bare + and - warning cases, and the +101 range-error case.
- Honor KCONFIG_WERROR for malformed format warnings, as discussed with Julian. Check unset, empty, 0 and 1 flag behavior.
- Compare 50% with 50:0:0 and -0 with 0 across 20 fixed seeds and check that each malformed input produces exactly one warning.
- Clarify that compatibility applies to malformed in-range inputs with KCONFIG_WERROR unset; values previously accepted by narrowing now fail.
Changes since v1:
- Warn about malformed in-range values while retaining their previous interpretation by default.
- Warn about leading whitespace and signs as undocumented input.
- Clarify that 50 gives tristate y/m/n probabilities of 25%/25%/50%, and compare it with the equivalent input 50:25:25 across 20 fixed seeds.
Validation on kbuild-next, x86_64, GCC 13.3.0:
- make testconfig with HOSTCFLAGS=-Werror: 138 passed. With the unpatched
conf: 74 failed, 64 passed; failures cover missing format warnings, unchecked narrowing and ignored KCONFIG_WERROR. The previous v3 draft fails only the nine strict-mode regression cases; 129 tests pass.
- ASan/UBSan at -O1, with leak detection disabled: the same 138 tests passed.
- Both v3 builds matched the submitted v2 on 1,635 inputs: 916 accepted values and 719 range errors, with KCONFIG_WERROR unset. Each also matched the original on 380 documented-input comparisons with fixed seeds.
- Each build passed 120 strict format-error checks: no output created in a fresh directory; existing .config, .config.old and input files stayed unchanged, including with KCONFIG_ALLCONFIG and KCONFIG_OVERWRITECONFIG. Valid inputs, range diagnostics and other configuration modes were also checked with KCONFIG_WERROR set.
- The probability tests also passed with conflicting probability and seed values and KCONFIG_WERROR=1 inherited from the test runner's environment.
- make defconfig, allnoconfig, allmodconfig and valid randconfig passed. KCONFIG_PROBABILITY=50% now produces a warning and remains compatible.
- No vmlinux build or boot test; this changes the host configuration tool. 32-bit hosts and other libc implementations were not tested.
scripts/kconfig/conf.c | 16 +- .../tests/randconfig_probability/Kconfig | 12 ++ .../tests/randconfig_probability/__init__.py | 137 ++++++++++++++++++ 3 files changed, 164 insertions(+), 1 deletion(-) create mode 100644 scripts/kconfig/tests/randconfig_probability/Kconfig create mode 100644 scripts/kconfig/tests/randconfig_probability/__init__.py
scripts/kconfig/conf.c +15 -1
@@ -186,12 +186,23 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)if (mode == def_random) {int n, p[3 ]; +bool warned = false;char *env =getenv ("KCONFIG_PROBABILITY" ); n =0 ;while (env && *env) {char *endp; -int tmp =strtol (env, &endp,10 ); +long tmp =strtol (env, &endp,10 ); + +if (!isdigit ((unsigned char )*env) || + (*endp && *endp !=':' ) || + (*endp ==':' && (!endp[1 ] || n ==2 ))) { +if (!warned) { +fprintf (stderr, +"warning: KCONFIG_PROBABILITY has malformed format\n" ); + warned = true; + } + }if (tmp >=0 && tmp <=100 ) { p[n++] = tmp;
@@ -227,6 +238,9 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)perror ("KCONFIG_PROBABILITY" );exit (1 ); } + +if (warned &&getenv ("KCONFIG_WERROR" )) +exit (1 ); }menu_for_each_entry (menu) {
scripts/kconfig/tests/randconfig_probability/Kconfig new file +12 -0
@@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config MODULES + bool + default y + modules + +config BOOL + bool "Bool" + +config TRI + tristate "Tristate"
scripts/kconfig/tests/randconfig_probability/__init__.py new file +137 -0
137 lines not on this page · raw message
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
Julian Braha
reviewerRe: [PATCH v3] kconfig: warn about malformed KCONFIG_PROBABILITY values
Quoted 138 lines · Dmitrii Tulnov
randconfig checks the numeric range of each probability but does not validate where
strtol()stops. For example, KCONFIG_PROBABILITY=50% is accepted as 50:0:0: the '%' is parsed repeatedly as zero. For the documented value 50, tristate y/m/n probabilities are 25%/25%/50%. With 50%, both y/m probabilities silently become zero, reducing the coverage of random configuration builds. Empty fields and extra fields are also accepted.Warn when the value does not follow the documented decimal format. For malformed inputs whose parsed probabilities are in range, preserve the existing interpretation unless
KCONFIG_WERRORis set. In that case, exit with an error before writing the configuration. Leading whitespace and signs are accepted bystrtol(), but also trigger the warning because they are outside the documented format.Keep the
strtol()result as long until the range check. This rejects out-of-range values that narrowing to int previously made valid, such as 4294967296 becoming zero on a 64-bit host, regardless ofKCONFIG_WERROR.Add regression tests for one warning per malformed input, preserved configurations,
KCONFIG_WERROR, out-of-range values, the supported probability formats, and the documented empty-value default.Fixes e43956e60769 ("kconfig: implement KCONFIG_PROBABILITY for randconfig")Assisted-by LLMSigned-off-by Dmitrii Tulnov <[email protected]>---
Changes since v2:
- Move the expected tristate distribution into the problem description.
- Simplify the initial format check to !isdigit((unsigned char)*env).
- Pass probability values and seeds throughconf._run_conf(extra_env=...). Use monkeypatch.delenv only to remove inherited variables.
- Add -0, +0, bare + and - warning cases, and the +101 range-error case.
- HonorKCONFIG_WERRORfor malformed format warnings, as discussed with Julian. Check unset, empty, 0 and 1 flag behavior.
- Compare 50% with 50:0:0 and -0 with 0 across 20 fixed seeds and check that each malformed input produces exactly one warning.
- Clarify that compatibility applies to malformed in-range inputs withKCONFIG_WERRORunset; values previously accepted by narrowing now fail.Changes since v1:
- Warn about malformed in-range values while retaining their previous interpretation by default.
- Warn about leading whitespace and signs as undocumented input.
- Clarify that 50 gives tristate y/m/n probabilities of 25%/25%/50%, and compare it with the equivalent input 50:25:25 across 20 fixed seeds.Validation on kbuild-next,
x86_64, GCC 13.3.0:
- make testconfig with HOSTCFLAGS=-Werror: 138 passed. With the unpatched
conf: 74 failed, 64 passed; failures cover missing format warnings, unchecked narrowing and ignoredKCONFIG_WERROR. The previous v3 draft fails only the nine strict-mode regression cases; 129 tests pass.
- ASan/UBSan at -O1, with leak detection disabled: the same 138 tests passed.
- Both v3 builds matched the submitted v2 on 1,635 inputs: 916 accepted values and 719 range errors, withKCONFIG_WERRORunset. Each also matched the original on 380 documented-input comparisons with fixed seeds.
- Each build passed 120 strict format-error checks: no output created in a fresh directory; existing .config, .config.old and input files stayed unchanged, including withKCONFIG_ALLCONFIGandKCONFIG_OVERWRITECONFIG. Valid inputs, range diagnostics and other configuration modes were also checked withKCONFIG_WERRORset.
- The probability tests also passed with conflicting probability and seed values and KCONFIG_WERROR=1 inherited from the test runner's environment.
- make defconfig, allnoconfig, allmodconfig and valid randconfig passed. KCONFIG_PROBABILITY=50% now produces a warning and remains compatible.
- No vmlinux build or boot test; this changes the host configuration tool. 32-bit hosts and other libc implementations were not tested.scripts/kconfig/conf.c | 16 +- .../tests/randconfig_probability/Kconfig | 12 ++ .../tests/randconfig_probability/__init__.py | 137 ++++++++++++++++++ 3 files changed, 164 insertions(+), 1 deletion(-) create mode 100644 scripts/kconfig/tests/randconfig_probability/Kconfig create mode 100644 scripts/kconfig/tests/randconfig_probability/__init__.pyscripts/kconfig/conf.c +15
-1@@ -186,12 +186,23 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)if (mode == def_random) {int n, p[3 ]; +bool warned = false;char *env =getenv ("KCONFIG_PROBABILITY" ); n =0 ;while (env && *env) {char *endp; -int tmp =strtol (env, &endp,10 ); +long tmp =strtol (env, &endp,10 ); + +if (!isdigit ((unsigned char )*env) || + (*endp && *endp !=':' ) || + (*endp ==':' && (!endp[1 ] || n ==2 ))) { +if (!warned) { +fprintf (stderr, +"warning: KCONFIG_PROBABILITY has malformed format\n" ); + warned = true; + } + }if (tmp >=0 && tmp <=100 ) { p[n++] = tmp;@@ -227,6 +238,9 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)perror ("KCONFIG_PROBABILITY" );exit (1 ); } + +if (warned &&getenv ("KCONFIG_WERROR" )) +exit (1 ); }menu_for_each_entry (menu) {scripts/kconfig/tests/randconfig_probability/Kconfig new file +12
-0@@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config MODULES + bool + default y + modules + +config BOOL + bool "Bool" + +config TRI + tristate "Tristate"scripts/kconfig/tests/randconfig_probability/__init__.py new file +0
-0
scripts/kconfig/tests/randconfig_probability/__init__.py line 10
Quoted 10 lines
scripts/kconfig/tests/randconfig_probability/__init__.py new file +10
-0@@ -0,0 +1,137 @@ +# SPDX-License-Identifier: GPL-2.0-only +"""Validate KCONFIG_PROBABILITY without changing the supported distributions.""" + +import pytest + + [email protected](autouse=True) +def clear_werror(monkeypatch): + # extra_env can set strict mode, but cannot remove an inherited flag. + monkeypatch.delenv('KCONFIG_WERROR', raising=False)
Clearing this environment variable doesn't belong here, it should be part of scripts/kconfig/tests/conftest.py.
- Julian Braha
PATCH v4 1/1 kconfig: warn about malformed KCONFIG_PROBABILITY values
randconfig checks the numeric range of each probability but does not validate where strtol() stops. For example, KCONFIG_PROBABILITY=50% is accepted as 50:0:0: the '%' is parsed repeatedly as zero. For the documented value 50, tristate y/m/n probabilities are 25%/25%/50%. With 50%, both y/m probabilities silently become zero, reducing the coverage of random configuration builds. Empty fields and extra fields are also accepted.
Warn when the value does not follow the documented decimal format. For malformed inputs whose parsed probabilities are in range, preserve the existing interpretation unless KCONFIG_WERROR is set. In that case, exit with an error before writing the configuration. Leading whitespace and signs are accepted by strtol(), but also trigger the warning because they are outside the documented format.
Keep the strtol() result as long until the range check. This rejects out-of-range values that narrowing to int previously made valid, such as 4294967296 becoming zero on a 64-bit host, regardless of KCONFIG_WERROR.
Add regression tests for one warning per malformed input, preserved configurations, KCONFIG_WERROR, out-of-range values, the supported probability formats, and the documented empty-value default.
---
Thanks, Julian. I've moved the KCONFIG_WERROR cleanup fixture to scripts/kconfig/tests/conftest.py.
Changes since v3:
- Move the KCONFIG_WERROR cleanup fixture to the shared conftest.py, as requested by Julian. Explicit extra_env settings still enable WERROR.
Changes since v2:
- Move the expected tristate distribution into the problem description.
- Simplify the initial format check to !isdigit((unsigned char)*env).
- Pass probability values and seeds through conf._run_conf(extra_env=...). Use monkeypatch.delenv only to remove inherited variables.
- Add -0, +0, bare + and - warning cases, and the +101 range-error case.
- Honor KCONFIG_WERROR for malformed format warnings, as discussed with Julian. Check unset, empty, 0 and 1 flag behavior.
- Compare 50% with 50:0:0 and -0 with 0 across 20 fixed seeds and check that each malformed input produces exactly one warning.
- Clarify that compatibility applies to malformed in-range inputs with KCONFIG_WERROR unset; values previously accepted by narrowing now fail.
Changes since v1:
- Warn about malformed in-range values while retaining their previous interpretation by default.
- Warn about leading whitespace and signs as undocumented input.
- Clarify that 50 gives tristate y/m/n probabilities of 25%/25%/50%, and compare it with the equivalent input 50:25:25 across 20 fixed seeds.
Validation on kbuild-next, x86_64, GCC 13.3.0:
- make testconfig with HOSTCFLAGS=-Werror: 138 passed. The same 138 tests passed with KCONFIG_WERROR=1 inherited from the test runner, including the explicit strict-mode cases for empty, 0 and 1 flag values.
- The same two runs passed on the existing ASan/UBSan build at -O1, with leak detection disabled: 138 passed in each run.
- C code, test inputs and assertions are unchanged from the submitted v3. Its earlier checks remain applicable: 74 regression failures on the original; 1,635 comparisons with v2 and 380 documented-input comparisons with the original per build; 120 strict error file-preservation checks per build, including ALLCONFIG and OVERWRITECONFIG. These broader C checks were not rerun for the fixture move.
- No vmlinux build or boot test; this changes the host configuration tool. 32-bit hosts and other libc implementations were not tested.
scripts/kconfig/conf.c | 16 ++- scripts/kconfig/tests/conftest.py | 6 + .../tests/randconfig_probability/Kconfig | 12 ++ .../tests/randconfig_probability/__init__.py | 131 ++++++++++++++++++ 4 files changed, 164 insertions(+), 1 deletion(-) create mode 100644 scripts/kconfig/tests/randconfig_probability/Kconfig create mode 100644 scripts/kconfig/tests/randconfig_probability/__init__.py
scripts/kconfig/conf.c +15 -1
@@ -186,12 +186,23 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)if (mode == def_random) {int n, p[3 ]; +bool warned = false;char *env =getenv ("KCONFIG_PROBABILITY" ); n =0 ;while (env && *env) {char *endp; -int tmp =strtol (env, &endp,10 ); +long tmp =strtol (env, &endp,10 ); + +if (!isdigit ((unsigned char )*env) || + (*endp && *endp !=':' ) || + (*endp ==':' && (!endp[1 ] || n ==2 ))) { +if (!warned) { +fprintf (stderr, +"warning: KCONFIG_PROBABILITY has malformed format\n" ); + warned = true; + } + }if (tmp >=0 && tmp <=100 ) { p[n++] = tmp;
@@ -227,6 +238,9 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)perror ("KCONFIG_PROBABILITY" );exit (1 ); } + +if (warned &&getenv ("KCONFIG_WERROR" )) +exit (1 ); }menu_for_each_entry (menu) {
scripts/kconfig/tests/conftest.py +6 -0
@@ -312,6 +312,12 @@ class Conf: return self._matches('stderr', expected) [email protected](autouse=True) +def clear_werror(monkeypatch): + # extra_env can set strict mode, but cannot remove an inherited flag. + monkeypatch.delenv('KCONFIG_WERROR', raising=False) + + @pytest.fixture(scope="module") def conf(request): """Create a Conf instance and provide it to test functions."""
scripts/kconfig/tests/randconfig_probability/Kconfig new file +12 -0
@@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config MODULES + bool + default y + modules + +config BOOL + bool "Bool" + +config TRI + tristate "Tristate"
scripts/kconfig/tests/randconfig_probability/__init__.py new file +131 -0
131 lines not on this page · raw message
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
Julian Braha
reviewerRe: [PATCH v4] kconfig: warn about malformed KCONFIG_PROBABILITY values
Hi Dmitrii,
Quoted 219 lines · Dmitrii Tulnov
randconfig checks the numeric range of each probability but does not validate where
strtol()stops. For example, KCONFIG_PROBABILITY=50% is accepted as 50:0:0: the '%' is parsed repeatedly as zero. For the documented value 50, tristate y/m/n probabilities are 25%/25%/50%. With 50%, both y/m probabilities silently become zero, reducing the coverage of random configuration builds. Empty fields and extra fields are also accepted.Warn when the value does not follow the documented decimal format. For malformed inputs whose parsed probabilities are in range, preserve the existing interpretation unless
KCONFIG_WERRORis set. In that case, exit with an error before writing the configuration. Leading whitespace and signs are accepted bystrtol(), but also trigger the warning because they are outside the documented format.Keep the
strtol()result as long until the range check. This rejects out-of-range values that narrowing to int previously made valid, such as 4294967296 becoming zero on a 64-bit host, regardless ofKCONFIG_WERROR.Add regression tests for one warning per malformed input, preserved configurations,
KCONFIG_WERROR, out-of-range values, the supported probability formats, and the documented empty-value default.Fixes e43956e60769 ("kconfig: implement KCONFIG_PROBABILITY for randconfig")Assisted-by LLMSigned-off-by Dmitrii Tulnov <[email protected]>---
Thanks, Julian. I've moved theKCONFIG_WERRORcleanup fixture toscripts/kconfig/tests/conftest.py.Changes since v3:
- Move theKCONFIG_WERRORcleanup fixture to the sharedconftest.py, as requested by Julian. Explicitextra_envsettings still enable WERROR.Changes since v2:
- Move the expected tristate distribution into the problem description.
- Simplify the initial format check to !isdigit((unsigned char)*env).
- Pass probability values and seeds throughconf._run_conf(extra_env=...). Use monkeypatch.delenv only to remove inherited variables.
- Add -0, +0, bare + and - warning cases, and the +101 range-error case.
- HonorKCONFIG_WERRORfor malformed format warnings, as discussed with Julian. Check unset, empty, 0 and 1 flag behavior.
- Compare 50% with 50:0:0 and -0 with 0 across 20 fixed seeds and check that each malformed input produces exactly one warning.
- Clarify that compatibility applies to malformed in-range inputs withKCONFIG_WERRORunset; values previously accepted by narrowing now fail.Changes since v1:
- Warn about malformed in-range values while retaining their previous interpretation by default.
- Warn about leading whitespace and signs as undocumented input.
- Clarify that 50 gives tristate y/m/n probabilities of 25%/25%/50%, and compare it with the equivalent input 50:25:25 across 20 fixed seeds.Validation on kbuild-next,
x86_64, GCC 13.3.0:
- make testconfig with HOSTCFLAGS=-Werror: 138 passed. The same 138 tests passed with KCONFIG_WERROR=1 inherited from the test runner, including the explicit strict-mode cases for empty, 0 and 1 flag values.
- The same two runs passed on the existing ASan/UBSan build at -O1, with leak detection disabled: 138 passed in each run.
- C code, test inputs and assertions are unchanged from the submitted v3. Its earlier checks remain applicable: 74 regression failures on the original; 1,635 comparisons with v2 and 380 documented-input comparisons with the original per build; 120 strict error file-preservation checks per build, including ALLCONFIG and OVERWRITECONFIG. These broader C checks were not rerun for the fixture move.
- No vmlinux build or boot test; this changes the host configuration tool. 32-bit hosts and other libc implementations were not tested.scripts/kconfig/conf.c | 16 ++- scripts/kconfig/tests/conftest.py | 6 + .../tests/randconfig_probability/Kconfig | 12 ++ .../tests/randconfig_probability/__init__.py | 131 ++++++++++++++++++ 4 files changed, 164 insertions(+), 1 deletion(-) create mode 100644 scripts/kconfig/tests/randconfig_probability/Kconfig create mode 100644 scripts/kconfig/tests/randconfig_probability/__init__.pyscripts/kconfig/conf.c +15
-1@@ -186,12 +186,23 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)if (mode == def_random) {int n, p[3 ]; +bool warned = false;char *env =getenv ("KCONFIG_PROBABILITY" ); n =0 ;while (env && *env) {char *endp; -int tmp =strtol (env, &endp,10 ); +long tmp =strtol (env, &endp,10 ); + +if (!isdigit ((unsigned char )*env) || + (*endp && *endp !=':' ) || + (*endp ==':' && (!endp[1 ] || n ==2 ))) { +if (!warned) { +fprintf (stderr, +"warning: KCONFIG_PROBABILITY has malformed format\n" ); + warned = true; + } + }if (tmp >=0 && tmp <=100 ) { p[n++] = tmp;@@ -227,6 +238,9 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)perror ("KCONFIG_PROBABILITY" );exit (1 ); } + +if (warned &&getenv ("KCONFIG_WERROR" )) +exit (1 ); }menu_for_each_entry (menu) {scripts/kconfig/tests/conftest.py +6
-0@@ -312,6 +312,12 @@ class Conf: return self._matches('stderr', expected) [email protected](autouse=True) +def clear_werror(monkeypatch): + # extra_env can set strict mode, but cannot remove an inherited flag. + monkeypatch.delenv('KCONFIG_WERROR', raising=False) + + @pytest.fixture(scope="module") def conf(request): """Create a Conf instance and provide it to test functions."""scripts/kconfig/tests/randconfig_probability/Kconfig new file +12
-0@@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config MODULES + bool + default y + modules + +config BOOL + bool "Bool" + +config TRI + tristate "Tristate"scripts/kconfig/tests/randconfig_probability/__init__.py new file +62
-0@@ -0,0 +1,131 @@ +# SPDX-License-Identifier: GPL-2.0-only +"""Validate KCONFIG_PROBABILITY without changing the supported distributions.""" + +import pytest + + [email protected]('probability', [ + 'invalid', ' ', '50%', '50 ', '0x32', '10 20', '10:20x', + '10:20:invalid', '10:20:30x', + ':50', '50:', '10::20', '10:20:', '10:20:30:', '10:20:30:40', + '-0', '+0', '+', '-', '+100:0', ' \t100:0', '0: \t100:0', +]) +def test_malformed_warns(conf, probability): + assert conf._run_conf('--randconfig', extra_env={ + 'KCONFIG_PROBABILITY': probability, + 'KCONFIG_SEED': '0', + }) == 0 + assert 'warning: KCONFIG_PROBABILITY has malformed format' in conf.stderr + assert conf.stderr.count('warning:') == 1 + assert conf.config is not None + + [email protected]('probability, equivalent', [ + ('50%', '50:0:0'), + ('-0', '0'), +]) [email protected]('seed', range(20)) +def test_malformed_preserves_config(conf, probability, equivalent, seed): + assert conf._run_conf('--randconfig', extra_env={ + 'KCONFIG_PROBABILITY': probability, + 'KCONFIG_SEED': hex(seed), + }) == 0 + assert 'warning: KCONFIG_PROBABILITY has malformed format' in conf.stderr + assert conf.stderr.count('warning:') == 1 + malformed = conf.config + + assert conf._run_conf('--randconfig', extra_env={ + 'KCONFIG_PROBABILITY': equivalent, + 'KCONFIG_SEED': hex(seed), + }) == 0 + assert 'warning:' not in conf.stderr + assert conf.config == malformed + + [email protected]('werror', ['', '0', '1']) [email protected]('probability, status', [ + ('', 0), ('50', 0), ('50%', 1), ('-0', 1), ('10:20:30:40', 1), +]) +def test_werror(conf, probability, status, werror): + assert conf._run_conf('--randconfig', extra_env={ + 'KCONFIG_PROBABILITY': probability, + 'KCONFIG_SEED': '0', + 'KCONFIG_WERROR': werror, + }) == status + if status: + assert 'warning: KCONFIG_PROBABILITY has malformed format' in conf.stderr + assert conf.stderr.count('warning:') == 1 + else: + assert 'warning:' not in conf.stderr + + [email protected]('probability', [
scripts/kconfig/tests/randconfig_probability/__init__.py line 72
Quoted 10 lines
scripts/kconfig/tests/randconfig_probability/__init__.py new file +10
-0+ '-1', '101', '+101', '0:101', '0:0:101', '60:41', '0:60:41', + '4294967296', '-4294967296', + '999999999999999999999999', '-999999999999999999999999', +]) +def test_out_of_range(conf, probability): + assert conf._run_conf('--randconfig', extra_env={ + 'KCONFIG_PROBABILITY': probability, + 'KCONFIG_SEED': '0', + }) == 1 + assert 'KCONFIG_PROBABILITY:' in conf.stderr
So I've finally had some time to test this, and I found that all of these pytest.mark.parametrize() calls are causing the number of test results to skyrocket. As in, this one test would actually make up the absolute majority of results.
Could you remove them (and then the 'import pytest') and simply loop over the values in a list? For example, something like this:
def test_out_of_range(conf):
probabilities = [
'-1', '101', '+101', '0:101', '0:0:101', '60:41', '0:60:41',
'4294967296', '-4294967296',
'999999999999999999999999', '-999999999999999999999999',
]
for probability in probabilities:
assert conf._run_conf('--randconfig', extra_env={
'KCONFIG_PROBABILITY': probability,
'KCONFIG_SEED': '0',
}) == 1, repr(probability)
assert 'KCONFIG_PROBABILITY:' in conf.stderr, repr(probability)- Julian Braha
PATCH v5 1/1 kconfig: warn about malformed KCONFIG_PROBABILITY values
randconfig checks the numeric range of each probability but does not validate where strtol() stops. For example, KCONFIG_PROBABILITY=50% is accepted as 50:0:0: the '%' is parsed repeatedly as zero. For the documented value 50, tristate y/m/n probabilities are 25%/25%/50%. With 50%, both y/m probabilities silently become zero, reducing the coverage of random configuration builds. Empty fields and extra fields are also accepted.
Warn when the value does not follow the documented decimal format. For malformed inputs whose parsed probabilities are in range, preserve the existing interpretation unless KCONFIG_WERROR is set. In that case, exit with an error before writing the configuration. Leading whitespace and signs are accepted by strtol(), but also trigger the warning because they are outside the documented format.
Keep the strtol() result as long until the range check. This rejects out-of-range values that narrowing to int previously made valid, such as 4294967296 becoming zero on a 64-bit host, regardless of KCONFIG_WERROR.
Add regression tests for one warning per malformed input, preserved configurations, KCONFIG_WERROR, out-of-range values, the supported probability formats, and the documented empty-value default.
---
Thanks, Julian. I've replaced the parametrization with loops and removed the unused pytest import. All input combinations and checks are preserved. Assertion messages identify the failing input and, where applicable, the seed and KCONFIG_WERROR value. Each test function now stops at its first failing case.
Changes since v4:
- Replace pytest.mark.parametrize with loops and remove the unused pytest import from the probability test module, as requested by Julian.
- Include the probability and, where applicable, seed and WERROR value in assertion messages. Preserve all input combinations and checks.
Changes since v3:
- Move the KCONFIG_WERROR cleanup fixture to the shared conftest.py, as requested by Julian. Explicit extra_env settings still enable WERROR.
Changes since v2:
- Move the expected tristate distribution into the problem description.
- Simplify the initial format check to !isdigit((unsigned char)*env).
- Pass probability values and seeds through conf._run_conf(extra_env=...). Use monkeypatch.delenv only to remove inherited variables.
- Add -0, +0, bare + and - warning cases, and the +101 range-error case.
- Honor KCONFIG_WERROR for malformed format warnings, as discussed with Julian. Check unset, empty, 0 and 1 flag behavior.
- Compare 50% with 50:0:0 and -0 with 0 across 20 fixed seeds and check that each malformed input produces exactly one warning.
- Clarify that compatibility applies to malformed in-range inputs with KCONFIG_WERROR unset; values previously accepted by narrowing now fail.
Changes since v1:
- Warn about malformed in-range values while retaining their previous interpretation by default.
- Warn about leading whitespace and signs as undocumented input.
- Clarify that 50 gives tristate y/m/n probabilities of 25%/25%/50%, and compare it with the equivalent input 50:25:25 across 20 fixed seeds.
Validation on kbuild-next, x86_64, GCC 13.3.0:
- make testconfig with HOSTCFLAGS=-Werror: 28 passed, both normally and with KCONFIG_WERROR=1 inherited from the test runner. Explicit strict cases for empty, 0 and 1 flag values still pass.
- The same suite passed on the existing ASan/UBSan build at -O1, with leak detection disabled: 28 passed in each environment.
- The probability module now reports 7 tests instead of 117. Tracing confirms that all 178 actual conf calls match v4 in order, inputs, exit status, stdout, stderr and generated configuration.
- The new suite gives 4 expected failures and 24 passes on the original binary; the version before WERROR gives 1 expected failure and 27 passes. Loops stop at their first failure, so these counts differ from the earlier parametrized regression results.
- C code, test Kconfig and shared fixtures are unchanged from v4. Earlier C compatibility and file-preservation checks were not rerun.
- No vmlinux build or boot test; this changes the host configuration tool. 32-bit hosts and other libc implementations were not tested.
scripts/kconfig/conf.c | 16 +- scripts/kconfig/tests/conftest.py | 6 + .../tests/randconfig_probability/Kconfig | 12 ++ .../tests/randconfig_probability/__init__.py | 139 ++++++++++++++++++ 4 files changed, 172 insertions(+), 1 deletion(-) create mode 100644 scripts/kconfig/tests/randconfig_probability/Kconfig create mode 100644 scripts/kconfig/tests/randconfig_probability/__init__.py
scripts/kconfig/conf.c +15 -1
@@ -186,12 +186,23 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)if (mode == def_random) {int n, p[3 ]; +bool warned = false;char *env =getenv ("KCONFIG_PROBABILITY" ); n =0 ;while (env && *env) {char *endp; -int tmp =strtol (env, &endp,10 ); +long tmp =strtol (env, &endp,10 ); + +if (!isdigit ((unsigned char )*env) || + (*endp && *endp !=':' ) || + (*endp ==':' && (!endp[1 ] || n ==2 ))) { +if (!warned) { +fprintf (stderr, +"warning: KCONFIG_PROBABILITY has malformed format\n" ); + warned = true; + } + }if (tmp >=0 && tmp <=100 ) { p[n++] = tmp;
@@ -227,6 +238,9 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)perror ("KCONFIG_PROBABILITY" );exit (1 ); } + +if (warned &&getenv ("KCONFIG_WERROR" )) +exit (1 ); }menu_for_each_entry (menu) {
scripts/kconfig/tests/conftest.py +6 -0
@@ -312,6 +312,12 @@ class Conf: return self._matches('stderr', expected) [email protected](autouse=True) +def clear_werror(monkeypatch): + # extra_env can set strict mode, but cannot remove an inherited flag. + monkeypatch.delenv('KCONFIG_WERROR', raising=False) + + @pytest.fixture(scope="module") def conf(request): """Create a Conf instance and provide it to test functions."""
scripts/kconfig/tests/randconfig_probability/Kconfig new file +12 -0
@@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config MODULES + bool + default y + modules + +config BOOL + bool "Bool" + +config TRI + tristate "Tristate"
scripts/kconfig/tests/randconfig_probability/__init__.py new file +139 -0
139 lines not on this page · raw message
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
Julian Braha
reviewerRe: [PATCH v5] kconfig: warn about malformed KCONFIG_PROBABILITY values
Quoted 286 lines · Dmitrii Tulnov
randconfig checks the numeric range of each probability but does not validate where
strtol()stops. For example, KCONFIG_PROBABILITY=50% is accepted as 50:0:0: the '%' is parsed repeatedly as zero. For the documented value 50, tristate y/m/n probabilities are 25%/25%/50%. With 50%, both y/m probabilities silently become zero, reducing the coverage of random configuration builds. Empty fields and extra fields are also accepted.Warn when the value does not follow the documented decimal format. For malformed inputs whose parsed probabilities are in range, preserve the existing interpretation unless
KCONFIG_WERRORis set. In that case, exit with an error before writing the configuration. Leading whitespace and signs are accepted bystrtol(), but also trigger the warning because they are outside the documented format.Keep the
strtol()result as long until the range check. This rejects out-of-range values that narrowing to int previously made valid, such as 4294967296 becoming zero on a 64-bit host, regardless ofKCONFIG_WERROR.Add regression tests for one warning per malformed input, preserved configurations,
KCONFIG_WERROR, out-of-range values, the supported probability formats, and the documented empty-value default.Fixes e43956e60769 ("kconfig: implement KCONFIG_PROBABILITY for randconfig")Assisted-by LLMSigned-off-by Dmitrii Tulnov <[email protected]>---
Thanks, Julian. I've replaced the parametrization with loops and removed the unused pytest import. All input combinations and checks are preserved. Assertion messages identify the failing input and, where applicable, the seed andKCONFIG_WERRORvalue. Each test function now stops at its first failing case.Changes since v4:
- Replace pytest.mark.parametrize with loops and remove the unused pytest import from the probability test module, as requested by Julian.
- Include the probability and, where applicable, seed and WERROR value in assertion messages. Preserve all input combinations and checks.Changes since v3:
- Move theKCONFIG_WERRORcleanup fixture to the sharedconftest.py, as requested by Julian. Explicitextra_envsettings still enable WERROR.Changes since v2:
- Move the expected tristate distribution into the problem description.
- Simplify the initial format check to !isdigit((unsigned char)*env).
- Pass probability values and seeds throughconf._run_conf(extra_env=...). Use monkeypatch.delenv only to remove inherited variables.
- Add -0, +0, bare + and - warning cases, and the +101 range-error case.
- HonorKCONFIG_WERRORfor malformed format warnings, as discussed with Julian. Check unset, empty, 0 and 1 flag behavior.
- Compare 50% with 50:0:0 and -0 with 0 across 20 fixed seeds and check that each malformed input produces exactly one warning.
- Clarify that compatibility applies to malformed in-range inputs withKCONFIG_WERRORunset; values previously accepted by narrowing now fail.Changes since v1:
- Warn about malformed in-range values while retaining their previous interpretation by default.
- Warn about leading whitespace and signs as undocumented input.
- Clarify that 50 gives tristate y/m/n probabilities of 25%/25%/50%, and compare it with the equivalent input 50:25:25 across 20 fixed seeds.Validation on kbuild-next,
x86_64, GCC 13.3.0:
- make testconfig with HOSTCFLAGS=-Werror: 28 passed, both normally and with KCONFIG_WERROR=1 inherited from the test runner. Explicit strict cases for empty, 0 and 1 flag values still pass.
- The same suite passed on the existing ASan/UBSan build at -O1, with leak detection disabled: 28 passed in each environment.
- The probability module now reports 7 tests instead of 117. Tracing confirms that all 178 actual conf calls match v4 in order, inputs, exit status, stdout, stderr and generated configuration.
- The new suite gives 4 expected failures and 24 passes on the original binary; the version before WERROR gives 1 expected failure and 27 passes. Loops stop at their first failure, so these counts differ from the earlier parametrized regression results.
- C code, testKconfigand shared fixtures are unchanged from v4. Earlier C compatibility and file-preservation checks were not rerun.
- No vmlinux build or boot test; this changes the host configuration tool. 32-bit hosts and other libc implementations were not tested.scripts/kconfig/conf.c | 16 +- scripts/kconfig/tests/conftest.py | 6 + .../tests/randconfig_probability/Kconfig | 12 ++ .../tests/randconfig_probability/__init__.py | 139 ++++++++++++++++++ 4 files changed, 172 insertions(+), 1 deletion(-) create mode 100644 scripts/kconfig/tests/randconfig_probability/Kconfig create mode 100644 scripts/kconfig/tests/randconfig_probability/__init__.pyscripts/kconfig/conf.c +15
-1@@ -186,12 +186,23 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)if (mode == def_random) {int n, p[3 ]; +bool warned = false;char *env =getenv ("KCONFIG_PROBABILITY" ); n =0 ;while (env && *env) {char *endp; -int tmp =strtol (env, &endp,10 ); +long tmp =strtol (env, &endp,10 ); + +if (!isdigit ((unsigned char )*env) || + (*endp && *endp !=':' ) || + (*endp ==':' && (!endp[1 ] || n ==2 ))) { +if (!warned) { +fprintf (stderr, +"warning: KCONFIG_PROBABILITY has malformed format\n" ); + warned = true; + } + }if (tmp >=0 && tmp <=100 ) { p[n++] = tmp;@@ -227,6 +238,9 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)perror ("KCONFIG_PROBABILITY" );exit (1 ); } + +if (warned &&getenv ("KCONFIG_WERROR" )) +exit (1 ); }menu_for_each_entry (menu) {scripts/kconfig/tests/conftest.py +6
-0@@ -312,6 +312,12 @@ class Conf: return self._matches('stderr', expected) [email protected](autouse=True) +def clear_werror(monkeypatch): + # extra_env can set strict mode, but cannot remove an inherited flag. + monkeypatch.delenv('KCONFIG_WERROR', raising=False) + + @pytest.fixture(scope="module") def conf(request): """Create a Conf instance and provide it to test functions."""scripts/kconfig/tests/randconfig_probability/Kconfig new file +12
-0@@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config MODULES + bool + default y + modules + +config BOOL + bool "Bool" + +config TRI + tristate "Tristate"scripts/kconfig/tests/randconfig_probability/__init__.py new file +117
-0@@ -0,0 +1,139 @@ +# SPDX-License-Identifier: GPL-2.0-only +"""Validate KCONFIG_PROBABILITY without changing the supported distributions.""" + + +def test_malformed_warns(conf): + probabilities = [ + 'invalid', ' ', '50%', '50 ', '0x32', '10 20', '10:20x', + '10:20:invalid', '10:20:30x', + ':50', '50:', '10::20', '10:20:', '10:20:30:', '10:20:30:40', + '-0', '+0', '+', '-', '+100:0', ' \t100:0', '0: \t100:0', + ] + for probability in probabilities: + assert conf._run_conf('--randconfig', extra_env={ + 'KCONFIG_PROBABILITY': probability, + 'KCONFIG_SEED': '0', + }) == 0, repr(probability) + assert ('warning: KCONFIG_PROBABILITY has malformed format' in + conf.stderr), repr(probability) + assert conf.stderr.count('warning:') == 1, repr(probability) + assert conf.config is not None, repr(probability) + + +def test_malformed_preserves_config(conf): + probabilities = [('50%', '50:0:0'), ('-0', '0')] + for seed in range(20): + for probability, equivalent in probabilities: + context = 'probability={!r}, equivalent={!r}, seed={}'.format( + probability, equivalent, seed) + assert conf._run_conf('--randconfig', extra_env={ + 'KCONFIG_PROBABILITY': probability, + 'KCONFIG_SEED': hex(seed), + }) == 0, context + assert ('warning: KCONFIG_PROBABILITY has malformed format' in + conf.stderr), context + assert conf.stderr.count('warning:') == 1, context + malformed = conf.config + + assert conf._run_conf('--randconfig', extra_env={ + 'KCONFIG_PROBABILITY': equivalent, + 'KCONFIG_SEED': hex(seed), + }) == 0, context + assert 'warning:' not in conf.stderr, context + assert conf.config == malformed, context + + +def test_werror(conf): + probabilities = [ + ('', 0), ('50', 0), ('50%', 1), ('-0', 1), ('10:20:30:40', 1), + ] + for probability, status in probabilities: + for werror in ['', '0', '1']: + context = 'probability={!r}, werror={!r}'.format( + probability, werror) + assert conf._run_conf('--randconfig', extra_env={ + 'KCONFIG_PROBABILITY': probability, + 'KCONFIG_SEED': '0', + 'KCONFIG_WERROR': werror, + }) == status, context + if status: + assert ('warning: KCONFIG_PROBABILITY has malformed format' in + conf.stderr), context + assert conf.stderr.count('warning:') == 1, context + else: + assert 'warning:' not in conf.stderr, context + + +def test_out_of_range(conf): + probabilities = [ + '-1', '101', '+101', '0:101', '0:0:101', '60:41', '0:60:41', + '4294967296', '-4294967296', + '999999999999999999999999', '-999999999999999999999999', + ] + for probability in probabilities: + assert conf._run_conf('--randconfig', extra_env={ + 'KCONFIG_PROBABILITY': probability, + 'KCONFIG_SEED': '0', + }) == 1, repr(probability) + assert 'KCONFIG_PROBABILITY:' in conf.stderr, repr(probability) + + +def test_valid(conf): + probabilities = [ + ('0', 'n', 'n'), + ('0:0', 'n', 'n'), + ('100:0', 'y', 'y'), + ('0:100', 'y', 'm'), + ('100:0:0', 'y', 'n'), + ('0:100:0', 'n', 'y'), + ('0:0:100', 'n', 'm'), + ('000:000:100', 'n', 'm'), + ] + for probability, boolean, tristate in probabilities: + assert conf._run_conf('--randconfig', extra_env={ + 'KCONFIG_PROBABILITY': probability, + 'KCONFIG_SEED': '0', + }) == 0, repr(probability) + assert 'warning:' not in conf.stderr, repr(probability) + for symbol, value in [('BOOL', boolean), ('TRI', tristate)]: + if value == 'n': + expected = '# CONFIG_{} is not set'.format(symbol) + else: + expected = 'CONFIG_{}={}'.format(symbol, value) + assert expected in conf.config.splitlines(), repr(probability) + + +def test_single_probability_matches_tristate_split(conf): + for seed in range(20): + context = 'seed={}'.format(seed) + assert conf._run_conf('--randconfig', extra_env={ + 'KCONFIG_PROBABILITY': '50', + 'KCONFIG_SEED': hex(seed), + }) == 0, context + assert 'warning:' not in conf.stderr, context + single = conf.config + + # 50% boolean y; 25% tristate y, 25% m, and 50% n. + assert conf._run_conf('--randconfig', extra_env={
scripts/kconfig/tests/randconfig_probability/__init__.py line 127
Quoted 10 lines
scripts/kconfig/tests/randconfig_probability/__init__.py new file +10
-0+ 'KCONFIG_PROBABILITY': '50:25:25', + 'KCONFIG_SEED': hex(seed), + }) == 0, context + assert 'warning:' not in conf.stderr, context + assert conf.config == single, context + + +def test_empty(conf, monkeypatch): + # extra_env overrides inherited variables, but cannot remove them. + monkeypatch.delenv('KCONFIG_PROBABILITY', raising=False)
It seems there's still a lingering unnecessary monkeypatch call here in test_empty. Otherwise, I think this patch is pretty close!
- Julian Braha
PATCH v6 1/1 kconfig: warn about malformed KCONFIG_PROBABILITY values
randconfig checks the numeric range of each probability but does not validate where strtol() stops. For example, KCONFIG_PROBABILITY=50% is accepted as 50:0:0: the '%' is parsed repeatedly as zero. For the documented value 50, tristate y/m/n probabilities are 25%/25%/50%. With 50%, both y/m probabilities silently become zero, reducing the coverage of random configuration builds. Empty fields and extra fields are also accepted.
Warn when the value does not follow the documented decimal format. For malformed inputs whose parsed probabilities are in range, preserve the existing interpretation unless KCONFIG_WERROR is set. In that case, exit with an error before writing the configuration. Leading whitespace and signs are accepted by strtol(), but also trigger the warning because they are outside the documented format.
Keep the strtol() result as long until the range check. This rejects out-of-range values that narrowing to int previously made valid, such as 4294967296 becoming zero on a 64-bit host, regardless of KCONFIG_WERROR.
Add regression tests for one warning per malformed input, preserved configurations, KCONFIG_WERROR, out-of-range values, the supported probability formats, and the documented empty-value default.
---
Thanks, Julian. I kept monkeypatch.delenv() to compare the unset and empty cases. I've dropped that comparison and removed the call; test_empty now just checks the empty value through extra_env.
Changes since v5:
- Remove the local monkeypatch fixture argument and the unset-variable comparison from test_empty.
- Keep the documented empty KCONFIG_PROBABILITY check explicit through extra_env, so the test does not depend on inherited environment.
Changes since v4:
- Replace pytest.mark.parametrize with loops and remove the unused pytest import from the probability test module, as requested by Julian.
- Include the probability and, where applicable, seed and WERROR value in assertion messages. Preserve all input combinations and checks.
Changes since v3:
- Move the KCONFIG_WERROR cleanup fixture to the shared conftest.py, as requested by Julian. Explicit extra_env settings still enable WERROR.
Changes since v2:
- Move the expected tristate distribution into the problem description.
- Simplify the initial format check to !isdigit((unsigned char)*env).
- Pass probability values and seeds through conf._run_conf(extra_env=...). The shared fixture clears inherited KCONFIG_WERROR before each test.
- Add -0, +0, bare + and - warning cases, and the +101 range-error case.
- Honor KCONFIG_WERROR for malformed format warnings, as discussed with Julian. Check unset, empty, 0 and 1 flag behavior.
- Compare 50% with 50:0:0 and -0 with 0 across 20 fixed seeds and check that each malformed input produces exactly one warning.
- Clarify that compatibility applies to malformed in-range inputs with KCONFIG_WERROR unset; values previously accepted by narrowing now fail.
Changes since v1:
- Warn about malformed in-range values while retaining their previous interpretation by default.
- Warn about leading whitespace and signs as undocumented input.
- Clarify that 50 gives tristate y/m/n probabilities of 25%/25%/50%, and compare it with the equivalent input 50:25:25 across 20 fixed seeds.
Validation on kbuild-next, x86_64, GCC 13.3.0:
- make testconfig with HOSTCFLAGS=-Werror: 28 passed, both normally and with KCONFIG_WERROR=1 inherited from the test runner. Explicit strict cases for empty, 0 and 1 flag values still pass.
- The same suite passed on the existing ASan/UBSan build at -O1, with leak detection disabled: 28 passed in each environment.
- The probability module reports 7 tests. Native and ASan/UBSan runs with inherited KCONFIG_PROBABILITY=101, KCONFIG_SEED=not-a-seed and each of KCONFIG_WERROR='', '0' and '1' all pass, including reverse test order.
- test_empty makes one explicit run with KCONFIG_PROBABILITY='' through extra_env; no local monkeypatch fixture is needed.
- The new suite gives 4 expected failures and 24 passes on the original binary; the version before WERROR gives 1 expected failure and 27 passes. Loops stop at their first failure, so these counts differ from the earlier parametrized regression results.
- C code, test Kconfig and shared fixtures are unchanged from v5. Earlier C compatibility and file-preservation checks were not rerun.
- No vmlinux build or boot test; this changes the host configuration tool. 32-bit hosts and other libc implementations were not tested.
scripts/kconfig/conf.c | 16 ++- scripts/kconfig/tests/conftest.py | 6 + .../tests/randconfig_probability/Kconfig | 12 ++ .../tests/randconfig_probability/__init__.py | 130 ++++++++++++++++++ 4 files changed, 163 insertions(+), 1 deletion(-) create mode 100644 scripts/kconfig/tests/randconfig_probability/Kconfig create mode 100644 scripts/kconfig/tests/randconfig_probability/__init__.py
scripts/kconfig/conf.c +15 -1
@@ -186,12 +186,23 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)if (mode == def_random) {int n, p[3 ]; +bool warned = false;char *env =getenv ("KCONFIG_PROBABILITY" ); n =0 ;while (env && *env) {char *endp; -int tmp =strtol (env, &endp,10 ); +long tmp =strtol (env, &endp,10 ); + +if (!isdigit ((unsigned char )*env) || + (*endp && *endp !=':' ) || + (*endp ==':' && (!endp[1 ] || n ==2 ))) { +if (!warned) { +fprintf (stderr, +"warning: KCONFIG_PROBABILITY has malformed format\n" ); + warned = true; + } + }if (tmp >=0 && tmp <=100 ) { p[n++] = tmp;
@@ -227,6 +238,9 @@ static void conf_set_all_new_symbols(enum conf_def_mode mode)perror ("KCONFIG_PROBABILITY" );exit (1 ); } + +if (warned &&getenv ("KCONFIG_WERROR" )) +exit (1 ); }menu_for_each_entry (menu) {
scripts/kconfig/tests/conftest.py +6 -0
@@ -312,6 +312,12 @@ class Conf: return self._matches('stderr', expected) [email protected](autouse=True) +def clear_werror(monkeypatch): + # extra_env can set strict mode, but cannot remove an inherited flag. + monkeypatch.delenv('KCONFIG_WERROR', raising=False) + + @pytest.fixture(scope="module") def conf(request): """Create a Conf instance and provide it to test functions."""
scripts/kconfig/tests/randconfig_probability/Kconfig new file +12 -0
@@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only + +config MODULES + bool + default y + modules + +config BOOL + bool "Bool" + +config TRI + tristate "Tristate"
scripts/kconfig/tests/randconfig_probability/__init__.py new file +130 -0
130 lines not on this page · raw message
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
Re: [PATCH v6] kconfig: warn about malformed KCONFIG_PROBABILITY values
Quoted 19 lines · Dmitrii Tulnov
randconfig checks the numeric range of each probability but does not validate where
strtol()stops. For example, KCONFIG_PROBABILITY=50% is accepted as 50:0:0: the '%' is parsed repeatedly as zero. For the documented value 50, tristate y/m/n probabilities are 25%/25%/50%. With 50%, both y/m probabilities silently become zero, reducing the coverage of random configuration builds. Empty fields and extra fields are also accepted.Warn when the value does not follow the documented decimal format. For malformed inputs whose parsed probabilities are in range, preserve the existing interpretation unless
KCONFIG_WERRORis set. In that case, exit with an error before writing the configuration. Leading whitespace and signs are accepted bystrtol(), but also trigger the warning because they are outside the documented format.Keep the
strtol()result as long until the range check. This rejects out-of-range values that narrowing to int previously made valid, such as 4294967296 becoming zero on a 64-bit host, regardless ofKCONFIG_WERROR.
Quoted 7 lines
Add regression tests for one warning per malformed input, preserved configurations,
KCONFIG_WERROR, out-of-range values, the supported probability formats, and the documented empty-value default.Fixes e43956e60769 ("kconfig: implement KCONFIG_PROBABILITY for randconfig")Assisted-by LLMSigned-off-by Dmitrii Tulnov <[email protected]>
Nathan Chancellor
maintainerRe: [PATCH v6] kconfig: warn about malformed KCONFIG_PROBABILITY values
In reply to Dmitrii TulnovQuoted 1 line · Dmitrii Tulnov
kconfig: warn about malformed
KCONFIG_PROBABILITYvalues
Applied to
https://git.kernel.org/pub/scm/linux/kernel/git/kbuild/linux.git kbuild-next-unstable
Thanks!
[1/1] kconfig: warn about malformed KCONFIG_PROBABILITY values
https://git.kernel.org/kbuild/c/9f01cd3a32439
Please look out for regression or issue reports or other follow up comments, as they may result in the patch/series getting dropped or reverted. Patches applied to an "unstable" branch are accepted pending wider testing in -next and any post-commit review; they will generally be moved to the main branch in a week if no issues are found.
Best regards,
Signature · 2 lines
Cheers, Nathan