Documentation: kunit: update kconfig options needed for UML coverage
Subsystems (4)
PATCH 1/1 Documentation: kunit: update kconfig options needed for UML coverage
Recent changes have made it so the current set is not sufficient. Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked.
Specifying a version of the debug info fixes this.
Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's hopefully less fragile (esp. given we're tied to GCC 6 and lower).
--- Documentation/dev-tools/kunit/running_tips.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Documentation/dev-tools/kunit/running_tips.rst +2 -1
@@ -114,6 +114,7 @@ Instead of enabling ``CONFIG_GCOV_KERNEL=y``, we can set these options: CONFIG_DEBUG_KERNEL=y CONFIG_DEBUG_INFO=y + CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y CONFIG_GCOV=y
@@ -122,7 +123,7 @@ Putting it together into a copy-pastable sequence of commands: .. code-block:: bash # Append coverage options to the current config - $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig $ ./tools/testing/kunit/kunit.py run # Extract the coverage information from the build dir (.kunit/) $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/
base-commit: b14ffae378aa1db993e62b01392e70d1e585fb23
Signature · 1 line
2.35.1.1021.g381101b075-goog
Quoted 10 lines · Daniel Latypov
Recent changes have made it so the current set is not sufficient. Namely,
CONFIG_DEBUG_INFOis not being set even when explicitly asked.Specifying a version of the debug info fixes this.
PickCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULTas an option that's hopefully less fragile (esp. given we're tied to GCC 6 and lower).Signed-off-by Daniel Latypov <[email protected]>---
Looks good to me. One idea below, but it's a bit subjective and I'm fine with this going in as-is if you prefer.
Cheers,
-- Daivd
Quoted 16 lines
Documentation/dev-tools/kunit/running_tips.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)Documentation/dev-tools/kunit/running_tips.rst +1
-0@@ -114,6 +114,7 @@ Instead of enabling ``CONFIG_GCOV_KERNEL=y``, we can set these options: CONFIG_DEBUG_KERNEL=y CONFIG_DEBUG_INFO=y + CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y CONFIG_GCOV=y
Documentation/dev-tools/kunit/running_tips.rst Putting it together into a copy-pastable sequence of commands: · line 127
Quoted 6 lines
Documentation/dev-tools/kunit/running_tips.rst +1
-1@@ -122,7 +123,7 @@ Putting it together into a copy-pastable sequence of commands: .. code-block:: bash # Append coverage options to the current config - $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig $ ./tools/testing/kunit/kunit.py run
Would we want to instead use a chain of --kconfig_add arguments? (I think there are advantages either way...)
Quoted 7 lines
Documentation/dev-tools/kunit/running_tips.rst +0
-0# Extract the coverage information from the build dir (.kunit/) $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/base-commit:
b14ffae378aa1db993e62b01392e70d1e585fb23--
2.35.1.1021.g381101b075-goog
Quoted 1 line · David Gow
<snip>
Quoted 7 lines
Documentation/dev-tools/kunit/running_tips.rst Putting it together into a copy-pastable sequence of commands: · line 127
Quoted 4 lines
# Append coverage options to the current config
Documentation/dev-tools/kunit/running_tips.rst +1
-1- $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig $ ./tools/testing/kunit/kunit.py runWould we want to instead use a chain of --kconfig_add arguments? (I think there are advantages either way...)
I've been considering this ever since the --kconfig_add patch was accepted. It's more compatible w/ commands using --kunitconfig, but it also looks very verbose.
E.g. it looks like
$ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6 --kconfig_add=CONFIG_DEBUG_INFO=y --kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y --kconfig_add=CONFIG_GCOV=y
Neither looks very appealing to me, so I've just kept it as-is for now.
Maybe there's something we can do to make this easier (e.g. allowing --kunitconfig to be repeated and mergable)?
Quoted 19 lines · Daniel Latypov
Quoted 1 line · David Gow
<snip>
Quoted 7 lines
Documentation/dev-tools/kunit/running_tips.rst Putting it together into a copy-pastable sequence of commands: · line 127
Quoted 4 lines
# Append coverage options to the current config
Documentation/dev-tools/kunit/running_tips.rst +1
-1- $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig $ ./tools/testing/kunit/kunit.py runWould we want to instead use a chain of --kconfig_add arguments? (I think there are advantages either way...)
I've been considering this ever since the --kconfig_add patch was accepted. It's more compatible w/ commands using --kunitconfig, but it also looks very verbose.
E.g. it looks like
Quoted 4 lines
$ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6 --kconfig_add=CONFIG_DEBUG_INFO=y --kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y --kconfig_add=CONFIG_GCOV=y
I don't think it's *that* much more verbose, but I see your point. I personally prefer this, but not enough to argue about it.
Quoted 4 lines
Neither looks very appealing to me, so I've just kept it as-is for now.
Maybe there's something we can do to make this easier (e.g. allowing --kunitconfig to be repeated and mergable)?
I would like --kunitconfig to be repeadable and mergable.
Quoted 21 lines · Brendan Higgins' via KUnit Development
Quoted 19 lines · Daniel Latypov
Quoted 1 line · David Gow
<snip>
Quoted 7 lines
Documentation/dev-tools/kunit/running_tips.rst Putting it together into a copy-pastable sequence of commands: · line 127
Quoted 4 lines
# Append coverage options to the current config
Documentation/dev-tools/kunit/running_tips.rst +1
-1- $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig $ ./tools/testing/kunit/kunit.py runWould we want to instead use a chain of --kconfig_add arguments? (I think there are advantages either way...)
I've been considering this ever since the --kconfig_add patch was accepted. It's more compatible w/ commands using --kunitconfig, but it also looks very verbose.
E.g. it looks like
Quoted 7 lines
Quoted 4 lines
$ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6 --kconfig_add=CONFIG_DEBUG_INFO=y --kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y --kconfig_add=CONFIG_GCOV=yI don't think it's *that* much more verbose, but I see your point. I personally prefer this, but not enough to argue about it.
I personally prefer it too, but I'm biased as the person who added --kconfig_add.
They're both ugly enough I'd figured I'd save the bikeshedding for another patch.
Quoted 7 lines
Quoted 5 lines
Neither looks very appealing to me, so I've just kept it as-is for now.
Maybe there's something we can do to make this easier (e.g. allowing --kunitconfig to be repeated and mergable)?
I would like --kunitconfig to be repeadable and mergable.
Ack.
There's some things to consider first.
1. This will conflict w/
https://patchwork.kernel.org/project/linux-kselftest/patch/[email protected]/,
so I'm going to wait until that gets merged first.
2. some kconfigs can be incompatible (e.g. these options only work on UML, can't combine w/ a non-UML compatible file)
How do we make this less of a footgun?
We'd talked about how it'd be nice if kconfig/"make olddefconfig" could print out *why* options get dropped (either they're not visible, have unmet deps, etc.). If we had that, I'd feel more comfortable w/ repeatable kunitconfig.
3. People have the ability to do this already if they're really sure it's safe
$ cat <files...> | ./tools/testing/kunit/kunit.py run --kunitconfig=/dev/stdin
4. are we committed to supporting a "uml_coverage.kunitconfig" file? As shown by the existence of this patch, we've let it get broken for a bit, at least against linux-next (afaik, it was working on torvalds/master up until the 5.18 window opened and we had some patches reworking CONFIG_DEBUG_INFO).
These instructions exist so others don't have to try and re-figure out the steps/workarounds.
But they're not more formally "part of KUnit" since no one has had the expertise to maintain it (and fix issues like the reliance on gcc-6), etc.
Creating a kunitconfig file for this will further imply ownership.
Quoted 40 lines · Daniel Latypov' via KUnit Development
Quoted 21 lines · Brendan Higgins' via KUnit Development
Quoted 19 lines · Daniel Latypov
Quoted 1 line · David Gow
<snip>
Quoted 7 lines
Documentation/dev-tools/kunit/running_tips.rst Putting it together into a copy-pastable sequence of commands: · line 127
Quoted 4 lines
# Append coverage options to the current config
Documentation/dev-tools/kunit/running_tips.rst +1
-1- $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig $ ./tools/testing/kunit/kunit.py runWould we want to instead use a chain of --kconfig_add arguments? (I think there are advantages either way...)
I've been considering this ever since the --kconfig_add patch was accepted. It's more compatible w/ commands using --kunitconfig, but it also looks very verbose.
E.g. it looks likeQuoted 7 lines
Quoted 4 lines
$ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6 --kconfig_add=CONFIG_DEBUG_INFO=y --kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y --kconfig_add=CONFIG_GCOV=yI don't think it's *that* much more verbose, but I see your point. I personally prefer this, but not enough to argue about it.
I personally prefer it too, but I'm biased as the person who added --kconfig_add.
They're both ugly enough I'd figured I'd save the bikeshedding for another patch.Quoted 3 lines
Quoted 3 lines
Neither looks very appealing to me, so I've just kept it as-is for now.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Maybe there's something we can do to make this easier (e.g. allowing --kunitconfig to be repeated and mergable)?
I would like --kunitconfig to be repeadable and mergable.
Ack.
There's some things to consider first.
I wasn't saying I want you to do it now. I just like the idea.
Quoted 3 lines
1. This will conflict w/
https://patchwork.kernel.org/project/linux-kselftest/patch/[email protected]/,
so I'm going to wait until that gets merged first.
Agreed.
Quoted 7 lines
2. some kconfigs can be incompatible (e.g. these options only work on UML, can't combine w/ a non-UML compatible file)
How do we make this less of a footgun?
We'd talked about how it'd be nice if kconfig/"make olddefconfig" could print out *why* options get dropped (either they're not visible, have unmet deps, etc.). If we had that, I'd feel more comfortable w/ repeatable kunitconfig.
Good point. Something to think about.
Quoted 2 lines
3. People have the ability to do this already if they're really sure it's safe
$ cat <files...> | ./tools/testing/kunit/kunit.py run --kunitconfig=/dev/stdin
Sure, but I still think multiple --kunitconfig s would be a good feature to add. I think it makes it a bit easier to think about mixing and matching kunitconfigs.
Quoted 5 lines
4. are we committed to supporting a "uml_coverage.kunitconfig" file? As shown by the existence of this patch, we've let it get broken for a bit, at least against linux-next (afaik, it was working on torvalds/master up until the 5.18 window opened and we had some patches reworking
CONFIG_DEBUG_INFO).
Good point. I don't think we want to get in that business. UML Coverage needs a lot of work, and I don't think we have the time or resources to own that work by ourselves.
If someone else wants to add - and then own - such a kunitconfig, I would fully support them and maybe even help them a bit, but I don't want to own such a file until UML coverage gets a bit more stable.
Quoted 7 lines
These instructions exist so others don't have to try and re-figure out the steps/workarounds.
But they're not more formally "part of KUnit" since no one has had the expertise to maintain it (and fix issues like the reliance on gcc-6), etc.Creating a kunitconfig file for this will further imply ownership.
Agreed.
Quoted 8 lines · Daniel Latypov
Recent changes have made it so the current set is not sufficient. Namely,
CONFIG_DEBUG_INFOis not being set even when explicitly asked.Specifying a version of the debug info fixes this.
PickCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULTas an option that's hopefully less fragile (esp. given we're tied to GCC 6 and lower).Signed-off-by Daniel Latypov <[email protected]>
Thanks!
Maxime
Quoted 9 lines · Daniel Latypov
Recent changes have made it so the current set is not sufficient. Namely,
CONFIG_DEBUG_INFOis not being set even when explicitly asked.Specifying a version of the debug info fixes this.
PickCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULTas an option that's hopefully less fragile (esp. given we're tied to GCC 6 and lower).Signed-off-by Daniel Latypov <[email protected]>
I agree with David's comment. Otherwise,