Skip to contentSkip to conversation

Documentation: kunit: update kconfig options needed for UML coverage

PATCH In review Daniel Latypov 4 participants 8 messages Mar 26, 2022 – Mar 28, 2022

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Documentation: kunit: update kconfig options needed for UML coverage

Recent changes have made it so the current set is not sufficient. Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked.

Specifying a version of the debug info fixes this.
Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's hopefully less fragile (esp. given we're tied to GCC 6 and lower).

---
 Documentation/dev-tools/kunit/running_tips.rst | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Documentation/dev-tools/kunit/running_tips.rst +2 -1
index 7b6d26a25959..c36f6760087d 100644
@@ -114,6 +114,7 @@ Instead of enabling ``CONFIG_GCOV_KERNEL=y``, we can set these options:
 
 	CONFIG_DEBUG_KERNEL=y
 	CONFIG_DEBUG_INFO=y
+	CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
 	CONFIG_GCOV=y
 
 
@@ -122,7 +123,7 @@ Putting it together into a copy-pastable sequence of commands:
 .. code-block:: bash
 
 	# Append coverage options to the current config
-	$ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
+	$ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
 	$ ./tools/testing/kunit/kunit.py run
 	# Extract the coverage information from the build dir (.kunit/)
 	$ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/

base-commit: b14ffae378aa1db993e62b01392e70d1e585fb23

Signature · 1 line
2.35.1.1021.g381101b075-goog
Reviewed-by on v1 1/1
#1 lore raw
Quoted 10 lines · Daniel Latypov

Recent changes have made it so the current set is not sufficient. Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked.

Specifying a version of the debug info fixes this.
Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's hopefully less fragile (esp. given we're tied to GCC 6 and lower).

---

Looks good to me. One idea below, but it's a bit subjective and I'm fine with this going in as-is if you prefer.

Cheers,
-- Daivd

Quoted 16 lines
 Documentation/dev-tools/kunit/running_tips.rst | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Documentation/dev-tools/kunit/running_tips.rst +1 -0
index 7b6d26a25959..c36f6760087d 100644
@@ -114,6 +114,7 @@ Instead of enabling ``CONFIG_GCOV_KERNEL=y``, we can set these options:
 
        CONFIG_DEBUG_KERNEL=y
        CONFIG_DEBUG_INFO=y
+       CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
        CONFIG_GCOV=y
 
 

Documentation/dev-tools/kunit/running_tips.rst Putting it together into a copy-pastable sequence of commands: · line 127

Quoted 6 lines
Documentation/dev-tools/kunit/running_tips.rst +1 -1
@@ -122,7 +123,7 @@ Putting it together into a copy-pastable sequence of commands:
 .. code-block:: bash
 
        # Append coverage options to the current config
-       $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
+       $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
        $ ./tools/testing/kunit/kunit.py run

Would we want to instead use a chain of --kconfig_add arguments? (I think there are advantages either way...)

Quoted 7 lines
Documentation/dev-tools/kunit/running_tips.rst +0 -0
        # Extract the coverage information from the build dir (.kunit/)
        $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/

base-commit: b14ffae378aa1db993e62b01392e70d1e585fb23 --
2.35.1.1021.g381101b075-goog

#2 lore raw
Quoted 1 line · David Gow

<snip>

Quoted 7 lines

Documentation/dev-tools/kunit/running_tips.rst Putting it together into a copy-pastable sequence of commands: · line 127

Quoted 4 lines

# Append coverage options to the current config

Documentation/dev-tools/kunit/running_tips.rst +1 -1
-       $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
+       $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
        $ ./tools/testing/kunit/kunit.py run

Would we want to instead use a chain of --kconfig_add arguments? (I think there are advantages either way...)

I've been considering this ever since the --kconfig_add patch was accepted. It's more compatible w/ commands using --kunitconfig, but it also looks very verbose.
E.g. it looks like

$ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6
--kconfig_add=CONFIG_DEBUG_INFO=y
--kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
--kconfig_add=CONFIG_GCOV=y

Neither looks very appealing to me, so I've just kept it as-is for now.

Maybe there's something we can do to make this easier (e.g. allowing --kunitconfig to be repeated and mergable)?

#3 lore raw
Quoted 19 lines · Daniel Latypov
Quoted 1 line · David Gow

<snip>

Quoted 7 lines

Documentation/dev-tools/kunit/running_tips.rst Putting it together into a copy-pastable sequence of commands: · line 127

Quoted 4 lines

# Append coverage options to the current config

Documentation/dev-tools/kunit/running_tips.rst +1 -1
-       $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
+       $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
        $ ./tools/testing/kunit/kunit.py run

Would we want to instead use a chain of --kconfig_add arguments? (I think there are advantages either way...)

I've been considering this ever since the --kconfig_add patch was accepted. It's more compatible w/ commands using --kunitconfig, but it also looks very verbose.
E.g. it looks like

Quoted 4 lines
$ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6
--kconfig_add=CONFIG_DEBUG_INFO=y
--kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
--kconfig_add=CONFIG_GCOV=y

I don't think it's *that* much more verbose, but I see your point. I personally prefer this, but not enough to argue about it.

Quoted 4 lines

Neither looks very appealing to me, so I've just kept it as-is for now.

Maybe there's something we can do to make this easier (e.g. allowing --kunitconfig to be repeated and mergable)?

I would like --kunitconfig to be repeadable and mergable.

#4 lore raw
Quoted 21 lines · Brendan Higgins' via KUnit Development
Quoted 19 lines · Daniel Latypov
Quoted 1 line · David Gow

<snip>

Quoted 7 lines

Documentation/dev-tools/kunit/running_tips.rst Putting it together into a copy-pastable sequence of commands: · line 127

Quoted 4 lines

# Append coverage options to the current config

Documentation/dev-tools/kunit/running_tips.rst +1 -1
-       $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
+       $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
        $ ./tools/testing/kunit/kunit.py run

Would we want to instead use a chain of --kconfig_add arguments? (I think there are advantages either way...)

I've been considering this ever since the --kconfig_add patch was accepted. It's more compatible w/ commands using --kunitconfig, but it also looks very verbose.
E.g. it looks like

Quoted 7 lines
Quoted 4 lines
$ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6
--kconfig_add=CONFIG_DEBUG_INFO=y
--kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
--kconfig_add=CONFIG_GCOV=y

I don't think it's *that* much more verbose, but I see your point. I personally prefer this, but not enough to argue about it.

I personally prefer it too, but I'm biased as the person who added --kconfig_add.
They're both ugly enough I'd figured I'd save the bikeshedding for another patch.

Quoted 7 lines
Quoted 5 lines

Neither looks very appealing to me, so I've just kept it as-is for now.

Maybe there's something we can do to make this easier (e.g. allowing --kunitconfig to be repeated and mergable)?

I would like --kunitconfig to be repeadable and mergable.

Ack.
There's some things to consider first.

1. This will conflict w/
https://patchwork.kernel.org/project/linux-kselftest/patch/[email protected]/,
so I'm going to wait until that gets merged first.

2. some kconfigs can be incompatible (e.g. these options only work on UML, can't combine w/ a non-UML compatible file)
How do we make this less of a footgun?
We'd talked about how it'd be nice if kconfig/"make olddefconfig" could print out *why* options get dropped (either they're not visible, have unmet deps, etc.). If we had that, I'd feel more comfortable w/ repeatable kunitconfig.

3. People have the ability to do this already if they're really sure it's safe

$ cat <files...> | ./tools/testing/kunit/kunit.py run --kunitconfig=/dev/stdin

4. are we committed to supporting a "uml_coverage.kunitconfig" file? As shown by the existence of this patch, we've let it get broken for a bit, at least against linux-next (afaik, it was working on torvalds/master up until the 5.18 window opened and we had some patches reworking CONFIG_DEBUG_INFO).

These instructions exist so others don't have to try and re-figure out the steps/workarounds.
But they're not more formally "part of KUnit" since no one has had the expertise to maintain it (and fix issues like the reliance on gcc-6), etc.

Creating a kunitconfig file for this will further imply ownership.

#5 lore raw
Quoted 40 lines · Daniel Latypov' via KUnit Development
Quoted 21 lines · Brendan Higgins' via KUnit Development
Quoted 19 lines · Daniel Latypov
Quoted 1 line · David Gow

<snip>

Quoted 7 lines

Documentation/dev-tools/kunit/running_tips.rst Putting it together into a copy-pastable sequence of commands: · line 127

Quoted 4 lines

# Append coverage options to the current config

Documentation/dev-tools/kunit/running_tips.rst +1 -1
-       $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
+       $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
        $ ./tools/testing/kunit/kunit.py run

Would we want to instead use a chain of --kconfig_add arguments? (I think there are advantages either way...)

I've been considering this ever since the --kconfig_add patch was accepted. It's more compatible w/ commands using --kunitconfig, but it also looks very verbose.
E.g. it looks like

Quoted 7 lines
Quoted 4 lines
$ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6
--kconfig_add=CONFIG_DEBUG_INFO=y
--kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
--kconfig_add=CONFIG_GCOV=y

I don't think it's *that* much more verbose, but I see your point. I personally prefer this, but not enough to argue about it.

I personally prefer it too, but I'm biased as the person who added --kconfig_add.
They're both ugly enough I'd figured I'd save the bikeshedding for another patch.

Quoted 3 lines
Quoted 3 lines

Neither looks very appealing to me, so I've just kept it as-is for now.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

Maybe there's something we can do to make this easier (e.g. allowing --kunitconfig to be repeated and mergable)?

I would like --kunitconfig to be repeadable and mergable.

Ack.
There's some things to consider first.

I wasn't saying I want you to do it now. I just like the idea.

Quoted 3 lines

1. This will conflict w/
https://patchwork.kernel.org/project/linux-kselftest/patch/[email protected]/,
so I'm going to wait until that gets merged first.

Agreed.

Quoted 7 lines

2. some kconfigs can be incompatible (e.g. these options only work on UML, can't combine w/ a non-UML compatible file)
How do we make this less of a footgun?
We'd talked about how it'd be nice if kconfig/"make olddefconfig" could print out *why* options get dropped (either they're not visible, have unmet deps, etc.). If we had that, I'd feel more comfortable w/ repeatable kunitconfig.

Good point. Something to think about.

Quoted 2 lines

3. People have the ability to do this already if they're really sure it's safe

$ cat <files...> | ./tools/testing/kunit/kunit.py run --kunitconfig=/dev/stdin

Sure, but I still think multiple --kunitconfig s would be a good feature to add. I think it makes it a bit easier to think about mixing and matching kunitconfigs.

Quoted 5 lines

4. are we committed to supporting a "uml_coverage.kunitconfig" file? As shown by the existence of this patch, we've let it get broken for a bit, at least against linux-next (afaik, it was working on torvalds/master up until the 5.18 window opened and we had some patches reworking CONFIG_DEBUG_INFO).

Good point. I don't think we want to get in that business. UML Coverage needs a lot of work, and I don't think we have the time or resources to own that work by ourselves.

If someone else wants to add - and then own - such a kunitconfig, I would fully support them and maybe even help them a bit, but I don't want to own such a file until UML coverage gets a bit more stable.

Quoted 7 lines

These instructions exist so others don't have to try and re-figure out the steps/workarounds.
But they're not more formally "part of KUnit" since no one has had the expertise to maintain it (and fix issues like the reliance on gcc-6), etc.

Creating a kunitconfig file for this will further imply ownership.

Agreed.

Tested-by on v1 1/1
#6 lore raw
In reply to Daniel Latypov
Quoted 8 lines · Daniel Latypov

Recent changes have made it so the current set is not sufficient. Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked.

Specifying a version of the debug info fixes this.
Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's hopefully less fragile (esp. given we're tied to GCC 6 and lower).

Thanks!
Maxime

Reviewed-by on v1 1/1
#7 lore raw
In reply to Daniel Latypov
Quoted 9 lines · Daniel Latypov

Recent changes have made it so the current set is not sufficient. Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked.

Specifying a version of the debug info fixes this.
Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's hopefully less fragile (esp. given we're tied to GCC 6 and lower).

I agree with David's comment. Otherwise,