clk: Add kunit tests for fixed rate and parent data

Document the linux,kunit board compatible string. This board is loaded into the Linux kernel when KUnit is testing devicetree dependent code.

Cc Rob Herring <[email protected]>

Cc Krzysztof Kozlowski <[email protected]>

Cc Brendan Higgins <[email protected]>

Cc David Gow <[email protected]>

Signed-off-by Stephen Boyd <[email protected]>

---
 .../bindings/kunit/linux,kunit.yaml           | 24 +++++++++++++++++++
 1 file changed, 24 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/kunit/linux,kunit.yaml

Documentation/devicetree/bindings/kunit/linux,kunit.yaml new file +24 -1

new file mode 100644
index 000000000000..dfe6da4796e8
@@ -0,0 +1,24 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/kunit/linux,kunit.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: KUnit
+
+maintainers:
+  - Brendan Higgins <[email protected]>
+  - David Gow <[email protected]>
+
+description:
+  KUnit board used to unit test the Linux kernel in User Mode Linux (UML).
+
+properties:
+  $nodename:
+    const: "/"
+  compatible:
+    const: linux,kunit
+
+additionalProperties: true
+
+...
--

https://git.kernel.org/pub/scm/linux/kernel/git/clk/linux.git/
https://git.kernel.org/pub/scm/linux/kernel/git/sboyd/spmi.git

Quoted 2 lines · Stephen Boyd

Document the linux,kunit board compatible string. This board is loaded into the Linux kernel when KUnit is testing devicetree dependent code.

As with the series as a whole, this might need to change a little bit if we want to either use devicetree overlays and/or other architectures.

That being said, I'm okay with having this until then: the only real topic for bikeshedding is the name.
- Is KUnit best as a board name, or part of the vendor name?
- Do we want to include the architecture in the name? Should it be "linux,kunit", "linux-kunit,uml", "linux,kunit-uml", etc?

Quoted 3 lines

Document the linux,kunit board compatible string. This board is loaded into the Linux kernel when KUnit is testing devicetree dependent code.

As with the series as a whole, this might need to change a little bit if we want to either use devicetree overlays and/or other architectures.

That being said, I'm okay with having this until then: the only real topic for bikeshedding is the name.
- Is KUnit best as a board name, or part of the vendor name?
- Do we want to include the architecture in the name? Should it be "linux,kunit", "linux-kunit,uml", "linux,kunit-uml", etc?

Quoting David Gow (2023-03-02 23:14:55)

Quoted 9 lines · Stephen Boyd

Quoted 3 lines

Document the linux,kunit board compatible string. This board is loaded into the Linux kernel when KUnit is testing devicetree dependent code.

As with the series as a whole, this might need to change a little bit if we want to either use devicetree overlays and/or other architectures.

Quoted 5 lines

That being said, I'm okay with having this until then: the only real topic for bikeshedding is the name.
- Is KUnit best as a board name, or part of the vendor name?
- Do we want to include the architecture in the name? Should it be "linux,kunit", "linux-kunit,uml", "linux,kunit-uml", etc?

I think I will drop this patch. I have overlays working. I hijacked of_core_init() to load the testcase data from drivers/of/unittest-data and made a container node for kunit overlays to apply to.

To fully exercise common clk framework code in KUnit we need to associate 'struct device' pointers with 'struct device_node' pointers so that things like clk_get() can parse DT nodes for 'clocks' and so that clk providers can use DT to provide clks; the most common mode of operation for clk providers.

Adding support to KUnit so that it loads a DTB is fairly simple after commit b31297f04e86 ("um: Add devicetree support"). We can simply pass a pre-compiled deviectree blob (DTB) on the kunit.py commandline and UML will load it. The problem is that tests won't know that the commandline has been modified, nor that a DTB has been loaded. Take a different approach so that tests can skip if a DTB hasn't been loaded.

Reuse the Makefile logic from the OF unittests to build a DTB into the kernel. This DTB will be for the mythical machine "linux,kunit", i.e. the devicetree for the KUnit "board". In practice, it is a dtsi file that will gather includes for kunit tests that rely in part on a devicetree being loaded. The devicetree should only be loaded if CONFIG_OF_KUNIT=y. Make that a choice config parallel to the existing CONFIG_OF_UNITTEST so that only one devicetree can be loaded in the system at a time. Similarly, the kernel commandline option to load a DTB is ignored if CONFIG_OF_KUNIT is enabled so that only one DTB is loaded at a time.

Add a simple unit test to confirm that the DTB loading worked. Future tests will add to the kunit.dtsi file to include their specific test nodes.

Cc Richard Weinberger <[email protected]>

Cc Anton Ivanov <[email protected]>

Cc Johannes Berg <[email protected]>

Cc Vincent Whitchurch <[email protected]>

Cc Rob Herring <[email protected]>

Cc Frank Rowand <[email protected]>

Signed-off-by Stephen Boyd <[email protected]>

---
 arch/um/kernel/dtb.c            | 29 +++++++++++++++--
 drivers/of/Kconfig              | 26 ++++++++++++++++
 drivers/of/Makefile             |  1 +
 drivers/of/kunit/.kunitconfig   |  4 +++
 drivers/of/kunit/Makefile       |  4 +++
 drivers/of/kunit/kunit.dtsi     |  8 +++++
 drivers/of/kunit/kunit.dtso     |  4 +++
 drivers/of/kunit/uml_dtb_test.c | 55 +++++++++++++++++++++++++++++++++
 8 files changed, 128 insertions(+), 3 deletions(-)
 create mode 100644 drivers/of/kunit/.kunitconfig
 create mode 100644 drivers/of/kunit/Makefile
 create mode 100644 drivers/of/kunit/kunit.dtsi
 create mode 100644 drivers/of/kunit/kunit.dtso
 create mode 100644 drivers/of/kunit/uml_dtb_test.c

arch/um/kernel/dtb.c +12 -3

index 484141b06938..ee63951b12df 100644
@@ -15,9 +15,32 @@ void uml_dtb_init(void)
        long long size;
        void *area;
 
-       area = uml_load_file(dtb, &size);
-       if (!area)
-               return;
+       if (IS_ENABLED(CONFIG_OF_KUNIT)) {
+               /*
+                * __dtbo_kunit_begin[] and __dtbo_kunit_end[] are magically
+                * created by cmd_dt_S_dtbo in scripts/Makefile.lib from the
+                * drivers/of/kunit/kunit.dtsi file.
+                */
+               extern uint8_t __dtbo_kunit_begin[];
+               extern uint8_t __dtbo_kunit_end[];
+
+               size = __dtbo_kunit_end - __dtbo_kunit_begin;
+               if (!size) {
+                       pr_warn("%s: kunit testcases is empty\n", __func__);

arch/um/kernel/dtb.c +4 -0

+               area = uml_load_file(dtb, &size);
+               if (!area)
+                       return;
+       }
 
        if (!early_init_dt_scan(area)) {
                pr_err("invalid DTB %s\n", dtb);

drivers/of/Kconfig +26 -0

index 80b5fd44ab1c..1f968b6a3dde 100644
@@ -12,6 +12,20 @@ menuconfig OF
 
 if OF
 
+choice
+       prompt "Devicetree Runtime Tests"
+       default OF_UNITTEST
+
+config OF_KUNIT
+       bool "Devicetree KUnit support" if KUNIT
+       depends on UML
+       select IRQ_DOMAIN
+       select OF_EARLY_FLATTREE
+       help
+         This option builds in KUnit test cases that rely on device tree infrastructure.
+         A fake Device Tree Blob (DTB) is loaded on the UML kernel running KUnit so that
+         KUnit tests can test device tree dependent code.
+
 config OF_UNITTEST
        bool "Device Tree runtime unit tests"
        depends on !SPARC

@@ -25,6 +39,18 @@ config OF_UNITTEST
 
          If unsure, say N here, but this option is safe to enable.
 
+endchoice
+
+config OF_DTB_KUNIT_TEST
+       tristate "Devicetree KUnit DTB Test" if !KUNIT_ALL_TESTS
+       depends on KUNIT
+       default KUNIT_ALL_TESTS
+       help
+         This option builds unit tests for the "linux,kunit" DTB built into
+         the UML kernel image.
+
+         If unsure, say N here, but this option is safe to enable.
+
 config OF_ALL_DTBS
        bool "Build all Device Tree Blobs"
        depends on COMPILE_TEST

drivers/of/Makefile +1 -0

index e0360a44306e..16eef3fdf60a 100644
@@ -19,4 +19,5 @@ obj-y += kexec.o
 endif
 endif
 
+obj-y += kunit/
 obj-$(CONFIG_OF_UNITTEST) += unittest-data/

drivers/of/kunit/.kunitconfig new file +4 -0

new file mode 100644
index 000000000000..1def0ad30d29
@@ -0,0 +1,4 @@
+CONFIG_KUNIT=y
+CONFIG_OF=y
+CONFIG_OF_KUNIT=y
+CONFIG_OF_DTB_KUNIT_TEST=y

drivers/of/kunit/Makefile new file +4 -0

new file mode 100644
index 000000000000..ffe0447e1ac7
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_OF_KUNIT) += kunit.dtbo.o
+
+obj-$(CONFIG_OF_DTB_KUNIT_TEST) += uml_dtb_test.o

drivers/of/kunit/kunit.dtsi new file +8 -0

new file mode 100644
index 000000000000..82f6c3e2b8d5
@@ -0,0 +1,8 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/ {
+       model = "KUnit UML";
+       compatible = "linux,kunit";
+};
+
+/* Include testcase dtsi files below */

drivers/of/kunit/kunit.dtso new file +4 -0

new file mode 100644
index 000000000000..50187e8d1422
@@ -0,0 +1,4 @@
+// SPDX-License-Identifier: GPL-2.0
+/dts-v1/;
+
+#include "kunit.dtsi"

drivers/of/kunit/uml_dtb_test.c new file +55 -1

new file mode 100644
index 000000000000..8966c9ebf51f
@@ -0,0 +1,55 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KUnit tests for DTB loading on UML
+ */
+#include <linux/kconfig.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
+
+#include <kunit/test.h>
+
+/*
+ * Test that of_machine_is_compatible() returns positive int when loaded DTB
+ * matches.
+ */
+static void uml_dtb_of_machine_compatible_test(struct kunit *test)
+{
+       KUNIT_EXPECT_GT(test, of_machine_is_compatible("linux,kunit"), 0);
+}
+
+/*
+ * Test that of_flat_dt_get_machine_name() returns the expected 'model' from the
+ * loaded DTB.
+ */
+static void uml_dtb_of_flat_dt_get_machine_name_test(struct kunit *test)
+{
+       KUNIT_EXPECT_STREQ(test, of_flat_dt_get_machine_name(), "KUnit UML");
+}
+
+static struct kunit_case uml_dtb_test_cases[] = {
+       KUNIT_CASE(uml_dtb_of_machine_compatible_test),
+       KUNIT_CASE(uml_dtb_of_flat_dt_get_machine_name_test),
+       {}
+};
+
+static int uml_dtb_test_init(struct kunit *test)
+{
+       if (!IS_ENABLED(CONFIG_OF_KUNIT))
+               kunit_skip(test, "requires CONFIG_OF_KUNIT");
+
+       return 0;
+}
+
+/*
+ * Test suite to confirm DTB is loaded on UML.
+ */
+static struct kunit_suite uml_dtb_suite = {
+       .name = "uml_dtb",
+       .init = uml_dtb_test_init,
+       .test_cases = uml_dtb_test_cases,
+};
+
+kunit_test_suites(
+       &uml_dtb_suite,
+);
+MODULE_LICENSE("GPL");
--

https://git.kernel.org/pub/scm/linux/kernel/git/clk/linux.git/
https://git.kernel.org/pub/scm/linux/kernel/git/sboyd/spmi.git

--
You received this message because you are subscribed to the Google Groups "KUnit Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/kunit-dev/20230302013822.1808711-3-sboyd%40kernel.org.

Quoted 14 lines

To fully exercise common clk framework code in KUnit we need to associate 'struct device' pointers with 'struct device_node' pointers so that things like clk_get() can parse DT nodes for 'clocks' and so that clk providers can use DT to provide clks; the most common mode of operation for clk providers.

Adding support to KUnit so that it loads a DTB is fairly simple after commit b31297f04e86 ("um: Add devicetree support"). We can simply pass a pre-compiled deviectree blob (DTB) on the kunit.py commandline and UML will load it. The problem is that tests won't know that the commandline has been modified, nor that a DTB has been loaded. Take a different approach so that tests can skip if a DTB hasn't been loaded.

Quoted 10 lines

Reuse the Makefile logic from the OF unittests to build a DTB into the kernel. This DTB will be for the mythical machine "linux,kunit", i.e. the devicetree for the KUnit "board". In practice, it is a dtsi file that will gather includes for kunit tests that rely in part on a devicetree being loaded. The devicetree should only be loaded if CONFIG_OF_KUNIT=y. Make that a choice config parallel to the existing CONFIG_OF_UNITTEST so that only one devicetree can be loaded in the system at a time. Similarly, the kernel commandline option to load a DTB is ignored if CONFIG_OF_KUNIT is enabled so that only one DTB is loaded at a time.

This feels a little bit like it's just papering over the real problem, which is that there's no way tests can skip themselves if no DTB is loaded.

That being said, I do think that there's probably some sense in supporting the compiled-in DTB as well (it's definitely simpler than patching kunit.py to always pass the extra command-line option in, for example).
But maybe it'd be nice to have the command-line option override the built-in one if present.

Quoting David Gow (2023-03-02 23:15:04)

Quoted 26 lines · Stephen Boyd

Quoted 24 lines

To fully exercise common clk framework code in KUnit we need to associate 'struct device' pointers with 'struct device_node' pointers so that things like clk_get() can parse DT nodes for 'clocks' and so that clk providers can use DT to provide clks; the most common mode of operation for clk providers.

Adding support to KUnit so that it loads a DTB is fairly simple after commit b31297f04e86 ("um: Add devicetree support"). We can simply pass a pre-compiled deviectree blob (DTB) on the kunit.py commandline and UML will load it. The problem is that tests won't know that the commandline has been modified, nor that a DTB has been loaded. Take a different approach so that tests can skip if a DTB hasn't been loaded.

Reuse the Makefile logic from the OF unittests to build a DTB into the kernel. This DTB will be for the mythical machine "linux,kunit", i.e. the devicetree for the KUnit "board". In practice, it is a dtsi file that will gather includes for kunit tests that rely in part on a devicetree being loaded. The devicetree should only be loaded if CONFIG_OF_KUNIT=y. Make that a choice config parallel to the existing CONFIG_OF_UNITTEST so that only one devicetree can be loaded in the system at a time. Similarly, the kernel commandline option to load a DTB is ignored if CONFIG_OF_KUNIT is enabled so that only one DTB is loaded at a time.

Quoted 3 lines

This feels a little bit like it's just papering over the real problem, which is that there's no way tests can skip themselves if no DTB is loaded.

Hmm. I think you're suggesting that the unit test data be loaded whenever CONFIG_OF=y and CONFIG_KUNIT=y. Then tests can check for CONFIG_OF and skip if it isn't enabled?

Quoted 7 lines

That being said, I do think that there's probably some sense in supporting the compiled-in DTB as well (it's definitely simpler than patching kunit.py to always pass the extra command-line option in, for example).
But maybe it'd be nice to have the command-line option override the built-in one if present.

Got it. I need to test loading another DTB on the commandline still, but I think this won't be a problem. We'll load the unittest-data DTB even with KUnit on UML, so assuming that works on UML right now it should be unchanged by this series once I resend.

Quoted 6 lines

Hmm. I think you're suggesting that the unit test data be loaded whenever CONFIG_OF=y and CONFIG_KUNIT=y. Then tests can check for CONFIG_OF and skip if it isn't enabled?

More of the opposite: that we should have some way of supporting tests which might want to use a DTB other than the built-in one. Mostly for non-UML situations where an actual devicetree is needed to even boot far enough to get test output (so we wouldn't be able to override it with a compiled-in test one).

I think moving to overlays probably will render this idea obsolete:
but the thought was to give test code a way to check for the required devicetree nodes at runtime, and skip the test if they weren't found. That way, the failure mode for trying to boot this on something which required another device tree for, e.g., serial, would be "these tests are skipped because the wrong device tree is loaded", not "I get no output because serial isn't working".

Again, though, it's only really needed for non-UML, and just loading overlays as needed should be much more sensible anyway.

Quoted 9 lines

Quoted 8 lines

That being said, I do think that there's probably some sense in supporting the compiled-in DTB as well (it's definitely simpler than patching kunit.py to always pass the extra command-line option in, for example).
But maybe it'd be nice to have the command-line option override the built-in one if present.

Quoted 4 lines

Got it. I need to test loading another DTB on the commandline still, but I think this won't be a problem. We'll load the unittest-data DTB even with KUnit on UML, so assuming that works on UML right now it should be unchanged by this series once I resend.

Again, moving to overlays should render this mostly obsolete, no? Or am I misunderstanding how the overlay stuff will work?

One possible future advantage of being able to test with custom DTs at boot time would be for fuzzing (provide random DT properties, see what happens in the test). We've got some vague plans to support a way of passing custom data to tests to support this kind of case (though, if we're using overlays, maybe the test could just patch those if we wanted to do that).

Quoting David Gow (2023-03-10 00:09:48)

Quoted 8 lines · Stephen Boyd

Quoted 6 lines

Hmm. I think you're suggesting that the unit test data be loaded whenever CONFIG_OF=y and CONFIG_KUNIT=y. Then tests can check for CONFIG_OF and skip if it isn't enabled?

Quoted 5 lines

More of the opposite: that we should have some way of supporting tests which might want to use a DTB other than the built-in one. Mostly for non-UML situations where an actual devicetree is needed to even boot far enough to get test output (so we wouldn't be able to override it with a compiled-in test one).

Ok, got it.

Quoted 11 lines

I think moving to overlays probably will render this idea obsolete:
but the thought was to give test code a way to check for the required devicetree nodes at runtime, and skip the test if they weren't found. That way, the failure mode for trying to boot this on something which required another device tree for, e.g., serial, would be "these tests are skipped because the wrong device tree is loaded", not "I get no output because serial isn't working".

Again, though, it's only really needed for non-UML, and just loading overlays as needed should be much more sensible anyway.

I still have one niggle here. Loading overlays requires CONFIG_OF_OVERLAY, and the overlay loading API returns -ENOTSUPP when CONFIG_OF_OVERLAY=n. For now I'm checking for the config being enabled in each test, but I'm thinking it may be better to simply call kunit_skip() from the overlay loading function if the config is disabled. This way tests can simply call the overlay loading function and we'll halt the test immediately if the config isn't enabled.

Quoted 9 lines

Quoted 9 lines

Quoted 8 lines

That being said, I do think that there's probably some sense in supporting the compiled-in DTB as well (it's definitely simpler than patching kunit.py to always pass the extra command-line option in, for example).
But maybe it'd be nice to have the command-line option override the built-in one if present.

Quoted 7 lines

Quoted 4 lines

Got it. I need to test loading another DTB on the commandline still, but I think this won't be a problem. We'll load the unittest-data DTB even with KUnit on UML, so assuming that works on UML right now it should be unchanged by this series once I resend.

Again, moving to overlays should render this mostly obsolete, no? Or am I misunderstanding how the overlay stuff will work?

Right, overlays make it largely a moot issue. The way the OF unit tests work today is by grafting a DTB onto the live tree. I'm reusing that logic to graft a container node target for kunit tests to add their overlays too. It will be clearer once I post v2.

Quoted 7 lines

One possible future advantage of being able to test with custom DTs at boot time would be for fuzzing (provide random DT properties, see what happens in the test). We've got some vague plans to support a way of passing custom data to tests to support this kind of case (though, if we're using overlays, maybe the test could just patch those if we wanted to do that).

Ah ok. I can see someone making a fuzzer that modifies devicetree properties randomly, e.g. using different strings for clock-names.

This reminds me of another issue I ran into. I wanted to test adding the same platform device to the platform bus twice to confirm that the second device can't be added. That prints a warning, which makes kunit.py think that the test has failed because it printed a warning. Is there some way to avoid that? I want something like

KUNIT_EXPECT_WARNING(test, <call some function>)

so I can test error cases.

Quoted 30 lines · Stephen Boyd

Quoting David Gow (2023-03-10 00:09:48)

Quoted 8 lines · Stephen Boyd

Quoted 6 lines

Hmm. I think you're suggesting that the unit test data be loaded whenever CONFIG_OF=y and CONFIG_KUNIT=y. Then tests can check for CONFIG_OF and skip if it isn't enabled?

Quoted 5 lines

More of the opposite: that we should have some way of supporting tests which might want to use a DTB other than the built-in one. Mostly for non-UML situations where an actual devicetree is needed to even boot far enough to get test output (so we wouldn't be able to override it with a compiled-in test one).

Ok, got it.

Quoted 11 lines

I think moving to overlays probably will render this idea obsolete:
but the thought was to give test code a way to check for the required devicetree nodes at runtime, and skip the test if they weren't found. That way, the failure mode for trying to boot this on something which required another device tree for, e.g., serial, would be "these tests are skipped because the wrong device tree is loaded", not "I get no output because serial isn't working".

Again, though, it's only really needed for non-UML, and just loading overlays as needed should be much more sensible anyway.

Quoted 8 lines

I still have one niggle here. Loading overlays requires CONFIG_OF_OVERLAY, and the overlay loading API returns -ENOTSUPP when CONFIG_OF_OVERLAY=n. For now I'm checking for the config being enabled in each test, but I'm thinking it may be better to simply call kunit_skip() from the overlay loading function if the config is disabled. This way tests can simply call the overlay loading function and we'll halt the test immediately if the config isn't enabled.

That sounds sensible, though there is a potential pitfall. If kunit_skip() is called directly from overlay code, might introduce a dependency on kunit.ko from the DT overlay, which we might not want. The solution there is either to have a kunit wrapper function (so the call is already in kunit.ko), or to have a hook to skip the current test (which probably makes sense to do anyway, but I think the wrapper is the better option).

Quoted 42 lines

Quoted 9 lines

Quoted 9 lines

Quoted 8 lines

That being said, I do think that there's probably some sense in supporting the compiled-in DTB as well (it's definitely simpler than patching kunit.py to always pass the extra command-line option in, for example).
But maybe it'd be nice to have the command-line option override the built-in one if present.

Quoted 7 lines

Quoted 4 lines

Got it. I need to test loading another DTB on the commandline still, but I think this won't be a problem. We'll load the unittest-data DTB even with KUnit on UML, so assuming that works on UML right now it should be unchanged by this series once I resend.

Again, moving to overlays should render this mostly obsolete, no? Or am I misunderstanding how the overlay stuff will work?

Right, overlays make it largely a moot issue. The way the OF unit tests work today is by grafting a DTB onto the live tree. I'm reusing that logic to graft a container node target for kunit tests to add their overlays too. It will be clearer once I post v2.

Quoted 7 lines

One possible future advantage of being able to test with custom DTs at boot time would be for fuzzing (provide random DT properties, see what happens in the test). We've got some vague plans to support a way of passing custom data to tests to support this kind of case (though, if we're using overlays, maybe the test could just patch those if we wanted to do that).

Ah ok. I can see someone making a fuzzer that modifies devicetree properties randomly, e.g. using different strings for clock-names.

This reminds me of another issue I ran into. I wanted to test adding the same platform device to the platform bus twice to confirm that the second device can't be added. That prints a warning, which makes kunit.py think that the test has failed because it printed a warning. Is there some way to avoid that? I want something like

KUNIT_EXPECT_WARNING(test, <call some function>)

so I can test error cases.

Hmm... I'd've thought that shouldn't be a problem: kunit.py should ignore most messages during a test, unless it can't find a valid result line. What does the raw KTAP output look like? (You can get it from kunit.py by passing the --raw_output option).

That being said, a KUNIT_EXPECT_LOG_MESSAGE() or similar is something we've wanted for a while. I think that the KASAN folks have been working on something similar using console tracepoints:
https://lore.kernel.org/all/ebf96ea600050f00ed567e80505ae8f242633640.1666113393.git.andreyknvl@google.com/

Cheers,
-- David

Quoted 40 lines · David Gow

Quoted 30 lines · Stephen Boyd

Quoting David Gow (2023-03-10 00:09:48)

Quoted 8 lines · Stephen Boyd

Quoted 6 lines

Hmm. I think you're suggesting that the unit test data be loaded whenever CONFIG_OF=y and CONFIG_KUNIT=y. Then tests can check for CONFIG_OF and skip if it isn't enabled?

Quoted 5 lines

More of the opposite: that we should have some way of supporting tests which might want to use a DTB other than the built-in one. Mostly for non-UML situations where an actual devicetree is needed to even boot far enough to get test output (so we wouldn't be able to override it with a compiled-in test one).

Ok, got it.

Quoted 11 lines

I think moving to overlays probably will render this idea obsolete:
but the thought was to give test code a way to check for the required devicetree nodes at runtime, and skip the test if they weren't found. That way, the failure mode for trying to boot this on something which required another device tree for, e.g., serial, would be "these tests are skipped because the wrong device tree is loaded", not "I get no output because serial isn't working".

Again, though, it's only really needed for non-UML, and just loading overlays as needed should be much more sensible anyway.

Quoted 8 lines

I still have one niggle here. Loading overlays requires CONFIG_OF_OVERLAY, and the overlay loading API returns -ENOTSUPP when CONFIG_OF_OVERLAY=n. For now I'm checking for the config being enabled in each test, but I'm thinking it may be better to simply call kunit_skip() from the overlay loading function if the config is disabled. This way tests can simply call the overlay loading function and we'll halt the test immediately if the config isn't enabled.

Quoted 51 lines

That sounds sensible, though there is a potential pitfall. If kunit_skip() is called directly from overlay code, might introduce a dependency on kunit.ko from the DT overlay, which we might not want. The solution there is either to have a kunit wrapper function (so the call is already in kunit.ko), or to have a hook to skip the current test (which probably makes sense to do anyway, but I think the wrapper is the better option).

Quoted 42 lines

Quoted 9 lines

Quoted 9 lines

Quoted 8 lines

That being said, I do think that there's probably some sense in supporting the compiled-in DTB as well (it's definitely simpler than patching kunit.py to always pass the extra command-line option in, for example).
But maybe it'd be nice to have the command-line option override the built-in one if present.

Quoted 7 lines

Quoted 4 lines

Got it. I need to test loading another DTB on the commandline still, but I think this won't be a problem. We'll load the unittest-data DTB even with KUnit on UML, so assuming that works on UML right now it should be unchanged by this series once I resend.

Again, moving to overlays should render this mostly obsolete, no? Or am I misunderstanding how the overlay stuff will work?

Right, overlays make it largely a moot issue. The way the OF unit tests work today is by grafting a DTB onto the live tree. I'm reusing that logic to graft a container node target for kunit tests to add their overlays too. It will be clearer once I post v2.

Quoted 7 lines

One possible future advantage of being able to test with custom DTs at boot time would be for fuzzing (provide random DT properties, see what happens in the test). We've got some vague plans to support a way of passing custom data to tests to support this kind of case (though, if we're using overlays, maybe the test could just patch those if we wanted to do that).

Ah ok. I can see someone making a fuzzer that modifies devicetree properties randomly, e.g. using different strings for clock-names.

This reminds me of another issue I ran into. I wanted to test adding the same platform device to the platform bus twice to confirm that the second device can't be added. That prints a warning, which makes kunit.py think that the test has failed because it printed a warning. Is there some way to avoid that? I want something like

KUNIT_EXPECT_WARNING(test, <call some function>)

so I can test error cases.

DT unittests already have a similar concept. A test can report that a kernel warning (or any other specific text) either (1) must occur for the test to pass or (2) must _not_ occur for the test to pass. The check for the kernel warning is done by the test output parsing program scripts/dtc/of_unittest_expect.

The reporting by a test of an expected error in drivers/of/unittest.c is done by EXPECT_BEGIN() and EXPECT_END(). These have been in unittest for a long time.

The reporting by a test of a not expected to occur error is done by EXPECT_NOT_BEGIN() and EXPECT_NOT_END(). These are added to unittest in linux 6.3-rc1.

I discussed this concept in one of the early TAP / KTAP discussion

threads and expect to start a discussion thread on this specific topic in the KTAP Specification V2 context. I expect the discussion to result in a different implementation than what DT unittests are using (bike shedding likely to ensue) but whatever is agreed to should be easy for DT to switch to.

Quoted 14 lines

Hmm... I'd've thought that shouldn't be a problem: kunit.py should ignore most messages during a test, unless it can't find a valid result line. What does the raw KTAP output look like? (You can get it from kunit.py by passing the --raw_output option).

That being said, a KUNIT_EXPECT_LOG_MESSAGE() or similar is something we've wanted for a while. I think that the KASAN folks have been working on something similar using console tracepoints:
https://lore.kernel.org/all/ebf96ea600050f00ed567e80505ae8f242633640.1666113393.git.andreyknvl@google.com/

Cheers,
-- David

Quoted 107 lines · Frank Rowand

Quoted 40 lines · David Gow

Quoted 30 lines · Stephen Boyd

Quoting David Gow (2023-03-10 00:09:48)

Quoted 8 lines · Stephen Boyd

Quoted 6 lines

Hmm. I think you're suggesting that the unit test data be loaded whenever CONFIG_OF=y and CONFIG_KUNIT=y. Then tests can check for CONFIG_OF and skip if it isn't enabled?

Quoted 5 lines

More of the opposite: that we should have some way of supporting tests which might want to use a DTB other than the built-in one. Mostly for non-UML situations where an actual devicetree is needed to even boot far enough to get test output (so we wouldn't be able to override it with a compiled-in test one).

Ok, got it.

Quoted 11 lines

I think moving to overlays probably will render this idea obsolete:
but the thought was to give test code a way to check for the required devicetree nodes at runtime, and skip the test if they weren't found. That way, the failure mode for trying to boot this on something which required another device tree for, e.g., serial, would be "these tests are skipped because the wrong device tree is loaded", not "I get no output because serial isn't working".

Again, though, it's only really needed for non-UML, and just loading overlays as needed should be much more sensible anyway.

Quoted 8 lines

I still have one niggle here. Loading overlays requires CONFIG_OF_OVERLAY, and the overlay loading API returns -ENOTSUPP when CONFIG_OF_OVERLAY=n. For now I'm checking for the config being enabled in each test, but I'm thinking it may be better to simply call kunit_skip() from the overlay loading function if the config is disabled. This way tests can simply call the overlay loading function and we'll halt the test immediately if the config isn't enabled.

Quoted 51 lines

That sounds sensible, though there is a potential pitfall. If kunit_skip() is called directly from overlay code, might introduce a dependency on kunit.ko from the DT overlay, which we might not want. The solution there is either to have a kunit wrapper function (so the call is already in kunit.ko), or to have a hook to skip the current test (which probably makes sense to do anyway, but I think the wrapper is the better option).

Quoted 42 lines

Quoted 9 lines

Quoted 9 lines

Quoted 8 lines

That being said, I do think that there's probably some sense in supporting the compiled-in DTB as well (it's definitely simpler than patching kunit.py to always pass the extra command-line option in, for example).
But maybe it'd be nice to have the command-line option override the built-in one if present.

Quoted 7 lines

Quoted 4 lines

Got it. I need to test loading another DTB on the commandline still, but I think this won't be a problem. We'll load the unittest-data DTB even with KUnit on UML, so assuming that works on UML right now it should be unchanged by this series once I resend.

Again, moving to overlays should render this mostly obsolete, no? Or am I misunderstanding how the overlay stuff will work?

Right, overlays make it largely a moot issue. The way the OF unit tests work today is by grafting a DTB onto the live tree. I'm reusing that logic to graft a container node target for kunit tests to add their overlays too. It will be clearer once I post v2.

Quoted 7 lines

One possible future advantage of being able to test with custom DTs at boot time would be for fuzzing (provide random DT properties, see what happens in the test). We've got some vague plans to support a way of passing custom data to tests to support this kind of case (though, if we're using overlays, maybe the test could just patch those if we wanted to do that).

Ah ok. I can see someone making a fuzzer that modifies devicetree properties randomly, e.g. using different strings for clock-names.

This reminds me of another issue I ran into. I wanted to test adding the same platform device to the platform bus twice to confirm that the second device can't be added. That prints a warning, which makes kunit.py think that the test has failed because it printed a warning. Is there some way to avoid that? I want something like

KUNIT_EXPECT_WARNING(test, <call some function>)

so I can test error cases.

DT unittests already have a similar concept. A test can report that a kernel warning (or any other specific text) either (1) must occur for the test to pass or (2) must _not_ occur for the test to pass. The check for the kernel warning is done by the test output parsing program scripts/dtc/of_unittest_expect.

The reporting by a test of an expected error in drivers/of/unittest.c is done by EXPECT_BEGIN() and EXPECT_END(). These have been in unittest for a long time.

The reporting by a test of a not expected to occur error is done by EXPECT_NOT_BEGIN() and EXPECT_NOT_END(). These are added to unittest in linux 6.3-rc1.

Quoted 1 line

I discussed this concept in one of the early TAP / KTAP discussion

The link to the early KTAP discussion on this concept is:

   https://lore.kernel.org/all/[email protected]/T/#u

Quoted 5 lines

threads and expect to start a discussion thread on this specific topic in the KTAP Specification V2 context. I expect the discussion to result in a different implementation than what DT unittests are using (bike shedding likely to ensue) but whatever is agreed to should be easy for DT to switch to.

The link to the KTAP Specification Version 2 process and progress is:

https://elinux.org/Test_Results_Format_Notes#KTAP_version_2

Quoted 15 lines

Quoted 14 lines

Hmm... I'd've thought that shouldn't be a problem: kunit.py should ignore most messages during a test, unless it can't find a valid result line. What does the raw KTAP output look like? (You can get it from kunit.py by passing the --raw_output option).

That being said, a KUNIT_EXPECT_LOG_MESSAGE() or similar is something we've wanted for a while. I think that the KASAN folks have been working on something similar using console tracepoints:
https://lore.kernel.org/all/ebf96ea600050f00ed567e80505ae8f242633640.1666113393.git.andreyknvl@google.com/

Cheers,
-- David

Quoted 109 lines · Frank Rowand

Quoted 107 lines · Frank Rowand

Quoted 40 lines · David Gow

Quoted 30 lines · Stephen Boyd

Quoting David Gow (2023-03-10 00:09:48)

Quoted 8 lines · Stephen Boyd

Quoted 6 lines

Hmm. I think you're suggesting that the unit test data be loaded whenever CONFIG_OF=y and CONFIG_KUNIT=y. Then tests can check for CONFIG_OF and skip if it isn't enabled?

Quoted 5 lines

More of the opposite: that we should have some way of supporting tests which might want to use a DTB other than the built-in one. Mostly for non-UML situations where an actual devicetree is needed to even boot far enough to get test output (so we wouldn't be able to override it with a compiled-in test one).

Ok, got it.

Quoted 11 lines

I think moving to overlays probably will render this idea obsolete:
but the thought was to give test code a way to check for the required devicetree nodes at runtime, and skip the test if they weren't found. That way, the failure mode for trying to boot this on something which required another device tree for, e.g., serial, would be "these tests are skipped because the wrong device tree is loaded", not "I get no output because serial isn't working".

Again, though, it's only really needed for non-UML, and just loading overlays as needed should be much more sensible anyway.

Quoted 8 lines

I still have one niggle here. Loading overlays requires CONFIG_OF_OVERLAY, and the overlay loading API returns -ENOTSUPP when CONFIG_OF_OVERLAY=n. For now I'm checking for the config being enabled in each test, but I'm thinking it may be better to simply call kunit_skip() from the overlay loading function if the config is disabled. This way tests can simply call the overlay loading function and we'll halt the test immediately if the config isn't enabled.

Quoted 51 lines

That sounds sensible, though there is a potential pitfall. If kunit_skip() is called directly from overlay code, might introduce a dependency on kunit.ko from the DT overlay, which we might not want. The solution there is either to have a kunit wrapper function (so the call is already in kunit.ko), or to have a hook to skip the current test (which probably makes sense to do anyway, but I think the wrapper is the better option).

Quoted 42 lines

Quoted 9 lines

Quoted 9 lines

Quoted 8 lines

That being said, I do think that there's probably some sense in supporting the compiled-in DTB as well (it's definitely simpler than patching kunit.py to always pass the extra command-line option in, for example).
But maybe it'd be nice to have the command-line option override the built-in one if present.

Quoted 7 lines

Quoted 4 lines

Got it. I need to test loading another DTB on the commandline still, but I think this won't be a problem. We'll load the unittest-data DTB even with KUnit on UML, so assuming that works on UML right now it should be unchanged by this series once I resend.

Again, moving to overlays should render this mostly obsolete, no? Or am I misunderstanding how the overlay stuff will work?

Right, overlays make it largely a moot issue. The way the OF unit tests work today is by grafting a DTB onto the live tree. I'm reusing that logic to graft a container node target for kunit tests to add their overlays too. It will be clearer once I post v2.

Quoted 7 lines

One possible future advantage of being able to test with custom DTs at boot time would be for fuzzing (provide random DT properties, see what happens in the test). We've got some vague plans to support a way of passing custom data to tests to support this kind of case (though, if we're using overlays, maybe the test could just patch those if we wanted to do that).

Ah ok. I can see someone making a fuzzer that modifies devicetree properties randomly, e.g. using different strings for clock-names.

This reminds me of another issue I ran into. I wanted to test adding the same platform device to the platform bus twice to confirm that the second device can't be added. That prints a warning, which makes kunit.py think that the test has failed because it printed a warning. Is there some way to avoid that? I want something like

KUNIT_EXPECT_WARNING(test, <call some function>)

so I can test error cases.

DT unittests already have a similar concept. A test can report that a kernel warning (or any other specific text) either (1) must occur for the test to pass or (2) must _not_ occur for the test to pass. The check for the kernel warning is done by the test output parsing program scripts/dtc/of_unittest_expect.

The reporting by a test of an expected error in drivers/of/unittest.c is done by EXPECT_BEGIN() and EXPECT_END(). These have been in unittest for a long time.

The reporting by a test of a not expected to occur error is done by EXPECT_NOT_BEGIN() and EXPECT_NOT_END(). These are added to unittest in linux 6.3-rc1.

Quoted 7 lines

Quoted 1 line

I discussed this concept in one of the early TAP / KTAP discussion

The link to the early KTAP discussion on this concept is:

   https://lore.kernel.org/all/[email protected]/T/#u

Thanks -- I'd totally forgotten about that!

I still personally would prefer a way of checking this from within the kernel, as if we're just printing out "EXPECT: " lines, then it's not possible to know if a test passes just from the raw results (and things like statistics can't be updated without a separate tool like kunit.py parsing the KTAP.

Indeed, my personal preference is that this log-based way of doing expectations is probably best kept as a last resort. i.e.,
1. Try to add a hook to the code which prints the message, which can then fail the test (or set a flag for the test to check later). This probably needs some better KUnit-side helpers to be truly ergonomic, but at least avoids too strict a dependency on the exact formatting of the log messages.

2. If that doesn't work, use console tracepoints or similar to implement an EXPECT_BEGIN() / EXPECT_END() or similar API entirely within the kernel.

3. Only if we can't come up with a working way of doing the former options, resort to adding "EXPECT:" lines and having a parser pick up on this.

One of the downsides of doing "EXPECT" lines in KTAP is that it'll suddenly be much more dependent on the exact layout of the tests, as we'd need to be able to override a test result if an expectation fails (at least, to maintain the KUnit structure). And overriding a result which is already in the output seems really, really ugly.

There's a patch to the KASAN tests to move from doing option 1 to option 2 above (in order to better support RCU, which didn't work with the hook):
https://lore.kernel.org/all/ebf96ea600050f00ed567e80505ae8f242633640.1666113393.git.andreyknvl@google.com/

Quoted 6 lines

Quoted 5 lines

threads and expect to start a discussion thread on this specific topic in the KTAP Specification V2 context. I expect the discussion to result in a different implementation than what DT unittests are using (bike shedding likely to ensue) but whatever is agreed to should be easy for DT to switch to.

Quoted 4 lines

The link to the KTAP Specification Version 2 process and progress is:

https://elinux.org/Test_Results_Format_Notes#KTAP_version_2

Thanks! We've got a few more KTAP ideas to air, so will hopefully send those out soon!

Cheers,
-- David

Quoted 16 lines

Quoted 15 lines

Quoted 14 lines

Hmm... I'd've thought that shouldn't be a problem: kunit.py should ignore most messages during a test, unless it can't find a valid result line. What does the raw KTAP output look like? (You can get it from kunit.py by passing the --raw_output option).

That being said, a KUNIT_EXPECT_LOG_MESSAGE() or similar is something we've wanted for a while. I think that the KASAN folks have been working on something similar using console tracepoints:
https://lore.kernel.org/all/ebf96ea600050f00ed567e80505ae8f242633640.1666113393.git.andreyknvl@google.com/

Cheers,
-- David

Quoted 118 lines · David Gow

Quoted 109 lines · Frank Rowand

Quoted 107 lines · Frank Rowand

Quoted 40 lines · David Gow

Quoted 30 lines · Stephen Boyd

Quoting David Gow (2023-03-10 00:09:48)

Quoted 8 lines · Stephen Boyd

Quoted 6 lines

Hmm. I think you're suggesting that the unit test data be loaded whenever CONFIG_OF=y and CONFIG_KUNIT=y. Then tests can check for CONFIG_OF and skip if it isn't enabled?

Quoted 5 lines

More of the opposite: that we should have some way of supporting tests which might want to use a DTB other than the built-in one. Mostly for non-UML situations where an actual devicetree is needed to even boot far enough to get test output (so we wouldn't be able to override it with a compiled-in test one).

Ok, got it.

Quoted 11 lines

I think moving to overlays probably will render this idea obsolete:
but the thought was to give test code a way to check for the required devicetree nodes at runtime, and skip the test if they weren't found. That way, the failure mode for trying to boot this on something which required another device tree for, e.g., serial, would be "these tests are skipped because the wrong device tree is loaded", not "I get no output because serial isn't working".

Again, though, it's only really needed for non-UML, and just loading overlays as needed should be much more sensible anyway.

Quoted 8 lines

I still have one niggle here. Loading overlays requires CONFIG_OF_OVERLAY, and the overlay loading API returns -ENOTSUPP when CONFIG_OF_OVERLAY=n. For now I'm checking for the config being enabled in each test, but I'm thinking it may be better to simply call kunit_skip() from the overlay loading function if the config is disabled. This way tests can simply call the overlay loading function and we'll halt the test immediately if the config isn't enabled.

Quoted 51 lines

That sounds sensible, though there is a potential pitfall. If kunit_skip() is called directly from overlay code, might introduce a dependency on kunit.ko from the DT overlay, which we might not want. The solution there is either to have a kunit wrapper function (so the call is already in kunit.ko), or to have a hook to skip the current test (which probably makes sense to do anyway, but I think the wrapper is the better option).

Quoted 42 lines

Quoted 9 lines

Quoted 9 lines

Quoted 8 lines

That being said, I do think that there's probably some sense in supporting the compiled-in DTB as well (it's definitely simpler than patching kunit.py to always pass the extra command-line option in, for example).
But maybe it'd be nice to have the command-line option override the built-in one if present.

Quoted 7 lines

Quoted 4 lines

Got it. I need to test loading another DTB on the commandline still, but I think this won't be a problem. We'll load the unittest-data DTB even with KUnit on UML, so assuming that works on UML right now it should be unchanged by this series once I resend.

Again, moving to overlays should render this mostly obsolete, no? Or am I misunderstanding how the overlay stuff will work?

Right, overlays make it largely a moot issue. The way the OF unit tests work today is by grafting a DTB onto the live tree. I'm reusing that logic to graft a container node target for kunit tests to add their overlays too. It will be clearer once I post v2.

Quoted 7 lines

One possible future advantage of being able to test with custom DTs at boot time would be for fuzzing (provide random DT properties, see what happens in the test). We've got some vague plans to support a way of passing custom data to tests to support this kind of case (though, if we're using overlays, maybe the test could just patch those if we wanted to do that).

Ah ok. I can see someone making a fuzzer that modifies devicetree properties randomly, e.g. using different strings for clock-names.

This reminds me of another issue I ran into. I wanted to test adding the same platform device to the platform bus twice to confirm that the second device can't be added. That prints a warning, which makes kunit.py think that the test has failed because it printed a warning. Is there some way to avoid that? I want something like

KUNIT_EXPECT_WARNING(test, <call some function>)

so I can test error cases.

DT unittests already have a similar concept. A test can report that a kernel warning (or any other specific text) either (1) must occur for the test to pass or (2) must _not_ occur for the test to pass. The check for the kernel warning is done by the test output parsing program scripts/dtc/of_unittest_expect.

The reporting by a test of an expected error in drivers/of/unittest.c is done by EXPECT_BEGIN() and EXPECT_END(). These have been in unittest for a long time.

The reporting by a test of a not expected to occur error is done by EXPECT_NOT_BEGIN() and EXPECT_NOT_END(). These are added to unittest in linux 6.3-rc1.

Quoted 7 lines

Quoted 1 line

I discussed this concept in one of the early TAP / KTAP discussion

The link to the early KTAP discussion on this concept is:

   https://lore.kernel.org/all/[email protected]/T/#u

Quoted 8 lines

Thanks -- I'd totally forgotten about that!

I still personally would prefer a way of checking this from within the kernel, as if we're just printing out "EXPECT: " lines, then it's not possible to know if a test passes just from the raw results (and things like statistics can't be updated without a separate tool like kunit.py parsing the KTAP.

Yes, I totally agree with that. If there is a reasonable way to implement. But in the DT unittest world, I have not found a reasonable way. Adding hooks is suggested below, but for DT unittest _I_ (opinion) do not find that reasonable. I voice no vote for kunit - that decision is up to the kunit crowd.

Quoted 8 lines

Indeed, my personal preference is that this log-based way of doing expectations is probably best kept as a last resort. i.e.,
1. Try to add a hook to the code which prints the message, which can then fail the test (or set a flag for the test to check later). This probably needs some better KUnit-side helpers to be truly ergonomic, but at least avoids too strict a dependency on the exact formatting of the log messages.

I'm not a fan of hooks. I see them as a maintenance burden, dependent upon the source version of the object being tested, yet another thing that can go wrong, and adds complexity to creating a test environment and running the test. Again, this just a personal opinion, and I'm not voting for or against this for kunit.

Quoted 3 lines

2. If that doesn't work, use console tracepoints or similar to implement an EXPECT_BEGIN() / EXPECT_END() or similar API entirely within the kernel.

Isn't this just another hook? So same opinion.

Quoted 3 lines

3. Only if we can't come up with a working way of doing the former options, resort to adding "EXPECT:" lines and having a parser pick up on this.

Again, don't let my opinion affect the voting between 1, 2, 3, or other for kunit.

Quoted 6 lines

One of the downsides of doing "EXPECT" lines in KTAP is that it'll suddenly be much more dependent on the exact layout of the tests, as we'd need to be able to override a test result if an expectation fails (at least, to maintain the KUnit structure). And overriding a result which is already in the output seems really, really ugly.

I don't understand "dependent on the exact layout of the tests". If you are saying that the test result parser has to figure out which test result to override, that has not been an issue in the cases that I use EXPECTs in DT unittest. The EXPECT begin and EXPECT end have always immediately surrounded a single test, so when the parser processes the EXPECT end, only the most recent test result could be overridden. This has worked because the kernel warning and error messages have been from kernel action that happens synchronously with the test. If the test prods the kernel in a way that results in the kernel performing an asynchronous activity (eg in another thread), then it becomes more complex to structure the EXPECT end -- I would imagine that the test would have to block on the asynchronous activity just before reporting the normal KTAP status result for the test (and the EXPECT end would normally be just after reporting the KTAP status result for the test).

I agree with overriding being ugly. For the DT unittest results parser, the EXPECT summary results are reported separately from the individual test summary results. The parser also flags the EXPECT failure in line with the normal individual test result lines.

I see both parsing results as valid, and as a policy choice for each test parser.

Quoted 13 lines

There's a patch to the KASAN tests to move from doing option 1 to option 2 above (in order to better support RCU, which didn't work with the hook):
https://lore.kernel.org/all/ebf96ea600050f00ed567e80505ae8f242633640.1666113393.git.andreyknvl@google.com/

Quoted 6 lines

Quoted 5 lines

threads and expect to start a discussion thread on this specific topic in the KTAP Specification V2 context. I expect the discussion to result in a different implementation than what DT unittests are using (bike shedding likely to ensue) but whatever is agreed to should be easy for DT to switch to.

Quoted 7 lines

Quoted 4 lines

The link to the KTAP Specification Version 2 process and progress is:

https://elinux.org/Test_Results_Format_Notes#KTAP_version_2

Thanks! We've got a few more KTAP ideas to air, so will hopefully send those out soon!

Glad to hear, I'm hoping that process starts progressing a bit.

-Frank

Quoted 20 lines

Cheers,
-- David

Quoted 16 lines

Quoted 15 lines

Quoted 14 lines

Hmm... I'd've thought that shouldn't be a problem: kunit.py should ignore most messages during a test, unless it can't find a valid result line. What does the raw KTAP output look like? (You can get it from kunit.py by passing the --raw_output option).

That being said, a KUNIT_EXPECT_LOG_MESSAGE() or similar is something we've wanted for a while. I think that the KASAN folks have been working on something similar using console tracepoints:
https://lore.kernel.org/all/ebf96ea600050f00ed567e80505ae8f242633640.1666113393.git.andreyknvl@google.com/

Cheers,
-- David

Quoted 143 lines · Frank Rowand

Quoted 118 lines · David Gow

Quoted 109 lines · Frank Rowand

Quoted 107 lines · Frank Rowand

Quoted 40 lines · David Gow

Quoted 30 lines · Stephen Boyd

Quoting David Gow (2023-03-10 00:09:48)

Quoted 8 lines · Stephen Boyd

Quoted 6 lines

Hmm. I think you're suggesting that the unit test data be loaded whenever CONFIG_OF=y and CONFIG_KUNIT=y. Then tests can check for CONFIG_OF and skip if it isn't enabled?

Quoted 5 lines

More of the opposite: that we should have some way of supporting tests which might want to use a DTB other than the built-in one. Mostly for non-UML situations where an actual devicetree is needed to even boot far enough to get test output (so we wouldn't be able to override it with a compiled-in test one).

Ok, got it.

Quoted 11 lines

I think moving to overlays probably will render this idea obsolete:
but the thought was to give test code a way to check for the required devicetree nodes at runtime, and skip the test if they weren't found. That way, the failure mode for trying to boot this on something which required another device tree for, e.g., serial, would be "these tests are skipped because the wrong device tree is loaded", not "I get no output because serial isn't working".

Again, though, it's only really needed for non-UML, and just loading overlays as needed should be much more sensible anyway.

Quoted 8 lines

I still have one niggle here. Loading overlays requires CONFIG_OF_OVERLAY, and the overlay loading API returns -ENOTSUPP when CONFIG_OF_OVERLAY=n. For now I'm checking for the config being enabled in each test, but I'm thinking it may be better to simply call kunit_skip() from the overlay loading function if the config is disabled. This way tests can simply call the overlay loading function and we'll halt the test immediately if the config isn't enabled.

Quoted 51 lines

That sounds sensible, though there is a potential pitfall. If kunit_skip() is called directly from overlay code, might introduce a dependency on kunit.ko from the DT overlay, which we might not want. The solution there is either to have a kunit wrapper function (so the call is already in kunit.ko), or to have a hook to skip the current test (which probably makes sense to do anyway, but I think the wrapper is the better option).

Quoted 42 lines

Quoted 9 lines

Quoted 9 lines

Quoted 8 lines

That being said, I do think that there's probably some sense in supporting the compiled-in DTB as well (it's definitely simpler than patching kunit.py to always pass the extra command-line option in, for example).
But maybe it'd be nice to have the command-line option override the built-in one if present.

Quoted 7 lines

Quoted 4 lines

Got it. I need to test loading another DTB on the commandline still, but I think this won't be a problem. We'll load the unittest-data DTB even with KUnit on UML, so assuming that works on UML right now it should be unchanged by this series once I resend.

Again, moving to overlays should render this mostly obsolete, no? Or am I misunderstanding how the overlay stuff will work?

Right, overlays make it largely a moot issue. The way the OF unit tests work today is by grafting a DTB onto the live tree. I'm reusing that logic to graft a container node target for kunit tests to add their overlays too. It will be clearer once I post v2.

Quoted 7 lines

One possible future advantage of being able to test with custom DTs at boot time would be for fuzzing (provide random DT properties, see what happens in the test). We've got some vague plans to support a way of passing custom data to tests to support this kind of case (though, if we're using overlays, maybe the test could just patch those if we wanted to do that).

Ah ok. I can see someone making a fuzzer that modifies devicetree properties randomly, e.g. using different strings for clock-names.

This reminds me of another issue I ran into. I wanted to test adding the same platform device to the platform bus twice to confirm that the second device can't be added. That prints a warning, which makes kunit.py think that the test has failed because it printed a warning. Is there some way to avoid that? I want something like

KUNIT_EXPECT_WARNING(test, <call some function>)

so I can test error cases.

DT unittests already have a similar concept. A test can report that a kernel warning (or any other specific text) either (1) must occur for the test to pass or (2) must _not_ occur for the test to pass. The check for the kernel warning is done by the test output parsing program scripts/dtc/of_unittest_expect.

The reporting by a test of an expected error in drivers/of/unittest.c is done by EXPECT_BEGIN() and EXPECT_END(). These have been in unittest for a long time.

The reporting by a test of a not expected to occur error is done by EXPECT_NOT_BEGIN() and EXPECT_NOT_END(). These are added to unittest in linux 6.3-rc1.

Quoted 7 lines

Quoted 1 line

I discussed this concept in one of the early TAP / KTAP discussion

The link to the early KTAP discussion on this concept is:

   https://lore.kernel.org/all/[email protected]/T/#u

Quoted 8 lines

Thanks -- I'd totally forgotten about that!

I still personally would prefer a way of checking this from within the kernel, as if we're just printing out "EXPECT: " lines, then it's not possible to know if a test passes just from the raw results (and things like statistics can't be updated without a separate tool like kunit.py parsing the KTAP.

Yes, I totally agree with that. If there is a reasonable way to implement. But in the DT unittest world, I have not found a reasonable way. Adding hooks is suggested below, but for DT unittest _I_ (opinion) do not find that reasonable. I voice no vote for kunit - that decision is up to the kunit crowd.

Quoted 8 lines

Indeed, my personal preference is that this log-based way of doing expectations is probably best kept as a last resort. i.e.,
1. Try to add a hook to the code which prints the message, which can then fail the test (or set a flag for the test to check later). This probably needs some better KUnit-side helpers to be truly ergonomic, but at least avoids too strict a dependency on the exact formatting of the log messages.

Quoted 6 lines

I'm not a fan of hooks. I see them as a maintenance burden, dependent upon the source version of the object being tested, yet another thing that can go wrong, and adds complexity to creating a test environment and running the test. Again, this just a personal opinion, and I'm not voting for or against this for kunit.

Quoted 6 lines

Quoted 3 lines

2. If that doesn't work, use console tracepoints or similar to implement an EXPECT_BEGIN() / EXPECT_END() or similar API entirely within the kernel.

Isn't this just another hook? So same opinion.

Quoted 3 lines

Quoted 3 lines

3. Only if we can't come up with a working way of doing the former options, resort to adding "EXPECT:" lines and having a parser pick up on this.

Adding one more thought here so I don't forget it before the topic picks up again in the KTAP version 2 world...

The test parser could generate an artificial subtest test case status line or normal test case status line to report the result of the EXPECT. This also is ugly because it is creating a new requirement on the parser vs the expectations in the KTAP plan line (the plan line could include the EXPECT in the number of tests count, but then the raw KTAP test output would be missing the artificial EXPECT test result). No need to hash out details here, just a thought...

-Frank

Quoted 27 lines

Again, don't let my opinion affect the voting between 1, 2, 3, or other for kunit.

Quoted 6 lines

One of the downsides of doing "EXPECT" lines in KTAP is that it'll suddenly be much more dependent on the exact layout of the tests, as we'd need to be able to override a test result if an expectation fails (at least, to maintain the KUnit structure). And overriding a result which is already in the output seems really, really ugly.

I don't understand "dependent on the exact layout of the tests". If you are saying that the test result parser has to figure out which test result to override, that has not been an issue in the cases that I use EXPECTs in DT unittest. The EXPECT begin and EXPECT end have always immediately surrounded a single test, so when the parser processes the EXPECT end, only the most recent test result could be overridden. This has worked because the kernel warning and error messages have been from kernel action that happens synchronously with the test. If the test prods the kernel in a way that results in the kernel performing an asynchronous activity (eg in another thread), then it becomes more complex to structure the EXPECT end -- I would imagine that the test would have to block on the asynchronous activity just before reporting the normal KTAP status result for the test (and the EXPECT end would normally be just after reporting the KTAP status result for the test).

Quoted 10 lines

I agree with overriding being ugly. For the DT unittest results parser, the EXPECT summary results are reported separately from the individual test summary results. The parser also flags the EXPECT failure in line with the normal individual test result lines.

I see both parsing results as valid, and as a policy choice for each test parser.

Quoted 15 lines

Quoted 15 lines

There's a patch to the KASAN tests to move from doing option 1 to option 2 above (in order to better support RCU, which didn't work with the hook):
https://lore.kernel.org/all/ebf96ea600050f00ed567e80505ae8f242633640.1666113393.git.andreyknvl@google.com/

Quoted 8 lines

Quoted 5 lines

threads and expect to start a discussion thread on this specific topic in the KTAP Specification V2 context. I expect the discussion to result in a different implementation than what DT unittests are using (bike shedding likely to ensue) but whatever is agreed to should be easy for DT to switch to.

The link to the KTAP Specification Version 2 process and progress is:

Quoted 8 lines

Quoted 5 lines

Quoted 2 lines

https://elinux.org/Test_Results_Format_Notes#KTAP_version_2

Thanks! We've got a few more KTAP ideas to air, so will hopefully send those out soon!

Glad to hear, I'm hoping that process starts progressing a bit.

Quoted 23 lines

-Frank

Quoted 20 lines

Cheers,
-- David

Quoted 16 lines

Quoted 15 lines

Quoted 14 lines

Hmm... I'd've thought that shouldn't be a problem: kunit.py should ignore most messages during a test, unless it can't find a valid result line. What does the raw KTAP output look like? (You can get it from kunit.py by passing the --raw_output option).

That being said, a KUNIT_EXPECT_LOG_MESSAGE() or similar is something we've wanted for a while. I think that the KASAN folks have been working on something similar using console tracepoints:
https://lore.kernel.org/all/ebf96ea600050f00ed567e80505ae8f242633640.1666113393.git.andreyknvl@google.com/

Cheers,
-- David

Hmm... I'd've thought that shouldn't be a problem: kunit.py should ignore most messages during a test, unless it can't find a valid result line. What does the raw KTAP output look like? (You can get it from kunit.py by passing the --raw_output option).

To fully exercise common clk framework code in KUnit we need to associate 'struct device' pointers with 'struct device_node' pointers so that things like clk_get() can parse DT nodes for 'clocks' and so that clk providers can use DT to provide clks; the most common mode of operation for clk providers.

Adding support to KUnit so that it loads a DTB is fairly simple after commit b31297f04e86 ("um: Add devicetree support"). We can simply pass a pre-compiled deviectree blob (DTB) on the kunit.py commandline and UML will load it. The problem is that tests won't know that the commandline has been modified, nor that a DTB has been loaded. Take a different approach so that tests can skip if a DTB hasn't been loaded.

Reuse the Makefile logic from the OF unittests to build a DTB into the kernel. This DTB will be for the mythical machine "linux,kunit", i.e. the devicetree for the KUnit "board". In practice, it is a dtsi file that will gather includes for kunit tests that rely in part on a devicetree being loaded. The devicetree should only be loaded if CONFIG_OF_KUNIT=y. Make that a choice config parallel to the existing CONFIG_OF_UNITTEST so that only one devicetree can be loaded in the system at a time. Similarly, the kernel commandline option to load a DTB is ignored if CONFIG_OF_KUNIT is enabled so that only one DTB is loaded at a time.

Add a simple unit test to confirm that the DTB loading worked. Future tests will add to the kunit.dtsi file to include their specific test nodes.

Cc Richard Weinberger <[email protected]>

Cc Anton Ivanov <[email protected]>

Cc Johannes Berg <[email protected]>

Cc Vincent Whitchurch <[email protected]>

Cc Rob Herring <[email protected]>

Cc Frank Rowand <[email protected]>

Signed-off-by Stephen Boyd <[email protected]>

---
 arch/um/kernel/dtb.c            | 29 +++++++++++++++--
 drivers/of/Kconfig              | 26 ++++++++++++++++
 drivers/of/Makefile             |  1 +
 drivers/of/kunit/.kunitconfig   |  4 +++
 drivers/of/kunit/Makefile       |  4 +++
 drivers/of/kunit/kunit.dtsi     |  8 +++++
 drivers/of/kunit/kunit.dtso     |  4 +++
 drivers/of/kunit/uml_dtb_test.c | 55 +++++++++++++++++++++++++++++++++
 8 files changed, 128 insertions(+), 3 deletions(-)
 create mode 100644 drivers/of/kunit/.kunitconfig
 create mode 100644 drivers/of/kunit/Makefile
 create mode 100644 drivers/of/kunit/kunit.dtsi
 create mode 100644 drivers/of/kunit/kunit.dtso
 create mode 100644 drivers/of/kunit/uml_dtb_test.c

arch/um/kernel/dtb.c +26 -3

index 484141b06938..ee63951b12df 100644
@@ -15,9 +15,32 @@ void uml_dtb_init(void)
 	long long size;
 	void *area;
 
-	area = uml_load_file(dtb, &size);
-	if (!area)
-		return;
+	if (IS_ENABLED(CONFIG_OF_KUNIT)) {
+		/*
+		 * __dtbo_kunit_begin[] and __dtbo_kunit_end[] are magically
+		 * created by cmd_dt_S_dtbo in scripts/Makefile.lib from the
+		 * drivers/of/kunit/kunit.dtsi file.
+		 */
+		extern uint8_t __dtbo_kunit_begin[];
+		extern uint8_t __dtbo_kunit_end[];
+
+		size = __dtbo_kunit_end - __dtbo_kunit_begin;
+		if (!size) {
+			pr_warn("%s: kunit testcases is empty\n", __func__);
+			return;
+		}
+
+		/* creating copy */
+		area = memblock_alloc(size, 8);
+		if (!area)
+			return;
+
+		memcpy(area, __dtbo_kunit_begin, size);
+	} else {
+		area = uml_load_file(dtb, &size);
+		if (!area)
+			return;
+	}
 
 	if (!early_init_dt_scan(area)) {
 		pr_err("invalid DTB %s\n", dtb);

drivers/of/Kconfig +0 -0

index 80b5fd44ab1c..1f968b6a3dde 100644

Introduce KUnit resource wrappers around platform_driver_register(), platform_device_alloc(), and platform_device_add() so that test authors can register platform drivers/devices from their tests and have the drivers/devices automatically be unregistered when the test is done.

This makes test setup code simpler when a platform driver or platform device is needed. Add a few test cases at the same time to make sure the APIs work as intended.

Cc Brendan Higgins <[email protected]>

Cc David Gow <[email protected]>

Cc Greg Kroah-Hartman <[email protected]>

Cc "Rafael J. Wysocki" <[email protected]>

Signed-off-by Stephen Boyd <[email protected]>

---

Should this be moved to drivers/base/ and called platform_kunit.c? The include/kunit/platform_driver.h could also be kunit/platform_device.h to match linux/platform_device.h if that is more familiar.

And I'm not super certain about allocating a driver structure and embedding it in a wrapper struct. Maybe the code should just use kunit_get_current_test() instead?

 include/kunit/platform_driver.h  |  15 +++
 lib/kunit/Makefile               |   6 +
 lib/kunit/platform_driver-test.c | 107 ++++++++++++++++
 lib/kunit/platform_driver.c      | 207 +++++++++++++++++++++++++++++++
 4 files changed, 335 insertions(+)
 create mode 100644 include/kunit/platform_driver.h
 create mode 100644 lib/kunit/platform_driver-test.c
 create mode 100644 lib/kunit/platform_driver.c

include/kunit/platform_driver.h new file +15 -0

new file mode 100644
index 000000000000..dc211ff8f893
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _KUNIT_PLATFORM_DRIVER_H
+#define _KUNIT_PLATFORM_DRIVER_H
+
+struct kunit;
+struct platform_device;
+struct platform_driver;
+
+struct platform_device *
+kunit_platform_device_alloc(struct kunit *test, const char *name, int id);
+int kunit_platform_device_add(struct kunit *test, struct platform_device *pdev);
+
+int kunit_platform_driver_register(struct kunit *test, struct platform_driver *drv);
+
+#endif

lib/kunit/Makefile +6 -0

index 29aff6562b42..5964d8231ff5 100644
@@ -1,5 +1,6 @@
 obj-$(CONFIG_KUNIT) +=                 kunit.o
 
+# Core KUnit code
 kunit-objs +=                          test.o \
                                        resource.o \
                                        string-stream.o \

@@ -11,7 +12,12 @@ ifeq ($(CONFIG_KUNIT_DEBUGFS),y)
 kunit-objs +=                          debugfs.o
 endif
 
+# KUnit helpers
+kunit-objs +=                          platform_driver.o
+
+# KUnit tests
 obj-$(CONFIG_KUNIT_TEST) +=            kunit-test.o
+obj-$(CONFIG_KUNIT_TEST) +=            platform_driver-test.o
 
 # string-stream-test compiles built-in only.
 ifeq ($(CONFIG_KUNIT_TEST),y)

lib/kunit/platform_driver-test.c new file +107 -0

new file mode 100644
index 000000000000..c926fe01b40a
@@ -0,0 +1,107 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KUnit test for platform driver infrastructure.
+ */
+
+#include <linux/platform_device.h>
+
+#include <kunit/platform_driver.h>
+#include <kunit/test.h>
+
+/*
+ * Test that kunit_platform_device_alloc() creates a platform device.
+ */
+static void kunit_platform_device_alloc_test(struct kunit *test)
+{
+       KUNIT_EXPECT_NOT_ERR_OR_NULL(test,
+                       kunit_platform_device_alloc(test, "kunit-platform", 1));
+}
+
+/*
+ * Test that kunit_platform_device_add() registers a platform device on the
+ * platform bus with the proper name and id.
+ */
+static void kunit_platform_device_add_test(struct kunit *test)
+{
+       struct platform_device *pdev;
+       const char *name = "kunit-platform";
+       const int id = -1;
+
+       pdev = kunit_platform_device_alloc(test, name, id);
+       KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pdev);
+
+       KUNIT_EXPECT_EQ(test, 0, kunit_platform_device_add(test, pdev));
+       KUNIT_EXPECT_TRUE(test, dev_is_platform(&pdev->dev));
+       KUNIT_EXPECT_STREQ(test, pdev->name, name);
+       KUNIT_EXPECT_EQ(test, pdev->id, id);
+}
+
+static struct kunit_case kunit_platform_device_test_cases[] = {
+       KUNIT_CASE(kunit_platform_device_alloc_test),
+       KUNIT_CASE(kunit_platform_device_add_test),
+       {}
+};
+
+static struct kunit_suite kunit_platform_device_suite = {
+       .name = "kunit_platform_device",
+       .test_cases = kunit_platform_device_test_cases,
+};
+
+struct kunit_platform_driver_test_context {
+       struct platform_driver pdrv;
+       const char *data;
+};
+
+static inline struct kunit_platform_driver_test_context *
+to_test_context(struct platform_device *pdev)
+{
+       return container_of(to_platform_driver(pdev->dev.driver),
+                           struct kunit_platform_driver_test_context,
+                           pdrv);
+}
+
+static int kunit_platform_driver_probe(struct platform_device *pdev)
+{
+       struct kunit_platform_driver_test_context *ctx;
+
+       ctx = to_test_context(pdev);
+       ctx->data = "test data";
+
+       return 0;
+}
+
+/* Test that kunit_platform_driver_register() registers a driver that probes. */
+static void kunit_platform_driver_register_test(struct kunit *test)
+{
+       struct platform_device *pdev;
+       struct kunit_platform_driver_test_context *ctx;
+
+       ctx = kunit_kzalloc(test, sizeof(*ctx), GFP_KERNEL);
+       KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ctx);
+
+       pdev = kunit_platform_device_alloc(test, "kunit-platform", -1);
+       KUNIT_ASSERT_NOT_ERR_OR_NULL(test, pdev);
+       KUNIT_ASSERT_EQ(test, 0, kunit_platform_device_add(test, pdev));
+
+       ctx->pdrv.probe = kunit_platform_driver_probe;
+       ctx->pdrv.driver.name = "kunit-platform";
+       ctx->pdrv.driver.owner = THIS_MODULE;
+
+       KUNIT_EXPECT_EQ(test, 0, kunit_platform_driver_register(test, &ctx->pdrv));
+       KUNIT_EXPECT_STREQ(test, ctx->data, "test data");
+}
+
+static struct kunit_case kunit_platform_driver_test_cases[] = {
+       KUNIT_CASE(kunit_platform_driver_register_test),
+       {}
+};
+
+static struct kunit_suite kunit_platform_driver_suite = {
+       .name = "kunit_platform_driver",
+       .test_cases = kunit_platform_driver_test_cases,
+};
+
+kunit_test_suites(&kunit_platform_device_suite,
+                 &kunit_platform_driver_suite);
+
+MODULE_LICENSE("GPL");

lib/kunit/platform_driver.c new file +4 -0

new file mode 100644
index 000000000000..11d155114936
@@ -0,0 +1,207 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Test managed platform driver
+ */

lib/kunit/platform_driver.c new file +193 -1

+
+static int kunit_platform_device_alloc_init(struct kunit_resource *res, void *context)
+{
+       struct kunit_platform_device_alloc_params *params = context;
+       struct platform_device *pdev;
+
+       pdev = platform_device_alloc(params->name, params->id);
+       if (!pdev)
+               return -ENOMEM;
+
+       res->data = pdev;
+
+       return 0;
+}
+
+static void kunit_platform_device_alloc_exit(struct kunit_resource *res)
+{
+       struct platform_device *pdev = res->data;
+
+       platform_device_put(pdev);
+}
+
+/**
+ * kunit_platform_device_alloc() - Allocate a KUnit test managed platform device
+ * @test: test context
+ * @dev: platform device to alloc
+ *
+ * Register a test managed platform device. The device is put when the test completes.
+ *
+ * Returns: 0 on success, negative errno on failure.
+ */
+struct platform_device *
+kunit_platform_device_alloc(struct kunit *test, const char *name, int id)
+{
+       struct platform_device *pdev;
+       struct kunit_platform_device_alloc_params params = {
+               .name = name,
+               .id = id,
+       };
+
+       pdev = kunit_alloc_resource(test,
+                                  kunit_platform_device_alloc_init,
+                                  kunit_platform_device_alloc_exit,
+                                  GFP_KERNEL, &params);
+       if (!pdev)
+               return ERR_PTR(-ENOMEM);
+
+       return pdev;
+}
+EXPORT_SYMBOL_GPL(kunit_platform_device_alloc);
+
+static int kunit_platform_device_add_init(struct kunit_resource *res, void *context)
+{
+       struct platform_device *pdev = context;
+       int ret;
+
+       ret = platform_device_add(pdev);
+       if (ret) {
+               platform_device_put(pdev);
+               return ret;
+       }
+       res->data = pdev;
+
+       return 0;
+}
+
+static void kunit_platform_device_add_exit(struct kunit_resource *res)
+{
+       struct platform_device *pdev = res->data;
+
+       platform_device_unregister(pdev);
+}
+
+/**
+ * kunit_platform_device_add() - Register a KUnit test managed platform device
+ * @test: test context
+ * @dev: platform device to add
+ *
+ * Register a test managed platform device. The device is unregistered when the
+ * test completes.
+ *
+ * Returns: 0 on success, negative errno on failure.
+ */
+int kunit_platform_device_add(struct kunit *test, struct platform_device *pdev)
+{
+       struct platform_device *res;
+
+       res = kunit_alloc_resource(test,
+                                  kunit_platform_device_add_init,
+                                  kunit_platform_device_add_exit,
+                                  GFP_KERNEL, pdev);
+       if (!res)
+               return -EINVAL;
+
+       return 0;
+}
+EXPORT_SYMBOL_GPL(kunit_platform_device_add);
+
+static int kunit_platform_driver_register_init(struct kunit_resource *res, void *context)
+{
+       struct platform_driver *drv = context;
+       int ret;
+
+       ret = platform_driver_register(drv);
+       if (ret)
+               return ret;
+       res->data = drv;
+
+       /*
+        * Wait for the driver to probe (or at least flush out of the deferred
+        * workqueue)
+        */
+       wait_for_device_probe();
+
+       return 0;
+}
+
+static void kunit_platform_driver_register_exit(struct kunit_resource *res)
+{
+       struct platform_driver *drv = res->data;
+
+       platform_driver_unregister(drv);
+}
+
+/**
+ * kunit_platform_driver_register() - Register a KUnit test managed platform driver
+ * @test: test context
+ * @drv: platform driver to register
+ *
+ * Register a test managed platform driver. This allows callers to embed the
+ * @drv in a container structure and use container_of() in the probe function
+ * to pass information to kunit tests. It can be assumed that the driver has
+ * probed when this function returns.
+ *
+ * Example:
+ *
+ * .. code-block:: c
+ *
+ *     struct kunit_test_context {
+ *             struct platform_driver pdrv;
+ *             const char *data;
+ *     };
+ *
+ *     static inline struct kunit_test_context *
+ *     to_test_context(struct platform_device *pdev)
+ *     {
+ *             return container_of(to_platform_driver(pdev->dev.driver),
+ *                                 struct kunit_test_context,
+ *                                 pdrv);
+ *     }
+ *
+ *     static int kunit_platform_driver_probe(struct platform_device *pdev)
+ *     {
+ *             struct kunit_test_context *ctx;
+ *
+ *             ctx = to_test_context(pdev);
+ *             ctx->data = "test data";
+ *
+ *             return 0;
+ *     }
+ *
+ *     static void kunit_platform_driver_test(struct kunit *test)
+ *     {
+ *             struct kunit_test_context *ctx;
+ *
+ *             ctx = kunit_kzalloc(test, sizeof(*ctx), GFP_KERNEL);
+ *             KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ctx);
+ *
+ *             ctx->pdrv.probe = kunit_platform_driver_probe;
+ *             ctx->pdrv.driver.name = "kunit-platform";
+ *             ctx->pdrv.driver.owner = THIS_MODULE;
+ *
+ *             KUNIT_EXPECT_EQ(test, 0, kunit_platform_driver_register(test, &ctx->pdrv));
+ *             KUNIT_EXPECT_STREQ(test, ctx->data, "test data");
+ *     }
+ *
+ * Returns: 0 on success, negative errno on failure.
+ */
+int kunit_platform_driver_register(struct kunit *test,
+                                  struct platform_driver *drv)
+{
+       struct platform_driver *res;
+
+       res = kunit_alloc_resource(test,
+                                  kunit_platform_driver_register_init,
+                                  kunit_platform_driver_register_exit,
+                                  GFP_KERNEL, drv);
+       if (!res)
+               return -EINVAL;
+
+       return 0;
+}
+EXPORT_SYMBOL_GPL(kunit_platform_driver_register);
--

https://git.kernel.org/pub/scm/linux/kernel/git/clk/linux.git/
https://git.kernel.org/pub/scm/linux/kernel/git/sboyd/spmi.git

Quoted 15 lines · Stephen Boyd

Introduce KUnit resource wrappers around platform_driver_register(), platform_device_alloc(), and platform_device_add() so that test authors can register platform drivers/devices from their tests and have the drivers/devices automatically be unregistered when the test is done.

This makes test setup code simpler when a platform driver or platform device is needed. Add a few test cases at the same time to make sure the APIs work as intended.

Cc Brendan Higgins <[email protected]>

Cc David Gow <[email protected]>

Cc Greg Kroah-Hartman <[email protected]>

Cc "Rafael J. Wysocki" <[email protected]>

Signed-off-by Stephen Boyd <[email protected]>

Quoted 6 lines

---

Should this be moved to drivers/base/ and called platform_kunit.c? The include/kunit/platform_driver.h could also be kunit/platform_device.h to match linux/platform_device.h if that is more familiar.

DRM has a similar thing already (albeit with a root_device, which is more common with KUnit tests generally):
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/drm/drm_kunit_helpers.h

But that's reasonably drm-specific, so it makes sense that it lives with DRM stuff. platform_device is a bit more generic.

Quoted 7 lines

DRM has a similar thing already (albeit with a root_device, which is more common with KUnit tests generally):
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/drm/drm_kunit_helpers.h

But that's reasonably drm-specific, so it makes sense that it lives with DRM stuff. platform_device is a bit more generic.

I'd be very happy to get something from the core to address the same thing.

I think the main thing we needed that isn't covered by this patch is we wanted the device to be bound to its driver, so with probe being called before calling the test (see 57a84a97bbda).

Quoting Maxime Ripard (2023-03-03 06:35:28)

Quoted 12 lines · David Gow

Quoted 7 lines

DRM has a similar thing already (albeit with a root_device, which is more common with KUnit tests generally):
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/drm/drm_kunit_helpers.h

But that's reasonably drm-specific, so it makes sense that it lives with DRM stuff. platform_device is a bit more generic.

I'd be very happy to get something from the core to address the same thing.

Quoted 3 lines

I think the main thing we needed that isn't covered by this patch is we wanted the device to be bound to its driver, so with probe being called before calling the test (see 57a84a97bbda).

Can you clarify? This patch makes a poor attempt at waiting for the platform driver to bind, but in reality it may not be bound by the time the driver register function returns.

Quoted 15 lines

Introduce KUnit resource wrappers around platform_driver_register(), platform_device_alloc(), and platform_device_add() so that test authors can register platform drivers/devices from their tests and have the drivers/devices automatically be unregistered when the test is done.

This makes test setup code simpler when a platform driver or platform device is needed. Add a few test cases at the same time to make sure the APIs work as intended.

Cc Brendan Higgins <[email protected]>

Cc David Gow <[email protected]>

Cc Greg Kroah-Hartman <[email protected]>

Cc "Rafael J. Wysocki" <[email protected]>

Signed-off-by Stephen Boyd <[email protected]>

Quoted 6 lines

---

Should this be moved to drivers/base/ and called platform_kunit.c? The include/kunit/platform_driver.h could also be kunit/platform_device.h to match linux/platform_device.h if that is more familiar.

DRM has a similar thing already (albeit with a root_device, which is more common with KUnit tests generally):
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/drm/drm_kunit_helpers.h

But that's reasonably drm-specific, so it makes sense that it lives with DRM stuff. platform_device is a bit more generic.

I'd probably personally err on the side of having these in drivers/base/, as I think we'll ultimately need similar things for a lot of different devices, and I'd rather not end up with things like USB device helpers living in the lib/kunit directory alongside the "core" KUnit code. But I could be persuaded otherwise.

Quoted 5 lines

And I'm not super certain about allocating a driver structure and embedding it in a wrapper struct. Maybe the code should just use kunit_get_current_test() instead?

I think there are enough cases througout the kernel where device/driver structs are needed that having this makes sense. Combined with the fact that, while kunit_get_current_test() can be used even when KUnit is not loaded, actually doing anything with the resulting struct kunit pointer will probably require (at least for the moment) KUnit functions to be reachable, so would break if CONFIG_KUNIT=m.

So, unless you actually find kunit_get_current_test() and friends to be easier to work with, I'd probably stick with this.

Quoted 17 lines

lib/kunit/platform_driver.c new file +11 -0

new file mode 100644
index 000000000000..11d155114936
@@ -0,0 +1,207 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Test managed platform driver
+ */
+
+#include <linux/device/driver.h>
+#include <linux/platform_device.h>
+
+#include <kunit/resource.h>
+
+struct kunit_platform_device_alloc_params {

lib/kunit/platform_driver.c line 14

Quoted 3 lines

lib/kunit/platform_driver.c new file +3 -0

+       const char *name;
+       int id;
+};

FYI: It's my plan to eventually get rid of (or at least de-emphasize) the whole 'init' function aspect of KUnit resources so we don't need all of these extra structs and the like. It probably won't make it in for 6.4, but we'll see...

Quoting David Gow (2023-03-02 23:15:31)

Quoted 27 lines · Stephen Boyd

Quoted 15 lines

Introduce KUnit resource wrappers around platform_driver_register(), platform_device_alloc(), and platform_device_add() so that test authors can register platform drivers/devices from their tests and have the drivers/devices automatically be unregistered when the test is done.

This makes test setup code simpler when a platform driver or platform device is needed. Add a few test cases at the same time to make sure the APIs work as intended.

Cc Brendan Higgins <[email protected]>

Cc David Gow <[email protected]>

Cc Greg Kroah-Hartman <[email protected]>

Cc "Rafael J. Wysocki" <[email protected]>

Signed-off-by Stephen Boyd <[email protected]>

Quoted 6 lines

---

Should this be moved to drivers/base/ and called platform_kunit.c? The include/kunit/platform_driver.h could also be kunit/platform_device.h to match linux/platform_device.h if that is more familiar.

DRM has a similar thing already (albeit with a root_device, which is more common with KUnit tests generally):
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/drm/drm_kunit_helpers.h

Quoted 8 lines

But that's reasonably drm-specific, so it makes sense that it lives with DRM stuff. platform_device is a bit more generic.

I'd probably personally err on the side of having these in drivers/base/, as I think we'll ultimately need similar things for a lot of different devices, and I'd rather not end up with things like USB device helpers living in the lib/kunit directory alongside the "core" KUnit code. But I could be persuaded otherwise.

Ok no problem. I'll move it.

Quoted 6 lines

Quoted 5 lines

And I'm not super certain about allocating a driver structure and embedding it in a wrapper struct. Maybe the code should just use kunit_get_current_test() instead?

Quoted 7 lines

I think there are enough cases througout the kernel where device/driver structs are needed that having this makes sense. Combined with the fact that, while kunit_get_current_test() can be used even when KUnit is not loaded, actually doing anything with the resulting struct kunit pointer will probably require (at least for the moment) KUnit functions to be reachable, so would break if CONFIG_KUNIT=m.

Wouldn't it still work in that case? The unit tests would be modular as well because they depend on CONFIG_KUNIT.

Quoted 4 lines

So, unless you actually find kunit_get_current_test() and friends to be easier to work with, I'd probably stick with this.

Alright thanks.

Quoted 17 lines

Quoted 17 lines

lib/kunit/platform_driver.c new file +11 -0

new file mode 100644
index 000000000000..11d155114936
@@ -0,0 +1,207 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Test managed platform driver
+ */
+
+#include <linux/device/driver.h>
+#include <linux/platform_device.h>
+
+#include <kunit/resource.h>
+
+struct kunit_platform_device_alloc_params {

Quoted 8 lines

lib/kunit/platform_driver.c line 14

Quoted 3 lines

lib/kunit/platform_driver.c new file +3 -0

+       const char *name;
+       int id;
+};

FYI: It's my plan to eventually get rid of (or at least de-emphasize) the whole 'init' function aspect of KUnit resources so we don't need all of these extra structs and the like. It probably won't make it in for 6.4, but we'll see...

Will we be able to get the error values out of the init function? It's annoying that the error values can't be returned as error pointers to kunit_alloc_resource(). I end up skipping init, and doing it directly before or after calling the kunit_alloc_resource() function. I'll try to avoid init functions in the allocations.

Unit tests are more ergonomic and simpler to understand if they don't have to hoist a bunch of code into the test harness init and exit functions. Add some test managed wrappers for the clk APIs so that clk unit tests can write more code in the actual test and less code in the harness.

Only add APIs that are used for now. More wrappers can be added in the future as necessary.

Cc Brendan Higgins <[email protected]>

Cc David Gow <[email protected]>

Signed-off-by Stephen Boyd <[email protected]>

---

 drivers/clk/Makefile    |   5 +
 drivers/clk/clk-kunit.c | 204 ++++++++++++++++++++++++++++++++++++++++
 drivers/clk/clk-kunit.h |  28 ++++++
 3 files changed, 237 insertions(+)
 create mode 100644 drivers/clk/clk-kunit.c
 create mode 100644 drivers/clk/clk-kunit.h

drivers/clk/Makefile +0 -0

index e3ca0d058a25..7efce649b0d3 100644

drivers/clk/Makefile +2 -0

+endif
+
 # hardware specific clock types
 # please keep this section sorted lexicographically by file path name
 obj-$(CONFIG_COMMON_CLK_APPLE_NCO)     += clk-apple-nco.o

drivers/clk/clk-kunit.c new file +5 -0

new file mode 100644
index 000000000000..78d85b3a7a4a
@@ -0,0 +1,204 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KUnit helpers for clk tests
+ */
+#include <linux/clk.h>

drivers/clk/clk-kunit.c new file +189 -0

+{
+       struct clk *clk = res->data;
+
+       clk_disable_unprepare(clk);
+}
+
+/**
+ * kunit_clk_prepare_enable() - Test managed clk_prepare_enable()
+ * @test: The test context
+ * @clk: clk to prepare and enable
+ *
+ * Returns: 0 on success, or negative errno on failure.
+ */
+int kunit_clk_prepare_enable(struct kunit *test, struct clk *clk)
+{
+       if (!kunit_alloc_resource(test, NULL, kunit_clk_disable_unprepare,
+                                 GFP_KERNEL, clk))
+               return -EINVAL;
+
+       return clk_prepare_enable(clk);
+}
+EXPORT_SYMBOL_GPL(kunit_clk_prepare_enable);
+
+static void kunit_clk_put(struct kunit_resource *res)
+{
+       struct clk *clk = res->data;
+
+       clk_put(clk);
+}
+
+/**
+ * kunit_clk_get() - Test managed clk_get()
+ * @test: The test context
+ * @dev: device for clock "consumer"
+ * @id: clock consumer ID
+ *
+ * Returns: new clk consumer or ERR_PTR on failure
+ */
+struct clk *
+kunit_clk_get(struct kunit *test, struct device *dev, const char *con_id)
+{
+       struct clk *clk;
+
+       clk = clk_get(dev, con_id);
+       if (IS_ERR(clk))
+               return clk;
+
+       if (!kunit_alloc_resource(test, NULL, kunit_clk_put, GFP_KERNEL, clk)) {
+               clk_put(clk);
+               return ERR_PTR(-EINVAL);
+       }
+
+       return clk;
+}
+EXPORT_SYMBOL_GPL(kunit_clk_get);
+
+/**
+ * kunit_of_clk_get() - Test managed of_clk_get()
+ * @test: The test context
+ * @np: device_node for clock "consumer"
+ * @index: index in 'clocks' property of @np
+ *
+ * Returns: new clk consumer or ERR_PTR on failure
+ */
+struct clk *
+kunit_of_clk_get(struct kunit *test, struct device_node *np, int index)
+{
+       struct clk *clk;
+
+       clk = of_clk_get(np, index);
+       if (IS_ERR(clk))
+               return clk;
+
+       if (!kunit_alloc_resource(test, NULL, kunit_clk_put, GFP_KERNEL, clk)) {
+               clk_put(clk);
+               return ERR_PTR(-EINVAL);
+       }
+
+       return clk;
+}
+EXPORT_SYMBOL_GPL(kunit_of_clk_get);
+
+/**
+ * kunit_clk_hw_get_clk() - Test managed clk_hw_get_clk()
+ * @test: The test context
+ * @hw: clk_hw associated with the clk being consumed
+ * @con_id: connection ID string on device
+ *
+ * Returns: new clk consumer or ERR_PTR on failure
+ */
+struct clk *
+kunit_clk_hw_get_clk(struct kunit *test, struct clk_hw *hw, const char *con_id)
+{
+       struct clk *clk;
+
+       clk = clk_hw_get_clk(hw, con_id);
+       if (IS_ERR(clk))
+               return clk;
+
+       if (!kunit_alloc_resource(test, NULL, kunit_clk_put, GFP_KERNEL, clk)) {
+               clk_put(clk);
+               return ERR_PTR(-EINVAL);
+       }
+
+       return clk;
+}
+EXPORT_SYMBOL_GPL(kunit_clk_hw_get_clk);
+
+/**
+ * kunit_clk_hw_get_clk_prepared_enabled() - Test managed clk_hw_get_clk() + clk_prepare_enable()
+ * @test: The test context
+ * @hw: clk_hw associated with the clk being consumed
+ * @con_id: connection ID string on device
+ *
+ * Returns: new clk consumer that is prepared and enabled or ERR_PTR on failure
+ */
+struct clk *
+kunit_clk_hw_get_clk_prepared_enabled(struct kunit *test, struct clk_hw *hw,
+                                     const char *con_id)
+{
+       int ret;
+       struct clk *clk;
+
+       clk = kunit_clk_hw_get_clk(test, hw, con_id);
+       if (IS_ERR(clk))
+               return clk;
+
+       ret = kunit_clk_prepare_enable(test, clk);
+       if (ret)
+               return ERR_PTR(ret);
+
+       return clk;
+}
+EXPORT_SYMBOL_GPL(kunit_clk_hw_get_clk_prepared_enabled);
+
+static void kunit_clk_hw_unregister(struct kunit_resource *res)
+{
+       struct clk_hw *hw = res->data;
+
+       clk_hw_unregister(hw);
+}
+
+/**
+ * kunit_clk_hw_register() - Test managed clk_hw_register()
+ * @test: The test context
+ * @dev: device that is registering this clock
+ * @hw: link to hardware-specific clock data
+ *
+ * Returns: 0 on success or a negative errno value on failure
+ */
+int kunit_clk_hw_register(struct kunit *test, struct device *dev, struct clk_hw *hw)
+{
+       int ret;
+
+       ret = clk_hw_register(dev, hw);
+       if (ret)
+               return ret;
+
+       if (!kunit_alloc_resource(test, NULL, kunit_clk_hw_unregister, GFP_KERNEL, hw)) {
+               clk_hw_unregister(hw);
+               return -EINVAL;
+       }
+
+       return 0;
+}
+
+/**
+ * kunit_of_clk_hw_register() - Test managed of_clk_hw_register()
+ * @test: The test context
+ * @node: device_node of device that is registering this clock
+ * @hw: link to hardware-specific clock data
+ *
+ * Returns: 0 on success or a negative errno value on failure
+ */
+int kunit_of_clk_hw_register(struct kunit *test, struct device_node *node, struct clk_hw *hw)
+{
+       int ret;
+
+       ret = of_clk_hw_register(node, hw);
+       if (ret)
+               return ret;
+
+       if (!kunit_alloc_resource(test, NULL, kunit_clk_hw_unregister, GFP_KERNEL, hw)) {
+               clk_hw_unregister(hw);
+               return -EINVAL;
+       }
+
+       return 0;
+}

drivers/clk/clk-kunit.h new file +28 -1

new file mode 100644
index 000000000000..153597d69269
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _CLK_KUNIT_H
+#define _CLK_KUNIT_H
+
+struct clk;
+struct clk_hw;
+struct device;
+struct device_node;
+struct kunit;
+
+struct clk *
+kunit_clk_get(struct kunit *test, struct device *dev, const char *con_id);
+struct clk *
+kunit_of_clk_get(struct kunit *test, struct device_node *np, int index);
+
+struct clk *
+kunit_clk_hw_get_clk(struct kunit *test, struct clk_hw *hw, const char *con_id);
+struct clk *
+kunit_clk_hw_get_clk_prepared_enabled(struct kunit *test, struct clk_hw *hw,
+                                     const char *con_id);
+
+int kunit_clk_prepare_enable(struct kunit *test, struct clk *clk);
+
+int kunit_clk_hw_register(struct kunit *test, struct device *dev, struct clk_hw *hw);
+int kunit_of_clk_hw_register(struct kunit *test, struct device_node *node,
+                            struct clk_hw *hw);
+
+#endif
--

https://git.kernel.org/pub/scm/linux/kernel/git/clk/linux.git/
https://git.kernel.org/pub/scm/linux/kernel/git/sboyd/spmi.git

Quoted 10 lines

Unit tests are more ergonomic and simpler to understand if they don't have to hoist a bunch of code into the test harness init and exit functions. Add some test managed wrappers for the clk APIs so that clk unit tests can write more code in the actual test and less code in the harness.

Only add APIs that are used for now. More wrappers can be added in the future as necessary.

Quoted 4 lines

Cc Brendan Higgins <[email protected]>

Cc David Gow <[email protected]>

Signed-off-by Stephen Boyd <[email protected]>

---

Looks good, modulo bikeshedding below.

Quoted 9 lines

drivers/clk/Makefile +0 -0

index e3ca0d058a25..7efce649b0d3 100644
@@ -17,6 +17,11 @@ ifeq ($(CONFIG_OF), y)
 obj-$(CONFIG_COMMON_CLK)       += clk-conf.o
 endif
 

drivers/clk/Makefile ifeq ($(CONFIG_OF), y) · line 22

Quoted 3 lines

drivers/clk/Makefile +3 -0

+# KUnit specific helpers
+ifeq ($(CONFIG_COMMON_CLK), y)
+obj-$(CONFIG_KUNIT)            += clk-kunit.o

Do we want to compile these in whenever KUnit is enabled, or only when we're building clk tests specifically? I suspect this would be served better by being under a CLK_KUNIT config option, which all of the tests then depend on. (Whether that's the existing CONFIG_CLK_KUNIT_TEST, and all of the clk tests live under the same config option, or a separate parent option would be up to you).

Equally, this could be a bit interesting if CONFIG_KUNIT=m. Given CONFIG_COMMON_CLK=y, this would end up as a clk-kunit module, no?

Quoted 17 lines

drivers/clk/Makefile +2 -0

+endif
+
 # hardware specific clock types
 # please keep this section sorted lexicographically by file path name
 obj-$(CONFIG_COMMON_CLK_APPLE_NCO)     += clk-apple-nco.o

drivers/clk/clk-kunit.c new file +5 -0

new file mode 100644
index 000000000000..78d85b3a7a4a
@@ -0,0 +1,204 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KUnit helpers for clk tests
+ */
+#include <linux/clk.h>

drivers/clk/clk-kunit.c line 15

Quoted 10 lines

drivers/clk/clk-kunit.c new file +10 -0

+#include <linux/clk-provider.h>
+#include <linux/err.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include <kunit/resource.h>
+
+#include "clk-kunit.h"
+
+static void kunit_clk_disable_unprepare(struct kunit_resource *res)

We need to decide on the naming scheme of these, and in particular if they should be kunit_clk or clk_kunit (or something else).

I'd lean to clk_kunit, if only to match DRM's KUnit helpers being drm_kunit_helper better, and so that these are more tightly bound to the subsystem being tested.
(i.e., so I don't have to scroll through every subsystem's helpers when autocompleting kunit_).

Quoting David Gow (2023-03-02 23:15:35)

Quoted 11 lines · Stephen Boyd

Quoted 10 lines

Unit tests are more ergonomic and simpler to understand if they don't have to hoist a bunch of code into the test harness init and exit functions. Add some test managed wrappers for the clk APIs so that clk unit tests can write more code in the actual test and less code in the harness.

Only add APIs that are used for now. More wrappers can be added in the future as necessary.

Quoted 6 lines

Quoted 4 lines

Cc Brendan Higgins <[email protected]>

Cc David Gow <[email protected]>

Signed-off-by Stephen Boyd <[email protected]>

---

Looks good, modulo bikeshedding below.

Cool!

Quoted 9 lines

Quoted 9 lines

drivers/clk/Makefile +0 -0

index e3ca0d058a25..7efce649b0d3 100644
@@ -17,6 +17,11 @@ ifeq ($(CONFIG_OF), y)
 obj-$(CONFIG_COMMON_CLK)       += clk-conf.o
 endif
 

Quoted 10 lines

drivers/clk/Makefile ifeq ($(CONFIG_OF), y) · line 22

Quoted 3 lines

drivers/clk/Makefile +3 -0

+# KUnit specific helpers
+ifeq ($(CONFIG_COMMON_CLK), y)
+obj-$(CONFIG_KUNIT)            += clk-kunit.o

Do we want to compile these in whenever KUnit is enabled, or only when we're building clk tests specifically? I suspect this would be served better by being under a CLK_KUNIT config option, which all of the tests then depend on. (Whether that's the existing CONFIG_CLK_KUNIT_TEST, and all of the clk tests live under the same config option, or a separate parent option would be up to you).

I was thinking of building it in with whatever mode CONFIG_KUNIT is built as. If this is a module because CONFIG_KUNIT=m, then unit tests would depend on that, and this would be a module as well. modprobe would know that some unit test module depends on symbols provided by clk-kunit.ko and thus load clk-kunit.ko first.

Quoted 3 lines

Equally, this could be a bit interesting if CONFIG_KUNIT=m. Given CONFIG_COMMON_CLK=y, this would end up as a clk-kunit module, no?

Yes, that is the intent.

Quoted 31 lines

Quoted 17 lines

drivers/clk/Makefile +2 -0

+endif
+
 # hardware specific clock types
 # please keep this section sorted lexicographically by file path name
 obj-$(CONFIG_COMMON_CLK_APPLE_NCO)     += clk-apple-nco.o

drivers/clk/clk-kunit.c new file +5 -0

new file mode 100644
index 000000000000..78d85b3a7a4a
@@ -0,0 +1,204 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KUnit helpers for clk tests
+ */
+#include <linux/clk.h>

drivers/clk/clk-kunit.c line 15

Quoted 10 lines

drivers/clk/clk-kunit.c new file +10 -0

+#include <linux/clk-provider.h>
+#include <linux/err.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include <kunit/resource.h>
+
+#include "clk-kunit.h"
+
+static void kunit_clk_disable_unprepare(struct kunit_resource *res)

We need to decide on the naming scheme of these, and in particular if they should be kunit_clk or clk_kunit (or something else).

Quoted 5 lines

I'd lean to clk_kunit, if only to match DRM's KUnit helpers being drm_kunit_helper better, and so that these are more tightly bound to the subsystem being tested.
(i.e., so I don't have to scroll through every subsystem's helpers when autocompleting kunit_).

Ok, got it. I was trying to match kunit_kzalloc() style. It makes it easy to slap the 'kunit_' prefix on existing auto-completed function names like kzalloc() or clk_prepare_enable().

I wasn't aware of drm_kunit_helper. That's a mouthful! We don't call it slab_kunit_helper_kzalloc(). Maybe to satisfy all conditions it should be:

clk_prepare_enable_kunit()

so that kunit_ autocomplete doesn't have a big scroll list, and clk subsystem autocompletes, and we know it is kunit specific.

Quoted 31 lines

Quoted 30 lines

Quoted 11 lines

drivers/clk/clk-kunit.c new file +5 -0

new file mode 100644
index 000000000000..78d85b3a7a4a
@@ -0,0 +1,204 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KUnit helpers for clk tests
+ */
+#include <linux/clk.h>

drivers/clk/clk-kunit.c line 15

Quoted 10 lines

drivers/clk/clk-kunit.c new file +10 -0

+#include <linux/clk-provider.h>
+#include <linux/err.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+
+#include <kunit/resource.h>
+
+#include "clk-kunit.h"
+
+static void kunit_clk_disable_unprepare(struct kunit_resource *res)

We need to decide on the naming scheme of these, and in particular if they should be kunit_clk or clk_kunit (or something else).

I'd lean to clk_kunit, if only to match DRM's KUnit helpers being drm_kunit_helper better, and so that these are more tightly bound to the subsystem being tested.
(i.e., so I don't have to scroll through every subsystem's helpers when autocompleting kunit_).

Quoted 3 lines

Ok, got it. I was trying to match kunit_kzalloc() style. It makes it easy to slap the 'kunit_' prefix on existing auto-completed function names like kzalloc() or clk_prepare_enable().

Yeah: my rule of thumb at the moment is to keep the kunit_ prefix for things which are generic across the whole kernel (and tend to be implemented in lib/kunit), and to use suffixes or infixes (whichever works best) for things which are subsystem-specific.

This patch series adds unit tests for the clk fixed rate basic type and the clk registration functions that use struct clk_parent_data. To get there, we add support for loading a DTB into the UML kernel that's running the unit tests along with probing platform drivers to bind to device nodes specified in DT.

With this series, we're able to exercise some of the code in the common clk framework that uses devicetree lookups to find parents and the fixed rate clk code that scans devicetree directly and creates clks. Please review.

I Cced everyone to all the patches so they get the full context. I'm hoping I can take the whole pile through the clk tree as they almost all depend on each other. In the future I imagine it will be easy to add more test nodes to the clk.dtsi file and not need to go across various maintainer trees like this series does.

Stephen Boyd (8):
dt-bindings: Add linux,kunit binding
of: Enable DTB loading on UML for KUnit tests
kunit: Add test managed platform_device/driver APIs
clk: Add test managed clk provider/consumer APIs
dt-bindings: kunit: Add fixed rate clk consumer test
clk: Add KUnit tests for clk fixed rate basic type
dt-bindings: clk: Add KUnit clk_parent_data test
clk: Add KUnit tests for clks registered with struct clk_parent_data

 .../clock/linux,clk-kunit-parent-data.yaml    |  47 ++
 .../kunit/linux,clk-kunit-fixed-rate.yaml     |  35 ++
 .../bindings/kunit/linux,kunit.yaml           |  24 +
 arch/um/kernel/dtb.c                          |  29 +-
 drivers/clk/.kunitconfig                      |   3 +
 drivers/clk/Kconfig                           |   7 +
 drivers/clk/Makefile                          |   6 +
 drivers/clk/clk-fixed-rate_test.c             | 296 ++++++++++++
 drivers/clk/clk-kunit.c                       | 204 ++++++++
 drivers/clk/clk-kunit.h                       |  28 ++
 drivers/clk/clk_test.c                        | 456 +++++++++++++++++-
 drivers/of/Kconfig                            |  26 +
 drivers/of/Makefile                           |   1 +
 drivers/of/kunit/.kunitconfig                 |   4 +
 drivers/of/kunit/Makefile                     |   4 +
 drivers/of/kunit/clk.dtsi                     |  30 ++
 drivers/of/kunit/kunit.dtsi                   |   9 +
 drivers/of/kunit/kunit.dtso                   |   4 +
 drivers/of/kunit/uml_dtb_test.c               |  55 +++
 include/kunit/platform_driver.h               |  15 +
 lib/kunit/Makefile                            |   6 +
 lib/kunit/platform_driver-test.c              | 107 ++++
 lib/kunit/platform_driver.c                   | 207 ++++++++
 23 files changed, 1599 insertions(+), 4 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/clock/linux,clk-kunit-parent-data.yaml
 create mode 100644 Documentation/devicetree/bindings/kunit/linux,clk-kunit-fixed-rate.yaml
 create mode 100644 Documentation/devicetree/bindings/kunit/linux,kunit.yaml
 create mode 100644 drivers/clk/clk-fixed-rate_test.c
 create mode 100644 drivers/clk/clk-kunit.c
 create mode 100644 drivers/clk/clk-kunit.h
 create mode 100644 drivers/of/kunit/.kunitconfig
 create mode 100644 drivers/of/kunit/Makefile
 create mode 100644 drivers/of/kunit/clk.dtsi
 create mode 100644 drivers/of/kunit/kunit.dtsi
 create mode 100644 drivers/of/kunit/kunit.dtso
 create mode 100644 drivers/of/kunit/uml_dtb_test.c
 create mode 100644 include/kunit/platform_driver.h
 create mode 100644 lib/kunit/platform_driver-test.c
 create mode 100644 lib/kunit/platform_driver.c

base-commit: c9c3395d5e3dcc6daee66c6908354d47bf98cb0c --
https://git.kernel.org/pub/scm/linux/kernel/git/clk/linux.git/
https://git.kernel.org/pub/scm/linux/kernel/git/sboyd/spmi.git

Quoted 7 lines · Stephen Boyd

This patch series adds unit tests for the clk fixed rate basic type and the clk registration functions that use struct clk_parent_data. To get there, we add support for loading a DTB into the UML kernel that's running the unit tests along with probing platform drivers to bind to device nodes specified in DT.

Quoted 5 lines

With this series, we're able to exercise some of the code in the common clk framework that uses devicetree lookups to find parents and the fixed rate clk code that scans devicetree directly and creates clks. Please review.

Thanks Stephen -- this is really neat!

This works well here, and I love all of the tests for the KUnit/device-tree integration as well.

I'm still looking through the details of it (alas, I've mostly lived in x86-land, so my device-tree knowledge is, uh, spotty to say the least), but apart from possibly renaming some things or similarly minor tweaks, I've not got any real suggestions thus far.

I do wonder whether we'll want, on the KUnit side, to have some way of supporting KUnit device trees on non-UML architecctures (e.g., if we need to test something architecture-specific, or on a big-endian platform, etc), but I think that's a question for the future, rather than something that affects this series.

Similarly, I wonder if there's something we could do with device tree overlays, in order to make it possible for tests to swap nodes in and out for testing.

Quoted 25 lines · Stephen Boyd

Quoted 7 lines

This patch series adds unit tests for the clk fixed rate basic type and the clk registration functions that use struct clk_parent_data. To get there, we add support for loading a DTB into the UML kernel that's running the unit tests along with probing platform drivers to bind to device nodes specified in DT.

Quoted 5 lines

With this series, we're able to exercise some of the code in the common clk framework that uses devicetree lookups to find parents and the fixed rate clk code that scans devicetree directly and creates clks. Please review.

Thanks Stephen -- this is really neat!

This works well here, and I love all of the tests for the KUnit/device-tree integration as well.

I'm still looking through the details of it (alas, I've mostly lived in x86-land, so my device-tree knowledge is, uh, spotty to say the least), but apart from possibly renaming some things or similarly minor tweaks, I've not got any real suggestions thus far.

Quoted 5 lines

I do wonder whether we'll want, on the KUnit side, to have some way of supporting KUnit device trees on non-UML architecctures (e.g., if we need to test something architecture-specific, or on a big-endian platform, etc), but I think that's a question for the future, rather than something that affects this series.

I'll say that's a requirement. We should be able to structure the tests to not interfere with the running system's DT. The DT unittest does that.

As a side topic, Is anyone looking at getting UML to work on arm64? It's surprising how much x86 stuff there is which is I guess one reason it hasn't happened.

Quoted 4 lines

Similarly, I wonder if there's something we could do with device tree overlays, in order to make it possible for tests to swap nodes in and out for testing.

Yes, that's how the DT unittest works. But it is pretty much one big overlay (ignoring the overlay tests). It could probably be more modular where it is apply overlay, test, remove overlay, repeat.

Quoting Rob Herring (2023-03-02 09:32:09)

Quoted 30 lines · David Gow

Quoted 28 lines · Stephen Boyd

Quoted 11 lines

This patch series adds unit tests for the clk fixed rate basic type and the clk registration functions that use struct clk_parent_data. To get there, we add support for loading a DTB into the UML kernel that's running the unit tests along with probing platform drivers to bind to device nodes specified in DT.

With this series, we're able to exercise some of the code in the common clk framework that uses devicetree lookups to find parents and the fixed rate clk code that scans devicetree directly and creates clks. Please review.

Thanks Stephen -- this is really neat!

This works well here, and I love all of the tests for the KUnit/device-tree integration as well.

I'm still looking through the details of it (alas, I've mostly lived in x86-land, so my device-tree knowledge is, uh, spotty to say the least), but apart from possibly renaming some things or similarly minor tweaks, I've not got any real suggestions thus far.

I do wonder whether we'll want, on the KUnit side, to have some way of supporting KUnit device trees on non-UML architecctures (e.g., if we need to test something architecture-specific, or on a big-endian platform, etc), but I think that's a question for the future, rather than something that affects this series.

Quoted 3 lines

I'll say that's a requirement. We should be able to structure the tests to not interfere with the running system's DT. The DT unittest does that.

That could be another choice in the unit test choice menu. CONFIG_OF_KUNIT_NOT_UML that injects some built-in DTB overlay on an architecture that wants to run tests.

Quoted 3 lines

As a side topic, Is anyone looking at getting UML to work on arm64? It's surprising how much x86 stuff there is which is I guess one reason it hasn't happened.

I've no idea but it would be nice indeed.

Quoted 7 lines

Quoted 3 lines

Similarly, I wonder if there's something we could do with device tree overlays, in order to make it possible for tests to swap nodes in and out for testing.

Yes, that's how the DT unittest works. But it is pretty much one big overlay (ignoring the overlay tests). It could probably be more modular where it is apply overlay, test, remove overlay, repeat.

I didn't want to rely on the overlay code to inject DT nodes. Having tests written for the fake KUnit machine is simple. It closely matches how clk code probes the DTB and how nodes are created and populated on the platform bus as devices. CLK_OF_DECLARE() would need the overlay to be applied early too, which doesn't happen otherwise as far as I know.

But perhaps this design is too much of an end-to-end test and not a unit test? In the spirit of unit testing we shouldn't care about how the node is added to the live devicetree, just that there is a devicetree at all.

Supporting overlays to more easily test combinations sounds like a good idea. Probably some kunit_*() prefixed functions could be used to apply a test managed overlay and automatically remove it when the test is over would work. The clk registration tests could use this API to inject an overlay and then manually call the of_platform_populate() function to create the platform device(s). The overlay could be built in drivers/clk/ too and then probably some macroish function can find the blob and apply it.

Is there some way to delete the platform devices that we populate from the overlay? I'd like the tests to be hermetic.

Hi Stephen,

Quoted 31 lines · Stephen Boyd

Quoting Rob Herring (2023-03-02 09:32:09)

Quoted 30 lines · David Gow

Quoted 28 lines · Stephen Boyd

Quoted 11 lines

This patch series adds unit tests for the clk fixed rate basic type and the clk registration functions that use struct clk_parent_data. To get there, we add support for loading a DTB into the UML kernel that's running the unit tests along with probing platform drivers to bind to device nodes specified in DT.

With this series, we're able to exercise some of the code in the common clk framework that uses devicetree lookups to find parents and the fixed rate clk code that scans devicetree directly and creates clks. Please review.

Thanks Stephen -- this is really neat!

This works well here, and I love all of the tests for the KUnit/device-tree integration as well.

I'm still looking through the details of it (alas, I've mostly lived in x86-land, so my device-tree knowledge is, uh, spotty to say the least), but apart from possibly renaming some things or similarly minor tweaks, I've not got any real suggestions thus far.

I do wonder whether we'll want, on the KUnit side, to have some way of supporting KUnit device trees on non-UML architecctures (e.g., if we need to test something architecture-specific, or on a big-endian platform, etc), but I think that's a question for the future, rather than something that affects this series.

Quoted 7 lines

Quoted 3 lines

I'll say that's a requirement. We should be able to structure the tests to not interfere with the running system's DT. The DT unittest does that.

That could be another choice in the unit test choice menu. CONFIG_OF_KUNIT_NOT_UML that injects some built-in DTB overlay on an architecture that wants to run tests.

As long as you use compatible values that don't exist elsewhere, and don't overwrite anything, you can load your kunit test overlays on any running system that has DT support.

Quoted 5 lines

Quoted 3 lines

As a side topic, Is anyone looking at getting UML to work on arm64? It's surprising how much x86 stuff there is which is I guess one reason it hasn't happened.

I've no idea but it would be nice indeed.

I believe that's non-trivial. At least for arm32 (I didn't have any arm64 systems last time I asked the experts).

Quoted 8 lines

Quoted 7 lines

Quoted 3 lines

Similarly, I wonder if there's something we could do with device tree overlays, in order to make it possible for tests to swap nodes in and out for testing.

Yes, that's how the DT unittest works. But it is pretty much one big overlay (ignoring the overlay tests). It could probably be more modular where it is apply overlay, test, remove overlay, repeat.

Quoted 5 lines

I didn't want to rely on the overlay code to inject DT nodes. Having tests written for the fake KUnit machine is simple. It closely matches how clk code probes the DTB and how nodes are created and populated on the platform bus as devices. CLK_OF_DECLARE() would need the overlay to be applied early too, which doesn't happen otherwise as far as I know.

Don't all generic clock drivers also create a platform driver? At least drivers/clk/clk-fixed-factor.c does.

Quoted 4 lines

But perhaps this design is too much of an end-to-end test and not a unit test? In the spirit of unit testing we shouldn't care about how the node is added to the live devicetree, just that there is a devicetree at all.

Quoted 8 lines

Supporting overlays to more easily test combinations sounds like a good idea. Probably some kunit_*() prefixed functions could be used to apply a test managed overlay and automatically remove it when the test is over would work. The clk registration tests could use this API to inject an overlay and then manually call the of_platform_populate() function to create the platform device(s). The overlay could be built in drivers/clk/ too and then probably some macroish function can find the blob and apply it.

No need to manually call of_platform_populate() to create the platform devices. That is taken care of automatically when applying an overlay.

Quoted 2 lines

Is there some way to delete the platform devices that we populate from the overlay? I'd like the tests to be hermetic.

Removing the overlay will delete the platform devices.

All of that works if you have your own code to apply a DT overlay. The recent fw_devlinks patches did cause some regressions, cfr.
https://lore.kernel.org/all/CAMuHMdXEnSD4rRJ-o90x4OprUacN_rJgyo8x6=9F9rZ+-KzjOg@mail.gmail.com

P.S. Shameless plug: for loading overlays from userspace, there are
my overlay branches, cfr. https://elinux.org/R-Car/DT-Overlays

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Quoted 89 lines · Geert Uytterhoeven

Quoted 31 lines · Stephen Boyd

Quoting Rob Herring (2023-03-02 09:32:09)

Quoted 30 lines · David Gow

Quoted 28 lines · Stephen Boyd

Quoted 11 lines

This patch series adds unit tests for the clk fixed rate basic type and the clk registration functions that use struct clk_parent_data. To get there, we add support for loading a DTB into the UML kernel that's running the unit tests along with probing platform drivers to bind to device nodes specified in DT.

With this series, we're able to exercise some of the code in the common clk framework that uses devicetree lookups to find parents and the fixed rate clk code that scans devicetree directly and creates clks. Please review.

Thanks Stephen -- this is really neat!

This works well here, and I love all of the tests for the KUnit/device-tree integration as well.

I'm still looking through the details of it (alas, I've mostly lived in x86-land, so my device-tree knowledge is, uh, spotty to say the least), but apart from possibly renaming some things or similarly minor tweaks, I've not got any real suggestions thus far.

I do wonder whether we'll want, on the KUnit side, to have some way of supporting KUnit device trees on non-UML architecctures (e.g., if we need to test something architecture-specific, or on a big-endian platform, etc), but I think that's a question for the future, rather than something that affects this series.

Quoted 7 lines

Quoted 3 lines

I'll say that's a requirement. We should be able to structure the tests to not interfere with the running system's DT. The DT unittest does that.

That could be another choice in the unit test choice menu. CONFIG_OF_KUNIT_NOT_UML that injects some built-in DTB overlay on an architecture that wants to run tests.

As long as you use compatible values that don't exist elsewhere, and don't overwrite anything, you can load your kunit test overlays on any running system that has DT support.

Quoted 5 lines

Quoted 3 lines

As a side topic, Is anyone looking at getting UML to work on arm64? It's surprising how much x86 stuff there is which is I guess one reason it hasn't happened.

I've no idea but it would be nice indeed.

I believe that's non-trivial. At least for arm32 (I didn't have any arm64 systems last time I asked the experts).

Quoted 8 lines

Quoted 7 lines

Quoted 3 lines

Similarly, I wonder if there's something we could do with device tree overlays, in order to make it possible for tests to swap nodes in and out for testing.

Yes, that's how the DT unittest works. But it is pretty much one big overlay (ignoring the overlay tests). It could probably be more modular where it is apply overlay, test, remove overlay, repeat.

Quoted 5 lines

I didn't want to rely on the overlay code to inject DT nodes. Having tests written for the fake KUnit machine is simple. It closely matches how clk code probes the DTB and how nodes are created and populated on the platform bus as devices. CLK_OF_DECLARE() would need the overlay to be applied early too, which doesn't happen otherwise as far as I know.

Don't all generic clock drivers also create a platform driver? At least drivers/clk/clk-fixed-factor.c does.

Quoted 4 lines

But perhaps this design is too much of an end-to-end test and not a unit test? In the spirit of unit testing we shouldn't care about how the node is added to the live devicetree, just that there is a devicetree at all.

Quoted 8 lines

Supporting overlays to more easily test combinations sounds like a good idea. Probably some kunit_*() prefixed functions could be used to apply a test managed overlay and automatically remove it when the test is over would work. The clk registration tests could use this API to inject an overlay and then manually call the of_platform_populate() function to create the platform device(s). The overlay could be built in drivers/clk/ too and then probably some macroish function can find the blob and apply it.

No need to manually call of_platform_populate() to create the platform devices. That is taken care of automatically when applying an overlay.

Quoted 2 lines

Is there some way to delete the platform devices that we populate from the overlay? I'd like the tests to be hermetic.

Quoted 1 line

Removing the overlay will delete the platform devices.

I _think_ that is incorrect. Do you have a pointer to the overlay code that deletes the device? (If I remember correctly, the overlay remove code does not even check whether the device exists and whether a driver is bound to it -- but this is on my todo list to look into.)

Quoted 3 lines

All of that works if you have your own code to apply a DT overlay. The recent fw_devlinks patches did cause some regressions, cfr.
https://lore.kernel.org/all/CAMuHMdXEnSD4rRJ-o90x4OprUacN_rJgyo8x6=9F9rZ+-KzjOg@mail.gmail.com

Hi Frank,

Quoted 91 lines · Frank Rowand

Quoted 89 lines · Geert Uytterhoeven

Quoted 31 lines · Stephen Boyd

Quoting Rob Herring (2023-03-02 09:32:09)

Quoted 30 lines · David Gow

Quoted 28 lines · Stephen Boyd

Quoted 11 lines

This patch series adds unit tests for the clk fixed rate basic type and the clk registration functions that use struct clk_parent_data. To get there, we add support for loading a DTB into the UML kernel that's running the unit tests along with probing platform drivers to bind to device nodes specified in DT.

With this series, we're able to exercise some of the code in the common clk framework that uses devicetree lookups to find parents and the fixed rate clk code that scans devicetree directly and creates clks. Please review.

Thanks Stephen -- this is really neat!

This works well here, and I love all of the tests for the KUnit/device-tree integration as well.

I'm still looking through the details of it (alas, I've mostly lived in x86-land, so my device-tree knowledge is, uh, spotty to say the least), but apart from possibly renaming some things or similarly minor tweaks, I've not got any real suggestions thus far.

I do wonder whether we'll want, on the KUnit side, to have some way of supporting KUnit device trees on non-UML architecctures (e.g., if we need to test something architecture-specific, or on a big-endian platform, etc), but I think that's a question for the future, rather than something that affects this series.

Quoted 7 lines

Quoted 3 lines

I'll say that's a requirement. We should be able to structure the tests to not interfere with the running system's DT. The DT unittest does that.

That could be another choice in the unit test choice menu. CONFIG_OF_KUNIT_NOT_UML that injects some built-in DTB overlay on an architecture that wants to run tests.

As long as you use compatible values that don't exist elsewhere, and don't overwrite anything, you can load your kunit test overlays on any running system that has DT support.

Quoted 5 lines

Quoted 3 lines

As a side topic, Is anyone looking at getting UML to work on arm64? It's surprising how much x86 stuff there is which is I guess one reason it hasn't happened.

I've no idea but it would be nice indeed.

I believe that's non-trivial. At least for arm32 (I didn't have any arm64 systems last time I asked the experts).

Quoted 8 lines

Quoted 7 lines

Quoted 3 lines

Similarly, I wonder if there's something we could do with device tree overlays, in order to make it possible for tests to swap nodes in and out for testing.

Yes, that's how the DT unittest works. But it is pretty much one big overlay (ignoring the overlay tests). It could probably be more modular where it is apply overlay, test, remove overlay, repeat.

Quoted 5 lines

I didn't want to rely on the overlay code to inject DT nodes. Having tests written for the fake KUnit machine is simple. It closely matches how clk code probes the DTB and how nodes are created and populated on the platform bus as devices. CLK_OF_DECLARE() would need the overlay to be applied early too, which doesn't happen otherwise as far as I know.

Don't all generic clock drivers also create a platform driver? At least drivers/clk/clk-fixed-factor.c does.

Quoted 4 lines

But perhaps this design is too much of an end-to-end test and not a unit test? In the spirit of unit testing we shouldn't care about how the node is added to the live devicetree, just that there is a devicetree at all.

Quoted 8 lines

Supporting overlays to more easily test combinations sounds like a good idea. Probably some kunit_*() prefixed functions could be used to apply a test managed overlay and automatically remove it when the test is over would work. The clk registration tests could use this API to inject an overlay and then manually call the of_platform_populate() function to create the platform device(s). The overlay could be built in drivers/clk/ too and then probably some macroish function can find the blob and apply it.

No need to manually call of_platform_populate() to create the platform devices. That is taken care of automatically when applying an overlay.

Quoted 2 lines

Is there some way to delete the platform devices that we populate from the overlay? I'd like the tests to be hermetic.

Quoted 6 lines

Quoted 1 line

Removing the overlay will delete the platform devices.

I _think_ that is incorrect. Do you have a pointer to the overlay code that deletes the device? (If I remember correctly, the overlay remove code does not even check whether the device exists and whether a driver is bound to it -- but this is on my todo list to look into.)

https://elixir.bootlin.com/linux/latest/source/drivers/of/platform.c#L769

Quoted 4 lines

Quoted 4 lines

All of that works if you have your own code to apply a DT overlay. The recent fw_devlinks patches did cause some regressions, cfr.
https://lore.kernel.org/all/CAMuHMdXEnSD4rRJ-o90x4OprUacN_rJgyo8x6=9F9rZ+-KzjOg@mail.gmail.com

Gr{oetje,eeting}s,

Geert

Quoting Rob Herring (2023-03-02 09:32:09)

Quoted 32 lines · David Gow

Quoted 25 lines · Stephen Boyd

Quoted 7 lines

This patch series adds unit tests for the clk fixed rate basic type and the clk registration functions that use struct clk_parent_data. To get there, we add support for loading a DTB into the UML kernel that's running the unit tests along with probing platform drivers to bind to device nodes specified in DT.

Quoted 5 lines

With this series, we're able to exercise some of the code in the common clk framework that uses devicetree lookups to find parents and the fixed rate clk code that scans devicetree directly and creates clks. Please review.

Thanks Stephen -- this is really neat!

This works well here, and I love all of the tests for the KUnit/device-tree integration as well.

I'm still looking through the details of it (alas, I've mostly lived in x86-land, so my device-tree knowledge is, uh, spotty to say the least), but apart from possibly renaming some things or similarly minor tweaks, I've not got any real suggestions thus far.

Quoted 5 lines

I do wonder whether we'll want, on the KUnit side, to have some way of supporting KUnit device trees on non-UML architecctures (e.g., if we need to test something architecture-specific, or on a big-endian platform, etc), but I think that's a question for the future, rather than something that affects this series.

Quoted 3 lines

I'll say that's a requirement. We should be able to structure the tests to not interfere with the running system's DT. The DT unittest does that.

That could be another choice in the unit test choice menu. CONFIG_OF_KUNIT_NOT_UML that injects some built-in DTB overlay on an architecture that wants to run tests.

Quoted 4 lines

As a side topic, Is anyone looking at getting UML to work on arm64? It's surprising how much x86 stuff there is which is I guess one reason it hasn't happened.

I've no idea but it would be nice indeed.

Quoted 9 lines

Quoted 4 lines

Similarly, I wonder if there's something we could do with device tree overlays, in order to make it possible for tests to swap nodes in and out for testing.

Yes, that's how the DT unittest works. But it is pretty much one big overlay (ignoring the overlay tests). It could probably be more modular where it is apply overlay, test, remove overlay, repeat.

I didn't want to rely on the overlay code to inject DT nodes. Having tests written for the fake KUnit machine is simple. It closely matches how clk code probes the DTB and how nodes are created and populated on the platform bus as devices. CLK_OF_DECLARE() would need the overlay to be applied early too, which doesn't happen otherwise as far as I know.

But perhaps this design is too much of an end-to-end test and not a unit test? In the spirit of unit testing we shouldn't care about how the node is added to the live devicetree, just that there is a devicetree at all.

Supporting overlays to more easily test combinations sounds like a good idea. Probably some kunit_*() prefixed functions could be used to

apply a test managed overlay and automatically remove it when the test is over would work. The clk registration tests could use this API to inject an overlay and then manually call the of_platform_populate() function to create the platform device(s). The overlay could be built in drivers/clk/ too and then probably some macroish function can find the blob and apply it.

Is there some way to delete the platform devices that we populate from the overlay? I'd like the tests to be hermetic.

Quoted 7 lines

But perhaps this design is too much of an end-to-end test and not a unit test? In the spirit of unit testing we shouldn't care about how the node is added to the live devicetree, just that there is a devicetree at all.

Supporting overlays to more easily test combinations sounds like a good idea. Probably some kunit_*() prefixed functions could be used to

In an imaginary world where overlay support was completed, then _maybe_.

To me, the most important environment to test is where the devictree data is populated in early boot from an FDT. This is the environment that drivers currently exist in.

Populating devicetree data via an overlay adds in the functioning of the overlay apply code (and how the rules behind that functioning may differ from devicetree data populated in early boot from an FDT).

Quoted 25 lines · David Gow

Quoted 7 lines · Stephen Boyd

This patch series adds unit tests for the clk fixed rate basic type and the clk registration functions that use struct clk_parent_data. To get there, we add support for loading a DTB into the UML kernel that's running the unit tests along with probing platform drivers to bind to device nodes specified in DT.

Quoted 5 lines

With this series, we're able to exercise some of the code in the common clk framework that uses devicetree lookups to find parents and the fixed rate clk code that scans devicetree directly and creates clks. Please review.

Thanks Stephen -- this is really neat!

This works well here, and I love all of the tests for the KUnit/device-tree integration as well.

I'm still looking through the details of it (alas, I've mostly lived in x86-land, so my device-tree knowledge is, uh, spotty to say the least), but apart from possibly renaming some things or similarly minor tweaks, I've not got any real suggestions thus far.

Quoted 5 lines

I do wonder whether we'll want, on the KUnit side, to have some way of supporting KUnit device trees on non-UML architecctures (e.g., if we need to test something architecture-specific, or on a big-endian platform, etc), but I think that's a question for the future, rather than something that affects this series.

I'll say that's a requirement. We should be able to structure the tests to not interfere with the running system's DT. The DT unittest does that.

As a side topic, Is anyone looking at getting UML to work on arm64? It's surprising how much x86 stuff there is which is I guess one reason it hasn't happened.

Quoted 3 lines

Similarly, I wonder if there's something we could do with device tree overlays, in order to make it possible for tests to swap nodes in and out for testing.

Yes, that's how the DT unittest works. But it is pretty much one big overlay (ignoring the overlay tests). It could probably be more modular where it is apply overlay, test, remove overlay, repeat.

Rob

This patch series adds unit tests for the clk fixed rate basic type and the clk registration functions that use struct clk_parent_data. To get there, we add support for loading a DTB into the UML kernel that's running the unit tests along with probing platform drivers to bind to device nodes specified in DT.

With this series, we're able to exercise some of the code in the common clk framework that uses devicetree lookups to find parents and the fixed rate clk code that scans devicetree directly and creates clks. Please review.

I Cced everyone to all the patches so they get the full context. I'm hoping I can take the whole pile through the clk tree as they almost all depend on each other. In the future I imagine it will be easy to add more test nodes to the clk.dtsi file and not need to go across various maintainer trees like this series does.

Stephen Boyd (8):
dt-bindings: Add linux,kunit binding
of: Enable DTB loading on UML for KUnit tests
kunit: Add test managed platform_device/driver APIs
clk: Add test managed clk provider/consumer APIs
dt-bindings: kunit: Add fixed rate clk consumer test
clk: Add KUnit tests for clk fixed rate basic type
dt-bindings: clk: Add KUnit clk_parent_data test
clk: Add KUnit tests for clks registered with struct clk_parent_data

 .../clock/linux,clk-kunit-parent-data.yaml    |  47 ++
 .../kunit/linux,clk-kunit-fixed-rate.yaml     |  35 ++
 .../bindings/kunit/linux,kunit.yaml           |  24 +
 arch/um/kernel/dtb.c                          |  29 +-
 drivers/clk/.kunitconfig                      |   3 +
 drivers/clk/Kconfig                           |   7 +
 drivers/clk/Makefile                          |   6 +
 drivers/clk/clk-fixed-rate_test.c             | 296 ++++++++++++
 drivers/clk/clk-kunit.c                       | 204 ++++++++
 drivers/clk/clk-kunit.h                       |  28 ++
 drivers/clk/clk_test.c                        | 456 +++++++++++++++++-
 drivers/of/Kconfig                            |  26 +
 drivers/of/Makefile                           |   1 +
 drivers/of/kunit/.kunitconfig                 |   4 +
 drivers/of/kunit/Makefile                     |   4 +

 drivers/of/kunit/clk.dtsi                     |  30 ++
 drivers/of/kunit/kunit.dtsi                   |   9 +
 drivers/of/kunit/kunit.dtso                   |   4 +
 drivers/of/kunit/uml_dtb_test.c               |  55 +++
 include/kunit/platform_driver.h               |  15 +
 lib/kunit/Makefile                            |   6 +
 lib/kunit/platform_driver-test.c              | 107 ++++
 lib/kunit/platform_driver.c                   | 207 ++++++++

Quoted 18 lines

This patch series adds unit tests for the clk fixed rate basic type and the clk registration functions that use struct clk_parent_data. To get there, we add support for loading a DTB into the UML kernel that's running the unit tests along with probing platform drivers to bind to device nodes specified in DT.

With this series, we're able to exercise some of the code in the common clk framework that uses devicetree lookups to find parents and the fixed rate clk code that scans devicetree directly and creates clks. Please review.

I Cced everyone to all the patches so they get the full context. I'm hoping I can take the whole pile through the clk tree as they almost all depend on each other. In the future I imagine it will be easy to add more test nodes to the clk.dtsi file and not need to go across various maintainer trees like this series does.

Quoted 9 lines

Stephen Boyd (8):
dt-bindings: Add linux,kunit binding
of: Enable DTB loading on UML for KUnit tests
kunit: Add test managed platform_device/driver APIs
clk: Add test managed clk provider/consumer APIs
dt-bindings: kunit: Add fixed rate clk consumer test
clk: Add KUnit tests for clk fixed rate basic type
dt-bindings: clk: Add KUnit clk_parent_data test
clk: Add KUnit tests for clks registered with struct clk_parent_data

Good to see bindings for this. I've been meaning to do something about the DT unittest ones being undocumented, but I hadn't really decided whether it was worth writing schemas for them. The compatibles at least show up with 'make dt_compatible_check'. Perhaps we want to just define some vendor (not 'linux') that's an exception rather than requiring schemas (actually, that already works for 'foo').

It's
likely that we want test DTs that fail normal checks and schemas get in the way of that as we don't have a way to turn off checks.