of: unittest: unflatten device tree on UML when testing
cover of: unittest: unflatten device tree on UML when testing
This patch set unflattens device trees when running DT's unittest. It is a follow up to
https://www.spinics.net/lists/linux-kselftest/msg05454.html part of the KUnit RFC v3 thread
(https://www.spinics.net/lists/linux-kselftest/msg05431.html). Rob asked me to submit this patch separately along with some modifications.
Signature · 1 line
2.20.1.791.gb4d0f1c61a-goog
PATCH 1/1 of: unittest: unflatten device tree on UML when testing
UML supports enabling OF, and is useful for running the device tree tests, so add support for unflattening device tree blobs so we can actually use it.
--- drivers/of/unittest.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/of/unittest.c +3 -0
@@ -2527,6 +2527,9 @@ static int __init of_unittest(void) }of_node_put (np); +if (IS_ENABLED (CONFIG_UML )) +unflatten_device_tree (); +pr_info ("start of unittest - you will see error messages\n" );of_unittest_check_tree_linkage ();of_unittest_check_phandles ();
Signature · 1 line
2.20.1.791.gb4d0f1c61a-goog
Quoted 6 lines · Brendan Higgins
UML supports enabling OF, and is useful for running the device tree tests, so add support for unflattening device tree blobs so we can actually use it.
Signed-off-by Brendan Higgins <[email protected]>---
For a single patch, no need for a cover letter. Just add any commentary here.
Quoted 2 lines
drivers/of/unittest.c | 3 +++ 1 file changed, 3 insertions(+)
Applied, thanks.
Rob
Quoted 15 lines · Brendan Higgins
UML supports enabling OF, and is useful for running the device tree tests, so add support for unflattening device tree blobs so we can actually use it.
Signed-off-by Brendan Higgins <[email protected]>--- drivers/of/unittest.c | 3 +++ 1 file changed, 3 insertions(+)drivers/of/unittest.c +0
-0
drivers/of/unittest.c static int __init of_unittest(void) · line 2535
Quoted 9 lines
drivers/of/unittest.c +3
-0@@ -2527,6 +2527,9 @@ static int __init of_unittest(void) }of_node_put (np); +if (IS_ENABLED (CONFIG_UML )) +unflatten_device_tree (); +pr_info ("start of unittest - you will see error messages\n" );of_unittest_check_tree_linkage ();of_unittest_check_phandles ();
(Insert my usual disclaimer that I am clueless about UML, I still need to learn about it...)
This does not look correct to me.
A few lines earlier in of_unittest(), the live devicetree needs to exist for unittest_data_data() and a few of_*() functions to succeed. So it seems that the unflatten_device_tree() for uml should be at the beginning of of_unittest().
Rob, if I am correct please revert this patch.
-Frank
Quoted 30 lines · Frank Rowand
Quoted 15 lines · Brendan Higgins
UML supports enabling OF, and is useful for running the device tree tests, so add support for unflattening device tree blobs so we can actually use it.
Signed-off-by Brendan Higgins <[email protected]>--- drivers/of/unittest.c | 3 +++ 1 file changed, 3 insertions(+)drivers/of/unittest.c +0
-0drivers/of/unittest.c static int __init of_unittest(void) · line 2535
Quoted 9 lines
drivers/of/unittest.c +3
-0@@ -2527,6 +2527,9 @@ static int __init of_unittest(void) }of_node_put (np); +if (IS_ENABLED (CONFIG_UML )) +unflatten_device_tree (); +pr_info ("start of unittest - you will see error messages\n" );of_unittest_check_tree_linkage ();of_unittest_check_phandles ();(Insert my usual disclaimer that I am clueless about UML, I still need to learn about it...)
Quoted 6 lines
This does not look correct to me.
A few lines earlier in
of_unittest(), the live devicetree needs to exist forunittest_data_data()and a few of_*() functions to succeed. So it seems that theunflatten_device_tree()for uml should be at the beginning ofof_unittest().
It is true that other functions ahead of it depend on the presence of a device tree, but an unflattened tree does get linked in somewhere else. Despite that, this is needed for overlay_base_root. I got similar behavior if you don't link in a flattened device tree on x86. Thus, the order in which you call them doesn't actually seem to matter. I found no difference from changing the order in UML myself.
Without my patch we get the following error,
### dt-test ### FAIL of_unittest_overlay_high_level():2372 overlay_base_root not initialized
### dt-test ### end of unittest - 219 passed, 1 failed
With my patch we get:
### dt-test ### end of unittest - 224 passed, 0 failed
I used the following .config for these results:CONFIG_OF=yCONFIG_OF_UNITTEST=yCONFIG_OF_OVERLAY=yCONFIG_I2C=yCONFIG_I2C_MUX=y
Quoted 3 lines
Rob, if I am correct please revert this patch.
Cheers
Quoted 45 lines · Brendan Higgins
Quoted 30 lines · Frank Rowand
Quoted 15 lines · Brendan Higgins
UML supports enabling OF, and is useful for running the device tree tests, so add support for unflattening device tree blobs so we can actually use it.
Signed-off-by Brendan Higgins <[email protected]>--- drivers/of/unittest.c | 3 +++ 1 file changed, 3 insertions(+)drivers/of/unittest.c +0
-0drivers/of/unittest.c static int __init of_unittest(void) · line 2535
Quoted 9 lines
drivers/of/unittest.c +3
-0@@ -2527,6 +2527,9 @@ static int __init of_unittest(void) }of_node_put (np); +if (IS_ENABLED (CONFIG_UML )) +unflatten_device_tree (); +pr_info ("start of unittest - you will see error messages\n" );of_unittest_check_tree_linkage ();of_unittest_check_phandles ();(Insert my usual disclaimer that I am clueless about UML, I still need to learn about it...)
Quoted 6 lines
This does not look correct to me.
A few lines earlier in
of_unittest(), the live devicetree needs to exist forunittest_data_data()and a few of_*() functions to succeed. So it seems that theunflatten_device_tree()for uml should be at the beginning ofof_unittest().It is true that other functions ahead of it depend on the presence of a device tree, but an unflattened tree does get linked in somewhere else. Despite that, this is needed for
overlay_base_root. I got similar behavior if you don't link in a flattened device tree on x86. Thus, the order in which you call them doesn't actually seem to matter. I found no difference from changing the order in UML myself.
Quoted 7 lines
Without my patch we get the following error,
### dt-test ### FAIL of_unittest_overlay_high_level():2372overlay_base_rootnot initialized
### dt-test ### end of unittest - 219 passed, 1 failedWith my patch we get:
### dt-test ### end of unittest - 224 passed, 0 failed
Thanks for reporting both the failure and the success cases, that helps me understand a little bit better.
If instead of the above patch, if you add the following (untested, not even compile tested) to the beginning of of_unittest():
if (IS_ENABLED(CONFIG_UML)) unittest_unflatten_overlay_base();
does that also result in a good test result of:
### dt-test ### end of unittest - 224 passed, 0 failed
I think I need to find some time to build and boot a UML kernel soon.
My current _guess_ is that the original problem was not a failure to unflatten any present devicetree in UML but instead that the UML kernel does not call unflatten_device_tree() and thus fails to indirectly call unittest_unflatten_overlay_base(), which is called by unflatten_device_tree().
unittest_unflatten_overlay_base() is an unfortunate wart that I added, but I don't have a better alternative yet.
-Frank
Quoted 14 lines
I used the following .config for these results:
CONFIG_OF=yCONFIG_OF_UNITTEST=yCONFIG_OF_OVERLAY=yCONFIG_I2C=yCONFIG_I2C_MUX=yQuoted 3 lines
Rob, if I am correct please revert this patch.
Cheers
Quoted 61 lines · Frank Rowand
Quoted 45 lines · Brendan Higgins
Quoted 30 lines · Frank Rowand
Quoted 15 lines · Brendan Higgins
UML supports enabling OF, and is useful for running the device tree tests, so add support for unflattening device tree blobs so we can actually use it.
Signed-off-by Brendan Higgins <[email protected]>--- drivers/of/unittest.c | 3 +++ 1 file changed, 3 insertions(+)drivers/of/unittest.c +0
-0drivers/of/unittest.c static int __init of_unittest(void) · line 2535
Quoted 9 lines
drivers/of/unittest.c +3
-0@@ -2527,6 +2527,9 @@ static int __init of_unittest(void) }of_node_put (np); +if (IS_ENABLED (CONFIG_UML )) +unflatten_device_tree (); +pr_info ("start of unittest - you will see error messages\n" );of_unittest_check_tree_linkage ();of_unittest_check_phandles ();(Insert my usual disclaimer that I am clueless about UML, I still need to learn about it...)
Quoted 6 lines
This does not look correct to me.
A few lines earlier in
of_unittest(), the live devicetree needs to exist forunittest_data_data()and a few of_*() functions to succeed. So it seems that theunflatten_device_tree()for uml should be at the beginning ofof_unittest().It is true that other functions ahead of it depend on the presence of a device tree, but an unflattened tree does get linked in somewhere else. Despite that, this is needed for
overlay_base_root. I got similar behavior if you don't link in a flattened device tree on x86. Thus, the order in which you call them doesn't actually seem to matter. I found no difference from changing the order in UML myself.Quoted 7 lines
Without my patch we get the following error,
### dt-test ### FAIL of_unittest_overlay_high_level():2372overlay_base_rootnot initialized
### dt-test ### end of unittest - 219 passed, 1 failedWith my patch we get:
### dt-test ### end of unittest - 224 passed, 0 failedThanks for reporting both the failure and the success cases, that helps me understand a little bit better.
If instead of the above patch, if you add the following (untested, not even compile tested) to the beginning of
of_unittest():
Quoted 6 lines
if (IS_ENABLED(CONFIG_UML)) unittest_unflatten_overlay_base();does that also result in a good test result of:
### dt-test ### end of unittest - 224 passed, 0 failed
Yep, I just tried it. It works as you suspected.
Quoted 2 lines
I think I need to find some time to build and boot a UML kernel soon.
It's really no big deal, just copy the .config I sent and build with make ARCH=um then you "boot" the kernel with ./linux (note this will mess up your terminal settings); that's it! (Shameless plug: you can also try it out with the KUnit patchset with ./tools/testing/kunit/kunit.py --timeout=30 --jobs=12 --defconfig, which builds the kernel with a pretty similar config, boots the kernel, and then parses the output for you. ;-) )
Quoted 6 lines
My current _guess_ is that the original problem was not a failure to unflatten any present devicetree in UML but instead that the UML kernel does not call
unflatten_device_tree()and thus fails to indirectly callunittest_unflatten_overlay_base(), which is called byunflatten_device_tree().
I think you are right. Sorry for not noticing this before making my change. Since it was pretty much the only architecture (the only one I care about) that does not unflatten DT, I assumed that was the problem. I didn't put too much thought into it after that point beyond making sure that it did what I wanted.
Quoted 3 lines
unittest_unflatten_overlay_base()is an unfortunate wart that I added, but I don't have a better alternative yet.
Hey, I get it. No worries.
In any case, it seems like unittest_unflatten_overlay_base() is the right function to call there. I will send out patch. Do you want me to send a patch on top of this one, or do you want to revert this one and for me to send a v2 follow up to this patch? I don't care either way, whatever you guys prefer.
Cheers
Quoted 100 lines · Brendan Higgins
Quoted 61 lines · Frank Rowand
Quoted 45 lines · Brendan Higgins
Quoted 30 lines · Frank Rowand
Quoted 15 lines · Brendan Higgins
UML supports enabling OF, and is useful for running the device tree tests, so add support for unflattening device tree blobs so we can actually use it.
Signed-off-by Brendan Higgins <[email protected]>--- drivers/of/unittest.c | 3 +++ 1 file changed, 3 insertions(+)drivers/of/unittest.c +0
-0drivers/of/unittest.c static int __init of_unittest(void) · line 2535
Quoted 9 lines
drivers/of/unittest.c +3
-0@@ -2527,6 +2527,9 @@ static int __init of_unittest(void) }of_node_put (np); +if (IS_ENABLED (CONFIG_UML )) +unflatten_device_tree (); +pr_info ("start of unittest - you will see error messages\n" );of_unittest_check_tree_linkage ();of_unittest_check_phandles ();(Insert my usual disclaimer that I am clueless about UML, I still need to learn about it...)
Quoted 6 lines
This does not look correct to me.
A few lines earlier in
of_unittest(), the live devicetree needs to exist forunittest_data_data()and a few of_*() functions to succeed. So it seems that theunflatten_device_tree()for uml should be at the beginning ofof_unittest().It is true that other functions ahead of it depend on the presence of a device tree, but an unflattened tree does get linked in somewhere else. Despite that, this is needed for
overlay_base_root. I got similar behavior if you don't link in a flattened device tree on x86. Thus, the order in which you call them doesn't actually seem to matter. I found no difference from changing the order in UML myself.Quoted 7 lines
Without my patch we get the following error,
### dt-test ### FAIL of_unittest_overlay_high_level():2372overlay_base_rootnot initialized
### dt-test ### end of unittest - 219 passed, 1 failedWith my patch we get:
### dt-test ### end of unittest - 224 passed, 0 failedThanks for reporting both the failure and the success cases, that helps me understand a little bit better.
If instead of the above patch, if you add the following (untested, not even compile tested) to the beginning of
of_unittest():Quoted 6 lines
if (IS_ENABLED(CONFIG_UML)) unittest_unflatten_overlay_base();does that also result in a good test result of:
### dt-test ### end of unittest - 224 passed, 0 failed
Yep, I just tried it. It works as you suspected.
Quoted 2 lines
I think I need to find some time to build and boot a UML kernel soon.
It's really no big deal, just copy the .config I sent and build with
make ARCH=umthen you "boot" the kernel with./linux(note this will mess up your terminal settings); that's it! (Shameless plug: you can also try it out with the KUnit patchset with./tools/testing/kunit/kunit.py --timeout=30 --jobs=12 --defconfig, which builds the kernel with a pretty similar config, boots the kernel, and then parses the output for you. ;-) )Quoted 6 lines
My current _guess_ is that the original problem was not a failure to unflatten any present devicetree in UML but instead that the UML kernel does not call
unflatten_device_tree()and thus fails to indirectly callunittest_unflatten_overlay_base(), which is called byunflatten_device_tree().I think you are right. Sorry for not noticing this before making my change. Since it was pretty much the only architecture (the only one I care about) that does not unflatten DT, I assumed that was the problem. I didn't put too much thought into it after that point beyond making sure that it did what I wanted.
Quoted 3 lines
unittest_unflatten_overlay_base()is an unfortunate wart that I added, but I don't have a better alternative yet.
Quoted 7 lines
Hey, I get it. No worries.
In any case, it seems like
unittest_unflatten_overlay_base()is the right function to call there. I will send out patch. Do you want me to send a patch on top of this one, or do you want to revert this one and for me to send a v2 follow up to this patch? I don't care either way, whatever you guys prefer.
I'll drop or revert the existing one, so against mainline is good.
Rob
Quoted 74 lines · Brendan Higgins
Quoted 61 lines · Frank Rowand
Quoted 45 lines · Brendan Higgins
Quoted 30 lines · Frank Rowand
Quoted 15 lines · Brendan Higgins
UML supports enabling OF, and is useful for running the device tree tests, so add support for unflattening device tree blobs so we can actually use it.
Signed-off-by Brendan Higgins <[email protected]>--- drivers/of/unittest.c | 3 +++ 1 file changed, 3 insertions(+)drivers/of/unittest.c +0
-0drivers/of/unittest.c static int __init of_unittest(void) · line 2535
Quoted 9 lines
drivers/of/unittest.c +3
-0@@ -2527,6 +2527,9 @@ static int __init of_unittest(void) }of_node_put (np); +if (IS_ENABLED (CONFIG_UML )) +unflatten_device_tree (); +pr_info ("start of unittest - you will see error messages\n" );of_unittest_check_tree_linkage ();of_unittest_check_phandles ();(Insert my usual disclaimer that I am clueless about UML, I still need to learn about it...)
Quoted 6 lines
This does not look correct to me.
A few lines earlier in
of_unittest(), the live devicetree needs to exist forunittest_data_data()and a few of_*() functions to succeed. So it seems that theunflatten_device_tree()for uml should be at the beginning ofof_unittest().It is true that other functions ahead of it depend on the presence of a device tree, but an unflattened tree does get linked in somewhere else. Despite that, this is needed for
overlay_base_root. I got similar behavior if you don't link in a flattened device tree on x86. Thus, the order in which you call them doesn't actually seem to matter. I found no difference from changing the order in UML myself.Quoted 7 lines
Without my patch we get the following error,
### dt-test ### FAIL of_unittest_overlay_high_level():2372overlay_base_rootnot initialized
### dt-test ### end of unittest - 219 passed, 1 failedWith my patch we get:
### dt-test ### end of unittest - 224 passed, 0 failedThanks for reporting both the failure and the success cases, that helps me understand a little bit better.
If instead of the above patch, if you add the following (untested, not even compile tested) to the beginning of
of_unittest():Quoted 6 lines
if (IS_ENABLED(CONFIG_UML)) unittest_unflatten_overlay_base();does that also result in a good test result of:
### dt-test ### end of unittest - 224 passed, 0 failed
Yep, I just tried it. It works as you suspected.
Quoted 2 lines
I think I need to find some time to build and boot a UML kernel soon.
Quoted 7 lines
It's really no big deal, just copy the .config I sent and build with
make ARCH=umthen you "boot" the kernel with./linux(note this will mess up your terminal settings); that's it! (Shameless plug: you can also try it out with the KUnit patchset with./tools/testing/kunit/kunit.py --timeout=30 --jobs=12 --defconfig, which builds the kernel with a pretty similar config, boots the kernel, and then parses the output for you. ;-) )
Thanks, that was enough info to prod me into building and "booting" a uml kernel. I have another framework that I use, so I did not try kunit.py, but reading that filled in any missing details that I needed.
As I mentioned, I used my own framework, but the commands that it emitted essentially boil down to a rather simple recipe:
export ARCH=um
make kunit_defconfig
make olddefconfig
make linux
# KBUILD_OUTPUT is my build directory
${KBUILD_OUTPUT}/linux mem=256mQuoted 15 lines
Quoted 7 lines
My current _guess_ is that the original problem was not a failure to unflatten any present devicetree in UML but instead that the UML kernel does not call
unflatten_device_tree()and thus fails to indirectly callunittest_unflatten_overlay_base(), which is called byunflatten_device_tree().I think you are right. Sorry for not noticing this before making my change. Since it was pretty much the only architecture (the only one I care about) that does not unflatten DT, I assumed that was the problem. I didn't put too much thought into it after that point beyond making sure that it did what I wanted.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
unittest_unflatten_overlay_base()is an unfortunate wart that I added, but I don't have a better alternative yet.Hey, I get it. No worries.
In any case, it seems like
unittest_unflatten_overlay_base()is the right function to call there. I will send out patch. Do you want me to
Thanks for the updated patch.
Quoted 6 lines
send a patch on top of this one, or do you want to revert this one and for me to send a v2 follow up to this patch? I don't care either way, whatever you guys prefer.
Cheers