Rust Oe-Selftest implementation
PATCH v7 1/1 Rust Oe-Selftest implementation
From: pgowda <[email protected]>
The patch implements Rust testing framework similar to other selftest, specifically the gcc selftest in OE. It uses the client and server based method to test the binaries for cross-target on the image. The test framework is a wrapper around the Rust build system as ./x.py test.
It tests many functionalities of Rust distribution like tools, documentation, libraries, packages, tools, Cargo, Crater etc. Please refer the following link for detailed description of Rust testing:-
https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests
To support the rust tests in oe-core, the following functions were added:-setup_cargo_environment(): Build bootstrap and some early stage tools. do_rust_setup_snapshot(): Install the snapshot version of rust binaries. do_configure(): To generate config.tomldo_compile(): To build "remote-test-server" for qemu target image.
Approximate Number of Tests Run in the Rust Testsuite :- 18000 Approximate Number of Tests that FAIL in bitbake environment :- 100-150 Normally majority of the testcases are present in major folder "test/" It contributes to more than 80% of the testcases present in Rust test framework. These tests pass as expected on any Rust versions without much fuss. The tests that fail are of less important and contribute to less than 2% of the total testcases. These minor tests are observed to work on some versions and fail on others. They have to be added, ignored or excluded for different versions as per the behavior. These tests have been ignored or excluded in the Rust selftest environment to generate success of completing the testsuite.
These tests work in parallel mode even in the skipped test mode as expected. Although the patch to disable tests is large, it is very simple in that it only disables tests. When updating to a newer version of Rust, the patch can usually be ported in a day.
--- meta/classes/rust-target-config.bbclass | 3 +- meta/lib/oeqa/selftest/cases/rust.py | 57 +++ meta/recipes-devtools/rust/rust-testsuite.inc | 164 ++++++ .../rust-testsuite/rust-oe-selftest.patch | 477 ++++++++++++++++++ .../rust/rust-testsuite_1.62.0.bb | 3 + 5 files changed, 703 insertions(+), 1 deletion(-) create mode 100644 meta/lib/oeqa/selftest/cases/rust.py create mode 100644 meta/recipes-devtools/rust/rust-testsuite.inc create mode 100644 meta/recipes-devtools/rust/rust-testsuite/rust-oe-selftest.patch create mode 100644 meta/recipes-devtools/rust/rust-testsuite_1.62.0.bb
meta/classes/rust-target-config.bbclass +2 -1
@@ -344,7 +344,8 @@ def rust_gen_target(d, thing, wd, arch): tspec['linker-is-gnu'] = True tspec['linker-flavor'] = "gcc" tspec['has-rpath'] = True - tspec['has-elf-tls'] = True + # FIXME: Around 270 testcases fail with elf-tls enabled. + # tspec['has-elf-tls'] = True tspec['position-independent-executables'] = True tspec['panic-strategy'] = d.getVar("RUST_PANIC_STRATEGY")
meta/lib/oeqa/selftest/cases/rust.py new file +57 -0
57 lines not on this page · raw message
meta/recipes-devtools/rust/rust-testsuite.inc new file +164 -0
164 lines not on this page · raw message
meta/recipes-devtools/rust/rust-testsuite/rust-oe-selftest.patch new file +477 -0
477 lines not on this page · raw message
meta/recipes-devtools/rust/rust-testsuite_1.62.0.bb new file +3 -0
@@ -0,0 +1,3 @@ +require rust-testsuite.inc +require rust-source.inc +require rust-snapshot.inc
Signature · 1 line
2.25.1
Quoted 32 lines · On Tue, 2 Aug 2022 17:50:05 +0530 "Pgowda" <[email protected]> wrote:
From: pgowda <[email protected]>
The patch implements Rust testing framework similar to other selftest, specifically the gcc selftest in OE. It uses the client and server based method to test the binaries for cross-target on the image. The test framework is a wrapper around the Rust build system as ./x.py test.
It tests many functionalities of Rust distribution like tools, documentation, libraries, packages, tools, Cargo, Crater etc. Please refer the following link for detailed description of Rust testing:-
https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-testsTo support the rust tests in oe-core, the following functions were added:-
setup_cargo_environment(): Build bootstrap and some early stage tools.do_rust_setup_snapshot(): Install the snapshot version of rust binaries.do_configure(): To generate config.tomldo_compile(): To build "remote-test-server" for qemu target image.Approximate Number of Tests Run in the Rust Testsuite :- 18000 Approximate Number of Tests that FAIL in bitbake environment :- 100-150 Normally majority of the testcases are present in major folder "test/" It contributes to more than 80% of the testcases present in Rust test framework. These tests pass as expected on any Rust versions without much fuss. The tests that fail are of less important and contribute to less than 2% of the total testcases. These minor tests are observed to work on some versions and fail on others. They have to be added, ignored or excluded for different versions as per the behavior. These tests have been ignored or excluded in the Rust selftest environment to generate success of completing the testsuite.
Quoted 7 lines
These tests work in parallel mode even in the skipped test mode as expected. Although the patch to disable tests is large, it is very simple in that it only disables tests. When updating to a newer version of Rust, the patch can usually be ported in a day.
Signed-off-by pgowda <[email protected]>Signed-off-by Vinay Kumar <[email protected]>
This patch fails because of:
AssertionError:
The following recipes do not have a maintainer assigned to them. Please add an entry to meta/conf/distro/include/maintainers.inc file. rust-testsuite (/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/recipes-devtools/rust/rust-testsuite_1.62.0.bb)
Some logs:
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3917/steps/15/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3866/steps/14/logs/stdio
Signature · 3 lines
Luca Ceresoli, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi Pgowda,
Quoted 45 lines · On Wed, 3 Aug 2022 15:40:15 +0200 "Luca Ceresoli via lists.openembedded.org"…
Quoted 32 lines · On Tue, 2 Aug 2022 17:50:05 +0530 "Pgowda" <[email protected]> wrote:
From: pgowda <[email protected]>
The patch implements Rust testing framework similar to other selftest, specifically the gcc selftest in OE. It uses the client and server based method to test the binaries for cross-target on the image. The test framework is a wrapper around the Rust build system as ./x.py test.
It tests many functionalities of Rust distribution like tools, documentation, libraries, packages, tools, Cargo, Crater etc. Please refer the following link for detailed description of Rust testing:-
https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-testsTo support the rust tests in oe-core, the following functions were added:-
setup_cargo_environment(): Build bootstrap and some early stage tools.do_rust_setup_snapshot(): Install the snapshot version of rust binaries.do_configure(): To generate config.tomldo_compile(): To build "remote-test-server" for qemu target image.Approximate Number of Tests Run in the Rust Testsuite :- 18000 Approximate Number of Tests that FAIL in bitbake environment :- 100-150 Normally majority of the testcases are present in major folder "test/" It contributes to more than 80% of the testcases present in Rust test framework. These tests pass as expected on any Rust versions without much fuss. The tests that fail are of less important and contribute to less than 2% of the total testcases. These minor tests are observed to work on some versions and fail on others. They have to be added, ignored or excluded for different versions as per the behavior. These tests have been ignored or excluded in the Rust selftest environment to generate success of completing the testsuite.
Quoted 7 lines
These tests work in parallel mode even in the skipped test mode as expected. Although the patch to disable tests is large, it is very simple in that it only disables tests. When updating to a newer version of Rust, the patch can usually be ported in a day.
Signed-off-by pgowda <[email protected]>Signed-off-by Vinay Kumar <[email protected]>This patch fails because of:
Quoted 8 lines
AssertionError:
The following recipes do not have a maintainer assigned to them. Please add an entry to meta/conf/distro/include/maintainers.inc file. rust-testsuite (/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/recipes-devtools/rust/rust-testsuite_1.62.0.bb)Some logs:
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3917/steps/15/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3866/steps/14/logs/stdio
...and there is another failure, this time it is an actual build error:
| CMake Error at /home/pokybuild/yocto-worker/qemux86-64/build/build-st-789666/tmp/work/core2-64-poky-linux/rust-testsuite/1.62.0-r0/recipe-sysroot-native/usr/share/cmake-3.23/Modules/CMakeTestCXXCompiler.cmake:62 (message): | The C++ compiler | | "/home/pokybuild/yocto-worker/qemux86-64/build/build-st-789666/tmp/work/core2-64-poky-linux/rust-testsuite/1.62.0-r0/wrapper/build-rust-cxx" | | is not able to compile a simple test program. | | It fails with the following output: | [...] | /home/pokybuild/yocto-worker/qemux86-64/build/build-st-789666/tmp/hosttools/ld: cannot find -lstdc++ | collect2: error: ld returned 1 exit status
Full log at:
https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/5569/steps/19/logs/stdio
Signature · 3 lines
Luca Ceresoli, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Hi Luca,
Updated the maintainers list and submitted the patch as:-
https://lists.openembedded.org/g/openembedded-core/message/168819
Thanks,
Pgowda
Quoted 85 lines · Luca Ceresoli
Hi Pgowda,
Quoted 45 lines · On Wed, 3 Aug 2022 15:40:15 +0200 "Luca Ceresoli via lists.openembedded.org"…
Quoted 32 lines · On Tue, 2 Aug 2022 17:50:05 +0530 "Pgowda" <[email protected]> wrote:
From: pgowda <[email protected]>
The patch implements Rust testing framework similar to other selftest, specifically the gcc selftest in OE. It uses the client and server based method to test the binaries for cross-target on the image. The test framework is a wrapper around the Rust build system as ./x.py test.
It tests many functionalities of Rust distribution like tools, documentation, libraries, packages, tools, Cargo, Crater etc. Please refer the following link for detailed description of Rust testing:-
https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-testsTo support the rust tests in oe-core, the following functions were added:-
setup_cargo_environment(): Build bootstrap and some early stage tools.do_rust_setup_snapshot(): Install the snapshot version of rust binaries.do_configure(): To generate config.tomldo_compile(): To build "remote-test-server" for qemu target image.Approximate Number of Tests Run in the Rust Testsuite :- 18000 Approximate Number of Tests that FAIL in bitbake environment :- 100-150 Normally majority of the testcases are present in major folder "test/" It contributes to more than 80% of the testcases present in Rust test framework. These tests pass as expected on any Rust versions without much fuss. The tests that fail are of less important and contribute to less than 2% of the total testcases. These minor tests are observed to work on some versions and fail on others. They have to be added, ignored or excluded for different versions as per the behavior. These tests have been ignored or excluded in the Rust selftest environment to generate success of completing the testsuite.
Quoted 7 lines
These tests work in parallel mode even in the skipped test mode as expected. Although the patch to disable tests is large, it is very simple in that it only disables tests. When updating to a newer version of Rust, the patch can usually be ported in a day.
Signed-off-by pgowda <[email protected]>Signed-off-by Vinay Kumar <[email protected]>This patch fails because of:
Quoted 8 lines
AssertionError:
The following recipes do not have a maintainer assigned to them. Please add an entry to meta/conf/distro/include/maintainers.inc file. rust-testsuite (/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/recipes-devtools/rust/rust-testsuite_1.62.0.bb)Some logs:
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3917/steps/15/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3866/steps/14/logs/stdio...and there is another failure, this time it is an actual build error:
| CMake Error at /home/pokybuild/yocto-worker/qemux86-64/build/build-st-789666/tmp/work/core2-64-poky-linux/rust-testsuite/1.62.0-r0/recipe-sysroot-native/usr/share/cmake-3.23/Modules/CMakeTestCXXCompiler.cmake:62 (message): | The C++ compiler | | "/home/pokybuild/yocto-worker/qemux86-64/build/build-st-789666/tmp/work/core2-64-poky-linux/rust-testsuite/1.62.0-r0/wrapper/build-rust-cxx" | | is not able to compile a simple test program. | | It fails with the following output: | [...] | /home/pokybuild/yocto-worker/qemux86-64/build/build-st-789666/tmp/hosttools/ld: cannot find -lstdc++ | collect2: error: ld returned 1 exit statusFull log at:
https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/5569/steps/19/logs/stdio
--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
Quoted 7 lines · Pgowda
Hi Luca,
Updated the maintainers list and submitted the patch as:-
https://lists.openembedded.org/g/openembedded-core/message/168819Thanks,
Pgowda
Thanks Naveen.
For future follow-ups on this thread and when there is more than one version, it's good to provide a brief cover letter / section that explains what has changed since the last version. eg:
See:
https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded
and part of:
https://www.kernel.org/doc/html/v4.17/process/submitting-patches.html
We don't require this but for large patches that have several iterations, it's very helpful for the sender to summarize what has changed so that the gatekeeper doesn't have to dig up the previous patch.
Thanks,
../Randy
Quoted 93 lines · Luca Ceresoli
Quoted 91 lines
Hi Pgowda,
Quoted 44 lines · On Wed, 3 Aug 2022 15:40:15 +0200 "Luca Ceresoli via lists.openembedded.org"…
Quoted 32 lines · On Tue, 2 Aug 2022 17:50:05 +0530 "Pgowda" <[email protected]> wrote:
From: pgowda <[email protected]>
The patch implements Rust testing framework similar to other selftest, specifically the gcc selftest in OE. It uses the client and server based method to test the binaries for cross-target on the image. The test framework is a wrapper around the Rust build system as ./x.py test.
It tests many functionalities of Rust distribution like tools, documentation, libraries, packages, tools, Cargo, Crater etc. Please refer the following link for detailed description of Rust testing:-
https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-testsTo support the rust tests in oe-core, the following functions were added:-
setup_cargo_environment(): Build bootstrap and some early stage tools.do_rust_setup_snapshot(): Install the snapshot version of rust binaries.do_configure(): To generate config.tomldo_compile(): To build "remote-test-server" for qemu target image.Approximate Number of Tests Run in the Rust Testsuite :- 18000 Approximate Number of Tests that FAIL in bitbake environment :- 100-150 Normally majority of the testcases are present in major folder "test/" It contributes to more than 80% of the testcases present in Rust test framework. These tests pass as expected on any Rust versions without much fuss. The tests that fail are of less important and contribute to less than 2% of the total testcases. These minor tests are observed to work on some versions and fail on others. They have to be added, ignored or excluded for different versions as per the behavior. These tests have been ignored or excluded in the Rust selftest environment to generate success of completing the testsuite.
Quoted 7 lines
These tests work in parallel mode even in the skipped test mode as expected. Although the patch to disable tests is large, it is very simple in that it only disables tests. When updating to a newer version of Rust, the patch can usually be ported in a day.
Signed-off-by pgowda <[email protected]>Signed-off-by Vinay Kumar <[email protected]>This patch fails because of:
Quoted 8 lines
AssertionError:
The following recipes do not have a maintainer assigned to them. Please add an entry to meta/conf/distro/include/maintainers.inc file. rust-testsuite (/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/recipes-devtools/rust/rust-testsuite_1.62.0.bb)Some logs:
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3917/steps/15/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/3866/steps/14/logs/stdio...and there is another failure, this time it is an actual build error:
| CMake Error at /home/pokybuild/yocto-worker/qemux86-64/build/build-st-789666/tmp/work/core2-64-poky-linux/rust-testsuite/1.62.0-r0/recipe-sysroot-native/usr/share/cmake-3.23/Modules/CMakeTestCXXCompiler.cmake:62 (message): | The C++ compiler | | "/home/pokybuild/yocto-worker/qemux86-64/build/build-st-789666/tmp/work/core2-64-poky-linux/rust-testsuite/1.62.0-r0/wrapper/build-rust-cxx" | | is not able to compile a simple test program. | | It fails with the following output: | [...] | /home/pokybuild/yocto-worker/qemux86-64/build/build-st-789666/tmp/hosttools/ld: cannot find -lstdc++ | collect2: error: ld returned 1 exit statusFull log at:
https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/5569/steps/19/logs/stdio
--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#168820): https://lists.openembedded.org/g/openembedded-core/message/168820
Mute This Topic: https://lists.openembedded.org/mt/92768863/3616765
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 2 lines
# Randy MacLeod # Wind River Linux
Hi Pgowda,
Quoted 4 lines · On Wed, 3 Aug 2022 19:58:30 +0530 pgowda cve <[email protected]> wrote:
Hi Luca,
Updated the maintainers list and submitted the patch as:-
https://lists.openembedded.org/g/openembedded-core/message/168819
Does this also fix the strictly-build failure, or only the lack of maintainer?
Signature · 3 lines
Luca Ceresoli, Bootlin Embedded Linux and Kernel engineering https://bootlin.com