Skip to contentSkip to conversation

Rust Oe-Selftest implementation

PATCH Superseded [email protected] 2 participants 4 messages Aug 3, 2022 – Aug 6, 2022

v8 of 12 Superseded Latest is v12 →

lore
Share & tools
lore
#0 lore raw

PATCH v8 1/1 Rust Oe-Selftest implementation

From: pgowda <[email protected]>

The patch implements Rust testing framework similar to other selftest, specifically the gcc selftest in OE. It uses the client and server based method to test the binaries for cross-target on the image. The test framework is a wrapper around the Rust build system as ./x.py test.
It tests many functionalities of Rust distribution like tools, documentation, libraries, packages, tools, Cargo, Crater etc. Please refer the following link for detailed description of Rust testing:-
https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests

To support the rust tests in oe-core, the following functions were added:-
setup_cargo_environment(): Build bootstrap and some early stage tools. do_rust_setup_snapshot(): Install the snapshot version of rust binaries. do_configure(): To generate config.toml
do_compile(): To build "remote-test-server" for qemu target image.

Approximate Number of Tests Run in the Rust Testsuite :- 18000 Approximate Number of Tests that FAIL in bitbake environment :- 100-150 Normally majority of the testcases are present in major folder "test/" It contributes to more than 80% of the testcases present in Rust test framework. These tests pass as expected on any Rust versions without much fuss. The tests that fail are of less important and contribute to less than 2% of the total testcases. These minor tests are observed to work on some versions and fail on others. They have to be added, ignored or excluded for different versions as per the behavior. These tests have been ignored or excluded in the Rust selftest environment to generate success of completing the testsuite.

These tests work in parallel mode even in the skipped test mode as expected. Although the patch to disable tests is large, it is very simple in that it only disables tests. When updating to a newer version of Rust, the patch can usually be ported in a day.

---
 meta/classes/rust-target-config.bbclass       |   3 +-
 meta/conf/distro/include/maintainers.inc      |   1 +
 meta/lib/oeqa/selftest/cases/rust.py          |  57 +++
 meta/recipes-devtools/rust/rust-testsuite.inc | 164 ++++++
 .../rust-testsuite/rust-oe-selftest.patch     | 477 ++++++++++++++++++
 .../rust/rust-testsuite_1.62.0.bb             |   3 +
 6 files changed, 704 insertions(+), 1 deletion(-)
 create mode 100644 meta/lib/oeqa/selftest/cases/rust.py
 create mode 100644 meta/recipes-devtools/rust/rust-testsuite.inc
 create mode 100644 meta/recipes-devtools/rust/rust-testsuite/rust-oe-selftest.patch
 create mode 100644 meta/recipes-devtools/rust/rust-testsuite_1.62.0.bb
meta/classes/rust-target-config.bbclass +2 -1
index 87b7dee3ed..17eb18eba1 100644
@@ -344,7 +344,8 @@ def rust_gen_target(d, thing, wd, arch):
     tspec['linker-is-gnu'] = True
     tspec['linker-flavor'] = "gcc"
     tspec['has-rpath'] = True
-    tspec['has-elf-tls'] = True
+    # FIXME: Around 270 testcases fail with elf-tls enabled.
+    # tspec['has-elf-tls'] = True
     tspec['position-independent-executables'] = True
     tspec['panic-strategy'] = d.getVar("RUST_PANIC_STRATEGY")
 
meta/conf/distro/include/maintainers.inc +1 -0
index e20275c674..9f6418456f 100644
@@ -723,6 +723,7 @@ RECIPE_MAINTAINER:pn-rust-crosssdk-${SDK_ARCH}-glibc = "Randy MacLeod <Randy.Mac
 RECIPE_MAINTAINER:pn-rust-cross-canadian-${TRANSLATED_TARGET_ARCH} = "Randy MacLeod <[email protected]>"
 RECIPE_MAINTAINER:pn-rust-hello-world = "Randy MacLeod <[email protected]>"
 RECIPE_MAINTAINER:pn-rust-llvm = "Randy MacLeod <[email protected]>"
+RECIPE_MAINTAINER:pn-rust-testsuite = "Randy MacLeod <[email protected]>"
 RECIPE_MAINTAINER:pn-rust-tools-cross-canadian-${TRANSLATED_TARGET_ARCH} = "Randy MacLeod <[email protected]>"
 RECIPE_MAINTAINER:pn-rxvt-unicode = "Unassigned <[email protected]>"
 RECIPE_MAINTAINER:pn-sato-screenshot = "Ross Burton <[email protected]>"
meta/lib/oeqa/selftest/cases/rust.py new file +57 -0

57 lines not on this page · raw message

meta/recipes-devtools/rust/rust-testsuite.inc new file +164 -0

164 lines not on this page · raw message

meta/recipes-devtools/rust/rust-testsuite/rust-oe-selftest.patch new file +477 -0

477 lines not on this page · raw message

meta/recipes-devtools/rust/rust-testsuite_1.62.0.bb new file +3 -0
new file mode 100644
index 0000000000..d334231c8f
@@ -0,0 +1,3 @@
+require rust-testsuite.inc
+require rust-source.inc
+require rust-snapshot.inc
Signature · 1 line
2.25.1
#1 lore raw

Hello Pgowda,

Quoted 32 lines · On Wed, 3 Aug 2022 19:57:34 +0530 "Pgowda" <[email protected]> wrote:

From: pgowda <[email protected]>

The patch implements Rust testing framework similar to other selftest, specifically the gcc selftest in OE. It uses the client and server based method to test the binaries for cross-target on the image. The test framework is a wrapper around the Rust build system as ./x.py test.
It tests many functionalities of Rust distribution like tools, documentation, libraries, packages, tools, Cargo, Crater etc. Please refer the following link for detailed description of Rust testing:-
https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests

To support the rust tests in oe-core, the following functions were added:-
setup_cargo_environment(): Build bootstrap and some early stage tools. do_rust_setup_snapshot(): Install the snapshot version of rust binaries. do_configure(): To generate config.toml
do_compile(): To build "remote-test-server" for qemu target image.

Approximate Number of Tests Run in the Rust Testsuite :- 18000 Approximate Number of Tests that FAIL in bitbake environment :- 100-150 Normally majority of the testcases are present in major folder "test/" It contributes to more than 80% of the testcases present in Rust test framework. These tests pass as expected on any Rust versions without much fuss. The tests that fail are of less important and contribute to less than 2% of the total testcases. These minor tests are observed to work on some versions and fail on others. They have to be added, ignored or excluded for different versions as per the behavior. These tests have been ignored or excluded in the Rust selftest environment to generate success of completing the testsuite.

Quoted 7 lines

These tests work in parallel mode even in the skipped test mode as expected. Although the patch to disable tests is large, it is very simple in that it only disables tests. When updating to a newer version of Rust, the patch can usually be ported in a day.

This patch triggers the same error I had reported for v7:

https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/5588/steps/18/logs/stdio

https://lists.openembedded.org/g/openembedded-core/message/168816

Can please try to reproduce the issue locally, to ensure you can investigate it?

Also, when sending a new version, don't forget to add a short changelog as suggested by Randy:
https://lists.openembedded.org/g/openembedded-core/message/168821

Signature · 3 lines
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
author
#2 lore raw

Hi Luca,

Thanks for the update.
The issue is due to the requirement of libstdc++ in rust-1.62.
It was not the same with 1.61 and earlier.
It requires the installation of libstdc++ in the host machine. Can you please let us know whether it should be fixed in rust-testsuite or can they be installed in the host machine as a prerequisite for rust-1.62??

Thanks,
Pgowda

Quoted 65 lines · Luca Ceresoli

Hello Pgowda,

Quoted 32 lines · On Wed, 3 Aug 2022 19:57:34 +0530 "Pgowda" <[email protected]> wrote:

From: pgowda <[email protected]>

The patch implements Rust testing framework similar to other selftest, specifically the gcc selftest in OE. It uses the client and server based method to test the binaries for cross-target on the image. The test framework is a wrapper around the Rust build system as ./x.py test.
It tests many functionalities of Rust distribution like tools, documentation, libraries, packages, tools, Cargo, Crater etc. Please refer the following link for detailed description of Rust testing:-
https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests

To support the rust tests in oe-core, the following functions were added:-
setup_cargo_environment(): Build bootstrap and some early stage tools. do_rust_setup_snapshot(): Install the snapshot version of rust binaries. do_configure(): To generate config.toml
do_compile(): To build "remote-test-server" for qemu target image.

Approximate Number of Tests Run in the Rust Testsuite :- 18000 Approximate Number of Tests that FAIL in bitbake environment :- 100-150 Normally majority of the testcases are present in major folder "test/" It contributes to more than 80% of the testcases present in Rust test framework. These tests pass as expected on any Rust versions without much fuss. The tests that fail are of less important and contribute to less than 2% of the total testcases. These minor tests are observed to work on some versions and fail on others. They have to be added, ignored or excluded for different versions as per the behavior. These tests have been ignored or excluded in the Rust selftest environment to generate success of completing the testsuite.

Quoted 7 lines

These tests work in parallel mode even in the skipped test mode as expected. Although the patch to disable tests is large, it is very simple in that it only disables tests. When updating to a newer version of Rust, the patch can usually be ported in a day.

This patch triggers the same error I had reported for v7:

https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/5588/steps/18/logs/stdio

https://lists.openembedded.org/g/openembedded-core/message/168816

Can please try to reproduce the issue locally, to ensure you can investigate it?

Also, when sending a new version, don't forget to add a short changelog as suggested by Randy:
https://lists.openembedded.org/g/openembedded-core/message/168821

--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

author
#3 lore raw
In reply to Luca Ceresoli

Hi Luca,

Thanks for informing me about the error.
The patch has been updated to fix the libstdc++ environment issue as reported by you.
https://lists.openembedded.org/g/openembedded-core/message/168974

Thanks,
Pgowda

Quoted 65 lines · Luca Ceresoli

Hello Pgowda,

Quoted 32 lines · On Wed, 3 Aug 2022 19:57:34 +0530 "Pgowda" <[email protected]> wrote:

From: pgowda <[email protected]>

The patch implements Rust testing framework similar to other selftest, specifically the gcc selftest in OE. It uses the client and server based method to test the binaries for cross-target on the image. The test framework is a wrapper around the Rust build system as ./x.py test.
It tests many functionalities of Rust distribution like tools, documentation, libraries, packages, tools, Cargo, Crater etc. Please refer the following link for detailed description of Rust testing:-
https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests

To support the rust tests in oe-core, the following functions were added:-
setup_cargo_environment(): Build bootstrap and some early stage tools. do_rust_setup_snapshot(): Install the snapshot version of rust binaries. do_configure(): To generate config.toml
do_compile(): To build "remote-test-server" for qemu target image.

Approximate Number of Tests Run in the Rust Testsuite :- 18000 Approximate Number of Tests that FAIL in bitbake environment :- 100-150 Normally majority of the testcases are present in major folder "test/" It contributes to more than 80% of the testcases present in Rust test framework. These tests pass as expected on any Rust versions without much fuss. The tests that fail are of less important and contribute to less than 2% of the total testcases. These minor tests are observed to work on some versions and fail on others. They have to be added, ignored or excluded for different versions as per the behavior. These tests have been ignored or excluded in the Rust selftest environment to generate success of completing the testsuite.

Quoted 7 lines

These tests work in parallel mode even in the skipped test mode as expected. Although the patch to disable tests is large, it is very simple in that it only disables tests. When updating to a newer version of Rust, the patch can usually be ported in a day.

This patch triggers the same error I had reported for v7:

https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/5588/steps/18/logs/stdio

https://lists.openembedded.org/g/openembedded-core/message/168816

Can please try to reproduce the issue locally, to ensure you can investigate it?

Also, when sending a new version, don't forget to add a short changelog as suggested by Randy:
https://lists.openembedded.org/g/openembedded-core/message/168821

--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com