Skip to contentSkip to conversation

listing of tests is broken

Buildroot New Yegor Yefremov 3 participants 9 messages Feb 9, 2018 – Feb 13, 2018

lore
Share & tools
lore
#0 lore raw

./support/testing/run-tests -l

File "/home/user/br_test/support/testing/infra/basetest.py", line
45, in __init__

    self.config += "\nBR2_JLEVEL={}\n".format(self.jlevel)

TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

Any idea?

Yegor

#1 lore raw

Hello,

Quoted 8 lines · Yegor Yefremov

./support/testing/run-tests -l

File "/home/user/br_test/support/testing/infra/basetest.py", line
45, in __init__

    self.config += "\nBR2_JLEVEL={}\n".format(self.jlevel)

TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

Any idea?

Yes, it is a fallout from the Rust tests, one of the class has no "config" variable, which causes this breakage.

I have not yet decided if the right fix is to replace config = None by config = list() in the base test class, or to fix the Rust test case.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com
#2 lore raw
Quoted 10 lines · Thomas Petazzoni

Hello,

Quoted 7 lines · Yegor Yefremov

./support/testing/run-tests -l

File "/home/user/br_test/support/testing/infra/basetest.py", line
45, in __init__

    self.config += "\nBR2_JLEVEL={}\n".format(self.jlevel)

TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

Quoted 7 lines
Quoted 1 line

Any idea?

Yes, it is a fallout from the Rust tests, one of the class has no "config" variable, which causes this breakage.

I have not yet decided if the right fix is to replace config = None by config = list() in the base test class, or to fix the Rust test case.

I'm for the first fix: config = list()

Yegor

#3 lore raw

Hello,

Quoted 4 lines · Yegor Yefremov
Quoted 2 lines

I have not yet decided if the right fix is to replace config = None by config = list() in the base test class, or to fix the Rust test case.

I'm for the first fix: config = list()

Before going with this fix, I'd like to understand if what the Rust tests are doing is legit or not. If it isn't, then we may not necessarily want to paper over the problem.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com
#4 lore raw

Hello,

It happens on the master branch too.

Quoted 10 lines · Thomas Petazzoni
Quoted 4 lines · Yegor Yefremov
Quoted 2 lines

I have not yet decided if the right fix is to replace config = None by config = list() in the base test class, or to fix the Rust test case.

I'm for the first fix: config = list()

Before going with this fix, I'd like to understand if what the Rust tests are doing is legit or not. If it isn't, then we may not necessarily want to paper over the problem.

The correct fix IMO is to change the Rust test case.

With the current code, nose2 will collect 3 test cases from the file:
TestRustBase, TestRustBin and TestRust. It happens because all three have a method called test_something. But TestRustBase is not a test case by itself.

So we should move the test_* method from the base class to the 2 derived classes (yes, duplicating the code), like it is already done by the 'init' test cases.

An alternative solution is to rename the method in the base class to not start with test_ and create a test_run method in each derived class calling it. The method is short, so I don't think we need to do this.

With the listing of tests broken, also 'make .gitlab-ci.yml' is broken. Based only on this I would say we should fix that on master branch. But see below... And also the update to .gitlab-ci.yml must be committed. This way we fix check-gitlab-ci.yml in GitLab:
https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035

But after fixing those, the TestRustBin still fails at GitLab:
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092045
from test_rust.py, line 39, like this:

Quoted 1 line

SystemError: Cargo init failed

The build.log ends in these messages:

Quoted 4 lines

error: Failed to create project hello-world at /tmp/tmp_US3dA-br2-testing-rust/hello-world

Caused by:
could not determine the current user, please set $USER

The same test case does not fail for me when it runs outside the Docker image, so I guess we will need to tweak the Docker image.

I don't have the result yet for TestRust in GitLab. It is still running (105 minutes and counting) by the time I write this.
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092044
But running outside the Docker image it also passes.

... Would it be the case to revert only the rust test case only from the master branch? If yes, extra care must be taken when merging the branches to not remove it from the next branch too.

Regards,
Ricardo

#5 lore raw

Hello,

Quoted 1 line · Ricardo Martincoski

The correct fix IMO is to change the Rust test case.

Agreed.

Quoted 10 lines

With the current code, nose2 will collect 3 test cases from the file:
TestRustBase, TestRustBin and TestRust. It happens because all three have a method called test_something. But TestRustBase is not a test case by itself.

So we should move the test_* method from the base class to the 2 derived classes (yes, duplicating the code), like it is already done by the 'init' test cases.

An alternative solution is to rename the method in the base class to not start with test_ and create a test_run method in each derived class calling it. The method is short, so I don't think we need to do this.

Yes, duplicating would be fine for now.

Quoted 19 lines

With the listing of tests broken, also 'make .gitlab-ci.yml' is broken. Based only on this I would say we should fix that on master branch. But see below... And also the update to .gitlab-ci.yml must be committed. This way we fix check-gitlab-ci.yml in GitLab:
https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035

But after fixing those, the TestRustBin still fails at GitLab:
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092045
from test_rust.py, line 39, like this:

Quoted 1 line

SystemError: Cargo init failed

The build.log ends in these messages:

Quoted 4 lines

error: Failed to create project hello-world at /tmp/tmp_US3dA-br2-testing-rust/hello-world

Caused by:
could not determine the current user, please set $USER

The same test case does not fail for me when it runs outside the Docker image, so I guess we will need to tweak the Docker image.

Quoted 8 lines

I don't have the result yet for TestRust in GitLab. It is still running (105 minutes and counting) by the time I write this.
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092044
But running outside the Docker image it also passes.

... Would it be the case to revert only the rust test case only from the master branch? If yes, extra care must be taken when merging the branches to not remove it from the next branch too.

I'd say, we fix the test enumeration problem, and then let the Rust test case fail. Hopefully Eric Le Bihan (added in Cc) will chime in and fix the Rust test failure.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com
#6 lore raw

PATCH 1/1 support/testing: fix job check-gitlab-ci.yml

Currently 'run-tests -l' is broken. It breaks 'make .gitlab-ci.yml' that in turn breaks the job in GitLab.

TestRustBase is not a test case by itself, so it can't have a method with the name starting with "test_" otherwise nose2 assumes it is a test case.
Move the test_run method from the base class to the derived classes.

While at it, update .gitlab-ci.yml with the new test cases.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035

Cc Eric Le Bihan <[email protected]>
Cc Thomas Petazzoni <[email protected]>
Cc Yann E. MORIN <[email protected]>
Cc Yegor Yefremov <[email protected]>

---
check-gitlab-ci.yml after the fix:
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092002
which is part of a full run:
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/17391318

---
 .gitlab-ci.yml                             |  2 ++
 support/testing/tests/package/test_rust.py | 18 ++++++++++++------
 2 files changed, 14 insertions(+), 6 deletions(-)
.gitlab-ci.yml +2 -0
index 30a4399..f76e53b 100644
@@ -282,6 +282,8 @@ tests.package.test_ipython.TestIPythonPy2: *runtime_test
 tests.package.test_ipython.TestIPythonPy3: *runtime_test
 tests.package.test_python.TestPython2: *runtime_test
 tests.package.test_python.TestPython3: *runtime_test
+tests.package.test_rust.TestRust: *runtime_test
+tests.package.test_rust.TestRustBin: *runtime_test
 tests.toolchain.test_external.TestExternalToolchainBuildrootMusl: *runtime_test
 tests.toolchain.test_external.TestExternalToolchainBuildrootuClibc: *runtime_test
 tests.toolchain.test_external.TestExternalToolchainCCache: *runtime_test
support/testing/tests/package/test_rust.py +12 -6
index 8035f8b..e6c0de2 100644
@@ -53,12 +53,6 @@ class TestRustBase(infra.basetest.BRTest):
         self.b.build()
         shutil.rmtree(workdir)
 
-    def test_run(self):
-        self.build_test_prog()
-        self.login()
-        _, exit_code = self.emulator.run(self.crate)
-        self.assertEqual(exit_code, 0)
-
 
 class TestRustBin(TestRustBase):
     config = \
@@ -82,6 +76,12 @@ class TestRustBin(TestRustBase):
              BR2_PACKAGE_HOST_RUSTC=y
              """
 
+    def test_run(self):
+        self.build_test_prog()
+        self.login()
+        _, exit_code = self.emulator.run(self.crate)
+        self.assertEqual(exit_code, 0)
+
 
 class TestRust(TestRustBase):
     config = \
@@ -105,3 +105,9 @@ class TestRust(TestRustBase):
              BR2_PACKAGE_HOST_RUSTC=y
              BR2_PACKAGE_HOST_RUST=y
              """
+
+    def test_run(self):
+        self.build_test_prog()
+        self.login()
+        _, exit_code = self.emulator.run(self.crate)
+        self.assertEqual(exit_code, 0)
Signature · 1 line
2.7.4
#7 lore raw

PATCH 1/1 support/testing: fix job check-gitlab-ci.yml

Quoted 13 lines · Ricardo Martincoski

Currently 'run-tests -l' is broken. It breaks 'make .gitlab-ci.yml' that in turn breaks the job in GitLab.

TestRustBase is not a test case by itself, so it can't have a method with the name starting with "test_" otherwise nose2 assumes it is a test case.
Move the test_run method from the base class to the derived classes.

While at it, update .gitlab-ci.yml with the new test cases.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035

Quoted 6 lines
Cc Eric Le Bihan <[email protected]>
Cc Thomas Petazzoni <[email protected]>
Cc Yann E. MORIN <[email protected]>
Cc Yegor Yefremov <[email protected]>
Quoted 66 lines

---
check-gitlab-ci.yml after the fix:
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092002
which is part of a full run:
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/17391318

---
 .gitlab-ci.yml                             |  2 ++
 support/testing/tests/package/test_rust.py | 18 ++++++++++++------
 2 files changed, 14 insertions(+), 6 deletions(-)
.gitlab-ci.yml +2 -0
index 30a4399..f76e53b 100644
@@ -282,6 +282,8 @@ tests.package.test_ipython.TestIPythonPy2: *runtime_test
 tests.package.test_ipython.TestIPythonPy3: *runtime_test
 tests.package.test_python.TestPython2: *runtime_test
 tests.package.test_python.TestPython3: *runtime_test
+tests.package.test_rust.TestRust: *runtime_test
+tests.package.test_rust.TestRustBin: *runtime_test
 tests.toolchain.test_external.TestExternalToolchainBuildrootMusl: *runtime_test
 tests.toolchain.test_external.TestExternalToolchainBuildrootuClibc: *runtime_test
 tests.toolchain.test_external.TestExternalToolchainCCache: *runtime_test
support/testing/tests/package/test_rust.py +12 -7
index 8035f8b..e6c0de2 100644
@@ -53,12 +53,6 @@ class TestRustBase(infra.basetest.BRTest):
         self.b.build()
         shutil.rmtree(workdir)
 
-    def test_run(self):
-        self.build_test_prog()
-        self.login()
-        _, exit_code = self.emulator.run(self.crate)
-        self.assertEqual(exit_code, 0)
-
 
 class TestRustBin(TestRustBase):
     config = \
@@ -82,6 +76,12 @@ class TestRustBin(TestRustBase):
              BR2_PACKAGE_HOST_RUSTC=y
              """
 
+    def test_run(self):
+        self.build_test_prog()
+        self.login()
+        _, exit_code = self.emulator.run(self.crate)
+        self.assertEqual(exit_code, 0)
+
 
 class TestRust(TestRustBase):
     config = \
@@ -105,3 +105,9 @@ class TestRust(TestRustBase):
              BR2_PACKAGE_HOST_RUSTC=y
              BR2_PACKAGE_HOST_RUST=y
              """
+
+    def test_run(self):
+        self.build_test_prog()
+        self.login()
+        _, exit_code = self.emulator.run(self.crate)
+        self.assertEqual(exit_code, 0)
--

2.7.4

#8 lore raw

PATCH 1/1 support/testing: fix job check-gitlab-ci.yml

In reply to Ricardo Martincoski

Hello,

Quoted 24 lines · Ricardo Martincoski

Currently 'run-tests -l' is broken. It breaks 'make .gitlab-ci.yml' that in turn breaks the job in GitLab.

TestRustBase is not a test case by itself, so it can't have a method with the name starting with "test_" otherwise nose2 assumes it is a test case.
Move the test_run method from the base class to the derived classes.

While at it, update .gitlab-ci.yml with the new test cases.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/52000035

Cc Eric Le Bihan <[email protected]>
Cc Thomas Petazzoni <[email protected]>
Cc Yann E. MORIN <[email protected]>
Cc Yegor Yefremov <[email protected]>

---
check-gitlab-ci.yml after the fix:
https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/52092002
which is part of a full run:
https://gitlab.com/RicardoMartincoski/buildroot/pipelines/17391318

Quoted 4 lines
---
 .gitlab-ci.yml                             |  2 ++
 support/testing/tests/package/test_rust.py | 18 ++++++++++++------
 2 files changed, 14 insertions(+), 6 deletions(-)

Applied to master, thanks.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com