Skip to contentSkip to conversation

support/testing: add openssh runtime test

PATCH New Romain Naour 3 participants 13 messages Mar 30, 2020 – Apr 4, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 support/testing: add openssh runtime test

This new runtime test is based on test_dropbear.py. The only required change is to use "-oStrictHostKeyChecking=no" instead of "-y" to accept the new key.

Since the base test infra only provide a uClibc-ng toolchain, add a second test using a glibc based internal toolchain.

For example, this allow to trigger the openssh 8.1p bug with glibc 2.31 [1].

[1] https://bugs.archlinux.org/task/65386

---
 .gitlab-ci.yml                                |  2 +
 DEVELOPERS                                    |  1 +
 support/testing/tests/package/test_openssh.py | 78 +++++++++++++++++++
 .../tests/package/test_openssh/post-build.sh  |  4 +
 4 files changed, 85 insertions(+)
 create mode 100644 support/testing/tests/package/test_openssh.py
 create mode 100755 support/testing/tests/package/test_openssh/post-build.sh
.gitlab-ci.yml +2 -0
index 416ed3c0ca..0fae2e4e53 100644
@@ -430,6 +430,8 @@ tests.package.test_lxc.TestLxc: { extends: .runtime_test }
 tests.package.test_lzlib.TestLuaLzlib: { extends: .runtime_test }
 tests.package.test_netdata.TestNetdata: { extends: .runtime_test }
 tests.package.test_openjdk.TestOpenJdk: { extends: .runtime_test }
+tests.package.test_openssh.TestOpenSshGlibc: { extends: .runtime_test }
+tests.package.test_openssh.TestOpenSshuClibc: { extends: .runtime_test }
 tests.package.test_opkg.TestOpkg: { extends: .runtime_test }
 tests.package.test_perl.TestPerl: { extends: .runtime_test }
 tests.package.test_perl_class_load.TestPerlClassLoad: { extends: .runtime_test }
DEVELOPERS +1 -0
index f1bf5c263c..009063347b 100644
@@ -2214,6 +2214,7 @@ F:	package/waffle/
 F:	package/xenomai/
 F:	package/zziplib/
 F:	support/testing/tests/package/test_glxinfo.py
+F:	support/testing/tests/package/test_openssh.py
 F:	toolchain/
 
 N:	Roman Gorbenkov <[email protected]>
support/testing/tests/package/test_openssh.py new file +78 -0

78 lines not on this page · raw message

support/testing/tests/package/test_openssh/post-build.sh new file +4 -0
new file mode 100755
index 0000000000..4c876ce554
@@ -0,0 +1,4 @@
+#!/usr/bin/env bash
+
+sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/g'  ${TARGET_DIR}/etc/ssh/sshd_config
+sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/g'  ${TARGET_DIR}/etc/ssh/sshd_config
Signature · 1 line
2.21.1
#1 lore raw

PATCH 2/2 package/openssh: bump to version 8.2p1

This new version is mandatory to allow the glibc package bump to version 2.31. Otherwise it's not possible to connect to the remote host, as reported by [1] [2]. Upstream commit [3][4] fixes the issue.

[1] https://bugs.archlinux.org/task/65386
[2] https://bugs.gentoo.org/708224
[3] https://github.com/openssh/openssh-portable/commit/beee0ef61866cb567b9abc23bd850f922e59e3f0
[4] https://github.com/openssh/openssh-portable/commit/69298ebfc2c066acee5d187eac8ce9f38c796630

Release Note:
https://www.openssh.com/txt/release-8.2

---
 package/openssh/openssh.hash | 4 ++--
 package/openssh/openssh.mk   | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
package/openssh/openssh.hash +2 -2
index 4db933f7b0..2f962d4337 100644
@@ -1,4 +1,4 @@
-# From https://www.openssh.com/txt/release-8.1 (base64 encoded)
-sha256 02f5dbef3835d0753556f973cd57b4c19b6b1f6cd24c03445e23ac77ca1b93ff  openssh-8.1p1.tar.gz
+# From https://www.openssh.com/txt/release-8.2 (base64 encoded)
+sha256 43925151e6cf6cee1450190c0e9af4dc36b41c12737619edff8bcebdff64e671  openssh-8.2p1.tar.gz
 # Locally calculated
 sha256 73d0db766229670c7b4e1ec5e6baed54977a0694a565e7cc878c45ee834045d7  LICENCE
package/openssh/openssh.mk +1 -1
index 0411133bdc..d50572128a 100644
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-OPENSSH_VERSION = 8.1p1
+OPENSSH_VERSION = 8.2p1
 OPENSSH_SITE = http://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable
 OPENSSH_LICENSE = BSD-3-Clause, BSD-2-Clause, Public Domain
 OPENSSH_LICENSE_FILES = LICENCE
Signature · 1 line
2.21.1
#2 lore raw

PATCH 2/2 package/openssh: bump to version 8.2p1

Hi,

I'm not able to complete the boot on qemu_aarch64_virt_defconfig. boot stuck after the network initialisation.

regards

Quoted 10 lines · Romain Naour

This new version is mandatory to allow the glibc package bump to version 2.31. Otherwise it's not possible to connect to the remote host, as reported by [1] [2]. Upstream commit [3][4] fixes the issue.

[1] https://bugs.archlinux.org/task/65386
[2] https://bugs.gentoo.org/708224
[3] https://github.com/openssh/openssh-portable/commit/beee0ef61866cb567b9abc23bd850f922e59e3f0
[4] https://github.com/openssh/openssh-portable/commit/69298ebfc2c066acee5d187eac8ce9f38c796630

Quoted 4 lines

Release Note:
https://www.openssh.com/txt/release-8.2

Quoted 36 lines
---
 package/openssh/openssh.hash | 4 ++--
 package/openssh/openssh.mk   | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
package/openssh/openssh.hash +2 -2
index 4db933f7b0..2f962d4337 100644
@@ -1,4 +1,4 @@
-# From https://www.openssh.com/txt/release-8.1 (base64 encoded)
-sha256 02f5dbef3835d0753556f973cd57b4c19b6b1f6cd24c03445e23ac77ca1b93ff  openssh-8.1p1.tar.gz
+# From https://www.openssh.com/txt/release-8.2 (base64 encoded)
+sha256 43925151e6cf6cee1450190c0e9af4dc36b41c12737619edff8bcebdff64e671  openssh-8.2p1.tar.gz
 # Locally calculated
 sha256 73d0db766229670c7b4e1ec5e6baed54977a0694a565e7cc878c45ee834045d7  LICENCE
package/openssh/openssh.mk +1 -2
index 0411133bdc..d50572128a 100644
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-OPENSSH_VERSION = 8.1p1
+OPENSSH_VERSION = 8.2p1
 OPENSSH_SITE = http://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable
 OPENSSH_LICENSE = BSD-3-Clause, BSD-2-Clause, Public Domain
 OPENSSH_LICENSE_FILES = LICENCE
--

2.21.1

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

#3 lore raw

PATCH 2/2 package/openssh: bump to version 8.2p1

Hi David,

Le 31/03/2020 ? 07:34, David PIERRET a ?crit?:

Quoted 4 lines

Hi,

I'm not able to complete the boot on qemu_aarch64_virt_defconfig. boot stuck after the network initialisation.

Thanks for testing!

Can you try by adding -device virtio-rng-pci on the Qemu command line ? The system emulated by Qemu lack of entropy sources, the boot process is stuck several dozen of seconde waiting for minimum entropy level.

Note, this is not related to the version bump. You should be able to reproduce the issue with openssh 8.1p1.

See:
https://git.buildroot.net/buildroot/commit/?id=450863097f588fe85d23e83cff14a75f1740d8fa

I'll double check on my side.

Best regards,
Romain

Quoted 55 lines

regards

Quoted 10 lines · Romain Naour

This new version is mandatory to allow the glibc package bump to version 2.31. Otherwise it's not possible to connect to the remote host, as reported by [1] [2]. Upstream commit [3][4] fixes the issue.

[1] https://bugs.archlinux.org/task/65386
[2] https://bugs.gentoo.org/708224
[3] https://github.com/openssh/openssh-portable/commit/beee0ef61866cb567b9abc23bd850f922e59e3f0
[4] https://github.com/openssh/openssh-portable/commit/69298ebfc2c066acee5d187eac8ce9f38c796630

Quoted 4 lines

Release Note:
https://www.openssh.com/txt/release-8.2

Quoted 36 lines
---
 package/openssh/openssh.hash | 4 ++--
 package/openssh/openssh.mk   | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
package/openssh/openssh.hash +2 -2
index 4db933f7b0..2f962d4337 100644
@@ -1,4 +1,4 @@
-# From https://www.openssh.com/txt/release-8.1 (base64 encoded)
-sha256 02f5dbef3835d0753556f973cd57b4c19b6b1f6cd24c03445e23ac77ca1b93ff  openssh-8.1p1.tar.gz
+# From https://www.openssh.com/txt/release-8.2 (base64 encoded)
+sha256 43925151e6cf6cee1450190c0e9af4dc36b41c12737619edff8bcebdff64e671  openssh-8.2p1.tar.gz
 # Locally calculated
 sha256 73d0db766229670c7b4e1ec5e6baed54977a0694a565e7cc878c45ee834045d7  LICENCE
package/openssh/openssh.mk +1 -2
index 0411133bdc..d50572128a 100644
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-OPENSSH_VERSION = 8.1p1
+OPENSSH_VERSION = 8.2p1
 OPENSSH_SITE = http://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable
 OPENSSH_LICENSE = BSD-3-Clause, BSD-2-Clause, Public Domain
 OPENSSH_LICENSE_FILES = LICENCE
--

2.21.1

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

#4 lore raw

PATCH 2/2 package/openssh: bump to version 8.2p1

Hi Romain,

Quoted 6 lines · Romain Naour

Hi David,

Le 31/03/2020 ? 07:34, David PIERRET a ?crit :

Quoted 2 lines

Hi,

Quoted 8 lines
Quoted 2 lines

I'm not able to complete the boot on qemu_aarch64_virt_defconfig. boot stuck after the network initialisation.

Thanks for testing!

Can you try by adding -device virtio-rng-pci on the Qemu command line ? The system emulated by Qemu lack of entropy sources, the boot process is stuck several dozen of seconde waiting for minimum entropy level.

With rng_tools enabled it works.

Quoted 30 lines

Note, this is not related to the version bump. You should be able to reproduce the issue with openssh 8.1p1.

See:
https://git.buildroot.net/buildroot/commit/?id=450863097f588fe85d23e83cff14a75f1740d8fa

I'll double check on my side.

Best regards,
Romain

Quoted 19 lines

regards

Quoted 10 lines · Romain Naour

This new version is mandatory to allow the glibc package bump to version 2.31. Otherwise it's not possible to connect to the remote host, as reported by [1] [2]. Upstream commit [3][4] fixes the issue.

[1] https://bugs.archlinux.org/task/65386
[2] https://bugs.gentoo.org/708224
[3] https://github.com/openssh/openssh-portable/commit/beee0ef61866cb567b9abc23bd850f922e59e3f0
[4] https://github.com/openssh/openssh-portable/commit/69298ebfc2c066acee5d187eac8ce9f38c796630

Quoted 4 lines

Release Note:
https://www.openssh.com/txt/release-8.2

Quoted 37 lines
Quoted 36 lines
Quoted 36 lines
---
 package/openssh/openssh.hash | 4 ++--
 package/openssh/openssh.mk   | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
package/openssh/openssh.hash +2 -2
index 4db933f7b0..2f962d4337 100644
@@ -1,4 +1,4 @@
-# From https://www.openssh.com/txt/release-8.1 (base64 encoded)
-sha256 02f5dbef3835d0753556f973cd57b4c19b6b1f6cd24c03445e23ac77ca1b93ff  openssh-8.1p1.tar.gz
+# From https://www.openssh.com/txt/release-8.2 (base64 encoded)
+sha256 43925151e6cf6cee1450190c0e9af4dc36b41c12737619edff8bcebdff64e671  openssh-8.2p1.tar.gz
 # Locally calculated
 sha256 73d0db766229670c7b4e1ec5e6baed54977a0694a565e7cc878c45ee834045d7  LICENCE
package/openssh/openssh.mk +1 -2
index 0411133bdc..d50572128a 100644
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-OPENSSH_VERSION = 8.1p1
+OPENSSH_VERSION = 8.2p1
 OPENSSH_SITE = http://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable
 OPENSSH_LICENSE = BSD-3-Clause, BSD-2-Clause, Public Domain
 OPENSSH_LICENSE_FILES = LICENCE
--

2.21.1

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

#5 lore raw

PATCH 2/2 package/openssh: bump to version 8.2p1

In reply to Romain Naour
Quoted 9 lines · On Mon, 30 Mar 2020 18:24:11 +0200 Romain Naour <[email protected]> wrote:

This new version is mandatory to allow the glibc package bump to version 2.31. Otherwise it's not possible to connect to the remote host, as reported by [1] [2]. Upstream commit [3][4] fixes the issue.

[1] https://bugs.archlinux.org/task/65386
[2] https://bugs.gentoo.org/708224
[3] https://github.com/openssh/openssh-portable/commit/beee0ef61866cb567b9abc23bd850f922e59e3f0
[4] https://github.com/openssh/openssh-portable/commit/69298ebfc2c066acee5d187eac8ce9f38c796630

Quoted 8 lines

Release Note:
https://www.openssh.com/txt/release-8.2

---
 package/openssh/openssh.hash | 4 ++--
 package/openssh/openssh.mk   | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

Applied to master, thanks.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#6 lore raw

PATCH 1/2 support/testing: add openssh runtime test

In reply to Romain Naour

support/testing/tests/package/test_openssh.py line 53

Quoted 6 lines · On Mon, 30 Mar 2020 18:24:10 +0200 Romain Naour <[email protected]> wrote:
support/testing/tests/package/test_openssh.py new file +6 -0
+class TestOpenSshGlibc(TestOpensshBase):
+    config = \
+        TestOpensshBase.opensshconfig + \
+        """
+        BR2_x86_64=y
+        BR2_x86_core2=y

Why use an x86-64 system? It forces you to build your own kernel? For this glibc configuration, you could build your own toolchain, but probably stick to using a prebuilt kernel.

Quoted 1 line

+ BR2_TOOLCHAIN_BUILDROOT_GLIBC=y

What about using the ARM ARM toolchain, and boot using the pre-built vexpress kernel ? This will significantly speed-up the build of the system for this runtime test.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#7 lore raw

PATCH 1/2 support/testing: add openssh runtime test

Hi Thomas,

Le 31/03/2020 ? 23:32, Thomas Petazzoni a ?crit?:

Quoted 10 lines

support/testing/tests/package/test_openssh.py line 53

Quoted 6 lines · On Mon, 30 Mar 2020 18:24:10 +0200 Romain Naour <[email protected]> wrote:
support/testing/tests/package/test_openssh.py new file +6 -0
+class TestOpenSshGlibc(TestOpensshBase):
+    config = \
+        TestOpensshBase.opensshconfig + \
+        """
+        BR2_x86_64=y
+        BR2_x86_core2=y
Quoted 3 lines

Why use an x86-64 system? It forces you to build your own kernel? For this glibc configuration, you could build your own toolchain, but probably stick to using a prebuilt kernel.

I reused the configuration for Docker, that's why I used x86_64.

But the testsuite basically test only with uClibc-ng with an old toolchain by default. It would be interesting to allow to select a custom toolchain.

Also, I noticed that the build continue even if the defconfig provided contain an option that is dropped by make olddefconfig.

For example, mesa3d:
https://git.buildroot.net/buildroot/commit/?id=faec5c583ec59e968fdb547aea93ab6ed001eb45

The symbol BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_SWRAST was missing since only one swrast driver can be enabled.

The test should stop is one of the provided symbol is missing (like autobuild-run do).

Quoted 6 lines
Quoted 2 lines

+ BR2_TOOLCHAIN_BUILDROOT_GLIBC=y

What about using the ARM ARM toolchain, and boot using the pre-built vexpress kernel ? This will significantly speed-up the build of the system for this runtime test.

Because I made this test to trigger the openssh issue with glibc 2.31 (local patch applied to do the version bump).

That's why It's important for the testsuite to allow testing with several toolchain (I believe).

Best regards,
Romain

Quoted 5 lines

Thanks!

Thomas

#8 lore raw

PATCH 1/2 support/testing: add openssh runtime test

Hello,

Quoted 5 lines · On Thu, 2 Apr 2020 13:48:15 +0200 Romain Naour <[email protected]> wrote:
Quoted 3 lines

Why use an x86-64 system? It forces you to build your own kernel? For this glibc configuration, you could build your own toolchain, but probably stick to using a prebuilt kernel.

I reused the configuration for Docker, that's why I used x86_64.

OK, but then you have to build your own kernel, which is no really needed.

Quoted 2 lines

But the testsuite basically test only with uClibc-ng with an old toolchain by default. It would be interesting to allow to select a custom toolchain.

You can already select whatever toolchain you want, since you provide for each test case the toolchain.

However, we could certainly update the default toolchain used in infra.basetest.BASIC_TOOLCHAIN_CONFIG

Quoted 11 lines

Also, I noticed that the build continue even if the defconfig provided contain an option that is dropped by make olddefconfig.

For example, mesa3d:
https://git.buildroot.net/buildroot/commit/?id=faec5c583ec59e968fdb547aea93ab6ed001eb45

The symbol BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_SWRAST was missing since only one swrast driver can be enabled.

The test should stop is one of the provided symbol is missing (like autobuild-run do).

Sure, that would make sense.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

+ BR2_TOOLCHAIN_BUILDROOT_GLIBC=y

What about using the ARM ARM toolchain, and boot using the pre-built vexpress kernel ? This will significantly speed-up the build of the system for this runtime test.

Because I made this test to trigger the openssh issue with glibc 2.31 (local patch applied to do the version bump).

Ah, OK, indeed.

Quoted 2 lines

That's why It's important for the testsuite to allow testing with several toolchain (I believe).

It already allows to do so: you did it!

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#9 lore raw

PATCH 1/2 support/testing: add openssh runtime test

Hello Thomas,

Le 02/04/2020 ? 14:16, Thomas Petazzoni a ?crit?:

Quoted 5 lines

Hello,

On Thu, 2 Apr 2020 13:48:15 +0200
Romain Naour <[email protected]> wrote:

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Why use an x86-64 system? It forces you to build your own kernel? For this glibc configuration, you could build your own toolchain, but probably stick to using a prebuilt kernel.

I reused the configuration for Docker, that's why I used x86_64.

OK, but then you have to build your own kernel, which is no really needed.

Ok

Quoted 6 lines
Quoted 3 lines

But the testsuite basically test only with uClibc-ng with an old toolchain by default. It would be interesting to allow to select a custom toolchain.

You can already select whatever toolchain you want, since you provide for each test case the toolchain.

I mean, I would be interesting to override or provide another toolchain. For example, a package test should be pass whatever the toolchain being used.

For the moment most of the tests are done with BASIC_TOOLCHAIN_CONFIG.

Quoted 3 lines

However, we could certainly update the default toolchain used in infra.basetest.BASIC_TOOLCHAIN_CONFIG

What about using br-arm-full-2020.02 ?

Quoted 24 lines
Quoted 4 lines

Also, I noticed that the build continue even if the defconfig provided contain an option that is dropped by make olddefconfig.

Quoted 8 lines

For example, mesa3d:
https://git.buildroot.net/buildroot/commit/?id=faec5c583ec59e968fdb547aea93ab6ed001eb45

The symbol BR2_PACKAGE_MESA3D_GALLIUM_DRIVER_SWRAST was missing since only one swrast driver can be enabled.

The test should stop is one of the provided symbol is missing (like autobuild-run do).

Sure, that would make sense.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

+ BR2_TOOLCHAIN_BUILDROOT_GLIBC=y

What about using the ARM ARM toolchain, and boot using the pre-built vexpress kernel ? This will significantly speed-up the build of the system for this runtime test.

Because I made this test to trigger the openssh issue with glibc 2.31 (local patch applied to do the version bump).

Quoted 6 lines

Ah, OK, indeed.

Quoted 2 lines

That's why It's important for the testsuite to allow testing with several toolchain (I believe).

It already allows to do so: you did it!

See my explanation above.

Best regards,
Romain

Quoted 5 lines

Best regards,

Thomas

#10 lore raw

PATCH 1/2 support/testing: add openssh runtime test

Quoted 8 lines · On Thu, 2 Apr 2020 21:38:42 +0200 Romain Naour <[email protected]> wrote:
Quoted 5 lines
Quoted 2 lines

But the testsuite basically test only with uClibc-ng with an old toolchain by default. It would be interesting to allow to select a custom toolchain.

You can already select whatever toolchain you want, since you provide for each test case the toolchain.

I mean, I would be interesting to override or provide another toolchain. For example, a package test should be pass whatever the toolchain being used.

How would you handle that? The point of runtime tests is that they should be reproducible so that we can track when they started failing. We also can't run all tests on all toolchains/architectures, that's not really realistic from a CPU time point of view, and from a test maintenance point of view.

Quoted 4 lines
Quoted 2 lines

However, we could certainly update the default toolchain used in infra.basetest.BASIC_TOOLCHAIN_CONFIG

What about using br-arm-full-2020.02 ?

Sure, would be good. Could you test this, do a Gitlab CI run, and see how it goes?

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#11 lore raw

PATCH 1/2 support/testing: add openssh runtime test

Hi Thomas,

Le 02/04/2020 ? 21:41, Thomas Petazzoni a ?crit?:

Quoted 9 lines
Quoted 6 lines · On Thu, 2 Apr 2020 21:38:42 +0200 Romain Naour <[email protected]> wrote:
Quoted 5 lines
Quoted 2 lines

But the testsuite basically test only with uClibc-ng with an old toolchain by default. It would be interesting to allow to select a custom toolchain.

You can already select whatever toolchain you want, since you provide for each test case the toolchain.

Quoted 8 lines
Quoted 2 lines

I mean, I would be interesting to override or provide another toolchain. For example, a package test should be pass whatever the toolchain being used.

How would you handle that? The point of runtime tests is that they should be reproducible so that we can track when they started failing. We also can't run all tests on all toolchains/architectures, that's not really realistic from a CPU time point of view, and from a test maintenance point of view.

My point is that we don't test with different gcc/binutils version and different libc implementation. But we have to use the same architecture since the Qemu command line is specific to the emulated architecture. If a test would be available for several architecture it would require to update the qemu command line (and the kernel configuration too).

At least testing with several gcc and libc implementation would be good enough?

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

However, we could certainly update the default toolchain used in infra.basetest.BASIC_TOOLCHAIN_CONFIG

What about using br-arm-full-2020.02 ?

Sure, would be good. Could you test this, do a Gitlab CI run, and see how it goes?

Ok.

Best regards,
Romain

Quoted 3 lines

Thomas

#12 lore raw

PATCH 1/2 support/testing: add openssh runtime test

Quoted 7 lines · On Thu, 2 Apr 2020 22:04:00 +0200 Romain Naour <[email protected]> wrote:

My point is that we don't test with different gcc/binutils version and different libc implementation. But we have to use the same architecture since the Qemu command line is specific to the emulated architecture. If a test would be available for several architecture it would require to update the qemu command line (and the kernel configuration too).

At least testing with several gcc and libc implementation would be good enough?

But would you do that for all tests ?

If you take our 172 existing tests, then run all of them on 3 different libcs, 3 different gcc versions, you suddenly have 1548 test cases. And that's just for *one* architecture.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com