Skip to contentSkip to conversation

gitlab Qemu runtime testing

PATCH Superseded Jugurtha BELKALEM 4 participants 21 messages Apr 29, 2019 – May 1, 2019

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover gitlab Qemu runtime testing

The tests that have been performed resulted in:
- 23 architectures passing the test.
- 12 architectures that failed the test.
More investigations need to be made for
architectures that did not pass the test.

special notes for :
- m64k_q800 has been blacklisted (as there is
no qemu support for it).
- sparc64 does not find a qemu shell.
- x86 and x86_64 do not boot properly (timeout).
- qemu ork1 and xtensa do not even compile.

Jugurtha BELKALEM (7):
support/docker/Dockerfile: new package expect package/qemu: enable nios2 support for host-qemu package/qemu: enable sparc64 support for host-qemu package/qemu: remove comment about sh64 configs/qemu-*: add host-qemu-system support/scripts/qemu-boot: gitlab tests for Qemu gitlab.yml.in: enable Qemu gitlab testing

 .gitlab-ci.yml                                |  3 +++
 .gitlab-ci.yml.in                             |  3 +++
 configs/qemu_aarch64_virt_defconfig           |  4 +++
 configs/qemu_arm_versatile_defconfig          |  4 +++
 configs/qemu_arm_versatile_nommu_defconfig    |  4 +++
 configs/qemu_arm_vexpress_defconfig           |  4 +++
 configs/qemu_m68k_mcf5208_defconfig           |  4 +++
 configs/qemu_m68k_q800_defconfig              |  4 +++
 configs/qemu_microblazebe_mmu_defconfig       |  4 +++
 configs/qemu_microblazeel_mmu_defconfig       |  4 +++
 configs/qemu_mips32r2_malta_defconfig         |  4 +++
 configs/qemu_mips32r2el_malta_defconfig       |  4 +++
 configs/qemu_mips32r6_malta_defconfig         |  4 +++
 configs/qemu_mips32r6el_malta_defconfig       |  4 +++
 configs/qemu_mips64_malta_defconfig           |  4 +++
 configs/qemu_mips64el_malta_defconfig         |  4 +++
 configs/qemu_mips64r6_malta_defconfig         |  4 +++
 configs/qemu_mips64r6el_malta_defconfig       |  4 +++
 configs/qemu_nios2_10m50_defconfig            |  4 +++
 configs/qemu_or1k_defconfig                   |  4 +++
 configs/qemu_ppc64_e5500_defconfig            |  4 +++
 configs/qemu_ppc64_pseries_defconfig          |  4 +++
 configs/qemu_ppc64le_pseries_defconfig        |  4 +++
 configs/qemu_ppc_g3beige_defconfig            |  4 +++
 configs/qemu_ppc_mpc8544ds_defconfig          |  4 +++
 configs/qemu_ppc_virtex_ml507_defconfig       |  4 +++
 configs/qemu_riscv32_virt_defconfig           |  4 +++
 configs/qemu_riscv64_virt_defconfig           |  4 +++
 configs/qemu_sh4_r2d_defconfig                |  4 +++
 configs/qemu_sh4eb_r2d_defconfig              |  4 +++
 configs/qemu_sparc64_sun4u_defconfig          |  4 +++
 configs/qemu_sparc_ss10_defconfig             |  4 +++
 configs/qemu_x86_64_defconfig                 |  4 +++
 configs/qemu_x86_defconfig                    |  4 +++
 configs/qemu_xtensa_lx60_defconfig            |  4 +++
 configs/qemu_xtensa_lx60_nommu_defconfig      |  4 +++
 package/qemu/Config.in.host                   |  2 ++
 package/qemu/qemu.mk                          |  3 ++-
 support/docker/Dockerfile                     |  1 +
 support/scripts/expect.sh                     | 22 +++++++++++++++++
 support/scripts/qemu-boot-checker.sh          | 35 +++++++++++++++++++++++++++
 support/scripts/qemu-boot-defconfig_config.sh | 11 +++++++++
 42 files changed, 215 insertions(+), 1 deletion(-)
 create mode 100755 support/scripts/expect.sh
 create mode 100755 support/scripts/qemu-boot-checker.sh
 create mode 100755 support/scripts/qemu-boot-defconfig_config.sh
Signature · 1 line
2.7.4
#1 lore raw

PATCH 1/7 support/docker/Dockerfile: new package expect

expect package is required for various automation tests. One use case is to integrate the script from toolchain builder :
https://github.com/bootlin/toolchains-builder/blob/master/build.sh
to check Qemu build states.
expect can be used to launch a qemu machine for a given architecture (x86, x86_64, mips, arm, sparc, nios2, ..., etc), send root password, wait for login and then perform shutdown.

Special note concerning python-pexpect package which can be used for the same task: After testing the package, We have noticed that it does not show the automation process correcly (you may not even see the message welcome to buildroot) and it is not verbose when errors are encountered.

---
v2: reorder alphabetically
Remove m68k kernel issue, since we use kernel 4.19 with qemu m68k defconfig. The patch fixing the issue was merged in kernel 4.17.

---
 support/docker/Dockerfile | 1 +
 1 file changed, 1 insertion(+)
support/docker/Dockerfile +1 -0
index 36cd3b2..8dcd3ef 100644
@@ -29,6 +29,7 @@ RUN apt-get install -y --no-install-recommends \
         cmake \
         cpio \
         cvs \
+        expect \
         file \
         g++-multilib \
         git \
Signature · 1 line
2.7.4
#2 lore raw

PATCH 1/7 support/docker/Dockerfile: new package expect

Quoted 5 lines · Jugurtha BELKALEM

Special note concerning python-pexpect package which can be used for the same task: After testing the package, We have noticed that it does not show the automation process correcly (you may not even see the message welcome to buildroot) and it is not verbose when errors are encountered.

Could you expand on this a little? We've been using pexpect (both fdpexpect and
pxssh) very successfully for our testing project. Admittedly, we only use Python3 so maybe that's where the problem is.

Clearly, for this simple use case it should be acceptable to use tk-expect
(even though I expect hardly anyone under the age of 40 to still know about it :-). But if we're going to reuse the same approach for other tests as well, we might benefit from a more powerful and python-integrated package.

Regards,
Arnout

#3 lore raw

PATCH 2/7 package/qemu: enable nios2 support for host-qemu

In reply to Jugurtha BELKALEM

This config enable host eqmu support for nios2 which allows to test this architecture in gitlab.

---
 package/qemu/Config.in.host | 1 +
 package/qemu/qemu.mk        | 1 +
 2 files changed, 2 insertions(+)
package/qemu/Config.in.host +1 -0
index 8a0df38..160725d 100644
@@ -9,6 +9,7 @@ config BR2_PACKAGE_HOST_QEMU_ARCH_SUPPORTS
 	default y if BR2_microblazebe
 	default y if BR2_mips
 	default y if BR2_mipsel
+	default y if BR2_nios2
 	default y if BR2_powerpc
 	default y if BR2_powerpc64
 	default y if BR2_powerpc64le
package/qemu/qemu.mk +1 -0
index 19bcafa..e82d372 100644
@@ -172,6 +172,7 @@ HOST_QEMU_DEPENDENCIES = host-pkgconf host-zlib host-libglib2 host-pixman
 #       mipsel          mipsel
 #       mips64          mips64
 #       mips64el        mips64el
+#       nios2           nios2
 #       powerpc         ppc
 #       powerpc64       ppc64
 #       powerpc64le     ppc64 (system) / ppc64le (usermode)
Signature · 1 line
2.7.4
#4 lore raw

PATCH 3/7 package/qemu: enable sparc64 support for host-qemu

In reply to Jugurtha BELKALEM

This config enable host eqmu support for sparc64 which allows to test this architecture in gitlab.

---
 package/qemu/Config.in.host | 1 +
 package/qemu/qemu.mk        | 1 +
 2 files changed, 2 insertions(+)
package/qemu/Config.in.host +1 -0
index 160725d..ea80d6f 100644
@@ -16,6 +16,7 @@ config BR2_PACKAGE_HOST_QEMU_ARCH_SUPPORTS
 	default y if BR2_riscv
 	default y if BR2_sh
 	default y if BR2_sparc
+	default y if BR2_sparc64
 	default y if BR2_x86_64
 	depends on !BR2_powerpc_620 && !BR2_powerpc_630 && !BR2_powerpc_970
 
package/qemu/qemu.mk +1 -0
index e82d372..b509472 100644
@@ -183,6 +183,7 @@ HOST_QEMU_DEPENDENCIES = host-pkgconf host-zlib host-libglib2 host-pixman
 #       sh4aeb          sh4eb
 #       sh64            not supported
 #       sparc           sparc
+#       sparc64         sparc64
 
 HOST_QEMU_ARCH = $(ARCH)
 ifeq ($(HOST_QEMU_ARCH),i486)
Signature · 1 line
2.7.4
#5 lore raw

PATCH 4/7 package/qemu: remove comment about sh64

In reply to Jugurtha BELKALEM

sh64 has been removed since commit [1]:

[1] 4338a319b72570e433fd7484f8a95e1e0e978941

---
 package/qemu/qemu.mk | 1 -
 1 file changed, 1 deletion(-)
package/qemu/qemu.mk +0 -1
index b509472..0dfb928 100644
@@ -181,7 +181,6 @@ HOST_QEMU_DEPENDENCIES = host-pkgconf host-zlib host-libglib2 host-pixman
 #       sh4eb           sh4eb
 #       sh4a            sh4
 #       sh4aeb          sh4eb
-#       sh64            not supported
 #       sparc           sparc
 #       sparc64         sparc64
 
Signature · 1 line
2.7.4
#6 lore raw

PATCH 5/7 configs/qemu-*: add host-qemu-system

In reply to Jugurtha BELKALEM

Such config allows to setup gitlab tests for various qemu architectures (x86, mips, nios2, microblaze, ..., etc) using the same Qemu version.

---
 configs/qemu_aarch64_virt_defconfig        | 4 ++++
 configs/qemu_arm_versatile_defconfig       | 4 ++++
 configs/qemu_arm_versatile_nommu_defconfig | 4 ++++
 configs/qemu_arm_vexpress_defconfig        | 4 ++++
 configs/qemu_m68k_mcf5208_defconfig        | 4 ++++
 configs/qemu_m68k_q800_defconfig           | 4 ++++
 configs/qemu_microblazebe_mmu_defconfig    | 4 ++++
 configs/qemu_microblazeel_mmu_defconfig    | 4 ++++
 configs/qemu_mips32r2_malta_defconfig      | 4 ++++
 configs/qemu_mips32r2el_malta_defconfig    | 4 ++++
 configs/qemu_mips32r6_malta_defconfig      | 4 ++++
 configs/qemu_mips32r6el_malta_defconfig    | 4 ++++
 configs/qemu_mips64_malta_defconfig        | 4 ++++
 configs/qemu_mips64el_malta_defconfig      | 4 ++++
 configs/qemu_mips64r6_malta_defconfig      | 4 ++++
 configs/qemu_mips64r6el_malta_defconfig    | 4 ++++
 configs/qemu_nios2_10m50_defconfig         | 4 ++++
 configs/qemu_or1k_defconfig                | 4 ++++
 configs/qemu_ppc64_e5500_defconfig         | 4 ++++
 configs/qemu_ppc64_pseries_defconfig       | 4 ++++
 configs/qemu_ppc64le_pseries_defconfig     | 4 ++++
 configs/qemu_ppc_g3beige_defconfig         | 4 ++++
 configs/qemu_ppc_mpc8544ds_defconfig       | 4 ++++
 configs/qemu_ppc_virtex_ml507_defconfig    | 4 ++++
 configs/qemu_riscv32_virt_defconfig        | 4 ++++
 configs/qemu_riscv64_virt_defconfig        | 4 ++++
 configs/qemu_sh4_r2d_defconfig             | 4 ++++
 configs/qemu_sh4eb_r2d_defconfig           | 4 ++++
 configs/qemu_sparc64_sun4u_defconfig       | 4 ++++
 configs/qemu_sparc_ss10_defconfig          | 4 ++++
 configs/qemu_x86_64_defconfig              | 4 ++++
 configs/qemu_x86_defconfig                 | 4 ++++
 configs/qemu_xtensa_lx60_defconfig         | 4 ++++
 configs/qemu_xtensa_lx60_nommu_defconfig   | 4 ++++
 34 files changed, 136 insertions(+)
configs/qemu_aarch64_virt_defconfig +4 -0
index 638e18b..212d779 100644
@@ -21,3 +21,7 @@ BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19.16"
 BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
 BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/aarch64-virt/linux.config"
 BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_arm_versatile_defconfig +4 -0
index ce7cb5e..1194644 100644
@@ -21,3 +21,7 @@ BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
 BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/arm-versatile/linux.config"
 BR2_LINUX_KERNEL_DTS_SUPPORT=y
 BR2_LINUX_KERNEL_INTREE_DTS_NAME="versatile-pb"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_arm_versatile_nommu_defconfig +4 -0
index 81fb5b6..ae4353d 100644
@@ -32,3 +32,7 @@ BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
 BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/arm-versatile/linux-nommu.config"
 BR2_LINUX_KERNEL_DTS_SUPPORT=y
 BR2_LINUX_KERNEL_INTREE_DTS_NAME="versatile-pb"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_arm_vexpress_defconfig +4 -0
index 66f198a..e52a140 100644
@@ -23,3 +23,7 @@ BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19.16"
 BR2_LINUX_KERNEL_DEFCONFIG="vexpress"
 BR2_LINUX_KERNEL_DTS_SUPPORT=y
 BR2_LINUX_KERNEL_INTREE_DTS_NAME="vexpress-v2p-ca9"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_m68k_mcf5208_defconfig +4 -0
index 44fd918..4dde10b 100644
@@ -26,3 +26,7 @@ BR2_TARGET_GENERIC_GETTY_PORT="ttyS0"
 
 # use minimal busybox with hush and networking tools
 BR2_PACKAGE_BUSYBOX_CONFIG="package/busybox/busybox-minimal.config"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_m68k_q800_defconfig +4 -0
index 0dd90a9..ce26443 100644
@@ -23,3 +23,7 @@ BR2_LINUX_KERNEL_VMLINUX=y
 # Serial port config
 BR2_TARGET_GENERIC_GETTY=y
 BR2_TARGET_GENERIC_GETTY_PORT="ttyS0"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_microblazebe_mmu_defconfig +4 -0
index 95fdfed..022d2b7 100644
@@ -21,3 +21,7 @@ BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
 BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/microblazebe-mmu/linux.config"
 BR2_LINUX_KERNEL_LINUX_BIN=y
 BR2_LINUX_KERNEL_PATCH="board/qemu/microblazebe-mmu/xilinx-xemaclite.patch"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_microblazeel_mmu_defconfig +4 -0
index 7400daf..fbe02ca 100644
@@ -21,3 +21,7 @@ BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
 BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/microblazeel-mmu/linux.config"
 BR2_LINUX_KERNEL_LINUX_BIN=y
 BR2_LINUX_KERNEL_PATCH="board/qemu/microblazeel-mmu/xilinx-xemaclite.patch"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_mips32r2_malta_defconfig +4 -0
index 0ee7a58..838a3d6 100644
@@ -23,3 +23,7 @@ BR2_LINUX_KERNEL_VMLINUX=y
 # Serial port config
 BR2_TARGET_GENERIC_GETTY=y
 BR2_TARGET_GENERIC_GETTY_PORT="ttyS0"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_mips32r2el_malta_defconfig +4 -0
index 075088d..aaf3821 100644
@@ -23,3 +23,7 @@ BR2_LINUX_KERNEL_VMLINUX=y
 # Serial port config
 BR2_TARGET_GENERIC_GETTY=y
 BR2_TARGET_GENERIC_GETTY_PORT="ttyS0"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_mips32r6_malta_defconfig +4 -0
index b0bf94c..c480b69 100644
@@ -27,3 +27,7 @@ BR2_GCC_VERSION_6_X=y
 # Serial port config
 BR2_TARGET_GENERIC_GETTY=y
 BR2_TARGET_GENERIC_GETTY_PORT="ttyS0"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_mips32r6el_malta_defconfig +4 -0
index 077d836..06f638e 100644
@@ -27,3 +27,7 @@ BR2_GCC_VERSION_6_X=y
 # Serial port config
 BR2_TARGET_GENERIC_GETTY=y
 BR2_TARGET_GENERIC_GETTY_PORT="ttyS0"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_mips64_malta_defconfig +4 -0
index 12502ea..20e368a 100644
@@ -23,3 +23,7 @@ BR2_LINUX_KERNEL_VMLINUX=y
 # Serial port config
 BR2_TARGET_GENERIC_GETTY=y
 BR2_TARGET_GENERIC_GETTY_PORT="ttyS0"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_mips64el_malta_defconfig +4 -0
index 5ddb759..2ec2843 100644
@@ -23,3 +23,7 @@ BR2_LINUX_KERNEL_VMLINUX=y
 # Serial port config
 BR2_TARGET_GENERIC_GETTY=y
 BR2_TARGET_GENERIC_GETTY_PORT="ttyS0"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_mips64r6_malta_defconfig +4 -0
index bfa32f6..a614b4c 100644
@@ -28,3 +28,7 @@ BR2_GCC_VERSION_6_X=y
 # Serial port config
 BR2_TARGET_GENERIC_GETTY=y
 BR2_TARGET_GENERIC_GETTY_PORT="ttyS0"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_mips64r6el_malta_defconfig +4 -0
index 2fd855d..d2b9e50 100644
@@ -28,3 +28,7 @@ BR2_GCC_VERSION_6_X=y
 # Serial port config
 BR2_TARGET_GENERIC_GETTY=y
 BR2_TARGET_GENERIC_GETTY_PORT="ttyS0"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_nios2_10m50_defconfig +4 -0
index c07abc3..24573fc 100644
@@ -11,3 +11,7 @@ BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_19=y
 
 # Kernel needs mkimage
 BR2_PACKAGE_HOST_UBOOT_TOOLS=y
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_or1k_defconfig +4 -0
index 3fc53f4..3d3fd53 100644
@@ -14,3 +14,7 @@ BR2_LINUX_KERNEL_CUSTOM_VERSION=y
 BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19.16"
 BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
 BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/or1k/linux.config"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_ppc64_e5500_defconfig +4 -0
index 59b8636..d5d6647 100644
@@ -19,3 +19,7 @@ BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y
 # Filesystem
 BR2_TARGET_ROOTFS_EXT2=y
 # BR2_TARGET_ROOTFS_TAR is not set
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_ppc64_pseries_defconfig +4 -0
index 5f55b24..200c50d 100644
@@ -19,3 +19,7 @@ BR2_LINUX_KERNEL_CUSTOM_VERSION=y
 BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19.16"
 BR2_LINUX_KERNEL_DEFCONFIG="pseries"
 BR2_LINUX_KERNEL_VMLINUX=y
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_ppc64le_pseries_defconfig +4 -0
index 8356e4e..0eb3f11 100644
@@ -19,3 +19,7 @@ BR2_LINUX_KERNEL_CUSTOM_VERSION=y
 BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19.16"
 BR2_LINUX_KERNEL_DEFCONFIG="pseries_le"
 BR2_LINUX_KERNEL_VMLINUX=y
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_ppc_g3beige_defconfig +4 -0
index 4b2f3ed..8034abc 100644
@@ -23,3 +23,7 @@ BR2_LINUX_KERNEL_VMLINUX=y
 # Serial port config
 BR2_TARGET_GENERIC_GETTY=y
 BR2_TARGET_GENERIC_GETTY_PORT="ttyS0"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_ppc_mpc8544ds_defconfig +4 -0
index 98985b7..9a529ae 100644
@@ -26,3 +26,7 @@ BR2_TARGET_GENERIC_GETTY_PORT="ttyS0"
 
 # Kernel needs mkimage
 BR2_PACKAGE_HOST_UBOOT_TOOLS=y
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_ppc_virtex_ml507_defconfig +4 -0
index eddaf35..4b60675 100644
@@ -24,3 +24,7 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="virtex440-ml507"
 
 # Kernel needs mkimage
 BR2_PACKAGE_HOST_UBOOT_TOOLS=y
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_riscv32_virt_defconfig +4 -0
index 84f7b6a..5a8e105 100644
@@ -25,3 +25,7 @@ BR2_LINUX_KERNEL_IMAGE=y
 BR2_TARGET_OPENSBI=y
 BR2_TARGET_OPENSBI_USE_PLAT=y
 BR2_TARGET_OPENSBI_PLAT="qemu/virt"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_riscv64_virt_defconfig +4 -0
index 12afac4..9590c61 100644
@@ -24,3 +24,7 @@ BR2_LINUX_KERNEL_IMAGE=y
 BR2_TARGET_OPENSBI=y
 BR2_TARGET_OPENSBI_USE_PLAT=y
 BR2_TARGET_OPENSBI_PLAT="qemu/virt"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_sh4_r2d_defconfig +4 -0
index e194d7e..ca7b25e 100644
@@ -21,3 +21,7 @@ BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19.16"
 BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
 BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/sh4-r2d/linux.config"
 BR2_LINUX_KERNEL_ZIMAGE=y
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_sh4eb_r2d_defconfig +4 -0
index f1aad8b..0664a08 100644
@@ -20,3 +20,7 @@ BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19.16"
 BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
 BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/sh4eb-r2d/linux.config"
 BR2_LINUX_KERNEL_ZIMAGE=y
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_sparc64_sun4u_defconfig +4 -0
index f4da2bc..70b4f13 100644
@@ -18,3 +18,7 @@ BR2_LINUX_KERNEL_CUSTOM_VERSION=y
 BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19.16"
 BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
 BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/sparc64-sun4u/linux.config"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_sparc_ss10_defconfig +4 -0
index e027b73..23733d8 100644
@@ -18,3 +18,7 @@ BR2_LINUX_KERNEL_CUSTOM_VERSION=y
 BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19.16"
 BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
 BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/sparc-ss10/linux.config"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_x86_64_defconfig +4 -0
index afa2396..c3c0991 100644
@@ -18,3 +18,7 @@ BR2_LINUX_KERNEL_CUSTOM_VERSION=y
 BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19.16"
 BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
 BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/x86_64/linux.config"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_x86_defconfig +4 -0
index 3441ad5..6d68b7d 100644
@@ -19,3 +19,7 @@ BR2_LINUX_KERNEL_CUSTOM_VERSION=y
 BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19.16"
 BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
 BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/x86/linux.config"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_xtensa_lx60_defconfig +4 -0
index 6bf48a2..f36b4fb 100644
@@ -23,3 +23,7 @@ BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/xtensa-lx60/linux.config"
 BR2_LINUX_KERNEL_IMAGE_TARGET_CUSTOM=y
 BR2_LINUX_KERNEL_IMAGE_NAME="Image.elf"
 BR2_LINUX_KERNEL_IMAGE_TARGET_NAME="Image"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
configs/qemu_xtensa_lx60_nommu_defconfig +4 -0
index 5ff9646..48746a9 100644
@@ -27,3 +27,7 @@ BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/xtensa-lx60/linux-nommu.config"
 BR2_LINUX_KERNEL_IMAGE_TARGET_CUSTOM=y
 BR2_LINUX_KERNEL_IMAGE_NAME="Image.elf"
 BR2_LINUX_KERNEL_IMAGE_TARGET_NAME="Image"
+
+# host-qemu for gitlab testing
+BR2_PACKAGE_HOST_QEMU=y
+BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
Signature · 1 line
2.7.4
#7 lore raw

PATCH 6/7 support/scripts/qemu-boot: gitlab tests for Qemu

In reply to Jugurtha BELKALEM

Enables to check various qemu architectures build states. These scripts were inspired from toolchain builder :
https://github.com/bootlin/toolchains-builder/blob/master/build.sh
to test qemu's build process.
This allows to troubleshoot different issues that may be associated with defective qemu builds by lanching a qemu machine, sending root password, waiting for login shell and then perform a shutdown.

The script expect.sh relies on expect package to automate the tests.
We should mention that python-pexpect can be tweeked for the same job but seems like it does hide the automation process as well as any errors that may be encountered.

The script qemu-boot-defconfig_config.sh is required for architectures that need special configuration before starting compilation (like setting the correct tty).

On the other side, qemu-boot-checker.sh is used to read the qemu command used to launch a qemu machine (by reading board/qemu/qemu_architecture/readme.txt) as well as setting the path to the qemu host and calling expect.sh.

---
 support/scripts/expect.sh                     | 22 +++++++++++++++++
 support/scripts/qemu-boot-checker.sh          | 35 +++++++++++++++++++++++++++
 support/scripts/qemu-boot-defconfig_config.sh | 11 +++++++++
 3 files changed, 68 insertions(+)
 create mode 100755 support/scripts/expect.sh
 create mode 100755 support/scripts/qemu-boot-checker.sh
 create mode 100755 support/scripts/qemu-boot-defconfig_config.sh
support/scripts/expect.sh new file +22 -0
new file mode 100755
index 0000000..6d65752
@@ -0,0 +1,22 @@
+#!/usr/bin/expect
+#
+
+set timeout 400
+
+log_file /tmp/expect_session.log
+
+eval spawn $env(QEMU_COMMAND)
+
+expect {
+    eof {puts "Connection problem, exiting."; exit 1}
+    timeout {puts "System did not boot in time, exiting."; exit 1}
+    "buildroot login:"
+}
+send "root\r"
+expect {
+    eof {puts "Connection problem, exiting."; exit 1}
+    timeout {puts "No shell, exiting."; exit 1}
+    "# "
+}
+send "poweroff\r"
+expect "System halted"
support/scripts/qemu-boot-checker.sh new file +35 -0
new file mode 100755
index 0000000..f036516
@@ -0,0 +1,35 @@
+#!/bin/bash
+
+if [[ $1 = qemu* ]]; then
+  device_name=$(echo $1  | sed -e 's#^qemu_##; s#_defconfig$##;' | sed -r 's/[_]/-/g')
+  if [ $device_name == "x86-64" ]; then
+    device_name="x86_64"
+  elif [ $device_name == "xtensa-lx60" ] || [ $device_name == "xtensa-lx60-nommu" ]; then
+    echo "xtensa cannot be tested"
+    exit 0
+  elif [ $device_name == "m68k-q800" ] || [ $device_name == "ork1k" ]; then
+    archQemuNoSupport
+  fi
+
+  test_qemu_cmd="$(grep qemu-system $2/board/qemu/${device_name}/readme.txt)"
+  qemu_command="$(echo "${test_qemu_cmd}"|tr -d '\n')"
+
+  export QEMU_COMMAND="${qemu_command}"
+
+  export PATH="$2/output/host/bin:$PATH"
+  echo $PATH
+
+  function archQemuNoSupport {
+    echo "cannot boot under qemu, support out of tree!"
+    exit 0
+  }
+
+  function boot_test {
+    if ! expect expect.sh ; then
+      echo "  booting test system ... FAILED"
+      return 1
+    fi
+    echo "  booting test system ... SUCCESS"
+  }
+  boot_test
+fi
support/scripts/qemu-boot-defconfig_config.sh new file +11 -0
new file mode 100755
index 0000000..526eee1
@@ -0,0 +1,11 @@
+#!/bin/bash
+
+if [[ $1 = qemu* ]]; then
+  device_name=$(echo $1  | sed -e 's#^qemu_##; s#_defconfig$##;' | sed -r 's/[_]/-/g') 
+  if [ $device_name == "x86-64" ]; then
+    device_name="x86_64"
+    sed -i "s/tty1/ttyS0/" $2/configs/$1
+  elif [ $device_name == "x86" ]; then
+    sed -i "s/tty1/ttyS0/" $2/configs/$1
+  fi
+fi
Signature · 1 line
2.7.4
#8 lore raw

PATCH 6/7 support/scripts/qemu-boot: gitlab tests for Qemu

Hi Jugurtha,

Thank you for this. It is certainly useful! I have a few comments though.

Quoted 48 lines · Jugurtha BELKALEM

Enables to check various qemu architectures build states. These scripts were inspired from toolchain builder :
https://github.com/bootlin/toolchains-builder/blob/master/build.sh
to test qemu's build process.
This allows to troubleshoot different issues that may be associated with defective qemu builds by lanching a qemu machine, sending root password, waiting for login shell and then perform a shutdown.

The script expect.sh relies on expect package to automate the tests.
We should mention that python-pexpect can be tweeked for the same job but seems like it does hide the automation process as well as any errors that may be encountered.

The script qemu-boot-defconfig_config.sh is required for architectures that need special configuration before starting compilation (like setting the correct tty).

On the other side, qemu-boot-checker.sh is used to read the qemu command used to launch a qemu machine (by reading board/qemu/qemu_architecture/readme.txt) as well as setting the path to the qemu host and calling expect.sh.

---
 support/scripts/expect.sh                     | 22 +++++++++++++++++
 support/scripts/qemu-boot-checker.sh          | 35 +++++++++++++++++++++++++++
 support/scripts/qemu-boot-defconfig_config.sh | 11 +++++++++
 3 files changed, 68 insertions(+)
 create mode 100755 support/scripts/expect.sh
 create mode 100755 support/scripts/qemu-boot-checker.sh
 create mode 100755 support/scripts/qemu-boot-defconfig_config.sh
support/scripts/expect.sh new file +8 -0
new file mode 100755
index 0000000..6d65752
@@ -0,0 +1,22 @@
+#!/usr/bin/expect
+#
+
+set timeout 400
+
+log_file /tmp/expect_session.log
+
+eval spawn $env(QEMU_COMMAND)

support/scripts/expect.sh line 18

Quoted 10 lines
support/scripts/expect.sh new file +10 -0
+
+expect {
+    eof {puts "Connection problem, exiting."; exit 1}
+    timeout {puts "System did not boot in time, exiting."; exit 1}
+    "buildroot login:"
+}
+send "root\r"
+expect {
+    eof {puts "Connection problem, exiting."; exit 1}
+    timeout {puts "No shell, exiting."; exit 1}

Here, the timeout should be *much* smaller than 400. Like, 2-3 seconds is more
appropriate.

support/scripts/expect.sh line 22

Quoted 4 lines
support/scripts/expect.sh new file +4 -0
+    "# "
+}
+send "poweroff\r"
+expect "System halted"

Can we also have an expectation that the spawned command exits here?

support/scripts/qemu-boot-checker.sh line 4

Quoted 10 lines
support/scripts/qemu-boot-checker.sh new file +4 -0
new file mode 100755
index 0000000..f036516
@@ -0,0 +1,35 @@
+#!/bin/bash
+
+if [[ $1 = qemu* ]]; then
+  device_name=$(echo $1  | sed -e 's#^qemu_##; s#_defconfig$##;' | sed -r 's/[_]/-/g')

We don't really have consistent indentation in shell scripts, but it's either 4
spaces or one tab. I prefer one tab myself.

support/scripts/qemu-boot-checker.sh line 6

Quoted 2 lines
support/scripts/qemu-boot-checker.sh new file +2 -0
+  if [ $device_name == "x86-64" ]; then
+    device_name="x86_64"

We should find a better way of handling this. E.g., renaming the directory to
x86-64 :-)

support/scripts/qemu-boot-checker.sh line 8

Quoted 2 lines
support/scripts/qemu-boot-checker.sh new file +2 -0
+  elif [ $device_name == "xtensa-lx60" ] || [ $device_name == "xtensa-lx60-nommu" ]; then
+    echo "xtensa cannot be tested"

If that is the case, why do we even have these qemu defconfigs? Max?

support/scripts/qemu-boot-checker.sh line 10

Quoted 2 lines
support/scripts/qemu-boot-checker.sh new file +2 -0
+    exit 0
+  elif [ $device_name == "m68k-q800" ] || [ $device_name == "ork1k" ]; then
                                                                ^^^^^or1k

support/scripts/qemu-boot-checker.sh line 11

Quoted 1 line
support/scripts/qemu-boot-checker.sh new file +1 -0
+    archQemuNoSupport

In that case, no qemu can be built, so I think it's better to just check for
the presence of the qemu binary.

support/scripts/qemu-boot-checker.sh line 15

Quoted 4 lines
support/scripts/qemu-boot-checker.sh new file +4 -0
+  fi
+
+  test_qemu_cmd="$(grep qemu-system $2/board/qemu/${device_name}/readme.txt)"
+  qemu_command="$(echo "${test_qemu_cmd}"|tr -d '\n')"

Instead of screenscraping this from the readme, I'd much prefer to rewrite it
so it can be called directly. E.g. for each defconfig have a launch command.

It might even make sense to have the launch command in
package/qemu/Config.in.host so it is part of the defconfig, and then have some way (e.g. utils/launch-qemu.sh) to launch it.

But for now, I'd already be happy with board/qemu/*/launch.sh.

support/scripts/qemu-boot-checker.sh line 22

Quoted 7 lines
support/scripts/qemu-boot-checker.sh new file +7 -0
+
+  export QEMU_COMMAND="${qemu_command}"
+
+  export PATH="$2/output/host/bin:$PATH"
+  echo $PATH
+
+  function archQemuNoSupport {

These functions should be declared before the main functionality of the script.

support/scripts/qemu-boot-checker.sh line 34

Quoted 12 lines
support/scripts/qemu-boot-checker.sh new file +12 -0
+    echo "cannot boot under qemu, support out of tree!"
+    exit 0
+  }
+
+  function boot_test {
+    if ! expect expect.sh ; then
+      echo "  booting test system ... FAILED"
+      return 1
+    fi
+    echo "  booting test system ... SUCCESS"
+  }
+  boot_test

IMO it doesn't make much sense to define a function and then immediately call
it. So you could just do:

printf " booting test system ... "
if ! expect expect.sh ; then
echo "FAILED"
exit 1
fi
echo "SUCCESS"

Also, the output of the spawned qemu should be logged somewhere, and that log
file should be a gitlab-ci artifact.

Quoted 7 lines
support/scripts/qemu-boot-checker.sh new file +1 -0
+fi
support/scripts/qemu-boot-defconfig_config.sh new file +0 -0
new file mode 100755
index 0000000..526eee1

support/scripts/qemu-boot-defconfig_config.sh line 9

Quoted 9 lines
support/scripts/qemu-boot-defconfig_config.sh new file +9 -0
@@ -0,0 +1,11 @@
+#!/bin/bash
+
+if [[ $1 = qemu* ]]; then
+  device_name=$(echo $1  | sed -e 's#^qemu_##; s#_defconfig$##;' | sed -r 's/[_]/-/g') 
+  if [ $device_name == "x86-64" ]; then
+    device_name="x86_64"
+    sed -i "s/tty1/ttyS0/" $2/configs/$1
+  elif [ $device_name == "x86" ]; then
+    sed -i "s/tty1/ttyS0/" $2/configs/$1

This sucks. I think we can just remove BR2_TARGET_GENERIC_GETTY_PORT from these
defconfigs and rely on the console (and make sure the console is set correctly in the append argument of qemu).

Regards,
Arnout

Quoted 3 lines
support/scripts/qemu-boot-defconfig_config.sh new file +2 -0
+  fi
+fi
#9 lore raw

PATCH 6/7 support/scripts/qemu-boot: gitlab tests for Qemu

Quoted 4 lines · Arnout Vandecappelle

support/scripts/qemu-boot-checker.sh line 8

Quoted 2 lines
support/scripts/qemu-boot-checker.sh new file +2 -0
+  elif [ $device_name == "xtensa-lx60" ] || [ $device_name == "xtensa-lx60-nommu" ]; then
+    echo "xtensa cannot be tested"

If that is the case, why do we even have these qemu defconfigs? Max?

Tested both configurations just now, with the current buildroot tip (2019.02-959-gd54b0e22aef3) and QEMU-4.0. Both build and boot for me with QEMU command line from the readme.
Curious as well.

Signature · 2 lines
Thanks.
-- Max
#10 lore raw

PATCH 6/7 support/scripts/qemu-boot: gitlab tests for Qemu

Thank you for you comments Arnout, I have corrected the indentation, timeout and function declaration issues.

However; I think that duplicating the content of readme.txt of each qemu architecture would be difficult to maintain (even if it is more convenient to use), this is why a project like toolchain builder :
https://github.com/bootlin/toolchains-builder/blob/master/build.sh is reading directly from readme.txt.
Concerning the naming of x86_64 folder in board/qemu, it is the only one that uses underscore instead of hyphen. Changing the directory name may cause issues with other works as maybe other scripts as relying on this name (especially that it exists since a long time ago).

I want to thank you also Max for your answer (We need to investigate more).

Regards.

Quoted 8 lines · Max Filippov
Quoted 1 line · Arnout Vandecappelle
Quoted 1 line

+ elif [ $device_name == "xtensa-lx60" ] || [ $device_name ==

"xtensa-lx60-nommu" ]; then

Quoted 3 lines
Quoted 1 line

+ echo "xtensa cannot be tested"

If that is the case, why do we even have these qemu defconfigs? Max?

Quoted 9 lines

Tested both configurations just now, with the current buildroot tip (2019.02-959-gd54b0e22aef3) and QEMU-4.0. Both build and boot for me with QEMU command line from the readme.
Curious as well.

--
Thanks.
-- Max

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190430/cf1acc4c/attachment.html>

#11 lore raw

PATCH 6/7 support/scripts/qemu-boot: gitlab tests for Qemu

Hi Jugurtha,

[Please don't top-post, but reply inline like I do below.]

Quoted 8 lines · Jugurtha BELKALEM

Thank you for you comments Arnout, I have corrected the indentation, timeout and? function declaration issues.

However; I think that duplicating the content of readme.txt of each qemu architecture would be difficult to maintain (even if it is more convenient to use), this is why a project like toolchain builder :
https://github.com/bootlin/toolchains-builder/blob/master/build.sh is reading directly from readme.txt.

The idea would not be to duplicate it, but to move it. I.e., readme.txt just
says "launch with board/qemu/x86_64/launch.sh", and the actual command is in that shell script.

But maybe we should see what other Buildroot maintainers have to say about it.
Thomas, Peter, Yann, Romain?

Quoted 4 lines

Concerning the naming of x86_64 folder in board/qemu, it is the only one that uses underscore instead of hyphen. Changing the directory name may cause issues with other works as maybe other scripts as relying on this name (especially that it exists since a long time ago).

OK, true. Let's keep that like you have it then.

Quoted 1 line

I want to thank you also Max for your answer (We need to investigate more).

If it doesn't work on gitlab-ci, I would just not enable the build of host-qemu
for those (and put some intelligence in the script that it only runs if host-qemu has been built).

Quoted 9 lines

On Tue, Apr 30, 2019 at 1:38 AM Max Filippov <[email protected] <mailto:[email protected]>> wrote:

On Mon, Apr 29, 2019 at 3:32 PM Arnout Vandecappelle <[email protected] <mailto:[email protected]>> wrote:
> > +? elif [ $device_name == "xtensa-lx60" ] || [ $device_name == "xtensa-lx60-nommu" ]; then
> > +? ? echo "xtensa cannot be tested"
>

Quoted 5 lines

>? If that is the case, why do we even have these qemu defconfigs? Max?

Tested both configurations just now, with the current buildroot tip (2019.02-959-gd54b0e22aef3) and QEMU-4.0. Both build and boot for me with QEMU command line from the readme.

Is this with the system-intalled qemu? I guess so, because Buildroot still has
3.1.0... That might be the issue :-)

Regards,
Arnout

#12 lore raw

PATCH 6/7 support/scripts/qemu-boot: gitlab tests for Qemu

Quoted 8 lines · Arnout Vandecappelle
Quoted 5 lines
Quoted 1 line

If that is the case, why do we even have these qemu defconfigs? Max?

Tested both configurations just now, with the current buildroot tip (2019.02-959-gd54b0e22aef3) and QEMU-4.0. Both build and boot for me with QEMU command line from the readme.

Is this with the system-intalled qemu? I guess so, because Buildroot still has
3.1.0... That might be the issue :-)

Just tested with that version of QEMU... Both work as expected here. Still puzzled (:

Signature · 2 lines
Thanks.
-- Max
#13 lore raw

PATCH 6/7 support/scripts/qemu-boot: gitlab tests for Qemu

In reply to Arnout Vandecappelle

Arnout, All,

On 2019-04-30 13:20 +0200, Arnout Vandecappelle spake thusly:

Quoted 10 lines · Jugurtha BELKALEM
Quoted 8 lines

Thank you for you comments Arnout, I have corrected the indentation, timeout and? function declaration issues.

However; I think that duplicating the content of readme.txt of each qemu architecture would be difficult to maintain (even if it is more convenient to use), this is why a project like toolchain builder :
https://github.com/bootlin/toolchains-builder/blob/master/build.sh is reading directly from readme.txt.

Quoted 6 lines

The idea would not be to duplicate it, but to move it. I.e., readme.txt just
says "launch with board/qemu/x86_64/launch.sh", and the actual command is in that shell script.

But maybe we should see what other Buildroot maintainers have to say about it.
Thomas, Peter, Yann, Romain?

Ideally, I'd go without the readme.txt altogether, or if we want to keep "backward compatibility", we'd make it a symlink to launch.sh.

Snf then, launch.sh would be an actual shell script that also contains the readme (possibly as -h).

Quoted 9 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line

+? elif [ $device_name == "xtensa-lx60" ] || [ $device_name ==

"xtensa-lx60-nommu" ]; then
> > +? ? echo "xtensa cannot be tested"
>? If that is the case, why do we even have these qemu defconfigs? Max? Tested both configurations just now, with the current buildroot tip (2019.02-959-gd54b0e22aef3) and QEMU-4.0. Both build and boot for me with QEMU command line from the readme.

Is this with the system-intalled qemu? I guess so, because Buildroot still has
3.1.0... That might be the issue :-)

I've started a test-build ehre, to check.

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#14 lore raw

PATCH 6/7 support/scripts/qemu-boot: gitlab tests for Qemu

Arnout, All,

On 2019-04-30 21:40 +0200, Yann E. MORIN spake thusly:

Quoted 1 line

On 2019-04-30 13:20 +0200, Arnout Vandecappelle spake thusly:

[--SNIP--]

Quoted 11 lines
Quoted 9 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line

+? elif [ $device_name == "xtensa-lx60" ] || [ $device_name ==

"xtensa-lx60-nommu" ]; then

Quoted 2 lines
Quoted 1 line

+? ? echo "xtensa cannot be tested"

? If that is the case, why do we even have these qemu defconfigs? Max?

Tested both configurations just now, with the current buildroot tip (2019.02-959-gd54b0e22aef3) and QEMU-4.0. Both build and boot for me with QEMU command line from the readme.

Is this with the system-intalled qemu? I guess so, because Buildroot still has
3.1.0... That might be the issue :-)

I've started a test-build ehre, to check.

And it does not build because:

UIMAGE arch/xtensa/boot/uImage
"mkimage" command not found - U-Boot images will not be built arch/xtensa/boot/Makefile:51: recipe for target 'arch/xtensa/boot/uImage' failed

So, it needs to have host-uboot-tools.

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#15 lore raw

PATCH 6/7 support/scripts/qemu-boot: gitlab tests for Qemu

Quoted 5 lines · Yann E. MORIN

And it does not build because:

UIMAGE arch/xtensa/boot/uImage
"mkimage" command not found - U-Boot images will not be built arch/xtensa/boot/Makefile:51: recipe for target 'arch/xtensa/boot/uImage' failed

Ah, yes, I have mkimage somewhere in my PATH.

Quoted 1 line

So, it needs to have host-uboot-tools.

But I don't think this is the right thing. What I see is that despite BR2_LINUX_KERNEL_IMAGE_TARGET_NAME is set to Image,
to only build self-contained ELF kernel, the initial kernel build is run as 'make ... all', and only the build with the cpio is run as 'make ... Image'. Why?

Signature · 2 lines
Thanks.
-- Max
#16 lore raw

PATCH 6/7 support/scripts/qemu-boot: gitlab tests for Qemu

Quoted 10 lines · Max Filippov
Quoted 5 lines · Yann E. MORIN

And it does not build because:

UIMAGE arch/xtensa/boot/uImage
"mkimage" command not found - U-Boot images will not be built arch/xtensa/boot/Makefile:51: recipe for target 'arch/xtensa/boot/uImage' failed

Ah, yes, I have mkimage somewhere in my PATH.

Quoted 7 lines
Quoted 1 line

So, it needs to have host-uboot-tools.

But I don't think this is the right thing. What I see is that despite BR2_LINUX_KERNEL_IMAGE_TARGET_NAME is set to Image,
to only build self-contained ELF kernel, the initial kernel build is run as 'make ... all', and only the build with the cpio is run as 'make ... Image'. Why?

Ok, I see why: because ffbe46a5295c ("linux: simplify LINUX_BUILD_CMDS"). It's a regression then.

Signature · 2 lines
Thanks.
-- Max
#17 lore raw

PATCH 6/7 support/scripts/qemu-boot: gitlab tests for Qemu

Max, All,

On 2019-04-30 13:23 -0700, Max Filippov spake thusly:

Quoted 13 lines · Max Filippov
Quoted 12 lines · Yann E. MORIN
Quoted 5 lines

And it does not build because:

UIMAGE arch/xtensa/boot/uImage
"mkimage" command not found - U-Boot images will not be built arch/xtensa/boot/Makefile:51: recipe for target 'arch/xtensa/boot/uImage' failed

Ah, yes, I have mkimage somewhere in my PATH.

Quoted 1 line

So, it needs to have host-uboot-tools.

Quoted 8 lines
Quoted 5 lines

But I don't think this is the right thing. What I see is that despite BR2_LINUX_KERNEL_IMAGE_TARGET_NAME is set to Image,
to only build self-contained ELF kernel, the initial kernel build is run as 'make ... all', and only the build with the cpio is run as 'make ... Image'. Why?

Ok, I see why: because ffbe46a5295c ("linux: simplify LINUX_BUILD_CMDS"). It's a regression then.

See also 2a7cf511f4 (linux: split calling "all" and "$(LINUX_TARGET_NAME)" targets) which is an attempt at fixing the regression in ffbe46a5295c.

Furthermore, there were a few patches adding host uboot tools to a few defconfigs that needed it, like 7cbf9c63e5 (configs/qemu_ppc_virtex_ml507:
kernel build needs mkimage)

Finally, see also the discussion on that series:

    http://lists.busybox.net/pipermail/buildroot/2019-April/248424.html

We may have missed a few. I'll send a patch adding it to the extensa defconfigs.

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#18 lore raw

PATCH 6/7 support/scripts/qemu-boot: gitlab tests for Qemu

In reply to Yann E. MORIN
Quoted 12 lines · Yann E. MORIN

Arnout, All,

On 2019-04-30 13:20 +0200, Arnout Vandecappelle spake thusly:

Quoted 9 lines · Jugurtha BELKALEM
Quoted 8 lines

Thank you for you comments Arnout, I have corrected the indentation, timeout and? function declaration issues.

However; I think that duplicating the content of readme.txt of each qemu architecture would be difficult to maintain (even if it is more convenient to use), this is why a project like toolchain builder :
https://github.com/bootlin/toolchains-builder/blob/master/build.sh is reading directly from readme.txt.

Quoted 8 lines
Quoted 6 lines

The idea would not be to duplicate it, but to move it. I.e., readme.txt just
says "launch with board/qemu/x86_64/launch.sh", and the actual command is in that shell script.

But maybe we should see what other Buildroot maintainers have to say about it.
Thomas, Peter, Yann, Romain?

Ideally, I'd go without the readme.txt altogether, or if we want to keep "backward compatibility", we'd make it a symlink to launch.sh.

Sounds good to me. Most of those don't contain any information at all.

Regards,
Arnout

#19 lore raw

PATCH 7/7 gitlab.yml.in: enable Qemu gitlab testing

In reply to Jugurtha BELKALEM

Used to launch qemu tests for various qemu architectures.
Neverthless to say that this file skips any other job different from qemu (only qemu configurations are runtime tested).

---
 .gitlab-ci.yml    | 3 +++
 .gitlab-ci.yml.in | 3 +++
 2 files changed, 6 insertions(+)
.gitlab-ci.yml +3 -0
index e9869be..5884716 100644
@@ -37,6 +37,8 @@ check-package:
 .defconfig_script:
     script:
         - echo 'Configure Buildroot'
+        - main_dir=$(pwd)
+        - ./support/scripts/qemu-boot-defconfig_config.sh "${CI_JOB_NAME}" "${main_dir}"
         - make ${CI_JOB_NAME}
         - echo 'Build buildroot'
         - |
@@ -45,6 +47,7 @@ check-package:
                 tail -200 build.log
                 exit 1
             }
+        - ./support/scripts/qemu-boot-checker.sh "${CI_JOB_NAME}" "${main_dir}"
 
 .defconfig:
     extends: .defconfig_script
.gitlab-ci.yml.in +3 -0
index ad61643..ac5ba57 100644
@@ -37,6 +37,8 @@ check-package:
 .defconfig_script:
     script:
         - echo 'Configure Buildroot'
+        - main_dir=$(pwd)
+        - ./support/scripts/qemu-boot-defconfig_config.sh "${CI_JOB_NAME}" "${main_dir}"
         - make ${CI_JOB_NAME}
         - echo 'Build buildroot'
         - |
@@ -45,6 +47,7 @@ check-package:
                 tail -200 build.log
                 exit 1
             }
+        - ./support/scripts/qemu-boot-checker.sh "${CI_JOB_NAME}" "${main_dir}"
 
 .defconfig:
     extends: .defconfig_script
Signature · 1 line
2.7.4
#20 lore raw

PATCH 7/7 gitlab.yml.in: enable Qemu gitlab testing

Quoted 17 lines · Jugurtha BELKALEM

Used to launch qemu tests for various qemu architectures.
Neverthless to say that this file skips any other job different from qemu (only qemu configurations are runtime tested).

---
 .gitlab-ci.yml    | 3 +++
 .gitlab-ci.yml.in | 3 +++
 2 files changed, 6 insertions(+)
.gitlab-ci.yml +0 -0
index e9869be..5884716 100644

.gitlab-ci.yml check-package: · line 41

Quoted 5 lines
.gitlab-ci.yml +2 -0
@@ -37,6 +37,8 @@ check-package:
 .defconfig_script:
     script:
         - echo 'Configure Buildroot'
+        - main_dir=$(pwd)
+        - ./support/scripts/qemu-boot-defconfig_config.sh "${CI_JOB_NAME}" "${main_dir}"

${main_dir} is still the current directory when the script is called, so it's
not needed to pass it as an argument.

.gitlab-ci.yml check-package: · line 50

Quoted 8 lines
.gitlab-ci.yml +1 -0
         - make ${CI_JOB_NAME}
         - echo 'Build buildroot'
         - |
@@ -45,6 +47,7 @@ check-package:
                 tail -200 build.log
                 exit 1
             }
+        - ./support/scripts/qemu-boot-checker.sh "${CI_JOB_NAME}" "${main_dir}"

Instead of calling this unconditionally for each defconfig, I'd prefer this to
be in a pipeline, i.e. the qemu run is done in a separate job and only if the corresponding build was successful. But it will take some getting familiar with gitlab-ci to get that correct, so this idea may be a little too ambitious.

Regards,
Arnout

Quoted 25 lines
.gitlab-ci.yml +0 -0
 .defconfig:
     extends: .defconfig_script
.gitlab-ci.yml.in +3 -0
index ad61643..ac5ba57 100644
@@ -37,6 +37,8 @@ check-package:
 .defconfig_script:
     script:
         - echo 'Configure Buildroot'
+        - main_dir=$(pwd)
+        - ./support/scripts/qemu-boot-defconfig_config.sh "${CI_JOB_NAME}" "${main_dir}"
         - make ${CI_JOB_NAME}
         - echo 'Build buildroot'
         - |
@@ -45,6 +47,7 @@ check-package:
                 tail -200 build.log
                 exit 1
             }
+        - ./support/scripts/qemu-boot-checker.sh "${CI_JOB_NAME}" "${main_dir}"
 
 .defconfig:
     extends: .defconfig_script