boot/arm-trusted-firmware: support alternate image files
PATCH v3 1/4 boot/arm-trusted-firmware: support alternate image files
Some platform may generate specific boot image files instead of the generic files *.bin when building TF-A package. This change introduces new configuration directive for the arm-trusted-firmware boot package.
BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_IMAGE_NAMES is boolean. When disabled, install boot image files are .../*.bin. When enabled, BR2_TARGET_ARM_TRUSTED_FIRMWARE_BL_IMAGE_NAMES shall lists the names of the generated boot image files.
---
Changes v2 -> v3:
- Replace BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_IMAGE_NAMES and BR2_TARGET_ARM_TRUSTED_FIRMWARE_BL_IMAGE_NAMES with unique config BR2_TARGET_ARM_TRUSTED_FIRMWARE_IMAGE_NAMES.
Changes v1 -> v2:
- No change
--- boot/arm-trusted-firmware/Config.in | 7 +++++++ boot/arm-trusted-firmware/arm-trusted-firmware.mk | 8 +++++++- 2 files changed, 14 insertions(+), 1 deletion(-)
boot/arm-trusted-firmware/Config.in +7 -0
@@ -124,6 +124,13 @@ config BR2_TARGET_ARM_TRUSTED_FIRMWARE_UBOOT_AS_BL33 gets built before ATF, and that the appropriate BL33 variable pointing to u-boot.bin is passed when building ATF. +config BR2_TARGET_ARM_TRUSTED_FIRMWARE_IMAGE_NAMES + string "Image names" + default "*.bin" + help + File names of the generated boot images to be copied to + output directory images/. + config BR2_TARGET_ARM_TRUSTED_FIRMWARE_ADDITIONAL_VARIABLES string "Additional ATF build variables" help
boot/arm-trusted-firmware/arm-trusted-firmware.mk +7 -1
@@ -136,6 +136,12 @@ define ARM_TRUSTED_FIRMWARE_BL31_UBOOT_INSTALL_ELF endef endif +define ARM_TRUSTED_FIRMWARE_BL_IMAGES_INSTALL + $(foreach f, $(call qstrip,$(BR2_TARGET_ARM_TRUSTED_FIRMWARE_IMAGE_NAMES)), + echo $(f) && $(INSTALL) -D -m 0644 -t $(BINARIES_DIR) $(ARM_TRUSTED_FIRMWARE_IMG_DIR)/$(f) + ) +endef + define ARM_TRUSTED_FIRMWARE_BUILD_CMDS $(ARM_TRUSTED_FIRMWARE_BUILD_FIPTOOL) $(TARGET_MAKE_ENV) $(MAKE) -C $(@D) $(ARM_TRUSTED_FIRMWARE_MAKE_OPTS) \
@@ -144,7 +150,7 @@ define ARM_TRUSTED_FIRMWARE_BUILD_CMDS endef define ARM_TRUSTED_FIRMWARE_INSTALL_IMAGES_CMDS - cp -dpf $(ARM_TRUSTED_FIRMWARE_IMG_DIR)/*.bin $(BINARIES_DIR)/ + $(ARM_TRUSTED_FIRMWARE_BL_IMAGES_INSTALL) $(ARM_TRUSTED_FIRMWARE_BL31_UBOOT_INSTALL) $(ARM_TRUSTED_FIRMWARE_BL31_UBOOT_INSTALL_ELF) endef
Signature · 1 line
2.17.1
PATCH v3 2/4 configs/qemu_arm_vexpress_tz: Armv7-A emulation with TrustZone services
This change introduces a Qemu board for an Armv7-A target executing with OP-TEE secure world services. The target Linux based normal world embeds the standard minimal filesystem with OP-TEE non-secure components embedded files from OP-TEE test, examples and benchmark packages.
qemu_arm_vexpress_tz_defconfig differs from qemu_arm_vexpress_defconfig. Supporting both secure and non-secure worlds on the Arm target mandates a secure world, here OP-TEE OS, and a bootloader to boot both worlds, here TF-A (boot/arm-trusted-firmware). Here non-secure Linux kernel is booted through U-boot
TF-A bootloader (BL1/BL2) => OP-TEE (BL32) => U-boot (BL33).
| Executes as secure | Secure | Execs as Non-secure | Loads BL32/BL33 in RAM | Jumps to BL33 | Always booted after | Jumps to BL32 once done | as Non-secure | secure world inits
Vexpress and vexpress-tz defconfigs also differs in that Qemu emulates a Cortex-A9 in the former and a Cortex-A15 in the later. Cortex-A15 is the Armv7-A CPU used in upstream TF-A and OP-TEE OS packages hence selected here.
Defconfig adds a fragment to the Linux kernel native configuration to enable OP-TEE driver support.
Defconfig adds a fragment to the U-Boot native configuration set boot command, enable semihosting and remove U-Boot persistent environment storage support.
The defconfig also enables build of the Qemu emulator in case the system installed Qemu does not yet support CPU TrustZone secure state.
---
Changes v2 -> v3:
- Rename board/config: qemu_armv7a_tz_virt => qemu_arm_vexpress_tz
- Remove some useless configuration from the defconfig: Linux 9P; Qemu virtfs; TF-A log level; OP-TEE log level.
- Add post_build.sh script to rename files in output/images/.
- Detail diffs between vexpress and vexpress-tz defconfigs in commit.
- Fix typo in defconfig about Linux version number.
- Remove info about virtfs from board readme.txt file.
- Update TF-A config from recent changes: image filenames; OP-TEE.
Changes v1 -> v2:
- Bump kernel to 4.19.
- Use kernel defconfig and add optee and 9p as config fragments.
- Remove ARM_ARCH_MAJOR from platform configuration directive BR2_TARGET_ARM_TRUSTED_FIRMWARE_ADDITIONAL_VARIABLES.
--- board/qemu/arm-vexpress-tz/linux.fragment | 3 + board/qemu/arm-vexpress-tz/post-build.sh | 10 ++ board/qemu/arm-vexpress-tz/readme.txt | 135 ++++++++++++++++++++++ board/qemu/arm-vexpress-tz/u-boot.config | 7 ++ configs/qemu_arm_vexpress_tz_defconfig | 47 ++++++++ 5 files changed, 202 insertions(+) create mode 100644 board/qemu/arm-vexpress-tz/linux.fragment create mode 100755 board/qemu/arm-vexpress-tz/post-build.sh create mode 100644 board/qemu/arm-vexpress-tz/readme.txt create mode 100644 board/qemu/arm-vexpress-tz/u-boot.config create mode 100644 configs/qemu_arm_vexpress_tz_defconfig
board/qemu/arm-vexpress-tz/linux.fragment new file +3 -0
@@ -0,0 +1,3 @@ +### Enable OP-TEE +CONFIG_TEE=y +CONFIG_OPTEE=y
board/qemu/arm-vexpress-tz/post-build.sh new file +10 -0
@@ -0,0 +1,10 @@ +#!/bin/sh + +set -u +set -e + +# Rename boot images for the dear TF-A +ln -sf u-boot.bin ${BINARIES_DIR}/bl33.bin +ln -sf tee-header_v2.bin ${BINARIES_DIR}/bl32.bin +ln -sf tee-pager_v2.bin ${BINARIES_DIR}/bl32_extra1.bin +ln -sf tee-pageable_v2.bin ${BINARIES_DIR}/bl32_extra2.bin
board/qemu/arm-vexpress-tz/readme.txt new file +135 -0
135 lines not on this page · raw message
board/qemu/arm-vexpress-tz/u-boot.config new file +7 -0
@@ -0,0 +1,7 @@ +CONFIG_SYS_TEXT_BASE=0x60000000 +CONFIG_BOOTCOMMAND="fdt addr ${fdt_addr} && fdt resize 1000 && smhload zImage ${kernel_addr_r} && smhload rootfs.cpio.gz ${ramdisk_addr_r} ramdisk_addr_end && setenv bootargs console=ttyAMA0,115200 earlyprintk=serial,ttyAMA0,115200 && fdt chosen ${ramdisk_addr_r} ${ramdisk_addr_end} && bootz ${kernel_addr_r} - ${fdt_addr}" +CONFIG_SEMIHOSTING=y +# Drop flash accesses +CONFIG_ENV_IS_IN_FLASH=n +CONFIG_MTD=n +CONFIG_MTD_NOR_FLASH=n
configs/qemu_arm_vexpress_tz_defconfig new file +47 -0
47 lines not on this page · raw message
Signature · 1 line
2.17.1
Arnout Vandecappelle
AppliedPATCH v3 2/4 configs/qemu_arm_vexpress_tz: Armv7-A emulation with TrustZone services
Hi Etienne,
Quoted 24 lines · Etienne Carriere
This change introduces a Qemu board for an Armv7-A target executing with OP-TEE secure world services. The target Linux based normal world embeds the standard minimal filesystem with OP-TEE non-secure components embedded files from OP-TEE test, examples and benchmark packages.
qemu_arm_vexpress_tz_defconfigdiffers fromqemu_arm_vexpress_defconfig. Supporting both secure and non-secure worlds on the Arm target mandates a secure world, here OP-TEE OS, and a bootloader to boot both worlds, here TF-A (boot/arm-trusted-firmware). Here non-secure Linux kernel is booted through U-bootTF-A bootloader (BL1/BL2) => OP-TEE (BL32) => U-boot (BL33).
| Executes as secure | Secure | Execs as Non-secure | Loads BL32/BL33 in RAM | Jumps to BL33 | Always booted after | Jumps to BL32 once done | as Non-secure | secure world initsVexpress and vexpress-tz defconfigs also differs in that Qemu emulates a Cortex-A9 in the former and a Cortex-A15 in the later. Cortex-A15 is the Armv7-A CPU used in upstream TF-A and OP-TEE OS packages hence selected here.
Defconfig adds a fragment to the Linux kernel native configuration to enable OP-TEE driver support.
Quoted 8 lines
Defconfig adds a fragment to the U-Boot native configuration set boot command, enable semihosting and remove U-Boot persistent environment storage support.
The defconfig also enables build of the Qemu emulator in case the system installed Qemu does not yet support CPU TrustZone secure state.
Signed-off-by Etienne Carriere <[email protected]>
Applied to master, thanks, but with some changes...
First of all, thank you for the very extensive and clear commit message and
readme file.
[snip]
board/qemu/arm-vexpress-tz/readme.txt line 13
Quoted 13 lines
board/qemu/arm-vexpress-tz/readme.txt new file +13
-0+Board qemu_arm_vexpress_tz builds a Qemu Armv7-A target system with +OP-TEE running in the TrustZone secure world and a Linux based +OS running in the non-secure world. The board configuration enable +builds of the Qemu host Arm target emulator. + + make qemu_arm_vexpress_tz_defconfig + make + +BIOS used in the Qemu host is the Arm Trusted Firmware-A (TF-A). TF-A +uses Qemu semihosting file access to access boot image files. The +Qemu platform is quite specific for that in TF-A and one needs to +run the emulation from the image directory for TF-A to boot the +secure and non-secure worlds.
This semihosting approach is not so nice, because it only works on qemu. It
would be nicer to have a single image that contains everything (except bl1 I guess) and use that as virtual flash, so it matches what would happen on a real board. But this is not a bad start, and it might make debugging the optee part easier.
board/qemu/arm-vexpress-tz/readme.txt line 21
Quoted 8 lines
board/qemu/arm-vexpress-tz/readme.txt new file +8
-0+ + cd output/images && ../host/bin/qemu-system-arm \ + -machine virt -machine secure=on -cpu cortex-a15 \ + -smp 1 -s -m 1024 -d unimp \ + -serial stdio \ + -netdev user,id=vmnic -device virtio-net-device,netdev=vmnic \ + -semihosting-config enable,target=native \ + -bios bl1.bin
I'm a bit worried that the script in the toolchains-builder will not be able to
parse this. But because of the cd, it will anyway not work, so OK. It anyway looks a lot nicer like this than how it's done in the other readmes.
[snip]
configs/qemu_arm_vexpress_tz_defconfig line 7
Quoted 8 lines
configs/qemu_arm_vexpress_tz_defconfig new file +7
-0@@ -0,0 +1,47 @@ +# Architecture +BR2_arm=y +BR2_cortex_a15=y +BR2_ARM_ENABLE_NEON=y +BR2_ARM_ENABLE_VFP=y +BR2_ARM_FPU_VFPV3D16=y +# System
Please add an empty line before the different sections.
configs/qemu_arm_vexpress_tz_defconfig line 12
Quoted 5 lines
configs/qemu_arm_vexpress_tz_defconfig new file +5
-0+BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0" +# Filesystems (support several boot config) +BR2_TARGET_ROOTFS_CPIO=y +BR2_TARGET_ROOTFS_CPIO_GZIP=y +BR2_TARGET_ROOTFS_EXT2=y
There's no reason at all to add ext2 and tar, so I removed both of them. If you
want to support several boot configs, it should be mentioned in the readme file how to do that.
Quoted 6 lines
configs/qemu_arm_vexpress_tz_defconfig new file +6
-0+# Generic +BR2_ROOTFS_POST_BUILD_SCRIPT="board/qemu/arm-vexpress-tz/post-build.sh" +# Linux 4.19 series +BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_19=y +BR2_LINUX_KERNEL=y +BR2_LINUX_KERNEL_CUSTOM_VERSION=y
configs/qemu_arm_vexpress_tz_defconfig line 28
Quoted 10 lines
configs/qemu_arm_vexpress_tz_defconfig new file +10
-0+BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19.16" +BR2_LINUX_KERNEL_DEFCONFIG="vexpress" +BR2_LINUX_KERNEL_CONFIG_FRAGMENT_FILES="board/qemu/arm-vexpress-tz/linux.fragment" +BR2_LINUX_KERNEL_DTS_SUPPORT=y +BR2_LINUX_KERNEL_INTREE_DTS_NAME="vexpress-v2p-ca15_a7" +# TF-A for booting OP-TEE secure and uboot/linux non secure +BR2_TARGET_ARM_TRUSTED_FIRMWARE=y +BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_GIT=y +BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_REPO_URL="https://github.com/ARM-software/arm-trusted-firmware.git" +BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_REPO_VERSION="v2.0"
There is a version selection available now, so I used that instead of the git
download.
BTW, our current ATF version is still v1.4, maybe it should be bumped?
configs/qemu_arm_vexpress_tz_defconfig line 40
Quoted 12 lines
configs/qemu_arm_vexpress_tz_defconfig new file +12
-0+BR2_TARGET_ARM_TRUSTED_FIRMWARE_PLATFORM="qemu" +BR2_TARGET_ARM_TRUSTED_FIRMWARE_BL32_OPTEE=y +BR2_TARGET_ARM_TRUSTED_FIRMWARE_UBOOT_AS_BL33=y +BR2_TARGET_ARM_TRUSTED_FIRMWARE_ADDITIONAL_VARIABLES="BL32_RAM_LOCATION=tdram" +# OP-TEE components +BR2_TARGET_OPTEE_OS=y +BR2_TARGET_OPTEE_OS_PLATFORM="vexpress-qemu_virt" +BR2_PACKAGE_OPTEE_CLIENT=y +BR2_PACKAGE_OPTEE_TEST=y +BR2_PACKAGE_OPTEE_EXAMPLES=y +BR2_PACKAGE_OPTEE_BENCHMARK=y +# U-boot for booting the dear Linux kernel
:-)
configs/qemu_arm_vexpress_tz_defconfig line 41
Quoted 1 line
configs/qemu_arm_vexpress_tz_defconfig new file +1
-0+BR2_TARGET_UBOOT=y
You have to specify the U-Boot version. I'm not sure what you tested with, but
I used 2019.01 and it worked.
configs/qemu_arm_vexpress_tz_defconfig line 45
Quoted 4 lines
configs/qemu_arm_vexpress_tz_defconfig new file +4
-0+BR2_TARGET_UBOOT_BUILD_SYSTEM_KCONFIG=y +BR2_TARGET_UBOOT_BOARD_DEFCONFIG="qemu_arm" +BR2_TARGET_UBOOT_CONFIG_FRAGMENT_FILES="board/qemu/arm-vexpress-tz/u-boot.config" +# Build Qemu emulator for the Arm target
I changed this in what we use everywhere else: host-qemu for gitlab testing
Regards,
Arnout
Quoted 3 lines
configs/qemu_arm_vexpress_tz_defconfig new file +2
-0+BR2_PACKAGE_HOST_QEMU=y +BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
PATCH v3 2/4 configs/qemu_arm_vexpress_tz: Armv7-A emulation with TrustZone services
Quoted 69 lines · Arnout Vandecappelle
Hi Etienne,
Quoted 24 lines · Etienne Carriere
This change introduces a Qemu board for an Armv7-A target executing with OP-TEE secure world services. The target Linux based normal world embeds the standard minimal filesystem with OP-TEE non-secure components embedded files from OP-TEE test, examples and benchmark packages.
qemu_arm_vexpress_tz_defconfigdiffers fromqemu_arm_vexpress_defconfig. Supporting both secure and non-secure worlds on the Arm target mandates a secure world, here OP-TEE OS, and a bootloader to boot both worlds, here TF-A (boot/arm-trusted-firmware). Here non-secure Linux kernel is booted through U-bootTF-A bootloader (BL1/BL2) => OP-TEE (BL32) => U-boot (BL33).
| Executes as secure | Secure | Execs as Non-secure | Loads BL32/BL33 in RAM | Jumps to BL33 | Always booted after | Jumps to BL32 once done | as Non-secure | secure world initsVexpress and vexpress-tz defconfigs also differs in that Qemu emulates a Cortex-A9 in the former and a Cortex-A15 in the later. Cortex-A15 is the Armv7-A CPU used in upstream TF-A and OP-TEE OS packages hence selected here.
Defconfig adds a fragment to the Linux kernel native configuration to enable OP-TEE driver support.
Quoted 8 lines
Defconfig adds a fragment to the U-Boot native configuration set boot command, enable semihosting and remove U-Boot persistent environment storage support.
The defconfig also enables build of the Qemu emulator in case the system installed Qemu does not yet support CPU TrustZone secure state.
Signed-off-by Etienne Carriere <[email protected]>Applied to master, thanks, but with some changes...
[snip]
board/qemu/arm-vexpress-tz/readme.txt line 13
Quoted 13 lines
board/qemu/arm-vexpress-tz/readme.txt new file +13
-0+Board qemu_arm_vexpress_tz builds a Qemu Armv7-A target system with +OP-TEE running in the TrustZone secure world and a Linux based +OS running in the non-secure world. The board configuration enable +builds of the Qemu host Arm target emulator. + + make qemu_arm_vexpress_tz_defconfig + make + +BIOS used in the Qemu host is the Arm Trusted Firmware-A (TF-A). TF-A +uses Qemu semihosting file access to access boot image files. The +Qemu platform is quite specific for that in TF-A and one needs to +run the emulation from the image directory for TF-A to boot the +secure and non-secure worlds.This semihosting approach is not so nice, because it only works on qemu. It
would be nicer to have a single image that contains everything (except bl1 I guess) and use that as virtual flash, so it matches what would happen on a real board. But this is not a bad start, and it might make debugging the optee part easier.board/qemu/arm-vexpress-tz/readme.txt line 21
Quoted 8 lines
board/qemu/arm-vexpress-tz/readme.txt new file +8
-0+ + cd output/images && ../host/bin/qemu-system-arm \ + -machine virt -machine secure=on -cpu cortex-a15 \ + -smp 1 -s -m 1024 -d unimp \ + -serial stdio \ + -netdev user,id=vmnic -device virtio-net-device,netdev=vmnic \ + -semihosting-config enable,target=native \ + -bios bl1.bin
Quoted 3 lines
I'm a bit worried that the script in the toolchains-builder will not be able to
parse this. But because of the cd, it will anyway not work, so OK. It anyway looks a lot nicer like this than how it's done in the other readmes.
Thanks,
Quoted 60 lines
[snip]
configs/qemu_arm_vexpress_tz_defconfig line 7
Quoted 8 lines
configs/qemu_arm_vexpress_tz_defconfig new file +7
-0@@ -0,0 +1,47 @@ +# Architecture +BR2_arm=y +BR2_cortex_a15=y +BR2_ARM_ENABLE_NEON=y +BR2_ARM_ENABLE_VFP=y +BR2_ARM_FPU_VFPV3D16=y +# SystemPlease add an empty line before the different sections.
configs/qemu_arm_vexpress_tz_defconfig line 12
Quoted 5 lines
configs/qemu_arm_vexpress_tz_defconfig new file +5
-0+BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0" +# Filesystems (support several boot config) +BR2_TARGET_ROOTFS_CPIO=y +BR2_TARGET_ROOTFS_CPIO_GZIP=y +BR2_TARGET_ROOTFS_EXT2=yThere's no reason at all to add ext2 and tar, so I removed both of them. If you
want to support several boot configs, it should be mentioned in the readme file how to do that.Quoted 6 lines
configs/qemu_arm_vexpress_tz_defconfig new file +6
-0+# Generic +BR2_ROOTFS_POST_BUILD_SCRIPT="board/qemu/arm-vexpress-tz/post-build.sh" +# Linux 4.19 series +BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_19=y +BR2_LINUX_KERNEL=y +BR2_LINUX_KERNEL_CUSTOM_VERSION=yconfigs/qemu_arm_vexpress_tz_defconfig line 28
Quoted 10 lines
configs/qemu_arm_vexpress_tz_defconfig new file +10
-0+BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19.16" +BR2_LINUX_KERNEL_DEFCONFIG="vexpress" +BR2_LINUX_KERNEL_CONFIG_FRAGMENT_FILES="board/qemu/arm-vexpress-tz/linux.fragment" +BR2_LINUX_KERNEL_DTS_SUPPORT=y +BR2_LINUX_KERNEL_INTREE_DTS_NAME="vexpress-v2p-ca15_a7" +# TF-A for booting OP-TEE secure and uboot/linux non secure +BR2_TARGET_ARM_TRUSTED_FIRMWARE=y +BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_GIT=y +BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_REPO_URL="https://github.com/ARM-software/arm-trusted-firmware.git" +BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_REPO_VERSION="v2.0"There is a version selection available now, so I used that instead of the git
download.BTW, our current ATF version is still v1.4, maybe it should be bumped?
configs/qemu_arm_vexpress_tz_defconfig line 40
Quoted 12 lines
configs/qemu_arm_vexpress_tz_defconfig new file +12
-0+BR2_TARGET_ARM_TRUSTED_FIRMWARE_PLATFORM="qemu" +BR2_TARGET_ARM_TRUSTED_FIRMWARE_BL32_OPTEE=y +BR2_TARGET_ARM_TRUSTED_FIRMWARE_UBOOT_AS_BL33=y +BR2_TARGET_ARM_TRUSTED_FIRMWARE_ADDITIONAL_VARIABLES="BL32_RAM_LOCATION=tdram" +# OP-TEE components +BR2_TARGET_OPTEE_OS=y +BR2_TARGET_OPTEE_OS_PLATFORM="vexpress-qemu_virt" +BR2_PACKAGE_OPTEE_CLIENT=y +BR2_PACKAGE_OPTEE_TEST=y +BR2_PACKAGE_OPTEE_EXAMPLES=y +BR2_PACKAGE_OPTEE_BENCHMARK=y +# U-boot for booting the dear Linux kernel:-)
Quoted 4 lines
configs/qemu_arm_vexpress_tz_defconfig line 41
Quoted 1 line
configs/qemu_arm_vexpress_tz_defconfig new file +1
-0+BR2_TARGET_UBOOT=yYou have to specify the U-Boot version. I'm not sure what you tested with, but
I used 2019.01 and it worked.
Would be nice to use BR2_TARGET_UBOOT_LATEST_VERSION. BR2_TARGET_UBOOT_LATEST_VERSION=y
As for the linux kernel, i wonder if the generic config would be better?
Quoted 7 lines
configs/qemu_arm_vexpress_tz_defconfig line 45
Quoted 5 lines
configs/qemu_arm_vexpress_tz_defconfig new file +4
-0+BR2_TARGET_UBOOT_BUILD_SYSTEM_KCONFIG=y +BR2_TARGET_UBOOT_BOARD_DEFCONFIG="qemu_arm" +BR2_TARGET_UBOOT_CONFIG_FRAGMENT_FILES="board/qemu/arm-vexpress-tz/u-boot.config" +# Build Qemu emulator for the Arm targetI changed this in what we use everywhere else: host-qemu for gitlab testing
Thanks.
Regards,
Etienne
Quoted 7 lines
Regards,
ArnoutQuoted 3 lines
configs/qemu_arm_vexpress_tz_defconfig new file +2
-0+BR2_PACKAGE_HOST_QEMU=y +BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
PATCH v3 2/4 configs/qemu_arm_vexpress_tz: Armv7-A emulation with TrustZone services
Quoted 6 lines · Etienne Carriere
Quoted 4 lines · Arnout Vandecappelle
[snip]
Quoted 1 line
+BR2_TARGET_UBOOT=y
Quoted 7 lines
Quoted 2 lines
You have to specify the U-Boot version. I'm not sure what you tested with, but
I used 2019.01 and it worked.Would be nice to use
BR2_TARGET_UBOOT_LATEST_VERSION. BR2_TARGET_UBOOT_LATEST_VERSION=yAs for the linux kernel, i wonder if the generic config would be better?
Sorry: mail popped from my fingers while I was typing.
I meant, would it be better if the qemu/tz defconfig is based on:
BR2_TARGET_UBOOT_LATEST_VERSION=y BR2_LINUX_KERNEL_LATEST_VERSION=y BR2_LINUX_KERNEL_USE_ARCH_DEFAULT_CONFIG=y
Regards,
Etienne
PATCH v3 2/4 configs/qemu_arm_vexpress_tz: Armv7-A emulation with TrustZone services
Quoted 14 lines · Etienne Carriere
Quoted 12 lines · Etienne Carriere
Quoted 6 lines · Arnout Vandecappelle
[snip]
Quoted 1 line
+BR2_TARGET_UBOOT=y
You have to specify the U-Boot version. I'm not sure what you tested with, but
I used 2019.01 and it worked.Would be nice to use
BR2_TARGET_UBOOT_LATEST_VERSION. BR2_TARGET_UBOOT_LATEST_VERSION=y
Quoted 8 lines
Quoted 1 line
As for the linux kernel, i wonder if the generic config would be better?
Sorry: mail popped from my fingers while I was typing.
I meant, would it be better if the qemu/tz defconfig is based on:
BR2_TARGET_UBOOT_LATEST_VERSION=y BR2_LINUX_KERNEL_LATEST_VERSION=y BR2_LINUX_KERNEL_USE_ARCH_DEFAULT_CONFIG=y
You original submission had this, and that didn't build because some of the
config options were invalid. That shows that you *have* to have the version.
Regards,
Arnout
PATCH v3 2/4 configs/qemu_arm_vexpress_tz: Armv7-A emulation with TrustZone services
In reply to Etienne CarriereHello Etienne,
Quoted 6 lines · On Fri, 22 Mar 2019 10:58:16 +0100 Etienne Carriere…
board/qemu/arm-vexpress-tz/linux.fragment | 3 + board/qemu/arm-vexpress-tz/post-build.sh | 10 ++ board/qemu/arm-vexpress-tz/readme.txt | 135 ++++++++++++++++++++++ board/qemu/arm-vexpress-tz/u-boot.config | 7 ++ configs/qemu_arm_vexpress_tz_defconfig | 47 ++++++++ 5 files changed, 202 insertions(+)
This defconfig causes a build failure, reported at
https://gitlab.com/buildroot.org/buildroot/-/jobs/389451835, like this:
486 Traceback (most recent call last):
487 File "scripts/pem_to_pub_c.py", line 61, in <module>
488 main()
489 File "scripts/pem_to_pub_c.py", line 24, in main
490 from Crypto.PublicKey import RSA
491 ImportError: No module named 'Crypto'
Could you have a look ?
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v3 2/4 configs/qemu_arm_vexpress_tz: Armv7-A emulation with TrustZone services
Hello Thomas and all,
My best wishes for this new year to you and your beloved.
Quoted 15 lines · Thomas Petazzoni
Hello Etienne,
Quoted 6 lines · On Fri, 22 Mar 2019 10:58:16 +0100 Etienne Carriere…
board/qemu/arm-vexpress-tz/linux.fragment | 3 + board/qemu/arm-vexpress-tz/post-build.sh | 10 ++ board/qemu/arm-vexpress-tz/readme.txt | 135 ++++++++++++++++++++++ board/qemu/arm-vexpress-tz/u-boot.config | 7 ++ configs/qemu_arm_vexpress_tz_defconfig | 47 ++++++++ 5 files changed, 202 insertions(+)This defconfig causes a build failure, reported at
https://gitlab.com/buildroot.org/buildroot/-/jobs/389451835, like this:
Quoted 9 lines
486 Traceback (most recent call last):
487 File "scripts/pem_to_pub_c.py", line 61, in <module>
488main()
489 File "scripts/pem_to_pub_c.py", line 24, in main
490 from Crypto.PublicKey import RSA
491 ImportError: No module named 'Crypto'Could you have a look ?
This issue was reported by Romain [1] when I posted an upgrade from optee-3.5.0 to 3.7.0 in BR.
Actually, I'm still not sure I really understand the issue since optee-os.mk already specifies pycrypto as a dependency.
OPTEE_OS_DEPENDENCIES = host-openssl host-python-pycrypto host-python-pyelftools
Maybe it is an issue related to python versioning (2 vs 3)? I'll try to have a look in the coming... days, but I quite busy on some other topics.
Will do my best :|
Regards,
Etienne
Quoted 8 lines
Best regards,
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200107/1b4a1431/attachment.html>
PATCH v3 2/4 configs/qemu_arm_vexpress_tz: Armv7-A emulation with TrustZone services
Hi Etienne,
Le 07/01/2020 ? 08:56, Etienne Carriere a ?crit?:
Quoted 39 lines
Hello Thomas and all,
My best wishes?for this new year to you and your beloved.
On Sat, 28 Dec 2019 at 12:35, Thomas Petazzoni <[email protected] <mailto:[email protected]>> wrote:
Hello Etienne, On Fri, 22 Mar 2019 10:58:16 +0100 Etienne Carriere <[email protected] <mailto:[email protected]>> wrote: >? board/qemu/arm-vexpress-tz/linux.fragment |? ?3 + >? board/qemu/arm-vexpress-tz/post-build.sh? |? 10 ++ >? board/qemu/arm-vexpress-tz/readme.txt? ? ?| 135 ++++++++++++++++++++++ >? board/qemu/arm-vexpress-tz/u-boot.config? |? ?7 ++ >? configs/qemu_arm_vexpress_tz_defconfig? ? |? 47 ++++++++ >? 5 files changed, 202 insertions(+) This defconfig causes a build failure, reported at https://gitlab.com/buildroot.org/buildroot/-/jobs/389451835, like this: 486 Traceback (most recent call last): 487? ?File "scripts/pem_to_pub_c.py", line 61, in <module> 488? ? ?main() 489? ?File "scripts/pem_to_pub_c.py", line 24, in main 490? ? ?from Crypto.PublicKey import RSA 491 ImportError: No module named 'Crypto' Could you have a look ?This issue was reported by Romain [1] when I posted an upgrade from optee-3.5.0 to 3.7.0 in BR.
Actually, I'm still not sure?I really understand the issue since optee-os.mk
<http://optee-os.mk> already specifies pycrypto as a dependency.
Quoted 6 lines
OPTEE_OS_DEPENDENCIES= host-openssl host-python-pycrypto host-python-pyelftoolsMaybe it is an issue related to?python?versioning (2 vs 3)?
I'll try to have a look in the coming... days, but I quite busy on some other topics.
Will do my best :|
Indeed, this is an issue related to host-python dependencies.
Since optee-3.7.0 all scripts are python3 only [1] but the optee package still depends on host-python (python2).
But even if we modify to use host-python3, we still need host-python3 modules being build for python3 (host-python-pycrypto host-python-pyelftools). Since we can't build host python modules for both python2 and python3, the issue appear again when host-python (python2) is used as host python interpreter.
This is a similar issue that for mesa3d where host-python3-make package was introduced to be able to build. We can fixes this issue by adding host-python3-pycrypto host-python3-pyelftools package (and reverse dependencies if any).
I guess we don't have the issue only if python3 and pycrypto pyelftools are installed on the host.
[1]
https://github.com/OP-TEE/optee_os/commit/bbaeed4dc6258006e846543197b8aff95d80abbf
Best regards,
Romain
Quoted 23 lines
Regards,
Etienne?
Best regards,
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
PATCH v3 2/4 configs/qemu_arm_vexpress_tz: Armv7-A emulation with TrustZone services
Hi Etienne,
Le 09/02/2020 ? 18:55, Romain Naour a ?crit?:
Quoted 64 lines
Hi Etienne,
Le 07/01/2020 ? 08:56, Etienne Carriere a ?crit?:
Quoted 39 lines
Hello Thomas and all,
My best wishes?for this new year to you and your beloved.
On Sat, 28 Dec 2019 at 12:35, Thomas Petazzoni <[email protected] <mailto:[email protected]>> wrote:
Hello Etienne, On Fri, 22 Mar 2019 10:58:16 +0100 Etienne Carriere <[email protected] <mailto:[email protected]>> wrote: >? board/qemu/arm-vexpress-tz/linux.fragment |? ?3 + >? board/qemu/arm-vexpress-tz/post-build.sh? |? 10 ++ >? board/qemu/arm-vexpress-tz/readme.txt? ? ?| 135 ++++++++++++++++++++++ >? board/qemu/arm-vexpress-tz/u-boot.config? |? ?7 ++ >? configs/qemu_arm_vexpress_tz_defconfig? ? |? 47 ++++++++ >? 5 files changed, 202 insertions(+) This defconfig causes a build failure, reported at https://gitlab.com/buildroot.org/buildroot/-/jobs/389451835, like this: 486 Traceback (most recent call last): 487? ?File "scripts/pem_to_pub_c.py", line 61, in <module> 488? ? ?main() 489? ?File "scripts/pem_to_pub_c.py", line 24, in main 490? ? ?from Crypto.PublicKey import RSA 491 ImportError: No module named 'Crypto' Could you have a look ?This issue was reported by Romain [1] when I posted an upgrade from optee-3.5.0 to 3.7.0 in BR.
Actually, I'm still not sure?I really understand the issue since optee-os.mk
<http://optee-os.mk> already specifies pycrypto as a dependency.Quoted 6 lines
OPTEE_OS_DEPENDENCIES= host-openssl host-python-pycrypto host-python-pyelftoolsMaybe it is an issue related to?python?versioning (2 vs 3)?
I'll try to have a look in the coming... days, but I quite busy on some other topics.
Will do my best :|Indeed, this is an issue related to host-python dependencies.
Since optee-3.7.0 all scripts are python3 only [1] but the optee package still depends on host-python (python2).
But even if we modify to use host-python3, we still need host-python3 modules being build for python3 (host-python-pycrypto host-python-pyelftools). Since we can't build host python modules for both python2 and python3, the issue appear again when host-python (python2) is used as host python interpreter.
This is a similar issue that for mesa3d where host-python3-make package was introduced to be able to build. We can fixes this issue by adding host-python3-pycrypto host-python3-pyelftools package (and reverse dependencies if any).
Quoted 5 lines
I guess we don't have the issue only if python3 and pycrypto pyelftools are installed on the host.
[1]
https://github.com/OP-TEE/optee_os/commit/bbaeed4dc6258006e846543197b8aff95d80abbf
I tried to build this defconfig with host-python3 as default python interpreter (by selecting python3 on the target) but there are several other issues:
First python-pycrypto doesn't work with python-3.8 due to time.clock() that was removed from python 3.8.
There is a patch pending:
https://github.com/dlitz/pycrypto/pull/296/commits/6d41ad025331afce9e495d7be3205730ddfa8f07
The build continue up to optee-test package:
Quoted 1 line
Quoted 1 line
Quoted 1 line
optee-test 3.7.0 Building
Traceback (most recent call last):
File "../../scripts/file_to_c.py", line 48, in <module>
[...]
TypeError: cannot use a str to initialize an array with typecode 'B'
So, it seems the optee stack is not in good shape due to python 3.8 bump.
Can you take a look?
Best regards,
Romain
Quoted 28 lines
Best regards,
RomainQuoted 23 lines
Regards,
Etienne?
Best regards,
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
PATCH v3 3/4 support/testing: test can use the locally generated qemu host tool
In reply to Etienne CarriereThis change adds argument --local-emulator to run-tests to use a locally built emulator for the runtime test. Target test shall provide a config_emulator attribute unless what test is inconsistent.
---
Changes v2 -> v3:
- Add an argument to run-tests to use locally built emulator. Test can still enforce use of locally built emulator from boot method argument local=True|False (still defaults to False).
Changes v1 -> v2:
- No commit not in v1 series.
--- support/testing/infra/basetest.py | 5 ++++- support/testing/infra/emulator.py | 19 ++++++++++++++++--- support/testing/run-tests | 4 ++++ 3 files changed, 24 insertions(+), 4 deletions(-)
support/testing/infra/basetest.py +4 -1
@@ -51,6 +51,8 @@ class BRConfigTest(unittest.TestCase): def setUp(self): self.show_msg("Starting") + if self.local_emulator: + self.config = self.config + self.config_emulator self.b = Builder(self.config, self.builddir, self.logtofile) if not self.keepbuilds:
@@ -78,7 +80,8 @@ class BRTest(BRConfigTest): self.show_msg("Building done") self.emulator = Emulator(self.builddir, self.downloaddir, - self.logtofile, self.timeout_multiplier) + self.logtofile, self.timeout_multiplier, + self.local_emulator) def tearDown(self): if self.emulator:
support/testing/infra/emulator.py +16 -3
@@ -1,14 +1,19 @@ import pexpect +import os + import infra class Emulator(object): - def __init__(self, builddir, downloaddir, logtofile, timeout_multiplier): + def __init__(self, builddir, downloaddir, logtofile, timeout_multiplier, + local_emulator): self.qemu = None self.downloaddir = downloaddir self.logfile = infra.open_log_file(builddir, "run", logtofile) + self.builddir = builddir + self.local_emulator = local_emulator # We use elastic runners on the cloud to runs our tests. Those runners # can take a long time to run the emulator. Use a timeout multiplier # when running the tests to avoid sporadic failures.
@@ -30,13 +35,21 @@ class Emulator(object): # # options: array of command line options to pass to Qemu # - def boot(self, arch, kernel=None, kernel_cmdline=None, options=None): + # local: if True, the locally built qemu host tool is used instead of a + # qemu host tool found from the PATH. + # + def boot(self, arch, kernel=None, kernel_cmdline=None, options=None, + local=None): if arch in ["armv7", "armv5"]: qemu_arch = "arm" else: qemu_arch = arch - qemu_cmd = ["qemu-system-{}".format(qemu_arch), + qemu_bin = "qemu-system-{}".format(qemu_arch) + if self.local_emulator or local: + qemu_bin = os.path.join(self.builddir, 'host', 'bin', qemu_bin) + + qemu_cmd = [qemu_bin, "-serial", "stdio", "-display", "none"]
support/testing/run-tests +4 -0
@@ -31,6 +31,8 @@ def main(): help='BR2_JLEVEL to use for each testcase') parser.add_argument('--timeout-multiplier', type=int, default=1, help='increase timeouts (useful for slow machines)') + parser.add_argument('--local-emulator', action='store_true', + help='use a locally built emulator') args = parser.parse_args()
@@ -107,6 +109,8 @@ def main(): return 1 BRConfigTest.timeout_multiplier = args.timeout_multiplier + BRConfigTest.local_emulator = args.local_emulator + nose2_args = ["-v", "-N", str(args.testcases), "-s", test_dir,
Signature · 1 line
2.17.1
PATCH v3 3/4 support/testing: test can use the locally generated qemu host tool
Hello,
+ Arnout
Quoted 10 lines · Etienne Carriere
This change adds argument --local-emulator to run-tests to use a locally built emulator for the runtime test. Target test shall provide a
config_emulatorattribute unless what test is inconsistent.Signed-off-by Etienne Carriere <[email protected]>---
Changes v2 -> v3:
- Add an argument to run-tests to use locally built emulator. Test can still enforce use of locally built emulator from boot method argument local=True|False (still defaults to False).
I am not strongly against it.
But let me ask:
Are you or someone else having trouble with any other test case when using "old" qemu versions from host?
If the answer is no, maybe we should keep things simple and use a patch very similar to v2 of this one, just adding the host-qemu to the test case config fragment and the 'local' argument to Emulator.run.
I say similar to v2 because I think your implementation in v3 with the 'qemu_bin' variable is better than v2.
If the answer is yes, be aware that one could download the docker image that we use in GitLab CI and run the tests inside it. This image has qemu 2.8.1 that currently run all tests we have without any problem AFAIK.
IMO, we remove --local-emulator and add local=True and host-qemu to the config fragment of any tests (currently only the one in the next patch) that needs a qemu newer than 2.8.1.
Another approach "to have an generic option to script run-tests to build the emulator within the test config" (quoting your reply to v2) would be:
Instead of adding a command line argument to run-tests, create a new flag in the test case class, perhaps named "build_emulator". If that flag is True, the host-qemu is automatically added to the config fragment by the test infra. The same flag could be propagated to the Emulator.init so we don't need to pass 'local' to Emulator.run.
If for some reason a flag (boolean) is not enough, for example if the host-qemu would not always be set with same options (I don't know if that is the case), the test case that needs the local build of emulator could pass config_emulator and that would make the test infra to know that it needs to build and use the locally built emulator. Again, no need to pass 'local' to Emulator.run or the user to pass a command line argument.
This way the test case tells the test infra that it needs an emulator built locally instead of the user that starts the test.
Thoughts?
[snip]
support/testing/infra/emulator.py class Emulator(object): · line 42
Quoted 10 lines
support/testing/infra/emulator.py +5
-1@@ -30,13 +35,21 @@ class Emulator(object): # # options: array of command line options to pass to Qemu # - def boot(self, arch, kernel=None, kernel_cmdline=None, options=None): + # local: if True, the locally built qemu host tool is used instead of a + # qemu host tool found from the PATH. + # + def boot(self, arch, kernel=None, kernel_cmdline=None, options=None, + local=None):
A better default value is local=False
support/testing/infra/emulator.py class Emulator(object): · line 54
Quoted 13 lines
support/testing/infra/emulator.py +5
-1if arch in ["armv7", "armv5"]: qemu_arch = "arm" else: qemu_arch = arch - qemu_cmd = ["qemu-system-{}".format(qemu_arch), + qemu_bin = "qemu-system-{}".format(qemu_arch) + if self.local_emulator or local: + qemu_bin = os.path.join(self.builddir, 'host', 'bin', qemu_bin) + + qemu_cmd = [qemu_bin, "-serial", "stdio", "-display", "none"]
Regards,
Ricardo
PATCH v3 3/4 support/testing: test can use the locally generated qemu host tool
Hello Ricardo and Etienne,
Quoted 10 lines · On Sat, 30 Mar 2019 00:30:01 -0300 Ricardo Martincoski…
I am not strongly against it.
But let me ask:
Are you or someone else having trouble with any other test case when using "old" qemu versions from host?If the answer is no, maybe we should keep things simple and use a patch very similar to v2 of this one, just adding the host-qemu to the test case config fragment and the 'local' argument to Emulator.run.
I say similar to v2 because I think your implementation in v3 with the 'qemu_bin' variable is better than v2.
I totally agree here. Having this option --local-emulator doesn't make much sense. We would have to use it for some tests, but not for some other tests.
Instead, we want each test to know whether it needs a recent enough version of Qemu, or whether the system-provided Qemu version is good enough.
Quoted 7 lines
Another approach "to have an generic option to script run-tests to build the emulator within the test config" (quoting your reply to v2) would be:
Instead of adding a command line argument to run-tests, create a new flag in the test case class, perhaps named "build_emulator". If that flag is True, the host-qemu is automatically added to the config fragment by the test infra. The same flag could be propagated to the Emulator.init so we don't need to pass 'local' to Emulator.run.
Yes, something like that sounds good.
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v3 3/4 support/testing: test can use the locally generated qemu host tool
Hello Ricardo,
Hello Thomas,
Thanks for your feedback and sorry for this late answers.
Quoted 16 lines · Thomas Petazzoni
Hello Ricardo and Etienne,
Quoted 10 lines · On Sat, 30 Mar 2019 00:30:01 -0300 Ricardo Martincoski…
I am not strongly against it.
But let me ask:
Are you or someone else having trouble with any other test case when using "old" qemu versions from host?If the answer is no, maybe we should keep things simple and use a patch very similar to v2 of this one, just adding the host-qemu to the test case config fragment and the 'local' argument to Emulator.run.
I say similar to v2 because I think your implementation in v3 with the 'qemu_bin' variable is better than v2.
Up to my knowledge, when running qemu-system-arm without secure world emulation, old Qemu versions are fine (I.e. v2.x).
But when emulating arm w/ trustzone (qemu-system-arm -machine secure=on ...), one must use at least Qemu v2.10, and recommended version is v3.1.
- v2.10 is necessary, because of commit b29fd33db578 ("target/arm: use DISAS_EXIT for eret handling") that is mandated.
- v3.1 is recommended since commit fb23d693a3e0 ("hw/arm/virt: add DT property /secure-chosen/stdout-path indicating secure UART"). Without this fix, secure/non-secure world consoles are mixed resulting in weird traces (unless one fully disables secure world runtime traces).
Quoted 8 lines
I totally agree here. Having this option --local-emulator doesn't make much sense. We would have to use it for some tests, but not for some other tests.
Instead, we want each test to know whether it needs a recent enough version of Qemu, or whether the system-provided Qemu version is good enough.
Fair enough.
Quoted 8 lines
Quoted 8 lines
Another approach "to have an generic option to script run-tests to build the emulator within the test config" (quoting your reply to v2) would be:
Instead of adding a command line argument to run-tests, create a new flag in the test case class, perhaps named "build_emulator". If that flag is True, the host-qemu is automatically added to the config fragment by the test infra. The same flag could be propagated to the Emulator.init so we don't need to pass 'local' to Emulator.run.
Ok, I think I see your idea.
I'll come up with a v4 for this whole patch series.
Thanks again
etienne
Quoted 10 lines
Yes, something like that sounds good.
Best regards,
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
PATCH v3 4/4 support/testing: test_optee.py: test optee boot and testsuite
In reply to Etienne CarriereRun a Qemu emulation over qemu_armv7a_tz_virt_defconfig and run the embedded OP-TEE regression test suite (xtest).
Tool xtest dumps traces that contain '# ' (hash + space) which corrupts infra/emulator.py sequence which use such traces to find shell prompt when command is completed. To overcome the issue the xtest traces are shown only if the test failed.
One can run the test from something like:
$> support/testing/run-tests \
-o output/optee-runtest -d output/dwl \
tests.package.test_optee---
Changes v2 -> v3:
- Test provides a config_emulator attribute and mandates use of the locally built emulator.
- Update qemu defconfig filename.
- Remove postprocessing of test image file since driven from the selected defconfig
Changes v1 -> v2:
- Add argument local=True to test emulator to use the qemu host built from test configuration.
- Fix typo in trace "Silent test takes a while, be patient..."
--- support/testing/tests/package/test_optee.py | 42 +++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 support/testing/tests/package/test_optee.py
support/testing/tests/package/test_optee.py new file +42 -0
@@ -0,0 +1,42 @@ +import os + +import infra.basetest + +# This test enforces locally built emulator to prevent old Qemu to +# dump secure trace to stdio and corrupting trace synchro expected +# through pexpect. + +class TestOptee(infra.basetest.BRTest): + + with open(os.path.join(os.getcwd(), 'configs', + 'qemu_arm_vexpress_tz_defconfig'), + 'r') as config_file: + config = "".join(line for line in config_file if line[:1] != '#') + \ + """ + BR2_PACKAGE_HOST_QEMU=y + BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y + BR2_TOOLCHAIN_EXTERNAL=y + """ + config_emulator = '' + + def test_run(self): + + qemu_options = ['-machine', 'virt,secure=on'] + qemu_options.extend(['-cpu', 'cortex-a15']) + qemu_options.extend(['-m', '1024']) + qemu_options.extend(['-semihosting-config', 'enable,target=native']) + qemu_options.extend(['-bios', 'bl1.bin']) + + # This test expects Qemu is run from the image direcotry + os.chdir(os.path.join(self.builddir, 'images')) + + self.emulator.boot(arch='armv7', options=qemu_options, local=True) + self.emulator.login() + + # Trick traces since xtest prints "# " which corrupts emulator run + # method. Tests are dumped only if test fails. + cmd = 'echo "Silent test takes a while, be patient..."; ' + \ + 'xtest -t regression > /tmp/xtest.log ||' + \ + '(cat /tmp/xtest.log && false)' + output, exit_code = self.emulator.run(cmd, timeout=240) + self.assertEqual(exit_code, 0)
Signature · 1 line
2.17.1
PATCH v3 4/4 support/testing: test_optee.py: test optee boot and testsuite
Hello,
Looks good, except for the chdir stuff.
Also a lot of nits below.
On Fri, Mar 22, 2019 at 06:58 AM, Etienne Carriere wrote:
[snip]
Quoted 4 lines
--- support/testing/tests/package/test_optee.py | 42 +++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 support/testing/tests/package/test_optee.py
Please run flake8 and fix the warning it generates. test_optee.py:9:1: E302 expected 2 blank lines, found 1
Please run 'make .gitlab-ci.yml'. BTW the same is also missing in the defconfig patch 2.
Quoted 10 lines
support/testing/tests/package/test_optee.py new file +3
-0@@ -0,0 +1,42 @@ +import os + +import infra.basetest
support/testing/tests/package/test_optee.py line 13
Quoted 10 lines
support/testing/tests/package/test_optee.py new file +10
-0+ +# This test enforces locally built emulator to prevent old Qemu to +# dump secure trace to stdio and corrupting trace synchro expected +# through pexpect. + +class TestOptee(infra.basetest.BRTest): + + with open(os.path.join(os.getcwd(), 'configs', + 'qemu_arm_vexpress_tz_defconfig'), + 'r') as config_file:
Regarding the use of open(), I see no better option here.
Regarding the use of os.getcwd(), I would prefer to have my patch applied before this one:
http://patchwork.ozlabs.org/patch/992697/
So here you could use:
with open(infra.basepath('configs/qemu_armv7a_tz_virt_defconfig'), 'r') as config_file:
This way we would keep the logic to get any path in a single point in the test infra.
But this suggestion depends on what the maintainers think about my patch.
If you like the idea you can add my patch to your series.
support/testing/tests/package/test_optee.py line 15
Quoted 2 lines
support/testing/tests/package/test_optee.py new file +2
-0+ config = "".join(line for line in config_file if line[:1] != '#') + \ + """
nit: for consistence with all other test cases, please use one less indent:
config = "".join(line for line in config_file if line[:1] != '#') + \
"""support/testing/tests/package/test_optee.py line 17
Quoted 2 lines
support/testing/tests/package/test_optee.py new file +2
-0+ BR2_PACKAGE_HOST_QEMU=y + BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
Shouldn't these 2 lines be in the config_emulator below ...
support/testing/tests/package/test_optee.py line 20
Quoted 3 lines
support/testing/tests/package/test_optee.py new file +3
-0+ BR2_TOOLCHAIN_EXTERNAL=y + """ + config_emulator = ''
... like this?
config_emulator = \
"""
BR2_PACKAGE_HOST_QEMU=y
BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
"""But see my question on patch 3 before changing this.
support/testing/tests/package/test_optee.py line 30
Quoted 10 lines
support/testing/tests/package/test_optee.py new file +10
-0+ + def test_run(self): + + qemu_options = ['-machine', 'virt,secure=on'] + qemu_options.extend(['-cpu', 'cortex-a15']) + qemu_options.extend(['-m', '1024']) + qemu_options.extend(['-semihosting-config', 'enable,target=native']) + qemu_options.extend(['-bios', 'bl1.bin']) + + # This test expects Qemu is run from the image direcotry
s/direcotry/directory/
support/testing/tests/package/test_optee.py line 31
Quoted 1 line
support/testing/tests/package/test_optee.py new file +1
-0+ os.chdir(os.path.join(self.builddir, 'images'))
This is not good.
When running multiple test cases, it will make any test case that runs after this one to fail, see:
$ ./support/testing/run-tests -o o -k \
tests.core.test_post_scripts.TestPostScripts \
tests.package.test_optee.TestOptee
20:14:02 TestOptee Starting
20:17:01 TestOptee Cleaning up
.20:17:01 TestPostScripts Starting
E
======================================================================
ERROR: test_run (tests.core.test_post_scripts.TestPostScripts)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/ricardo/src/buildroot/support/testing/infra/basetest.py", line 76, in setUp
super(BRTest, self).setUp()
File "/home/ricardo/src/buildroot/support/testing/infra/basetest.py", line 62, in setUp
self.b.configure(make_extra_opts=["BR2_EXTERNAL={}".format(":".join(self.br2_external))])
File "/home/ricardo/src/buildroot/support/testing/infra/builder.py", line 48, in configure
raise SystemError("Cannot olddefconfig")
SystemError: Cannot olddefconfigA better solution is to add a new optional parameter to Emulator.boot method, perhaps named "cwd" that defaults to None and is passed to pexpect.spawn(..., cwd=cwd) (not tested).
This way the Emulator class is aware that qemu must run from a directory, only for the test cases that pass this.
support/testing/tests/package/test_optee.py line 41
Quoted 10 lines
support/testing/tests/package/test_optee.py new file +10
-0+ + self.emulator.boot(arch='armv7', options=qemu_options, local=True) + self.emulator.login() + + # Trick traces since xtest prints "# " which corrupts emulator run + # method. Tests are dumped only if test fails. + cmd = 'echo "Silent test takes a while, be patient..."; ' + \ + 'xtest -t regression > /tmp/xtest.log ||' + \ + '(cat /tmp/xtest.log && false)' + output, exit_code = self.emulator.run(cmd, timeout=240)
'output' is currently not tested.
It's better to use the convention:
_, exit_code = self.emulator.run(cmd, timeout=240)
Quoted 3 lines
support/testing/tests/package/test_optee.py new file +1
-0+ self.assertEqual(exit_code, 0)Signature · 1 line
2.17.1
Regards,
Ricardo
PATCH v3 4/4 support/testing: test_optee.py: test optee boot and testsuite
Hello Ricardo,
Quoted 8 lines · Ricardo Martincoski
Hello,
Looks good, except for the chdir stuff.
Also a lot of nits below.On Fri, Mar 22, 2019 at 06:58 AM, Etienne Carriere wrote:
Quoted 8 lines
[snip]
Quoted 4 lines
--- support/testing/tests/package/test_optee.py | 42 +++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 support/testing/tests/package/test_optee.pyPlease run flake8 and fix the warning it generates. test_optee.py:9:1: E302 expected 2 blank lines, found 1
Sure, sorry. Python newbie bad habit.
Quoted 3 lines
Please run 'make .gitlab-ci.yml'. BTW the same is also missing in the defconfig patch 2.
Ok.
Quoted 24 lines
Quoted 11 lines
support/testing/tests/package/test_optee.py new file +3
-0@@ -0,0 +1,42 @@ +import os + +import infra.basetestsupport/testing/tests/package/test_optee.py line 13
Quoted 10 lines
support/testing/tests/package/test_optee.py new file +10
-0+ +# This test enforces locally built emulator to prevent old Qemu to +# dump secure trace to stdio and corrupting trace synchro expected +# through pexpect. + +class TestOptee(infra.basetest.BRTest): + + with open(os.path.join(os.getcwd(), 'configs', + 'qemu_arm_vexpress_tz_defconfig'), + 'r') as config_file:Regarding the use of
open(), I see no better option here.
Quoted 9 lines
Regarding the use of
os.getcwd(), I would prefer to have my patch applied before this one:
http://patchwork.ozlabs.org/patch/992697/
So here you could use:
withopen(infra.basepath('configs/qemu_armv7a_tz_virt_defconfig'), 'r')asconfig_file:
This way we would keep the logic to get any path in a single point in the test infra.
But this suggestion depends on what the maintainers think about my patch.
If you like the idea you can add my patch to your series.
Ok, thanks, it will be more consistent with the runtime env. Maybe I could simply merge your proposal in my change (and append your s-o-b tag to it), if you agree.
Quoted 7 lines
support/testing/tests/package/test_optee.py line 15
Quoted 3 lines
support/testing/tests/package/test_optee.py new file +2
-0+ config = "".join(line for line in config_file if line[:1] != '#') + \ + """nit: for consistence with all other test cases, please use one less indent:
config = "".join(line for line in config_file if line[:1] != '#') + \ """
ok.
Quoted 10 lines
support/testing/tests/package/test_optee.py line 17
Quoted 3 lines
support/testing/tests/package/test_optee.py new file +2
-0+ BR2_PACKAGE_HOST_QEMU=y + BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=yShouldn't these 2 lines be in the
config_emulatorbelow ...support/testing/tests/package/test_optee.py line 20
Quoted 3 lines
support/testing/tests/package/test_optee.py new file +3
-0+ BR2_TOOLCHAIN_EXTERNAL=y + """ + config_emulator = ''
Quoted 6 lines
... like this?
config_emulator = \ """ BR2_PACKAGE_HOST_QEMU=y BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y """
yes. true.
Quoted 2 lines
But see my question on patch 3 before changing this.
true again, I think I will remove this config_emulator from the test case and move it to a generic config in basetest.py.
Quoted 13 lines
support/testing/tests/package/test_optee.py line 30
Quoted 11 lines
support/testing/tests/package/test_optee.py new file +10
-0+ + def test_run(self): + + qemu_options = ['-machine', 'virt,secure=on'] + qemu_options.extend(['-cpu', 'cortex-a15']) + qemu_options.extend(['-m', '1024']) + qemu_options.extend(['-semihosting-config', 'enable,target=native']) + qemu_options.extend(['-bios', 'bl1.bin']) + + # This test expects Qemu is run from the image direcotrys/direcotry/directory/
thanks.
Quoted 26 lines
support/testing/tests/package/test_optee.py line 31
Quoted 2 lines
support/testing/tests/package/test_optee.py new file +1
-0+ os.chdir(os.path.join(self.builddir, 'images'))This is not good.
When running multiple test cases, it will make any test case that runs after this one to fail, see:$ ./support/testing/run-tests -o o -k \ tests.core.test_post_scripts.TestPostScripts \ tests.package.test_optee.TestOptee 20:14:02 TestOptee Starting 20:17:01 TestOptee Cleaning up .20:17:01 TestPostScripts Starting E ====================================================================== ERROR: test_run (tests.core.test_post_scripts.TestPostScripts) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/ricardo/src/buildroot/support/testing/infra/basetest.py", line 76, in setUp super(BRTest, self).setUp() File "/home/ricardo/src/buildroot/support/testing/infra/basetest.py", line 62, in setUp self.b.configure(make_extra_opts=["BR2_EXTERNAL={}".format(":".join(self.br2_external))]) File "/home/ricardo/src/buildroot/support/testing/infra/builder.py", line 48, in configure raise SystemError("Cannot olddefconfig") SystemError: Cannot olddefconfig
Quoted 5 lines
A better solution is to add a new optional parameter to Emulator.boot method, perhaps named "cwd" that defaults to None and is passed to
pexpect.spawn(..., cwd=cwd)(not tested).
This way the Emulator class is aware that qemu must run from a directory, only for the test cases that pass this.
Ok. Indeed my proposal is bad.
I will go this way, and try to come up with something not too ugly.
Quoted 14 lines
support/testing/tests/package/test_optee.py line 41
Quoted 11 lines
support/testing/tests/package/test_optee.py new file +10
-0+ + self.emulator.boot(arch='armv7', options=qemu_options, local=True) + self.emulator.login() + + # Trick traces since xtest prints "# " which corrupts emulator run + # method. Tests are dumped only if test fails. + cmd = 'echo "Silent test takes a while, be patient..."; ' + \ + 'xtest -t regression > /tmp/xtest.log ||' + \ + '(cat /tmp/xtest.log && false)' + output, exit_code = self.emulator.run(cmd, timeout=240)'output' is currently not tested.
It's better to use the convention:
Quoted 1 line
_, exit_code = self.emulator.run(cmd, timeout=240)
Ok thanks, will fix.
Quoted 8 lines
Quoted 4 lines
support/testing/tests/package/test_optee.py new file +1
-1+ self.assertEqual(exit_code, 0) --2.17.1
Regards,
Ricardo
Thanks again for your feedback.
Regards,
etienne
PATCH v3 4/4 support/testing: test_optee.py: test optee boot and testsuite
Hello,
On Wed, Apr 03, 2019 at 07:19 AM, Etienne Carriere wrote:
[snip]
Quoted 18 lines
Quoted 8 lines
support/testing/tests/package/test_optee.py line 13
Quoted 5 lines
support/testing/tests/package/test_optee.py new file +5
-0+class TestOptee(infra.basetest.BRTest): + + with open(os.path.join(os.getcwd(), 'configs', + 'qemu_arm_vexpress_tz_defconfig'), + 'r') as config_file:Regarding the use of
open(), I see no better option here.Quoted 9 lines
Regarding the use of
os.getcwd(), I would prefer to have my patch applied before this one:
http://patchwork.ozlabs.org/patch/992697/
So here you could use:
withopen(infra.basepath('configs/qemu_armv7a_tz_virt_defconfig'), 'r')asconfig_file:
This way we would keep the logic to get any path in a single point in the test infra.
But this suggestion depends on what the maintainers think about my patch.
If you like the idea you can add my patch to your series.
Quoted 3 lines
Ok, thanks, it will be more consistent with the runtime env. Maybe I could simply merge your proposal in my change (and append your s-o-b tag to it), if you agree.
I prefer my patch to stay as a separate commit because it changes how run-tests can be called. Currently we can chdir to one buildroot tree and call run-tests from another buildroot tree. Currently we also need to be in the buildroot top dir when we call run-tests. After that patch when run-tests is called it will execute in (and consequently test) the buildroot tree it belongs to, no matter which is the current directory when run-tests is called.
That patch can still be applied with 'git am -3'.
But feel free to just grab only part of the code from there if you want to. No need for my SoB if you use only part of it. Maybe something like this is enough:
def basepath(relpath=""):
return os.path.join(os.getcwd(), relpath)
This way we have all calls to os.getcwd() in support/testing/infra/__init__.py and I resend my patch later.
But also feel free to not do this right now. Up to you.
Regards,
Ricardo
PATCH v3 4/4 support/testing: test_optee.py: test optee boot and testsuite
Quoted 24 lines · Ricardo Martincoski
Hello,
On Wed, Apr 03, 2019 at 07:19 AM, Etienne Carriere wrote:
[snip]
Quoted 18 lines
Quoted 17 lines
support/testing/tests/package/test_optee.py line 13
Quoted 5 lines
support/testing/tests/package/test_optee.py new file +5
-0+class TestOptee(infra.basetest.BRTest): + + with open(os.path.join(os.getcwd(), 'configs', + 'qemu_arm_vexpress_tz_defconfig'), + 'r') as config_file:Regarding the use of
open(), I see no better option here.Regarding the use of
os.getcwd(), I would prefer to have my patch applied before this one:
http://patchwork.ozlabs.org/patch/992697/
So here you could use:
withopen(infra.basepath('configs/qemu_armv7a_tz_virt_defconfig'), 'r')asconfig_file:
This way we would keep the logic to get any path in a single point in the test infra.
But this suggestion depends on what the maintainers think about my patch.
If you like the idea you can add my patch to your series.
Quoted 6 lines
Quoted 3 lines
Ok, thanks, it will be more consistent with the runtime env. Maybe I could simply merge your proposal in my change (and append your s-o-b tag to it), if you agree.
I prefer my patch to stay as a separate commit because it changes how run-tests can be called. (...)
Oh, yes, sorry! I misunderstood your previous comment. Sure, I won't merge your change and mine :)
Thanks for the clarification. I'll focus on this cwd() issue you suggest and o which extend I need to take your proposal into account in my change.
Regards,
etienne
Quoted 21 lines
(...) Currently we can chdir to one buildroot tree and call run-tests from another buildroot tree. Currently we also need to be in the buildroot top dir when we call run-tests. After that patch when run-tests is called it will execute in (and consequently test) the buildroot tree it belongs to, no matter which is the current directory when run-tests is called.
That patch can still be applied with 'git am -3'.But feel free to just grab only part of the code from there if you want to. No need for my SoB if you use only part of it. Maybe something like this is enough:
def
basepath(relpath=""):
returnos.path.join(os.getcwd(), relpath)This way we have all calls to
os.getcwd()insupport/testing/infra/__init__.pyand I resend my patch later.But also feel free to not do this right now. Up to you.
Regards,
Ricardo
PATCH v3 1/4 boot/arm-trusted-firmware: support alternate image files
In reply to Etienne CarriereQuoted 2 lines · Etienne Carriere
Some platform may generate specific boot image files instead of the generic files *.bin when building TF-A package.
This is still very vague, and it doesn't have any in-tree user. Therefore,
we're not prepared to merge this patch. Do you have a concrete example, preferably with a defconfig that actually makes use of it?
I've therefore marked this patch as Changes Requested.
Regards,
Arnout
Quoted 10 lines
This change
introduces new configuration directive for the arm-trusted-firmware boot package.
BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_IMAGE_NAMESis boolean. When disabled, install boot image files are .../*.bin. When enabled,BR2_TARGET_ARM_TRUSTED_FIRMWARE_BL_IMAGE_NAMESshall lists the names of the generated boot image files.Signed-off-by Etienne Carriere <[email protected]>
[snip]
PATCH v3 1/4 boot/arm-trusted-firmware: support alternate image files
Hello Thomans and all,
Quoted 10 lines · Arnout Vandecappelle
Quoted 2 lines · Etienne Carriere
Some platform may generate specific boot image files instead of the generic files *.bin when building TF-A package.
This is still very vague, and it doesn't have any in-tree user. Therefore,
we're not prepared to merge this patch. Do you have a concrete example, preferably with a defconfig that actually makes use of it?
Well, I will consider when there is an in-tree board that needs this :)
Actually, this is some kind of equivalent for optee-os BR2_TARGET_OPTEE_OS_IMAGE_NAMES
or even u-boot BR2_TARGET_UBOOT_FORMAT_CUSTOM.
Regards,
Etienne
Quoted 16 lines
I've therefore marked this patch as Changes Requested.
Regards,
ArnoutQuoted 10 lines
This change
introduces new configuration directive for the arm-trusted-firmware boot package.
BR2_TARGET_ARM_TRUSTED_FIRMWARE_CUSTOM_IMAGE_NAMESis boolean. When disabled, install boot image files are .../*.bin. When enabled,BR2_TARGET_ARM_TRUSTED_FIRMWARE_BL_IMAGE_NAMESshall lists the names of the generated boot image files.Signed-off-by Etienne Carriere <[email protected]>[snip]