Skip to contentSkip to conversation

Create a test job for testing static memory on qemu

PATCH Superseded Xenia Ragiadakou 5 participants 19 messages Jul 7, 2022 – Jul 12, 2022

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover Create a test job for testing static memory on qemu

This patch series
- removes all the references to the XEN_CONFIG_EXPERT environmental variable which is not used anymore
- creates a trivial arm64 test job that boots xen on qemu with a direct mapped dom0less domu with static memory and verifies, based on its logs, that the domu's memory node ranges are the same as the static memory ranges with which it was configured

Xenia Ragiadakou (2):
automation: Remove XEN_CONFIG_EXPERT leftovers
automation: arm64: Create a test job for testing static allocation on
qemu

 automation/build/README.md                 |   3 -
 automation/configs/arm/static_mem          |   3 +
 automation/gitlab-ci/test.yaml             |  24 +++++
 automation/scripts/build                   |   8 +-
 automation/scripts/containerize            |   1 -
 automation/scripts/qemu-staticmem-arm64.sh | 114 +++++++++++++++++++++
 6 files changed, 147 insertions(+), 6 deletions(-)
 create mode 100644 automation/configs/arm/static_mem
 create mode 100755 automation/scripts/qemu-staticmem-arm64.sh
Signature · 1 line
2.34.1
#1 lore raw

PATCH 1/2 automation: Remove XEN_CONFIG_EXPERT leftovers

The EXPERT config option cannot anymore be selected via the environmental variable XEN_CONFIG_EXPERT. Remove stale references to XEN_CONFIG_EXPERT from the automation code.

---
 automation/build/README.md      | 3 ---
 automation/scripts/build        | 4 ++--
 automation/scripts/containerize | 1 -
 3 files changed, 2 insertions(+), 6 deletions(-)
automation/build/README.md +0 -3
index 2137957408..00305eed03 100644
@@ -65,9 +65,6 @@ understands.
 - CONTAINER_NO_PULL: If set to 1, the script will not pull from docker hub.
   This is useful when testing container locally.
 
-- XEN_CONFIG_EXPERT: If this is defined in your shell it will be
-  automatically passed through to the container.
-
 If your docker host has Linux kernel > 4.11, and you want to use containers
 that run old glibc (for example, CentOS 6 or SLES11SP4), you may need to add
 
automation/scripts/build +2 -2
index 281f8b1fcc..21b3bc57c8 100755
@@ -91,6 +91,6 @@ for cfg in `ls ${cfg_dir}`; do
     echo "Building $cfg"
     make -j$(nproc) -C xen clean
     rm -f xen/.config
-    make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} XEN_CONFIG_EXPERT=y defconfig
-    make -j$(nproc) -C xen XEN_CONFIG_EXPERT=y
+    make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} defconfig
+    make -j$(nproc) -C xen
 done
automation/scripts/containerize +0 -1
index 8992c67278..9d4beca4fa 100755
@@ -101,7 +101,6 @@ exec ${docker_cmd} run \
     -v "${CONTAINER_PATH}":/build:rw${selinux} \
     -v "${HOME}/.ssh":/root/.ssh:ro \
     ${SSH_AUTH_DIR:+-v "${SSH_AUTH_DIR}":/tmp/ssh-agent${selinux}} \
-    ${XEN_CONFIG_EXPERT:+-e XEN_CONFIG_EXPERT=${XEN_CONFIG_EXPERT}} \
     ${CONTAINER_ARGS} \
     -${termint}i --rm -- \
     ${CONTAINER} \
Signature · 1 line
2.34.1
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] automation: Remove XEN_CONFIG_EXPERT leftovers

Quoted 5 lines · Xenia Ragiadakou

The EXPERT config option cannot anymore be selected via the environmental variable XEN_CONFIG_EXPERT. Remove stale references to XEN_CONFIG_EXPERT from the automation code.

Quoted 48 lines
---
 automation/build/README.md      | 3 ---
 automation/scripts/build        | 4 ++--
 automation/scripts/containerize | 1 -
 3 files changed, 2 insertions(+), 6 deletions(-)
automation/build/README.md +0 -3
index 2137957408..00305eed03 100644
@@ -65,9 +65,6 @@ understands.
 - CONTAINER_NO_PULL: If set to 1, the script will not pull from docker hub.
   This is useful when testing container locally.
 
-- XEN_CONFIG_EXPERT: If this is defined in your shell it will be
-  automatically passed through to the container.
-
 If your docker host has Linux kernel > 4.11, and you want to use containers
 that run old glibc (for example, CentOS 6 or SLES11SP4), you may need to add
 
automation/scripts/build +2 -2
index 281f8b1fcc..21b3bc57c8 100755
@@ -91,6 +91,6 @@ for cfg in `ls ${cfg_dir}`; do
     echo "Building $cfg"
     make -j$(nproc) -C xen clean
     rm -f xen/.config
-    make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} XEN_CONFIG_EXPERT=y defconfig
-    make -j$(nproc) -C xen XEN_CONFIG_EXPERT=y
+    make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} defconfig
+    make -j$(nproc) -C xen
 done
automation/scripts/containerize +0 -1
index 8992c67278..9d4beca4fa 100755
@@ -101,7 +101,6 @@ exec ${docker_cmd} run \
     -v "${CONTAINER_PATH}":/build:rw${selinux} \
     -v "${HOME}/.ssh":/root/.ssh:ro \
     ${SSH_AUTH_DIR:+-v "${SSH_AUTH_DIR}":/tmp/ssh-agent${selinux}} \
-    ${XEN_CONFIG_EXPERT:+-e XEN_CONFIG_EXPERT=${XEN_CONFIG_EXPERT}} \
     ${CONTAINER_ARGS} \
     -${termint}i --rm -- \
     ${CONTAINER} \
Signature · 1 line
2.34.1
#3 lore raw

PATCH 2/2 automation: arm64: Create a test job for testing static allocation on qemu

In reply to Xenia Ragiadakou

Add an arm subdirectory under automation/configs for the arm specific configs and add a config that enables static allocation.

Modify the build script to search for configs also in this subdirectory and to keep the generated xen binary, suffixed with the config file name, as artifact.

Create a test job that
- boots xen on qemu with a single direct mapped dom0less guest configured with statically allocated memory
- verifies that the memory ranges reported in the guest's logs are the same with the provided static memory regions

For guest kernel, use the 5.9.9 kernel from the tests-artifacts containers. Use busybox-static package, to create the guest ramdisk.
To generate the u-boot script, use ImageBuilder.
Use the qemu from the tests-artifacts containers.

---
 automation/configs/arm/static_mem          |   3 +
 automation/gitlab-ci/test.yaml             |  24 +++++
 automation/scripts/build                   |   4 +
 automation/scripts/qemu-staticmem-arm64.sh | 114 +++++++++++++++++++++
 4 files changed, 145 insertions(+)
 create mode 100644 automation/configs/arm/static_mem
 create mode 100755 automation/scripts/qemu-staticmem-arm64.sh
automation/configs/arm/static_mem new file +3 -0
new file mode 100644
index 0000000000..84675ddf4e
@@ -0,0 +1,3 @@
+CONFIG_EXPERT=y
+CONFIG_UNSUPPORTED=y
+CONFIG_STATIC_MEMORY=y
\ No newline at end of file
automation/gitlab-ci/test.yaml +24 -0
index 42cd725a12..dc181f3777 100644
@@ -205,3 +205,27 @@ qemu-smoke-x86-64-clang-pvh:
     - smoke
     - /^coverity-tested\/.*/
     - /^stable-.*/
+
+qemu-staticmem-arm64-gcc:
+  stage: test
+  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
+  variables:
+    CONTAINER: debian:unstable-arm64v8
+  script:
+    - ./automation/scripts/qemu-staticmem-arm64.sh 2>&1 | tee qemu-staticmem-arm64.log
+  dependencies:
+    - debian-unstable-gcc-arm64
+    - kernel-5.9.9-arm64-export
+    - qemu-system-aarch64-6.0.0-arm64-export
+  artifacts:
+    paths:
+      - qemu-staticmem.serial
+      - '*.log'
+    when: always
+  tags:
+    - arm64
+  except:
+    - master
+    - smoke
+    - /^coverity-tested\/.*/
+    - /^stable-.*/
automation/scripts/build +4 -0
index 21b3bc57c8..9c6196d9bd 100755
@@ -83,6 +83,7 @@ fi
 # Build all the configs we care about
 case ${XEN_TARGET_ARCH} in
     x86_64) arch=x86 ;;
+    arm64) arch=arm ;;
     *) exit 0 ;;
 esac
 
@@ -93,4 +94,7 @@ for cfg in `ls ${cfg_dir}`; do
     rm -f xen/.config
     make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} defconfig
     make -j$(nproc) -C xen
+    if [[ ${arch} == "arm" ]]; then
+        cp xen/xen binaries/xen-${cfg}
+    fi
 done
automation/scripts/qemu-staticmem-arm64.sh new file +114 -0

114 lines not on this page · raw message

Signature · 1 line
2.34.1
#4 lore raw

Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Hi Xenia,

Quoted 33 lines · Xenia Ragiadakou

Add an arm subdirectory under automation/configs for the arm specific configs and add a config that enables static allocation.

Modify the build script to search for configs also in this subdirectory and to keep the generated xen binary, suffixed with the config file name, as artifact.

Create a test job that
- boots xen on qemu with a single direct mapped dom0less guest configured with statically allocated memory
- verifies that the memory ranges reported in the guest's logs are the same with the provided static memory regions

For guest kernel, use the 5.9.9 kernel from the tests-artifacts containers. Use busybox-static package, to create the guest ramdisk.
To generate the u-boot script, use ImageBuilder.
Use the qemu from the tests-artifacts containers.

---
  automation/configs/arm/static_mem          |   3 +
  automation/gitlab-ci/test.yaml             |  24 +++++
  automation/scripts/build                   |   4 +
  automation/scripts/qemu-staticmem-arm64.sh | 114 +++++++++++++++++++++
  4 files changed, 145 insertions(+)

create mode 100644 automation/configs/arm/static_mem create mode 100755 automation/scripts/qemu-staticmem-arm64.sh

automation/configs/arm/static_mem new file +0 -0
new file mode 100644
index 0000000000..84675ddf4e

automation/configs/arm/static_mem line 3

Quoted 4 lines
automation/configs/arm/static_mem new file +3 -0
@@ -0,0 +1,3 @@
+CONFIG_EXPERT=y
+CONFIG_UNSUPPORTED=y
+CONFIG_STATIC_MEMORY=y
\ No newline at end of file

Any particular reason to build a new Xen rather enable CONFIG_STATIC_MEMORY in the existing build?

Quoted 13 lines
automation/scripts/build +1 -0
index 21b3bc57c8..9c6196d9bd 100755
@@ -83,6 +83,7 @@ fi
  # Build all the configs we care about
  case ${XEN_TARGET_ARCH} in
      x86_64) arch=x86 ;;
+    arm64) arch=arm ;;
      *) exit 0 ;;
  esac
 

automation/scripts/build for cfg in `ls ${cfg_dir}`; do · line 99

Quoted 6 lines
automation/scripts/build +3 -0
@@ -93,4 +94,7 @@ for cfg in `ls ${cfg_dir}`; do
      rm -f xen/.config
      make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} defconfig
      make -j$(nproc) -C xen
+    if [[ ${arch} == "arm" ]]; then
+        cp xen/xen binaries/xen-${cfg}
+    fi

This feels a bit of a hack to be arm only. Can you explain why this is not enabled for x86 (other than this is not yet used)?

automation/scripts/qemu-staticmem-arm64.sh line 4

Quoted 11 lines
automation/scripts/build +0 -0
  done
automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
new file mode 100755
index 0000000000..5b89a151aa
@@ -0,0 +1,114 @@
+#!/bin/bash
+
+base=(0x50000000 0x100000000)
+size=(0x10000000 0x10000000)

From the name, it is not clear what the base and size refers too. Looking a bit below, it seems to be referring to the domain memory. If so, I would suggest to comment and rename to "domu_{base, size}".

Quoted 51 lines
automation/scripts/qemu-staticmem-arm64.sh new file +51 -0
+
+set -ex
+
+apt-get -qy update
+apt-get -qy install --no-install-recommends u-boot-qemu \
+                                            u-boot-tools \
+                                            device-tree-compiler \
+                                            cpio \
+                                            curl \
+                                            busybox-static
+
+# DomU Busybox
+cd binaries
+mkdir -p initrd
+mkdir -p initrd/bin
+mkdir -p initrd/sbin
+mkdir -p initrd/etc
+mkdir -p initrd/dev
+mkdir -p initrd/proc
+mkdir -p initrd/sys
+mkdir -p initrd/lib
+mkdir -p initrd/var
+mkdir -p initrd/mnt
+cp /bin/busybox initrd/bin/busybox
+initrd/bin/busybox --install initrd/bin
+echo "#!/bin/sh
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+mount -t devtmpfs devtmpfs /dev
+/bin/sh" > initrd/init
+chmod +x initrd/init
+cd initrd
+find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+cd ../..
+
+# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
+curl -fsSLO https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
+
+./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -M dumpdtb=binaries/virt-gicv2.dtb
+
+#dtc -I dtb -O dts binaries/virt-gicv2.dtb > binaries/virt-gicv2.dts
+
+# ImageBuilder

automation/scripts/qemu-staticmem-arm64.sh line 65

Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+
+echo "MEMORY_START=\"0x40000000\"
+MEMORY_END=\"0x0200000000\"
+
+DEVICE_TREE=\"virt-gicv2.dtb\"
+
+XEN=\"xen-static_mem\"
+XEN_CMD=\"console=dtuart earlyprintk xsm=dummy\"

AFAIK, earlyprintk is not an option for Xen on Arm (at least). It is also not clear why you need to pass xsm=dummy.

Quoted 22 lines
automation/scripts/qemu-staticmem-arm64.sh new file +22 -0
+
+NUM_DOMUS=1
+DOMU_MEM[0]=512
+DOMU_VCPUS[0]=1
+DOMU_KERNEL[0]=\"Image\"
+DOMU_RAMDISK[0]=\"initrd.cpio.gz\"
+DOMU_CMD[0]=\"earlyprintk console=ttyAMA0\"
+DOMU_STATIC_MEM[0]=\"${base[0]} ${size[0]} ${base[1]} ${size[1]}\"
+
+UBOOT_SOURCE=\"boot.source\"
+UBOOT_SCRIPT=\"boot.scr\"" > binaries/imagebuilder_config
+
+bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/imagebuilder_config
+
+# Run the test
+rm -f qemu-staticmem.serial
+set +e
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"| \
+timeout -k 1 60 ./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \

automation/scripts/qemu-staticmem-arm64.sh line 97

Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -no-reboot \
+    -device virtio-net-pci,netdev=vnet0 -netdev user,id=vnet0,tftp=binaries \
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin \
+    -dtb ./binaries/virt-gicv2.dtb \
+    |& tee qemu-staticmem.serial
+
+set -e

A lot of the code above is duplicated from qemu-smoke-arm64.sh. I think it would be better to consolidate in a single script. Looking briefly throught the existing scripts, it looks like it is possible to pass arguments (see qemu-smoke-x86-64.sh).

automation/scripts/qemu-staticmem-arm64.sh line 110

Quoted 13 lines
automation/scripts/qemu-staticmem-arm64.sh new file +13 -0
+
+(grep -q "Xen dom0less mode detected" qemu-staticmem.serial) || exit 1
+
+for ((i=0; i<${#base[@]}; i++))
+do
+    start="$(printf "0x%016x" ${base[$i]})"
+    end="$(printf "0x%016x" $((${base[$i]} + ${size[$i]} - 1)))"
+    grep -q "node   0: \[mem ${start}-${end}\]" qemu-staticmem.serial
+    if test $? -eq 1
+    then
+        exit 1
+    fi
+done

Please add a comment on top to explain what this is meant to do. However, I think we should avoid relying on output that we have written ourself. IOW, relying on Xen/Linux to always write the same message is risky because they can change at any time.

automation/scripts/qemu-staticmem-arm64.sh line 112

Quoted 2 lines
automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+
+(grep -q "BusyBox" qemu-staticmem.serial) || exit 1

Cheers,

Signature · 1 line
Julien Grall
#5 lore raw

Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Quoted 184 lines · Julien Grall

Hi Xenia,

Quoted 38 lines · Xenia Ragiadakou

Add an arm subdirectory under automation/configs for the arm specific configs
and add a config that enables static allocation.

Modify the build script to search for configs also in this subdirectory and to
keep the generated xen binary, suffixed with the config file name, as artifact.

Create a test job that
- boots xen on qemu with a single direct mapped dom0less guest configured with
statically allocated memory
- verifies that the memory ranges reported in the guest's logs are the same with the provided static memory regions

For guest kernel, use the 5.9.9 kernel from the tests-artifacts containers. Use busybox-static package, to create the guest ramdisk.
To generate the u-boot script, use ImageBuilder.
Use the qemu from the tests-artifacts containers.

---
  automation/configs/arm/static_mem          |   3 +
  automation/gitlab-ci/test.yaml             |  24 +++++
  automation/scripts/build                   |   4 +
  automation/scripts/qemu-staticmem-arm64.sh | 114 +++++++++++++++++++++
  4 files changed, 145 insertions(+)

create mode 100644 automation/configs/arm/static_mem create mode 100755 automation/scripts/qemu-staticmem-arm64.sh

automation/configs/arm/static_mem new file +0 -0
new file mode 100644
index 0000000000..84675ddf4e

automation/configs/arm/static_mem line 3

Quoted 4 lines
automation/configs/arm/static_mem new file +3 -0
@@ -0,0 +1,3 @@
+CONFIG_EXPERT=y
+CONFIG_UNSUPPORTED=y
+CONFIG_STATIC_MEMORY=y
\ No newline at end of file

Any particular reason to build a new Xen rather enable CONFIG_STATIC_MEMORY in the existing build?

Quoted 15 lines
automation/scripts/build +1 -0
index 21b3bc57c8..9c6196d9bd 100755
@@ -83,6 +83,7 @@ fi
  # Build all the configs we care about
  case ${XEN_TARGET_ARCH} in
      x86_64) arch=x86 ;;
+    arm64) arch=arm ;;
      *) exit 0 ;;
  esac
  @@ -93,4 +94,7 @@ for cfg in `ls ${cfg_dir}`; do
      rm -f xen/.config
      make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} defconfig
      make -j$(nproc) -C xen

automation/scripts/build for cfg in `ls ${cfg_dir}`; do · line 99

Quoted 3 lines
automation/scripts/build +3 -0
+    if [[ ${arch} == "arm" ]]; then
+        cp xen/xen binaries/xen-${cfg}
+    fi

This feels a bit of a hack to be arm only. Can you explain why this is not enabled for x86 (other than this is not yet used)?

automation/scripts/qemu-staticmem-arm64.sh line 4

Quoted 12 lines

done

automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
new file mode 100755
index 0000000000..5b89a151aa
@@ -0,0 +1,114 @@
+#!/bin/bash
+
+base=(0x50000000 0x100000000)
+size=(0x10000000 0x10000000)

From the name, it is not clear what the base and size refers too. Looking a bit below, it seems to be referring to the domain memory. If so, I would suggest to comment and rename to "domu_{base, size}".

Quoted 51 lines
automation/scripts/qemu-staticmem-arm64.sh new file +51 -0
+
+set -ex
+
+apt-get -qy update
+apt-get -qy install --no-install-recommends u-boot-qemu \
+                                            u-boot-tools \
+                                            device-tree-compiler \
+                                            cpio \
+                                            curl \
+                                            busybox-static
+
+# DomU Busybox
+cd binaries
+mkdir -p initrd
+mkdir -p initrd/bin
+mkdir -p initrd/sbin
+mkdir -p initrd/etc
+mkdir -p initrd/dev
+mkdir -p initrd/proc
+mkdir -p initrd/sys
+mkdir -p initrd/lib
+mkdir -p initrd/var
+mkdir -p initrd/mnt
+cp /bin/busybox initrd/bin/busybox
+initrd/bin/busybox --install initrd/bin
+echo "#!/bin/sh
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+mount -t devtmpfs devtmpfs /dev
+/bin/sh" > initrd/init
+chmod +x initrd/init
+cd initrd
+find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+cd ../..
+
+# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
+curl -fsSLO https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
+
+./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -M dumpdtb=binaries/virt-gicv2.dtb
+
+#dtc -I dtb -O dts binaries/virt-gicv2.dtb > binaries/virt-gicv2.dts
+
+# ImageBuilder

automation/scripts/qemu-staticmem-arm64.sh line 65

Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+
+echo "MEMORY_START=\"0x40000000\"
+MEMORY_END=\"0x0200000000\"
+
+DEVICE_TREE=\"virt-gicv2.dtb\"
+
+XEN=\"xen-static_mem\"
+XEN_CMD=\"console=dtuart earlyprintk xsm=dummy\"

AFAIK, earlyprintk is not an option for Xen on Arm (at least). It is also not clear why you need to pass xsm=dummy.

Quoted 31 lines
automation/scripts/qemu-staticmem-arm64.sh new file +13 -0
+
+NUM_DOMUS=1
+DOMU_MEM[0]=512
+DOMU_VCPUS[0]=1
+DOMU_KERNEL[0]=\"Image\"
+DOMU_RAMDISK[0]=\"initrd.cpio.gz\"
+DOMU_CMD[0]=\"earlyprintk console=ttyAMA0\"
+DOMU_STATIC_MEM[0]=\"${base[0]} ${size[0]} ${base[1]} ${size[1]}\"
+
+UBOOT_SOURCE=\"boot.source\"
+UBOOT_SCRIPT=\"boot.scr\"" > binaries/imagebuilder_config
+
+bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c

binaries/imagebuilder_config

automation/scripts/qemu-staticmem-arm64.sh new file +14 -0
+
+# Run the test
+rm -f qemu-staticmem.serial
+set +e
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"| \
+timeout -k 1 60 ./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -no-reboot \
+    -device virtio-net-pci,netdev=vnet0 -netdev user,id=vnet0,tftp=binaries

\

automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin \
+    -dtb ./binaries/virt-gicv2.dtb \
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh line 97

Quoted 3 lines
automation/scripts/qemu-staticmem-arm64.sh new file +3 -0
+    |& tee qemu-staticmem.serial
+
+set -e

A lot of the code above is duplicated from qemu-smoke-arm64.sh. I think it would be better to consolidate in a single script. Looking briefly throught the existing scripts, it looks like it is possible to pass arguments (see qemu-smoke-x86-64.sh).

One idea would be to make the script common and "source" a second script or config file with just the ImageBuilder configuration because it looks like it is the only thing different.

Quoted 10 lines
Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+
+(grep -q "Xen dom0less mode detected" qemu-staticmem.serial) || exit 1
+
+for ((i=0; i<${#base[@]}; i++))
+do
+    start="$(printf "0x%016x" ${base[$i]})"
+    end="$(printf "0x%016x" $((${base[$i]} + ${size[$i]} - 1)))"
+    grep -q "node   0: \[mem ${start}-${end}\]" qemu-staticmem.serial
+    if test $? -eq 1
+    then
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh line 110

Quoted 3 lines
automation/scripts/qemu-staticmem-arm64.sh new file +3 -0
+        exit 1
+    fi
+done

Please add a comment on top to explain what this is meant to do. However, I think we should avoid relying on output that we have written ourself. IOW, relying on Xen/Linux to always write the same message is risky because they can change at any time.

Especially if we make the script common, then we could just rely on the existing check to see if the guest started correctly (no special check for static memory).

Quoted 2 lines
Quoted 2 lines
automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+
+(grep -q "BusyBox" qemu-staticmem.serial) || exit 1
#6 lore raw

Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Hi,

Quoted 26 lines · Stefano Stabellini
Quoted 16 lines · Julien Grall
Quoted 16 lines
automation/scripts/qemu-staticmem-arm64.sh new file +13 -0
+# Run the test
+rm -f qemu-staticmem.serial
+set +e
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"| \
+timeout -k 1 60 ./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -no-reboot \
+    -device virtio-net-pci,netdev=vnet0 -netdev user,id=vnet0,tftp=binaries

\

automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin \
+    -dtb ./binaries/virt-gicv2.dtb \
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh line 97

Quoted 3 lines
automation/scripts/qemu-staticmem-arm64.sh new file +3 -0
+    |& tee qemu-staticmem.serial
+
+set -e

A lot of the code above is duplicated from qemu-smoke-arm64.sh. I think it would be better to consolidate in a single script. Looking briefly throught the existing scripts, it looks like it is possible to pass arguments (see qemu-smoke-x86-64.sh).

Quoted 3 lines

One idea would be to make the script common and "source" a second script or config file with just the ImageBuilder configuration because it looks like it is the only thing different.

This would mean creating a new bash script for every new test. This sounds a bit pointless if the only difference is the ImageBuilder configuration. Instead, it would be better to pass an argument to the script (like qemu-smoke-x86-64.sh) indicating which test we are going to perform.

Cheers,

Signature · 1 line
Julien Grall
#7 lore raw

Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Quoted 30 lines · Julien Grall

Hi,

Quoted 27 lines · Stefano Stabellini
Quoted 26 lines · Julien Grall
Quoted 19 lines
automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
+# Run the test
+rm -f qemu-staticmem.serial
+set +e
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 

0x40000000"| \
+timeout -k 1 60 ./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -no-reboot \
+    -device virtio-net-pci,netdev=vnet0 -netdev user,id=vnet0,tftp=binaries
\
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin \
+    -dtb ./binaries/virt-gicv2.dtb \
+    |& tee qemu-staticmem.serial

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+set -e

A lot of the code above is duplicated from qemu-smoke-arm64.sh. I think it would be better to consolidate in a single script. Looking briefly throught the existing scripts, it looks like it is possible to pass arguments (see qemu-smoke-x86-64.sh).

Quoted 5 lines
Quoted 3 lines

One idea would be to make the script common and "source" a second script or config file with just the ImageBuilder configuration because it looks like it is the only thing different.

This would mean creating a new bash script for every new test. This sounds a bit pointless if the only difference is the ImageBuilder configuration. Instead, it would be better to pass an argument to the script (like qemu-smoke-x86-64.sh) indicating which test we are going to perform.

I agree with Julien, also because the ImageBuilder script depends on how qemu is configured. It is not completely independent.

Signature · 1 line
Xenia
#8 lore raw

Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

In reply to Stefano Stabellini

Hi Stefano,

Quoted 218 lines · Stefano Stabellini
Quoted 184 lines · Julien Grall

Hi Xenia,

Quoted 38 lines · Xenia Ragiadakou

Add an arm subdirectory under automation/configs for the arm specific configs
and add a config that enables static allocation.

Modify the build script to search for configs also in this subdirectory and to
keep the generated xen binary, suffixed with the config file name, as artifact.

Create a test job that
- boots xen on qemu with a single direct mapped dom0less guest configured with
statically allocated memory
- verifies that the memory ranges reported in the guest's logs are the same with the provided static memory regions

For guest kernel, use the 5.9.9 kernel from the tests-artifacts containers. Use busybox-static package, to create the guest ramdisk.
To generate the u-boot script, use ImageBuilder.
Use the qemu from the tests-artifacts containers.

---
   automation/configs/arm/static_mem          |   3 +
   automation/gitlab-ci/test.yaml             |  24 +++++
   automation/scripts/build                   |   4 +
   automation/scripts/qemu-staticmem-arm64.sh | 114 +++++++++++++++++++++
   4 files changed, 145 insertions(+)

create mode 100644 automation/configs/arm/static_mem create mode 100755 automation/scripts/qemu-staticmem-arm64.sh

automation/configs/arm/static_mem new file +0 -0
new file mode 100644
index 0000000000..84675ddf4e

automation/configs/arm/static_mem line 3

Quoted 4 lines
automation/configs/arm/static_mem new file +3 -0
@@ -0,0 +1,3 @@
+CONFIG_EXPERT=y
+CONFIG_UNSUPPORTED=y
+CONFIG_STATIC_MEMORY=y
\ No newline at end of file

Any particular reason to build a new Xen rather enable CONFIG_STATIC_MEMORY in the existing build?

Quoted 15 lines
automation/scripts/build +1 -0
index 21b3bc57c8..9c6196d9bd 100755
@@ -83,6 +83,7 @@ fi
   # Build all the configs we care about
   case ${XEN_TARGET_ARCH} in
       x86_64) arch=x86 ;;
+    arm64) arch=arm ;;
       *) exit 0 ;;
   esac
   @@ -93,4 +94,7 @@ for cfg in `ls ${cfg_dir}`; do
       rm -f xen/.config
       make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} defconfig
       make -j$(nproc) -C xen

automation/scripts/build for cfg in `ls ${cfg_dir}`; do · line 99

Quoted 3 lines
automation/scripts/build +3 -0
+    if [[ ${arch} == "arm" ]]; then
+        cp xen/xen binaries/xen-${cfg}
+    fi

This feels a bit of a hack to be arm only. Can you explain why this is not enabled for x86 (other than this is not yet used)?

automation/scripts/qemu-staticmem-arm64.sh line 4

Quoted 12 lines

done

automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
new file mode 100755
index 0000000000..5b89a151aa
@@ -0,0 +1,114 @@
+#!/bin/bash
+
+base=(0x50000000 0x100000000)
+size=(0x10000000 0x10000000)

From the name, it is not clear what the base and size refers too. Looking a
bit below, it seems to be referring to the domain memory. If so, I would suggest to comment and rename to "domu_{base, size}".

Quoted 51 lines
automation/scripts/qemu-staticmem-arm64.sh new file +51 -0
+
+set -ex
+
+apt-get -qy update
+apt-get -qy install --no-install-recommends u-boot-qemu \
+                                            u-boot-tools \
+                                            device-tree-compiler \
+                                            cpio \
+                                            curl \
+                                            busybox-static
+
+# DomU Busybox
+cd binaries
+mkdir -p initrd
+mkdir -p initrd/bin
+mkdir -p initrd/sbin
+mkdir -p initrd/etc
+mkdir -p initrd/dev
+mkdir -p initrd/proc
+mkdir -p initrd/sys
+mkdir -p initrd/lib
+mkdir -p initrd/var
+mkdir -p initrd/mnt
+cp /bin/busybox initrd/bin/busybox
+initrd/bin/busybox --install initrd/bin
+echo "#!/bin/sh
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+mount -t devtmpfs devtmpfs /dev
+/bin/sh" > initrd/init
+chmod +x initrd/init
+cd initrd
+find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+cd ../..
+
+# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
+curl -fsSLO https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
+
+./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -M dumpdtb=binaries/virt-gicv2.dtb
+
+#dtc -I dtb -O dts binaries/virt-gicv2.dtb > binaries/virt-gicv2.dts
+
+# ImageBuilder

automation/scripts/qemu-staticmem-arm64.sh line 65

Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+
+echo "MEMORY_START=\"0x40000000\"
+MEMORY_END=\"0x0200000000\"
+
+DEVICE_TREE=\"virt-gicv2.dtb\"
+
+XEN=\"xen-static_mem\"
+XEN_CMD=\"console=dtuart earlyprintk xsm=dummy\"

AFAIK, earlyprintk is not an option for Xen on Arm (at least). It is also not clear why you need to pass xsm=dummy.

Quoted 31 lines
automation/scripts/qemu-staticmem-arm64.sh new file +13 -0
+
+NUM_DOMUS=1
+DOMU_MEM[0]=512
+DOMU_VCPUS[0]=1
+DOMU_KERNEL[0]=\"Image\"
+DOMU_RAMDISK[0]=\"initrd.cpio.gz\"
+DOMU_CMD[0]=\"earlyprintk console=ttyAMA0\"
+DOMU_STATIC_MEM[0]=\"${base[0]} ${size[0]} ${base[1]} ${size[1]}\"
+
+UBOOT_SOURCE=\"boot.source\"
+UBOOT_SCRIPT=\"boot.scr\"" > binaries/imagebuilder_config
+
+bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c

binaries/imagebuilder_config

automation/scripts/qemu-staticmem-arm64.sh new file +14 -0
+
+# Run the test
+rm -f qemu-staticmem.serial
+set +e
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"| \
+timeout -k 1 60 ./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -no-reboot \
+    -device virtio-net-pci,netdev=vnet0 -netdev user,id=vnet0,tftp=binaries

\

automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin \
+    -dtb ./binaries/virt-gicv2.dtb \
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh line 97

Quoted 3 lines
automation/scripts/qemu-staticmem-arm64.sh new file +3 -0
+    |& tee qemu-staticmem.serial
+
+set -e

A lot of the code above is duplicated from qemu-smoke-arm64.sh. I think it would be better to consolidate in a single script. Looking briefly throught the existing scripts, it looks like it is possible to pass arguments (see qemu-smoke-x86-64.sh).

One idea would be to make the script common and "source" a second script or config file with just the ImageBuilder configuration because it looks like it is the only thing different.

Quoted 10 lines
Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+
+(grep -q "Xen dom0less mode detected" qemu-staticmem.serial) || exit 1
+
+for ((i=0; i<${#base[@]}; i++))
+do
+    start="$(printf "0x%016x" ${base[$i]})"
+    end="$(printf "0x%016x" $((${base[$i]} + ${size[$i]} - 1)))"
+    grep -q "node   0: \[mem ${start}-${end}\]" qemu-staticmem.serial
+    if test $? -eq 1
+    then
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh line 110

Quoted 3 lines
automation/scripts/qemu-staticmem-arm64.sh new file +3 -0
+        exit 1
+    fi
+done

Please add a comment on top to explain what this is meant to do. However, I think we should avoid relying on output that we have written ourself. IOW, relying on Xen/Linux to always write the same message is risky because they can change at any time.

Quoted 3 lines

Especially if we make the script common, then we could just rely on the existing check to see if the guest started correctly (no special check for static memory).

In this case, how the test will verify that the static memory configuration has been taken into account and has not just been ignored?

Quoted 3 lines
Quoted 2 lines
Quoted 2 lines
automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+
+(grep -q "BusyBox" qemu-staticmem.serial) || exit 1
Signature · 1 line
Xenia
#9 lore raw

Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Quoted 233 lines · Xenia Ragiadakou
Quoted 232 lines · Stefano Stabellini
Quoted 197 lines · Julien Grall

Hi Xenia,

Quoted 43 lines · Xenia Ragiadakou

Add an arm subdirectory under automation/configs for the arm specific configs
and add a config that enables static allocation.

Modify the build script to search for configs also in this subdirectory and
to
keep the generated xen binary, suffixed with the config file name, as artifact.

Create a test job that
- boots xen on qemu with a single direct mapped dom0less guest configured
with
statically allocated memory
- verifies that the memory ranges reported in the guest's logs are the same
with the provided static memory regions

For guest kernel, use the 5.9.9 kernel from the tests-artifacts containers.
Use busybox-static package, to create the guest ramdisk. To generate the u-boot script, use ImageBuilder.
Use the qemu from the tests-artifacts containers.

---
   automation/configs/arm/static_mem          |   3 +
   automation/gitlab-ci/test.yaml             |  24 +++++
   automation/scripts/build                   |   4 +
   automation/scripts/qemu-staticmem-arm64.sh | 114
+++++++++++++++++++++
   4 files changed, 145 insertions(+)

create mode 100644 automation/configs/arm/static_mem create mode 100755 automation/scripts/qemu-staticmem-arm64.sh

automation/configs/arm/static_mem new file +0 -0
new file mode 100644
index 0000000000..84675ddf4e

automation/configs/arm/static_mem line 3

Quoted 4 lines
automation/configs/arm/static_mem new file +3 -0
@@ -0,0 +1,3 @@
+CONFIG_EXPERT=y
+CONFIG_UNSUPPORTED=y
+CONFIG_STATIC_MEMORY=y
\ No newline at end of file

Any particular reason to build a new Xen rather enable CONFIG_STATIC_MEMORY in
the existing build?

Quoted 16 lines
automation/scripts/build +1 -0
index 21b3bc57c8..9c6196d9bd 100755
@@ -83,6 +83,7 @@ fi
   # Build all the configs we care about
   case ${XEN_TARGET_ARCH} in
       x86_64) arch=x86 ;;
+    arm64) arch=arm ;;
       *) exit 0 ;;
   esac
   @@ -93,4 +94,7 @@ for cfg in `ls ${cfg_dir}`; do

rm -f xen/.config
make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg}
defconfig

       make -j$(nproc) -C xen

automation/scripts/build for cfg in `ls ${cfg_dir}`; do · line 99

Quoted 3 lines
automation/scripts/build +3 -0
+    if [[ ${arch} == "arm" ]]; then
+        cp xen/xen binaries/xen-${cfg}
+    fi

This feels a bit of a hack to be arm only. Can you explain why this is not enabled for x86 (other than this is not yet used)?

automation/scripts/qemu-staticmem-arm64.sh line 4

Quoted 12 lines

done

automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
new file mode 100755
index 0000000000..5b89a151aa
@@ -0,0 +1,114 @@
+#!/bin/bash
+
+base=(0x50000000 0x100000000)
+size=(0x10000000 0x10000000)

From the name, it is not clear what the base and size refers too. Looking
a
bit below, it seems to be referring to the domain memory. If so, I would suggest to comment and rename to "domu_{base, size}".

Quoted 52 lines
automation/scripts/qemu-staticmem-arm64.sh new file +38 -0
+
+set -ex
+
+apt-get -qy update
+apt-get -qy install --no-install-recommends u-boot-qemu \
+                                            u-boot-tools \
+                                            device-tree-compiler \
+                                            cpio \
+                                            curl \
+                                            busybox-static
+
+# DomU Busybox
+cd binaries
+mkdir -p initrd
+mkdir -p initrd/bin
+mkdir -p initrd/sbin
+mkdir -p initrd/etc
+mkdir -p initrd/dev
+mkdir -p initrd/proc
+mkdir -p initrd/sys
+mkdir -p initrd/lib
+mkdir -p initrd/var
+mkdir -p initrd/mnt
+cp /bin/busybox initrd/bin/busybox
+initrd/bin/busybox --install initrd/bin
+echo "#!/bin/sh
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+mount -t devtmpfs devtmpfs /dev
+/bin/sh" > initrd/init
+chmod +x initrd/init
+cd initrd
+find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+cd ../..
+
+# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
+curl -fsSLO

https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom

automation/scripts/qemu-staticmem-arm64.sh new file +13 -0
+
+./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -M dumpdtb=binaries/virt-gicv2.dtb
+
+#dtc -I dtb -O dts binaries/virt-gicv2.dtb > binaries/virt-gicv2.dts
+
+# ImageBuilder

automation/scripts/qemu-staticmem-arm64.sh line 65

Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+
+echo "MEMORY_START=\"0x40000000\"
+MEMORY_END=\"0x0200000000\"
+
+DEVICE_TREE=\"virt-gicv2.dtb\"
+
+XEN=\"xen-static_mem\"
+XEN_CMD=\"console=dtuart earlyprintk xsm=dummy\"

AFAIK, earlyprintk is not an option for Xen on Arm (at least). It is also not
clear why you need to pass xsm=dummy.

Quoted 34 lines
automation/scripts/qemu-staticmem-arm64.sh new file +13 -0
+
+NUM_DOMUS=1
+DOMU_MEM[0]=512
+DOMU_VCPUS[0]=1
+DOMU_KERNEL[0]=\"Image\"
+DOMU_RAMDISK[0]=\"initrd.cpio.gz\"
+DOMU_CMD[0]=\"earlyprintk console=ttyAMA0\"
+DOMU_STATIC_MEM[0]=\"${base[0]} ${size[0]} ${base[1]} ${size[1]}\"
+
+UBOOT_SOURCE=\"boot.source\"
+UBOOT_SCRIPT=\"boot.scr\"" > binaries/imagebuilder_config
+
+bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c

binaries/imagebuilder_config

automation/scripts/qemu-staticmem-arm64.sh new file +5 -0
+
+# Run the test
+rm -f qemu-staticmem.serial
+set +e
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source

0x40000000"| \

automation/scripts/qemu-staticmem-arm64.sh new file +9 -0
+timeout -k 1 60 ./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -no-reboot \
+    -device virtio-net-pci,netdev=vnet0 -netdev

user,id=vnet0,tftp=binaries
\

automation/scripts/qemu-staticmem-arm64.sh new file +3 -0
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin \
+    -dtb ./binaries/virt-gicv2.dtb \
+    |& tee qemu-staticmem.serial
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh

Quoted 2 lines
automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+
+set -e

A lot of the code above is duplicated from qemu-smoke-arm64.sh. I think it would be better to consolidate in a single script. Looking briefly throught
the existing scripts, it looks like it is possible to pass arguments (see qemu-smoke-x86-64.sh).

One idea would be to make the script common and "source" a second
script or config file with just the ImageBuilder configuration because it looks like it is the only thing different.

Quoted 20 lines

automation/scripts/qemu-staticmem-arm64.sh line 110

Quoted 13 lines
automation/scripts/qemu-staticmem-arm64.sh new file +13 -0
+
+(grep -q "Xen dom0less mode detected" qemu-staticmem.serial) || exit 1
+
+for ((i=0; i<${#base[@]}; i++))
+do
+    start="$(printf "0x%016x" ${base[$i]})"
+    end="$(printf "0x%016x" $((${base[$i]} + ${size[$i]} - 1)))"
+    grep -q "node   0: \[mem ${start}-${end}\]" qemu-staticmem.serial
+    if test $? -eq 1
+    then
+        exit 1
+    fi
+done

Please add a comment on top to explain what this is meant to do. However, I
think we should avoid relying on output that we have written ourself. IOW, relying on Xen/Linux to always write the same message is risky because they
can change at any time.

Quoted 6 lines
Quoted 3 lines

Especially if we make the script common, then we could just rely on the existing check to see if the guest started correctly (no special check for static memory).

In this case, how the test will verify that the static memory configuration has been taken into account and has not just been ignored?

There is no simple way that I can think of.

We can trust that it worked, without checking that the ranges were actually static as requested.

We can parse the Xen output like you did, although if it changes then the test will break.

Or we could add a script to detect and print specific output but I don't know if there is something under /proc or /sys that we could simply "cat" from bash to check it.

If there is a simple way to do this by cat'ing /proc or /sys then I think that would be great. Otherwise I think we could do as you did in this patch, which is not perfect but it works and if something changes in the Xen output we'll detect it right away given that gitlab-ci is run pre-commit.

#10 lore raw

Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Hi,

Quoted 3 lines · Stefano Stabellini

Or we could add a script to detect and print specific output but I don't know if there is something under /proc or /sys that we could simply "cat" from bash to check it.

The domain device-tree should be /proc/device-tree. So you could check the properties from there.

Cheers,

Signature · 1 line
Julien Grall
#11 lore raw

Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

In reply to Stefano Stabellini
Quoted 253 lines · Stefano Stabellini
Quoted 233 lines · Xenia Ragiadakou
Quoted 232 lines · Stefano Stabellini
Quoted 197 lines · Julien Grall

Hi Xenia,

Quoted 43 lines · Xenia Ragiadakou

Add an arm subdirectory under automation/configs for the arm specific configs
and add a config that enables static allocation.

Modify the build script to search for configs also in this subdirectory and
to
keep the generated xen binary, suffixed with the config file name, as artifact.

Create a test job that
- boots xen on qemu with a single direct mapped dom0less guest configured
with
statically allocated memory
- verifies that the memory ranges reported in the guest's logs are the same
with the provided static memory regions

For guest kernel, use the 5.9.9 kernel from the tests-artifacts containers.
Use busybox-static package, to create the guest ramdisk. To generate the u-boot script, use ImageBuilder.
Use the qemu from the tests-artifacts containers.

---
    automation/configs/arm/static_mem          |   3 +
    automation/gitlab-ci/test.yaml             |  24 +++++
    automation/scripts/build                   |   4 +
    automation/scripts/qemu-staticmem-arm64.sh | 114
+++++++++++++++++++++
    4 files changed, 145 insertions(+)

create mode 100644 automation/configs/arm/static_mem create mode 100755 automation/scripts/qemu-staticmem-arm64.sh

automation/configs/arm/static_mem new file +0 -0
new file mode 100644
index 0000000000..84675ddf4e

automation/configs/arm/static_mem line 3

Quoted 4 lines
automation/configs/arm/static_mem new file +3 -0
@@ -0,0 +1,3 @@
+CONFIG_EXPERT=y
+CONFIG_UNSUPPORTED=y
+CONFIG_STATIC_MEMORY=y
\ No newline at end of file

Any particular reason to build a new Xen rather enable CONFIG_STATIC_MEMORY in
the existing build?

Quoted 16 lines
automation/scripts/build +1 -0
index 21b3bc57c8..9c6196d9bd 100755
@@ -83,6 +83,7 @@ fi
    # Build all the configs we care about
    case ${XEN_TARGET_ARCH} in
        x86_64) arch=x86 ;;
+    arm64) arch=arm ;;
        *) exit 0 ;;
    esac
    @@ -93,4 +94,7 @@ for cfg in `ls ${cfg_dir}`; do

rm -f xen/.config
make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg}
defconfig

        make -j$(nproc) -C xen

automation/scripts/build for cfg in `ls ${cfg_dir}`; do · line 99

Quoted 3 lines
automation/scripts/build +3 -0
+    if [[ ${arch} == "arm" ]]; then
+        cp xen/xen binaries/xen-${cfg}
+    fi

This feels a bit of a hack to be arm only. Can you explain why this is not enabled for x86 (other than this is not yet used)?

automation/scripts/qemu-staticmem-arm64.sh line 4

Quoted 12 lines

done

automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
new file mode 100755
index 0000000000..5b89a151aa
@@ -0,0 +1,114 @@
+#!/bin/bash
+
+base=(0x50000000 0x100000000)
+size=(0x10000000 0x10000000)

From the name, it is not clear what the base and size refers too. Looking
a
bit below, it seems to be referring to the domain memory. If so, I would suggest to comment and rename to "domu_{base, size}".

Quoted 52 lines
automation/scripts/qemu-staticmem-arm64.sh new file +38 -0
+
+set -ex
+
+apt-get -qy update
+apt-get -qy install --no-install-recommends u-boot-qemu \
+                                            u-boot-tools \
+                                            device-tree-compiler \
+                                            cpio \
+                                            curl \
+                                            busybox-static
+
+# DomU Busybox
+cd binaries
+mkdir -p initrd
+mkdir -p initrd/bin
+mkdir -p initrd/sbin
+mkdir -p initrd/etc
+mkdir -p initrd/dev
+mkdir -p initrd/proc
+mkdir -p initrd/sys
+mkdir -p initrd/lib
+mkdir -p initrd/var
+mkdir -p initrd/mnt
+cp /bin/busybox initrd/bin/busybox
+initrd/bin/busybox --install initrd/bin
+echo "#!/bin/sh
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+mount -t devtmpfs devtmpfs /dev
+/bin/sh" > initrd/init
+chmod +x initrd/init
+cd initrd
+find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+cd ../..
+
+# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
+curl -fsSLO

https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom

automation/scripts/qemu-staticmem-arm64.sh new file +13 -0
+
+./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -M dumpdtb=binaries/virt-gicv2.dtb
+
+#dtc -I dtb -O dts binaries/virt-gicv2.dtb > binaries/virt-gicv2.dts
+
+# ImageBuilder

automation/scripts/qemu-staticmem-arm64.sh line 65

Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+
+echo "MEMORY_START=\"0x40000000\"
+MEMORY_END=\"0x0200000000\"
+
+DEVICE_TREE=\"virt-gicv2.dtb\"
+
+XEN=\"xen-static_mem\"
+XEN_CMD=\"console=dtuart earlyprintk xsm=dummy\"

AFAIK, earlyprintk is not an option for Xen on Arm (at least). It is also not
clear why you need to pass xsm=dummy.

Quoted 34 lines
automation/scripts/qemu-staticmem-arm64.sh new file +13 -0
+
+NUM_DOMUS=1
+DOMU_MEM[0]=512
+DOMU_VCPUS[0]=1
+DOMU_KERNEL[0]=\"Image\"
+DOMU_RAMDISK[0]=\"initrd.cpio.gz\"
+DOMU_CMD[0]=\"earlyprintk console=ttyAMA0\"
+DOMU_STATIC_MEM[0]=\"${base[0]} ${size[0]} ${base[1]} ${size[1]}\"
+
+UBOOT_SOURCE=\"boot.source\"
+UBOOT_SCRIPT=\"boot.scr\"" > binaries/imagebuilder_config
+
+bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c

binaries/imagebuilder_config

automation/scripts/qemu-staticmem-arm64.sh new file +5 -0
+
+# Run the test
+rm -f qemu-staticmem.serial
+set +e
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source

0x40000000"| \

automation/scripts/qemu-staticmem-arm64.sh new file +9 -0
+timeout -k 1 60 ./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -no-reboot \
+    -device virtio-net-pci,netdev=vnet0 -netdev

user,id=vnet0,tftp=binaries
\

automation/scripts/qemu-staticmem-arm64.sh new file +3 -0
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin \
+    -dtb ./binaries/virt-gicv2.dtb \
+    |& tee qemu-staticmem.serial
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh

Quoted 2 lines
automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+
+set -e

A lot of the code above is duplicated from qemu-smoke-arm64.sh. I think it would be better to consolidate in a single script. Looking briefly throught
the existing scripts, it looks like it is possible to pass arguments (see qemu-smoke-x86-64.sh).

One idea would be to make the script common and "source" a second
script or config file with just the ImageBuilder configuration because it looks like it is the only thing different.

Quoted 20 lines

automation/scripts/qemu-staticmem-arm64.sh line 110

Quoted 13 lines
automation/scripts/qemu-staticmem-arm64.sh new file +13 -0
+
+(grep -q "Xen dom0less mode detected" qemu-staticmem.serial) || exit 1
+
+for ((i=0; i<${#base[@]}; i++))
+do
+    start="$(printf "0x%016x" ${base[$i]})"
+    end="$(printf "0x%016x" $((${base[$i]} + ${size[$i]} - 1)))"
+    grep -q "node   0: \[mem ${start}-${end}\]" qemu-staticmem.serial
+    if test $? -eq 1
+    then
+        exit 1
+    fi
+done

Please add a comment on top to explain what this is meant to do. However, I
think we should avoid relying on output that we have written ourself. IOW, relying on Xen/Linux to always write the same message is risky because they
can change at any time.

Quoted 6 lines
Quoted 3 lines

Especially if we make the script common, then we could just rely on the existing check to see if the guest started correctly (no special check for static memory).

In this case, how the test will verify that the static memory configuration has been taken into account and has not just been ignored?

There is no simple way that I can think of.

We can trust that it worked, without checking that the ranges were actually static as requested.

We can parse the Xen output like you did, although if it changes then the test will break.

Or we could add a script to detect and print specific output but I don't know if there is something under /proc or /sys that we could simply "cat" from bash to check it.

Quoted 5 lines

If there is a simple way to do this by cat'ing /proc or /sys then I think that would be great. Otherwise I think we could do as you did in this patch, which is not perfect but it works and if something changes in the Xen output we'll detect it right away given that gitlab-ci is run pre-commit.

Ok, I 'll send another patch that will enable static memory for all arm xen builds and will use a single script. For the static memory case, I will try to verify the values of the guest's dt memory node via /proc/device-tree.

Also, I will do some minor cleanups in qemu-smoke-arm64.sh because there are some inconsistencies for instance the dtb is named virt-gicv3 while runs with gicv2, a comment states that qemu gets installed while it is taken from test-artifacts container.

Thanks to both of you for the review.

Signature · 1 line
Xenia
#12 lore raw

RE: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

In reply to Xenia Ragiadakou

Hi Xenia

Quoted 9 lines

-----Original Message-----
From: Xen-devel <[email protected]> On Behalf Of Xenia Ragiadakou
Sent: Friday, July 8, 2022 5:54 PM
To: Stefano Stabellini <[email protected]>; Julien Grall <[email protected]>
Cc: [email protected]; Doug Goldstein <[email protected]>
Subject: Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Quoted 17 lines

Hi Stefano,

Quoted 14 lines · Stefano Stabellini
Quoted 13 lines · Julien Grall

Hi Xenia,

Quoted 10 lines · Xenia Ragiadakou

Add an arm subdirectory under automation/configs for the arm specific configs and add a config that enables static allocation.

Modify the build script to search for configs also in this subdirectory and to keep the generated xen binary, suffixed with the config file name, as artifact.

Create a test job that
- boots xen on qemu with a single direct mapped dom0less guest configured with statically allocated memory

Although you said booting a single direct mapped dom0less guest configured with statically allocated memory here, later in code, you are only enabling statically allocated memory in the ImageBuilder script, missing the direct-map property.

Quoted 139 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

- verifies that the memory ranges reported in the guest's logs are the same with the provided static memory regions

For guest kernel, use the 5.9.9 kernel from the tests-artifacts containers. Use busybox-static package, to create the guest ramdisk.
To generate the u-boot script, use ImageBuilder.
Use the qemu from the tests-artifacts containers.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
---
   automation/configs/arm/static_mem          |   3 +
   automation/gitlab-ci/test.yaml             |  24 +++++
   automation/scripts/build                   |   4 +
   automation/scripts/qemu-staticmem-arm64.sh | 114

+++++++++++++++++++++

Quoted 124 lines
Quoted 124 lines
Quoted 11 lines
   4 files changed, 145 insertions(+)

create mode 100644 automation/configs/arm/static_mem create mode 100755 automation/scripts/qemu-staticmem-arm64.sh

automation/configs/arm/static_mem new file +0 -0
new file mode 100644
index 0000000000..84675ddf4e

automation/configs/arm/static_mem line 3

Quoted 4 lines
automation/configs/arm/static_mem new file +3 -0
@@ -0,0 +1,3 @@
+CONFIG_EXPERT=y
+CONFIG_UNSUPPORTED=y
+CONFIG_STATIC_MEMORY=y
\ No newline at end of file

Any particular reason to build a new Xen rather enable CONFIG_STATIC_MEMORY in the existing build?

Quoted 15 lines
automation/scripts/build +1 -0
index 21b3bc57c8..9c6196d9bd 100755
@@ -83,6 +83,7 @@ fi
   # Build all the configs we care about
   case ${XEN_TARGET_ARCH} in
       x86_64) arch=x86 ;;
+    arm64) arch=arm ;;
       *) exit 0 ;;
   esac
   @@ -93,4 +94,7 @@ for cfg in `ls ${cfg_dir}`; do
       rm -f xen/.config
       make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} defconfig
       make -j$(nproc) -C xen

automation/scripts/build for cfg in `ls ${cfg_dir}`; do · line 99

Quoted 3 lines
automation/scripts/build +3 -0
+    if [[ ${arch} == "arm" ]]; then
+        cp xen/xen binaries/xen-${cfg}
+    fi

This feels a bit of a hack to be arm only. Can you explain why this is not enabled for x86 (other than this is not yet used)?

automation/scripts/qemu-staticmem-arm64.sh line 4

Quoted 12 lines

done

automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
new file mode 100755
index 0000000000..5b89a151aa
@@ -0,0 +1,114 @@
+#!/bin/bash
+
+base=(0x50000000 0x100000000)
+size=(0x10000000 0x10000000)

From the name, it is not clear what the base and size refers too.
Looking a bit below, it seems to be referring to the domain memory. If so, I would suggest to comment and rename to "domu_{base, size}".

Quoted 52 lines
automation/scripts/qemu-staticmem-arm64.sh new file +52 -0
+
+set -ex
+
+apt-get -qy update
+apt-get -qy install --no-install-recommends u-boot-qemu \
+                                            u-boot-tools \
+                                            device-tree-compiler \
+                                            cpio \
+                                            curl \
+                                            busybox-static
+
+# DomU Busybox
+cd binaries
+mkdir -p initrd
+mkdir -p initrd/bin
+mkdir -p initrd/sbin
+mkdir -p initrd/etc
+mkdir -p initrd/dev
+mkdir -p initrd/proc
+mkdir -p initrd/sys
+mkdir -p initrd/lib
+mkdir -p initrd/var
+mkdir -p initrd/mnt
+cp /bin/busybox initrd/bin/busybox
+initrd/bin/busybox --install initrd/bin echo "#!/bin/sh
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+mount -t devtmpfs devtmpfs /dev
+/bin/sh" > initrd/init
+chmod +x initrd/init
+cd initrd
+find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+cd ../..
+
+# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded curl
+-fsSLO
+https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
+
+./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -M dumpdtb=binaries/virt-gicv2.dtb
+
+#dtc -I dtb -O dts binaries/virt-gicv2.dtb >
+binaries/virt-gicv2.dts
+
+# ImageBuilder

automation/scripts/qemu-staticmem-arm64.sh line 65

Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+
+echo "MEMORY_START=\"0x40000000\"
+MEMORY_END=\"0x0200000000\"
+
+DEVICE_TREE=\"virt-gicv2.dtb\"
+
+XEN=\"xen-static_mem\"
+XEN_CMD=\"console=dtuart earlyprintk xsm=dummy\"

AFAIK, earlyprintk is not an option for Xen on Arm (at least). It is also not clear why you need to pass xsm=dummy.

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

automation/scripts/qemu-staticmem-arm64.sh line 74

Quoted 9 lines
automation/scripts/qemu-staticmem-arm64.sh new file +9 -0
+
+NUM_DOMUS=1
+DOMU_MEM[0]=512
+DOMU_VCPUS[0]=1
+DOMU_KERNEL[0]=\"Image\"
+DOMU_RAMDISK[0]=\"initrd.cpio.gz\"
+DOMU_CMD[0]=\"earlyprintk console=ttyAMA0\"
+DOMU_STATIC_MEM[0]=\"${base[0]} ${size[0]} ${base[1]} ${size[1]}\"
+

You would like to add DOMU_DIRECT_MAP[0] = 1 to enable direct-map.

Quoted 58 lines
Quoted 11 lines
Quoted 11 lines

automation/scripts/qemu-staticmem-arm64.sh

Quoted 11 lines
automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
+UBOOT_SOURCE=\"boot.source\"
+UBOOT_SCRIPT=\"boot.scr\"" > binaries/imagebuilder_config
+
+bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c

binaries/imagebuilder_config

automation/scripts/qemu-staticmem-arm64.sh new file +6 -0
+
+# Run the test
+rm -f qemu-staticmem.serial
+set +e
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source
+0x40000000"| \ timeout -k 1 60 ./binaries/qemu-system-aarch64 -

nographic \

Quoted 46 lines
Quoted 12 lines
Quoted 12 lines
automation/scripts/qemu-staticmem-arm64.sh new file +9 -0
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -no-reboot \
+    -device virtio-net-pci,netdev=vnet0 -netdev
+user,id=vnet0,tftp=binaries

\

automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin \
+    -dtb ./binaries/virt-gicv2.dtb \
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh line 97

Quoted 3 lines
automation/scripts/qemu-staticmem-arm64.sh new file +3 -0
+    |& tee qemu-staticmem.serial
+
+set -e

A lot of the code above is duplicated from qemu-smoke-arm64.sh. I think it would be better to consolidate in a single script. Looking briefly throught the existing scripts, it looks like it is possible to pass arguments (see qemu-smoke-x86-64.sh).

One idea would be to make the script common and "source" a second script or config file with just the ImageBuilder configuration because it looks like it is the only thing different.

Quoted 19 lines
Quoted 4 lines
automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
+
+(grep -q "Xen dom0less mode detected" qemu-staticmem.serial) ||
+exit 1
+

automation/scripts/qemu-staticmem-arm64.sh line 110

Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+for ((i=0; i<${#base[@]}; i++))
+do
+    start="$(printf "0x%016x" ${base[$i]})"
+    end="$(printf "0x%016x" $((${base[$i]} + ${size[$i]} - 1)))"
+    grep -q "node   0: \[mem ${start}-${end}\]" qemu-staticmem.serial
+    if test $? -eq 1
+    then
+        exit 1
+    fi
+done

Please add a comment on top to explain what this is meant to do. However, I think we should avoid relying on output that we have written ourself. IOW, relying on Xen/Linux to always write the same message is risky because they can change at any time.

Quoted 7 lines
Quoted 3 lines

Especially if we make the script common, then we could just rely on the existing check to see if the guest started correctly (no special check for static memory).

In this case, how the test will verify that the static memory configuration has been taken into account and has not just been ignored?

If only statically allocated memory is enabled, guest memory address will still be mapped to GUEST_RAM_BASE(0x40000000)

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

automation/scripts/qemu-staticmem-arm64.sh

Quoted 2 lines
automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+
+(grep -q "BusyBox" qemu-staticmem.serial) || exit 1

--
Xenia

---
Cheers,
Penny Zheng

#13 lore raw

Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Hi Penny,

Quoted 173 lines · Penny Zheng

Hi Xenia

Quoted 9 lines

-----Original Message-----
From: Xen-devel <[email protected]> On Behalf Of Xenia Ragiadakou
Sent: Friday, July 8, 2022 5:54 PM
To: Stefano Stabellini <[email protected]>; Julien Grall <[email protected]>
Cc: [email protected]; Doug Goldstein <[email protected]>
Subject: Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Quoted 17 lines

Hi Stefano,

Quoted 14 lines · Stefano Stabellini
Quoted 13 lines · Julien Grall

Hi Xenia,

Quoted 10 lines · Xenia Ragiadakou

Add an arm subdirectory under automation/configs for the arm specific configs and add a config that enables static allocation.

Modify the build script to search for configs also in this subdirectory and to keep the generated xen binary, suffixed with the config file name, as artifact.

Create a test job that
- boots xen on qemu with a single direct mapped dom0less guest configured with statically allocated memory

Although you said booting a single direct mapped dom0less guest configured with statically allocated memory here, later in code, you are only enabling statically allocated memory in the ImageBuilder script, missing the direct-map property.

Quoted 139 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

- verifies that the memory ranges reported in the guest's logs are the same with the provided static memory regions

For guest kernel, use the 5.9.9 kernel from the tests-artifacts containers. Use busybox-static package, to create the guest ramdisk.
To generate the u-boot script, use ImageBuilder.
Use the qemu from the tests-artifacts containers.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
---
    automation/configs/arm/static_mem          |   3 +
    automation/gitlab-ci/test.yaml             |  24 +++++
    automation/scripts/build                   |   4 +
    automation/scripts/qemu-staticmem-arm64.sh | 114

+++++++++++++++++++++

Quoted 124 lines
Quoted 124 lines
Quoted 11 lines
    4 files changed, 145 insertions(+)

create mode 100644 automation/configs/arm/static_mem create mode 100755 automation/scripts/qemu-staticmem-arm64.sh

automation/configs/arm/static_mem new file +0 -0
new file mode 100644
index 0000000000..84675ddf4e

automation/configs/arm/static_mem line 3

Quoted 4 lines
automation/configs/arm/static_mem new file +3 -0
@@ -0,0 +1,3 @@
+CONFIG_EXPERT=y
+CONFIG_UNSUPPORTED=y
+CONFIG_STATIC_MEMORY=y
\ No newline at end of file

Any particular reason to build a new Xen rather enable CONFIG_STATIC_MEMORY in the existing build?

Quoted 15 lines
automation/scripts/build +1 -0
index 21b3bc57c8..9c6196d9bd 100755
@@ -83,6 +83,7 @@ fi
    # Build all the configs we care about
    case ${XEN_TARGET_ARCH} in
        x86_64) arch=x86 ;;
+    arm64) arch=arm ;;
        *) exit 0 ;;
    esac
    @@ -93,4 +94,7 @@ for cfg in `ls ${cfg_dir}`; do
        rm -f xen/.config
        make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} defconfig
        make -j$(nproc) -C xen

automation/scripts/build for cfg in `ls ${cfg_dir}`; do · line 99

Quoted 3 lines
automation/scripts/build +3 -0
+    if [[ ${arch} == "arm" ]]; then
+        cp xen/xen binaries/xen-${cfg}
+    fi

This feels a bit of a hack to be arm only. Can you explain why this is not enabled for x86 (other than this is not yet used)?

automation/scripts/qemu-staticmem-arm64.sh line 4

Quoted 12 lines

done

automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
new file mode 100755
index 0000000000..5b89a151aa
@@ -0,0 +1,114 @@
+#!/bin/bash
+
+base=(0x50000000 0x100000000)
+size=(0x10000000 0x10000000)

From the name, it is not clear what the base and size refers too.
Looking a bit below, it seems to be referring to the domain memory. If so, I would suggest to comment and rename to "domu_{base, size}".

Quoted 52 lines
automation/scripts/qemu-staticmem-arm64.sh new file +52 -0
+
+set -ex
+
+apt-get -qy update
+apt-get -qy install --no-install-recommends u-boot-qemu \
+                                            u-boot-tools \
+                                            device-tree-compiler \
+                                            cpio \
+                                            curl \
+                                            busybox-static
+
+# DomU Busybox
+cd binaries
+mkdir -p initrd
+mkdir -p initrd/bin
+mkdir -p initrd/sbin
+mkdir -p initrd/etc
+mkdir -p initrd/dev
+mkdir -p initrd/proc
+mkdir -p initrd/sys
+mkdir -p initrd/lib
+mkdir -p initrd/var
+mkdir -p initrd/mnt
+cp /bin/busybox initrd/bin/busybox
+initrd/bin/busybox --install initrd/bin echo "#!/bin/sh
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+mount -t devtmpfs devtmpfs /dev
+/bin/sh" > initrd/init
+chmod +x initrd/init
+cd initrd
+find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+cd ../..
+
+# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded curl
+-fsSLO
+https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
+
+./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -M dumpdtb=binaries/virt-gicv2.dtb
+
+#dtc -I dtb -O dts binaries/virt-gicv2.dtb >
+binaries/virt-gicv2.dts
+
+# ImageBuilder

automation/scripts/qemu-staticmem-arm64.sh line 65

Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+
+echo "MEMORY_START=\"0x40000000\"
+MEMORY_END=\"0x0200000000\"
+
+DEVICE_TREE=\"virt-gicv2.dtb\"
+
+XEN=\"xen-static_mem\"
+XEN_CMD=\"console=dtuart earlyprintk xsm=dummy\"

AFAIK, earlyprintk is not an option for Xen on Arm (at least). It is also not clear why you need to pass xsm=dummy.

Quoted 11 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

automation/scripts/qemu-staticmem-arm64.sh line 74

Quoted 9 lines
automation/scripts/qemu-staticmem-arm64.sh new file +9 -0
+
+NUM_DOMUS=1
+DOMU_MEM[0]=512
+DOMU_VCPUS[0]=1
+DOMU_KERNEL[0]=\"Image\"
+DOMU_RAMDISK[0]=\"initrd.cpio.gz\"
+DOMU_CMD[0]=\"earlyprintk console=ttyAMA0\"
+DOMU_STATIC_MEM[0]=\"${base[0]} ${size[0]} ${base[1]} ${size[1]}\"
+

You would like to add DOMU_DIRECT_MAP[0] = 1 to enable direct-map.

The imagebuilder configuration option DOMU_DIRECT_MAP defaults to 1.

But I could add DOMU_DIRECT_MAP[0]=1 in the script to make it clearer.

Quoted 83 lines
Quoted 58 lines
Quoted 11 lines
Quoted 11 lines

automation/scripts/qemu-staticmem-arm64.sh

Quoted 11 lines
automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
+UBOOT_SOURCE=\"boot.source\"
+UBOOT_SCRIPT=\"boot.scr\"" > binaries/imagebuilder_config
+
+bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c

binaries/imagebuilder_config

automation/scripts/qemu-staticmem-arm64.sh new file +6 -0
+
+# Run the test
+rm -f qemu-staticmem.serial
+set +e
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source
+0x40000000"| \ timeout -k 1 60 ./binaries/qemu-system-aarch64 -

nographic \

Quoted 46 lines
Quoted 12 lines
Quoted 12 lines
automation/scripts/qemu-staticmem-arm64.sh new file +9 -0
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -no-reboot \
+    -device virtio-net-pci,netdev=vnet0 -netdev
+user,id=vnet0,tftp=binaries

\

automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin \
+    -dtb ./binaries/virt-gicv2.dtb \
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh line 97

Quoted 3 lines
automation/scripts/qemu-staticmem-arm64.sh new file +3 -0
+    |& tee qemu-staticmem.serial
+
+set -e

A lot of the code above is duplicated from qemu-smoke-arm64.sh. I think it would be better to consolidate in a single script. Looking briefly throught the existing scripts, it looks like it is possible to pass arguments (see qemu-smoke-x86-64.sh).

One idea would be to make the script common and "source" a second script or config file with just the ImageBuilder configuration because it looks like it is the only thing different.

Quoted 19 lines
Quoted 4 lines
automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
+
+(grep -q "Xen dom0less mode detected" qemu-staticmem.serial) ||
+exit 1
+

automation/scripts/qemu-staticmem-arm64.sh line 110

Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+for ((i=0; i<${#base[@]}; i++))
+do
+    start="$(printf "0x%016x" ${base[$i]})"
+    end="$(printf "0x%016x" $((${base[$i]} + ${size[$i]} - 1)))"
+    grep -q "node   0: \[mem ${start}-${end}\]" qemu-staticmem.serial
+    if test $? -eq 1
+    then
+        exit 1
+    fi
+done

Please add a comment on top to explain what this is meant to do. However, I think we should avoid relying on output that we have written ourself. IOW, relying on Xen/Linux to always write the same message is risky because they can change at any time.

Quoted 7 lines
Quoted 3 lines

Especially if we make the script common, then we could just rely on the existing check to see if the guest started correctly (no special check for static memory).

In this case, how the test will verify that the static memory configuration has been taken into account and has not just been ignored?

If only statically allocated memory is enabled, guest memory address will still be mapped to GUEST_RAM_BASE(0x40000000)

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

automation/scripts/qemu-staticmem-arm64.sh

Quoted 2 lines
automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+
+(grep -q "BusyBox" qemu-staticmem.serial) || exit 1

--
Xenia

---
Cheers,
Penny Zheng

#14 lore raw

RE: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Hi Xenia

Quoted 198 lines

-----Original Message-----
From: Xenia Ragiadakou <[email protected]>
Sent: Monday, July 11, 2022 11:29 PM
To: Penny Zheng <[email protected]>; Stefano Stabellini <[email protected]>; Julien Grall <[email protected]>
Cc: [email protected]; Doug Goldstein <[email protected]>
Subject: Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Hi Penny,

Quoted 27 lines · Penny Zheng

Hi Xenia

Quoted 25 lines

-----Original Message-----
From: Xen-devel <[email protected]> On Behalf Of Xenia Ragiadakou
Sent: Friday, July 8, 2022 5:54 PM
To: Stefano Stabellini <[email protected]>; Julien Grall <[email protected]>
Cc: [email protected]; Doug Goldstein <[email protected]>
Subject: Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Hi Stefano,

Quoted 11 lines · Stefano Stabellini
Quoted 10 lines · Julien Grall

Hi Xenia,

Quoted 7 lines · Xenia Ragiadakou

Add an arm subdirectory under automation/configs for the arm specific configs and add a config that enables static allocation.

Modify the build script to search for configs also in this subdirectory and to keep the generated xen binary, suffixed with the config file name, as artifact.

Quoted 13 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Create a test job that
- boots xen on qemu with a single direct mapped dom0less guest configured with statically allocated memory

Although you said booting a single direct mapped dom0less guest configured with statically allocated memory here, later in code, you are only enabling statically allocated memory in the ImageBuilder script, missing the direct-map property.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

- verifies that the memory ranges reported in the guest's logs are the same with the provided static memory regions

For guest kernel, use the 5.9.9 kernel from the tests-artifacts

containers.

Quoted 30 lines
Quoted 30 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

Use busybox-static package, to create the guest ramdisk. To generate the u-boot script, use ImageBuilder. Use the qemu from the tests-artifacts containers.

---
    automation/configs/arm/static_mem          |   3 +
    automation/gitlab-ci/test.yaml             |  24 +++++
    automation/scripts/build                   |   4 +
    automation/scripts/qemu-staticmem-arm64.sh | 114

+++++++++++++++++++++

Quoted 19 lines
Quoted 19 lines
Quoted 11 lines
    4 files changed, 145 insertions(+)

create mode 100644 automation/configs/arm/static_mem create mode 100755 automation/scripts/qemu-staticmem-arm64.sh

automation/configs/arm/static_mem new file +0 -0
new file mode 100644
index 0000000000..84675ddf4e

automation/configs/arm/static_mem line 3

Quoted 4 lines
automation/configs/arm/static_mem new file +3 -0
@@ -0,0 +1,3 @@
+CONFIG_EXPERT=y
+CONFIG_UNSUPPORTED=y
+CONFIG_STATIC_MEMORY=y
\ No newline at end of file

Any particular reason to build a new Xen rather enable CONFIG_STATIC_MEMORY in the existing build?

Quoted 14 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
automation/scripts/build +0 -0
index 21b3bc57c8..9c6196d9bd 100755
@@ -83,6 +83,7 @@ fi
    # Build all the configs we care about
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
automation/scripts/build +1 -0
    case ${XEN_TARGET_ARCH} in
        x86_64) arch=x86 ;;
+    arm64) arch=arm ;;
        *) exit 0 ;;
    esac
    @@ -93,4 +94,7 @@ for cfg in `ls ${cfg_dir}`; do

rm -f xen/.config
make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg}

defconfig

Quoted 91 lines
Quoted 91 lines
Quoted 91 lines
Quoted 91 lines

automation/scripts/build for cfg in `ls ${cfg_dir}`; do · line 99

Quoted 4 lines
        make -j$(nproc) -C xen
automation/scripts/build +3 -0
+    if [[ ${arch} == "arm" ]]; then
+        cp xen/xen binaries/xen-${cfg}
+    fi

This feels a bit of a hack to be arm only. Can you explain why this is not enabled for x86 (other than this is not yet used)?

automation/scripts/qemu-staticmem-arm64.sh line 4

Quoted 12 lines

done

automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
new file mode 100755
index 0000000000..5b89a151aa
@@ -0,0 +1,114 @@
+#!/bin/bash
+
+base=(0x50000000 0x100000000)
+size=(0x10000000 0x10000000)

From the name, it is not clear what the base and size refers too.
Looking a bit below, it seems to be referring to the domain memory. If so, I would suggest to comment and rename to "domu_{base, size}".

Quoted 52 lines
automation/scripts/qemu-staticmem-arm64.sh new file +52 -0
+
+set -ex
+
+apt-get -qy update
+apt-get -qy install --no-install-recommends u-boot-qemu \
+                                            u-boot-tools \
+                                            device-tree-compiler \
+                                            cpio \
+                                            curl \
+                                            busybox-static
+
+# DomU Busybox
+cd binaries
+mkdir -p initrd
+mkdir -p initrd/bin
+mkdir -p initrd/sbin
+mkdir -p initrd/etc
+mkdir -p initrd/dev
+mkdir -p initrd/proc
+mkdir -p initrd/sys
+mkdir -p initrd/lib
+mkdir -p initrd/var
+mkdir -p initrd/mnt
+cp /bin/busybox initrd/bin/busybox initrd/bin/busybox --install
+initrd/bin echo "#!/bin/sh
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+mount -t devtmpfs devtmpfs /dev
+/bin/sh" > initrd/init
+chmod +x initrd/init
+cd initrd
+find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+cd ../..
+
+# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded curl
+-fsSLO
+https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
+
+./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -M dumpdtb=binaries/virt-gicv2.dtb
+
+#dtc -I dtb -O dts binaries/virt-gicv2.dtb >
+binaries/virt-gicv2.dts
+
+# ImageBuilder

automation/scripts/qemu-staticmem-arm64.sh line 65

Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+
+echo "MEMORY_START=\"0x40000000\"
+MEMORY_END=\"0x0200000000\"
+
+DEVICE_TREE=\"virt-gicv2.dtb\"
+
+XEN=\"xen-static_mem\"
+XEN_CMD=\"console=dtuart earlyprintk xsm=dummy\"

AFAIK, earlyprintk is not an option for Xen on Arm (at least). It is also not clear why you need to pass xsm=dummy.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh

Quoted 8 lines
automation/scripts/qemu-staticmem-arm64.sh new file +8 -0
+
+NUM_DOMUS=1
+DOMU_MEM[0]=512
+DOMU_VCPUS[0]=1
+DOMU_KERNEL[0]=\"Image\"
+DOMU_RAMDISK[0]=\"initrd.cpio.gz\"
+DOMU_CMD[0]=\"earlyprintk console=ttyAMA0\"
+DOMU_STATIC_MEM[0]=\"${base[0]} ${size[0]} ${base[1]}

${size[1]}\"

Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

automation/scripts/qemu-staticmem-arm64.sh line 75

Quoted 1 line
automation/scripts/qemu-staticmem-arm64.sh new file +1 -0
+

You would like to add DOMU_DIRECT_MAP[0] = 1 to enable direct-map.

The imagebuilder configuration option DOMU_DIRECT_MAP defaults to 1.

But I could add DOMU_DIRECT_MAP[0]=1 in the script to make it clearer.

Oh, sorry about that. Then everything is set clearly for me~~~

Quoted 84 lines
Quoted 84 lines
Quoted 59 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
+UBOOT_SOURCE=\"boot.source\"
+UBOOT_SCRIPT=\"boot.scr\"" > binaries/imagebuilder_config
+
+bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/
Quoted 8 lines
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh

Quoted 8 lines
automation/scripts/qemu-staticmem-arm64.sh new file +1 -0
+-c

binaries/imagebuilder_config

automation/scripts/qemu-staticmem-arm64.sh new file +6 -0
+
+# Run the test
+rm -f qemu-staticmem.serial
+set +e
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source
+0x40000000"| \ timeout -k 1 60 ./binaries/qemu-system-aarch64 -

nographic \

Quoted 46 lines
Quoted 12 lines
Quoted 12 lines
automation/scripts/qemu-staticmem-arm64.sh new file +9 -0
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -no-reboot \
+    -device virtio-net-pci,netdev=vnet0 -netdev
+user,id=vnet0,tftp=binaries

\

automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin \
+    -dtb ./binaries/virt-gicv2.dtb \
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh line 97

Quoted 3 lines
automation/scripts/qemu-staticmem-arm64.sh new file +3 -0
+    |& tee qemu-staticmem.serial
+
+set -e

A lot of the code above is duplicated from qemu-smoke-arm64.sh. I think it would be better to consolidate in a single script. Looking briefly throught the existing scripts, it looks like it is possible to pass arguments (see qemu-smoke-x86-64.sh).

One idea would be to make the script common and "source" a second script or config file with just the ImageBuilder configuration because it looks like it is the only thing different.

Quoted 19 lines
Quoted 4 lines
automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
+
+(grep -q "Xen dom0less mode detected" qemu-staticmem.serial) ||
+exit 1
+

automation/scripts/qemu-staticmem-arm64.sh line 110

Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+for ((i=0; i<${#base[@]}; i++))
+do
+    start="$(printf "0x%016x" ${base[$i]})"
+    end="$(printf "0x%016x" $((${base[$i]} + ${size[$i]} - 1)))"
+    grep -q "node   0: \[mem ${start}-${end}\]" qemu-staticmem.serial
+    if test $? -eq 1
+    then
+        exit 1
+    fi
+done

Please add a comment on top to explain what this is meant to do. However, I think we should avoid relying on output that we have written ourself. IOW, relying on Xen/Linux to always write the same message is risky because they can change at any time.

Quoted 7 lines
Quoted 3 lines

Especially if we make the script common, then we could just rely on the existing check to see if the guest started correctly (no special check for static memory).

In this case, how the test will verify that the static memory configuration has been taken into account and has not just been ignored?

If only statically allocated memory is enabled, guest memory address will still be mapped to GUEST_RAM_BASE(0x40000000)

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

automation/scripts/qemu-staticmem-arm64.sh

Quoted 2 lines
automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+
+(grep -q "BusyBox" qemu-staticmem.serial) || exit 1

--
Xenia

---
Cheers,
Penny Zheng

#15 lore raw

Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

In reply to Julien Grall

Hi Julien,

Quoted 32 lines · Julien Grall

Hi Xenia,

Quoted 29 lines · Xenia Ragiadakou

Add an arm subdirectory under automation/configs for the arm specific configs and add a config that enables static allocation.

Modify the build script to search for configs also in this subdirectory and to keep the generated xen binary, suffixed with the config file name, as artifact.

Create a test job that
- boots xen on qemu with a single direct mapped dom0less guest configured with
statically allocated memory
- verifies that the memory ranges reported in the guest's logs are the same
with the provided static memory regions

For guest kernel, use the 5.9.9 kernel from the tests-artifacts containers. Use busybox-static package, to create the guest ramdisk.
To generate the u-boot script, use ImageBuilder.
Use the qemu from the tests-artifacts containers.

---
  automation/configs/arm/static_mem          |   3 + automation/gitlab-ci/test.yaml             |  24 +++++ automation/scripts/build                   |   4 + automation/scripts/qemu-staticmem-arm64.sh | 114 +++++++++++++++++++++

  4 files changed, 145 insertions(+)

  create mode 100644 automation/configs/arm/static_mem create mode 100755 automation/scripts/qemu-staticmem-arm64.sh

Quoted 12 lines

automation/configs/arm/static_mem line 3

Quoted 10 lines
automation/configs/arm/static_mem new file +3 -0
new file mode 100644
index 0000000000..84675ddf4e
@@ -0,0 +1,3 @@
+CONFIG_EXPERT=y
+CONFIG_UNSUPPORTED=y
+CONFIG_STATIC_MEMORY=y
\ No newline at end of file

Any particular reason to build a new Xen rather enable CONFIG_STATIC_MEMORY in the existing build

IIUC, the xen binary (built with the arm64_defconfig) is used by the two other arm64 test jobs qemu-smoke-arm64-gcc and qemu-alpine-arm64-gcc. I did not want to change its configuration.

If there is no issue with changing its configuration, I can enable static memory and use this one. But to be honest, I would like to be able also to test with custom configs.

Quoted 12 lines
Quoted 12 lines
automation/scripts/build +0 -0
index 21b3bc57c8..9c6196d9bd 100755
@@ -83,6 +83,7 @@ fi
   # Build all the configs we care about

  case ${XEN_TARGET_ARCH} in
      x86_64) arch=x86 ;;
+    arm64) arch=arm ;;
      *) exit 0 ;;
  esac

automation/scripts/build +0 -0
@@ -93,4 +94,7 @@ for cfg in `ls ${cfg_dir}`; do
Quoted 8 lines
Quoted 6 lines

      rm -f xen/.config
      make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} defconfig make -j$(nproc) -C xen
+    if [[ ${arch} == "arm" ]]; then
+        cp xen/xen binaries/xen-${cfg}
+    fi

This feels a bit of a hack to be arm only. Can you explain why this is not enabled for x86 (other than this is not yet used)?

I did not want to change the existing behavior for x86.

Quoted 6 lines
Quoted 6 lines

  done

automation/scripts/qemu-staticmem-arm64.sh new file +0 -0
new file mode 100755
index 0000000000..5b89a151aa
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh line 4

Quoted 5 lines
automation/scripts/qemu-staticmem-arm64.sh new file +4 -0
@@ -0,0 +1,114 @@
+#!/bin/bash
+
+base=(0x50000000 0x100000000)
+size=(0x10000000 0x10000000)

From the name, it is not clear what the base and size refers too.
Looking a bit below, it seems to be referring to the domain memory. If so, I would suggest to comment and rename to "domu_{base, size}".

They are the static memory regions allocated to the domu.

Quoted 56 lines
Quoted 56 lines
automation/scripts/qemu-staticmem-arm64.sh new file +38 -0
+
+set -ex
+
+apt-get -qy update
+apt-get -qy install --no-install-recommends u-boot-qemu \
+                                            u-boot-tools \
+                                            device-tree-compiler \
+                                            cpio \
+                                            curl \
+                                            busybox-static
+
+# DomU Busybox
+cd binaries
+mkdir -p initrd
+mkdir -p initrd/bin
+mkdir -p initrd/sbin
+mkdir -p initrd/etc
+mkdir -p initrd/dev
+mkdir -p initrd/proc
+mkdir -p initrd/sys
+mkdir -p initrd/lib
+mkdir -p initrd/var
+mkdir -p initrd/mnt
+cp /bin/busybox initrd/bin/busybox
+initrd/bin/busybox --install initrd/bin
+echo "#!/bin/sh
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+mount -t devtmpfs devtmpfs /dev
+/bin/sh" > initrd/init
+chmod +x initrd/init
+cd initrd
+find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+cd ../..
+
+# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
+curl -fsSLO 

https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom

automation/scripts/qemu-staticmem-arm64.sh new file +17 -0
+
+./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -M dumpdtb=binaries/virt-gicv2.dtb
+
+#dtc -I dtb -O dts binaries/virt-gicv2.dtb > binaries/virt-gicv2.dts
+
+# ImageBuilder
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+
+echo "MEMORY_START=\"0x40000000\"
Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh line 65

Quoted 6 lines
automation/scripts/qemu-staticmem-arm64.sh new file +6 -0
+MEMORY_END=\"0x0200000000\"
+
+DEVICE_TREE=\"virt-gicv2.dtb\"
+
+XEN=\"xen-static_mem\"
+XEN_CMD=\"console=dtuart earlyprintk xsm=dummy\"

AFAIK, earlyprintk is not an option for Xen on Arm (at least). It is also not clear why you need to pass xsm=dummy.

It is not clear to me either :). I will remove them.

Quoted 29 lines
Quoted 29 lines
automation/scripts/qemu-staticmem-arm64.sh new file +13 -0
+
+NUM_DOMUS=1
+DOMU_MEM[0]=512
+DOMU_VCPUS[0]=1
+DOMU_KERNEL[0]=\"Image\"
+DOMU_RAMDISK[0]=\"initrd.cpio.gz\"
+DOMU_CMD[0]=\"earlyprintk console=ttyAMA0\"
+DOMU_STATIC_MEM[0]=\"${base[0]} ${size[0]} ${base[1]} ${size[1]}\"
+
+UBOOT_SOURCE=\"boot.source\"
+UBOOT_SCRIPT=\"boot.scr\"" > binaries/imagebuilder_config
+
+bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c 

binaries/imagebuilder_config

automation/scripts/qemu-staticmem-arm64.sh new file +5 -0
+
+# Run the test
+rm -f qemu-staticmem.serial
+set +e
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 

0x40000000"| \
+timeout -k 1 60 ./binaries/qemu-system-aarch64 -nographic \
+    -M virtualization=true \
+    -M virt \
+    -M virt,gic-version=2 \
+    -cpu cortex-a57 \
+    -smp 2 \
+    -m 8G \
+    -no-reboot \
+    -device virtio-net-pci,netdev=vnet0 -netdev

Quoted 8 lines

automation/scripts/qemu-staticmem-arm64.sh

Quoted 6 lines

user,id=vnet0,tftp=binaries \
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin \
+    -dtb ./binaries/virt-gicv2.dtb \
+    |& tee qemu-staticmem.serial

automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+
+set -e

A lot of the code above is duplicated from qemu-smoke-arm64.sh. I think it would be better to consolidate in a single script. Looking briefly throught the existing scripts, it looks like it is possible to pass arguments (see qemu-smoke-x86-64.sh).

Yes, if there is no issue with changing qemu-smoke-arm64.sh, this is a very good idea.

Quoted 7 lines
Quoted 7 lines
automation/scripts/qemu-staticmem-arm64.sh new file +7 -0
+
+(grep -q "Xen dom0less mode detected" qemu-staticmem.serial) || exit 1
+
+for ((i=0; i<${#base[@]}; i++))
+do
+    start="$(printf "0x%016x" ${base[$i]})"
+    end="$(printf "0x%016x" $((${base[$i]} + ${size[$i]} - 1)))"
Quoted 8 lines
Quoted 6 lines

+    grep -q "node   0: \[mem ${start}-${end}\]" qemu-staticmem.serial +    if test $? -eq 1
+    then
+        exit 1
+    fi
+done

Please add a comment on top to explain what this is meant to do. However, I think we should avoid relying on output that we have written ourself. IOW, relying on Xen/Linux to always write the same message is risky because they can change at any time.

The kernel is taken from a test-artifact container, so, IIUC, it won't change.

Quoted 5 lines

automation/scripts/qemu-staticmem-arm64.sh

Quoted 2 lines
automation/scripts/qemu-staticmem-arm64.sh new file +2 -0
+
+(grep -q "BusyBox" qemu-staticmem.serial) || exit 1

Cheers,

Signature · 1 line
Xenia
#16 lore raw

Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Quoted 1 line · Xenia Ragiadakou

Hi Julien,

Hi Xenia,

Quoted 46 lines · Julien Grall
Quoted 32 lines

Hi Xenia,

Quoted 29 lines · Xenia Ragiadakou

Add an arm subdirectory under automation/configs for the arm specific configs and add a config that enables static allocation.

Modify the build script to search for configs also in this subdirectory and to keep the generated xen binary, suffixed with the config file name, as artifact.

Create a test job that
- boots xen on qemu with a single direct mapped dom0less guest configured with
statically allocated memory
- verifies that the memory ranges reported in the guest's logs are the same
with the provided static memory regions

For guest kernel, use the 5.9.9 kernel from the tests-artifacts containers. Use busybox-static package, to create the guest ramdisk.
To generate the u-boot script, use ImageBuilder.
Use the qemu from the tests-artifacts containers.

---
  automation/configs/arm/static_mem          |   3 + automation/gitlab-ci/test.yaml             |  24 +++++ automation/scripts/build                   |   4 + automation/scripts/qemu-staticmem-arm64.sh | 114 +++++++++++++++++++++

  4 files changed, 145 insertions(+)

  create mode 100644 automation/configs/arm/static_mem create mode 100755 automation/scripts/qemu-staticmem-arm64.sh

Quoted 12 lines

automation/configs/arm/static_mem line 3

Quoted 10 lines
automation/configs/arm/static_mem new file +3 -0
new file mode 100644
index 0000000000..84675ddf4e
@@ -0,0 +1,3 @@
+CONFIG_EXPERT=y
+CONFIG_UNSUPPORTED=y
+CONFIG_STATIC_MEMORY=y
\ No newline at end of file

Any particular reason to build a new Xen rather enable CONFIG_STATIC_MEMORY in the existing build

Quoted 3 lines

IIUC, the xen binary (built with the arm64_defconfig) is used by the two other arm64 test jobs qemu-smoke-arm64-gcc and qemu-alpine-arm64-gcc. I did not want to change its configuration.

If there is no issue with changing its configuration, I can enable static memory and use this one.

I would expect a Xen built to CONFIG_STATIC_MEMORY to continue to work in normal case. So it should be fine to enable by default.

Quoted 1 line

But to be honest, I would like to be able also to test with custom configs.

That's fine. But in this case...

Quoted 22 lines
Quoted 13 lines
Quoted 13 lines
automation/scripts/build +0 -0
index 21b3bc57c8..9c6196d9bd 100755
@@ -83,6 +83,7 @@ fi
   # Build all the configs we care about

  case ${XEN_TARGET_ARCH} in
      x86_64) arch=x86 ;;
+    arm64) arch=arm ;;
      *) exit 0 ;;
  esac

automation/scripts/build +0 -0
@@ -93,4 +94,7 @@ for cfg in `ls ${cfg_dir}`; do
Quoted 8 lines
Quoted 6 lines

      rm -f xen/.config
      make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} defconfig make -j$(nproc) -C xen
+    if [[ ${arch} == "arm" ]]; then
+        cp xen/xen binaries/xen-${cfg}
+    fi

This feels a bit of a hack to be arm only. Can you explain why this is not enabled for x86 (other than this is not yet used)?

Quoted 1 line

I did not want to change the existing behavior for x86.

... I don't think this should be restricted to arm. I would also consider to do this change separately to avoid mixing infrastructure change and new test.

[...]

Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
automation/scripts/build +5 -0
+# ImageBuilder
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+
+echo "MEMORY_START=\"0x40000000\"
Quoted 8 lines

automation/scripts/build

Quoted 6 lines
automation/scripts/build +6 -0
+MEMORY_END=\"0x0200000000\"
+
+DEVICE_TREE=\"virt-gicv2.dtb\"
+
+XEN=\"xen-static_mem\"
+XEN_CMD=\"console=dtuart earlyprintk xsm=dummy\"

AFAIK, earlyprintk is not an option for Xen on Arm (at least). It is also not clear why you need to pass xsm=dummy.

Quoted 1 line

It is not clear to me either :). I will remove them.

Where was this command line copied from? If it is an Arm documentation (or script), then they should be corrected.

Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
automation/scripts/build +7 -0
+
+(grep -q "Xen dom0less mode detected" qemu-staticmem.serial) || exit 1
+
+for ((i=0; i<${#base[@]}; i++))
+do
+    start="$(printf "0x%016x" ${base[$i]})"
+    end="$(printf "0x%016x" $((${base[$i]} + ${size[$i]} - 1)))"
Quoted 8 lines
Quoted 6 lines

+    grep -q "node   0: \[mem ${start}-${end}\]" qemu-staticmem.serial +    if test $? -eq 1
+    then
+        exit 1
+    fi
+done

Please add a comment on top to explain what this is meant to do. However, I think we should avoid relying on output that we have written ourself. IOW, relying on Xen/Linux to always write the same message is risky because they can change at any time.

Quoted 1 line

The kernel is taken from a test-artifact container, so, IIUC, it won't change.

This statement is correct today. However, we may decide to update the kernel or test multiple kernels (with different ouput).

In the first case, it would be a matter of updating the script. This is annoying but not too bad. In the second case, we would need to have "if version X ... else if version Y ... ".

Cheers,

Signature · 1 line
Julien Grall
#17 lore raw

Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Quoted 86 lines · Julien Grall
Quoted 1 line · Xenia Ragiadakou

Hi Julien,

Hi Xenia,

Quoted 47 lines · Julien Grall
Quoted 33 lines

Hi Xenia,

Quoted 30 lines · Xenia Ragiadakou

Add an arm subdirectory under automation/configs for the arm specific configs and add a config that enables static allocation.

Modify the build script to search for configs also in this subdirectory and to keep the generated xen binary, suffixed with the config file name, as artifact.

Create a test job that
- boots xen on qemu with a single direct mapped dom0less guest configured with
statically allocated memory
- verifies that the memory ranges reported in the guest's logs are the same
with the provided static memory regions

For guest kernel, use the 5.9.9 kernel from the tests-artifacts containers. Use busybox-static package, to create the guest ramdisk.
To generate the u-boot script, use ImageBuilder.
Use the qemu from the tests-artifacts containers.

---
  automation/configs/arm/static_mem          |   3 + automation/gitlab-ci/test.yaml             |  24 +++++ automation/scripts/build                   |   4 + automation/scripts/qemu-staticmem-arm64.sh | 114
+++++++++++++++++++++

  4 files changed, 145 insertions(+)

  create mode 100644 automation/configs/arm/static_mem create mode 100755 automation/scripts/qemu-staticmem-arm64.sh

Quoted 12 lines

automation/configs/arm/static_mem line 3

Quoted 10 lines
automation/configs/arm/static_mem new file +3 -0
new file mode 100644
index 0000000000..84675ddf4e
@@ -0,0 +1,3 @@
+CONFIG_EXPERT=y
+CONFIG_UNSUPPORTED=y
+CONFIG_STATIC_MEMORY=y
\ No newline at end of file

Any particular reason to build a new Xen rather enable CONFIG_STATIC_MEMORY in the existing build

Quoted 3 lines

IIUC, the xen binary (built with the arm64_defconfig) is used by the two other arm64 test jobs qemu-smoke-arm64-gcc and qemu-alpine-arm64-gcc. I did not want to change its configuration.

If there is no issue with changing its configuration, I can enable static memory and use this one.

I would expect a Xen built to CONFIG_STATIC_MEMORY to continue to work in normal case. So it should be fine to enable by default.

Quoted 1 line

But to be honest, I would like to be able also to test with custom configs.

That's fine. But in this case...

Quoted 22 lines
Quoted 21 lines
Quoted 13 lines
automation/scripts/build +0 -0
index 21b3bc57c8..9c6196d9bd 100755
@@ -83,6 +83,7 @@ fi
   # Build all the configs we care about

  case ${XEN_TARGET_ARCH} in
      x86_64) arch=x86 ;;
+    arm64) arch=arm ;;
      *) exit 0 ;;
  esac

automation/scripts/build +0 -0
@@ -93,4 +94,7 @@ for cfg in `ls ${cfg_dir}`; do
Quoted 6 lines

      rm -f xen/.config
      make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} defconfig make -j$(nproc) -C xen
+    if [[ ${arch} == "arm" ]]; then
+        cp xen/xen binaries/xen-${cfg}
+    fi

This feels a bit of a hack to be arm only. Can you explain why this is not enabled for x86 (other than this is not yet used)?

Quoted 4 lines
Quoted 1 line

I did not want to change the existing behavior for x86.

... I don't think this should be restricted to arm. I would also consider to do this change separately to avoid mixing infrastructure change and new test.

Sure.

Quoted 17 lines

[...]

Quoted 14 lines
Quoted 13 lines

automation/scripts/build

Quoted 11 lines
automation/scripts/build +11 -0
+# ImageBuilder
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+
+echo "MEMORY_START=\"0x40000000\"
+MEMORY_END=\"0x0200000000\"
+
+DEVICE_TREE=\"virt-gicv2.dtb\"
+
+XEN=\"xen-static_mem\"
+XEN_CMD=\"console=dtuart earlyprintk xsm=dummy\"

AFAIK, earlyprintk is not an option for Xen on Arm (at least). It is also not clear why you need to pass xsm=dummy.

Quoted 3 lines
Quoted 1 line

It is not clear to me either :). I will remove them.

Where was this command line copied from? If it is an Arm documentation (or script), then they should be corrected.

Don't worry :) I was using them when debugging a script for parsing the xen cmdline and I dragged them around. Sorry about that.

Quoted 16 lines
Quoted 16 lines
Quoted 15 lines

automation/scripts/build

Quoted 13 lines
automation/scripts/build +13 -0
+
+(grep -q "Xen dom0less mode detected" qemu-staticmem.serial) || exit 1
+
+for ((i=0; i<${#base[@]}; i++))
+do
+    start="$(printf "0x%016x" ${base[$i]})"
+    end="$(printf "0x%016x" $((${base[$i]} + ${size[$i]} - 1)))"
+    grep -q "node   0: \[mem ${start}-${end}\]" qemu-staticmem.serial
+    if test $? -eq 1
+    then
+        exit 1
+    fi
+done

Please add a comment on top to explain what this is meant to do. However, I think we should avoid relying on output that we have written ourself. IOW, relying on Xen/Linux to always write the same message is risky because they can change at any time.

Quoted 5 lines
Quoted 1 line

The kernel is taken from a test-artifact container, so, IIUC, it won't change.

This statement is correct today. However, we may decide to update the kernel or test multiple kernels (with different ouput).

In the first case, it would be a matter of updating the script. This is annoying but not too bad. In the second case, we would need to have "if version X ... else if version Y ... ".

The particular test was relying and had a dependency on this kernel.
If the test is merged into the qemu-smoke-arm64.sh, the check above will leave and it will be tested whether the guest makes it to the busybox, based on the busybox logs, which also may change at any time.

Signature · 1 line
Xenia
#18 lore raw

Re: [PATCH 2/2] automation: arm64: Create a test job for testing static allocation on qemu

Quoted 17 lines · Xenia Ragiadakou
Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
Quoted 4 lines
(quoted diff) +2 -0
+
+(grep -q "Xen dom0less mode detected" qemu-staticmem.serial) || 

exit 1

automation/scripts/qemu-staticmem-arm64.sh new file +1 -0
+

automation/scripts/qemu-staticmem-arm64.sh line 110

Quoted 10 lines
automation/scripts/qemu-staticmem-arm64.sh new file +10 -0
+for ((i=0; i<${#base[@]}; i++))
+do
+    start="$(printf "0x%016x" ${base[$i]})"
+    end="$(printf "0x%016x" $((${base[$i]} + ${size[$i]} - 1)))"
+    grep -q "node   0: \[mem ${start}-${end}\]" qemu-staticmem.serial
+    if test $? -eq 1
+    then
+        exit 1
+    fi
+done

Please add a comment on top to explain what this is meant to do. However, I think we should avoid relying on output that we have written ourself. IOW, relying on Xen/Linux to always write the same message is risky because they can change at any time.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

The kernel is taken from a test-artifact container, so, IIUC, it won't change.

This statement is correct today. However, we may decide to update the kernel or test multiple kernels (with different ouput).

In the first case, it would be a matter of updating the script. This is annoying but not too bad. In the second case, we would need to have "if version X ... else if version Y ... ".

The particular test was relying and had a dependency on this kernel.

I think you missed my point. I don't disagree that the test today expects a specific version. However, there are nothing preventing us to change that so we long we have a matching initramfs/kernel.

Quoted 1 line

If the test is merged into the qemu-smoke-arm64.sh, the check above will leave and it will be tested whether the guest makes it to the busybox, based on the busybox logs, which also may change at any time.

I don't think I suggested that relying on busybox is better. Ideally we should run a script that prints a message.

But this is not something I am going to argue for in this patch. Relying on one piece of line is already far better than trying to check logs for each components (e.g. xen, kernel, busysbox).

Cheers,

Signature · 1 line
Julien Grall