Skip to contentSkip to conversation

arm64: runtime scripts improvements on efi

PATCH Superseded Shaoqin Huang 2 participants 12 messages Nov 29, 2023 – Nov 30, 2023

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover arm64: runtime scripts improvements on efi

When I run the arm64 tests on efi, I found several runtime scripts issues. This patch series try to fix all the issues.

The first patch add a missing error reporting.

The second patch skip the migration tests when run on efi since it will always fail.

The thrid patch fix the issue when parallel running tests on efi.

Shaoqin Huang (3):
runtime: Fix the missing last_line
runtime: arm64: Skip the migration tests when run on EFI
arm64: efi: Make running tests on EFI can be parallel

 arm/efi/run          | 16 +++++++++-------
 scripts/runtime.bash |  7 +++++++
 2 files changed, 16 insertions(+), 7 deletions(-)
Signature · 1 line
2.40.1
#1 lore raw

PATCH 1/3 runtime: Fix the missing last_line

The last_line is deleted by the 2607d2d6 ("arm64: Add an efi/run script"). This lead to when SKIP test, the reason is missing. Fix the problem by adding last_line back.

---
 scripts/runtime.bash | 2 ++
 1 file changed, 2 insertions(+)
scripts/runtime.bash +2 -0
index fc156f2f..d7054b80 100644
@@ -148,6 +148,8 @@ function run()
             fi
         fi
 
+	last_line=$(tail -1 <<<"$log")
+
         if [ ${skip} == true ]; then
             print_result "SKIP" $testname "" "$last_line"
             return 77
Signature · 1 line
2.40.1
#2 lore raw

Re: [kvm-unit-tests PATCH v1 1/3] runtime: Fix the missing last_line

Quoted 5 lines · Shaoqin Huang

The last_line is deleted by the 2607d2d6 ("arm64: Add an efi/run script"). This lead to when SKIP test, the reason is missing. Fix the problem by adding last_line back.

Fixes 2607d2d6 ("arm64: Add an efi/run script")
Quoted 9 lines
---
 scripts/runtime.bash | 2 ++
 1 file changed, 2 insertions(+)
scripts/runtime.bash +0 -0
index fc156f2f..d7054b80 100644

scripts/runtime.bash function run() · line 154

Quoted 7 lines
scripts/runtime.bash +2 -0
@@ -148,6 +148,8 @@ function run()
             fi
         fi
 
+	last_line=$(tail -1 <<<"$log")
+
         if [ ${skip} == true ]; then
             print_result "SKIP" $testname "" "$last_line"

We can just change this one use of $last_line to $(tail -1 <<<"$log")

Quoted 4 lines
scripts/runtime.bash +0 -0
             return 77
Signature · 1 line
2.40.1

Thanks,
drew

#3 lore raw

Re: [kvm-unit-tests PATCH v1 1/3] runtime: Fix the missing last_line

Hi drew,

Quoted 9 lines · Andrew Jones
Quoted 5 lines · Shaoqin Huang

The last_line is deleted by the 2607d2d6 ("arm64: Add an efi/run script"). This lead to when SKIP test, the reason is missing. Fix the problem by adding last_line back.

Fixes 2607d2d6 ("arm64: Add an efi/run script")

Will add it.

Quoted 12 lines
Quoted 12 lines
---
  scripts/runtime.bash | 2 ++
  1 file changed, 2 insertions(+)
scripts/runtime.bash +0 -0
index fc156f2f..d7054b80 100644
@@ -148,6 +148,8 @@ function run()
              fi
          fi
 
Quoted 7 lines

scripts/runtime.bash function run() · line 154

Quoted 4 lines
scripts/runtime.bash +2 -0
+	last_line=$(tail -1 <<<"$log")
+
          if [ ${skip} == true ]; then
              print_result "SKIP" $testname "" "$last_line"

We can just change this one use of $last_line to $(tail -1 <<<"$log")

Got it. Looks good.

Thanks,
Shaoqin

Quoted 8 lines
Quoted 4 lines
scripts/runtime.bash +0 -0
              return 77
Signature · 1 line
2.40.1

Thanks,
drew

Signature · 1 line
Shaoqin
#4 lore raw

PATCH 2/3 runtime: arm64: Skip the migration tests when run on EFI

In reply to Shaoqin Huang

When running the migration tests on EFI, the migration will always fail since the efi/run use the vvfat format to run test, but the vvfat format does not support live migration. So those migration tests will always fail.

Instead of waiting for fail everytime when run migration tests on EFI, skip those tests if running on EFI.

---
 scripts/runtime.bash | 5 +++++
 1 file changed, 5 insertions(+)
scripts/runtime.bash +5 -0
index d7054b80..b7105c19 100644
@@ -156,6 +156,11 @@ function run()
         fi
     }
 
+    if [ "${CONFIG_EFI}" == "y" ] && find_word "migration" "$groups"; then
+        print_result "SKIP" $testname "" "migration test is not support in efi"
+        return 2
+    fi
+
     cmdline=$(get_cmdline $kernel)
     if find_word "migration" "$groups"; then
         cmdline="MIGRATION=yes $cmdline"
Signature · 1 line
2.40.1
#5 lore raw

Re: [kvm-unit-tests PATCH v1 2/3] runtime: arm64: Skip the migration tests when run on EFI

Quoted 18 lines · Shaoqin Huang

When running the migration tests on EFI, the migration will always fail since the efi/run use the vvfat format to run test, but the vvfat format does not support live migration. So those migration tests will always fail.

Instead of waiting for fail everytime when run migration tests on EFI, skip those tests if running on EFI.

---
 scripts/runtime.bash | 5 +++++
 1 file changed, 5 insertions(+)
scripts/runtime.bash +0 -0
index d7054b80..b7105c19 100644

scripts/runtime.bash function run() · line 160

Quoted 5 lines
scripts/runtime.bash +2 -0
@@ -156,6 +156,11 @@ function run()
         fi
     }
 
+    if [ "${CONFIG_EFI}" == "y" ] && find_word "migration" "$groups"; then
+        print_result "SKIP" $testname "" "migration test is not support in efi"

migration tests are not supported with efi

scripts/runtime.bash function run() · line 166

Quoted 6 lines
scripts/runtime.bash +3 -0
+        return 2
+    fi
+
     cmdline=$(get_cmdline $kernel)
     if find_word "migration" "$groups"; then
         cmdline="MIGRATION=yes $cmdline"

We don't need to do the find_word twice,

cmdline=$(get_cmdline $kernel)
if find_word "migration" "$groups"; then
if [ "${CONFIG_EFI}" == "y" ]; then
print_result "SKIP" $testname "" "migration tests are not supported with efi" return 2
fi
cmdline="MIGRATION=yes $cmdline"

Quoted 3 lines
Signature · 1 line
2.40.1

Thanks,
drew

#6 lore raw

Re: [kvm-unit-tests PATCH v1 2/3] runtime: arm64: Skip the migration tests when run on EFI

Hi drew,

Quoted 37 lines · Andrew Jones
Quoted 18 lines · Shaoqin Huang

When running the migration tests on EFI, the migration will always fail since the efi/run use the vvfat format to run test, but the vvfat format does not support live migration. So those migration tests will always fail.

Instead of waiting for fail everytime when run migration tests on EFI, skip those tests if running on EFI.

---
  scripts/runtime.bash | 5 +++++
  1 file changed, 5 insertions(+)
scripts/runtime.bash +0 -0
index d7054b80..b7105c19 100644

scripts/runtime.bash function run() · line 160

Quoted 5 lines
scripts/runtime.bash +2 -0
@@ -156,6 +156,11 @@ function run()
          fi
      }
 
+    if [ "${CONFIG_EFI}" == "y" ] && find_word "migration" "$groups"; then
+        print_result "SKIP" $testname "" "migration test is not support in efi"

migration tests are not supported with efi

scripts/runtime.bash function run() · line 166

Quoted 6 lines
scripts/runtime.bash +3 -0
+        return 2
+    fi
+
      cmdline=$(get_cmdline $kernel)
      if find_word "migration" "$groups"; then
          cmdline="MIGRATION=yes $cmdline"

We don't need to do the find_word twice,

Quoted 8 lines

cmdline=$(get_cmdline $kernel)
if find_word "migration" "$groups"; then
if [ "${CONFIG_EFI}" == "y" ]; then
print_result "SKIP" $testname "" "migration tests are not supported with efi" return 2
fi
cmdline="MIGRATION=yes $cmdline"

Thanks for your better solution. I can do that.

Thanks,
Shaoqin

Quoted 7 lines
Quoted 3 lines
Signature · 1 line
2.40.1

Thanks,
drew

Signature · 1 line
Shaoqin
#7 lore raw

PATCH 3/3 arm64: efi: Make running tests on EFI can be parallel

In reply to Shaoqin Huang

Currently running tests on EFI in parallel can cause part of tests to fail, this is because arm/efi/run script use the EFI_CASE to create the subdir under the efi-tests, and the EFI_CASE is the filename of the test, when running tests in parallel, the multiple tests exist in the same filename will execute at the same time, which will use the same directory and write the test specific things into it, this cause chaotic and make some tests fail.

To Fix this things, use the testname instead of the filename to create the subdir under the efi-tests. We use the EFI_TESTNAME to replace the EFI_CASE in script. Since every testname is specific, now the tests can be run parallel. It also considers when user directly use the arm/efi/run to run test, in this case, still use the filename.

Besides, replace multiple $EFI_TEST/$EFI_CASE to the $EFI_CASE_DIR, this makes the script looks more clean and we don'e need to replace many EFI_CASE to EFI_TESTNAME.

---
 arm/efi/run | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)
arm/efi/run +9 -7
index 6872c337..03bfbef4 100755
@@ -24,6 +24,8 @@ fi
 : "${EFI_SRC:=$TEST_DIR}"
 : "${EFI_UEFI:=$DEFAULT_UEFI}"
 : "${EFI_TEST:=efi-tests}"
+: "${EFI_TESTNAME:=$TESTNAME}"
+: "${EFI_TESTNAME:=$(basename $1 .efi)}"
 : "${EFI_CASE:=$(basename $1 .efi)}"
 : "${EFI_VAR_GUID:=97ef3e03-7329-4a6a-b9ba-6c1fdcc5f823}"
 
@@ -56,20 +58,20 @@ if [ "$EFI_CASE" = "_NO_FILE_4Uhere_" ]; then
 	EFI_CASE=dummy
 fi
 
-: "${EFI_CASE_DIR:="$EFI_TEST/$EFI_CASE"}"
+: "${EFI_CASE_DIR:="$EFI_TEST/$EFI_TESTNAME"}"
 mkdir -p "$EFI_CASE_DIR"
 
-cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_TEST/$EFI_CASE/"
-echo "@echo -off" > "$EFI_TEST/$EFI_CASE/startup.nsh"
+cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_CASE_DIR/"
+echo "@echo -off" > "$EFI_CASE_DIR/startup.nsh"
 if [ "$EFI_USE_DTB" = "y" ]; then
 	qemu_args+=(-machine acpi=off)
 	FDT_BASENAME="dtb"
-	$(EFI_RUN=y $TEST_DIR/run -machine dumpdtb="$EFI_TEST/$EFI_CASE/$FDT_BASENAME" "${qemu_args[@]}")
-	echo "setvar fdtfile -guid $EFI_VAR_GUID -rt =L\"$FDT_BASENAME\""  >> "$EFI_TEST/$EFI_CASE/startup.nsh"
+	$(EFI_RUN=y $TEST_DIR/run -machine dumpdtb="$EFI_CASE_DIR/$FDT_BASENAME" "${qemu_args[@]}")
+	echo "setvar fdtfile -guid $EFI_VAR_GUID -rt =L\"$FDT_BASENAME\""  >> "$EFI_CASE_DIR/startup.nsh"
 fi
-echo "$EFI_CASE.efi" "${cmd_args[@]}" >> "$EFI_TEST/$EFI_CASE/startup.nsh"
+echo "$EFI_CASE.efi" "${cmd_args[@]}" >> "$EFI_CASE_DIR/startup.nsh"
 
 EFI_RUN=y $TEST_DIR/run \
        -bios "$EFI_UEFI" \
-       -drive file.dir="$EFI_TEST/$EFI_CASE/",file.driver=vvfat,file.rw=on,format=raw,if=virtio \
+       -drive file.dir="$EFI_CASE_DIR/",file.driver=vvfat,file.rw=on,format=raw,if=virtio \
        "${qemu_args[@]}"
Signature · 1 line
2.40.1
#8 lore raw

Re: [kvm-unit-tests PATCH v1 3/3] arm64: efi: Make running tests on EFI can be parallel

Quoted 7 lines · Shaoqin Huang

Currently running tests on EFI in parallel can cause part of tests to fail, this is because arm/efi/run script use the EFI_CASE to create the subdir under the efi-tests, and the EFI_CASE is the filename of the test, when running tests in parallel, the multiple tests exist in the same filename will execute at the same time, which will use the same directory and write the test specific things into it, this cause chaotic and make some tests fail.

How do they fail? iiuc, we're switching from having one of each unique binary on the efi partition to multiple redundant binaries, since we copy the binary for each test, even when it's the same as for other tests. It seems like we should be able to keep single unique binaries and resolve the parallel execution failure by just checking for existence of the binaries or only creating test-specific data directories or something.

Quoted 31 lines

To Fix this things, use the testname instead of the filename to create the subdir under the efi-tests. We use the EFI_TESTNAME to replace the EFI_CASE in script. Since every testname is specific, now the tests can be run parallel. It also considers when user directly use the arm/efi/run to run test, in this case, still use the filename.

Besides, replace multiple $EFI_TEST/$EFI_CASE to the $EFI_CASE_DIR, this makes the script looks more clean and we don'e need to replace many EFI_CASE to EFI_TESTNAME.

---
 arm/efi/run | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)
arm/efi/run +2 -0
index 6872c337..03bfbef4 100755
@@ -24,6 +24,8 @@ fi
 : "${EFI_SRC:=$TEST_DIR}"
 : "${EFI_UEFI:=$DEFAULT_UEFI}"
 : "${EFI_TEST:=efi-tests}"
+: "${EFI_TESTNAME:=$TESTNAME}"
+: "${EFI_TESTNAME:=$(basename $1 .efi)}"
 : "${EFI_CASE:=$(basename $1 .efi)}"
 : "${EFI_VAR_GUID:=97ef3e03-7329-4a6a-b9ba-6c1fdcc5f823}"
 
@@ -56,20 +58,20 @@ if [ "$EFI_CASE" = "_NO_FILE_4Uhere_" ]; then
 	EFI_CASE=dummy

arm/efi/run if [ "$EFI_CASE" = "_NO_FILE_4Uhere_" ]; then · line 65

Quoted 10 lines
arm/efi/run +3 -3
 fi
 
-: "${EFI_CASE_DIR:="$EFI_TEST/$EFI_CASE"}"
+: "${EFI_CASE_DIR:="$EFI_TEST/$EFI_TESTNAME"}"
 mkdir -p "$EFI_CASE_DIR"
 
-cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_TEST/$EFI_CASE/"
-echo "@echo -off" > "$EFI_TEST/$EFI_CASE/startup.nsh"
+cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_CASE_DIR/"
+echo "@echo -off" > "$EFI_CASE_DIR/startup.nsh"

Unrelated change, should be a separate patch.

Quoted 19 lines
arm/efi/run +4 -4
 if [ "$EFI_USE_DTB" = "y" ]; then
 	qemu_args+=(-machine acpi=off)
 	FDT_BASENAME="dtb"
-	$(EFI_RUN=y $TEST_DIR/run -machine dumpdtb="$EFI_TEST/$EFI_CASE/$FDT_BASENAME" "${qemu_args[@]}")
-	echo "setvar fdtfile -guid $EFI_VAR_GUID -rt =L\"$FDT_BASENAME\""  >> "$EFI_TEST/$EFI_CASE/startup.nsh"
+	$(EFI_RUN=y $TEST_DIR/run -machine dumpdtb="$EFI_CASE_DIR/$FDT_BASENAME" "${qemu_args[@]}")
+	echo "setvar fdtfile -guid $EFI_VAR_GUID -rt =L\"$FDT_BASENAME\""  >> "$EFI_CASE_DIR/startup.nsh"
 fi
-echo "$EFI_CASE.efi" "${cmd_args[@]}" >> "$EFI_TEST/$EFI_CASE/startup.nsh"
+echo "$EFI_CASE.efi" "${cmd_args[@]}" >> "$EFI_CASE_DIR/startup.nsh"
 
 EFI_RUN=y $TEST_DIR/run \
        -bios "$EFI_UEFI" \
-       -drive file.dir="$EFI_TEST/$EFI_CASE/",file.driver=vvfat,file.rw=on,format=raw,if=virtio \
+       -drive file.dir="$EFI_CASE_DIR/",file.driver=vvfat,file.rw=on,format=raw,if=virtio \
        "${qemu_args[@]}"
Signature · 1 line
2.40.1

Thanks,
drew

#9 lore raw

Re: [kvm-unit-tests PATCH v1 3/3] arm64: efi: Make running tests on EFI can be parallel

Hi drew,

Quoted 9 lines · Andrew Jones
Quoted 7 lines · Shaoqin Huang

Currently running tests on EFI in parallel can cause part of tests to fail, this is because arm/efi/run script use the EFI_CASE to create the subdir under the efi-tests, and the EFI_CASE is the filename of the test, when running tests in parallel, the multiple tests exist in the same filename will execute at the same time, which will use the same directory and write the test specific things into it, this cause chaotic and make some tests fail.

Quoted 8 lines

How do they fail? iiuc, we're switching from having one of each unique binary on the efi partition to multiple redundant binaries, since we copy the binary for each test, even when it's the same as for other tests. It seems like we should be able to keep single unique binaries and resolve the parallel execution failure by just checking for existence of the binaries or only creating test-specific data directories or something.

The problem comes from the arm/efi/run script, as you can see. If we parallel running multiple tests on efi, for example, running the pmu-sw-incr and pmu-chained-counters and other pmu tests at the same time, the EFI_CASE will be pmu. So they will write their $cmd_args to the $EFI/TEST/pmu/startup.nsh at the same time, which will corrupt the startup.nsh file.

cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_TEST/$EFI_CASE/"
echo "@echo -off" > "$EFI_TEST/$EFI_CASE/startup.nsh"
...
echo "$EFI_CASE.efi" "${cmd_args[@]}" >> "$EFI_TEST/$EFI_CASE/startup.nsh"

And I can get the log which outputs:

* pmu-sw-incr.log:
- ABORT: pmu: Unknown sub-test 'pmu-mem-acce'
* pmu-chained-counters.log
- ABORT: pmu: Unknown sub-test 'pmu-mem-access-reliab'

And the efi-tests/pmu/startup.nsh:

@echo -off
setvar fdtfile -guid 97ef3e03-7329-4a6a-b9ba-6c1fdcc5f823 -rt =L"dtb" pmu.efi pmu-mem-access-reliability
setvar fdtfile -guid 97ef3e03-7329-4a6a-b9ba-6c1fdcc5f823 -rt =L"dtb" pmu.efi pmu-chained-sw-incr

Thus when running parallel, some of them will fail. So I create different sub-dir in the efi-tests for each small test.

Thanks,
Shaoqin

Quoted 37 lines
Quoted 37 lines

To Fix this things, use the testname instead of the filename to create the subdir under the efi-tests. We use the EFI_TESTNAME to replace the EFI_CASE in script. Since every testname is specific, now the tests can be run parallel. It also considers when user directly use the arm/efi/run to run test, in this case, still use the filename.

Besides, replace multiple $EFI_TEST/$EFI_CASE to the $EFI_CASE_DIR, this makes the script looks more clean and we don'e need to replace many EFI_CASE to EFI_TESTNAME.

---
  arm/efi/run | 16 +++++++++-------
  1 file changed, 9 insertions(+), 7 deletions(-)
arm/efi/run +3 -1
index 6872c337..03bfbef4 100755
@@ -24,6 +24,8 @@ fi
  : "${EFI_SRC:=$TEST_DIR}"
  : "${EFI_UEFI:=$DEFAULT_UEFI}"
  : "${EFI_TEST:=efi-tests}"
+: "${EFI_TESTNAME:=$TESTNAME}"
+: "${EFI_TESTNAME:=$(basename $1 .efi)}"
  : "${EFI_CASE:=$(basename $1 .efi)}"
  : "${EFI_VAR_GUID:=97ef3e03-7329-4a6a-b9ba-6c1fdcc5f823}"
 
@@ -56,20 +58,20 @@ if [ "$EFI_CASE" = "_NO_FILE_4Uhere_" ]; then
  	EFI_CASE=dummy
  fi
 
-: "${EFI_CASE_DIR:="$EFI_TEST/$EFI_CASE"}"
+: "${EFI_CASE_DIR:="$EFI_TEST/$EFI_TESTNAME"}"
  mkdir -p "$EFI_CASE_DIR"
 
Quoted 7 lines

arm/efi/run if [ "$EFI_CASE" = "_NO_FILE_4Uhere_" ]; then · line 65

Quoted 4 lines
arm/efi/run +2 -2
-cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_TEST/$EFI_CASE/"
-echo "@echo -off" > "$EFI_TEST/$EFI_CASE/startup.nsh"
+cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_CASE_DIR/"
+echo "@echo -off" > "$EFI_CASE_DIR/startup.nsh"

Unrelated change, should be a separate patch.

Ok, Will separate it.

Quoted 23 lines
Quoted 19 lines
arm/efi/run +4 -4
  if [ "$EFI_USE_DTB" = "y" ]; then
  	qemu_args+=(-machine acpi=off)
  	FDT_BASENAME="dtb"
-	$(EFI_RUN=y $TEST_DIR/run -machine dumpdtb="$EFI_TEST/$EFI_CASE/$FDT_BASENAME" "${qemu_args[@]}")
-	echo "setvar fdtfile -guid $EFI_VAR_GUID -rt =L\"$FDT_BASENAME\""  >> "$EFI_TEST/$EFI_CASE/startup.nsh"
+	$(EFI_RUN=y $TEST_DIR/run -machine dumpdtb="$EFI_CASE_DIR/$FDT_BASENAME" "${qemu_args[@]}")
+	echo "setvar fdtfile -guid $EFI_VAR_GUID -rt =L\"$FDT_BASENAME\""  >> "$EFI_CASE_DIR/startup.nsh"
  fi
-echo "$EFI_CASE.efi" "${cmd_args[@]}" >> "$EFI_TEST/$EFI_CASE/startup.nsh"
+echo "$EFI_CASE.efi" "${cmd_args[@]}" >> "$EFI_CASE_DIR/startup.nsh"
 
  EFI_RUN=y $TEST_DIR/run \
         -bios "$EFI_UEFI" \
-       -drive file.dir="$EFI_TEST/$EFI_CASE/",file.driver=vvfat,file.rw=on,format=raw,if=virtio \
+       -drive file.dir="$EFI_CASE_DIR/",file.driver=vvfat,file.rw=on,format=raw,if=virtio \
         "${qemu_args[@]}"
Signature · 1 line
2.40.1

Thanks,
drew

Signature · 1 line
Shaoqin
#10 lore raw

Re: [kvm-unit-tests PATCH v1 3/3] arm64: efi: Make running tests on EFI can be parallel

Quoted 49 lines · Shaoqin Huang

Hi drew,

Quoted 9 lines · Andrew Jones
Quoted 7 lines · Shaoqin Huang

Currently running tests on EFI in parallel can cause part of tests to fail, this is because arm/efi/run script use the EFI_CASE to create the subdir under the efi-tests, and the EFI_CASE is the filename of the test, when running tests in parallel, the multiple tests exist in the same filename will execute at the same time, which will use the same directory and write the test specific things into it, this cause chaotic and make some tests fail.

Quoted 8 lines

How do they fail? iiuc, we're switching from having one of each unique binary on the efi partition to multiple redundant binaries, since we copy the binary for each test, even when it's the same as for other tests. It seems like we should be able to keep single unique binaries and resolve the parallel execution failure by just checking for existence of the binaries or only creating test-specific data directories or something.

The problem comes from the arm/efi/run script, as you can see. If we parallel running multiple tests on efi, for example, running the pmu-sw-incr and pmu-chained-counters and other pmu tests at the same time, the EFI_CASE will be pmu. So they will write their $cmd_args to the $EFI/TEST/pmu/startup.nsh at the same time, which will corrupt the startup.nsh file.

cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_TEST/$EFI_CASE/"
echo "@echo -off" > "$EFI_TEST/$EFI_CASE/startup.nsh"
...
echo "$EFI_CASE.efi" "${cmd_args[@]}" >> "$EFI_TEST/$EFI_CASE/startup.nsh"

And I can get the log which outputs:

* pmu-sw-incr.log:
- ABORT: pmu: Unknown sub-test 'pmu-mem-acce'
* pmu-chained-counters.log
- ABORT: pmu: Unknown sub-test 'pmu-mem-access-reliab'

And the efi-tests/pmu/startup.nsh:

@echo -off
setvar fdtfile -guid 97ef3e03-7329-4a6a-b9ba-6c1fdcc5f823 -rt =L"dtb" pmu.efi pmu-mem-access-reliability
setvar fdtfile -guid 97ef3e03-7329-4a6a-b9ba-6c1fdcc5f823 -rt =L"dtb" pmu.efi pmu-chained-sw-incr

Quoted 2 lines

Thus when running parallel, some of them will fail. So I create different sub-dir in the efi-tests for each small test.

Ok, I was guessing it was something like that. Maybe we should create a "bin" type of named directory for all the binaries and then create a separate subdir for each test and its startup.nsh, rather than copying the binaries multiple times.

Quoted 16 lines
Quoted 6 lines
Quoted 6 lines
  : "${EFI_CASE:=$(basename $1 .efi)}"
  : "${EFI_VAR_GUID:=97ef3e03-7329-4a6a-b9ba-6c1fdcc5f823}"
arm/efi/run +0 -1
@@ -56,20 +58,20 @@ if [ "$EFI_CASE" = "_NO_FILE_4Uhere_" ]; then
  	EFI_CASE=dummy
  fi
-: "${EFI_CASE_DIR:="$EFI_TEST/$EFI_CASE"}"
Quoted 9 lines

arm/efi/run if [ "$EFI_CASE" = "_NO_FILE_4Uhere_" ]; then · line 65

Quoted 6 lines
arm/efi/run +3 -2
+: "${EFI_CASE_DIR:="$EFI_TEST/$EFI_TESTNAME"}"
  mkdir -p "$EFI_CASE_DIR"
-cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_TEST/$EFI_CASE/"
-echo "@echo -off" > "$EFI_TEST/$EFI_CASE/startup.nsh"
+cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_CASE_DIR/"
+echo "@echo -off" > "$EFI_CASE_DIR/startup.nsh"

Unrelated change, should be a separate patch.

Quoted 1 line

Ok, Will separate it.

Actually, disregard my comment. I was too hasty here and thought the '@echo -off' was getting added, but now I see only the path was getting updated. It's correct to make this change here in this patch.

Thanks,
drew

#11 lore raw

Re: [kvm-unit-tests PATCH v1 3/3] arm64: efi: Make running tests on EFI can be parallel

Hi drew,

Quoted 50 lines · Andrew Jones
Quoted 49 lines · Shaoqin Huang

Hi drew,

Quoted 9 lines · Andrew Jones
Quoted 7 lines · Shaoqin Huang

Currently running tests on EFI in parallel can cause part of tests to fail, this is because arm/efi/run script use the EFI_CASE to create the subdir under the efi-tests, and the EFI_CASE is the filename of the test, when running tests in parallel, the multiple tests exist in the same filename will execute at the same time, which will use the same directory and write the test specific things into it, this cause chaotic and make some tests fail.

Quoted 8 lines

How do they fail? iiuc, we're switching from having one of each unique binary on the efi partition to multiple redundant binaries, since we copy the binary for each test, even when it's the same as for other tests. It seems like we should be able to keep single unique binaries and resolve the parallel execution failure by just checking for existence of the binaries or only creating test-specific data directories or something.

The problem comes from the arm/efi/run script, as you can see. If we parallel running multiple tests on efi, for example, running the pmu-sw-incr and pmu-chained-counters and other pmu tests at the same time, the EFI_CASE will be pmu. So they will write their $cmd_args to the $EFI/TEST/pmu/startup.nsh at the same time, which will corrupt the startup.nsh file.

cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_TEST/$EFI_CASE/"
echo "@echo -off" > "$EFI_TEST/$EFI_CASE/startup.nsh"
...
echo "$EFI_CASE.efi" "${cmd_args[@]}" >> "$EFI_TEST/$EFI_CASE/startup.nsh"

And I can get the log which outputs:

* pmu-sw-incr.log:
- ABORT: pmu: Unknown sub-test 'pmu-mem-acce'
* pmu-chained-counters.log
- ABORT: pmu: Unknown sub-test 'pmu-mem-access-reliab'

And the efi-tests/pmu/startup.nsh:

@echo -off
setvar fdtfile -guid 97ef3e03-7329-4a6a-b9ba-6c1fdcc5f823 -rt =L"dtb" pmu.efi pmu-mem-access-reliability
setvar fdtfile -guid 97ef3e03-7329-4a6a-b9ba-6c1fdcc5f823 -rt =L"dtb" pmu.efi pmu-chained-sw-incr

Quoted 8 lines
Quoted 2 lines

Thus when running parallel, some of them will fail. So I create different sub-dir in the efi-tests for each small test.

Ok, I was guessing it was something like that. Maybe we should create a "bin" type of named directory for all the binaries and then create a separate subdir for each test and its startup.nsh, rather than copying the binaries multiple times.

I can put this kind of improvement into another patch. Currently I want to use the safety way to fix this problem, create different directory and copy the .efi for each of them will not cause any interfere for each of the test. We can tolerate the redundant, it doesn't cost many space.

Thanks,
Shaoqin

Quoted 25 lines
Quoted 16 lines
Quoted 6 lines
Quoted 6 lines
   : "${EFI_CASE:=$(basename $1 .efi)}"
   : "${EFI_VAR_GUID:=97ef3e03-7329-4a6a-b9ba-6c1fdcc5f823}"
arm/efi/run +0 -1
@@ -56,20 +58,20 @@ if [ "$EFI_CASE" = "_NO_FILE_4Uhere_" ]; then
   	EFI_CASE=dummy
   fi
-: "${EFI_CASE_DIR:="$EFI_TEST/$EFI_CASE"}"
Quoted 9 lines

arm/efi/run if [ "$EFI_CASE" = "_NO_FILE_4Uhere_" ]; then · line 65

Quoted 6 lines
arm/efi/run +3 -2
+: "${EFI_CASE_DIR:="$EFI_TEST/$EFI_TESTNAME"}"
   mkdir -p "$EFI_CASE_DIR"
-cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_TEST/$EFI_CASE/"
-echo "@echo -off" > "$EFI_TEST/$EFI_CASE/startup.nsh"
+cp "$EFI_SRC/$EFI_CASE.efi" "$EFI_CASE_DIR/"
+echo "@echo -off" > "$EFI_CASE_DIR/startup.nsh"

Unrelated change, should be a separate patch.

Quoted 1 line

Ok, Will separate it.

Actually, disregard my comment. I was too hasty here and thought the '@echo -off' was getting added, but now I see only the path was getting updated. It's correct to make this change here in this patch.

Thanks,
drew

Signature · 1 line
Shaoqin