Skip to contentSkip to conversation

Speed up migration tests

PATCH In review Thomas Huth 4 participants 10 messages Aug 19, 2022 – Aug 22, 2022

lore
Share & tools
lore
#0 lore raw

cover Speed up migration tests

We are currently facing the problem that the "gcov-gprof" CI jobs in the gitlab-CI are running way too long - which happens since the migration-tests have been enabled there recently.

These patches now speed up the migration tests, so that the CI job should be fine again.

This is how it looked like before my modifications:

https://gitlab.com/thuth/qemu/-/jobs/2888957948#L46
...

 5/243 qemu:qtest+qtest-aarch64 / qtest-aarch64/migration-test  OK  1265.22s
 8/243 qemu:qtest+qtest-x86_64 / qtest-x86_64/migration-test    OK  1138.82s

And this is how it looks like after the patches have been applied:

https://gitlab.com/thuth/qemu/-/jobs/2905108018#L48
...

 5/243 qemu:qtest+qtest-aarch64 / qtest-aarch64/migration-test  OK   251.14s
 8/243 qemu:qtest+qtest-x86_64 / qtest-x86_64/migration-test    OK   336.94s

That means the CI job is running ca. 30 minutes faster here now!

Thomas Huth (4):
tests/qtest/migration-test: Only wait for serial output where
migration succeeds
tests/migration/aarch64: Speed up the aarch64 migration test tests/migration/i386: Speed up the i386 migration test (when using
TCG)
tests/qtest/migration-test: Remove duplicated test_postcopy from the
test plan

 tests/migration/aarch64/a-b-kernel.h | 10 +++++-----
 tests/migration/i386/a-b-bootblock.h | 12 ++++++------
 tests/qtest/migration-test.c         |  5 +++--
 tests/migration/aarch64/a-b-kernel.S |  3 +--
 tests/migration/i386/a-b-bootblock.S |  1 +
 5 files changed, 16 insertions(+), 15 deletions(-)
Signature · 1 line
2.31.1
#1 lore raw

PATCH 1/4 tests/qtest/migration-test: Only wait for serial output where migration succeeds

Waiting for the serial output can take a couple of seconds - and since we're doing a lot of migration tests, this time easily sums up to multiple minutes. But if a test is supposed to fail, it does not make much sense to wait for the source to be in the right state first, so we can skip the waiting here. This way we can speed up all tests where the migration is supposed to fail. In the gitlab-CI gprov-gcov test, each of the migration-tests now run two minutes faster!

---
 tests/qtest/migration-test.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tests/qtest/migration-test.c +3 -1
index 520a5f917c..7be321b62d 100644
@@ -1307,7 +1307,9 @@ static void test_precopy_common(MigrateCommon *args)
     }
 
     /* Wait for the first serial output from the source */
-    wait_for_serial("src_serial");
+    if (args->result == MIG_TEST_SUCCEED) {
+        wait_for_serial("src_serial");
+    }
 
     if (!args->connect_uri) {
         g_autofree char *local_connect_uri =
Signature · 1 line
2.31.1
#2 lore raw

PATCH 2/4 tests/migration/aarch64: Speed up the aarch64 migration test

In reply to Thomas Huth

The migration tests spend a lot of time waiting for a sign of live of the guest on the serial console. The aarch64 migration code only outputs "B"s every couple of seconds (at least it takes more than 4 seconds between each characeter on my x86 laptop). There are a lot of migration tests, and if each test that checks for a successful migration waits for these characters before and after migration, the wait time sums up to multiple minutes! Let's use a shorter delay to speed things up.

While we're at it, also remove a superfluous masking with 0xff - we're reading and storing bytes, so the upper bits of the register do not matter anyway.

With these changes, the test runs twice as fast on my laptop, decreasing the total run time from approx. 8 minutes to only 4 minutes!

---
 tests/migration/aarch64/a-b-kernel.h | 10 +++++-----
 tests/migration/aarch64/a-b-kernel.S |  3 +--
 2 files changed, 6 insertions(+), 7 deletions(-)
tests/migration/aarch64/a-b-kernel.h +5 -5
index 0a9b01137e..34e518d061 100644
@@ -10,9 +10,9 @@ unsigned char aarch64_kernel[] = {
   0x03, 0x00, 0x80, 0x52, 0xe4, 0x03, 0x00, 0xaa, 0x83, 0x00, 0x00, 0x39,
   0x84, 0x04, 0x40, 0x91, 0x9f, 0x00, 0x01, 0xeb, 0xad, 0xff, 0xff, 0x54,
   0x05, 0x00, 0x80, 0x52, 0xe4, 0x03, 0x00, 0xaa, 0x83, 0x00, 0x40, 0x39,
-  0x63, 0x04, 0x00, 0x11, 0x63, 0x1c, 0x00, 0x12, 0x83, 0x00, 0x00, 0x39,
-  0x24, 0x7e, 0x0b, 0xd5, 0x84, 0x04, 0x40, 0x91, 0x9f, 0x00, 0x01, 0xeb,
-  0x2b, 0xff, 0xff, 0x54, 0xa5, 0x04, 0x00, 0x11, 0xa5, 0x1c, 0x00, 0x12,
-  0xbf, 0x00, 0x00, 0x71, 0x81, 0xfe, 0xff, 0x54, 0x43, 0x08, 0x80, 0x52,
-  0x43, 0x00, 0x00, 0x39, 0xf1, 0xff, 0xff, 0x17
+  0x63, 0x04, 0x00, 0x11, 0x83, 0x00, 0x00, 0x39, 0x24, 0x7e, 0x0b, 0xd5,
+  0x84, 0x04, 0x40, 0x91, 0x9f, 0x00, 0x01, 0xeb, 0x4b, 0xff, 0xff, 0x54,
+  0xa5, 0x04, 0x00, 0x11, 0xa5, 0x10, 0x00, 0x12, 0xbf, 0x00, 0x00, 0x71,
+  0xa1, 0xfe, 0xff, 0x54, 0x43, 0x08, 0x80, 0x52, 0x43, 0x00, 0x00, 0x39,
+  0xf2, 0xff, 0xff, 0x17
 };
tests/migration/aarch64/a-b-kernel.S +1 -2
index 0225945348..a4103ecb71 100644
@@ -53,7 +53,6 @@ innerloop:
         /* increment the first byte of each page by 1 */
         ldrb    w3, [x4]
         add     w3, w3, #1
-        and     w3, w3, #0xff
         strb    w3, [x4]
 
         /* make sure QEMU user space can see consistent data as MMU is off */
@@ -64,7 +63,7 @@ innerloop:
         blt     innerloop
 
         add     w5, w5, #1
-        and     w5, w5, #0xff
+        and     w5, w5, #0x1f
         cmp     w5, #0
         bne     mainloop
 
Signature · 1 line
2.31.1
Reviewed-by on v1 2/4
#3 lore raw

Re: [PATCH 2/4] tests/migration/aarch64: Speed up the aarch64 migration test

Quoted 13 lines · Thomas Huth

The migration tests spend a lot of time waiting for a sign of live of the guest on the serial console. The aarch64 migration code only outputs "B"s every couple of seconds (at least it takes more than 4 seconds between each characeter on my x86 laptop). There are a lot of migration tests, and if each test that checks for a successful migration waits for these characters before and after migration, the wait time sums up to multiple minutes! Let's use a shorter delay to speed things up.

While we're at it, also remove a superfluous masking with 0xff - we're reading and storing bytes, so the upper bits of the register do not matter anyway.

Quoted 8 lines

With these changes, the test runs twice as fast on my laptop, decreasing the total run time from approx. 8 minutes to only 4 minutes!

---
 tests/migration/aarch64/a-b-kernel.h | 10 +++++-----
 tests/migration/aarch64/a-b-kernel.S |  3 +--
 2 files changed, 6 insertions(+), 7 deletions(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#4 lore raw

PATCH 3/4 tests/migration/i386: Speed up the i386 migration test (when using TCG)

In reply to Thomas Huth

When KVM is not available, the i386 migration test also runs in a rather slow fashion, since the guest code takes a couple of seconds to print the "B"s on the serial console, and the migration test has to wait for this each time. Let's increase the frequency here, too, so that the delays in the migration tests get smaller.

---
 tests/migration/i386/a-b-bootblock.h | 12 ++++++------
 tests/migration/i386/a-b-bootblock.S |  1 +
 2 files changed, 7 insertions(+), 6 deletions(-)
tests/migration/i386/a-b-bootblock.h +6 -6
index 7d459d4fde..b7b0fce2ee 100644
@@ -4,17 +4,17 @@
  * the header and the assembler differences in your patch submission.
  */
 unsigned char x86_bootsect[] = {
-  0xfa, 0x0f, 0x01, 0x16, 0x74, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
+  0xfa, 0x0f, 0x01, 0x16, 0x78, 0x7c, 0x66, 0xb8, 0x01, 0x00, 0x00, 0x00,
   0x0f, 0x22, 0xc0, 0x66, 0xea, 0x20, 0x7c, 0x00, 0x00, 0x08, 0x00, 0x00,
   0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe4, 0x92, 0x0c, 0x02,
   0xe6, 0x92, 0xb8, 0x10, 0x00, 0x00, 0x00, 0x8e, 0xd8, 0x66, 0xb8, 0x41,
   0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xb3, 0x00, 0xb8, 0x00, 0x00, 0x10,
   0x00, 0xfe, 0x00, 0x05, 0x00, 0x10, 0x00, 0x00, 0x3d, 0x00, 0x00, 0x40,
-  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x75, 0xe9, 0x66, 0xb8, 0x42, 0x00, 0x66,
-  0xba, 0xf8, 0x03, 0xee, 0xeb, 0xde, 0x66, 0x90, 0x00, 0x00, 0x00, 0x00,
-  0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x9a, 0xcf, 0x00,
-  0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00, 0x27, 0x00, 0x5c, 0x7c,
-  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+  0x06, 0x7c, 0xf2, 0xfe, 0xc3, 0x80, 0xe3, 0x3f, 0x75, 0xe6, 0x66, 0xb8,
+  0x42, 0x00, 0x66, 0xba, 0xf8, 0x03, 0xee, 0xeb, 0xdb, 0x8d, 0x76, 0x00,
+  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0x00, 0x00,
+  0x00, 0x9a, 0xcf, 0x00, 0xff, 0xff, 0x00, 0x00, 0x00, 0x92, 0xcf, 0x00,
+  0x27, 0x00, 0x60, 0x7c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
   0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
   0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
   0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
tests/migration/i386/a-b-bootblock.S +1 -0
index 3f97f28023..3d464c7568 100644
@@ -50,6 +50,7 @@ innerloop:
         jl innerloop
 
         inc %bl
+        andb $0x3f,%bl
         jnz mainloop
 
         mov $66,%ax
Signature · 1 line
2.31.1
Reviewed-by on v1 3/4
#5 lore raw

Re: [PATCH 3/4] tests/migration/i386: Speed up the i386 migration test (when using TCG)

Quoted 11 lines · Thomas Huth

When KVM is not available, the i386 migration test also runs in a rather slow fashion, since the guest code takes a couple of seconds to print the "B"s on the serial console, and the migration test has to wait for this each time. Let's increase the frequency here, too, so that the delays in the migration tests get smaller.

---
 tests/migration/i386/a-b-bootblock.h | 12 ++++++------
 tests/migration/i386/a-b-bootblock.S |  1 +
 2 files changed, 7 insertions(+), 6 deletions(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#6 lore raw

PATCH 4/4 tests/qtest/migration-test: Remove duplicated test_postcopy from the test plan

In reply to Thomas Huth

test_postcopy() is currently run twice - which is just a waste of resources and time. The commit d1a27b169b2d that introduced the duplicate talked about renaming the "postcopy/unix" test, but apparently it forgot to remove the old entry. Let's do that now.

Fixes d1a27b169b ("tests: Add postcopy tls migration test")
---
 tests/qtest/migration-test.c | 1 -
 1 file changed, 1 deletion(-)
tests/qtest/migration-test.c +0 -1
index 7be321b62d..f63edd0bc8 100644
@@ -2461,7 +2461,6 @@ int main(int argc, char **argv)
     module_call_init(MODULE_INIT_QOM);
 
     if (has_uffd) {
-        qtest_add_func("/migration/postcopy/unix", test_postcopy);
         qtest_add_func("/migration/postcopy/plain", test_postcopy);
         qtest_add_func("/migration/postcopy/recovery/plain",
                        test_postcopy_recovery);
Signature · 1 line
2.31.1
Reviewed-by on v1 4/4
#7 lore raw

Re: [PATCH 4/4] tests/qtest/migration-test: Remove duplicated test_postcopy from the test plan

Quoted 4 lines · Thomas Huth

test_postcopy() is currently run twice - which is just a waste of resources and time. The commit d1a27b169b2d that introduced the duplicate talked about renaming the "postcopy/unix" test, but apparently it forgot to remove the old entry. Let's do that now.

It was probably a merge error

Quoted 6 lines
Fixes d1a27b169b ("tests: Add postcopy tls migration test")
---
 tests/qtest/migration-test.c | 1 -
 1 file changed, 1 deletion(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#8 lore raw

Re: [PATCH 4/4] tests/qtest/migration-test: Remove duplicated test_postcopy from the test plan

Quoted 7 lines · * Daniel P. Berrangé ([email protected]) wrote:
Quoted 4 lines · Thomas Huth

test_postcopy() is currently run twice - which is just a waste of resources and time. The commit d1a27b169b2d that introduced the duplicate talked about renaming the "postcopy/unix" test, but apparently it forgot to remove the old entry. Let's do that now.

It was probably a merge error

Yeh I remember having to do a lot of manual merges around htere.

Dave

Quoted 17 lines
Quoted 6 lines
Fixes d1a27b169b ("tests: Add postcopy tls migration test")
---
 tests/qtest/migration-test.c | 1 -
 1 file changed, 1 deletion(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#9 lore raw
In reply to Thomas Huth
Quoted 6 lines · Thomas Huth

We are currently facing the problem that the "gcov-gprof" CI jobs in the gitlab-CI are running way too long - which happens since the migration-tests have been enabled there recently.

These patches now speed up the migration tests, so that the CI job should be fine again.

Awesome stuff, queued to testing/next, thanks.

Signature · 1 line
Alex Bennée