Skip to contentSkip to conversation

Support chain migration in test

PATCH In review [email protected] 2 participants 6 messages Jul 9 – Aug 20

lore
Share & tools
lore
#0 lore raw

cover Support chain migration in test

From: Nguyen Dinh Phi <[email protected]>

This series adds test infrastructure and test cases for multi-hop chain migration, where a VM migrates from host A to B and then from B to C.

The migration test TODO list mentions ping-pong migration (A→B→A) as a desirable scenario; the true ping-pong (A→B→A) would require the original source A to become a destination for the return leg.
This is not possible: a QEMU process launched without -incoming cannot later accept an incoming migration stream. Instead, this series implements chain migration (A→B→C): B was already launched with -incoming and can be re-wired as the source for the next hop, while a fresh QEMU instance C is started as the new destination.

Nguyen Dinh Phi (3):
tests/migration: Factor out per-process launch and setup helpers tests/migration: Retrieve serial path from QTestState tests/migration: Add chain (A to B to C) precopy migration tests

 tests/qtest/libqtest.c                |  16 +
 tests/qtest/libqtest.h                |   8 +
 tests/qtest/migration/colo-tests.c    |  14 +-
 tests/qtest/migration/cpr-tests.c     |   8 +-
 tests/qtest/migration/framework.c     | 475 ++++++++++++++++----------
 tests/qtest/migration/framework.h     |  21 +-
 tests/qtest/migration/misc-tests.c    |   4 +-
 tests/qtest/migration/precopy-tests.c |  51 ++-
 8 files changed, 389 insertions(+), 208 deletions(-)
Signature · 1 line
2.50.1 (Apple Git-155)
#1 lore raw

PATCH 1/3 tests/migration: Factor out per-process launch and setup helpers

From: Nguyen Dinh Phi <[email protected]>

migrate_start() always spawned source and destination together, and migrate_start_set_capabilities() required both QTestState pointers at once. This coupling prevents launching a third QEMU instance or re-configuring an existing destination as a new source.

Split the logic into three public helpers:

migrate_launch_source() / migrate_launch_dest()
Spawn a single QEMU process and record its serial file path on the returned QTestState.

migrate_setup_instance()
Register the event callback and apply migration capabilities to one already-running process. Can be called again to re-wire the same process for a different role.

migrate_start() is kept for backward compatibility and now delegates to these helpers internally.

---
 tests/qtest/migration/framework.c | 253 +++++++++++++++++-------------
 tests/qtest/migration/framework.h |  18 ++-
 2 files changed, 165 insertions(+), 106 deletions(-)
tests/qtest/migration/framework.c +148 -105
index a830b96f41..305b8968ff 100644
@@ -208,9 +208,11 @@ static QList *migrate_start_get_qmp_capabilities(const MigrateStart *args)
     return capabilities;
 }
 
-static void migrate_start_set_capabilities(QTestState *from, QTestState *to,
-                                           MigrateStart *args)
+void migrate_setup_instance(QTestState *who, QTestMigrationState *state,
+                   MigrateStart *args)
 {
+    qtest_qmp_set_event_callback(who, migrate_watch_for_events, state);
+    *state = (QTestMigrationState) { .suspend_me = args->suspend_me };
     /*
      * MigrationCapability_lookup and MIGRATION_CAPABILITY_ constants
      * are from qapi-types-migration.h.
@@ -220,25 +222,13 @@ static void migrate_start_set_capabilities(QTestState *from, QTestState *to,
      * Enable return path first, since other features depend on it.
      */
     if (args->caps[MIGRATION_CAPABILITY_RETURN_PATH]) {
-        if (from) {
-            migrate_set_capability(from, "return-path", true);
-        }
-        if (to) {
-            migrate_set_capability(to, "return-path", true);
-        }
+        migrate_set_capability(who, "return-path", true);
     }
 
     for (uint8_t i = 0; i < MIGRATION_CAPABILITY__MAX; i++) {
-        if (!args->caps[i]) {
-            continue;
-        }
-        if (from) {
-            migrate_set_capability(from,
-                            MigrationCapability_lookup.array[i], true);
-        }
-        if (to) {
-            migrate_set_capability(to,
-                            MigrationCapability_lookup.array[i], true);
+        if (args->caps[i]) {
+            migrate_set_capability(who, MigrationCapability_lookup.array[i],
+                                   true);
         }
     }
 
@@ -246,26 +236,17 @@ static void migrate_start_set_capabilities(QTestState *from, QTestState *to,
      * Always enable migration events.  Libvirt always uses it, let's try
      * to mimic as closer as that.
      */
-    migrate_set_capability(from, "events", true);
-    if (!args->defer_target_connect && to) {
-        migrate_set_capability(to, "events", true);
-    }
+    migrate_set_capability(who, "events", true);
 
-    /*
+     /*
      * Default number of channels should be fine for most
      * tests. Individual tests can override by calling
      * migrate_set_parameter() directly.
      */
     if (args->caps[MIGRATION_CAPABILITY_MULTIFD]) {
-        migrate_set_parameter_int(from, "multifd-channels",
+        migrate_set_parameter_int(who, "multifd-channels",
                                   MULTIFD_TEST_CHANNELS);
-        if (to) {
-            migrate_set_parameter_int(to, "multifd-channels",
-                                      MULTIFD_TEST_CHANNELS);
-        }
     }
-
-    return;
 }
 
 static char *test_shmem_path(void)
@@ -308,12 +289,28 @@ static char *migrate_mem_type_get_opts(MemType type, const char *memory_size)
     return opts;
 }
 
-int migrate_args(char **from, char **to, MigrateStart *args)
+/*
+ * Build a QEMU command line for one migration participant.
+ *
+ * @serial_name  : serial output file basename under tmpfs
+ * @is_target    : set to true if the new qemu process is destination side
+ * @extra_opts   : extra arguments
+ * @defer_events : when true and is_target is set, use the command-line
+ *                 event flag instead of the QMP capability (for deferred
+ *                 monitor connections)
+ * @args         : common process settings
+ *
+ * Returns 0 and sets *cmd_out on success.  Returns -1 and calls
+ * g_test_skip() if the machine type is not available.
+ */
+static int migrate_build_cmd(const char *serial_name,
+                             bool is_target,
+                             const char *extra_opts,
+                             bool defer_events,
+                             MigrateStart *args,
+                             char **cmd_out)
 {
-    /* options for source and target */
     g_autofree gchar *arch_opts = NULL;
-    gchar *cmd_source = NULL;
-    gchar *cmd_target = NULL;
     const gchar *ignore_stderr;
     g_autofree char *mem_object = NULL;
     const char *kvm_opts = NULL;
@@ -323,16 +320,10 @@ int migrate_args(char **from, char **to, MigrateStart *args)
     g_autofree char *machine = NULL;
     const char *bootpath = bootfile_get();
     g_autofree char *memory_backend = NULL;
-    const char *events;
 
     if (strcmp(arch, "i386") == 0 || strcmp(arch, "x86_64") == 0) {
         memory_size = "150M";
-
-        if (g_str_equal(arch, "i386")) {
-            machine_alias = "pc";
-        } else {
-            machine_alias = "q35";
-        }
+        machine_alias = g_str_equal(arch, "i386") ? "pc" : "q35";
         arch_opts = g_strdup_printf(
             "-drive if=none,id=d0,file=%s,format=raw "
             "-device ide-hd,drive=d0,secs=1,cyls=1,heads=1", bootpath);
@@ -346,13 +337,13 @@ int migrate_args(char **from, char **to, MigrateStart *args)
         end_address = S390_TEST_MEM_END;
     } else if (strcmp(arch, "ppc64") == 0) {
         memory_size = "256M";
-        start_address = PPC_TEST_MEM_START;
-        end_address = PPC_TEST_MEM_END;
         machine_alias = "pseries";
         machine_opts = "vsmt=8";
         arch_opts = g_strdup_printf(
             "-nodefaults -machine " PSERIES_DEFAULT_CAPABILITIES " "
             "-bios %s", bootpath);
+        start_address = PPC_TEST_MEM_START;
+        end_address = PPC_TEST_MEM_END;
     } else if (strcmp(arch, "aarch64") == 0) {
         memory_size = "150M";
         machine_alias = "virt";
@@ -394,53 +385,103 @@ int migrate_args(char **from, char **to, MigrateStart *args)
     }
 
     if (!qtest_has_machine(machine_alias)) {
-        g_autofree char *msg = g_strdup_printf("machine %s not supported", machine_alias);
+        g_autofree char *msg = g_strdup_printf("machine %s not supported",
+                                               machine_alias);
         g_test_skip(msg);
         return -1;
     }
 
     machine = resolve_machine_version(machine_alias, QEMU_ENV_SRC,
                                       QEMU_ENV_DST);
-
     g_test_message("Using machine type: %s", machine);
 
-    cmd_source = g_strdup_printf("-accel kvm%s -accel tcg "
-                                 "-machine %s,%s "
-                                 "-name source,debug-threads=on "
-                                 "%s "
-                                 "-serial file:%s/src_serial "
-                                 "%s %s %s",
-                                 kvm_opts ? kvm_opts : "",
-                                 machine, machine_opts,
-                                 memory_backend, tmpfs,
-                                 arch_opts ? arch_opts : "",
-                                 args->opts_source ? args->opts_source : "",
-                                 ignore_stderr);
+    if (is_target) {
+        /*
+         * If the monitor connection is deferred, enable events on the command
+         * line so none are missed. This is for testing only, do not set
+         * migration options like this in general.
+         */
+        const char *events = defer_events ? "-global migration.x-events=on"
+                                          : "";
+        *cmd_out = g_strdup_printf(
+            "-accel kvm%s -accel tcg "
+            "-machine %s,%s "
+            "-name target,debug-threads=on "
+            "%s "
+            "-serial file:%s/%s "
+            "-incoming defer "
+            "%s %s %s %s",
+            kvm_opts ? kvm_opts : "",
+            machine, machine_opts,
+            memory_backend, tmpfs, serial_name,
+            events,
+            arch_opts ? arch_opts : "",
+            extra_opts ? extra_opts : "",
+            ignore_stderr);
+    } else {
+        *cmd_out = g_strdup_printf(
+            "-accel kvm%s -accel tcg "
+            "-machine %s,%s "
+            "-name source,debug-threads=on "
+            "%s "
+            "-serial file:%s/%s "
+            "%s %s %s",
+            kvm_opts ? kvm_opts : "",
+            machine, machine_opts,
+            memory_backend, tmpfs, serial_name,
+            arch_opts ? arch_opts : "",
+            extra_opts ? extra_opts : "",
+            ignore_stderr);
+    }
 
-    /*
-     * If the monitor connection is deferred, enable events on the command line
-     * so none are missed.  This is for testing only, do not set migration
-     * options like this in general.
-     */
-    events = args->defer_target_connect ? "-global migration.x-events=on" : "";
-
-    cmd_target = g_strdup_printf("-accel kvm%s -accel tcg "
-                                 "-machine %s,%s "
-                                 "-name target,debug-threads=on "
-                                 "%s "
-                                 "-serial file:%s/dest_serial "
-                                 "-incoming defer "
-                                 "%s %s %s %s",
-                                 kvm_opts ? kvm_opts : "",
-                                 machine, machine_opts,
-                                 memory_backend, tmpfs,
-                                 events,
-                                 arch_opts ? arch_opts : "",
-                                 args->opts_target ? args->opts_target : "",
-                                 ignore_stderr);
-
-    *from = cmd_source;
-    *to = cmd_target;
+    return 0;
+}
+
+/*
+ * Launch a QEMU process as a migration source.
+ * Returns the QTestState on success, or NULL otherwise
+ */
+QTestState *migrate_launch_source(const char *serial_name, MigrateStart *args)
+{
+    g_autofree char *cmd = NULL;
+    g_autoptr(QList) capabilities = migrate_start_get_qmp_capabilities(args);
+
+    if (migrate_build_cmd(serial_name, false, args->opts_source,
+                          false, args, &cmd)) {
+        return NULL;
+    }
+    return qtest_init_ext(QEMU_ENV_SRC, cmd, capabilities, true);
+}
+
+/*
+ * Launch a QEMU process as a migration destination.
+ * Returns the QTestState on success, or NULL otherwise
+ */
+QTestState *migrate_launch_dest(const char *serial_name, MigrateStart *args)
+{
+    g_autofree char *cmd = NULL;
+    g_autoptr(QList) capabilities = migrate_start_get_qmp_capabilities(args);
+
+    if (migrate_build_cmd(serial_name, true, args->opts_target,
+                          args->defer_target_connect, args, &cmd)) {
+        return NULL;
+    }
+    return qtest_init_ext(QEMU_ENV_DST, cmd, capabilities,
+                          !args->defer_target_connect);
+}
+
+int migrate_args(char **from, char **to, MigrateStart *args)
+{
+    if (migrate_build_cmd("src_serial", false, args->opts_source,
+                          false, args, from)) {
+        return -1;
+    }
+    if (migrate_build_cmd("dest_serial", true, args->opts_target,
+                          args->defer_target_connect, args, to)) {
+        g_free(*from);
+        *from = NULL;
+        return -1;
+    }
     return 0;
 }
 
@@ -482,43 +523,45 @@ static void migrate_mem_type_cleanup(MemType type)
 
 int migrate_start(QTestState **from, QTestState **to, MigrateStart *args)
 {
-    g_autofree gchar *cmd_source = NULL;
-    g_autofree gchar *cmd_target = NULL;
-    g_autoptr(QList) capabilities = migrate_start_get_qmp_capabilities(args);
-
     if (!migrate_mem_type_prepare(args->mem_type)) {
         return -1;
     }
 
-    dst_state = (QTestMigrationState) { };
-    src_state = (QTestMigrationState) { };
+    src_state = (QTestMigrationState){};
+    dst_state = (QTestMigrationState){};
     bootfile_create(qtest_get_arch(), tmpfs, args->suspend_me);
-    src_state.suspend_me = args->suspend_me;
-
-    if (migrate_args(&cmd_source, &cmd_target, args)) {
-        return -1;
-    }
 
     if (!args->only_target) {
-        *from = qtest_init_ext(QEMU_ENV_SRC, cmd_source, capabilities, true);
-        qtest_qmp_set_event_callback(*from,
-                                     migrate_watch_for_events,
-                                     &src_state);
+        *from = migrate_launch_source("src_serial", args);
+        if (!*from) {
+            return -1;
+        }
+        migrate_setup_instance(*from, &src_state, args);
     }
 
     if (!args->only_source) {
-        *to = qtest_init_ext(QEMU_ENV_DST, cmd_target, capabilities,
-                             !args->defer_target_connect);
-        qtest_qmp_set_event_callback(*to,
-                                     migrate_watch_for_events,
-                                     &dst_state);
+        *to = migrate_launch_dest("dest_serial", args);
+        if (!*to) {
+            if (!args->only_target) {
+                qtest_quit(*from);
+            }
+            return -1;
+        }
+        if (!args->defer_target_connect) {
+            migrate_setup_instance(*to, &dst_state, args);
+        } else {
+            /*
+             * The monitor is not connected yet; only register the event
+             * callback so no events are missed.
+             * The caller must call migrate_setup_instance() after
+             * qtest_connect() if it wants to setup any capability
+             */
+            qtest_qmp_set_event_callback(*to, migrate_watch_for_events,
+                                         &dst_state);
+        }
     }
 
     migrate_mem_type_cleanup(args->mem_type);
-    migrate_start_set_capabilities(*from,
-                                   args->only_source ? NULL : *to,
-                                   args);
-
     return 0;
 }
 
tests/qtest/migration/framework.h +17 -1
index 941cbd7102..f054ef8f4d 100644
@@ -225,10 +225,27 @@ typedef struct {
     bool live;
 } MigrateCommon;
 
+typedef struct QTestMigrationState QTestMigrationState;
+
 void wait_for_serial(const char *side);
 void migrate_prepare_for_dirty_mem(QTestState *from);
 void migrate_wait_for_dirty_mem(QTestState *from, QTestState *to);
 
+/*
+ * Start a QEMU process which has serial set to @serial_name
+ */
+QTestState *migrate_launch_source(const char *serial_name, MigrateStart *args);
+/*
+ * Start a QEMU process which has serial set to @serial_name
+ */
+QTestState *migrate_launch_dest(const char *serial_name, MigrateStart *args);
+/*
+ * Registers the event callback and sets migration capabilities on an
+ * already-running process
+ */
+void migrate_setup_instance(QTestState *who, QTestMigrationState *state,
+                   MigrateStart *args);
+
 int migrate_args(char **from, char **to, MigrateStart *args);
 int migrate_start(QTestState **from, QTestState **to, MigrateStart *args);
 void migrate_end(QTestState *from, QTestState *to, bool test_dest);
@@ -240,7 +257,6 @@ int test_precopy_common(MigrateCommon *args);
 void test_precopy_unix_common(MigrateCommon *args);
 void test_file_common(MigrateCommon *args, bool stop_src);
 
-typedef struct QTestMigrationState QTestMigrationState;
 QTestMigrationState *get_src(void);
 QTestMigrationState *get_dst(void);
 
Signature · 1 line
2.50.1 (Apple Git-155)
#2 lore raw

Re: [PATCH 1/3] tests/migration: Factor out per-process launch and setup helpers

Quoted 2 lines · [email protected] writes:

From: Nguyen Dinh Phi <[email protected]>

Hi, sorry for the delay, this series conflicts with some ongoing work I'm doing so I had to find time to double check.

Quoted 5 lines

migrate_start() always spawned source and destination together, and migrate_start_set_capabilities() required both QTestState pointers at once. This coupling prevents launching a third QEMU instance or re-configuring an existing destination as a new source.

What about migrate_end? Don't you need to allow instances to be freed individually as well?

Quoted 25 lines

Split the logic into three public helpers:

migrate_launch_source() / migrate_launch_dest()
Spawn a single QEMU process and record its serial file path on the returned QTestState.

migrate_setup_instance()
Register the event callback and apply migration capabilities to one already-running process. Can be called again to re-wire the same process for a different role.

migrate_start() is kept for backward compatibility and now delegates to these helpers internally.

---
 tests/qtest/migration/framework.c | 253 +++++++++++++++++-------------
 tests/qtest/migration/framework.h |  18 ++-
 2 files changed, 165 insertions(+), 106 deletions(-)
tests/qtest/migration/framework.c +0 -0
index a830b96f41..305b8968ff 100644

tests/qtest/migration/framework.c static QList *migrate_start_get_qmp_capabilities(const MigrateStart *args) · line 214

Quoted 9 lines
tests/qtest/migration/framework.c +3 -2
@@ -208,9 +208,11 @@ static QList *migrate_start_get_qmp_capabilities(const MigrateStart *args)
     return capabilities;
 }
 
-static void migrate_start_set_capabilities(QTestState *from, QTestState *to,
-                                           MigrateStart *args)
+void migrate_setup_instance(QTestState *who, QTestMigrationState *state,
+                   MigrateStart *args)
 {
+    qtest_qmp_set_event_callback(who, migrate_watch_for_events, state);

You could pass in a defer_connect boolean in here and exit at this point, no need to set the callback operation in the caller as well.

Quoted 36 lines
tests/qtest/migration/framework.c +5 -16
+    *state = (QTestMigrationState) { .suspend_me = args->suspend_me };
     /*
      * MigrationCapability_lookup and MIGRATION_CAPABILITY_ constants
      * are from qapi-types-migration.h.
@@ -220,25 +222,13 @@ static void migrate_start_set_capabilities(QTestState *from, QTestState *to,
      * Enable return path first, since other features depend on it.
      */
     if (args->caps[MIGRATION_CAPABILITY_RETURN_PATH]) {
-        if (from) {
-            migrate_set_capability(from, "return-path", true);
-        }
-        if (to) {
-            migrate_set_capability(to, "return-path", true);
-        }
+        migrate_set_capability(who, "return-path", true);
     }
 
     for (uint8_t i = 0; i < MIGRATION_CAPABILITY__MAX; i++) {
-        if (!args->caps[i]) {
-            continue;
-        }
-        if (from) {
-            migrate_set_capability(from,
-                            MigrationCapability_lookup.array[i], true);
-        }
-        if (to) {
-            migrate_set_capability(to,
-                            MigrationCapability_lookup.array[i], true);
+        if (args->caps[i]) {
+            migrate_set_capability(who, MigrationCapability_lookup.array[i],
+                                   true);
         }
     }
 
@@ -246,26 +236,17 @@ static void migrate_start_set_capabilities(QTestState *from, QTestState *to,
      * Always enable migration events.  Libvirt always uses it, let's try

tests/qtest/migration/framework.c static void migrate_start_set_capabilities(QTestState *from, QTestState *to, · line 241

Quoted 10 lines
tests/qtest/migration/framework.c +2 -5
      * to mimic as closer as that.
      */
-    migrate_set_capability(from, "events", true);
-    if (!args->defer_target_connect && to) {
-        migrate_set_capability(to, "events", true);
-    }
+    migrate_set_capability(who, "events", true);
 
-    /*
+     /*

Please keep the open comment mark in the correct place.

Quoted 19 lines
tests/qtest/migration/framework.c +1 -7
      * Default number of channels should be fine for most
      * tests. Individual tests can override by calling
      * migrate_set_parameter() directly.
      */
     if (args->caps[MIGRATION_CAPABILITY_MULTIFD]) {
-        migrate_set_parameter_int(from, "multifd-channels",
+        migrate_set_parameter_int(who, "multifd-channels",
                                   MULTIFD_TEST_CHANNELS);
-        if (to) {
-            migrate_set_parameter_int(to, "multifd-channels",
-                                      MULTIFD_TEST_CHANNELS);
-        }
     }
-
-    return;
 }
 
 static char *test_shmem_path(void)

tests/qtest/migration/framework.c static char *migrate_mem_type_get_opts(MemType type, const char *memory_size) · line 296

Quoted 9 lines
tests/qtest/migration/framework.c +5 -1
@@ -308,12 +289,28 @@ static char *migrate_mem_type_get_opts(MemType type, const char *memory_size)
     return opts;
 }
 
-int migrate_args(char **from, char **to, MigrateStart *args)
+/*
+ * Build a QEMU command line for one migration participant.
+ *
+ * @serial_name  : serial output file basename under tmpfs
+ * @is_target    : set to true if the new qemu process is destination side

If we now have launch_source|dest, we shouldn't need to pass this flag in.

tests/qtest/migration/framework.c static char *migrate_mem_type_get_opts(MemType type, const char *memory_size) · line 300

Quoted 4 lines
tests/qtest/migration/framework.c +4 -0
+ * @extra_opts   : extra arguments
+ * @defer_events : when true and is_target is set, use the command-line
+ *                 event flag instead of the QMP capability (for deferred
+ *                 monitor connections)

Same here.

Quoted 24 lines
tests/qtest/migration/framework.c +11 -4
+ * @args         : common process settings
+ *
+ * Returns 0 and sets *cmd_out on success.  Returns -1 and calls
+ * g_test_skip() if the machine type is not available.
+ */
+static int migrate_build_cmd(const char *serial_name,
+                             bool is_target,
+                             const char *extra_opts,
+                             bool defer_events,
+                             MigrateStart *args,
+                             char **cmd_out)
 {
-    /* options for source and target */
     g_autofree gchar *arch_opts = NULL;
-    gchar *cmd_source = NULL;
-    gchar *cmd_target = NULL;
     const gchar *ignore_stderr;
     g_autofree char *mem_object = NULL;
     const char *kvm_opts = NULL;
@@ -323,16 +320,10 @@ int migrate_args(char **from, char **to, MigrateStart *args)
     g_autofree char *machine = NULL;
     const char *bootpath = bootfile_get();
     g_autofree char *memory_backend = NULL;
-    const char *events;

tests/qtest/migration/framework.c int migrate_args(char **from, char **to, MigrateStart *args) · line 326

Quoted 10 lines
tests/qtest/migration/framework.c +1 -6
 
     if (strcmp(arch, "i386") == 0 || strcmp(arch, "x86_64") == 0) {
         memory_size = "150M";
-
-        if (g_str_equal(arch, "i386")) {
-            machine_alias = "pc";
-        } else {
-            machine_alias = "q35";
-        }
+        machine_alias = g_str_equal(arch, "i386") ? "pc" : "q35";

We usually wouldn't include changes of this kind among other things. This could be an individual patch if you think it's worth it. (I think we should just leave it alone)

Quoted 6 lines
tests/qtest/migration/framework.c +0 -0
         arch_opts = g_strdup_printf(
             "-drive if=none,id=d0,file=%s,format=raw "
             "-device ide-hd,drive=d0,secs=1,cyls=1,heads=1", bootpath);
@@ -346,13 +337,13 @@ int migrate_args(char **from, char **to, MigrateStart *args)
         end_address = S390_TEST_MEM_END;
     } else if (strcmp(arch, "ppc64") == 0) {

tests/qtest/migration/framework.c int migrate_args(char **from, char **to, MigrateStart *args) · line 346

Quoted 10 lines
tests/qtest/migration/framework.c +2 -2
         memory_size = "256M";
-        start_address = PPC_TEST_MEM_START;
-        end_address = PPC_TEST_MEM_END;
         machine_alias = "pseries";
         machine_opts = "vsmt=8";
         arch_opts = g_strdup_printf(
             "-nodefaults -machine " PSERIES_DEFAULT_CAPABILITIES " "
             "-bios %s", bootpath);
+        start_address = PPC_TEST_MEM_START;
+        end_address = PPC_TEST_MEM_END;

This moved for no reason...

tests/qtest/migration/framework.c int migrate_args(char **from, char **to, MigrateStart *args) · line 389

Quoted 10 lines
tests/qtest/migration/framework.c +2 -1
     } else if (strcmp(arch, "aarch64") == 0) {
         memory_size = "150M";
         machine_alias = "virt";
@@ -394,53 +385,103 @@ int migrate_args(char **from, char **to, MigrateStart *args)
     }
 
     if (!qtest_has_machine(machine_alias)) {
-        g_autofree char *msg = g_strdup_printf("machine %s not supported", machine_alias);
+        g_autofree char *msg = g_strdup_printf("machine %s not supported",
+                                               machine_alias);

Same here, this can stay as it is.

Quoted 49 lines
tests/qtest/migration/framework.c +28 -13
         g_test_skip(msg);
         return -1;
     }
 
     machine = resolve_machine_version(machine_alias, QEMU_ENV_SRC,
                                       QEMU_ENV_DST);
-
     g_test_message("Using machine type: %s", machine);
 
-    cmd_source = g_strdup_printf("-accel kvm%s -accel tcg "
-                                 "-machine %s,%s "
-                                 "-name source,debug-threads=on "
-                                 "%s "
-                                 "-serial file:%s/src_serial "
-                                 "%s %s %s",
-                                 kvm_opts ? kvm_opts : "",
-                                 machine, machine_opts,
-                                 memory_backend, tmpfs,
-                                 arch_opts ? arch_opts : "",
-                                 args->opts_source ? args->opts_source : "",
-                                 ignore_stderr);
+    if (is_target) {
+        /*
+         * If the monitor connection is deferred, enable events on the command
+         * line so none are missed. This is for testing only, do not set
+         * migration options like this in general.
+         */
+        const char *events = defer_events ? "-global migration.x-events=on"
+                                          : "";
+        *cmd_out = g_strdup_printf(
+            "-accel kvm%s -accel tcg "
+            "-machine %s,%s "
+            "-name target,debug-threads=on "
+            "%s "
+            "-serial file:%s/%s "
+            "-incoming defer "
+            "%s %s %s %s",
+            kvm_opts ? kvm_opts : "",
+            machine, machine_opts,
+            memory_backend, tmpfs, serial_name,
+            events,
+            arch_opts ? arch_opts : "",
+            extra_opts ? extra_opts : "",
+            ignore_stderr);
+    } else {
+        *cmd_out = g_strdup_printf(
+            "-accel kvm%s -accel tcg "
+            "-machine %s,%s "
+            "-name source,debug-threads=on "

tests/qtest/migration/framework.c int migrate_args(char **from, char **to, MigrateStart *args) · line 435

Quoted 10 lines
tests/qtest/migration/framework.c +10 -0
+            "%s "
+            "-serial file:%s/%s "
+            "%s %s %s",
+            kvm_opts ? kvm_opts : "",
+            machine, machine_opts,
+            memory_backend, tmpfs, serial_name,
+            arch_opts ? arch_opts : "",
+            extra_opts ? extra_opts : "",
+            ignore_stderr);
+    }

Let's move all the instance-specific information into the upper levels, this conditional can go away, all information comes in via extra_opts:

    *cmd_out = g_strdup_printf(
        "-accel kvm%s -accel tcg "
        "-machine %s,%s "
        "%s "
        "-serial file:%s/%s "
        "%s %s %s",
        kvm_opts ? kvm_opts : "",
        machine, machine_opts,
        memory_backend, tmpfs, serial_name,
        arch_opts ? arch_opts : "",
        extra_opts ? extra_opts : "",
        ignore_stderr);
Quoted 23 lines
tests/qtest/migration/framework.c +0 -22
-    /*
-     * If the monitor connection is deferred, enable events on the command line
-     * so none are missed.  This is for testing only, do not set migration
-     * options like this in general.
-     */
-    events = args->defer_target_connect ? "-global migration.x-events=on" : "";
-
-    cmd_target = g_strdup_printf("-accel kvm%s -accel tcg "
-                                 "-machine %s,%s "
-                                 "-name target,debug-threads=on "
-                                 "%s "
-                                 "-serial file:%s/dest_serial "
-                                 "-incoming defer "
-                                 "%s %s %s %s",
-                                 kvm_opts ? kvm_opts : "",
-                                 machine, machine_opts,
-                                 memory_backend, tmpfs,
-                                 events,
-                                 arch_opts ? arch_opts : "",
-                                 args->opts_target ? args->opts_target : "",
-                                 ignore_stderr);
-

tests/qtest/migration/framework.c int migrate_args(char **from, char **to, MigrateStart *args) · line 444

Quoted 10 lines
tests/qtest/migration/framework.c +8 -2
-    *from = cmd_source;
-    *to = cmd_target;
+    return 0;
+}
+
+/*
+ * Launch a QEMU process as a migration source.
+ * Returns the QTestState on success, or NULL otherwise
+ */
+QTestState *migrate_launch_source(const char *serial_name, MigrateStart *args)

If this is "launch_source" we already know the serial name is "src_serial", no need to pass in.

The launch_* functions could return bool and receive the **qts as first argument.

tests/qtest/migration/framework.c int migrate_args(char **from, char **to, MigrateStart *args) · line 450

Quoted 6 lines
tests/qtest/migration/framework.c +6 -0
+{
+    g_autofree char *cmd = NULL;
+    g_autoptr(QList) capabilities = migrate_start_get_qmp_capabilities(args);
+
+    if (migrate_build_cmd(serial_name, false, args->opts_source,
+                          false, args, &cmd)) {

Here's what I'm thinking to remove the instance-specific information from migrate_build_cmd:

static bool migrate_launch_instance(QTestState **qts, MigrateStart *args,
QTestMigrationState *state, const char *env, bool defer_connect, char *cmd)

{
    g_autoptr(QList) capabilities = migrate_start_get_qmp_capabilities(args);

    *qts = qtest_init_ext(env, cmd, capabilities, !defer_connect);
    if (*qts) {
        migrate_setup_instance(*qts, state, args, defer_connect);
    }

    return !!*qts;
}

static bool migrate_build_dest(MigrateStart *args, char **cmd)

{
    g_autofree char *opts = NULL;
    int r;

    if (args->only_source) {
        return true;
    }

    /*
     * If the monitor connection is deferred, enable events on the command
     * line so none are missed. This is for testing only, do not set
     * migration options like this in general.
     */

    opts = g_strdup_printf(
        "-name target,debug-threads=on "
        "-incoming defer "
        "%s %s",
        args->defer_target_connect ? "-global migration.x-events=on" : "",
        args->opts_target ?: "");

    r = migrate_build_cmd("dest_serial", opts, args, cmd);
    return r == 0;
}

static bool migrate_launch_dest(QTestState **qts, MigrateStart *args)

{
    g_autofree char *cmd = NULL;

    if (!migrate_build_dest(args, &cmd)){
        return false;
    }

    if (cmd && !migrate_launch_instance(qts, args, &dst_state, QEMU_ENV_DST,
                                        args->defer_target_connect, cmd)) {
        return false;
    }

    return true;
}

migrate_args -> migrate_build_dest
migrate_start -> migrate_launch_dest -> migrate_build_dest

And similarly for the source.

What do you think?

Quoted 51 lines
tests/qtest/migration/framework.c +34 -6
+        return NULL;
+    }
+    return qtest_init_ext(QEMU_ENV_SRC, cmd, capabilities, true);
+}
+
+/*
+ * Launch a QEMU process as a migration destination.
+ * Returns the QTestState on success, or NULL otherwise
+ */
+QTestState *migrate_launch_dest(const char *serial_name, MigrateStart *args)
+{
+    g_autofree char *cmd = NULL;
+    g_autoptr(QList) capabilities = migrate_start_get_qmp_capabilities(args);
+
+    if (migrate_build_cmd(serial_name, true, args->opts_target,
+                          args->defer_target_connect, args, &cmd)) {
+        return NULL;
+    }
+    return qtest_init_ext(QEMU_ENV_DST, cmd, capabilities,
+                          !args->defer_target_connect);
+}
+
+int migrate_args(char **from, char **to, MigrateStart *args)
+{
+    if (migrate_build_cmd("src_serial", false, args->opts_source,
+                          false, args, from)) {
+        return -1;
+    }
+    if (migrate_build_cmd("dest_serial", true, args->opts_target,
+                          args->defer_target_connect, args, to)) {
+        g_free(*from);
+        *from = NULL;
+        return -1;
+    }
     return 0;
 }
 
@@ -482,43 +523,45 @@ static void migrate_mem_type_cleanup(MemType type)
 
 int migrate_start(QTestState **from, QTestState **to, MigrateStart *args)
 {
-    g_autofree gchar *cmd_source = NULL;
-    g_autofree gchar *cmd_target = NULL;
-    g_autoptr(QList) capabilities = migrate_start_get_qmp_capabilities(args);
-
     if (!migrate_mem_type_prepare(args->mem_type)) {
         return -1;
     }
 
-    dst_state = (QTestMigrationState) { };
-    src_state = (QTestMigrationState) { };

tests/qtest/migration/framework.c static void migrate_mem_type_cleanup(MemType type) · line 534

Quoted 10 lines
tests/qtest/migration/framework.c +2 -5
+    src_state = (QTestMigrationState){};
+    dst_state = (QTestMigrationState){};
     bootfile_create(qtest_get_arch(), tmpfs, args->suspend_me);
-    src_state.suspend_me = args->suspend_me;
-
-    if (migrate_args(&cmd_source, &cmd_target, args)) {
-        return -1;
-    }
 
     if (!args->only_target) {

The only_* checks can go into the _launch_source level, keeps this function cleaner.

Quoted 15 lines
tests/qtest/migration/framework.c +5 -7
-        *from = qtest_init_ext(QEMU_ENV_SRC, cmd_source, capabilities, true);
-        qtest_qmp_set_event_callback(*from,
-                                     migrate_watch_for_events,
-                                     &src_state);
+        *from = migrate_launch_source("src_serial", args);
+        if (!*from) {
+            return -1;
+        }
+        migrate_setup_instance(*from, &src_state, args);
     }
 
     if (!args->only_source) {
-        *to = qtest_init_ext(QEMU_ENV_DST, cmd_target, capabilities,
-                             !args->defer_target_connect);
-        qtest_qmp_set_event_callback(*to,

tests/qtest/migration/framework.c static void migrate_mem_type_cleanup(MemType type) · line 550

Quoted 10 lines
tests/qtest/migration/framework.c +8 -2
-                                     migrate_watch_for_events,
-                                     &dst_state);
+        *to = migrate_launch_dest("dest_serial", args);
+        if (!*to) {
+            if (!args->only_target) {
+                qtest_quit(*from);
+            }
+            return -1;
+        }
+        if (!args->defer_target_connect) {

This check would move into migrate_setup_instance.

Quoted 39 lines
tests/qtest/migration/framework.c +11 -4
+            migrate_setup_instance(*to, &dst_state, args);
+        } else {
+            /*
+             * The monitor is not connected yet; only register the event
+             * callback so no events are missed.
+             * The caller must call migrate_setup_instance() after
+             * qtest_connect() if it wants to setup any capability
+             */
+            qtest_qmp_set_event_callback(*to, migrate_watch_for_events,
+                                         &dst_state);
+        }
     }
 
     migrate_mem_type_cleanup(args->mem_type);
-    migrate_start_set_capabilities(*from,
-                                   args->only_source ? NULL : *to,
-                                   args);
-
     return 0;
 }
 
tests/qtest/migration/framework.h +6 -0
index 941cbd7102..f054ef8f4d 100644
@@ -225,10 +225,27 @@ typedef struct {
     bool live;
 } MigrateCommon;
 
+typedef struct QTestMigrationState QTestMigrationState;
+
 void wait_for_serial(const char *side);
 void migrate_prepare_for_dirty_mem(QTestState *from);
 void migrate_wait_for_dirty_mem(QTestState *from, QTestState *to);
 
+/*
+ * Start a QEMU process which has serial set to @serial_name
+ */
+QTestState *migrate_launch_source(const char *serial_name, MigrateStart *args);

tests/qtest/migration/framework.h typedef struct { · line 247

Quoted 10 lines
tests/qtest/migration/framework.h +10 -0
+/*
+ * Start a QEMU process which has serial set to @serial_name
+ */
+QTestState *migrate_launch_dest(const char *serial_name, MigrateStart *args);
+/*
+ * Registers the event callback and sets migration capabilities on an
+ * already-running process
+ */
+void migrate_setup_instance(QTestState *who, QTestMigrationState *state,
+                   MigrateStart *args);

These don't need to be exported, keep them static.

Quoted 11 lines
tests/qtest/migration/framework.h +1 -1
+
 int migrate_args(char **from, char **to, MigrateStart *args);
 int migrate_start(QTestState **from, QTestState **to, MigrateStart *args);
 void migrate_end(QTestState *from, QTestState *to, bool test_dest);
@@ -240,7 +257,6 @@ int test_precopy_common(MigrateCommon *args);
 void test_precopy_unix_common(MigrateCommon *args);
 void test_file_common(MigrateCommon *args, bool stop_src);
 
-typedef struct QTestMigrationState QTestMigrationState;
 QTestMigrationState *get_src(void);
 QTestMigrationState *get_dst(void);
#3 lore raw

PATCH 2/3 tests/migration: Retrieve serial path from QTestState

In reply to [email protected]

From: Nguyen Dinh Phi <[email protected]>

Replace hard-coded side names ("src_serial", "dest_serial") passed to wait_for_serial() with qtest_get_serial_path(). The path is stored on the QTestState at launch time by migrate_launch_source/dest().

Hard-coded names break when a process changes role, for example when a completed destination is re-used as a new source in a chain migration.

---
 tests/qtest/libqtest.c                | 16 ++++++++++++
 tests/qtest/libqtest.h                |  8 ++++++
 tests/qtest/migration/colo-tests.c    | 14 +++++-----
 tests/qtest/migration/cpr-tests.c     |  8 +++---
 tests/qtest/migration/framework.c     | 37 ++++++++++++++++-----------
 tests/qtest/migration/framework.h     |  2 +-
 tests/qtest/migration/misc-tests.c    |  4 +--
 tests/qtest/migration/precopy-tests.c | 28 +++++++++++---------
 8 files changed, 76 insertions(+), 41 deletions(-)
tests/qtest/libqtest.c +16 -0
index c33c799c92..3fccd089dd 100644
@@ -93,6 +93,7 @@ struct QTestState
     GList *pending_events;
     QTestQMPEventCallback eventCB;
     void *eventData;
+    char *serial_path;
 };
 
 static GHookList abrt_hooks;
@@ -688,10 +689,25 @@ void qtest_quit(QTestState *s)
     }
 
     g_list_free(s->pending_events);
+    if (s->serial_path) {
+        unlink(s->serial_path);
+        g_free(s->serial_path);
+    }
 
     g_free(s);
 }
 
+void qtest_set_serial_path(QTestState *s, const char *path)
+{
+    g_free(s->serial_path);
+    s->serial_path = g_strdup(path);
+}
+
+const char *qtest_get_serial_path(QTestState *s)
+{
+    return s->serial_path;
+}
+
 static void socket_send(int fd, const char *buf, size_t size)
 {
     ssize_t res = qemu_send_full(fd, buf, size);
tests/qtest/libqtest.h +8 -0
index 45217fb8dc..0958cf1662 100644
@@ -98,6 +98,14 @@ QTestState *qtest_init(const char *extra_args);
 QTestState *qtest_init_ext(const char *var, const char *extra_args,
                            QList *capabilities, bool do_connect);
 
+/**
+ * qtest_set_serial_path:
+ * @s: #QTestState instance
+ * @path: full path to the serial output file to unlink on qtest_quit()
+ */
+void qtest_set_serial_path(QTestState *s, const char *path);
+const char *qtest_get_serial_path(QTestState *s);
+
 /**
  * qtest_init_without_qmp_handshake:
  * @extra_args: other arguments to pass to QEMU.  CAUTION: these
tests/qtest/migration/colo-tests.c +7 -7
index f7f9ba491b..c6b6faa578 100644
@@ -58,21 +58,21 @@ static int test_colo_common(MigrateCommon *args,
     migrate_incoming_qmp(to, args->uri, NULL, "{}");
 
     migrate_ensure_converge(from);
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
 
     migrate_qmp(from, to, NULL, NULL, "{}");
 
     wait_for_migration_status(from, "colo", NULL);
     wait_for_resume(to, get_dst());
 
-    wait_for_serial("src_serial");
-    wait_for_serial("dest_serial");
+    wait_for_serial(qtest_get_serial_path(from));
+    wait_for_serial(qtest_get_serial_path(to));
 
     /* wait for 3 checkpoints */
     for (int i = 0; i < 3; i++) {
         qtest_qmp_eventwait(to, "RESUME");
-        wait_for_serial("src_serial");
-        wait_for_serial("dest_serial");
+        wait_for_serial(qtest_get_serial_path(from));
+        wait_for_serial(qtest_get_serial_path(to));
     }
 
     if (failover_during_checkpoint) {
@@ -83,13 +83,13 @@ static int test_colo_common(MigrateCommon *args,
                                             "'arguments': {'instances':"
                                                 "[{'type': 'migration'}]}}");
         qtest_qmp_assert_success(from, "{'execute': 'x-colo-lost-heartbeat'}");
-        wait_for_serial("src_serial");
+        wait_for_serial(qtest_get_serial_path(from));
     } else {
         qtest_qmp_assert_success(to, "{'exec-oob': 'yank', 'id': 'yank-cmd', "
                                         "'arguments': {'instances':"
                                             "[{'type': 'migration'}]}}");
         qtest_qmp_assert_success(to, "{'execute': 'x-colo-lost-heartbeat'}");
-        wait_for_serial("dest_serial");
+        wait_for_serial(qtest_get_serial_path(to));
     }
 
     if (args->end_hook) {
tests/qtest/migration/cpr-tests.c +4 -4
index 0bec753b4a..f56bbe48fe 100644
@@ -66,7 +66,7 @@ static int test_transfer(MigrateCommon *args, const char *cpr_channel,
 
     migrate_set_parameter_str(from, "mode", "cpr-transfer");
 
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
 
     qtest_qmp_assert_success(from, "{ 'execute' : 'stop'}");
     wait_for_stop(from, get_src());
@@ -89,7 +89,7 @@ static int test_transfer(MigrateCommon *args, const char *cpr_channel,
     qtest_qmp_assert_success(to, "{ 'execute' : 'cont'}");
 
     wait_for_resume(to, get_dst());
-    wait_for_serial("dest_serial");
+    wait_for_serial(qtest_get_serial_path(to));
 
     migrate_end(from, to, true);
 
@@ -241,7 +241,7 @@ static void test_cpr_exec(MigrateCommon *args)
         data_hook = args->start_hook(from, NULL);
     }
 
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
     set_cpr_exec_args(from, args);
     migrate_set_capability(from, "events", true);
     migrate_qmp(from, NULL, connect_uri, NULL, "{}");
@@ -260,7 +260,7 @@ static void test_cpr_exec(MigrateCommon *args)
 
     wait_for_resume(to, get_dst());
     /* Device on target is still named src_serial because args do not change */
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
 
     if (args->end_hook) {
         args->end_hook(from, to, data_hook);
tests/qtest/migration/framework.c +22 -15
index 305b8968ff..a7dffeb82c 100644
@@ -58,10 +58,9 @@ static char *tmpfs;
  * we get an 'A' followed by an endless string of 'B's
  * but on the destination we won't have the A (unless we enabled suspend/resume)
  */
-void wait_for_serial(const char *side)
+void wait_for_serial(const char *path)
 {
-    g_autofree char *serialpath = g_strdup_printf("%s/%s", tmpfs, side);
-    FILE *serialfile = fopen(serialpath, "r");
+    FILE *serialfile = fopen(path, "r");
 
     do {
         int readvalue = fgetc(serialfile);
@@ -82,7 +81,7 @@ void wait_for_serial(const char *side)
             break;
 
         default:
-            fprintf(stderr, "Unexpected %d on %s serial\n", readvalue, side);
+            fprintf(stderr, "Unexpected %d on %s serial\n", readvalue, path);
             g_assert_not_reached();
         }
     } while (true);
@@ -450,7 +449,12 @@ QTestState *migrate_launch_source(const char *serial_name, MigrateStart *args)
                           false, args, &cmd)) {
         return NULL;
     }
-    return qtest_init_ext(QEMU_ENV_SRC, cmd, capabilities, true);
+    QTestState *s = qtest_init_ext(QEMU_ENV_SRC, cmd, capabilities, true);
+    if (s) {
+        g_autofree char *path = g_strdup_printf("%s/%s", tmpfs, serial_name);
+        qtest_set_serial_path(s, path);
+    }
+    return s;
 }
 
 /*
@@ -466,8 +470,13 @@ QTestState *migrate_launch_dest(const char *serial_name, MigrateStart *args)
                           args->defer_target_connect, args, &cmd)) {
         return NULL;
     }
-    return qtest_init_ext(QEMU_ENV_DST, cmd, capabilities,
-                          !args->defer_target_connect);
+    QTestState *s = qtest_init_ext(QEMU_ENV_DST, cmd, capabilities,
+                                   !args->defer_target_connect);
+    if (s) {
+        g_autofree char *path = g_strdup_printf("%s/%s", tmpfs, serial_name);
+        qtest_set_serial_path(s, path);
+    }
+    return s;
 }
 
 int migrate_args(char **from, char **to, MigrateStart *args)
@@ -595,8 +604,6 @@ void migrate_end(QTestState *from, QTestState *to, bool test_dest)
 
     cleanup("migsocket");
     cleanup("cpr.sock");
-    cleanup("src_serial");
-    cleanup("dest_serial");
     cleanup(FILE_TEST_FILENAME);
 }
 
@@ -631,7 +638,7 @@ static int migrate_postcopy_prepare(QTestState **from_ptr,
                              "                'port': '0' } } ] } }");
 
     /* Wait for the first serial output from the source */
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
     wait_for_suspend(from, &src_state);
 
     migrate_qmp(from, to, NULL, NULL, "{}");
@@ -657,7 +664,7 @@ static void migrate_postcopy_complete(QTestState *from, QTestState *to,
     }
 
     /* Make sure we get at least one "B" on destination */
-    wait_for_serial("dest_serial");
+    wait_for_serial(qtest_get_serial_path(to));
 
     if (env->uffd_feature_thread_id) {
         read_blocktime(to);
@@ -895,7 +902,7 @@ int test_precopy_common(MigrateCommon *args)
 
     /* Wait for the first serial output from the source */
     if (args->result == MIG_TEST_SUCCEED) {
-        wait_for_serial("src_serial");
+        wait_for_serial(qtest_get_serial_path(from));
         wait_for_suspend(from, &src_state);
     }
 
@@ -972,7 +979,7 @@ int test_precopy_common(MigrateCommon *args)
             qtest_qmp_assert_success(to, "{'execute': 'system_wakeup'}");
         }
 
-        wait_for_serial("dest_serial");
+        wait_for_serial(qtest_get_serial_path(to));
     }
 
 finish:
@@ -1063,7 +1070,7 @@ void test_file_common(MigrateCommon *args, bool stop_src)
     }
 
     migrate_ensure_converge(from);
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
 
     if (stop_src) {
         qtest_qmp_assert_success(from, "{ 'execute' : 'stop'}");
@@ -1090,7 +1097,7 @@ void test_file_common(MigrateCommon *args, bool stop_src)
     }
     wait_for_resume(to, &dst_state);
 
-    wait_for_serial("dest_serial");
+    wait_for_serial(qtest_get_serial_path(to));
 
     if (check_offset) {
         file_check_offset_region();
tests/qtest/migration/framework.h +1 -1
index f054ef8f4d..f6936eb8c6 100644
@@ -227,7 +227,7 @@ typedef struct {
 
 typedef struct QTestMigrationState QTestMigrationState;
 
-void wait_for_serial(const char *side);
+void wait_for_serial(const char *path);
 void migrate_prepare_for_dirty_mem(QTestState *from);
 void migrate_wait_for_dirty_mem(QTestState *from, QTestState *to);
 
tests/qtest/migration/misc-tests.c +2 -2
index ec6d438cdc..73750335f1 100644
@@ -128,7 +128,7 @@ static void do_test_validate_uuid(MigrateStart *args, bool should_fail)
     migrate_set_capability(from, "validate-uuid", true);
 
     /* Wait for the first serial output from the source */
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
 
     migrate_incoming_qmp(to, uri, NULL, "{}");
     migrate_qmp(from, to, uri, NULL, "{}");
@@ -185,7 +185,7 @@ static void do_test_validate_uri_channel(MigrateCommon *args)
     }
 
     /* Wait for the first serial output from the source */
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
 
     migrate_incoming_qmp(to, "tcp:127.0.0.1:0", NULL, "{}");
 
tests/qtest/migration/precopy-tests.c +16 -12
index a23d51126b..194462c841 100644
@@ -282,7 +282,8 @@ static void test_auto_converge(char *name, MigrateCommon *args)
     /* To check remaining size after precopy */
     migrate_set_capability(from, "pause-before-switchover", true);
 
-    wait_for_serial("src_serial");
+    /* Wait for the first serial output from the source */
+    wait_for_serial(qtest_get_serial_path(from));
 
     migrate_incoming_qmp(to, uri, NULL, "{}");
     migrate_qmp(from, to, uri, NULL, "{}");
@@ -317,7 +318,7 @@ static void test_auto_converge(char *name, MigrateCommon *args)
 
     qtest_qmp_eventwait(to, "RESUME");
 
-    wait_for_serial("dest_serial");
+    wait_for_serial(qtest_get_serial_path(to));
     wait_for_migration_complete(from);
 
     migrate_end(from, to, true);
@@ -436,7 +437,7 @@ static void test_multifd_tcp_cancel(MigrateCommon *args, bool postcopy_ram)
     migrate_incoming_qmp(to, "tcp:127.0.0.1:0", NULL, "{}");
 
     /* Wait for the first serial output from the source */
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
 
     migrate_qmp(from, to, NULL, NULL, "{}");
 
@@ -486,7 +487,7 @@ static void test_multifd_tcp_cancel(MigrateCommon *args, bool postcopy_ram)
     wait_for_stop(from, get_src());
     qtest_qmp_eventwait(to2, "RESUME");
 
-    wait_for_serial("dest_serial");
+    wait_for_serial(qtest_get_serial_path(to2));
     wait_for_migration_complete(from);
     migrate_end(from, to2, true);
 }
@@ -510,7 +511,7 @@ static void test_cancel_src_after_failed(QTestState *from, QTestState *to,
      * failed state during migrate_qmp().
      */
 
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
     migrate_ensure_converge(from);
 
     migrate_qmp(from, to, uri, NULL, "{}");
@@ -535,7 +536,7 @@ static void test_cancel_src_after_cancelled(QTestState *from, QTestState *to,
 {
     migrate_incoming_qmp(to, uri, NULL, "{}");
 
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
     migrate_ensure_converge(from);
 
     migrate_qmp(from, to, uri, NULL, "{}");
@@ -560,7 +561,7 @@ static void test_cancel_src_after_complete(QTestState *from, QTestState *to,
 {
     migrate_incoming_qmp(to, uri, NULL, "{}");
 
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
     migrate_ensure_converge(from);
 
     migrate_qmp(from, to, uri, NULL, "{}");
@@ -586,7 +587,7 @@ static void test_cancel_src_after_none(QTestState *from, QTestState *to,
      */
     migrate_cancel(to);
 
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
     migrate_cancel(from);
 
     migrate_incoming_qmp(to, uri, NULL, "{}");
@@ -610,7 +611,7 @@ static void test_cancel_src_pre_switchover(QTestState *from, QTestState *to,
 
     migrate_incoming_qmp(to, uri, NULL, "{}");
 
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
     migrate_ensure_converge(from);
 
     migrate_qmp(from, to, uri, NULL, "{}");
@@ -831,7 +832,10 @@ static void test_vcpu_dirty_limit(char *name, MigrateCommon *args)
     vm = dirtylimit_start_vm();
 
     /* Wait for the first serial output from the vm*/
-    wait_for_serial("vm_serial");
+    {
+        g_autofree char *vm_serial = g_strdup_printf("%s/vm_serial", tmpfs);
+        wait_for_serial(vm_serial);
+    }
 
     /* Do dirtyrate measurement with calc time equals 1s */
     calc_dirty_rate(vm, 1);
@@ -924,7 +928,7 @@ static void migrate_dirty_limit_wait_showup(QTestState *from,
     migrate_set_capability(from, "pause-before-switchover", true);
 
     /* Wait for the serial output from the source */
-    wait_for_serial("src_serial");
+    wait_for_serial(qtest_get_serial_path(from));
 }
 
 /*
@@ -1062,7 +1066,7 @@ static void test_dirty_limit(char *name, MigrateCommon *args)
 
     qtest_qmp_eventwait(to, "RESUME");
 
-    wait_for_serial("dest_serial");
+    wait_for_serial(qtest_get_serial_path(to));
     wait_for_migration_complete(from);
 
     migrate_end(from, to, true);
Signature · 1 line
2.50.1 (Apple Git-155)
#4 lore raw

PATCH 3/3 tests/migration: Add chain (A to B to C) precopy migration tests

In reply to [email protected]

From: Nguyen Dinh Phi <[email protected]>

Extract migrate_precopy_hop() from the SUCCESS path of test_precopy_common() to wrap the per-hop migration sequence of a succesfull migration. test_precopy_common() delegates its success path to this helper; failure paths are handled directly.

Introduce test_precopy_chain(), a helper that drives two consecutive migration hops on already-running processes: first A→B, then B→C, both hops are expected to be success.

Add two test cases exercising the new helper:
- /migration/precopy/tcp/plain/chain
- /migration/multifd/tcp/channels/plain/chain

---
 tests/qtest/migration/framework.c     | 195 +++++++++++++++++---------
 tests/qtest/migration/framework.h     |   1 +
 tests/qtest/migration/precopy-tests.c |  23 +++
 3 files changed, 153 insertions(+), 66 deletions(-)
tests/qtest/migration/framework.c +129 -66
index a7dffeb82c..b4218b2f23 100644
@@ -883,28 +883,25 @@ void test_postcopy_recovery_common(MigrateCommon *args,
     migrate_postcopy_complete(from, to, hook_data, args);
 }
 
-int test_precopy_common(MigrateCommon *args)
+/*
+ * Perform one precopy migration hop on an already-running (from, to) pair.
+ * Both processes must have migrate_setup_instance() called before this
+ * function.
+ * Always expects the hop to succeed.
+ *
+ */
+static void migrate_precopy_hop(QTestState *from, QTestState *to,
+                                 const char *listen_uri,
+                                 const char *connect_uri,
+                                 MigrateCommon *args)
 {
-    QTestState *from, *to;
-    void *data_hook = NULL;
     QObject *channels = NULL;
-    const char *listen_uri = args->uri ?: "tcp:127.0.0.1:0";
-
-    if (migrate_start(&from, &to, &args->start)) {
-        return -1;
-    }
-
-    if (args->start_hook) {
-        data_hook = args->start_hook(from, to);
-    }
+    int iters;
 
     migrate_incoming_qmp(to, listen_uri, NULL, "{}");
 
-    /* Wait for the first serial output from the source */
-    if (args->result == MIG_TEST_SUCCEED) {
-        wait_for_serial(qtest_get_serial_path(from));
-        wait_for_suspend(from, &src_state);
-    }
+    wait_for_serial(qtest_get_serial_path(from));
+    wait_for_suspend(from, &src_state);
 
     if (args->live) {
         migrate_ensure_non_converge(from);
@@ -913,76 +910,104 @@ int test_precopy_common(MigrateCommon *args)
         /*
          * Testing non-live migration, we allow it to run at
          * full speed to ensure short test case duration.
-         * For tests expected to fail, we don't need to
-         * change anything.
          */
-        if (args->result == MIG_TEST_SUCCEED) {
-            qtest_qmp_assert_success(from, "{ 'execute' : 'stop'}");
-            wait_for_stop(from, &src_state);
-            migrate_ensure_converge(from);
-        }
+        qtest_qmp_assert_success(from, "{ 'execute' : 'stop'}");
+        wait_for_stop(from, &src_state);
+        migrate_ensure_converge(from);
     }
 
     if (args->connect_channels) {
         channels = qobject_from_json(args->connect_channels, &error_abort);
     }
 
-    if (args->result == MIG_TEST_QMP_ERROR) {
-        migrate_qmp_fail(from, args->uri, channels, "{}");
-        goto finish;
-    }
+    migrate_qmp(from, to, connect_uri, channels, "{}");
 
-    migrate_qmp(from, to, args->uri, channels, "{}");
+    if (args->live) {
+        /*
+         * For initial iteration(s) we must do a full pass,
+         * but for the final iteration, we need only wait
+         * for some dirty mem before switching to converge.
+         */
+        /*
+         * Avoid changing the args->iterations by using a local copy
+         */
+        iters = args->iterations;
+        while (iters > 1) {
+            wait_for_migration_pass(from, &src_state);
+            iters--;
+        }
+        migrate_wait_for_dirty_mem(from, to);
+
+        migrate_ensure_converge(from);
+
+        /*
+         * We do this first, as it has a timeout to stop us
+         * hanging forever if migration didn't converge
+         */
+        wait_for_migration_complete(from);
+
+        wait_for_stop(from, &src_state);
 
-    if (args->result != MIG_TEST_SUCCEED) {
-        bool allow_active = args->result == MIG_TEST_FAIL;
-        wait_for_migration_fail(from, allow_active);
     } else {
-        if (args->live) {
-            /*
-             * For initial iteration(s) we must do a full pass,
-             * but for the final iteration, we need only wait
-             * for some dirty mem before switching to converge
-             */
-            while (args->iterations > 1) {
-                wait_for_migration_pass(from, &src_state);
-                args->iterations--;
-            }
-            migrate_wait_for_dirty_mem(from, to);
+        wait_for_migration_complete(from);
+        /*
+         * Must wait for dst to finish reading all incoming
+         * data on the socket before issuing 'cont' otherwise
+         * it'll be ignored
+         */
+        wait_for_migration_complete(to);
 
-            migrate_ensure_converge(from);
+        qtest_qmp_assert_success(to, "{ 'execute' : 'cont'}");
+    }
 
-            /*
-             * We do this first, as it has a timeout to stop us
-             * hanging forever if migration didn't converge
-             */
-            wait_for_migration_complete(from);
+    wait_for_resume(to, &dst_state);
+
+    if (args->start.suspend_me) {
+        /* wakeup succeeds only if guest is suspended */
+        qtest_qmp_assert_success(to, "{'execute': 'system_wakeup'}");
+    }
 
-            wait_for_stop(from, &src_state);
+    wait_for_serial(qtest_get_serial_path(to));
+}
 
-        } else {
-            wait_for_migration_complete(from);
-            /*
-             * Must wait for dst to finish reading all incoming
-             * data on the socket before issuing 'cont' otherwise
-             * it'll be ignored
-             */
-            wait_for_migration_complete(to);
+int test_precopy_common(MigrateCommon *args)
+{
+    QTestState *from, *to;
+    void *data_hook = NULL;
+    const char *listen_uri = args->uri ?: "tcp:127.0.0.1:0";
 
-            qtest_qmp_assert_success(to, "{ 'execute' : 'cont'}");
-        }
+    if (migrate_start(&from, &to, &args->start)) {
+        return -1;
+    }
+
+    if (args->start_hook) {
+        data_hook = args->start_hook(from, to);
+    }
 
-        wait_for_resume(to, &dst_state);
+    if (args->result == MIG_TEST_SUCCEED) {
+        migrate_precopy_hop(from, to, listen_uri, args->uri, args);
+    } else {
+        QObject *channels = NULL;
+
+        migrate_incoming_qmp(to, listen_uri, NULL, "{}");
+
+        if (args->live) {
+            migrate_ensure_non_converge(from);
+            migrate_prepare_for_dirty_mem(from);
+        }
 
-        if (args->start.suspend_me) {
-            /* wakeup succeeds only if guest is suspended */
-            qtest_qmp_assert_success(to, "{'execute': 'system_wakeup'}");
+        if (args->connect_channels) {
+            channels = qobject_from_json(args->connect_channels, &error_abort);
         }
 
-        wait_for_serial(qtest_get_serial_path(to));
+        if (args->result == MIG_TEST_QMP_ERROR) {
+            migrate_qmp_fail(from, args->uri, channels, "{}");
+        } else {
+            migrate_qmp(from, to, args->uri, channels, "{}");
+            wait_for_migration_fail(from, args->result == MIG_TEST_FAIL);
+        }
     }
 
-finish:
     if (args->end_hook) {
         args->end_hook(from, to, data_hook);
     }
@@ -1000,6 +1025,44 @@ void test_precopy_unix_common(MigrateCommon *args)
     test_precopy_common(args);
 }
 
+int test_precopy_chain(MigrateCommon *args)
+{
+    QTestState *from, *to, *to2;
+    const char *listen_uri = args->uri ?: "tcp:127.0.0.1:0";
+
+    /*
+     * Hook callbacks operate on a fixed (from, to) pair; they do not
+     * compose across multiple migration legs.
+     */
+    g_assert(!args->start_hook && !args->end_hook);
+
+    if (migrate_start(&from, &to, &args->start)) {
+        return -1;
+    }
+
+    /* First leg: A -> B */
+    migrate_precopy_hop(from, to, listen_uri, args->uri, args);
+
+    /*
+     * Second leg: B -> C.
+     */
+    qtest_quit(from);
+    migrate_setup_instance(to, &src_state, &args->start);
+
+    to2 = migrate_launch_dest("dest_serial2", &args->start);
+    if (!to2) {
+        qtest_quit(to);
+        return -1;
+    }
+    migrate_setup_instance(to2, &dst_state, &args->start);
+
+    migrate_precopy_hop(to, to2, listen_uri, NULL, args);
+
+    migrate_end(to, to2, true);
+
+    return 0;
+}
+
 static void file_dirty_offset_region(void)
 {
     g_autofree char *path = g_strdup_printf("%s/%s", tmpfs, FILE_TEST_FILENAME);
tests/qtest/migration/framework.h +1 -0
index f6936eb8c6..ca5289329a 100644
@@ -254,6 +254,7 @@ void test_postcopy_common(MigrateCommon *args);
 void test_postcopy_recovery_common(MigrateCommon *args,
                                    PostcopyRecoveryFailStage fail_stage);
 int test_precopy_common(MigrateCommon *args);
+int test_precopy_chain(MigrateCommon *args);
 void test_precopy_unix_common(MigrateCommon *args);
 void test_file_common(MigrateCommon *args, bool stop_src);
 
tests/qtest/migration/precopy-tests.c +23 -0
index 194462c841..fccff76ef6 100644
@@ -182,6 +182,11 @@ static void test_precopy_tcp_plain(char *name, MigrateCommon *args)
     test_precopy_common(args);
 }
 
+static void test_precopy_tcp_plain_chain(char *name, MigrateCommon *args)
+{
+    test_precopy_chain(args);
+}
+
 static void test_precopy_tcp_switchover_ack(char *name, MigrateCommon *args)
 {
     /*
@@ -398,6 +403,20 @@ static void test_multifd_tcp_channels_none(char *name, MigrateCommon *args)
     test_precopy_common(args);
 }
 
+static void test_multifd_tcp_channels_chain(char *name, MigrateCommon *args)
+{
+    args->live = true;
+    args->connect_channels = ("[ { 'channel-type': 'main',"
+                             "    'addr': { 'transport': 'socket',"
+                             "              'type': 'inet',"
+                             "              'host': '127.0.0.1',"
+                             "              'port': '0' } } ]");
+
+    args->start.caps[MIGRATION_CAPABILITY_MULTIFD] = true;
+
+    test_precopy_chain(args);
+}
+
 /*
  * This test does:
  *  source               target
@@ -1112,6 +1131,8 @@ void migration_test_add_precopy(MigrationTestEnv *env)
         return;
     }
 
+    migration_test_add("/migration/precopy/tcp/plain/chain",
+                       test_precopy_tcp_plain_chain);
     migration_test_add("/migration/precopy/tcp/plain/switchover-ack",
                        test_precopy_tcp_switchover_ack);
 
@@ -1133,6 +1154,8 @@ void migration_test_add_precopy(MigrationTestEnv *env)
     }
     migration_test_add("/migration/multifd/tcp/channels/plain/none",
                        test_multifd_tcp_channels_none);
+    migration_test_add("/migration/multifd/tcp/channels/plain/chain",
+                       test_multifd_tcp_channels_chain);
     migration_test_add("/migration/multifd/tcp/plain/zero-page/legacy",
                        test_multifd_tcp_zero_page_legacy);
     migration_test_add("/migration/multifd/tcp/plain/zero-page/none",
Signature · 1 line
2.50.1 (Apple Git-155)
#5 lore raw
In reply to [email protected]
Quoted 19 lines · [email protected] writes:

From: Nguyen Dinh Phi <[email protected]>

This series adds test infrastructure and test cases for multi-hop chain migration, where a VM migrates from host A to B and then from B to C.

The migration test TODO list mentions ping-pong migration (A→B→A) as a desirable scenario; the true ping-pong (A→B→A) would require the original source A to become a destination for the return leg.
This is not possible: a QEMU process launched without -incoming cannot later accept an incoming migration stream. Instead, this series implements chain migration (A→B→C): B was already launched with -incoming and can be re-wired as the source for the next hop, while a fresh QEMU instance C is started as the new destination.

Nguyen Dinh Phi (3):
tests/migration: Factor out per-process launch and setup helpers tests/migration: Retrieve serial path from QTestState tests/migration: Add chain (A to B to C) precopy migration tests

Quoted 9 lines
 tests/qtest/libqtest.c                |  16 +
 tests/qtest/libqtest.h                |   8 +
 tests/qtest/migration/colo-tests.c    |  14 +-
 tests/qtest/migration/cpr-tests.c     |   8 +-
 tests/qtest/migration/framework.c     | 475 ++++++++++++++++----------
 tests/qtest/migration/framework.h     |  21 +-
 tests/qtest/migration/misc-tests.c    |   4 +-
 tests/qtest/migration/precopy-tests.c |  51 ++-
 8 files changed, 389 insertions(+), 208 deletions(-)

This has been attempted before, but we ended up not continuing the work for other reasons. I think your ideas on the first patch are a step in the right direction. As for the serial work, I'm not so sure we should be adding migration-specific data into QTestState, even though the serial is not technically exclusive to migration. I need to give it a bit more thought.

Here are the links to the previous work (for reference, I don't have plans to carry it forward at the moment):

Cleanups and dealing with serial:
https://lore.kernel.org/all/[email protected]/
https://lore.kernel.org/all/[email protected]/
https://lore.kernel.org/all/[email protected]/

Ping-pong test:
https://lore.kernel.org/all/[email protected]/