Skip to contentSkip to conversation

tests: introduce testing coverage for TLS with migration

PATCH Superseded Daniel P. Berrangé 5 participants 48 messages Mar 2, 2022 – Mar 10, 2022

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover tests: introduce testing coverage for TLS with migration

This significantly expands the migration test suite to cover testing with TLS over TCP and UNIX sockets, with both PSK (pre shared keys) and x509 credentials, and for both single and multifd scenarios.

It identified one bug in handling PSK credentials with UNIX sockets, but other than that everything was operating as expected.

To minimize the impact on code duplication alopt of refactoring is done of the migration tests to introduce a common helper for running the migration process. The various tests mostly just have to provide a callback to set a few parameters/capabilities before migration starts, and sometimes a callback to cleanup or validate after completion/failure.

Daniel P. Berrangé (18):
tests: fix encoding of IP addresses in x509 certs
tests: improve error message when saving TLS PSK file fails
tests: support QTEST_TRACE env variable
tests: print newline after QMP response in qtest logs
tests: add more helper macros for creating TLS x509 certs
crypto: mandate a hostname when checking x509 creds on a client
migration: fix use of TLS PSK credentials with a UNIX socket
tests: merge code for UNIX and TCP migration pre-copy tests
tests: introduce ability to provide hooks for migration precopy test
tests: switch migration FD passing test to use common precopy helper
tests: expand the migration precopy helper to support failures
tests: add migration tests of TLS with PSK credentials
tests: add migration tests of TLS with x509 credentials
tests: convert XBZRLE migration test to use common helper
tests: convert multifd migration tests to use common helper
tests: add multifd migration tests of TLS with PSK credentials
tests: add multifd migration tests of TLS with x509 credentials
tests: ensure migration status isn't reported as failed

 crypto/tlssession.c                  |    6 +
 meson.build                          |    1 +
 migration/tls.c                      |    4 -
 tests/qtest/libqtest.c               |   11 +-
 tests/qtest/meson.build              |   12 +-
 tests/qtest/migration-helpers.c      |   13 +
 tests/qtest/migration-helpers.h      |    1 +
 tests/qtest/migration-test.c         | 1041 +++++++++++++++++++++-----
 tests/unit/crypto-tls-psk-helpers.c  |   20 +-
 tests/unit/crypto-tls-psk-helpers.h  |    1 +
 tests/unit/crypto-tls-x509-helpers.c |   16 +-
 tests/unit/crypto-tls-x509-helpers.h |   53 ++
 tests/unit/test-crypto-tlssession.c  |   11 +-
 13 files changed, 1004 insertions(+), 186 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH 1/18 tests: fix encoding of IP addresses in x509 certs

We need to encode just the address bytes, not the whole struct sockaddr data. Add a test case to validate that we're matching on SAN IP addresses correctly.

---
 tests/unit/crypto-tls-x509-helpers.c | 16 +++++++++++++---
 tests/unit/test-crypto-tlssession.c  | 11 +++++++++--
 2 files changed, 22 insertions(+), 5 deletions(-)
tests/unit/crypto-tls-x509-helpers.c +13 -3
index fc609b3fd4..e9937f60d8 100644
@@ -168,9 +168,19 @@ test_tls_get_ipaddr(const char *addrstr,
     hints.ai_flags = AI_NUMERICHOST;
     g_assert(getaddrinfo(addrstr, NULL, &hints, &res) == 0);
 
-    *datalen = res->ai_addrlen;
-    *data = g_new(char, *datalen);
-    memcpy(*data, res->ai_addr, *datalen);
+    if (res->ai_family == AF_INET) {
+        struct sockaddr_in *in = (struct sockaddr_in *)res->ai_addr;
+        *datalen = sizeof(in->sin_addr);
+        *data = g_new(char, *datalen);
+        memcpy(*data, &in->sin_addr, *datalen);
+    } else if (res->ai_family == AF_INET6) {
+        struct sockaddr_in6 *in = (struct sockaddr_in6 *)res->ai_addr;
+        *datalen = sizeof(in->sin6_addr);
+        *data = g_new(char, *datalen);
+        memcpy(*data, &in->sin6_addr, *datalen);
+    } else {
+        g_assert_not_reached();
+    }
     freeaddrinfo(res);
 }
 
tests/unit/test-crypto-tlssession.c +9 -2
index 5f0da9192c..a6935d8497 100644
@@ -512,12 +512,19 @@ int main(int argc, char **argv)
                   false, true, "wiki.qemu.org", NULL);
 
     TEST_SESS_REG(altname4, cacertreq.filename,
+                  servercertalt1req.filename, clientcertreq.filename,
+                  false, false, "192.168.122.1", NULL);
+    TEST_SESS_REG(altname5, cacertreq.filename,
+                  servercertalt1req.filename, clientcertreq.filename,
+                  false, false, "fec0::dead:beaf", NULL);
+
+    TEST_SESS_REG(altname6, cacertreq.filename,
                   servercertalt2req.filename, clientcertreq.filename,
                   false, true, "qemu.org", NULL);
-    TEST_SESS_REG(altname5, cacertreq.filename,
+    TEST_SESS_REG(altname7, cacertreq.filename,
                   servercertalt2req.filename, clientcertreq.filename,
                   false, false, "www.qemu.org", NULL);
-    TEST_SESS_REG(altname6, cacertreq.filename,
+    TEST_SESS_REG(altname8, cacertreq.filename,
                   servercertalt2req.filename, clientcertreq.filename,
                   false, false, "wiki.qemu.org", NULL);
 
Signature · 1 line
2.34.1
#2 lore raw

PATCH 2/18 tests: improve error message when saving TLS PSK file fails

In reply to Daniel P. Berrangé
---
 tests/unit/crypto-tls-psk-helpers.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/unit/crypto-tls-psk-helpers.c +1 -1
index 7f8a488961..4bea7c6fa2 100644
@@ -30,7 +30,7 @@ void test_tls_psk_init(const char *pskfile)
 
     fp = fopen(pskfile, "w");
     if (fp == NULL) {
-        g_critical("Failed to create pskfile %s", pskfile);
+        g_critical("Failed to create pskfile %s: %s", pskfile, strerror(errno));
         abort();
     }
     /* Don't do this in real applications!  Use psktool. */
Signature · 1 line
2.34.1
#4 lore raw

PATCH 3/18 tests: support QTEST_TRACE env variable

In reply to Daniel P. Berrangé

When debugging failing qtests it is useful to be able to turn on trace output to stderr. The QTEST_TRACE env variable contents get injected as a '-trace <str>' command line arg

---
 tests/qtest/libqtest.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tests/qtest/libqtest.c +6 -2
index 41f4da4e54..a85f8a6d05 100644
@@ -260,6 +260,9 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args)
     gchar *qmp_socket_path;
     gchar *command;
     const char *qemu_binary = qtest_qemu_binary();
+    const char *trace = g_getenv("QTEST_TRACE");
+    g_autofree char *tracearg = trace ?
+        g_strdup_printf("-trace %s ", trace) : g_strdup("");
 
     s = g_new(QTestState, 1);
 
@@ -282,14 +285,15 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args)
 
     qtest_add_abrt_handler(kill_qemu_hook_func, s);
 
-    command = g_strdup_printf("exec %s "
+    command = g_strdup_printf("exec %s %s"
                               "-qtest unix:%s "
                               "-qtest-log %s "
                               "-chardev socket,path=%s,id=char0 "
                               "-mon chardev=char0,mode=control "
                               "-display none "
                               "%s"
-                              " -accel qtest", qemu_binary, socket_path,
+                              " -accel qtest",
+                              qemu_binary, tracearg, socket_path,
                               getenv("QTEST_LOG") ? "/dev/fd/2" : "/dev/null",
                               qmp_socket_path,
                               extra_args ?: "");
Signature · 1 line
2.34.1
Reviewed-by on v1 3/18
#5 lore raw

Re: [PATCH 03/18] tests: support QTEST_TRACE env variable

Quoted 5 lines · Daniel P. Berrangé

When debugging failing qtests it is useful to be able to turn on trace output to stderr. The QTEST_TRACE env variable contents get injected as a '-trace <str>' command line arg

Signature · 1 line
Peter Xu
Reviewed-by on v1 3/18
#6 lore raw

Re: [PATCH 03/18] tests: support QTEST_TRACE env variable

In reply to Daniel P. Berrangé
Quoted 31 lines · Daniel P. Berrangé

When debugging failing qtests it is useful to be able to turn on trace output to stderr. The QTEST_TRACE env variable contents get injected as a '-trace <str>' command line arg

---
  tests/qtest/libqtest.c | 8 ++++++--
  1 file changed, 6 insertions(+), 2 deletions(-)
tests/qtest/libqtest.c +4 -1
index 41f4da4e54..a85f8a6d05 100644
@@ -260,6 +260,9 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args)
      gchar *qmp_socket_path;
      gchar *command;
      const char *qemu_binary = qtest_qemu_binary();
+    const char *trace = g_getenv("QTEST_TRACE");
+    g_autofree char *tracearg = trace ?
+        g_strdup_printf("-trace %s ", trace) : g_strdup("");
 
      s = g_new(QTestState, 1);
 
@@ -282,14 +285,15 @@ QTestState *qtest_init_without_qmp_handshake(const char *extra_args)
 
      qtest_add_abrt_handler(kill_qemu_hook_func, s);
 
-    command = g_strdup_printf("exec %s "
+    command = g_strdup_printf("exec %s %s"
                                "-qtest unix:%s "
                                "-qtest-log %s "

tests/qtest/libqtest.c QTestState *qtest_init_without_qmp_handshake(const char *extra_args) · line 299

Quoted 10 lines
tests/qtest/libqtest.c +2 -1
                                "-chardev socket,path=%s,id=char0 "
                                "-mon chardev=char0,mode=control "
                                "-display none "
                                "%s"
-                              " -accel qtest", qemu_binary, socket_path,
+                              " -accel qtest",
+                              qemu_binary, tracearg, socket_path,
                                getenv("QTEST_LOG") ? "/dev/fd/2" : "/dev/null",
                                qmp_socket_path,
                                extra_args ?: "");
#7 lore raw

PATCH 4/18 tests: print newline after QMP response in qtest logs

In reply to Daniel P. Berrangé

The QMP commands have a trailing newline, but the response does not. This makes the qtest logs hard to follow as the next QMP command appears in the same line as the previous QMP response.

---
 tests/qtest/libqtest.c | 3 +++
 1 file changed, 3 insertions(+)
tests/qtest/libqtest.c +3 -0
index a85f8a6d05..79c3edcf4b 100644
@@ -629,6 +629,9 @@ QDict *qmp_fd_receive(int fd)
         }
         json_message_parser_feed(&qmp.parser, &c, 1);
     }
+    if (log) {
+        g_assert(write(2, "\n", 1) == 1);
+    }
     json_message_parser_destroy(&qmp.parser);
 
     return qmp.response;
Signature · 1 line
2.34.1
#8 lore raw

Re: [PATCH 04/18] tests: print newline after QMP response in qtest logs

Quoted 14 lines · Daniel P. Berrangé

The QMP commands have a trailing newline, but the response does not. This makes the qtest logs hard to follow as the next QMP command appears in the same line as the previous QMP response.

---
 tests/qtest/libqtest.c | 3 +++
 1 file changed, 3 insertions(+)
tests/qtest/libqtest.c +0 -0
index a85f8a6d05..79c3edcf4b 100644

tests/qtest/libqtest.c QDict *qmp_fd_receive(int fd) · line 634

Quoted 6 lines
tests/qtest/libqtest.c +3 -0
@@ -629,6 +629,9 @@ QDict *qmp_fd_receive(int fd)
         }
         json_message_parser_feed(&qmp.parser, &c, 1);
     }
+    if (log) {
+        g_assert(write(2, "\n", 1) == 1);
+    }

Drop the g_assert() to remove side effect of G_DISABLE_ASSERT?

Signature · 1 line
Peter Xu
#9 lore raw

Re: [PATCH 04/18] tests: print newline after QMP response in qtest logs

Quoted 10 lines · Peter Xu
Quoted 9 lines · Daniel P. Berrangé

The QMP commands have a trailing newline, but the response does not. This makes the qtest logs hard to follow as the next QMP command appears in the same line as the previous QMP response.

---
 tests/qtest/libqtest.c | 3 +++
 1 file changed, 3 insertions(+)
Quoted 13 lines

tests/qtest/libqtest.c QDict *qmp_fd_receive(int fd) · line 634

Quoted 11 lines
tests/qtest/libqtest.c +3 -0
index a85f8a6d05..79c3edcf4b 100644
@@ -629,6 +629,9 @@ QDict *qmp_fd_receive(int fd)
         }
         json_message_parser_feed(&qmp.parser, &c, 1);
     }
+    if (log) {
+        g_assert(write(2, "\n", 1) == 1);
+    }

Drop the g_assert() to remove side effect of G_DISABLE_ASSERT?

You need to check the return value of write() otherwise you'll get a compile failure due to a warn_unused_result attribute annotation.

I don't think G_DISABLE_ASSERT is a problem as we're not defining that in our code.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by
#10 lore raw

Re: [PATCH 04/18] tests: print newline after QMP response in qtest logs

Quoted 25 lines · Daniel P. Berrangé
Quoted 10 lines · Peter Xu
Quoted 9 lines · Daniel P. Berrangé

The QMP commands have a trailing newline, but the response does not. This makes the qtest logs hard to follow as the next QMP command appears in the same line as the previous QMP response.

---
  tests/qtest/libqtest.c | 3 +++
  1 file changed, 3 insertions(+)
Quoted 13 lines

tests/qtest/libqtest.c QDict *qmp_fd_receive(int fd) · line 634

Quoted 11 lines
tests/qtest/libqtest.c +3 -0
index a85f8a6d05..79c3edcf4b 100644
@@ -629,6 +629,9 @@ QDict *qmp_fd_receive(int fd)
          }
          json_message_parser_feed(&qmp.parser, &c, 1);
      }
+    if (log) {
+        g_assert(write(2, "\n", 1) == 1);
+    }

Drop the g_assert() to remove side effect of G_DISABLE_ASSERT?

Quoted 5 lines

You need to check the return value of write() otherwise you'll get a compile failure due to a warn_unused_result attribute annotation.

I don't think G_DISABLE_ASSERT is a problem as we're not defining that in our code.

You could use g_assert_true() - that's not affected by G_DISABLE_ASSERT.

Anyway:

#11 lore raw

Re: [PATCH 04/18] tests: print newline after QMP response in qtest logs

Quoted 26 lines · Thomas Huth
Quoted 25 lines · Daniel P. Berrangé
Quoted 23 lines · Peter Xu
Quoted 14 lines · Daniel P. Berrangé

The QMP commands have a trailing newline, but the response does not. This makes the qtest logs hard to follow as the next QMP command appears in the same line as the previous QMP response.

---
  tests/qtest/libqtest.c | 3 +++
  1 file changed, 3 insertions(+)
tests/qtest/libqtest.c +0 -0
index a85f8a6d05..79c3edcf4b 100644

tests/qtest/libqtest.c QDict *qmp_fd_receive(int fd) · line 634

Quoted 6 lines
tests/qtest/libqtest.c +3 -0
@@ -629,6 +629,9 @@ QDict *qmp_fd_receive(int fd)
          }
          json_message_parser_feed(&qmp.parser, &c, 1);
      }
+    if (log) {
+        g_assert(write(2, "\n", 1) == 1);
+    }

Drop the g_assert() to remove side effect of G_DISABLE_ASSERT?

Quoted 7 lines
Quoted 5 lines

You need to check the return value of write() otherwise you'll get a compile failure due to a warn_unused_result attribute annotation.

I don't think G_DISABLE_ASSERT is a problem as we're not defining that in our code.

You could use g_assert_true() - that's not affected by G_DISABLE_ASSERT.

Ah that's better indeed. :)

We could also fix the write() above this one too - that one captures the retcode into len but it simply dropped it, afaict.

Thanks,

Signature · 1 line
Peter Xu
#12 lore raw

Re: [PATCH 04/18] tests: print newline after QMP response in qtest logs

In reply to Thomas Huth
Quoted 26 lines · Thomas Huth
Quoted 25 lines · Daniel P. Berrangé
Quoted 23 lines · Peter Xu
Quoted 14 lines · Daniel P. Berrangé

The QMP commands have a trailing newline, but the response does not. This makes the qtest logs hard to follow as the next QMP command appears in the same line as the previous QMP response.

---
  tests/qtest/libqtest.c | 3 +++
  1 file changed, 3 insertions(+)
tests/qtest/libqtest.c +0 -0
index a85f8a6d05..79c3edcf4b 100644

tests/qtest/libqtest.c QDict *qmp_fd_receive(int fd) · line 634

Quoted 6 lines
tests/qtest/libqtest.c +3 -0
@@ -629,6 +629,9 @@ QDict *qmp_fd_receive(int fd)
          }
          json_message_parser_feed(&qmp.parser, &c, 1);
      }
+    if (log) {
+        g_assert(write(2, "\n", 1) == 1);
+    }

Drop the g_assert() to remove side effect of G_DISABLE_ASSERT?

Quoted 7 lines
Quoted 5 lines

You need to check the return value of write() otherwise you'll get a compile failure due to a warn_unused_result attribute annotation.

I don't think G_DISABLE_ASSERT is a problem as we're not defining that in our code.

You could use g_assert_true() - that's not affected by G_DISABLE_ASSERT.

I don't think we need to do that, per existing common practice:

$ git grep '\bg_assert('  | wc -l
2912
$ git grep '\bg_assert(' tests | wc -l
2296

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#13 lore raw

Re: [PATCH 04/18] tests: print newline after QMP response in qtest logs

Hi

Quoted 36 lines · Daniel P. Berrangé
Quoted 4 lines · Thomas Huth
Quoted 3 lines · Daniel P. Berrangé
Quoted 2 lines · Peter Xu
Quoted 1 line · Daniel P. Berrangé

The QMP commands have a trailing newline, but the response does

not.

Quoted 22 lines
Quoted 22 lines
Quoted 21 lines
Quoted 13 lines

This makes the qtest logs hard to follow as the next QMP command appears in the same line as the previous QMP response.

---
  tests/qtest/libqtest.c | 3 +++
  1 file changed, 3 insertions(+)
tests/qtest/libqtest.c +0 -0
index a85f8a6d05..79c3edcf4b 100644

tests/qtest/libqtest.c QDict *qmp_fd_receive(int fd) · line 634

Quoted 6 lines
tests/qtest/libqtest.c +3 -0
@@ -629,6 +629,9 @@ QDict *qmp_fd_receive(int fd)
          }
          json_message_parser_feed(&qmp.parser, &c, 1);
      }
+    if (log) {
+        g_assert(write(2, "\n", 1) == 1);
+    }

Drop the g_assert() to remove side effect of G_DISABLE_ASSERT?

Quoted 7 lines
Quoted 5 lines

You need to check the return value of write() otherwise you'll get a compile failure due to a warn_unused_result attribute annotation.

I don't think G_DISABLE_ASSERT is a problem as we're not defining that in our code.

You could use g_assert_true() - that's not affected by G_DISABLE_ASSERT.

Quoted 9 lines

I don't think we need to do that, per existing common practice:

$ git grep '\bg_assert('  | wc -l
2912
$ git grep '\bg_assert(' tests | wc -l
2296

On the topic of assert() usage, it would be nice to state clearly when to assert() or g_assert().

g_assert() behaviour is claimed to be more consistent than assert() across platforms.

Also -DNDEBUG is less obvious than -DG_DISABLE_CHECKS or -DG_DISABLE_ASSERT.

I would remove assert.h and prevent from using it back, but I might be missing some reasons to still use it.

Signature · 1 line
Marc-André Lureau
#14 lore raw

Re: [PATCH 04/18] tests: print newline after QMP response in qtest logs

Quoted 50 lines · Marc-André Lureau

Hi

Quoted 36 lines · Daniel P. Berrangé
Quoted 4 lines · Thomas Huth
Quoted 3 lines · Daniel P. Berrangé
Quoted 2 lines · Peter Xu
Quoted 1 line · Daniel P. Berrangé

The QMP commands have a trailing newline, but the response does

not.

Quoted 22 lines
Quoted 22 lines
Quoted 21 lines
Quoted 13 lines

This makes the qtest logs hard to follow as the next QMP command appears in the same line as the previous QMP response.

---
  tests/qtest/libqtest.c | 3 +++
  1 file changed, 3 insertions(+)
tests/qtest/libqtest.c +0 -0
index a85f8a6d05..79c3edcf4b 100644

tests/qtest/libqtest.c QDict *qmp_fd_receive(int fd) · line 634

Quoted 6 lines
tests/qtest/libqtest.c +3 -0
@@ -629,6 +629,9 @@ QDict *qmp_fd_receive(int fd)
          }
          json_message_parser_feed(&qmp.parser, &c, 1);
      }
+    if (log) {
+        g_assert(write(2, "\n", 1) == 1);
+    }

Drop the g_assert() to remove side effect of G_DISABLE_ASSERT?

Quoted 7 lines
Quoted 5 lines

You need to check the return value of write() otherwise you'll get a compile failure due to a warn_unused_result attribute annotation.

I don't think G_DISABLE_ASSERT is a problem as we're not defining that in our code.

You could use g_assert_true() - that's not affected by G_DISABLE_ASSERT.

Quoted 9 lines

I don't think we need to do that, per existing common practice:

$ git grep '\bg_assert('  | wc -l
2912
$ git grep '\bg_assert(' tests | wc -l
2296
Quoted 7 lines

On the topic of assert() usage, it would be nice to state clearly when to assert() or g_assert().

g_assert() behaviour is claimed to be more consistent than assert() across platforms.

Also -DNDEBUG is less obvious than -DG_DISABLE_CHECKS or -DG_DISABLE_ASSERT.

Personally I would make QEMU build error if NDEBUG or G_DISABLE_ASSERT was defined, as running with asserts disabled will lead to unsafe code. We rely on asserts firing to avoid compromise of QEMU when faced with malicious data.

Quoted 2 lines

I would remove assert.h and prevent from using it back, but I might be missing some reasons to still use it.

As a metric we've got massive use of both families of asset

$ git grep -E -o '\b(assert|g_assert(_[a-z]+)?)\b\s?\(' | sed -e 's/.*://' | sort | uniq -c
     17 assert (
   5196 assert(
   2913 g_assert(
     29 g_assert_cmpfloat(
    662 g_assert_cmphex(
   1745 g_assert_cmpint(
     20 g_assert_cmpmem(
    407 g_assert_cmpstr(
    756 g_assert_cmpuint(
    169 g_assert_false(
    138 g_assert_nonnull(
     22 g_assert_null(
    167 g_assert_true(

But for the tests/ subdir, g_assert family is a clear favourite

$ git grep -E -o '\b(assert|g_assert(_[a-z]+)?)\b\s?\(' tests  | sed -e 's/.*://' | sort | uniq -c
      1 assert (
    759 assert(
   2297 g_assert(
     29 g_assert_cmpfloat(
    661 g_assert_cmphex(
   1744 g_assert_cmpint(
     20 g_assert_cmpmem(
    406 g_assert_cmpstr(
    754 g_assert_cmpuint(
    169 g_assert_false(
    138 g_assert_nonnull(
     22 g_assert_null(
    167 g_assert_true(

This split doesn't appear to have changed all that much over time. Going back to v3.0.0 we see similar ballpark figures

$ git grep -E -o '\b(assert|g_assert(_[a-z]+)?)\b\s?\(' | sed -e 's/.*://' | sort | uniq -c 
     18 assert (
   3766 assert(
   2210 g_assert(
     23 g_assert_cmpfloat(
    310 g_assert_cmphex(
   1352 g_assert_cmpint(
     11 g_assert_cmpmem(
    324 g_assert_cmpstr(
    489 g_assert_cmpuint(
     88 g_assert_false(
     73 g_assert_nonnull(
      8 g_assert_null(
     46 g_assert_true(
[berrange@catbus qemu]$ git grep -E -o '\b(assert|g_assert(_[a-z]+)?)\b\s?\(' tests | sed -e 's/.*://' | sort | uniq -c 
    566 assert(
   1876 g_assert(
     23 g_assert_cmpfloat(
    309 g_assert_cmphex(
   1351 g_assert_cmpint(
     10 g_assert_cmpmem(
    323 g_assert_cmpstr(
    488 g_assert_cmpuint(
     88 g_assert_false(
     73 g_assert_nonnull(
      8 g_assert_null(
     46 g_assert_true(

Removing either 'assert' or g_assert would be a massive amount of code churn, for no real functional benefit.

I would personally encourage the more specific g_assert_* variants, to improve the error messages, but that's really minor.

IMHO we can accept that all of the above are valid to use and worry about more important things.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#15 lore raw

Re: [PATCH 04/18] tests: print newline after QMP response in qtest logs

Quoted 42 lines · Daniel P. Berrangé
Quoted 41 lines · Marc-André Lureau

Hi

Quoted 36 lines · Daniel P. Berrangé
Quoted 4 lines · Thomas Huth
Quoted 3 lines · Daniel P. Berrangé
Quoted 2 lines · Peter Xu
Quoted 1 line · Daniel P. Berrangé

The QMP commands have a trailing newline, but the response does

not.

Quoted 29 lines
Quoted 22 lines
Quoted 8 lines
Quoted 8 lines

This makes the qtest logs hard to follow as the next QMP command appears in the same line as the previous QMP response.

---
   tests/qtest/libqtest.c | 3 +++
   1 file changed, 3 insertions(+)
Quoted 13 lines

tests/qtest/libqtest.c QDict *qmp_fd_receive(int fd) · line 634

Quoted 11 lines
tests/qtest/libqtest.c +3 -0
index a85f8a6d05..79c3edcf4b 100644
@@ -629,6 +629,9 @@ QDict *qmp_fd_receive(int fd)
           }
           json_message_parser_feed(&qmp.parser, &c, 1);
       }
+    if (log) {
+        g_assert(write(2, "\n", 1) == 1);
+    }

Drop the g_assert() to remove side effect of G_DISABLE_ASSERT?

Quoted 5 lines

You need to check the return value of write() otherwise you'll get a compile failure due to a warn_unused_result attribute annotation.

I don't think G_DISABLE_ASSERT is a problem as we're not defining that in our code.

You could use g_assert_true() - that's not affected by G_DISABLE_ASSERT.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

I don't think we need to do that, per existing common practice:

$ git grep '\bg_assert('  | wc -l
2912
$ git grep '\bg_assert(' tests | wc -l
2296

I said, you *could* do that, not that you *must* do that ;-)

Since we require compiling without G_DISABLE_ASSERT in the QEMU code base, it doesn't really matter for us. But if you're involved in other projects, too, where GI_DISABLE_ASSERT might be allowed, it might be good habit to do it "right", i.e. use g_assert_true() in case the expression has side effects and must never be disabled.

Quoted 7 lines
Quoted 7 lines
Quoted 1 line

On the topic of assert() usage, it would be nice to state clearly when to assert() or g_assert().

g_assert() behaviour is claimed to be more consistent than assert() across platforms.

Quoted 6 lines
Quoted 1 line

Also -DNDEBUG is less obvious than -DG_DISABLE_CHECKS or -DG_DISABLE_ASSERT.

Personally I would make QEMU build error if NDEBUG or G_DISABLE_ASSERT was defined, as running with asserts disabled will lead to unsafe code. We rely on asserts firing to avoid compromise of QEMU when faced with malicious data.

We already do that, see osdep.h:

#ifdef NDEBUG
#error building with NDEBUG is not supported #endif
#ifdef G_DISABLE_ASSERT
#error building with G_DISABLE_ASSERT is not supported #endif

Quoted 2 lines
Quoted 2 lines

I would remove assert.h and prevent from using it back, but I might be missing some reasons to still use it.

I guess it's just grown historically.

First step would be to add an entry to our coding styles, I think.

Then, if we don't care about the code churn, this would be a nice BiteSizeTask for new contributors, I think (GSoC etc.), so we could create some tickets in the Gitlab issue tracker for this (e.g. split up by subsystem).

Thomas

#16 lore raw

Re: [PATCH 04/18] tests: print newline after QMP response in qtest logs

In reply to Daniel P. Berrangé

Hi

Quoted 5 lines · Daniel P. Berrangé

Removing either 'assert' or g_assert would be a massive amount of code churn, for no real functional benefit.

Well, a few thousands of lines that are trivially regexp. And we can make use of git blame ignore-rev (https://michaelheap.com/git-ignore-rev/) to skip those cleanups.

Signature · 1 line
Marc-André Lureau
#17 lore raw

Re: [PATCH 04/18] tests: print newline after QMP response in qtest logs

In reply to Daniel P. Berrangé

Hi

Quoted 62 lines · Daniel P. Berrangé
Quoted 9 lines · Marc-André Lureau

Hi

Quoted 4 lines · Daniel P. Berrangé
Quoted 3 lines · Thomas Huth
Quoted 2 lines · Daniel P. Berrangé
Quoted 1 line · Peter Xu

On Wed, Mar 02, 2022 at 05:49:18PM +0000, Daniel P. Berrangé

Quoted 3 lines · wrote:
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

The QMP commands have a trailing newline, but the response does

not.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This makes the qtest logs hard to follow as the next QMP

command

Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 12 lines

appears in the same line as the previous QMP response.

---
  tests/qtest/libqtest.c | 3 +++
  1 file changed, 3 insertions(+)
tests/qtest/libqtest.c +0 -0
index a85f8a6d05..79c3edcf4b 100644

tests/qtest/libqtest.c QDict *qmp_fd_receive(int fd) · line 634

Quoted 6 lines
tests/qtest/libqtest.c +3 -0
@@ -629,6 +629,9 @@ QDict *qmp_fd_receive(int fd)
          }
          json_message_parser_feed(&qmp.parser, &c, 1);
      }
+    if (log) {
+        g_assert(write(2, "\n", 1) == 1);
+    }

Drop the g_assert() to remove side effect of G_DISABLE_ASSERT?

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

You need to check the return value of write() otherwise you'll get

a

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

compile failure due to a warn_unused_result attribute annotation.

I don't think G_DISABLE_ASSERT is a problem as we're not defining that in our code.

You could use g_assert_true() - that's not affected by

G_DISABLE_ASSERT.

Quoted 6 lines
Quoted 6 lines

I don't think we need to do that, per existing common practice:

$ git grep '\bg_assert('  | wc -l
2912
Quoted 8 lines
Quoted 4 lines
$ git grep '\bg_assert(' tests | wc -l
2296

On the topic of assert() usage, it would be nice to state clearly when to assert() or g_assert().

g_assert() behaviour is claimed to be more consistent than assert()

across

Quoted 2 lines

platforms.

Quoted 8 lines
Quoted 1 line

Also -DNDEBUG is less obvious than -DG_DISABLE_CHECKS or

-DG_DISABLE_ASSERT.

Personally I would make QEMU build error if NDEBUG or G_DISABLE_ASSERT was defined, as running with asserts disabled will lead to unsafe code. We rely on asserts firing to avoid compromise of QEMU when faced with malicious data.

agreed

Quoted 76 lines
Quoted 3 lines

I would remove assert.h and prevent from using it back, but I might be missing some reasons to still use it.

As a metric we've got massive use of both families of asset

$ git grep -E -o '\b(assert|g_assert(_[a-z]+)?)\b\s?\(' | sed -e 's/.*://'
| sort | uniq -c
     17 assert (
   5196 assert(
   2913 g_assert(
     29 g_assert_cmpfloat(
    662 g_assert_cmphex(
   1745 g_assert_cmpint(
     20 g_assert_cmpmem(
    407 g_assert_cmpstr(
    756 g_assert_cmpuint(
    169 g_assert_false(
    138 g_assert_nonnull(
     22 g_assert_null(
    167 g_assert_true(

But for the tests/ subdir, g_assert family is a clear favourite

$ git grep -E -o '\b(assert|g_assert(_[a-z]+)?)\b\s?\(' tests  | sed -e
's/.*://' | sort | uniq -c
      1 assert (
    759 assert(
   2297 g_assert(
     29 g_assert_cmpfloat(
    661 g_assert_cmphex(
   1744 g_assert_cmpint(
     20 g_assert_cmpmem(
    406 g_assert_cmpstr(
    754 g_assert_cmpuint(
    169 g_assert_false(
    138 g_assert_nonnull(
     22 g_assert_null(
    167 g_assert_true(

This split doesn't appear to have changed all that much over time. Going back to v3.0.0 we see similar ballpark figures

$ git grep -E -o '\b(assert|g_assert(_[a-z]+)?)\b\s?\(' | sed -e 's/.*://'
| sort | uniq -c
     18 assert (
   3766 assert(
   2210 g_assert(
     23 g_assert_cmpfloat(
    310 g_assert_cmphex(
   1352 g_assert_cmpint(
     11 g_assert_cmpmem(
    324 g_assert_cmpstr(
    489 g_assert_cmpuint(
     88 g_assert_false(
     73 g_assert_nonnull(
      8 g_assert_null(
     46 g_assert_true(
[berrange@catbus qemu]$ git grep -E -o
'\b(assert|g_assert(_[a-z]+)?)\b\s?\(' tests | sed -e 's/.*://' | sort |
uniq -c
    566 assert(
   1876 g_assert(
     23 g_assert_cmpfloat(
    309 g_assert_cmphex(
   1351 g_assert_cmpint(
     10 g_assert_cmpmem(
    323 g_assert_cmpstr(
    488 g_assert_cmpuint(
     88 g_assert_false(
     73 g_assert_nonnull(
      8 g_assert_null(
     46 g_assert_true(
Quoted 9 lines

Removing either 'assert' or g_assert would be a massive amount of code churn, for no real functional benefit.

I would personally encourage the more specific g_assert_* variants, to improve the error messages, but that's really minor.

IMHO we can accept that all of the above are valid to use and worry about more important things.

Beside the usage inconsistency & the potential platform inconsistencies, it makes code reorganization/split (libslirp-like) a bit more annoying, because <assert.h> is only included in osdep.h.

Also g_assert*() family of functions help more precise usage patterns. g_assert_not_reached() is better than assert(false) or just abort() imho (it seems we use both).

Signature · 1 line
Marc-André Lureau
#18 lore raw

Re: [PATCH 04/18] tests: print newline after QMP response in qtest logs

Quoted 70 lines · Marc-André Lureau

Hi

Quoted 65 lines · Daniel P. Berrangé
Quoted 9 lines · Marc-André Lureau

Hi

Quoted 4 lines · Daniel P. Berrangé
Quoted 3 lines · Thomas Huth
Quoted 2 lines · Daniel P. Berrangé
Quoted 1 line · Peter Xu

On Wed, Mar 02, 2022 at 05:49:18PM +0000, Daniel P. Berrangé

Quoted 3 lines · wrote:
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

The QMP commands have a trailing newline, but the response does

not.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

This makes the qtest logs hard to follow as the next QMP

command

Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 12 lines

appears in the same line as the previous QMP response.

---
  tests/qtest/libqtest.c | 3 +++
  1 file changed, 3 insertions(+)
tests/qtest/libqtest.c +0 -0
index a85f8a6d05..79c3edcf4b 100644

tests/qtest/libqtest.c QDict *qmp_fd_receive(int fd) · line 634

Quoted 6 lines
tests/qtest/libqtest.c +3 -0
@@ -629,6 +629,9 @@ QDict *qmp_fd_receive(int fd)
          }
          json_message_parser_feed(&qmp.parser, &c, 1);
      }
+    if (log) {
+        g_assert(write(2, "\n", 1) == 1);
+    }

Drop the g_assert() to remove side effect of G_DISABLE_ASSERT?

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

You need to check the return value of write() otherwise you'll get

a

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

compile failure due to a warn_unused_result attribute annotation.

I don't think G_DISABLE_ASSERT is a problem as we're not defining that in our code.

You could use g_assert_true() - that's not affected by

G_DISABLE_ASSERT.

Quoted 6 lines
Quoted 6 lines

I don't think we need to do that, per existing common practice:

$ git grep '\bg_assert('  | wc -l
2912
Quoted 8 lines
Quoted 4 lines
$ git grep '\bg_assert(' tests | wc -l
2296

On the topic of assert() usage, it would be nice to state clearly when to assert() or g_assert().

g_assert() behaviour is claimed to be more consistent than assert()

across

Quoted 3 lines

platforms.

Also -DNDEBUG is less obvious than -DG_DISABLE_CHECKS or

-DG_DISABLE_ASSERT.

Quoted 7 lines
Quoted 5 lines

Personally I would make QEMU build error if NDEBUG or G_DISABLE_ASSERT was defined, as running with asserts disabled will lead to unsafe code. We rely on asserts firing to avoid compromise of QEMU when faced with malicious data.

agreed

Such a good idea that it was already done 5 years ago :-)

commit 262a69f4282e44426c7a132138581d400053e0a1

Author: Eric Blake <[email protected]>
Date:   Mon Sep 11 16:13:20 2017 -0500

osdep.h: Prohibit disabling assert() in supported builds

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#19 lore raw

Re: [PATCH 04/18] tests: print newline after QMP response in qtest logs

In reply to Daniel P. Berrangé
Quoted 57 lines · * Daniel P. Berrangé ([email protected]) wrote:
Quoted 56 lines · Marc-André Lureau

Hi

Quoted 36 lines · Daniel P. Berrangé
Quoted 4 lines · Thomas Huth
Quoted 3 lines · Daniel P. Berrangé
Quoted 2 lines · Peter Xu
Quoted 1 line · Daniel P. Berrangé

The QMP commands have a trailing newline, but the response does

not.

Quoted 22 lines
Quoted 22 lines
Quoted 21 lines
Quoted 13 lines

This makes the qtest logs hard to follow as the next QMP command appears in the same line as the previous QMP response.

---
  tests/qtest/libqtest.c | 3 +++
  1 file changed, 3 insertions(+)
tests/qtest/libqtest.c +0 -0
index a85f8a6d05..79c3edcf4b 100644

tests/qtest/libqtest.c QDict *qmp_fd_receive(int fd) · line 634

Quoted 6 lines
tests/qtest/libqtest.c +3 -0
@@ -629,6 +629,9 @@ QDict *qmp_fd_receive(int fd)
          }
          json_message_parser_feed(&qmp.parser, &c, 1);
      }
+    if (log) {
+        g_assert(write(2, "\n", 1) == 1);
+    }

Drop the g_assert() to remove side effect of G_DISABLE_ASSERT?

Quoted 7 lines
Quoted 5 lines

You need to check the return value of write() otherwise you'll get a compile failure due to a warn_unused_result attribute annotation.

I don't think G_DISABLE_ASSERT is a problem as we're not defining that in our code.

You could use g_assert_true() - that's not affected by G_DISABLE_ASSERT.

Quoted 9 lines

I don't think we need to do that, per existing common practice:

$ git grep '\bg_assert('  | wc -l
2912
$ git grep '\bg_assert(' tests | wc -l
2296

On the topic of assert() usage, it would be nice to state clearly when to assert() or g_assert().

g_assert() behaviour is claimed to be more consistent than assert() across platforms.

Quoted 6 lines
Quoted 1 line

Also -DNDEBUG is less obvious than -DG_DISABLE_CHECKS or -DG_DISABLE_ASSERT.

Personally I would make QEMU build error if NDEBUG or G_DISABLE_ASSERT was defined, as running with asserts disabled will lead to unsafe code. We rely on asserts firing to avoid compromise of QEMU when faced with malicious data.

That's not enough; glib allows many of the asserts to be made non-fatal by a runtime flag, by calling g_test_set_nonfatal_assertions (see glib commit a6a875068779) - I made checkpatch disallow the use of these g_assert's in the main code in commit 6e93895

Dave

Quoted 86 lines
Quoted 2 lines

I would remove assert.h and prevent from using it back, but I might be missing some reasons to still use it.

As a metric we've got massive use of both families of asset

$ git grep -E -o '\b(assert|g_assert(_[a-z]+)?)\b\s?\(' | sed -e 's/.*://' | sort | uniq -c
     17 assert (
   5196 assert(
   2913 g_assert(
     29 g_assert_cmpfloat(
    662 g_assert_cmphex(
   1745 g_assert_cmpint(
     20 g_assert_cmpmem(
    407 g_assert_cmpstr(
    756 g_assert_cmpuint(
    169 g_assert_false(
    138 g_assert_nonnull(
     22 g_assert_null(
    167 g_assert_true(

But for the tests/ subdir, g_assert family is a clear favourite

$ git grep -E -o '\b(assert|g_assert(_[a-z]+)?)\b\s?\(' tests  | sed -e 's/.*://' | sort | uniq -c
      1 assert (
    759 assert(
   2297 g_assert(
     29 g_assert_cmpfloat(
    661 g_assert_cmphex(
   1744 g_assert_cmpint(
     20 g_assert_cmpmem(
    406 g_assert_cmpstr(
    754 g_assert_cmpuint(
    169 g_assert_false(
    138 g_assert_nonnull(
     22 g_assert_null(
    167 g_assert_true(

This split doesn't appear to have changed all that much over time. Going back to v3.0.0 we see similar ballpark figures

$ git grep -E -o '\b(assert|g_assert(_[a-z]+)?)\b\s?\(' | sed -e 's/.*://' | sort | uniq -c 
     18 assert (
   3766 assert(
   2210 g_assert(
     23 g_assert_cmpfloat(
    310 g_assert_cmphex(
   1352 g_assert_cmpint(
     11 g_assert_cmpmem(
    324 g_assert_cmpstr(
    489 g_assert_cmpuint(
     88 g_assert_false(
     73 g_assert_nonnull(
      8 g_assert_null(
     46 g_assert_true(
[berrange@catbus qemu]$ git grep -E -o '\b(assert|g_assert(_[a-z]+)?)\b\s?\(' tests | sed -e 's/.*://' | sort | uniq -c 
    566 assert(
   1876 g_assert(
     23 g_assert_cmpfloat(
    309 g_assert_cmphex(
   1351 g_assert_cmpint(
     10 g_assert_cmpmem(
    323 g_assert_cmpstr(
    488 g_assert_cmpuint(
     88 g_assert_false(
     73 g_assert_nonnull(
      8 g_assert_null(
     46 g_assert_true(

Removing either 'assert' or g_assert would be a massive amount of code churn, for no real functional benefit.

I would personally encourage the more specific g_assert_* variants, to improve the error messages, but that's really minor.

IMHO we can accept that all of the above are valid to use and worry about more important things.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#20 lore raw

PATCH 5/18 tests: add more helper macros for creating TLS x509 certs

In reply to Daniel P. Berrangé

These macros are more suited to the general consumers of certs in the test suite, where we don't need to exercise every single possible permutation.

---
 tests/unit/crypto-tls-x509-helpers.h | 53 ++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)
tests/unit/crypto-tls-x509-helpers.h +53 -0
index cf6329e653..247e7160eb 100644
@@ -26,6 +26,9 @@
 #include <libtasn1.h>
 
 
+#define QCRYPTO_TLS_TEST_CLIENT_NAME "ACME QEMU Client"
+#define QCRYPTO_TLS_TEST_CLIENT_HOSTILE_NAME "ACME Hostile Client"
+
 /*
  * This contains parameter about how to generate
  * certificates.
@@ -118,6 +121,56 @@ void test_tls_cleanup(const char *keyfile);
     };                                                                  \
     test_tls_generate_cert(&varname, NULL)
 
+# define TLS_ROOT_REQ_SIMPLE(varname, fname)                            \
+    QCryptoTLSTestCertReq varname = {                                   \
+        .filename = fname,                                              \
+        .cn = "qemu-CA",                                                \
+        .basicConstraintsEnable = true,                                 \
+        .basicConstraintsCritical = true,                               \
+        .basicConstraintsIsCA = true,                                   \
+        .keyUsageEnable = true,                                         \
+        .keyUsageCritical = true,                                       \
+        .keyUsageValue = GNUTLS_KEY_KEY_CERT_SIGN,                      \
+    };                                                                  \
+    test_tls_generate_cert(&varname, NULL)
+
+# define TLS_CERT_REQ_SIMPLE_CLIENT(varname, cavarname, cname, fname)   \
+    QCryptoTLSTestCertReq varname = {                                   \
+        .filename = fname,                                              \
+        .cn = cname,                                                    \
+        .basicConstraintsEnable = true,                                 \
+        .basicConstraintsCritical = true,                               \
+        .basicConstraintsIsCA = false,                                  \
+        .keyUsageEnable = true,                                         \
+        .keyUsageCritical = true,                                       \
+        .keyUsageValue =                                                \
+        GNUTLS_KEY_DIGITAL_SIGNATURE | GNUTLS_KEY_KEY_ENCIPHERMENT,     \
+        .keyPurposeEnable = true,                                       \
+        .keyPurposeCritical = true,                                     \
+        .keyPurposeOID1 = GNUTLS_KP_TLS_WWW_CLIENT,                     \
+    };                                                                  \
+    test_tls_generate_cert(&varname, cavarname.crt)
+
+# define TLS_CERT_REQ_SIMPLE_SERVER(varname, cavarname, fname,          \
+                                    hostname, ipaddr)                   \
+    QCryptoTLSTestCertReq varname = {                                   \
+        .filename = fname,                                              \
+        .cn = hostname ? hostname : ipaddr,                             \
+        .altname1 = hostname,                                           \
+        .ipaddr1 = ipaddr,                                              \
+        .basicConstraintsEnable = true,                                 \
+        .basicConstraintsCritical = true,                               \
+        .basicConstraintsIsCA = false,                                  \
+        .keyUsageEnable = true,                                         \
+        .keyUsageCritical = true,                                       \
+        .keyUsageValue =                                                \
+        GNUTLS_KEY_DIGITAL_SIGNATURE | GNUTLS_KEY_KEY_ENCIPHERMENT,     \
+        .keyPurposeEnable = true,                                       \
+        .keyPurposeCritical = true,                                     \
+        .keyPurposeOID1 = GNUTLS_KP_TLS_WWW_SERVER,                     \
+    };                                                                  \
+    test_tls_generate_cert(&varname, cavarname.crt)
+
 extern const asn1_static_node pkix_asn1_tab[];
 
 #endif
Signature · 1 line
2.34.1
#21 lore raw

PATCH 6/18 crypto: mandate a hostname when checking x509 creds on a client

In reply to Daniel P. Berrangé

Currently the TLS session object assumes that the caller will always provide a hostname when using x509 creds on a client endpoint. This relies on the caller to detect and report an error if the user has configured QEMU with x509 credentials on a UNIX socket. The migration code has such a check, but it is too broad, reporting an error when the user has configured QEMU with PSK credentials on a UNIX socket, where hostnames are irrelevant.

Putting the check into the TLS session object credentials validation code ensures we report errors in only the scenario that matters.

---
 crypto/tlssession.c | 6 ++++++
 1 file changed, 6 insertions(+)
crypto/tlssession.c +6 -0
index a8db8c76d1..b302d835d2 100644
@@ -373,6 +373,12 @@ qcrypto_tls_session_check_certificate(QCryptoTLSSession *session,
                                session->hostname);
                     goto error;
                 }
+            } else {
+                if (session->creds->endpoint ==
+                    QCRYPTO_TLS_CREDS_ENDPOINT_CLIENT) {
+                    error_setg(errp, "No hostname for certificate validation");
+                    goto error;
+                }
             }
         }
 
Signature · 1 line
2.34.1
#22 lore raw

PATCH 7/18 migration: fix use of TLS PSK credentials with a UNIX socket

In reply to Daniel P. Berrangé

The migration TLS code has a check mandating that a hostname be available when starting a TLS session. This is expected when using x509 credentials, but is bogus for PSK and anonymous credentials as neither involve hostname validation.

The TLS crdentials object gained suitable error reporting in the case of TLS with x509 credentials, so there is no longer any need for the migration code to do its own (incorrect) validation.

---
 migration/tls.c | 4 ----
 1 file changed, 4 deletions(-)
migration/tls.c +0 -4
index ca1ea3bbdd..32c384a8b6 100644
@@ -137,10 +137,6 @@ QIOChannelTLS *migration_tls_client_create(MigrationState *s,
     if (s->parameters.tls_hostname && *s->parameters.tls_hostname) {
         hostname = s->parameters.tls_hostname;
     }
-    if (!hostname) {
-        error_setg(errp, "No hostname available for TLS");
-        return NULL;
-    }
 
     tioc = qio_channel_tls_new_client(
         ioc, creds, hostname, errp);
Signature · 1 line
2.34.1
#23 lore raw

Re: [PATCH 07/18] migration: fix use of TLS PSK credentials with a UNIX socket

Quoted 10 lines · Daniel P. Berrangé

The migration TLS code has a check mandating that a hostname be available when starting a TLS session. This is expected when using x509 credentials, but is bogus for PSK and anonymous credentials as neither involve hostname validation.

The TLS crdentials object gained suitable error reporting in the case of TLS with x509 credentials, so there is no longer any need for the migration code to do its own (incorrect) validation.

Do we need a Fixes tag for this?

Signature · 1 line
Peter Xu
#24 lore raw

Re: [PATCH 07/18] migration: fix use of TLS PSK credentials with a UNIX socket

Quoted 6 lines · Peter Xu
Quoted 5 lines · Daniel P. Berrangé

The migration TLS code has a check mandating that a hostname be available when starting a TLS session. This is expected when using x509 credentials, but is bogus for PSK and anonymous credentials as neither involve hostname validation.

Quoted 7 lines
Quoted 5 lines

The TLS crdentials object gained suitable error reporting in the case of TLS with x509 credentials, so there is no longer any need for the migration code to do its own (incorrect) validation.

Do we need a Fixes tag for this?

It is fuzzy as we never really intended for UNIX sockets to use TLS originally.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#25 lore raw

PATCH 8/18 tests: merge code for UNIX and TCP migration pre-copy tests

In reply to Daniel P. Berrangé

The test cases differ only in the URI they provide to the migration commands, and the ability to set the dirty_ring mode. This code is trivially merged into a common helper.

---
 roms/seabios                 |  2 +-
 tests/qtest/migration-test.c | 86 ++++++++++++++++--------------------
 2 files changed, 40 insertions(+), 48 deletions(-)
roms/seabios +1 -1
index 6a62e0cb0d..2dd4b9b3f8 160000
@@ -1 +1 @@
-Subproject commit 6a62e0cb0dfe9cd28b70547dbea5caf76847c3a9
+Subproject commit 2dd4b9b3f84019668719344b40dba79d681be41c
tests/qtest/migration-test.c +39 -47

125 lines not on this page · raw message

Signature · 1 line
2.34.1
Reviewed-by on v1 8/18
#26 lore raw

Re: [PATCH 08/18] tests: merge code for UNIX and TCP migration pre-copy tests

Quoted 5 lines · Daniel P. Berrangé

The test cases differ only in the URI they provide to the migration commands, and the ability to set the dirty_ring mode. This code is trivially merged into a common helper.

Signature · 1 line
Peter Xu
#27 lore raw

Re: [PATCH 08/18] tests: merge code for UNIX and TCP migration pre-copy tests

In reply to Daniel P. Berrangé
Quoted 15 lines · Daniel P. Berrangé

The test cases differ only in the URI they provide to the migration commands, and the ability to set the dirty_ring mode. This code is trivially merged into a common helper.

---
  roms/seabios                 |  2 +-
  tests/qtest/migration-test.c | 86 ++++++++++++++++--------------------
  2 files changed, 40 insertions(+), 48 deletions(-)
roms/seabios +0 -0
index 6a62e0cb0d..2dd4b9b3f8 160000

roms/seabios line 1

Quoted 2 lines
roms/seabios +1 -1
@@ -1 +1 @@
-Subproject commit 6a62e0cb0dfe9cd28b70547dbea5caf76847c3a9
+Subproject commit 2dd4b9b3f84019668719344b40dba79d681be41c

Did you really want to update the submodule here?

Thomas

#28 lore raw

Re: [PATCH 08/18] tests: merge code for UNIX and TCP migration pre-copy tests

Quoted 11 lines · Thomas Huth
Quoted 10 lines · Daniel P. Berrangé

The test cases differ only in the URI they provide to the migration commands, and the ability to set the dirty_ring mode. This code is trivially merged into a common helper.

---
  roms/seabios                 |  2 +-
  tests/qtest/migration-test.c | 86 ++++++++++++++++--------------------
  2 files changed, 40 insertions(+), 48 deletions(-)
Quoted 9 lines

roms/seabios line 1

Quoted 7 lines
roms/seabios +1 -1
index 6a62e0cb0d..2dd4b9b3f8 160000
@@ -1 +1 @@
-Subproject commit 6a62e0cb0dfe9cd28b70547dbea5caf76847c3a9
+Subproject commit 2dd4b9b3f84019668719344b40dba79d681be41c

Did you really want to update the submodule here?

Of course not :-(

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#29 lore raw

PATCH 9/18 tests: introduce ability to provide hooks for migration precopy test

In reply to Daniel P. Berrangé

There are alot of different scenarios to test with migration due to the wide number of parameters and capabilities available. To enable sharing of the basic precopy test scenario, we need to be able to set arbitrary parameters and capabilities before the migration is initiated, but don't want to have all this logic in the common helper function. Solve this by defining two hooks that can be provided by the test case, one before migration starts and one after migration finishes.

---
 tests/qtest/migration-test.c | 41 ++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)
tests/qtest/migration-test.c +41 -0
index c1058dc944..2f2059cebc 100644
@@ -796,11 +796,37 @@ static void test_baddest(void)
     test_migrate_end(from, to, false);
 }
 
+/*
+ * A hook that runs after the src and dst QEMUs have been
+ * created, but before the migration is started. This can
+ * be used to set migration parameters and capabilities.
+ *
+ * Returns: NULL, or a pointer to opaque state to be
+ *          later passed to the TestMigrateFinishHook
+ */
+typedef void * (*TestMigrateStartHook)(QTestState *from,
+                                       QTestState *to);
+
+/*
+ * A hook that runs after the migration has finished,
+ * regardless of whether it succeeded or failed, but
+ * before QEMU has terminated (unless it self-terminated
+ * due to migration error)
+ *
+ * @opaque is a pointer to state previously returned
+ * by the TestMigrateStartHook if any, or NULL.
+ */
+typedef void (*TestMigrateFinishHook)(QTestState *from,
+                                      QTestState *to,
+                                      void *opaque);
+
 /*
  * Common helper for running a precopy migration test
  *
  * @listen_uri: the URI for the dst QEMU to listen on
  * @connect_uri: the URI for the src QEMU to connect to
+ * @start_hook: (optional) callback to run at start to set migration parameters
+ * @finish_hook: (optional) callback to run at finish to cleanup
  * @dirty_ring: true to use dirty ring tracking
  *
  * If @connect_uri is NULL, then it will query the dst
@@ -810,11 +836,14 @@ static void test_baddest(void)
  */
 static void test_precopy_common(const char *listen_uri,
                                 const char *connect_uri,
+                                TestMigrateStartHook start_hook,
+                                TestMigrateFinishHook finish_hook,
                                 bool dirty_ring)
 {
     MigrateStart *args = migrate_start_new();
     g_autofree char *local_connect_uri = NULL;
     QTestState *from, *to;
+    void *data_hook = NULL;
 
     args->use_dirty_ring = dirty_ring;
 
@@ -832,6 +861,10 @@ static void test_precopy_common(const char *listen_uri,
     /* 1GB/s */
     migrate_set_parameter_int(from, "max-bandwidth", 1000000000);
 
+    if (start_hook) {
+        data_hook = start_hook(from, to);
+    }
+
     /* Wait for the first serial output from the source */
     wait_for_serial("src_serial");
 
@@ -855,6 +888,10 @@ static void test_precopy_common(const char *listen_uri,
     wait_for_serial("dest_serial");
     wait_for_migration_complete(from);
 
+    if (finish_hook) {
+        finish_hook(from, to, data_hook);
+    }
+
     test_migrate_end(from, to, true);
 }
 
@@ -864,6 +901,8 @@ static void test_precopy_unix_common(bool dirty_ring)
 
     test_precopy_common(uri,
                         uri,
+                        NULL, /* start_hook */
+                        NULL, /* finish_hook */
                         dirty_ring);
 }
 
@@ -971,6 +1010,8 @@ static void test_precopy_tcp(void)
 {
     test_precopy_common("tcp:127.0.0.1:0",
                         NULL, /* connect_uri */
+                        NULL, /* start_hook */
+                        NULL, /* finish_hook */
                         false /* dirty_ring */);
 }
 
Signature · 1 line
2.34.1
Reviewed-by on v1 9/18
#30 lore raw

Re: [PATCH 09/18] tests: introduce ability to provide hooks for migration precopy test

Quoted 9 lines · Daniel P. Berrangé

There are alot of different scenarios to test with migration due to the wide number of parameters and capabilities available. To enable sharing of the basic precopy test scenario, we need to be able to set arbitrary parameters and capabilities before the migration is initiated, but don't want to have all this logic in the common helper function. Solve this by defining two hooks that can be provided by the test case, one before migration starts and one after migration finishes.

Signature · 1 line
Peter Xu
#31 lore raw

PATCH 10/18 tests: switch migration FD passing test to use common precopy helper

In reply to Daniel P. Berrangé

The combination of the start and finish hooks allow the FD passing code to use the precopy helper

---
 tests/qtest/migration-test.c | 55 +++++++++++++-----------------------
 1 file changed, 19 insertions(+), 36 deletions(-)
tests/qtest/migration-test.c +19 -36
index 2f2059cebc..2082c58e8b 100644
@@ -1015,31 +1015,12 @@ static void test_precopy_tcp(void)
                         false /* dirty_ring */);
 }
 
-static void test_migrate_fd_proto(void)
+static void *test_migrate_fd_start_hook(QTestState *from,
+                                        QTestState *to)
 {
-    MigrateStart *args = migrate_start_new();
-    QTestState *from, *to;
+    QDict *rsp;
     int ret;
     int pair[2];
-    QDict *rsp;
-    const char *error_desc;
-
-    if (test_migrate_start(&from, &to, "defer", args)) {
-        return;
-    }
-
-    /*
-     * We want to pick a speed slow enough that the test completes
-     * quickly, but that it doesn't complete precopy even on a slow
-     * machine, so also set the downtime.
-     */
-    /* 1 ms should make it not converge */
-    migrate_set_parameter_int(from, "downtime-limit", 1);
-    /* 1GB/s */
-    migrate_set_parameter_int(from, "max-bandwidth", 1000000000);
-
-    /* Wait for the first serial output from the source */
-    wait_for_serial("src_serial");
 
     /* Create two connected sockets for migration */
     ret = socketpair(PF_LOCAL, SOCK_STREAM, 0, pair);
@@ -1064,17 +1045,15 @@ static void test_migrate_fd_proto(void)
     qobject_unref(rsp);
     close(pair[1]);
 
-    /* Start migration to the 2nd socket*/
-    migrate_qmp(from, "fd:fd-mig", "{}");
-
-    wait_for_migration_pass(from);
-
-    migrate_set_parameter_int(from, "downtime-limit", CONVERGE_DOWNTIME);
+    return NULL;
+}
 
-    if (!got_stop) {
-        qtest_qmp_eventwait(from, "STOP");
-    }
-    qtest_qmp_eventwait(to, "RESUME");
+static void test_migrate_fd_finish_hook(QTestState *from,
+                                        QTestState *to,
+                                        void *opaque)
+{
+    QDict *rsp;
+    const char *error_desc;
 
     /* Test closing fds */
     /* We assume, that QEMU removes named fd from its list,
@@ -1092,11 +1071,15 @@ static void test_migrate_fd_proto(void)
     error_desc = qdict_get_str(qdict_get_qdict(rsp, "error"), "desc");
     g_assert_cmpstr(error_desc, ==, "File descriptor named 'fd-mig' not found");
     qobject_unref(rsp);
+}
 
-    /* Complete migration */
-    wait_for_serial("dest_serial");
-    wait_for_migration_complete(from);
-    test_migrate_end(from, to, true);
+static void test_migrate_fd_proto(void)
+{
+    test_precopy_common("defer",
+                        "fd:fd-mig",
+                        test_migrate_fd_start_hook,
+                        test_migrate_fd_finish_hook,
+                        false /* dirty_ring */);
 }
 
 static void do_test_validate_uuid(MigrateStart *args, bool should_fail)
Signature · 1 line
2.34.1
#33 lore raw

PATCH 11/18 tests: expand the migration precopy helper to support failures

In reply to Daniel P. Berrangé

The migration precopy testing helper function always expects the migration to run to a completion state. There will be test scenarios for TLS where expect either the client or server to fail the migration. This expands the helper to cope with these scenarios.

---
 tests/qtest/migration-test.c | 47 +++++++++++++++++++++++++++++-------
 1 file changed, 38 insertions(+), 9 deletions(-)
tests/qtest/migration-test.c +38 -9
index 2082c58e8b..e40b408988 100644
@@ -827,17 +827,32 @@ typedef void (*TestMigrateFinishHook)(QTestState *from,
  * @connect_uri: the URI for the src QEMU to connect to
  * @start_hook: (optional) callback to run at start to set migration parameters
  * @finish_hook: (optional) callback to run at finish to cleanup
+ * @expect_fail: true if we expect migration to fail
+ * @dst_quit: true if we expect the dst QEMU to quit with an
+ *            abnormal exit status on failure
  * @dirty_ring: true to use dirty ring tracking
  *
  * If @connect_uri is NULL, then it will query the dst
  * QEMU for its actual listening address and use that
  * as the connect address. This allows for dynamically
  * picking a free TCP port.
+ *
+ * If @expect_fail is true then we expect the migration process to
+ * fail instead of completing. There can be a variety of reasons
+ * and stages in which this may happen. If a failure is expected
+ * to happen at time of establishing the connection, then @dst_quit
+ * should be false to indicate that the dst QEMU is espected to
+ * stay running and accept future migration connections. If a
+ * failure is expected to happen while processing the migration
+ * stream, then @dst_quit should be true to indicate that the
+ * dst QEMU is expected to quit with non-zero exit status
  */
 static void test_precopy_common(const char *listen_uri,
                                 const char *connect_uri,
                                 TestMigrateStartHook start_hook,
                                 TestMigrateFinishHook finish_hook,
+                                bool expect_fail,
+                                bool dst_quit,
                                 bool dirty_ring)
 {
     MigrateStart *args = migrate_start_new();
@@ -875,24 +890,32 @@ static void test_precopy_common(const char *listen_uri,
 
     migrate_qmp(from, connect_uri, "{}");
 
-    wait_for_migration_pass(from);
+    if (expect_fail) {
+        wait_for_migration_fail(from, !dst_quit);
 
-    migrate_set_parameter_int(from, "downtime-limit", CONVERGE_DOWNTIME);
+        if (dst_quit) {
+            qtest_set_expected_status(to, 1);
+        }
+    } else {
+        wait_for_migration_pass(from);
 
-    if (!got_stop) {
-        qtest_qmp_eventwait(from, "STOP");
-    }
+        migrate_set_parameter_int(from, "downtime-limit", CONVERGE_DOWNTIME);
 
-    qtest_qmp_eventwait(to, "RESUME");
+        if (!got_stop) {
+            qtest_qmp_eventwait(from, "STOP");
+        }
 
-    wait_for_serial("dest_serial");
-    wait_for_migration_complete(from);
+        qtest_qmp_eventwait(to, "RESUME");
+
+        wait_for_serial("dest_serial");
+        wait_for_migration_complete(from);
+    }
 
     if (finish_hook) {
         finish_hook(from, to, data_hook);
     }
 
-    test_migrate_end(from, to, true);
+    test_migrate_end(from, to, !expect_fail);
 }
 
 static void test_precopy_unix_common(bool dirty_ring)
@@ -903,6 +926,8 @@ static void test_precopy_unix_common(bool dirty_ring)
                         uri,
                         NULL, /* start_hook */
                         NULL, /* finish_hook */
+                        false, /* expect_fail */
+                        false, /* dst_quit */
                         dirty_ring);
 }
 
@@ -1012,6 +1037,8 @@ static void test_precopy_tcp(void)
                         NULL, /* connect_uri */
                         NULL, /* start_hook */
                         NULL, /* finish_hook */
+                        false, /* expect_fail */
+                        false, /* dst_quit */
                         false /* dirty_ring */);
 }
 
@@ -1079,6 +1106,8 @@ static void test_migrate_fd_proto(void)
                         "fd:fd-mig",
                         test_migrate_fd_start_hook,
                         test_migrate_fd_finish_hook,
+                        false, /* expect_fail */
+                        false, /* dst_quit */
                         false /* dirty_ring */);
 }
 
Signature · 1 line
2.34.1
#34 lore raw

Re: [PATCH 11/18] tests: expand the migration precopy helper to support failures

Quoted 15 lines · Daniel P. Berrangé

The migration precopy testing helper function always expects the migration to run to a completion state. There will be test scenarios for TLS where expect either the client or server to fail the migration. This expands the helper to cope with these scenarios.

---
 tests/qtest/migration-test.c | 47 +++++++++++++++++++++++++++++-------
 1 file changed, 38 insertions(+), 9 deletions(-)
tests/qtest/migration-test.c +0 -0
index 2082c58e8b..e40b408988 100644

tests/qtest/migration-test.c typedef void (*TestMigrateFinishHook)(QTestState *from, · line 832

Quoted 6 lines
tests/qtest/migration-test.c +3 -0
@@ -827,17 +827,32 @@ typedef void (*TestMigrateFinishHook)(QTestState *from,
  * @connect_uri: the URI for the src QEMU to connect to
  * @start_hook: (optional) callback to run at start to set migration parameters
  * @finish_hook: (optional) callback to run at finish to cleanup
+ * @expect_fail: true if we expect migration to fail
+ * @dst_quit: true if we expect the dst QEMU to quit with an
+ *            abnormal exit status on failure

"dst_quit" sounds a bit confusing to me, as setting dst_quit=false seems to mean "dest qemu should not quit" but it's actually for checking an abnormal quit only.

Rename may work. Or, IMHO it's nicer if we could merge the two parameters:

@expected_result: What is the expectation of this migration test

  typedef enum {
    /* This test should succeed, the default */
    MIG_TEST_SUCCEED = 0,
    /* This test should fail, dest qemu should keep alive */
    MIG_TEST_FAIL,
    /* This test should fail, dest qemu should fail with abnormal status */
    MIG_TEST_FAIL_DEST_QUIT_ERR,

};

Because fundamentally the two parameters are correlated, e.g. there is no combination of expect_fail==false && dst_quit==true.

No strong opinion, though.

Thanks,

Signature · 1 line
Peter Xu
#35 lore raw

Re: [PATCH 11/18] tests: expand the migration precopy helper to support failures

Quoted 38 lines · Peter Xu
Quoted 15 lines · Daniel P. Berrangé

The migration precopy testing helper function always expects the migration to run to a completion state. There will be test scenarios for TLS where expect either the client or server to fail the migration. This expands the helper to cope with these scenarios.

---
 tests/qtest/migration-test.c | 47 +++++++++++++++++++++++++++++-------
 1 file changed, 38 insertions(+), 9 deletions(-)
tests/qtest/migration-test.c +0 -0
index 2082c58e8b..e40b408988 100644

tests/qtest/migration-test.c typedef void (*TestMigrateFinishHook)(QTestState *from, · line 832

Quoted 6 lines
tests/qtest/migration-test.c +3 -0
@@ -827,17 +827,32 @@ typedef void (*TestMigrateFinishHook)(QTestState *from,
  * @connect_uri: the URI for the src QEMU to connect to
  * @start_hook: (optional) callback to run at start to set migration parameters
  * @finish_hook: (optional) callback to run at finish to cleanup
+ * @expect_fail: true if we expect migration to fail
+ * @dst_quit: true if we expect the dst QEMU to quit with an
+ *            abnormal exit status on failure

"dst_quit" sounds a bit confusing to me, as setting dst_quit=false seems to mean "dest qemu should not quit" but it's actually for checking an abnormal quit only.

Rename may work. Or, IMHO it's nicer if we could merge the two parameters:

@expected_result: What is the expectation of this migration test

  typedef enum {
    /* This test should succeed, the default */
    MIG_TEST_SUCCEED = 0,
    /* This test should fail, dest qemu should keep alive */
    MIG_TEST_FAIL,
    /* This test should fail, dest qemu should fail with abnormal status */
    MIG_TEST_FAIL_DEST_QUIT_ERR,
Quoted 6 lines

};

Because fundamentally the two parameters are correlated, e.g. there is no combination of expect_fail==false && dst_quit==true.

No strong opinion, though.

Using enums is more clear to someone reading code, so a good idea.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#36 lore raw

Re: [PATCH 11/18] tests: expand the migration precopy helper to support failures

In reply to Daniel P. Berrangé
Quoted 10 lines · Daniel P. Berrangé

static void test_precopy_common(const char *listen_uri,
const char *connect_uri, TestMigrateStartHook start_hook, TestMigrateFinishHook finish_hook,

tests/qtest/migration-test.c +2 -0
+                                bool expect_fail,
+                                bool dst_quit,
                                 bool dirty_ring)
 {
     MigrateStart *args = migrate_start_new();

tests/qtest/migration-test.c static void test_precopy_common(const char *listen_uri, · line 894

Quoted 6 lines
tests/qtest/migration-test.c +2 -1
@@ -875,24 +890,32 @@ static void test_precopy_common(const char *listen_uri,
 
     migrate_qmp(from, connect_uri, "{}");
 
-    wait_for_migration_pass(from);
+    if (expect_fail) {
+        wait_for_migration_fail(from, !dst_quit);

Two more thoughts..

(1) Shall we move MigrateStart creation to be even upper? Then we avoid passing over these parameters but merge these new parameters into MigrateStart too. After all we used to have similar long lists of params and we merged them into MigrateStart.

(2) Shall we leverage MigrateStart.hide_stderr? I saw a bunch of errors dumped even if all things run as expected.

Signature · 1 line
Peter Xu
#37 lore raw

Re: [PATCH 11/18] tests: expand the migration precopy helper to support failures

Quoted 20 lines · Peter Xu
Quoted 10 lines · Daniel P. Berrangé

static void test_precopy_common(const char *listen_uri,
const char *connect_uri, TestMigrateStartHook start_hook, TestMigrateFinishHook finish_hook,

tests/qtest/migration-test.c +2 -0
+                                bool expect_fail,
+                                bool dst_quit,
                                 bool dirty_ring)
 {
     MigrateStart *args = migrate_start_new();

tests/qtest/migration-test.c static void test_precopy_common(const char *listen_uri, · line 894

Quoted 6 lines
tests/qtest/migration-test.c +2 -1
@@ -875,24 +890,32 @@ static void test_precopy_common(const char *listen_uri,
 
     migrate_qmp(from, connect_uri, "{}");
 
-    wait_for_migration_pass(from);
+    if (expect_fail) {
+        wait_for_migration_fail(from, !dst_quit);

Two more thoughts..

Quoted 4 lines

(1) Shall we move MigrateStart creation to be even upper? Then we avoid passing over these parameters but merge these new parameters into MigrateStart too. After all we used to have similar long lists of params and we merged them into MigrateStart.

I don't to use MigrateStart as these new parameters are not common to all migration tests. I have come up with an equivalent approach though.

Quoted 2 lines

(2) Shall we leverage MigrateStart.hide_stderr? I saw a bunch of errors dumped even if all things run as expected.

Yes.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#38 lore raw

PATCH 12/18 tests: add migration tests of TLS with PSK credentials

In reply to Daniel P. Berrangé

This validates that we correctly handle migration success and failure scenarios when using TLS with pre shared keys.

---
 roms/seabios                        |   2 +-
 tests/qtest/meson.build             |   7 +-
 tests/qtest/migration-test.c        | 180 ++++++++++++++++++++++++++--
 tests/unit/crypto-tls-psk-helpers.c |  18 ++-
 tests/unit/crypto-tls-psk-helpers.h |   1 +
 5 files changed, 190 insertions(+), 18 deletions(-)
roms/seabios +1 -1
index 2dd4b9b3f8..6a62e0cb0d 160000
@@ -1 +1 @@
-Subproject commit 2dd4b9b3f84019668719344b40dba79d681be41c
+Subproject commit 6a62e0cb0dfe9cd28b70547dbea5caf76847c3a9
tests/qtest/meson.build +6 -1
index f33d84d19b..a95bb5def3 100644
@@ -276,13 +276,18 @@ endif
 
 tpmemu_files = ['tpm-emu.c', 'tpm-util.c', 'tpm-tests.c']
 
+migration_files = [files('migration-helpers.c')]
+if gnutls.found()
+  migration_files += [files('../unit/crypto-tls-psk-helpers.c'), gnutls]
+endif
+
 qtests = {
   'bios-tables-test': [io, 'boot-sector.c', 'acpi-utils.c', 'tpm-emu.c'],
   'cdrom-test': files('boot-sector.c'),
   'dbus-vmstate-test': files('migration-helpers.c') + dbus_vmstate1,
   'erst-test': files('erst-test.c'),
   'ivshmem-test': [rt, '../../contrib/ivshmem-server/ivshmem-server.c'],
-  'migration-test': files('migration-helpers.c'),
+  'migration-test': migration_files,
   'pxe-test': files('boot-sector.c'),
   'qos-test': [chardev, io, qos_test_ss.apply(config_host, strict: false).sources()],
   'tpm-crb-swtpm-test': [io, tpmemu_files],
tests/qtest/migration-test.c +167 -13

234 lines not on this page · raw message

tests/unit/crypto-tls-psk-helpers.c +15 -3
index 4bea7c6fa2..511e08cc9c 100644
@@ -24,7 +24,8 @@
 #include "crypto-tls-psk-helpers.h"
 #include "qemu/sockets.h"
 
-void test_tls_psk_init(const char *pskfile)
+static void
+test_tls_psk_init_common(const char *pskfile, const char *user, const char *key)
 {
     FILE *fp;
 
@@ -33,11 +34,22 @@ void test_tls_psk_init(const char *pskfile)
         g_critical("Failed to create pskfile %s: %s", pskfile, strerror(errno));
         abort();
     }
-    /* Don't do this in real applications!  Use psktool. */
-    fprintf(fp, "qemu:009d5638c40fde0c\n");
+    fprintf(fp, "%s:%s\n", user, key);
     fclose(fp);
 }
 
+void test_tls_psk_init(const char *pskfile)
+{
+    /* Don't hard code a key like this in real applications!  Use psktool. */
+    test_tls_psk_init_common(pskfile, "qemu", "009d5638c40fde0c");
+}
+
+void test_tls_psk_init_alt(const char *pskfile)
+{
+    /* Don't hard code a key like this in real applications!  Use psktool. */
+    test_tls_psk_init_common(pskfile, "qemu", "10ffa6a2c42f0388");
+}
+
 void test_tls_psk_cleanup(const char *pskfile)
 {
     unlink(pskfile);
tests/unit/crypto-tls-psk-helpers.h +1 -0
index faa645c629..67f8bdda71 100644
@@ -24,6 +24,7 @@
 #include <gnutls/gnutls.h>
 
 void test_tls_psk_init(const char *keyfile);
+void test_tls_psk_init_alt(const char *keyfile);
 void test_tls_psk_cleanup(const char *keyfile);
 
 #endif
Signature · 1 line
2.34.1
#39 lore raw

Re: [PATCH 12/18] tests: add migration tests of TLS with PSK credentials

Quoted 17 lines · Daniel P. Berrangé

This validates that we correctly handle migration success and failure scenarios when using TLS with pre shared keys.

---
  roms/seabios                        |   2 +-
  tests/qtest/meson.build             |   7 +-
  tests/qtest/migration-test.c        | 180 ++++++++++++++++++++++++++--
  tests/unit/crypto-tls-psk-helpers.c |  18 ++-
  tests/unit/crypto-tls-psk-helpers.h |   1 +
  5 files changed, 190 insertions(+), 18 deletions(-)
roms/seabios +0 -0
index 2dd4b9b3f8..6a62e0cb0d 160000

roms/seabios line 1

Quoted 2 lines
roms/seabios +1 -1
@@ -1 +1 @@
-Subproject commit 2dd4b9b3f84019668719344b40dba79d681be41c
+Subproject commit 6a62e0cb0dfe9cd28b70547dbea5caf76847c3a9

Ah, here is the revert - so the previous change was by accident, indeed!

Thomas

#40 lore raw

PATCH 13/18 tests: add migration tests of TLS with x509 credentials

In reply to Daniel P. Berrangé

This validates that we correctly handle migration success and failure scenarios when using TLS with x509 certificates. There are quite a few different scenarios that matter in relation to hostname validation.

---
 meson.build                  |   1 +
 tests/qtest/meson.build      |   5 +
 tests/qtest/migration-test.c | 366 +++++++++++++++++++++++++++++++++--
 3 files changed, 361 insertions(+), 11 deletions(-)
meson.build +1 -0
index 8df40bfac4..06d2175bdf 100644
@@ -1548,6 +1548,7 @@ config_host_data.set('CONFIG_KEYUTILS', keyutils.found())
 config_host_data.set('CONFIG_GETTID', has_gettid)
 config_host_data.set('CONFIG_GNUTLS', gnutls.found())
 config_host_data.set('CONFIG_GNUTLS_CRYPTO', gnutls_crypto.found())
+config_host_data.set('CONFIG_TASN1', tasn1.found())
 config_host_data.set('CONFIG_GCRYPT', gcrypt.found())
 config_host_data.set('CONFIG_NETTLE', nettle.found())
 config_host_data.set('CONFIG_QEMU_PRIVATE_XTS', xts == 'private')
tests/qtest/meson.build +5 -0
index a95bb5def3..91dc36fb9b 100644
@@ -279,6 +279,11 @@ tpmemu_files = ['tpm-emu.c', 'tpm-util.c', 'tpm-tests.c']
 migration_files = [files('migration-helpers.c')]
 if gnutls.found()
   migration_files += [files('../unit/crypto-tls-psk-helpers.c'), gnutls]
+
+  if tasn1.found()
+    migration_files += [files('../unit/crypto-tls-x509-helpers.c',
+                              '../unit/pkix_asn1_tab.c'), tasn1]
+  endif
 endif
 
 qtests = {
tests/qtest/migration-test.c +355 -11

452 lines not on this page · raw message

Signature · 1 line
2.34.1
#41 lore raw

PATCH 14/18 tests: convert XBZRLE migration test to use common helper

In reply to Daniel P. Berrangé

Most of the XBZRLE migration test logic is common with the rest of the precopy tests, so it can use the helper with just one small tweak.

---
 tests/qtest/migration-test.c | 70 ++++++++++++++----------------------
 1 file changed, 26 insertions(+), 44 deletions(-)
tests/qtest/migration-test.c +26 -44
index 4040443caa..9896fcb134 100644
@@ -1162,6 +1162,7 @@ typedef void (*TestMigrateFinishHook)(QTestState *from,
  * @expect_fail: true if we expect migration to fail
  * @dst_quit: true if we expect the dst QEMU to quit with an
  *            abnormal exit status on failure
+ * @iterations: number of migration passes to wait for
  * @dirty_ring: true to use dirty ring tracking
  *
  * If @connect_uri is NULL, then it will query the dst
@@ -1185,6 +1186,7 @@ static void test_precopy_common(const char *listen_uri,
                                 TestMigrateFinishHook finish_hook,
                                 bool expect_fail,
                                 bool dst_quit,
+                                unsigned int iterations,
                                 bool dirty_ring)
 {
     MigrateStart *args = migrate_start_new();
@@ -1229,7 +1231,9 @@ static void test_precopy_common(const char *listen_uri,
             qtest_set_expected_status(to, 1);
         }
     } else {
-        wait_for_migration_pass(from);
+        while (iterations--) {
+            wait_for_migration_pass(from);
+        }
 
         migrate_set_parameter_int(from, "downtime-limit", CONVERGE_DOWNTIME);
 
@@ -1255,6 +1259,7 @@ static void test_precopy_unix_common(TestMigrateStartHook start_hook,
                                      TestMigrateFinishHook finish_hook,
                                      bool expect_fail,
                                      bool dst_quit,
+                                     unsigned int iterations,
                                      bool dirty_ring)
 {
     g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
@@ -1265,6 +1270,7 @@ static void test_precopy_unix_common(TestMigrateStartHook start_hook,
                         finish_hook,
                         expect_fail,
                         dst_quit,
+                        iterations,
                         dirty_ring);
 }
 
@@ -1274,6 +1280,7 @@ static void test_precopy_unix_plain(void)
                              NULL, /* finish_hook */
                              false, /* expect_fail */
                              false, /* dst_quit */
+                             1, /* iterations */
                              false /* dirty_ring */);
 }
 
@@ -1283,6 +1290,7 @@ static void test_precopy_unix_dirty_ring(void)
                              NULL, /* finish_hook */
                              false, /* clientReject */
                              false, /* dst_quit */
+                             1, /* iterations */
                              true /* dirty_ring */);
 }
 
@@ -1293,6 +1301,7 @@ static void test_precopy_unix_tls_psk(void)
                              test_migrate_tls_psk_finish,
                              false, /* expect_fail */
                              false, /* dst_quit */
+                             1, /* iterations */
                              false /* dirty_ring */);
 }
 
@@ -1303,6 +1312,7 @@ static void test_precopy_unix_tls_x509_default_host(void)
                              test_migrate_tls_x509_finish,
                              true, /* expect_fail */
                              true, /* dst_quit */
+                             1, /* iterations */
                              false /* dirty_ring */);
 }
 
@@ -1312,6 +1322,7 @@ static void test_precopy_unix_tls_x509_override_host(void)
                              test_migrate_tls_x509_finish,
                              false, /* expect_fail */
                              false, /* dst_quit */
+                             1, /* iterations */
                              false /* dirty_ring */);
 }
 #endif /* CONFIG_TASN1 */
@@ -1354,57 +1365,26 @@ static void test_ignore_shared(void)
 }
 #endif
 
-static void test_xbzrle(const char *uri)
+static void *
+test_migrate_xbzrle_start(QTestState *from,
+                          QTestState *to)
 {
-    MigrateStart *args = migrate_start_new();
-    QTestState *from, *to;
-
-    if (test_migrate_start(&from, &to, uri, args)) {
-        return;
-    }
-
-    /*
-     * We want to pick a speed slow enough that the test completes
-     * quickly, but that it doesn't complete precopy even on a slow
-     * machine, so also set the downtime.
-     */
-    /* 1 ms should make it not converge*/
-    migrate_set_parameter_int(from, "downtime-limit", 1);
-    /* 1GB/s */
-    migrate_set_parameter_int(from, "max-bandwidth", 1000000000);
-
     migrate_set_parameter_int(from, "xbzrle-cache-size", 33554432);
 
     migrate_set_capability(from, "xbzrle", true);
     migrate_set_capability(to, "xbzrle", true);
-    /* Wait for the first serial output from the source */
-    wait_for_serial("src_serial");
 
-    migrate_qmp(from, uri, "{}");
-
-    wait_for_migration_pass(from);
-    /* Make sure we have 2 passes, so the xbzrle cache gets a workout */
-    wait_for_migration_pass(from);
-
-    /* 1000ms should converge */
-    migrate_set_parameter_int(from, "downtime-limit", 1000);
-
-    if (!got_stop) {
-        qtest_qmp_eventwait(from, "STOP");
-    }
-    qtest_qmp_eventwait(to, "RESUME");
-
-    wait_for_serial("dest_serial");
-    wait_for_migration_complete(from);
-
-    test_migrate_end(from, to, true);
+    return NULL;
 }
 
-static void test_xbzrle_unix(void)
+static void test_precopy_unix_xbzrle(void)
 {
-    g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
-
-    test_xbzrle(uri);
+    test_precopy_unix_common(test_migrate_xbzrle_start,
+                             NULL, /* finish_hook */
+                             false, /* expect_fail */
+                             false, /* dst_quit */
+                             2, /* iterations */
+                             false /* dirty_ring */);
 }
 
 static void test_precopy_tcp_common(TestMigrateStartHook start_hook,
@@ -1418,6 +1398,7 @@ static void test_precopy_tcp_common(TestMigrateStartHook start_hook,
                         finish_hook,
                         expect_fail,
                         dst_quit,
+                        1, /* iterations */
                         false /* dirty_ring */);
 }
 
@@ -1572,6 +1553,7 @@ static void test_migrate_fd_proto(void)
                         test_migrate_fd_finish_hook,
                         false, /* expect_fail */
                         false, /* dst_quit */
+                        1, /* iterations */
                         false /* dirty_ring */);
 }
 
@@ -1970,6 +1952,7 @@ int main(int argc, char **argv)
     qtest_add_func("/migration/postcopy/recovery", test_postcopy_recovery);
     qtest_add_func("/migration/bad_dest", test_baddest);
     qtest_add_func("/migration/precopy/unix/plain", test_precopy_unix_plain);
+    qtest_add_func("/migration/precopy/unix/xbzrle", test_precopy_unix_xbzrle);
 #ifdef CONFIG_GNUTLS
     qtest_add_func("/migration/precopy/unix/tls/psk",
                    test_precopy_unix_tls_psk);
@@ -2006,7 +1989,6 @@ int main(int argc, char **argv)
 #endif /* CONFIG_GNUTLS */
 
     /* qtest_add_func("/migration/ignore_shared", test_ignore_shared); */
-    qtest_add_func("/migration/xbzrle/unix", test_xbzrle_unix);
     qtest_add_func("/migration/fd_proto", test_migrate_fd_proto);
     qtest_add_func("/migration/validate_uuid", test_validate_uuid);
     qtest_add_func("/migration/validate_uuid_error", test_validate_uuid_error);
Signature · 1 line
2.34.1
#42 lore raw

Re: [PATCH 14/18] tests: convert XBZRLE migration test to use common helper

Quoted 49 lines · Daniel P. Berrangé
tests/qtest/migration-test.c +6 -0
@@ -1255,6 +1259,7 @@ static void test_precopy_unix_common(TestMigrateStartHook start_hook,
                                      TestMigrateFinishHook finish_hook,
                                      bool expect_fail,
                                      bool dst_quit,
+                                     unsigned int iterations,
                                      bool dirty_ring)
 {
     g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
@@ -1265,6 +1270,7 @@ static void test_precopy_unix_common(TestMigrateStartHook start_hook,
                         finish_hook,
                         expect_fail,
                         dst_quit,
+                        iterations,
                         dirty_ring);
 }
 
@@ -1274,6 +1280,7 @@ static void test_precopy_unix_plain(void)
                              NULL, /* finish_hook */
                              false, /* expect_fail */
                              false, /* dst_quit */
+                             1, /* iterations */
                              false /* dirty_ring */);
 }
 
@@ -1283,6 +1290,7 @@ static void test_precopy_unix_dirty_ring(void)
                              NULL, /* finish_hook */
                              false, /* clientReject */
                              false, /* dst_quit */
+                             1, /* iterations */
                              true /* dirty_ring */);
 }
 
@@ -1293,6 +1301,7 @@ static void test_precopy_unix_tls_psk(void)
                              test_migrate_tls_psk_finish,
                              false, /* expect_fail */
                              false, /* dst_quit */
+                             1, /* iterations */
                              false /* dirty_ring */);
 }
 
@@ -1303,6 +1312,7 @@ static void test_precopy_unix_tls_x509_default_host(void)
                              test_migrate_tls_x509_finish,
                              true, /* expect_fail */
                              true, /* dst_quit */
+                             1, /* iterations */
                              false /* dirty_ring */);
 }
 

tests/qtest/migration-test.c static void test_precopy_unix_tls_x509_override_host(void) · line 1326

Quoted 5 lines
tests/qtest/migration-test.c +1 -0
@@ -1312,6 +1322,7 @@ static void test_precopy_unix_tls_x509_override_host(void)
                              test_migrate_tls_x509_finish,
                              false, /* expect_fail */
                              false, /* dst_quit */
+                             1, /* iterations */
                              false /* dirty_ring */);

Another side benefit to merge parameters (e.g. the new "iterations") into MigrateStart (which I mentioned in the other thread) is that we don't need to touch the value in every test if there's a default, because we will set the default in migrate_start_new() and we only need a tweak on tests that want to overwrite the default values.

Thanks,

Signature · 1 line
Peter Xu
#43 lore raw

PATCH 15/18 tests: convert multifd migration tests to use common helper

In reply to Daniel P. Berrangé

Most of the multifd migration test logic is common with the rest of the precopy tests, so it can use the helper without difficulty. The only exception of the multifd cancellation test which tries to run multiple migrations in a row.

---
 tests/qtest/migration-test.c | 75 +++++++++++++++++++-----------------
 1 file changed, 39 insertions(+), 36 deletions(-)
tests/qtest/migration-test.c +39 -36
index 9896fcb134..7c69268aa8 100644
@@ -1716,26 +1716,12 @@ static void test_migrate_auto_converge(void)
     test_migrate_end(from, to, true);
 }
 
-static void test_multifd_tcp(const char *method)
+static void *
+test_migration_precopy_tcp_multifd_start_common(QTestState *from,
+                                                QTestState *to,
+                                                const char *method)
 {
-    MigrateStart *args = migrate_start_new();
-    QTestState *from, *to;
     QDict *rsp;
-    g_autofree char *uri = NULL;
-
-    if (test_migrate_start(&from, &to, "defer", args)) {
-        return;
-    }
-
-    /*
-     * We want to pick a speed slow enough that the test completes
-     * quickly, but that it doesn't complete precopy even on a slow
-     * machine, so also set the downtime.
-     */
-    /* 1 ms should make it not converge*/
-    migrate_set_parameter_int(from, "downtime-limit", 1);
-    /* 1GB/s */
-    migrate_set_parameter_int(from, "max-bandwidth", 1000000000);
 
     migrate_set_parameter_int(from, "multifd-channels", 16);
     migrate_set_parameter_int(to, "multifd-channels", 16);
@@ -1751,41 +1737,58 @@ static void test_multifd_tcp(const char *method)
                            "  'arguments': { 'uri': 'tcp:127.0.0.1:0' }}");
     qobject_unref(rsp);
 
-    /* Wait for the first serial output from the source */
-    wait_for_serial("src_serial");
-
-    uri = migrate_get_socket_address(to, "socket-address");
-
-    migrate_qmp(from, uri, "{}");
+    return NULL;
+}
 
-    wait_for_migration_pass(from);
+static void *
+test_migration_precopy_tcp_multifd_start(QTestState *from,
+                                         QTestState *to)
+{
+    return test_migration_precopy_tcp_multifd_start_common(from, to, "none");
+}
 
-    migrate_set_parameter_int(from, "downtime-limit", CONVERGE_DOWNTIME);
+static void *
+test_migration_precopy_tcp_multifd_zlib_start(QTestState *from,
+                                              QTestState *to)
+{
+    return test_migration_precopy_tcp_multifd_start_common(from, to, "zlib");
+}
 
-    if (!got_stop) {
-        qtest_qmp_eventwait(from, "STOP");
-    }
-    qtest_qmp_eventwait(to, "RESUME");
+#ifdef CONFIG_ZSTD
+static void *
+test_migration_precopy_tcp_multifd_zstd_start(QTestState *from,
+                                              QTestState *to)
+{
+    return test_migration_precopy_tcp_multifd_start_common(from, to, "zstd");
+}
+#endif /* CONFIG_ZSTD */
 
-    wait_for_serial("dest_serial");
-    wait_for_migration_complete(from);
-    test_migrate_end(from, to, true);
+static void test_multifd_tcp_common(TestMigrateStartHook start_hook)
+{
+    test_precopy_common("defer",
+                        NULL, /* connect_uri */
+                        start_hook,
+                        NULL, /* finish_hook */
+                        false, /* expect_fail */
+                        false, /* dst_quit */
+                        1, /* iterations */
+                        false /* dirty_ring */);
 }
 
 static void test_multifd_tcp_none(void)
 {
-    test_multifd_tcp("none");
+    test_multifd_tcp_common(test_migration_precopy_tcp_multifd_start);
 }
 
 static void test_multifd_tcp_zlib(void)
 {
-    test_multifd_tcp("zlib");
+    test_multifd_tcp_common(test_migration_precopy_tcp_multifd_zlib_start);
 }
 
 #ifdef CONFIG_ZSTD
 static void test_multifd_tcp_zstd(void)
 {
-    test_multifd_tcp("zstd");
+    test_multifd_tcp_common(test_migration_precopy_tcp_multifd_zstd_start);
 }
 #endif
 
Signature · 1 line
2.34.1
#44 lore raw

PATCH 16/18 tests: add multifd migration tests of TLS with PSK credentials

In reply to Daniel P. Berrangé

This validates that we correctly handle multifd migration success and failure scenarios when using TLS with pre shared keys.

---
 tests/qtest/migration-test.c | 94 ++++++++++++++++++++++++++++--------
 1 file changed, 75 insertions(+), 19 deletions(-)
tests/qtest/migration-test.c +75 -19
index 7c69268aa8..506c6996e0 100644
@@ -1717,9 +1717,9 @@ static void test_migrate_auto_converge(void)
 }
 
 static void *
-test_migration_precopy_tcp_multifd_start_common(QTestState *from,
-                                                QTestState *to,
-                                                const char *method)
+test_migrate_precopy_tcp_multifd_start_common(QTestState *from,
+                                              QTestState *to,
+                                              const char *method)
 {
     QDict *rsp;
 
@@ -1741,25 +1741,25 @@ test_migration_precopy_tcp_multifd_start_common(QTestState *from,
 }
 
 static void *
-test_migration_precopy_tcp_multifd_start(QTestState *from,
-                                         QTestState *to)
+test_migrate_precopy_tcp_multifd_start(QTestState *from,
+                                       QTestState *to)
 {
-    return test_migration_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
 }
 
 static void *
-test_migration_precopy_tcp_multifd_zlib_start(QTestState *from,
-                                              QTestState *to)
+test_migrate_precopy_tcp_multifd_zlib_start(QTestState *from,
+                                            QTestState *to)
 {
-    return test_migration_precopy_tcp_multifd_start_common(from, to, "zlib");
+    return test_migrate_precopy_tcp_multifd_start_common(from, to, "zlib");
 }
 
 #ifdef CONFIG_ZSTD
 static void *
-test_migration_precopy_tcp_multifd_zstd_start(QTestState *from,
-                                              QTestState *to)
+test_migrate_precopy_tcp_multifd_zstd_start(QTestState *from,
+                                            QTestState *to)
 {
-    return test_migration_precopy_tcp_multifd_start_common(from, to, "zstd");
+    return test_migrate_precopy_tcp_multifd_start_common(from, to, "zstd");
 }
 #endif /* CONFIG_ZSTD */
 
@@ -1777,18 +1777,64 @@ static void test_multifd_tcp_common(TestMigrateStartHook start_hook)
 
 static void test_multifd_tcp_none(void)
 {
-    test_multifd_tcp_common(test_migration_precopy_tcp_multifd_start);
+    test_multifd_tcp_common(test_migrate_precopy_tcp_multifd_start);
 }
 
 static void test_multifd_tcp_zlib(void)
 {
-    test_multifd_tcp_common(test_migration_precopy_tcp_multifd_zlib_start);
+    test_multifd_tcp_common(test_migrate_precopy_tcp_multifd_zlib_start);
 }
 
 #ifdef CONFIG_ZSTD
 static void test_multifd_tcp_zstd(void)
 {
-    test_multifd_tcp_common(test_migration_precopy_tcp_multifd_zstd_start);
+    test_multifd_tcp_common(test_migrate_precopy_tcp_multifd_zstd_start);
+}
+#endif
+
+#ifdef CONFIG_GNUTLS
+static void test_multifd_tcp_tls_common(TestMigrateStartHook start_hook,
+                                        TestMigrateFinishHook finish_hook,
+                                        bool expect_fail)
+{
+    test_precopy_common("defer",
+                        NULL, /* connect_uri */
+                        start_hook,
+                        finish_hook,
+                        expect_fail,
+                        false, /* dst_quit */
+                        1, /* iterations */
+                        false /* dirty_ring */);
+}
+
+static void *
+test_migrate_multifd_tcp_tls_psk_start_match(QTestState *from,
+                                             QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_psk_start_match(from, to);
+}
+
+static void *
+test_migrate_multifd_tcp_tls_psk_start_mismatch(QTestState *from,
+                                                QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_psk_start_mismatch(from, to);
+}
+
+static void test_multifd_tcp_tls_psk_match(void)
+{
+    test_multifd_tcp_tls_common(test_migrate_multifd_tcp_tls_psk_start_match,
+                                test_migrate_tls_psk_finish,
+                                false /* expect_fail */);
+}
+
+static void test_multifd_tcp_tls_psk_mismatch(void)
+{
+    test_multifd_tcp_tls_common(test_migrate_multifd_tcp_tls_psk_start_mismatch,
+                                test_migrate_tls_psk_finish,
+                                true /* expect_fail */);
 }
 #endif
 
@@ -2001,12 +2047,22 @@ int main(int argc, char **argv)
                    test_validate_uuid_dst_not_set);
 
     qtest_add_func("/migration/auto_converge", test_migrate_auto_converge);
-    qtest_add_func("/migration/multifd/tcp/none", test_multifd_tcp_none);
-    qtest_add_func("/migration/multifd/tcp/cancel", test_multifd_tcp_cancel);
-    qtest_add_func("/migration/multifd/tcp/zlib", test_multifd_tcp_zlib);
+    qtest_add_func("/migration/multifd/tcp/plain/none",
+                   test_multifd_tcp_none);
+    qtest_add_func("/migration/multifd/tcp/plain/cancel",
+                   test_multifd_tcp_cancel);
+    qtest_add_func("/migration/multifd/tcp/plain/zlib",
+                   test_multifd_tcp_zlib);
 #ifdef CONFIG_ZSTD
-    qtest_add_func("/migration/multifd/tcp/zstd", test_multifd_tcp_zstd);
+    qtest_add_func("/migration/multifd/tcp/plain/zstd",
+                   test_multifd_tcp_zstd);
 #endif
+#ifdef CONFIG_GNUTLS
+    qtest_add_func("/migration/multifd/tcp/tls/psk/match",
+                   test_multifd_tcp_tls_psk_match);
+    qtest_add_func("/migration/multifd/tcp/tls/psk/mismatch",
+                   test_multifd_tcp_tls_psk_mismatch);
+#endif /* CONFIG_GNUTLS */
 
     if (kvm_dirty_ring_supported()) {
         qtest_add_func("/migration/dirty_ring",
Signature · 1 line
2.34.1
#45 lore raw

PATCH 17/18 tests: add multifd migration tests of TLS with x509 credentials

In reply to Daniel P. Berrangé

This validates that we correctly handle multifd migration success and failure scenarios when using TLS with x509 certificates. There are quite a few different scenarios that matter in relation to hostname validation, but we skip a couple as we can assume that the non-multifd coverage applies to some extent.

---
 tests/qtest/migration-test.c | 135 ++++++++++++++++++++++++++++++++---
 1 file changed, 126 insertions(+), 9 deletions(-)
tests/qtest/migration-test.c +126 -9
index 506c6996e0..95ae843e1b 100644
@@ -1795,20 +1795,21 @@ static void test_multifd_tcp_zstd(void)
 #ifdef CONFIG_GNUTLS
 static void test_multifd_tcp_tls_common(TestMigrateStartHook start_hook,
                                         TestMigrateFinishHook finish_hook,
-                                        bool expect_fail)
+                                        bool expect_fail,
+                                        bool dst_quit)
 {
     test_precopy_common("defer",
                         NULL, /* connect_uri */
                         start_hook,
                         finish_hook,
                         expect_fail,
-                        false, /* dst_quit */
+                        dst_quit,
                         1, /* iterations */
                         false /* dirty_ring */);
 }
 
 static void *
-test_migrate_multifd_tcp_tls_psk_start_match(QTestState *from,
+test_migrate_multifd_tls_psk_start_match(QTestState *from,
                                              QTestState *to)
 {
     test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
@@ -1816,27 +1817,131 @@ test_migrate_multifd_tcp_tls_psk_start_match(QTestState *from,
 }
 
 static void *
-test_migrate_multifd_tcp_tls_psk_start_mismatch(QTestState *from,
+test_migrate_multifd_tls_psk_start_mismatch(QTestState *from,
                                                 QTestState *to)
 {
     test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
     return test_migrate_tls_psk_start_mismatch(from, to);
 }
 
+#ifdef CONFIG_TASN1
+static void *
+test_migrate_multifd_tls_x509_start_default_host(QTestState *from,
+                                                 QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_x509_start_default_host(from, to);
+}
+
+static void *
+test_migrate_multifd_tls_x509_start_override_host(QTestState *from,
+                                                  QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_x509_start_override_host(from, to);
+}
+
+static void *
+test_migrate_multifd_tls_x509_start_mismatch_host(QTestState *from,
+                                                  QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_x509_start_mismatch_host(from, to);
+}
+
+static void *
+test_migrate_multifd_tls_x509_start_allow_anonymous_client(QTestState *from,
+                                                           QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_x509_start_allow_anonymous_client(from, to);
+}
+
+static void *
+test_migrate_multifd_tls_x509_start_reject_anonymous_client(QTestState *from,
+                                                            QTestState *to)
+{
+    test_migrate_precopy_tcp_multifd_start_common(from, to, "none");
+    return test_migrate_tls_x509_start_reject_anonymous_client(from, to);
+}
+#endif /* CONFIG_TASN1 */
+
 static void test_multifd_tcp_tls_psk_match(void)
 {
-    test_multifd_tcp_tls_common(test_migrate_multifd_tcp_tls_psk_start_match,
+    test_multifd_tcp_tls_common(test_migrate_multifd_tls_psk_start_match,
                                 test_migrate_tls_psk_finish,
-                                false /* expect_fail */);
+                                false, /* expect_fail */
+                                false /* dst_quit */);
 }
 
 static void test_multifd_tcp_tls_psk_mismatch(void)
 {
-    test_multifd_tcp_tls_common(test_migrate_multifd_tcp_tls_psk_start_mismatch,
+    test_multifd_tcp_tls_common(test_migrate_multifd_tls_psk_start_mismatch,
                                 test_migrate_tls_psk_finish,
-                                true /* expect_fail */);
+                                true, /* expect_fail */
+                                false /* dst_quit */);
 }
-#endif
+
+#ifdef CONFIG_TASN1
+static void test_multifd_tcp_tls_x509_default_host(void)
+{
+    test_multifd_tcp_tls_common(
+        test_migrate_multifd_tls_x509_start_default_host,
+        test_migrate_tls_x509_finish,
+        false, /* expect_fail */
+        false /* dst_quit */);
+}
+
+static void test_multifd_tcp_tls_x509_override_host(void)
+{
+    test_multifd_tcp_tls_common(
+        test_migrate_multifd_tls_x509_start_override_host,
+        test_migrate_tls_x509_finish,
+        false, /* expect_fail */
+        false /* dst_quit */);
+}
+
+static void test_multifd_tcp_tls_x509_mismatch_host(void)
+{
+    /*
+     * This has different behaviour to the non-multifd case.
+     *
+     * In non-multifd case when client aborts due to mismatched
+     * cert host, the server has already started trying to load
+     * migration state, and so it exits with I/O  failure.
+     *
+     * In multifd case when client aborts due to mismatched
+     * cert host, the server is still waiting for the other
+     * multifd connections to arrive so hasn't started trying
+     * to load migration state, and thus just aborts the migration
+     * without exiting
+     */
+    test_multifd_tcp_tls_common(
+        test_migrate_multifd_tls_x509_start_mismatch_host,
+        test_migrate_tls_x509_finish,
+        true, /* expect_fail */
+        false /* dst_quit */);
+}
+
+static void test_multifd_tcp_tls_x509_allow_anonymous_client(void)
+{
+    test_multifd_tcp_tls_common(
+        test_migrate_multifd_tls_x509_start_allow_anonymous_client,
+        test_migrate_tls_x509_finish,
+        false, /* expect_fail */
+        false /* dst_quit */);
+}
+
+static void test_multifd_tcp_tls_x509_reject_anonymous_client(void)
+{
+    test_multifd_tcp_tls_common(
+        test_migrate_multifd_tls_x509_start_reject_anonymous_client,
+        test_migrate_tls_x509_finish,
+        true, /* expect_fail */
+        false /* dst_quit */);
+}
+#endif /* CONFIG_TASN1 */
+#endif /* CONFIHG_GNUTLS */
 
 /*
  * This test does:
@@ -2062,6 +2167,18 @@ int main(int argc, char **argv)
                    test_multifd_tcp_tls_psk_match);
     qtest_add_func("/migration/multifd/tcp/tls/psk/mismatch",
                    test_multifd_tcp_tls_psk_mismatch);
+#ifdef CONFIG_TASN1
+    qtest_add_func("/migration/multifd/tcp/tls/x509/default-host",
+                   test_multifd_tcp_tls_x509_default_host);
+    qtest_add_func("/migration/multifd/tcp/tls/x509/override-host",
+                   test_multifd_tcp_tls_x509_override_host);
+    qtest_add_func("/migration/multifd/tcp/tls/x509/mismatch-host",
+                   test_multifd_tcp_tls_x509_mismatch_host);
+    qtest_add_func("/migration/multifd/tcp/tls/x509/allow-anonymous-client",
+                   test_multifd_tcp_tls_x509_allow_anonymous_client);
+    qtest_add_func("/migration/multifd/tcp/tls/x509/reject-anonymous-client",
+                   test_multifd_tcp_tls_x509_reject_anonymous_client);
+#endif /* CONFIG_TASN1 */
 #endif /* CONFIG_GNUTLS */
 
     if (kvm_dirty_ring_supported()) {
Signature · 1 line
2.34.1
#46 lore raw

PATCH 18/18 tests: ensure migration status isn't reported as failed

In reply to Daniel P. Berrangé

Various methods in the migration test call 'query_migrate' to fetch the current status and then access a particular field. Almost all of these cases expect the migration to be in a non-failed state. In the case of 'wait_for_migration_pass' in particular, if the status is 'failed' then it will get into an infinite loop. By validating that the status is not 'failed' the test suite will assert rather than hang when getting into an unexpected state.

---
 tests/qtest/migration-helpers.c | 13 +++++++++++++
 tests/qtest/migration-helpers.h |  1 +
 tests/qtest/migration-test.c    |  6 +++---
 3 files changed, 17 insertions(+), 3 deletions(-)
tests/qtest/migration-helpers.c +13 -0
index 4ee26014b7..a6aa59e4e6 100644
@@ -107,6 +107,19 @@ QDict *migrate_query(QTestState *who)
     return wait_command(who, "{ 'execute': 'query-migrate' }");
 }
 
+QDict *migrate_query_not_failed(QTestState *who)
+{
+    const char *status;
+    QDict *rsp = migrate_query(who);
+    status = qdict_get_str(rsp, "status");
+    if (g_str_equal(status, "failed")) {
+        g_printerr("query-migrate shows failed migration: %s\n",
+                   qdict_get_str(rsp, "error-desc"));
+    }
+    g_assert(!g_str_equal(status, "failed"));
+    return rsp;
+}
+
 /*
  * Note: caller is responsible to free the returned object via
  * g_free() after use
tests/qtest/migration-helpers.h +1 -0
index d63bba9630..b710ece67e 100644
@@ -26,6 +26,7 @@ GCC_FMT_ATTR(3, 4)
 void migrate_qmp(QTestState *who, const char *uri, const char *fmt, ...);
 
 QDict *migrate_query(QTestState *who);
+QDict *migrate_query_not_failed(QTestState *who);
 
 void wait_for_migration_status(QTestState *who,
                                const char *goal, const char **ungoals);
tests/qtest/migration-test.c +3 -3
index 95ae843e1b..3570a7895c 100644
@@ -181,7 +181,7 @@ static int64_t read_ram_property_int(QTestState *who, const char *property)
     QDict *rsp_return, *rsp_ram;
     int64_t result;
 
-    rsp_return = migrate_query(who);
+    rsp_return = migrate_query_not_failed(who);
     if (!qdict_haskey(rsp_return, "ram")) {
         /* Still in setup */
         result = 0;
@@ -198,7 +198,7 @@ static int64_t read_migrate_property_int(QTestState *who, const char *property)
     QDict *rsp_return;
     int64_t result;
 
-    rsp_return = migrate_query(who);
+    rsp_return = migrate_query_not_failed(who);
     result = qdict_get_try_int(rsp_return, property, 0);
     qobject_unref(rsp_return);
     return result;
@@ -213,7 +213,7 @@ static void read_blocktime(QTestState *who)
 {
     QDict *rsp_return;
 
-    rsp_return = migrate_query(who);
+    rsp_return = migrate_query_not_failed(who);
     g_assert(qdict_haskey(rsp_return, "postcopy-blocktime"));
     qobject_unref(rsp_return);
 }
Signature · 1 line
2.34.1
Reviewed-by on v1 18/18
#47 lore raw

Re: [PATCH 18/18] tests: ensure migration status isn't reported as failed

Quoted 9 lines · Daniel P. Berrangé

Various methods in the migration test call 'query_migrate' to fetch the current status and then access a particular field. Almost all of these cases expect the migration to be in a non-failed state. In the case of 'wait_for_migration_pass' in particular, if the status is 'failed' then it will get into an infinite loop. By validating that the status is not 'failed' the test suite will assert rather than hang when getting into an unexpected state.

Signature · 1 line
Peter Xu