migration: Reduce migration-test time for non-KVM archs
cover migration: Reduce migration-test time for non-KVM archs
Hi, continuing the work from the previous[1] series to reduce the time migration-test takes during make check, here's a couple of patches to create a smaller set of tests.
The change is that from now on, ./migration-test will only run a limited set of tests (~12), while the full set (64) requires that the --full flag is passed on. Only those calling the test manually need to care about this new flag.
For everyone else, qtest/meson.build will automatically add --full in case KVM is available for the target architecture. All other architectures run the small set.
The choice of tests for the small set has been arbitrarily made in patch 2. I expect that once we close in on a list of tests, new tests will only be added under the --full flag and the small set will be kept stable.
An unwritten but maybe obvious rule is that slow or flaky tests should not be added to the small set and once detected should be moved to the full set.
More details in patch 1.
Thanks!
CI run: https://gitlab.com/farosas/qemu/-/pipelines/1647478788
1- https://lore.kernel.org/r/[email protected]
Fabiano Rosas (2):
tests/qtest/migration: Add --full option
tests/qtest/migration: Pick smoke tests
tests/qtest/meson.build | 11 +++++++++- tests/qtest/migration-test.c | 25 +++++++++++++++++++++++ tests/qtest/migration/compression-tests.c | 15 +++++++++++--- tests/qtest/migration/cpr-tests.c | 6 ++++++ tests/qtest/migration/file-tests.c | 21 +++++++++++++------ tests/qtest/migration/framework.h | 1 + tests/qtest/migration/misc-tests.c | 16 ++++++++++++--- tests/qtest/migration/postcopy-tests.c | 14 ++++++++++++- tests/qtest/migration/precopy-tests.c | 23 ++++++++++++++------- tests/qtest/migration/tls-tests.c | 14 +++++++++++-- 10 files changed, 123 insertions(+), 23 deletions(-)
Signature · 1 line
2.35.3
PATCH 1/2 tests/qtest/migration: Add --full option
Add a new command line option to allow selecting between running the full set of tests or a smaller set of tests. The default will be to run the small set (i.e. no comand line option provided) so we can reduce the amount of tests run by default. Only hosts which support KVM for the target architecture being tested will run the complete set of tests.
Adjust the meson.build file to pass in the --full option when appropriate.
(for now, set the option unconditionally until the next patch actually creates the small set)
Use cases:
configure --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-softmmu
| before - 615s/244 tests | after - 244s/100 tests
------------------------+--------------------------+-----------------------------
make check | full set for all archs | full set for the KVM arch,
make check-qtest | | small set for the rest
| |
qemu-system-$ARCH | full set for $ARCH | small set for $ARCH, KVM or
./migration-test | | TCG automatically chosen
| |
qemu-system-$ARCH | N/A | full set for $ARCH, KVM or
./migration-test --full | | TCG automatically chosen
| |
migration-compat-x86_64 | full set for x86_64 | small set for x86_64
CI job | |------------------------+--------------------------+-----------------------------
--- tests/qtest/meson.build | 11 +++++++++- tests/qtest/migration-test.c | 26 +++++++++++++++++++++++ tests/qtest/migration/compression-tests.c | 4 ++++ tests/qtest/migration/cpr-tests.c | 4 ++++ tests/qtest/migration/file-tests.c | 4 ++++ tests/qtest/migration/framework.h | 1 + tests/qtest/migration/misc-tests.c | 4 ++++ tests/qtest/migration/postcopy-tests.c | 4 ++++ tests/qtest/migration/precopy-tests.c | 4 ++++ tests/qtest/migration/tls-tests.c | 4 ++++ 10 files changed, 65 insertions(+), 1 deletion(-)
tests/qtest/meson.build +10 -1
@@ -402,6 +402,8 @@ foreach dir : target_dirs target_base = dir.split('-')[0] qtest_emulator = emulators['qemu-system-' + target_base] target_qtests = get_variable('qtests_' + target_base, []) + qtests_generic + has_kvm = ('CONFIG_KVM' in config_all_accel and host_os == 'linux' + and cpu == target_base and fs.exists('/dev/kvm')) test_deps = roms qtest_env = environment()
@@ -435,11 +437,18 @@ foreach dir : target_dirs test: executable(test, src, dependencies: deps) } endif + + test_args = ['--tap', '-k'] + + if test == 'migration-test' and has_kvm + test_args += ['--full'] + endif + test('qtest-@0@/@1@'.format(target_base, test), qtest_executables[test], depends: [test_deps, qtest_emulator, emulator_modules], env: qtest_env, - args: ['--tap', '-k'], + args: test_args, protocol: 'tap', timeout: slow_qtests.get(test, 60), priority: slow_qtests.get(test, 60),
tests/qtest/migration-test.c +26 -0
@@ -14,13 +14,39 @@#include "migration/framework.h" #include "qemu/module.h" +static void parse_args (int *argc_p,char ***argv_p,bool *full_set) +{ +int argc = *argc_p; +char **argv = *argv_p; +int i, j; + + j =1 ; +for (i =1 ; i < argc; i++) { +if (g_str_equal (argv[i],"--full" )) { + *full_set = true; +continue ; + } + argv[j++] = argv[i]; +if (i >= j) { + argv[i] =NULL ; + } + } + *argc_p = j; +} +int main (int argc,char **argv) {MigrationTestEnv *env;int ret; +bool full_set = false; + +/* strip the --full option if it's present */ +parse_args (&argc, &argv, &full_set);g_test_init (&argc, &argv,NULL ); env =migration_get_env (); + env->full_set = full_set; + env->full_set = true;/* temporary */ module_call_init (MODULE_INIT_QOM );migration_test_add_tls (env);
tests/qtest/migration/compression-tests.c +4 -0
@@ -155,6 +155,10 @@ void migration_test_add_compression(MigrationTestEnv *env) { tmpfs = env->tmpfs; +if (!env->full_set) { +return ; + } +#ifdef CONFIG_ZSTD migration_test_add ("/migration/multifd/tcp/plain/zstd" , test_multifd_tcp_zstd);
tests/qtest/migration/cpr-tests.c +4 -0
@@ -48,6 +48,10 @@ void migration_test_add_cpr(MigrationTestEnv *env) { tmpfs = env->tmpfs; +if (!env->full_set) { +return ; + } +/* * Our CI system has problems with shared memory. * Don't run this test until we find a workaround.
tests/qtest/migration/file-tests.c +4 -0
@@ -304,6 +304,10 @@ void migration_test_add_file(MigrationTestEnv *env) { tmpfs = env->tmpfs; +if (!env->full_set) { +return ; + } +migration_test_add ("/migration/precopy/file" , test_precopy_file);
tests/qtest/migration/framework.h +1 -0
@@ -24,6 +24,7 @@ typedef struct MigrationTestEnv {bool uffd_feature_thread_id;bool has_dirty_ring;bool is_x86; +bool full_set;const char *arch;const char *qemu_src;const char *qemu_dst;
tests/qtest/migration/misc-tests.c +4 -0
@@ -255,6 +255,10 @@ void migration_test_add_misc(MigrationTestEnv *env) { tmpfs = env->tmpfs; +if (!env->full_set) { +return ; + } +migration_test_add ("/migration/bad_dest" , test_baddest);#ifndef _WIN32migration_test_add ("/migration/analyze-script" , test_analyze_script);
tests/qtest/migration/postcopy-tests.c +4 -0
@@ -81,6 +81,10 @@ static void test_postcopy_preempt_recovery(void)void migration_test_add_postcopy (MigrationTestEnv *env) { +if (!env->full_set) { +return ; + } +if (env->has_uffd) {migration_test_add ("/migration/postcopy/plain" , test_postcopy); migration_test_add("/migration/postcopy/recovery/plain",
tests/qtest/migration/precopy-tests.c +4 -0
@@ -955,6 +955,10 @@ void migration_test_add_precopy(MigrationTestEnv *env) { tmpfs = env->tmpfs; +if (!env->full_set) { +return ; + } +if (env->is_x86) {migration_test_add ("/migration/precopy/unix/suspend/live" , test_precopy_unix_suspend_live);
tests/qtest/migration/tls-tests.c +4 -0
@@ -726,6 +726,10 @@ void migration_test_add_tls(MigrationTestEnv *env) { tmpfs = env->tmpfs; +if (!env->full_set) { +return ; + } +migration_test_add ("/migration/precopy/unix/tls/psk" , test_precopy_unix_tls_psk);
Signature · 1 line
2.35.3
PATCH 2/2 tests/qtest/migration: Pick smoke tests
In reply to Fabiano RosasChoose a few tests per group and move them from the full set to the smoke set.
--- tests/qtest/migration-test.c | 1 - tests/qtest/migration/compression-tests.c | 11 ++++++--- tests/qtest/migration/cpr-tests.c | 2 ++ tests/qtest/migration/file-tests.c | 17 +++++++++----- tests/qtest/migration/misc-tests.c | 12 +++++++--- tests/qtest/migration/postcopy-tests.c | 18 ++++++++++----- tests/qtest/migration/precopy-tests.c | 27 ++++++++++++++--------- tests/qtest/migration/tls-tests.c | 10 +++++++-- 8 files changed, 67 insertions(+), 31 deletions(-)
tests/qtest/migration-test.c +0 -1
@@ -46,7 +46,6 @@ int main(int argc, char **argv)g_test_init (&argc, &argv,NULL ); env =migration_get_env (); env->full_set = full_set; - env->full_set = true;/* temporary */ module_call_init (MODULE_INIT_QOM );migration_test_add_tls (env);
tests/qtest/migration/compression-tests.c +8 -3
@@ -151,10 +151,18 @@ static void test_multifd_tcp_zlib(void)test_precopy_common (&args); } +static void migration_test_add_compression_smoke (MigrationTestEnv *env) +{ +migration_test_add ("/migration/multifd/tcp/plain/zlib" , + test_multifd_tcp_zlib); +} +void migration_test_add_compression (MigrationTestEnv *env) { tmpfs = env->tmpfs; +migration_test_add_compression_smoke (env); +if (!env->full_set) {return ; }
@@ -183,7 +191,4 @@ void migration_test_add_compression(MigrationTestEnv *env)migration_test_add ("/migration/precopy/unix/xbzrle" , test_precopy_unix_xbzrle); } - -migration_test_add ("/migration/multifd/tcp/plain/zlib" , - test_multifd_tcp_zlib); }
tests/qtest/migration/cpr-tests.c +2 -0
@@ -48,6 +48,8 @@ void migration_test_add_cpr(MigrationTestEnv *env) { tmpfs = env->tmpfs; +/* no tests in the smoke set for now */ +if (!env->full_set) {return ; }
tests/qtest/migration/file-tests.c +11 -6
@@ -300,17 +300,25 @@ static void test_multifd_file_mapped_ram_fdset_dio(void) }#endif /* !_WIN32 */ +static void migration_test_add_file_smoke (MigrationTestEnv *env) +{ +migration_test_add ("/migration/precopy/file" , + test_precopy_file); + +migration_test_add ("/migration/multifd/file/mapped-ram/dio" , + test_multifd_file_mapped_ram_dio); +} +void migration_test_add_file (MigrationTestEnv *env) { tmpfs = env->tmpfs; +migration_test_add_file_smoke (env); +if (!env->full_set) {return ; } -migration_test_add ("/migration/precopy/file" , - test_precopy_file); -migration_test_add ("/migration/precopy/file/offset" , test_precopy_file_offset);#ifndef _WIN32
@@ -330,9 +338,6 @@ void migration_test_add_file(MigrationTestEnv *env)migration_test_add ("/migration/multifd/file/mapped-ram/live" , test_multifd_file_mapped_ram_live); -migration_test_add ("/migration/multifd/file/mapped-ram/dio" , - test_multifd_file_mapped_ram_dio); -#ifndef _WIN32migration_test_add ("/migration/multifd/file/mapped-ram/fdset" , test_multifd_file_mapped_ram_fdset);
tests/qtest/migration/misc-tests.c +9 -3
@@ -251,18 +251,24 @@ static void test_validate_uri_channels_none_set(void)do_test_validate_uri_channel (&args); } +static void migration_test_add_misc_smoke (MigrationTestEnv *env) +{ +#ifndef _WIN32 +migration_test_add ("/migration/analyze-script" , test_analyze_script); +#endif +} +void migration_test_add_misc (MigrationTestEnv *env) { tmpfs = env->tmpfs; +migration_test_add_misc_smoke (env); +if (!env->full_set) {return ; }migration_test_add ("/migration/bad_dest" , test_baddest); -#ifndef _WIN32 -migration_test_add ("/migration/analyze-script" , test_analyze_script); -#endif /* * Our CI system has problems with shared memory.
tests/qtest/migration/postcopy-tests.c +13 -5
@@ -79,18 +79,26 @@ static void test_postcopy_preempt_recovery(void)test_postcopy_recovery_common (&args); } -void migration_test_add_postcopy (MigrationTestEnv *env) +static void migration_test_add_postcopy_smoke (MigrationTestEnv *env) { -if (!env->full_set) { -return ; - } -if (env->has_uffd) {migration_test_add ("/migration/postcopy/plain" , test_postcopy);migration_test_add ("/migration/postcopy/recovery/plain" , test_postcopy_recovery);migration_test_add ("/migration/postcopy/preempt/plain" , test_postcopy_preempt); + } +} + +void migration_test_add_postcopy (MigrationTestEnv *env) +{ +migration_test_add_postcopy_smoke (env); + +if (!env->full_set) { +return ; + } + +if (env->has_uffd) {migration_test_add ("/migration/postcopy/preempt/recovery/plain" , test_postcopy_preempt_recovery);
tests/qtest/migration/precopy-tests.c +16 -11
@@ -951,14 +951,8 @@ static void test_dirty_limit(void)migrate_end (from, to, true); } -void migration_test_add_precopy (MigrationTestEnv *env) +static void migration_test_add_precopy_smoke (MigrationTestEnv *env) { - tmpfs = env->tmpfs; - -if (!env->full_set) { -return ; - } -if (env->is_x86) {migration_test_add ("/migration/precopy/unix/suspend/live" , test_precopy_unix_suspend_live);
@@ -970,6 +964,21 @@ void migration_test_add_precopy(MigrationTestEnv *env) test_precopy_unix_plain);migration_test_add ("/migration/precopy/tcp/plain" , test_precopy_tcp_plain); +migration_test_add ("/migration/multifd/tcp/uri/plain/none" , + test_multifd_tcp_uri_none); +migration_test_add ("/migration/multifd/tcp/plain/cancel" , + test_multifd_tcp_cancel); +} + +void migration_test_add_precopy (MigrationTestEnv *env) +{ + tmpfs = env->tmpfs; + +migration_test_add_precopy_smoke (env); + +if (!env->full_set) { +return ; + }migration_test_add ("/migration/precopy/tcp/plain/switchover-ack" , test_precopy_tcp_switchover_ack);
@@ -993,16 +1002,12 @@ void migration_test_add_precopy(MigrationTestEnv *env) test_dirty_limit); } } -migration_test_add ("/migration/multifd/tcp/uri/plain/none" , - test_multifd_tcp_uri_none);migration_test_add ("/migration/multifd/tcp/channels/plain/none" , test_multifd_tcp_channels_none);migration_test_add ("/migration/multifd/tcp/plain/zero-page/legacy" , test_multifd_tcp_zero_page_legacy);migration_test_add ("/migration/multifd/tcp/plain/zero-page/none" , test_multifd_tcp_no_zero_page); -migration_test_add ("/migration/multifd/tcp/plain/cancel" , - test_multifd_tcp_cancel);if (g_str_equal (env->arch,"x86_64" ) && env->has_kvm && env->has_dirty_ring) {
tests/qtest/migration/tls-tests.c +8 -2
@@ -722,10 +722,18 @@ static void test_multifd_tcp_tls_x509_reject_anon_client(void) }#endif /* CONFIG_TASN1 */ +static void migration_test_add_tls_smoke (MigrationTestEnv *env) +{ +migration_test_add ("/migration/precopy/tcp/tls/psk/match" , + test_precopy_tcp_tls_psk_match); +} +void migration_test_add_tls (MigrationTestEnv *env) { tmpfs = env->tmpfs; +migration_test_add_tls_smoke (env); +if (!env->full_set) {return ; }
@@ -755,8 +763,6 @@ void migration_test_add_tls(MigrationTestEnv *env) test_precopy_unix_tls_x509_override_host);#endif /* CONFIG_TASN1 */ -migration_test_add ("/migration/precopy/tcp/tls/psk/match" , - test_precopy_tcp_tls_psk_match);migration_test_add ("/migration/precopy/tcp/tls/psk/mismatch" , test_precopy_tcp_tls_psk_mismatch);#ifdef CONFIG_TASN1
Signature · 1 line
2.35.3
Quoted 18 lines · Fabiano Rosas
Hi, continuing the work from the previous[1] series to reduce the time migration-test takes during make check, here's a couple of patches to create a smaller set of tests.
The change is that from now on, ./migration-test will only run a limited set of tests (~12), while the full set (64) requires that the --full flag is passed on. Only those calling the test manually need to care about this new flag.
For everyone else, qtest/meson.build will automatically add --full in case KVM is available for the target architecture. All other architectures run the small set.
The choice of tests for the small set has been arbitrarily made in patch 2. I expect that once we close in on a list of tests, new tests will only be added under the --full flag and the small set will be kept stable.
Quoted 5 lines
An unwritten but maybe obvious rule is that slow or flaky tests should not be added to the small set and once detected should be moved to the full set.
More details in patch 1.
Looks all good here, thanks!
Signature · 1 line
Peter Xu
Quoted 3 lines · Fabiano Rosas
Hi, continuing the work from the previous[1] series to reduce the time migration-test takes during make check, here's a couple of patches to create a smaller set of tests.
Queued to testing/next, thanks.
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro