Add option to disable building tests
PATCH 1/1 Add option to disable building tests
There are situations in which you might want to build QEMU without building the full test suite, which in some configurations can take a considerable amount of time to build. This is especially true when using LTO with clang. For example:
$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto-objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target-list=x86_64-softmmu [...] $ time make -j8 [...] [3078/3078] Linking target tests/qtest/qos-test real 6m43.250s user 101m15.967s sys 4m3.813s
$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto-objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target-list=x86_64-softmmu --disable-tests [...] $ time make -j8 [...] [2024/2024] Linking target qemu-system-x86_64
real 3m14.277s user 33m13.174s sys 1m36.642s
Add a toggle to optionally disable building tests.
--- meson.build | 2 +- meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 3 files changed, 6 insertions(+), 1 deletion(-)
meson.build +1 -1
@@ -4605,7 +4605,7 @@ subdir('docs') subdir('pyvenv') # Tests are disabled on emscripten because they rely on host features that aren't # supported by emscripten (e.g. fork and unix socket). -if host_os != 'emscripten' +if get_option('tests').allowed() and host_os != 'emscripten' subdir('tests') endif if gtk.found()
meson_options.txt +2 -0
@@ -43,6 +43,8 @@ option('gdb', type: 'string', value: '', # on the configure script command line. After adding an option # here make sure to run "make update-buildoptions". +option('tests', type: 'feature', value: 'auto', + description: 'Build the test suite') option('docs', type : 'feature', value : 'auto', description: 'Documentations build support') option('fuzzing', type : 'boolean', value: false,
scripts/meson-buildoptions.sh +3 -0
@@ -194,6 +194,7 @@ meson_options_help() { printf "%s\n" ' spice-protocol Spice protocol support' printf "%s\n" ' stack-protector compiler-provided stack protection' printf "%s\n" ' tcg TCG support' + printf "%s\n" ' tests build test suite' printf "%s\n" ' tools build support utilities that come with QEMU' printf "%s\n" ' tpm TPM support' printf "%s\n" ' u2f U2F emulation support'
@@ -515,6 +516,8 @@ _meson_option_parse() { --enable-tcg-interpreter) printf "%s" -Dtcg_interpreter=true ;; --disable-tcg-interpreter) printf "%s" -Dtcg_interpreter=false ;; --tls-priority=*) quote_sh "-Dtls_priority=$2" ;; + --enable-tests) printf "%s" -Dtests=enabled ;; + --disable-tests) printf "%s" -Dtests=disabled ;; --enable-tools) printf "%s" -Dtools=enabled ;; --disable-tools) printf "%s" -Dtools=disabled ;; --enable-tpm) printf "%s" -Dtpm=enabled ;;
Signature · 1 line
2.47.3
Quoted 72 lines · Florian Schmidt
There are situations in which you might want to build QEMU without building the full test suite, which in some configurations can take a considerable amount of time to build. This is especially true when using LTO with clang. For example:
$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto-objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target-list=x86_64-softmmu [...] $ time make -j8 [...] [3078/3078] Linking target tests/qtest/qos-test real 6m43.250s user 101m15.967s sys 4m3.813s$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto-objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target-list=x86_64-softmmu --disable-tests [...] $ time make -j8 [...] [2024/2024] Linking target qemu-system-x86_64real 3m14.277s user 33m13.174s sys 1m36.642sAdd a toggle to optionally disable building tests.
Signed-off-by Florian Schmidt <[email protected]>--- meson.build | 2 +- meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 3 files changed, 6 insertions(+), 1 deletion(-)meson.build +1
-1@@ -4605,7 +4605,7 @@ subdir('docs') subdir('pyvenv') # Tests are disabled on emscripten because they rely on host features that aren't # supported by emscripten (e.g. fork and unix socket). -if host_os != 'emscripten' +if get_option('tests').allowed() and host_os != 'emscripten' subdir('tests') endif if gtk.found()meson_options.txt +2
-0@@ -43,6 +43,8 @@ option('gdb', type: 'string', value: '', # on the configure script command line. After adding an option # here make sure to run "make update-buildoptions". +option('tests', type: 'feature', value: 'auto', + description: 'Build the test suite') option('docs', type : 'feature', value : 'auto', description: 'Documentations build support') option('fuzzing', type : 'boolean', value: false,scripts/meson-buildoptions.sh +1
-0@@ -194,6 +194,7 @@ meson_options_help() { printf "%s\n" ' spice-protocol Spice protocol support' printf "%s\n" ' stack-protector compiler-provided stack protection' printf "%s\n" ' tcg TCG support' + printf "%s\n" ' tests build test suite' printf "%s\n" ' tools build support utilities that come with QEMU' printf "%s\n" ' tpm TPM support' printf "%s\n" ' u2f U2F emulation support'
scripts/meson-buildoptions.sh _meson_option_parse() { · line 523
Quoted 8 lines
scripts/meson-buildoptions.sh +2
-0@@ -515,6 +516,8 @@ _meson_option_parse() { --enable-tcg-interpreter) printf "%s" -Dtcg_interpreter=true ;; --disable-tcg-interpreter) printf "%s" -Dtcg_interpreter=false ;; --tls-priority=*) quote_sh "-Dtls_priority=$2" ;; + --enable-tests) printf "%s" -Dtests=enabled ;; + --disable-tests) printf "%s" -Dtests=disabled ;; --enable-tools) printf "%s" -Dtools=enabled ;; --disable-tools) printf "%s" -Dtools=disabled ;; --enable-tpm) printf "%s" -Dtpm=enabled ;;
That's a great addition, including just for speeding up normal builds.
If we want to overengineer the thing, it could be possible to still declare all tests, but not build them by default. However, it's probably too error prone and much less simple than this patch. So I don't think it's a good idea.
Regards,
Pierrick
Quoted 87 lines · Pierrick Bouvier
Quoted 72 lines · Florian Schmidt
There are situations in which you might want to build QEMU without building the full test suite, which in some configurations can take a considerable amount of time to build. This is especially true when using LTO with clang. For example:
$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto-objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target-list=x86_64-softmmu [...] $ time make -j8 [...] [3078/3078] Linking target tests/qtest/qos-test real 6m43.250s user 101m15.967s sys 4m3.813s$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto-objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target-list=x86_64-softmmu --disable-tests [...] $ time make -j8 [...] [2024/2024] Linking target qemu-system-x86_64real 3m14.277s user 33m13.174s sys 1m36.642sAdd a toggle to optionally disable building tests.
Signed-off-by Florian Schmidt <[email protected]>--- meson.build | 2 +- meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 3 files changed, 6 insertions(+), 1 deletion(-)meson.build +1
-1@@ -4605,7 +4605,7 @@ subdir('docs') subdir('pyvenv') # Tests are disabled on emscripten because they rely on host features that aren't # supported by emscripten (e.g. fork and unix socket). -if host_os != 'emscripten' +if get_option('tests').allowed() and host_os != 'emscripten' subdir('tests') endif if gtk.found()meson_options.txt +2
-0@@ -43,6 +43,8 @@ option('gdb', type: 'string', value: '', # on the configure script command line. After adding an option # here make sure to run "make update-buildoptions". +option('tests', type: 'feature', value: 'auto', + description: 'Build the test suite') option('docs', type : 'feature', value : 'auto', description: 'Documentations build support') option('fuzzing', type : 'boolean', value: false,scripts/meson-buildoptions.sh +1
-0@@ -194,6 +194,7 @@ meson_options_help() { printf "%s\n" ' spice-protocol Spice protocol support' printf "%s\n" ' stack-protector compiler-provided stack protection' printf "%s\n" ' tcg TCG support' + printf "%s\n" ' tests build test suite' printf "%s\n" ' tools build support utilities that come with QEMU' printf "%s\n" ' tpm TPM support' printf "%s\n" ' u2f U2F emulation support'scripts/meson-buildoptions.sh _meson_option_parse() { · line 523
Quoted 8 lines
scripts/meson-buildoptions.sh +2
-0@@ -515,6 +516,8 @@ _meson_option_parse() { --enable-tcg-interpreter) printf "%s" -Dtcg_interpreter=true ;; --disable-tcg-interpreter) printf "%s" -Dtcg_interpreter=false ;; --tls-priority=*) quote_sh "-Dtls_priority=$2" ;; + --enable-tests) printf "%s" -Dtests=enabled ;; + --disable-tests) printf "%s" -Dtests=disabled ;; --enable-tools) printf "%s" -Dtools=enabled ;; --disable-tools) printf "%s" -Dtools=disabled ;; --enable-tpm) printf "%s" -Dtpm=enabled ;;That's a great addition, including just for speeding up normal builds.
Reviewed-by Pierrick Bouvier <[email protected]>Tested-by Pierrick Bouvier <[email protected]>
Quoted 8 lines
If we want to overengineer the thing, it could be possible to still declare all tests, but not build them by default. However, it's probably too error prone and much less simple than this patch. So I don't think it's a good idea.
Regards,
Pierrick
When is it useful to build without the provided test suite? When another one is better? Why not contribute it then?
Or is is just that you don't want to test your changes?
The problem might be we build the tests by default but don't run them by default.
Quoted 4 lines · Philippe Mathieu-Daudé
When is it useful to build without the provided test suite? When another one is better? Why not contribute it then?
Or is is just that you don't want to test your changes?
In our case, we have pipelines that rebuild/repackage QEMU after a full build+test, without running the test suite again. In those cases, building the tests is wasted effort, and considerably increases the overall build times.
Quoted 2 lines
The problem might be we build the tests by default but don't run them by default.
That's fair, but I would argue that in that case, there should still be some way to build the binaries without building and running the tests.
Cheers,
Florian
Quoted 10 lines · Florian Schmidt
Quoted 4 lines · Philippe Mathieu-Daudé
When is it useful to build without the provided test suite? When another one is better? Why not contribute it then?
Or is is just that you don't want to test your changes?
In our case, we have pipelines that rebuild/repackage QEMU after a full build+test, without running the test suite again. In those cases, building the tests is wasted effort, and considerably increases the overall build times.
I would suggest that building a second time is wasted effort too. If the 2nd build is byte-for-byte identical to the 1st build, then it is pointless. If it is not byte-for-byte identical, then you'll be taking binaries that have not been tested.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com ~~ https://hachyderm.io/@berrange :| |: https://libvirt.org ~~ https://entangle-photo.org :| |: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
Quoted 4 lines · Daniel P. Berrangé
I would suggest that building a second time is wasted effort too. If the 2nd build is byte-for-byte identical to the 1st build, then it is pointless. If it is not byte-for-byte identical, then you'll be taking binaries that have not been tested.
You're absolutely right. They are bit-to-bit-identical, and it is wasted effort, too. That is something we'll look at internally.
However, it felt like a genuinely useful feature to split build and test phase more strongly, by having the ability to only build the product binaries if desired.
There are other options, of course. A new build target such as "make bin" or similar could only build the product binaries, and a following "make test" would then build + run the tests. This could be combined with Philippe's idea to run (and not only build) the tests by default, though I'm always wary of changing the behaviour of default targets. That's obviously a maintainers' decision.
Cheers,
Florian
Quoted 92 lines · Philippe Mathieu-Daudé
Quoted 91 lines · Pierrick Bouvier
Quoted 71 lines · Florian Schmidt
There are situations in which you might want to build QEMU without building the full test suite, which in some configurations can take a considerable amount of time to build. This is especially true when using LTO with clang. For example:
$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto-objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target-list=x86_64-softmmu [...] $ time make -j8 [...] [3078/3078] Linking target tests/qtest/qos-test real 6m43.250s user 101m15.967s sys 4m3.813s$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto-objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target-list=x86_64- softmmu --disable-tests [...] $ time make -j8 [...] [2024/2024] Linking target qemu-system-x86_64real 3m14.277s
user 33m13.174s
sys 1m36.642sAdd a toggle to optionally disable building tests.
Signed-off-by Florian Schmidt <[email protected]>---
meson.build | 2 +-
meson_options.txt | 2 ++
scripts/meson-buildoptions.sh| 3 +++3 files changed, 6 insertions(+), 1 deletion(-)meson.build +1
-1@@ -4605,7 +4605,7 @@ subdir('docs') subdir('pyvenv') # Tests are disabled on emscripten because they rely on host features that aren't # supported by emscripten (e.g. fork and unix socket). -if host_os != 'emscripten' +if get_option('tests').allowed() and host_os != 'emscripten' subdir('tests')endif
ifgtk.found()meson_options.txt +2
-0@@ -43,6 +43,8 @@ option('gdb', type: 'string', value: '', # on the configure script command line. After adding an option # here make sure to run "make update-buildoptions". +option('tests', type: 'feature', value: 'auto', + description: 'Build the test suite') option('docs', type : 'feature', value : 'auto', description: 'Documentations build support') option('fuzzing', type : 'boolean', value: false,scripts/meson-buildoptions.sh +0
-0@@ -194,6 +194,7 @@ meson_options_help() {printf "%s\n" ' spice-protocol Spice protocol support'
printf "%s\n" ' stack-protector compiler-provided stack protection' printf "%s\n" ' tcg TCG support'
+ printf "%s\n" ' tests build test suite'
printf "%s\n" ' tools build support utilities that come with QEMU' printf "%s\n" ' tpm TPM support'
printf "%s\n" ' u2f U2F emulation support'scripts/meson-buildoptions.sh _meson_option_parse() { · line 523
Quoted 8 lines
scripts/meson-buildoptions.sh +2
-0@@ -515,6 +516,8 @@ _meson_option_parse() { --enable-tcg-interpreter) printf "%s" -Dtcg_interpreter=true ;; --disable-tcg-interpreter) printf "%s" -Dtcg_interpreter=false ;; --tls-priority=*) quote_sh "-Dtls_priority=$2" ;; + --enable-tests) printf "%s" -Dtests=enabled ;; + --disable-tests) printf "%s" -Dtests=disabled ;; --enable-tools) printf "%s" -Dtools=enabled ;; --disable-tools) printf "%s" -Dtools=disabled ;; --enable-tpm) printf "%s" -Dtpm=enabled ;;That's a great addition, including just for speeding up normal builds.
Reviewed-by Pierrick Bouvier <[email protected]>Tested-by Pierrick Bouvier <[email protected]>If we want to overengineer the thing, it could be possible to still declare all tests, but not build them by default. However, it's probably too error prone and much less simple than this patch. So I don't think it's a good idea.
Quoted 5 lines
Quoted 3 lines
Regards,
PierrickWhen is it useful to build without the provided test suite?
I sometimes wished indeed for a --disable-tests switch in the past already (was just too lazy to contribute a patch): I'm sometimes building QEMU in a separate directory, just for a very special case like testing a patch with an --enable-asan build, or for doing a "git bisect". In such cases, I never want to run the normal tests since it's simply not necessary. I run the tests from my main build directory instead once I have a final patch and want to contribute it to upstream. So IMHO this patch is a good idea.
Thomas
Quoted 94 lines · Thomas Huth
Quoted 93 lines · Philippe Mathieu-Daudé
Quoted 92 lines · Pierrick Bouvier
Quoted 72 lines · Florian Schmidt
There are situations in which you might want to build QEMU without building the full test suite, which in some configurations can take a considerable amount of time to build. This is especially true when using LTO with clang. For example:
$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto-objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target-list=x86_64-softmmu [...] $ time make -j8 [...] [3078/3078] Linking target tests/qtest/qos-test real 6m43.250s user 101m15.967s sys 4m3.813s$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto-objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target-list=x86_64- softmmu --disable-tests [...] $ time make -j8 [...] [2024/2024] Linking target qemu-system-x86_64real 3m14.277s
user 33m13.174s
sys 1m36.642sAdd a toggle to optionally disable building tests.
Signed-off-by Florian Schmidt <[email protected]>---
meson.build | 2 +-
meson_options.txt | 2 ++
scripts/meson-buildoptions.sh| 3 +++3 files changed, 6 insertions(+), 1 deletion(-)meson.build +1
-1@@ -4605,7 +4605,7 @@ subdir('docs') subdir('pyvenv') # Tests are disabled on emscripten because they rely on host features that aren't # supported by emscripten (e.g. fork and unix socket). -if host_os != 'emscripten' +if get_option('tests').allowed() and host_os != 'emscripten' subdir('tests')endif
ifgtk.found()meson_options.txt +2
-0@@ -43,6 +43,8 @@ option('gdb', type: 'string', value: '', # on the configure script command line. After adding an option # here make sure to run "make update-buildoptions". +option('tests', type: 'feature', value: 'auto', + description: 'Build the test suite') option('docs', type : 'feature', value : 'auto', description: 'Documentations build support') option('fuzzing', type : 'boolean', value: false,scripts/meson-buildoptions.sh +0
-0@@ -194,6 +194,7 @@ meson_options_help() {printf "%s\n" ' spice-protocol Spice protocol support'
printf "%s\n" ' stack-protector compiler-provided stack protection' printf "%s\n" ' tcg TCG support'
+ printf "%s\n" ' tests build test suite'
printf "%s\n" ' tools build support utilities that come with QEMU' printf "%s\n" ' tpm TPM support'
printf "%s\n" ' u2f U2F emulation support'scripts/meson-buildoptions.sh _meson_option_parse() { · line 523
Quoted 8 lines
scripts/meson-buildoptions.sh +2
-0@@ -515,6 +516,8 @@ _meson_option_parse() { --enable-tcg-interpreter) printf "%s" -Dtcg_interpreter=true ;; --disable-tcg-interpreter) printf "%s" -Dtcg_interpreter=false ;; --tls-priority=*) quote_sh "-Dtls_priority=$2" ;; + --enable-tests) printf "%s" -Dtests=enabled ;; + --disable-tests) printf "%s" -Dtests=disabled ;; --enable-tools) printf "%s" -Dtools=enabled ;; --disable-tools) printf "%s" -Dtools=disabled ;; --enable-tpm) printf "%s" -Dtpm=enabled ;;That's a great addition, including just for speeding up normal builds.
Reviewed-by Pierrick Bouvier <[email protected]>Tested-by Pierrick Bouvier <[email protected]>If we want to overengineer the thing, it could be possible to still declare all tests, but not build them by default. However, it's probably too error prone and much less simple than this patch. So I don't think it's a good idea.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Regards,
PierrickWhen is it useful to build without the provided test suite?
I sometimes wished indeed for a --disable-tests switch in the past already (was just too lazy to contribute a patch): I'm sometimes building QEMU in a separate directory, just for a very special case like testing a patch with an --enable-asan build, or for doing a "git bisect". In such cases, I never want
to run the normal tests since it's simply not necessary. I run the tests from my main build directory instead once I have a final patch and want to contribute it to upstream. So IMHO this patch is a good idea.
I similarly have a local patch for this for such usage, that I never cleaned up to submit but I think it's a good idea.
Regards,
BALATON Zoltan
Quoted 94 lines · BALATON Zoltan
Quoted 93 lines · Thomas Huth
Quoted 92 lines · Philippe Mathieu-Daudé
Quoted 91 lines · Pierrick Bouvier
Quoted 71 lines · Florian Schmidt
There are situations in which you might want to build QEMU without building the full test suite, which in some configurations can take a considerable amount of time to build. This is especially true when using LTO with clang. For example:
$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto- objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target- list=x86_64-softmmu [...] $ time make -j8 [...] [3078/3078] Linking target tests/qtest/qos-test real 6m43.250s user 101m15.967s sys 4m3.813s$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto- objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target- list=x86_64- softmmu --disable-tests [...] $ time make -j8 [...] [2024/2024] Linking target qemu-system-x86_64real 3m14.277s
user 33m13.174s
sys 1m36.642sAdd a toggle to optionally disable building tests.
Signed-off-by Florian Schmidt <[email protected]>---
meson.build | 2 +-
meson_options.txt | 2 ++
scripts/meson-buildoptions.sh| 3 +++3 files changed, 6 insertions(+), 1 deletion(-)meson.build +1
-1@@ -4605,7 +4605,7 @@ subdir('docs') subdir('pyvenv') # Tests are disabled on emscripten because they rely on host features that aren't # supported by emscripten (e.g. fork and unix socket). -if host_os != 'emscripten' +if get_option('tests').allowed() and host_os != 'emscripten' subdir('tests')endif
ifgtk.found()meson_options.txt +2
-0@@ -43,6 +43,8 @@ option('gdb', type: 'string', value: '', # on the configure script command line. After adding an option # here make sure to run "make update-buildoptions". +option('tests', type: 'feature', value: 'auto', + description: 'Build the test suite') option('docs', type : 'feature', value : 'auto', description: 'Documentations build support') option('fuzzing', type : 'boolean', value: false,scripts/meson-buildoptions.sh +0
-0@@ -194,6 +194,7 @@ meson_options_help() {printf "%s\n" ' spice-protocol Spice protocol support'
printf "%s\n" ' stack-protector compiler-provided stack protection' printf "%s\n" ' tcg TCG support'
+ printf "%s\n" ' tests build test suite'
printf "%s\n" ' tools build support utilities that come with QEMU' printf "%s\n" ' tpm TPM support'
printf "%s\n" ' u2f U2F emulation support'scripts/meson-buildoptions.sh _meson_option_parse() { · line 523
Quoted 8 lines
scripts/meson-buildoptions.sh +2
-0@@ -515,6 +516,8 @@ _meson_option_parse() { --enable-tcg-interpreter) printf "%s" -Dtcg_interpreter=true ;; --disable-tcg-interpreter) printf "%s" - Dtcg_interpreter=false ;; --tls-priority=*) quote_sh "-Dtls_priority=$2" ;; + --enable-tests) printf "%s" -Dtests=enabled ;; + --disable-tests) printf "%s" -Dtests=disabled ;; --enable-tools) printf "%s" -Dtools=enabled ;; --disable-tools) printf "%s" -Dtools=disabled ;; --enable-tpm) printf "%s" -Dtpm=enabled ;;That's a great addition, including just for speeding up normal builds.
Reviewed-by Pierrick Bouvier <[email protected]>Tested-by Pierrick Bouvier <[email protected]>If we want to overengineer the thing, it could be possible to still declare all tests, but not build them by default. However, it's probably too error prone and much less simple than this patch. So I don't think it's a good idea.
Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines
Regards,
PierrickWhen is it useful to build without the provided test suite?
I sometimes wished indeed for a --disable-tests switch in the past already (was just too lazy to contribute a patch): I'm sometimes building QEMU in a separate directory, just for a very special case like testing a patch with an --enable-asan build, or for doing a "git bisect". In such cases, I never want to run the normal tests since it's simply not necessary. I run the tests from my main build directory instead once I have a final patch and want to contribute it to upstream. So IMHO this patch is a good idea.
I similarly have a local patch for this for such usage, that I never cleaned up to submit but I think it's a good idea.
I'm not saying this patch is a bad idea, I'll likely use it too; I want to clarify what are the valid use cases the community sees here. Building pointless things clearly has a negative impact on resources and our time, but not testing changes also has. IOW in somes cases this change is acceptable, but I'm a little wary on massive use by default.
Anyway I didn't wanted to start yet another endless discussion so I won't intervene further in this thread, sorry for the usual noise.
Quoted 120 lines · Philippe Mathieu-Daudé
Quoted 119 lines · BALATON Zoltan
Quoted 117 lines · Thomas Huth
Quoted 103 lines · Philippe Mathieu-Daudé
Quoted 102 lines · Pierrick Bouvier
Quoted 81 lines · Florian Schmidt
There are situations in which you might want to build QEMU without building the full test suite, which in some configurations can take a considerable amount of time to build. This is especially true when using
LTO with clang. For example:$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto- objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target- list=x86_64-softmmu [...] $ time make -j8 [...] [3078/3078] Linking target tests/qtest/qos-test real 6m43.250s user 101m15.967s sys 4m3.813s$ ../configure --cc=clang '--extra-ldflags=-flto=thin -ffat-lto- objects' '--extra-cflags=-flto=thin -ffat-lto-objects' --target- list=x86_64- softmmu --disable-tests [...] $ time make -j8 [...] [2024/2024] Linking target qemu-system-x86_64real 3m14.277s
user 33m13.174s
sys 1m36.642sAdd a toggle to optionally disable building tests.
Signed-off-by Florian Schmidt <[email protected]>---
meson.build | 2 +-
meson_options.txt | 2 ++
scripts/meson-buildoptions.sh| 3 +++3 files changed, 6 insertions(+), 1 deletion(-)meson.build +0
-0@@ -4605,7 +4605,7 @@ subdir('docs') subdir('pyvenv') # Tests are disabled on emscripten because they rely on hostfeatures that aren't
# supported by emscripten (e.g. fork and unix socket).meson.build +1
-1-if host_os != 'emscripten' +if get_option('tests').allowed() and host_os != 'emscripten'subdir('tests')
endif
ifgtk.found()meson_options.txt +2
-0@@ -43,6 +43,8 @@ option('gdb', type: 'string', value: '', # on the configure script command line. After adding an option # here make sure to run "make update-buildoptions". +option('tests', type: 'feature', value: 'auto', + description: 'Build the test suite') option('docs', type : 'feature', value : 'auto', description: 'Documentations build support') option('fuzzing', type : 'boolean', value: false,scripts/meson- renamed from scripts/meson-buildoptions.sh +0
-0
buildoptions.sh
indexc003985047..3fec13a336100644scripts/meson-buildoptions.sh +0
-0@@ -194,6 +194,7 @@ meson_options_help() {printf "%s\n" ' spice-protocol Spice protocol support' printf "%s\n" ' stack-protector compiler-provided stack protection'
printf "%s\n" ' tcg TCG support'
+ printf "%s\n" ' tests build test suite' printf "%s\n" ' tools build support utilities that come with QEMU'
printf "%s\n" ' tpm TPM support' printf "%s\n" ' u2f U2F emulation support'scripts/meson-buildoptions.sh +0
-0@@ -515,6 +516,8 @@ _meson_option_parse() { --enable-tcg-interpreter) printf "%s" -Dtcg_interpreter=true ;;Quoted 8 lines
--disable-tcg-interpreter) printf "%s" - Dtcg_interpreter=false ;;
--tls-priority=*)quote_sh"-Dtls_priority=$2" ;; + --enable-tests) printf "%s" -Dtests=enabled ;; + --disable-tests) printf "%s" -Dtests=disabled ;; --enable-tools) printf "%s" -Dtools=enabled ;; --disable-tools) printf "%s" -Dtools=disabled ;; --enable-tpm) printf "%s" -Dtpm=enabled ;;That's a great addition, including just for speeding up normal builds.
Reviewed-by Pierrick Bouvier <[email protected]>Tested-by Pierrick Bouvier <[email protected]>If we want to overengineer the thing, it could be possible to still declare all tests, but not build them by default. However, it's probably
too error prone and much less simple than this patch. So I don't think it's a good idea.Quoted 5 lines
Quoted 3 lines
Regards,
PierrickWhen is it useful to build without the provided test suite?
I sometimes wished indeed for a --disable-tests switch in the past already (was just too lazy to contribute a patch): I'm sometimes building QEMU in a separate directory, just for a very special case like testing a patch with an --enable-asan build, or for doing a "git bisect". In such cases, I never want to run the normal tests since it's simply not necessary. I run the tests from my main build directory instead once I have a final patch and want to contribute it to upstream. So IMHO this patch is a good idea.
Quoted 7 lines
Quoted 2 lines
I similarly have a local patch for this for such usage, that I never cleaned up to submit but I think it's a good idea.
I'm not saying this patch is a bad idea, I'll likely use it too; I want to clarify what are the valid use cases the community sees here. Building pointless things clearly has a negative impact on resources and our time, but not testing changes also has.
Perhaps we could add something to checkpatch that emits an error if config-status has --disable-tests.
Quoted 5 lines
IOW in somes cases this change is acceptable, but I'm a little wary on massive use by default.
Anyway I didn't wanted to start yet another endless discussion so I won't intervene further in this thread, sorry for the usual noise.
I think we'd rather have your comments and everyone else's, there's no deadline for anything.
Quoted 9 lines · Fabiano Rosas
Quoted 4 lines · Philippe Mathieu-Daudé
I'm not saying this patch is a bad idea, I'll likely use it too; I want to clarify what are the valid use cases the community sees here. Building pointless things clearly has a negative impact on resources and our time, but not testing changes also has.
Perhaps we could add something to checkpatch that emits an error if config-status has --disable-tests.
I think that's probably overkill. People who don't run tests are much more likely to be doing it by simply not running "make check" and "make check-functional" than by using this new --disable-tests configure option :-)
I think if --disable-tests isn't the default then that's enough to mean that it only gets used for the situations where it matters to cut down that build time (like the bisection-test cases etc).
(Also, checkpatch runs in a source tree and can't know about configure options, which are per-build-tree.)
thanks
-- PMM
Quoted 11 lines · Peter Maydell
Quoted 9 lines · Fabiano Rosas
Quoted 4 lines · Philippe Mathieu-Daudé
I'm not saying this patch is a bad idea, I'll likely use it too; I want to clarify what are the valid use cases the community sees here. Building pointless things clearly has a negative impact on resources and our time, but not testing changes also has.
Perhaps we could add something to checkpatch that emits an error if config-status has --disable-tests.
Quoted 8 lines
I think that's probably overkill. People who don't run tests are much more likely to be doing it by simply not running "make check" and "make check-functional" than by using this new --disable-tests configure option :-)
I think if --disable-tests isn't the default then that's enough to mean that it only gets used for the situations where it matters to cut down that build time (like the bisection-test cases etc).
+1 from me:
Add "--disable-tests", but don't make it default.
Those tests just stole lots of my time while running git-bisect.....
Helge