gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job
PATCH 1/1 gitlab-ci.yml: Run check-qtest and check-unit at the end of the fuzzer job
The fuzzer job finishes quite early, so we can run the unit tests and qtests with -fsanitize=address here without extending the total test time.
--- .gitlab-ci.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
.gitlab-ci.yml +3 -2
@@ -256,13 +256,14 @@ build-oss-fuzz: - mkdir build-oss-fuzz - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address" ./scripts/oss-fuzz/build.sh + - export ASAN_OPTIONS="fast_unwind_on_malloc=0" - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable -type f | grep -v slirp); do grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 || continue ; echo Testing ${fuzzer} ... ; - ASAN_OPTIONS="fast_unwind_on_malloc=0" - "${fuzzer}" -runs=1000 -seed=1 || exit 1 ; + "${fuzzer}" -runs=1000 -seed=1 || exit 1 ; done + - cd build-oss-fuzz && make check-qtest-i386 check-unit build-tci: <<: *native_build_job_definition
Signature · 1 line
2.18.2
Quoted 3 lines · Thomas Huth
The fuzzer job finishes quite early, so we can run the unit tests and qtests with -fsanitize=address here without extending the total test time.
Sounds good to me, though its too bad that this is limited to i386 and we aren't using the --enable-sanitizers configure argument due to the strange oss-fuzz build script.
Speaking of testing with sanitizers, does it make sense to have a job that does check-qtest with --enable-tsan, now that we have ThreadSanitizer support?
Can --enable-sanitizers --enable-tsan be added to some existing job, without severely increasing the pipeline's exeuction time?
Quoted 29 lines
Signed-off-by Thomas Huth <[email protected]>--- .gitlab-ci.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-).gitlab-ci.yml +3
-2@@ -256,13 +256,14 @@ build-oss-fuzz: - mkdir build-oss-fuzz - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address" ./scripts/oss-fuzz/build.sh + - export ASAN_OPTIONS="fast_unwind_on_malloc=0" - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable -type f | grep -v slirp); do grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 || continue ; echo Testing ${fuzzer} ... ; - ASAN_OPTIONS="fast_unwind_on_malloc=0" - "${fuzzer}" -runs=1000 -seed=1 || exit 1 ; + "${fuzzer}" -runs=1000 -seed=1 || exit 1 ; done + - cd build-oss-fuzz && make check-qtest-i386 check-unit build-tci: <<: *native_build_job_definitionSignature · 1 line
2.18.2
Quoted 9 lines · Alexander Bulekov
Quoted 3 lines · Thomas Huth
The fuzzer job finishes quite early, so we can run the unit tests and qtests with -fsanitize=address here without extending the total test time.
Sounds good to me, though its too bad that this is limited to i386 and we aren't using the --enable-sanitizers configure argument due to the strange oss-fuzz build script.
Quoted 6 lines
Speaking of testing with sanitizers, does it make sense to have a job that does check-qtest with --enable-tsan, now that we have ThreadSanitizer support?
Can --enable-sanitizers --enable-tsan be added to some existing job, without severely increasing the pipeline's exeuction time?
Good idea, I'll have a try when I've got some spare time again (probably not this week, though, it's a pretty busy one for me).
Quoted 1 line
Reviewed-by Alexander Bulekov <[email protected]>
Thanks!
Thomas
Hi Thomas,
Quoted 18 lines · Thomas Huth
The fuzzer job finishes quite early, so we can run the unit tests and qtests with -fsanitize=address here without extending the total test time.
Signed-off-by Thomas Huth <[email protected]>--- .gitlab-ci.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-).gitlab-ci.yml +1
-0@@ -256,13 +256,14 @@ build-oss-fuzz: - mkdir build-oss-fuzz - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address" ./scripts/oss-fuzz/build.sh + - export ASAN_OPTIONS="fast_unwind_on_malloc=0" - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable -type f
.gitlab-ci.yml build-oss-fuzz: · line 268
Quoted 10 lines
.gitlab-ci.yml +2
-2| grep -v slirp); do grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 || continue ; echo Testing ${fuzzer} ... ; - ASAN_OPTIONS="fast_unwind_on_malloc=0" - "${fuzzer}" -runs=1000 -seed=1 || exit 1 ; + "${fuzzer}" -runs=1000 -seed=1 || exit 1 ; done + - cd build-oss-fuzz && make check-qtest-i386 check-unit
As this does not use the fuzzer main entry point, what is the point of running that?
Quoted 7 lines
.gitlab-ci.yml +0
-0build-tci:<<:
*native_build_job_definition
--
2.18.2
Quoted 36 lines · Philippe Mathieu-Daudé
Hi Thomas,
Le lun. 31 août 2020 17:33, Thomas Huth <[email protected] <mailto:[email protected]>> a écrit :
The fuzzer job finishes quite early, so we can run the unit tests and qtests with -fsanitize=address here without extending the total test time. Signed-off-by: Thomas Huth <[email protected] <mailto:[email protected]>> --- .gitlab-ci.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 17f1f8fad9..417fda6909 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -256,13 +256,14 @@ build-oss-fuzz: - mkdir build-oss-fuzz - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address" ./scripts/oss-fuzz/build.sh + - export ASAN_OPTIONS="fast_unwind_on_malloc=0" - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable -type f | grep -v slirp); do grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 || continue ; echo Testing ${fuzzer} ... ; - ASAN_OPTIONS="fast_unwind_on_malloc=0" - "${fuzzer}" -runs=1000 -seed=1 || exit 1 ; + "${fuzzer}" -runs=1000 -seed=1 || exit 1 ; done + - cd build-oss-fuzz && make check-qtest-i386 check-unit
Quoted 2 lines
As this does not use the fuzzer main entry point, what is the point of running that?
Read the friendly patch description ;-) - the idea here is that we run the qtests and unit tests with -fsanitize=address in one of the pipelines. We could also use a different task for this, but since the fuzzer tests finish quite early and we already compiled the code with -fsanitize=address here anyway, it's IMHO simply a good fit here.
Thomas
Quoted 38 lines · Thomas Huth
Quoted 36 lines · Philippe Mathieu-Daudé
Hi Thomas,
Le lun. 31 août 2020 17:33, Thomas Huth <[email protected] <mailto:[email protected]>> a écrit :
The fuzzer job finishes quite early, so we can run the unit tests and qtests with -fsanitize=address here without extending the total test time.
Signed-off-by Thomas Huth <[email protected] <mailto:[email protected]>>---
.gitlab-ci.yml | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)diff --git
a/.gitlab-ci.ymlb/.gitlab-ci.yml
index17f1f8fad9..417fda6909100644
---a/.gitlab-ci.yml
+++b/.gitlab-ci.yml
@@ -256,13 +256,14 @@ build-oss-fuzz:
- mkdir build-oss-fuzz
- CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"./scripts/oss-fuzz/build.sh
+ - export ASAN_OPTIONS="fast_unwind_on_malloc=0"
- for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable
-type f
| grep -v slirp); do
grep "LLVMFuzzerTestOneInput" ${fuzzer} >/dev/null2>&1 ||
continue ;
echo Testing ${fuzzer} ... ;
- ASAN_OPTIONS="fast_unwind_on_malloc=0"
- "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
+ "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
done
+ - cd build-oss-fuzz && make check-qtest-i386 check-unit
Quoted 6 lines
Quoted 2 lines
As this does not use the fuzzer main entry point, what is the point of running that?
Read the friendly patch description ;-) - the idea here is that we run the qtests and unit tests with -fsanitize=address in one of the pipelines.
Sorry, the description was not that obvious to me. Can you add a comment before the 'make check*' line?
# Unrelated to fuzzer: run tests with -fsanitize=address
Quoted 7 lines
We could also use a different task for this, but since the fuzzer tests finish quite early and we already compiled the code with -fsanitize=address here anyway, it's IMHO simply a good fit here.
Thomas
Quoted 42 lines · Philippe Mathieu-Daudé
Quoted 41 lines · Thomas Huth
Quoted 38 lines · Philippe Mathieu-Daudé
Hi Thomas,
Le lun. 31 août 2020 17:33, Thomas Huth <[email protected] <mailto:[email protected]>> a écrit :
The fuzzer job finishes quite early, so we can run the unit tests and qtests with -fsanitize=address here without extending the total test time.
Signed-off-by Thomas Huth <[email protected] <mailto:[email protected]>>---
.gitlab-ci.yml | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)diff --git
a/.gitlab-ci.ymlb/.gitlab-ci.yml
index17f1f8fad9..417fda6909100644
---a/.gitlab-ci.yml
+++b/.gitlab-ci.yml
@@ -256,13 +256,14 @@ build-oss-fuzz:
- mkdir build-oss-fuzz
- CC="clang" CXX="clang++" CFLAGS="-fsanitize=address"./scripts/oss-fuzz/build.sh
+ - export ASAN_OPTIONS="fast_unwind_on_malloc=0"
- for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable
-type f
| grep -v slirp); do
grep "LLVMFuzzerTestOneInput" ${fuzzer} >/dev/null2>&1 ||
continue ;
echo Testing ${fuzzer} ... ;
- ASAN_OPTIONS="fast_unwind_on_malloc=0"
- "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
+ "${fuzzer}" -runs=1000 -seed=1 || exit 1 ;
done
+ - cd build-oss-fuzz && make check-qtest-i386 check-unitAs this does not use the fuzzer main entry point, what is the point of running that?
Quoted 8 lines
Quoted 3 lines
Read the friendly patch description ;-) - the idea here is that we run the qtests and unit tests with -fsanitize=address in one of the pipelines.
Sorry, the description was not that obvious to me. Can you add a comment before the 'make check*' line?
# Unrelated to fuzzer: run tests with -fsanitize=address
Sure, will do.
Thomas