Skip to contentSkip to conversation

tests/tcg: Skip failing ppc64 multi-threaded tests

PATCH In review Nicholas Piggin 3 participants 7 messages Jul 25, 2024 – Jul 26, 2024

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 tests/tcg: Skip failing ppc64 multi-threaded tests

In Gitlab CI, some ppc64 multi-threaded tcg tests crash when run in the clang-user job with an assertion failure in glibc that seems to indicate corruption:

signals: allocatestack.c:223: allocate_stack:
Assertion `powerof2 (pagesize_m1 + 1)' failed.

Disable these tests for now.

---
 tests/tcg/ppc64/Makefile.target | 12 ++++++++++++
 1 file changed, 12 insertions(+)
tests/tcg/ppc64/Makefile.target +12 -0
index 8c3e4e4038..509a20be2b 100644
@@ -11,6 +11,18 @@ config-cc.mak: Makefile
 
 -include config-cc.mak
 
+# multi-threaded tests are known to fail (e.g., clang-user CI job)
+# See: https://gitlab.com/qemu-project/qemu/-/issues/2456
+run-signals: signals
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+run-plugin-signals-with-%:
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+
+run-threadcount: threadcount
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+run-plugin-threadcount-with-%:
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+
 ifneq ($(CROSS_CC_HAS_POWER8_VECTOR),)
 PPC64_TESTS=bcdsub non_signalling_xscv
 endif
Signature · 1 line
2.45.2
#1 lore raw
Quoted 19 lines · Nicholas Piggin

In Gitlab CI, some ppc64 multi-threaded tcg tests crash when run in the clang-user job with an assertion failure in glibc that seems to indicate corruption:

signals: allocatestack.c:223: allocate_stack:
Assertion `powerof2 (pagesize_m1 + 1)' failed.

Disable these tests for now.

---
 tests/tcg/ppc64/Makefile.target | 12 ++++++++++++
 1 file changed, 12 insertions(+)
tests/tcg/ppc64/Makefile.target +0 -0
index 8c3e4e4038..509a20be2b 100644

tests/tcg/ppc64/Makefile.target config-cc.mak: Makefile · line 15

Quoted 5 lines
tests/tcg/ppc64/Makefile.target +2 -0
@@ -11,6 +11,18 @@ config-cc.mak: Makefile
 
 -include config-cc.mak
 
+# multi-threaded tests are known to fail (e.g., clang-user CI job)
+# See: https://gitlab.com/qemu-project/qemu/-/issues/2456

Given this is only a problem with clang can we only apply these workaround if we detect "clang" in $(CC)?

Quoted 13 lines
tests/tcg/ppc64/Makefile.target +10 -0
+run-signals: signals
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+run-plugin-signals-with-%:
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+
+run-threadcount: threadcount
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+run-plugin-threadcount-with-%:
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+
 ifneq ($(CROSS_CC_HAS_POWER8_VECTOR),)
 PPC64_TESTS=bcdsub non_signalling_xscv
 endif
Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro
#2 lore raw
Quoted 22 lines · Alex Bennée
Quoted 20 lines · Nicholas Piggin

In Gitlab CI, some ppc64 multi-threaded tcg tests crash when run in the clang-user job with an assertion failure in glibc that seems to indicate corruption:

signals: allocatestack.c:223: allocate_stack:
Assertion `powerof2 (pagesize_m1 + 1)' failed.

Disable these tests for now.

---
 tests/tcg/ppc64/Makefile.target | 12 ++++++++++++
 1 file changed, 12 insertions(+)
tests/tcg/ppc64/Makefile.target +0 -0
index 8c3e4e4038..509a20be2b 100644
@@ -11,6 +11,18 @@ config-cc.mak: Makefile
 
Quoted 7 lines

tests/tcg/ppc64/Makefile.target config-cc.mak: Makefile · line 15

Quoted 4 lines
tests/tcg/ppc64/Makefile.target +2 -0
 -include config-cc.mak
 
+# multi-threaded tests are known to fail (e.g., clang-user CI job)
+# See: https://gitlab.com/qemu-project/qemu/-/issues/2456

Given this is only a problem with clang can we only apply these workaround if we detect "clang" in $(CC)?

ifeq ($(findstring clang,$(CC)),clang)
...
endif

should do the trick

Quoted 14 lines
Quoted 14 lines
tests/tcg/ppc64/Makefile.target +10 -0
+run-signals: signals
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+run-plugin-signals-with-%:
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+
+run-threadcount: threadcount
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+run-plugin-threadcount-with-%:
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+
 ifneq ($(CROSS_CC_HAS_POWER8_VECTOR),)
 PPC64_TESTS=bcdsub non_signalling_xscv
 endif
Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro
#3 lore raw
Quoted 32 lines · Alex Bennée
Quoted 22 lines · Alex Bennée
Quoted 20 lines · Nicholas Piggin

In Gitlab CI, some ppc64 multi-threaded tcg tests crash when run in the clang-user job with an assertion failure in glibc that seems to indicate corruption:

signals: allocatestack.c:223: allocate_stack:
Assertion `powerof2 (pagesize_m1 + 1)' failed.

Disable these tests for now.

---
 tests/tcg/ppc64/Makefile.target | 12 ++++++++++++
 1 file changed, 12 insertions(+)
tests/tcg/ppc64/Makefile.target +0 -0
index 8c3e4e4038..509a20be2b 100644
@@ -11,6 +11,18 @@ config-cc.mak: Makefile
 
Quoted 7 lines

tests/tcg/ppc64/Makefile.target config-cc.mak: Makefile · line 15

Quoted 4 lines
tests/tcg/ppc64/Makefile.target +2 -0
 -include config-cc.mak
 
+# multi-threaded tests are known to fail (e.g., clang-user CI job)
+# See: https://gitlab.com/qemu-project/qemu/-/issues/2456

Given this is only a problem with clang can we only apply these workaround if we detect "clang" in $(CC)?

Quoted 5 lines

ifeq ($(findstring clang,$(CC)),clang)
...
endif

should do the trick

I did try that, but unfortunately the target CC (ppc64 in this case). I'll just send the big hammer to CI unstuck, and I'll try to work it out later.

Thanks,
Nick

#4 lore raw
In reply to Nicholas Piggin
Quoted 27 lines · Nicholas Piggin

In Gitlab CI, some ppc64 multi-threaded tcg tests crash when run in the clang-user job with an assertion failure in glibc that seems to indicate corruption:

signals: allocatestack.c:223: allocate_stack:
Assertion `powerof2 (pagesize_m1 + 1)' failed.

Disable these tests for now.

---
  tests/tcg/ppc64/Makefile.target | 12 ++++++++++++
  1 file changed, 12 insertions(+)
tests/tcg/ppc64/Makefile.target +5 -0
index 8c3e4e4038..509a20be2b 100644
@@ -11,6 +11,18 @@ config-cc.mak: Makefile
 
  -include config-cc.mak
 
+# multi-threaded tests are known to fail (e.g., clang-user CI job)
+# See: https://gitlab.com/qemu-project/qemu/-/issues/2456
+run-signals: signals
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+run-plugin-signals-with-%:

tests/tcg/ppc64/Makefile.target config-cc.mak: Makefile · line 28

Quoted 10 lines
tests/tcg/ppc64/Makefile.target +7 -0
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+
+run-threadcount: threadcount
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+run-plugin-threadcount-with-%:
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+
  ifneq ($(CROSS_CC_HAS_POWER8_VECTOR),)
  PPC64_TESTS=bcdsub non_signalling_xscv
  endif

Could you please check whether this is already fixed by Richard's patch:

https://gitlab.com/qemu-project/qemu/-/commit/8e466dd092469e5ab0f355775c57

?

Thanks,
Thomas

#5 lore raw
Quoted 41 lines · Thomas Huth
Quoted 27 lines · Nicholas Piggin

In Gitlab CI, some ppc64 multi-threaded tcg tests crash when run in the clang-user job with an assertion failure in glibc that seems to indicate corruption:

signals: allocatestack.c:223: allocate_stack:
Assertion `powerof2 (pagesize_m1 + 1)' failed.

Disable these tests for now.

---
  tests/tcg/ppc64/Makefile.target | 12 ++++++++++++
  1 file changed, 12 insertions(+)
tests/tcg/ppc64/Makefile.target +5 -0
index 8c3e4e4038..509a20be2b 100644
@@ -11,6 +11,18 @@ config-cc.mak: Makefile
 
  -include config-cc.mak
 
+# multi-threaded tests are known to fail (e.g., clang-user CI job)
+# See: https://gitlab.com/qemu-project/qemu/-/issues/2456
+run-signals: signals
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+run-plugin-signals-with-%:

tests/tcg/ppc64/Makefile.target config-cc.mak: Makefile · line 28

Quoted 10 lines
tests/tcg/ppc64/Makefile.target +7 -0
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+
+run-threadcount: threadcount
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+run-plugin-threadcount-with-%:
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+
  ifneq ($(CROSS_CC_HAS_POWER8_VECTOR),)
  PPC64_TESTS=bcdsub non_signalling_xscv
  endif

Could you please check whether this is already fixed by Richard's patch:

Quoted 3 lines

https://gitlab.com/qemu-project/qemu/-/commit/8e466dd092469e5ab0f355775c57

?

No, doesn't seem to unfortunately. Here is the same fail -

https://gitlab.com/npiggin/qemu/-/jobs/7436325582

I did build it with clang and using the same sanitize flags on my local system, and could not reproduce. So not sure.

I might try run the clang-user without any sanitize flags.

I sent this patch with ppc pull request already, I think we just do that for now to get clang-user passing. Simple enough to revert once fixed.

Thanks,
Nick

#6 lore raw
In reply to Thomas Huth
Quoted 41 lines · Thomas Huth
Quoted 27 lines · Nicholas Piggin

In Gitlab CI, some ppc64 multi-threaded tcg tests crash when run in the clang-user job with an assertion failure in glibc that seems to indicate corruption:

signals: allocatestack.c:223: allocate_stack:
Assertion `powerof2 (pagesize_m1 + 1)' failed.

Disable these tests for now.

---
  tests/tcg/ppc64/Makefile.target | 12 ++++++++++++
  1 file changed, 12 insertions(+)
tests/tcg/ppc64/Makefile.target +5 -0
index 8c3e4e4038..509a20be2b 100644
@@ -11,6 +11,18 @@ config-cc.mak: Makefile
 
  -include config-cc.mak
 
+# multi-threaded tests are known to fail (e.g., clang-user CI job)
+# See: https://gitlab.com/qemu-project/qemu/-/issues/2456
+run-signals: signals
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+run-plugin-signals-with-%:

tests/tcg/ppc64/Makefile.target config-cc.mak: Makefile · line 28

Quoted 10 lines
tests/tcg/ppc64/Makefile.target +7 -0
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+
+run-threadcount: threadcount
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+run-plugin-threadcount-with-%:
+	$(call skip-test, $<, "BROKEN (flaky with clang) ")
+
  ifneq ($(CROSS_CC_HAS_POWER8_VECTOR),)
  PPC64_TESTS=bcdsub non_signalling_xscv
  endif

Could you please check whether this is already fixed by Richard's patch:

Quoted 1 line

https://gitlab.com/qemu-project/qemu/-/commit/8e466dd092469e5ab0f355775c57

Okay removing the sanitizing entirely fixes it, e.g., this patch

https://gitlab.com/npiggin/qemu/-/commit/6160a7dd834b2d0e7bb08f13f709693ffa7c8d06

Result:

https://gitlab.com/npiggin/qemu/-/jobs/7436997610

Thanks,
Nick