Skip to contentSkip to conversation

ruby: workaround ptest hang problem

PATCH Superseded [email protected] 2 participants 4 messages Nov 5, 2021 – Nov 8, 2021

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 ruby: workaround ptest hang problem

From: Changqing Li <[email protected]>

since openssl 3 not compatible problem, ruby have disable openssl extention. But disable openssl extention make test_smtp.rs hang at test case "test_start".

Net::TestSMTP#test_start:
NameError: uninitialized constant Net::SMTP::OpenSSL
Did you mean? Open3 /usr/lib64/ruby/3.0.0/net/smtp.rb:195:in default_ssl_context' /usr/lib64/ruby/3.0.0/net/smtp.rb:552:in start' /usr/lib64/ruby/3.0.0/net/smtp.rb:475:in start' /usr/lib64/ruby/ptest/test/net/smtp/test_smtp.rb:199:in test_start'

temporarily remove the hang case to make other testcases can be run.

Meantime, move ruby-ptest out of the PTESTS_PROBLEMS list. On 48 core host, run ruby ptest in qemux86-64:

root@qemux86-64:/usr/lib64/ruby/ptest# time ./run-ptest
PASS: test/test_set.rb
PASS: test/stringio/test_stringio.rb
...
PASS: test/did_you_mean/test_tree_spell_checker.rb
PASS: test/test_mutex_m.rb

real 5m42.872s
user 3m50.923s
sys 0m44.136s

---
 meta/conf/distro/include/ptest-packagelists.inc | 3 +--
 meta/recipes-devtools/ruby/ruby.inc             | 4 ++++
 2 files changed, 5 insertions(+), 2 deletions(-)
meta/conf/distro/include/ptest-packagelists.inc +1 -2
index 2e324f8da4..f1c40ac813 100644
@@ -81,6 +81,7 @@ PTESTS_SLOW = "\
     openssl-ptest \
     perl-ptest \
     python3-ptest \
+    ruby-ptest \
     strace-ptest \
     tcl-ptest \
     util-linux-ptest \
@@ -90,7 +91,6 @@ PTESTS_SLOW = "\
 PTESTS_SLOW:remove:riscv64 = "valgrind-ptest"
 PTESTS_PROBLEMS:append:riscv64 = "valgrind-ptest"
 
-#    ruby-ptest \ # Timeout
 #    lz4-ptest \ # Needs a rewrite
 #    rt-tests-ptest \ # Needs to be checked whether it runs at all
 #    bash-ptest \ # Test outcomes are non-deterministic by design
@@ -103,7 +103,6 @@ PTESTS_PROBLEMS:append:riscv64 = "valgrind-ptest"
 #    python3-numpy-ptest \ # requires even more RAM and (possibly) disk space; multiple failures
 
 PTESTS_PROBLEMS = "\
-    ruby-ptest \
     lz4-ptest \
     rt-tests-ptest \
     bash-ptest \
meta/recipes-devtools/ruby/ruby.inc +4 -0
index 2b5caf35f5..776c409f6c 100644
@@ -42,3 +42,7 @@ do_configure:prepend() {
 do_configure:prepend() {
     rm -rf ${S}/ext/openssl/extconf.rb
 }
+# Remove this test temporarily until it compatible with openssl 3
+do_install_ptest:append() {
+    rm -rf ${D}${PTEST_PATH}/test/net/smtp/test_smtp.rb
+}
Signature · 1 line
2.32.0
#1 lore raw
Quoted 29 lines · Changqing Li

From: Changqing Li <[email protected]>

since openssl 3 not compatible problem, ruby have disable openssl extention. But disable openssl extention make test_smtp.rs hang at test case "test_start".

Net::TestSMTP#test_start:
NameError: uninitialized constant Net::SMTP::OpenSSL
Did you mean? Open3 /usr/lib64/ruby/3.0.0/net/smtp.rb:195:in default_ssl_context' /usr/lib64/ruby/3.0.0/net/smtp.rb:552:in start' /usr/lib64/ruby/3.0.0/net/smtp.rb:475:in start' /usr/lib64/ruby/ptest/test/net/smtp/test_smtp.rb:199:in test_start'

temporarily remove the hang case to make other testcases can be run.

Meantime, move ruby-ptest out of the PTESTS_PROBLEMS list. On 48 core host, run ruby ptest in qemux86-64:

root@qemux86-64:/usr/lib64/ruby/ptest# time ./run-ptest
PASS: test/test_set.rb
PASS: test/stringio/test_stringio.rb
...
PASS: test/did_you_mean/test_tree_spell_checker.rb
PASS: test/test_mutex_m.rb

real 5m42.872s
user 3m50.923s
sys 0m44.136s

Quoted 5 lines
---
 meta/conf/distro/include/ptest-packagelists.inc | 3 +--
 meta/recipes-devtools/ruby/ruby.inc             | 4 ++++
 2 files changed, 5 insertions(+), 2 deletions(-)

Unfortunately the success rate isn't like that when we tried this on the autobuilder, it failed on both arm64 and x86_64 with multiple failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2791
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/2507

Cheers,

Richard

#2 lore raw
Quoted 37 lines · Richard Purdie

[Please note: This e-mail is from an EXTERNAL e-mail address]

Quoted 29 lines · Changqing Li

From: Changqing Li <[email protected]>

since openssl 3 not compatible problem, ruby have disable openssl extention. But disable openssl extention make test_smtp.rs hang at test case "test_start".

Net::TestSMTP#test_start:
NameError: uninitialized constant Net::SMTP::OpenSSL
Did you mean? Open3 /usr/lib64/ruby/3.0.0/net/smtp.rb:195:in default_ssl_context' /usr/lib64/ruby/3.0.0/net/smtp.rb:552:in start' /usr/lib64/ruby/3.0.0/net/smtp.rb:475:in start' /usr/lib64/ruby/ptest/test/net/smtp/test_smtp.rb:199:in test_start'

temporarily remove the hang case to make other testcases can be run.

Meantime, move ruby-ptest out of the PTESTS_PROBLEMS list. On 48 core host, run ruby ptest in qemux86-64:

root@qemux86-64:/usr/lib64/ruby/ptest# time ./run-ptest
PASS: test/test_set.rb
PASS: test/stringio/test_stringio.rb
...
PASS: test/did_you_mean/test_tree_spell_checker.rb
PASS: test/test_mutex_m.rb
real  5m42.872s
user  3m50.923s
sys   0m44.136s
Quoted 5 lines
---
  meta/conf/distro/include/ptest-packagelists.inc | 3 +--
  meta/recipes-devtools/ruby/ruby.inc             | 4 ++++
  2 files changed, 5 insertions(+), 2 deletions(-)
Quoted 5 lines

Unfortunately the success rate isn't like that when we tried this on the autobuilder, it failed on both arm64 and x86_64 with multiple failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2791
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/2507

Sorry,  I thought this is expected.  Ruby have 800+ test cases, and failed about

30+ all the time.  so I think we still need to put ruby ptest in the  PTESTS_PROBLEMS list until

we address all the failed cases.

Quoted 6 lines

Cheers,

Richard

#3 lore raw
Quoted 44 lines · Changqing Li
Quoted 37 lines · Richard Purdie

[Please note: This e-mail is from an EXTERNAL e-mail address]

Quoted 29 lines · Changqing Li

From: Changqing Li <[email protected]>

since openssl 3 not compatible problem, ruby have disable openssl extention. But disable openssl extention make test_smtp.rs hang at test case "test_start".

Net::TestSMTP#test_start:
NameError: uninitialized constant Net::SMTP::OpenSSL
Did you mean? Open3 /usr/lib64/ruby/3.0.0/net/smtp.rb:195:in default_ssl_context' /usr/lib64/ruby/3.0.0/net/smtp.rb:552:in start' /usr/lib64/ruby/3.0.0/net/smtp.rb:475:in start' /usr/lib64/ruby/ptest/test/net/smtp/test_smtp.rb:199:in test_start'

temporarily remove the hang case to make other testcases can be run.

Meantime, move ruby-ptest out of the PTESTS_PROBLEMS list. On 48 core host, run ruby ptest in qemux86-64:

root@qemux86-64:/usr/lib64/ruby/ptest# time ./run-ptest
PASS: test/test_set.rb
PASS: test/stringio/test_stringio.rb
...
PASS: test/did_you_mean/test_tree_spell_checker.rb
PASS: test/test_mutex_m.rb
real  5m42.872s
user  3m50.923s
sys   0m44.136s
Quoted 5 lines
---
  meta/conf/distro/include/ptest-packagelists.inc | 3 +--
  meta/recipes-devtools/ruby/ruby.inc             | 4 ++++
  2 files changed, 5 insertions(+), 2 deletions(-)
Quoted 5 lines

Unfortunately the success rate isn't like that when we tried this on the autobuilder, it failed on both arm64 and x86_64 with multiple failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/2791
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/2507

Quoted 7 lines

Sorry,  I thought this is expected.  Ruby have 800+ test cases, and failed about

30+ all the time.  so I think we still need to put ruby ptest in the PTESTS_PROBLEMS list until

we address all the failed cases.

What we could do is mask out the failing cases. The issue is that we'll see warnings on the autobuilder for cases that fail so we need to remove any "known to fail" cases before we can enable by default there.

Cheers,

Richard