oeqa/runtime/ssh: Rework ssh timeout
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 34 lines · Richard Purdie via lists.openembedded.org
After the changes to improve this test, we keep seeing image testing ssh failures, particularly on mips.
It looks like part of the problem is that on a loaded system, 5s is too short for mips to reliably establish an ssh connection. I've seen logs where it keeps timing out and fails to work, then the debug code successfully uses ssh later after everything else fails.
Change the timings/retries to give slow platforms enough time to respond.
Signed-off-by Richard Purdie <[email protected]>---
meta/lib/oeqa/runtime/cases/ssh.py | 4 ++--1 file changed, 2 insertions(+), 2 deletions(-)meta/lib/oeqa/runtime/cases/ssh.py +0
-0@@ -16,7 +16,7 @@ class SSHTest(OERuntimeTestCase): @OETestDepends(['ping.PingTest.test_ping']) @OEHasPackage(['dropbear', 'openssh-sshd'])def
test_ssh(self):
- for i in range(20):
+ for i in range(5):
status, output =self.target.run("uname -a", timeout=5)if status == 0:
breakmeta/lib/oeqa/runtime/cases/ssh.py +1
-1@@ -29,7 +29,7 @@ class SSHTest(OERuntimeTestCase): # give it time for the port to open. # We sometimes see -15 (SIGTERM) on slow emulation machines too, likely # from boot/init not being 100% complete, retry for these too. - time.sleep(5) + time.sleep(30)
Quoted 3 lines
continue
else:
self.fail("uname failed with \"%s\" (exit code %s)" % (output, status))
The ssh test changes and resulting failures is making me quite grumpy. We took code which was working reliably, changed to make it much nicer and in the process made the autobuilder builds unstable.
I've mentioned it several times and whilst people have looked at and said "don't know" or "it shouldn't do that", nobody is actually picking up the issue and resolving it.
I could have reverted it but I agree the new code is nicer. I'd have been asked to "prove" it was the commit, or to revert the revert at some point and then we'd be back to the same instability again.
I have looked at a log of one of the failing builds:
https://valkyrie.yocto.io/pub/shared-failure-data/log.do_testimage.2960382
from
https://valkyrie.yoctoproject.org/#/builders/32/builds/198
and concluded it is simply just slower than the 5s being allowed. Hopefully this patch fixes the issue. If it does not, I would appreciate some help in resolving these failures.
Cheers,
Richard