tests/avocado/machine_aspeed: Fix the broken ast2[56]00_evb_sdk tests
cover tests/avocado/machine_aspeed: Fix the broken ast2[56]00_evb_sdk tests
Fix the broken ast2500_evb_sdk and ast2600_evb_sdk avocado tests. See the patch description of the second patch for details. Also add the test to the MAINTAINERS file (third patch).
Thomas Huth (3):
tests/avocado: Make ssh_command_output_contains() globally available tests/avocado/machine_aspeed: Fix the broken ast2[56]00_evb_sdk tests
MAINTAINERS: Cover tests/avocado/machine_aspeed.py
MAINTAINERS | 2 +- tests/avocado/avocado_qemu/__init__.py | 8 +++++++ tests/avocado/linux_ssh_mips_malta.py | 8 ------- tests/avocado/machine_aspeed.py | 31 +++++++++++++++----------- 4 files changed, 27 insertions(+), 22 deletions(-)
Signature · 1 line
2.31.1
PATCH 1/3 tests/avocado: Make ssh_command_output_contains() globally available
This function will be useful in other tests, too, so move it to the core LinuxSSHMixIn class.
--- tests/avocado/avocado_qemu/__init__.py | 8 ++++++++ tests/avocado/linux_ssh_mips_malta.py | 8 -------- 2 files changed, 8 insertions(+), 8 deletions(-)
tests/avocado/avocado_qemu/__init__.py +8 -0
@@ -431,6 +431,14 @@ def ssh_command(self, command): f'Guest command failed: {command}') return stdout_lines, stderr_lines + def ssh_command_output_contains(self, cmd, exp): + stdout, _ = self.ssh_command(cmd) + for line in stdout: + if exp in line: + break + else: + self.fail('"%s" output does not contain "%s"' % (cmd, exp)) + class LinuxDistro: """Represents a Linux distribution
tests/avocado/linux_ssh_mips_malta.py +0 -8
@@ -101,14 +101,6 @@ def shutdown_via_ssh(self): self.ssh_disconnect_vm() wait_for_console_pattern(self, 'Power down', 'Oops') - def ssh_command_output_contains(self, cmd, exp): - stdout, _ = self.ssh_command(cmd) - for line in stdout: - if exp in line: - break - else: - self.fail('"%s" output does not contain "%s"' % (cmd, exp)) - def run_common_commands(self, wordsize): self.ssh_command_output_contains( 'cat /proc/cpuinfo',
Signature · 1 line
2.31.1
Re: [PATCH 1/3] tests/avocado: Make ssh_command_output_contains() globally available
Quoted 4 lines · Thomas Huth
This function will be useful in other tests, too, so move it to the core LinuxSSHMixIn class.
Signed-off-by Thomas Huth <[email protected]>
Thanks,
C.
Quoted 43 lines
--- tests/avocado/avocado_qemu/__init__.py | 8 ++++++++ tests/avocado/linux_ssh_mips_malta.py | 8 -------- 2 files changed, 8 insertions(+), 8 deletions(-)tests/avocado/avocado_qemu/__init__.py +8
-0@@ -431,6 +431,14 @@ def ssh_command(self, command): f'Guest command failed: {command}') return stdout_lines, stderr_lines + def ssh_command_output_contains(self, cmd, exp): + stdout, _ = self.ssh_command(cmd) + for line in stdout: + if exp in line: + break + else: + self.fail('"%s" output does not contain "%s"' % (cmd, exp)) + class LinuxDistro: """Represents a Linux distributiontests/avocado/linux_ssh_mips_malta.py +0
-8@@ -101,14 +101,6 @@ def shutdown_via_ssh(self): self.ssh_disconnect_vm() wait_for_console_pattern(self, 'Power down', 'Oops') - def ssh_command_output_contains(self, cmd, exp): - stdout, _ = self.ssh_command(cmd) - for line in stdout: - if exp in line: - break - else: - self.fail('"%s" output does not contain "%s"' % (cmd, exp)) - def run_common_commands(self, wordsize): self.ssh_command_output_contains( 'cat /proc/cpuinfo',
PATCH 2/3 tests/avocado/machine_aspeed: Fix the broken ast2[56]00_evb_sdk tests
In reply to Thomas Huthtest_arm_ast2500_evb_sdk and test_arm_ast2600_evb_sdk are currently failing. The problem is that they are trying to look for the login prompt that does not have a newline at the end - but the logic in _console_interaction() only handles full lines. It used to work by accident in the past since there were sometimes kernel (warning and error) messages popping up that finally provided a newline character in the output, but since the tests have been changed to run with the "quiet" kernel parameter, this is not working anymore.
To make this work reliably, we must not look for the "login:" prompt, but have to use some text ending with a newline instead. And in the ast2600 test, switch to ssh instead of trying to log into the serial console - this works much more reliable and also has the benefit of excercising the network interface here a little bit, too.
--- tests/avocado/machine_aspeed.py | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-)
tests/avocado/machine_aspeed.py +18 -13
@@ -8,6 +8,7 @@ import time import os +from avocado_qemu import LinuxSSHMixIn from avocado_qemu import QemuSystemTest from avocado_qemu import wait_for_console_pattern from avocado_qemu import exec_command
@@ -230,7 +231,7 @@ def test_arm_ast2600_evb_buildroot(self): self.do_test_arm_aspeed_buildroot_poweroff() -class AST2x00MachineSDK(QemuSystemTest): +class AST2x00MachineSDK(QemuSystemTest, LinuxSSHMixIn): EXTRA_BOOTARGS = ( 'quiet '
@@ -257,7 +258,7 @@ def do_test_arm_aspeed_sdk_start(self, image): self.require_netdev('user') self.vm.set_console() self.vm.add_args('-drive', 'file=' + image + ',if=mtd,format=raw', - '-net', 'nic', '-net', 'user') + '-net', 'nic', '-net', 'user,hostfwd=:127.0.0.1:0-:22') self.vm.launch() self.wait_for_console_pattern('U-Boot 2019.04')
@@ -285,7 +286,7 @@ def test_arm_ast2500_evb_sdk(self): self.do_test_arm_aspeed_sdk_start( self.workdir + '/ast2500-default/image-bmc') - self.wait_for_console_pattern('ast2500-default login:') + self.wait_for_console_pattern('nodistro.0 ast2500-default ttyS4') @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') def test_arm_ast2600_evb_sdk(self):
@@ -305,24 +306,28 @@ def test_arm_ast2600_evb_sdk(self): 'tmp105,bus=aspeed.i2c.bus.5,address=0x4d,id=tmp-test'); self.vm.add_args('-device', 'ds1338,bus=aspeed.i2c.bus.5,address=0x32'); + self.log.info('going to starrt *******') self.do_test_arm_aspeed_sdk_start( self.workdir + '/ast2600-default/image-bmc') - self.wait_for_console_pattern('ast2600-default login:') - exec_command_and_wait_for_pattern(self, 'root', 'Password:') - exec_command_and_wait_for_pattern(self, '0penBmc', 'root@ast2600-default:~#') + self.wait_for_console_pattern('nodistro.0 ast2600-default ttyS4') - exec_command_and_wait_for_pattern(self, - 'echo lm75 0x4d > /sys/class/i2c-dev/i2c-5/device/new_device', + self.ssh_connect('root', '0penBmc', False) + self.ssh_command('dmesg -c > /dev/null') + + self.ssh_command_output_contains( + 'echo lm75 0x4d > /sys/class/i2c-dev/i2c-5/device/new_device ; ' + 'dmesg -c', 'i2c i2c-5: new_device: Instantiated device lm75 at 0x4d'); - exec_command_and_wait_for_pattern(self, + self.ssh_command_output_contains( 'cat /sys/class/hwmon/hwmon19/temp1_input', '0') self.vm.command('qom-set', path='/machine/peripheral/tmp-test', property='temperature', value=18000); - exec_command_and_wait_for_pattern(self, + self.ssh_command_output_contains( 'cat /sys/class/hwmon/hwmon19/temp1_input', '18000') - exec_command_and_wait_for_pattern(self, - 'echo ds1307 0x32 > /sys/class/i2c-dev/i2c-5/device/new_device', + self.ssh_command_output_contains( + 'echo ds1307 0x32 > /sys/class/i2c-dev/i2c-5/device/new_device ; ' + 'dmesg -c', 'i2c i2c-5: new_device: Instantiated device ds1307 at 0x32'); year = time.strftime("%Y") - exec_command_and_wait_for_pattern(self, 'hwclock -f /dev/rtc1', year); + self.ssh_command_output_contains('/sbin/hwclock -f /dev/rtc1', year);
Signature · 1 line
2.31.1
Quoted 9 lines · Thomas Huth
test_arm_ast2500_evb_sdkandtest_arm_ast2600_evb_sdkare currently failing. The problem is that they are trying to look for the login prompt that does not have a newline at the end - but the logic in_console_interaction()only handles full lines. It used to work by accident in the past since there were sometimes kernel (warning and error) messages popping up that finally provided a newline character in the output, but since the tests have been changed to run with the "quiet" kernel parameter, this is not working anymore.
Quoted 4 lines
To make this work reliably, we must not look for the "login:" prompt, but have to use some text ending with a newline instead. And in the ast2600 test, switch to ssh instead of trying to log into the serial console - this works much more reliable and also has the benefit of
Yes. The test ran ~100 times without a failure. The 'dmesg -c' call is inelegant but there is not much other solutions.
Quoted 1 line
excercising the network interface here a little bit, too.
exercising
Quoted 1 line
Signed-off-by Thomas Huth <[email protected]>
(one little typo below)
Thanks,
C.
Quoted 44 lines
--- tests/avocado/machine_aspeed.py | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-)tests/avocado/machine_aspeed.py +4
-3@@ -8,6 +8,7 @@ import time import os +from avocado_qemu import LinuxSSHMixIn from avocado_qemu import QemuSystemTest from avocado_qemu import wait_for_console_pattern from avocado_qemu import exec_command@@ -230,7 +231,7 @@ def test_arm_ast2600_evb_buildroot(self): self.do_test_arm_aspeed_buildroot_poweroff() -class AST2x00MachineSDK(QemuSystemTest): +class AST2x00MachineSDK(QemuSystemTest, LinuxSSHMixIn): EXTRA_BOOTARGS = ( 'quiet '@@ -257,7 +258,7 @@ def do_test_arm_aspeed_sdk_start(self, image): self.require_netdev('user') self.vm.set_console() self.vm.add_args('-drive', 'file=' + image + ',if=mtd,format=raw', - '-net', 'nic', '-net', 'user') + '-net', 'nic', '-net', 'user,hostfwd=:127.0.0.1:0-:22') self.vm.launch() self.wait_for_console_pattern('U-Boot 2019.04')@@ -285,7 +286,7 @@ def test_arm_ast2500_evb_sdk(self): self.do_test_arm_aspeed_sdk_start( self.workdir + '/ast2500-default/image-bmc') - self.wait_for_console_pattern('ast2500-default login:') + self.wait_for_console_pattern('nodistro.0 ast2500-default ttyS4') @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') def test_arm_ast2600_evb_sdk(self):
tests/avocado/machine_aspeed.py def test_arm_ast2600_evb_sdk(self): · line 309
Quoted 4 lines
tests/avocado/machine_aspeed.py +1
-0@@ -305,24 +306,28 @@ def test_arm_ast2600_evb_sdk(self): 'tmp105,bus=aspeed.i2c.bus.5,address=0x4d,id=tmp-test'); self.vm.add_args('-device', 'ds1338,bus=aspeed.i2c.bus.5,address=0x32'); + self.log.info('going to starrt *******')
's/rr/r'
Quoted 34 lines
tests/avocado/machine_aspeed.py +13
-10self.do_test_arm_aspeed_sdk_start( self.workdir + '/ast2600-default/image-bmc') - self.wait_for_console_pattern('ast2600-default login:') - exec_command_and_wait_for_pattern(self, 'root', 'Password:') - exec_command_and_wait_for_pattern(self, '0penBmc', 'root@ast2600-default:~#') + self.wait_for_console_pattern('nodistro.0 ast2600-default ttyS4') - exec_command_and_wait_for_pattern(self, - 'echo lm75 0x4d > /sys/class/i2c-dev/i2c-5/device/new_device', + self.ssh_connect('root', '0penBmc', False) + self.ssh_command('dmesg -c > /dev/null') + + self.ssh_command_output_contains( + 'echo lm75 0x4d > /sys/class/i2c-dev/i2c-5/device/new_device ; ' + 'dmesg -c', 'i2c i2c-5: new_device: Instantiated device lm75 at 0x4d'); - exec_command_and_wait_for_pattern(self, + self.ssh_command_output_contains( 'cat /sys/class/hwmon/hwmon19/temp1_input', '0') self.vm.command('qom-set', path='/machine/peripheral/tmp-test', property='temperature', value=18000); - exec_command_and_wait_for_pattern(self, + self.ssh_command_output_contains( 'cat /sys/class/hwmon/hwmon19/temp1_input', '18000') - exec_command_and_wait_for_pattern(self, - 'echo ds1307 0x32 > /sys/class/i2c-dev/i2c-5/device/new_device', + self.ssh_command_output_contains( + 'echo ds1307 0x32 > /sys/class/i2c-dev/i2c-5/device/new_device ; ' + 'dmesg -c', 'i2c i2c-5: new_device: Instantiated device ds1307 at 0x32'); year = time.strftime("%Y") - exec_command_and_wait_for_pattern(self, 'hwclock -f /dev/rtc1', year); + self.ssh_command_output_contains('/sbin/hwclock -f /dev/rtc1', year);
Quoted 71 lines · Cédric Le Goater
Quoted 9 lines · Thomas Huth
test_arm_ast2500_evb_sdkandtest_arm_ast2600_evb_sdkare currently failing. The problem is that they are trying to look for the login prompt that does not have a newline at the end - but the logic in_console_interaction()only handles full lines. It used to work by accident in the past since there were sometimes kernel (warning and error) messages popping up that finally provided a newline character in the output, but since the tests have been changed to run with the "quiet" kernel parameter, this is not working anymore.Quoted 4 lines
To make this work reliably, we must not look for the "login:" prompt, but have to use some text ending with a newline instead. And in the ast2600 test, switch to ssh instead of trying to log into the serial console - this works much more reliable and also has the benefit of
Yes. The test ran ~100 times without a failure. The 'dmesg -c' call is inelegant but there is not much other solutions.
Quoted 1 line
excercising the network interface here a little bit, too.
exercising
Quoted 1 line
Signed-off-by Thomas Huth <[email protected]>Reviewed-by Cédric Le Goater <[email protected]>(one little typo below)
Thanks,
C.
Quoted 39 lines
---
tests/avocado/machine_aspeed.py| 31 ++++++++++++++++++-------------1 file changed, 18 insertions(+), 13 deletions(-)tests/avocado/machine_aspeed.py +0
-0@@ -8,6 +8,7 @@import time
import os
+fromavocado_qemuimport LinuxSSHMixIn fromavocado_qemuimport QemuSystemTest fromavocado_qemuimportwait_for_console_patternfromavocado_qemuimportexec_commandtests/avocado/machine_aspeed.py +1
-1@@ -230,7 +231,7 @@ def test_arm_ast2600_evb_buildroot(self): self.do_test_arm_aspeed_buildroot_poweroff() -class AST2x00MachineSDK(QemuSystemTest): +class AST2x00MachineSDK(QemuSystemTest, LinuxSSHMixIn):
EXTRA_BOOTARGS= (
'quiet 'tests/avocado/machine_aspeed.py +1
-1@@ -257,7 +258,7 @@ def do_test_arm_aspeed_sdk_start(self, image): self.require_netdev('user') self.vm.set_console() self.vm.add_args('-drive', 'file=' + image + ',if=mtd,format=raw', - '-net', 'nic', '-net', 'user') + '-net', 'nic', '-net', 'user,hostfwd=:127.0.0.1:0-:22') self.vm.launch() self.wait_for_console_pattern('U-Boot 2019.04')@@ -285,7 +286,7 @@ def test_arm_ast2500_evb_sdk(self):self.do_test_arm_aspeed_sdk_start(
self.workdir + '/ast2500-default/image-bmc')
-self.wait_for_console_pattern('ast2500-default login:')+self.wait_for_console_pattern('nodistro.0 ast2500-default ttyS4')@skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
deftest_arm_ast2600_evb_sdk(self):tests/avocado/machine_aspeed.py +0
-0@@ -305,24 +306,28 @@ def test_arm_ast2600_evb_sdk(self):
Quoted 6 lines
tests/avocado/machine_aspeed.py def test_arm_ast2600_evb_sdk(self): · line 309
Quoted 4 lines
tests/avocado/machine_aspeed.py +1
-0'tmp105,bus=aspeed.i2c.bus.5,address=0x4d,id=tmp-test'); self.vm.add_args('-device', 'ds1338,bus=aspeed.i2c.bus.5,address=0x32'); + self.log.info('going to starrt *******')'s/rr/r'
Drat, that whole line was just a debug print that I wanted to remove before sending out the patch ... I'll fix it.
Thomas
Quoted 39 lines
Quoted 38 lines
self.do_test_arm_aspeed_sdk_start(
self.workdir + '/ast2600-default/image-bmc')
-self.wait_for_console_pattern('ast2600-default login:')
-exec_command_and_wait_for_pattern(self, 'root', 'Password:')-exec_command_and_wait_for_pattern(self, '0penBmc', 'root@ast2600-default:~#')
+self.wait_for_console_pattern('nodistro.0 ast2600-default ttyS4')- exec_command_and_wait_for_pattern(self,
- 'echo lm750x4d>/sys/class/i2c-dev/i2c-5/device/new_device', +self.ssh_connect('root', '0penBmc', False)
+self.ssh_command('dmesg -c > /dev/null')
+
+ self.ssh_command_output_contains(
+ 'echo lm750x4d>/sys/class/i2c-dev/i2c-5/device/new_device; '
+ 'dmesg -c',
'i2c i2c-5:new_device: Instantiated device lm75 at0x4d'); - exec_command_and_wait_for_pattern(self,
+ self.ssh_command_output_contains(
'cat/sys/class/hwmon/hwmon19/temp1_input', '0')self.vm.command('qom-set', path='/machine/peripheral/tmp-test', property='temperature', value=18000);
- exec_command_and_wait_for_pattern(self,
+ self.ssh_command_output_contains(
'cat/sys/class/hwmon/hwmon19/temp1_input', '18000') - exec_command_and_wait_for_pattern(self,
- 'echo ds13070x32>/sys/class/i2c-dev/i2c-5/device/new_device',
+ self.ssh_command_output_contains(
+ 'echo ds13070x32>/sys/class/i2c-dev/i2c-5/device/new_device; '
+ 'dmesg -c',
'i2c i2c-5:new_device: Instantiated device ds1307 at0x32'); year =time.strftime("%Y")
-exec_command_and_wait_for_pattern(self, 'hwclock -f /dev/rtc1', year);
+self.ssh_command_output_contains('/sbin/hwclock -f /dev/rtc1', year);
PATCH 3/3 MAINTAINERS: Cover tests/avocado/machine_aspeed.py
In reply to Thomas Huthtests/avocado/machine_aspeed.py should belong to the ASPEED section in the maintainers file. Improve the wildcards here a little bit, so that it is covered, too.
--- MAINTAINERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
MAINTAINERS +1 -1
@@ -1111,7 +1111,7 @@ F: include/hw/misc/pca9552*.h F: hw/net/ftgmac100.c F: include/hw/net/ftgmac100.h F: docs/system/arm/aspeed.rst -F: tests/qtest/*aspeed* +F: tests/*/*aspeed* F: hw/arm/fby35.c NRF51
Signature · 1 line
2.31.1
Re: [PATCH 3/3] MAINTAINERS: Cover tests/avocado/machine_aspeed.py
Quoted 5 lines · Thomas Huth
tests/avocado/machine_aspeed.pyshould belong to the ASPEED section in the maintainers file. Improve the wildcards here a little bit, so that it is covered, too.Signed-off-by Thomas Huth <[email protected]>
Thanks,
C.
Quoted 17 lines
--- MAINTAINERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)MAINTAINERS +1
-1@@ -1111,7 +1111,7 @@ F: include/hw/misc/pca9552*.h F: hw/net/ftgmac100.c F: include/hw/net/ftgmac100.h F: docs/system/arm/aspeed.rst -F: tests/qtest/*aspeed* +F: tests/*/*aspeed* F: hw/arm/fby35.c NRF51
Quoted 9 lines · Thomas Huth
Fix the broken
ast2500_evb_sdkandast2600_evb_sdkavocado tests. See the patch description of the second patch for details. Also add the test to theMAINTAINERSfile (third patch).Thomas Huth (3):
tests/avocado: Makessh_command_output_contains()globally available tests/avocado/machine_aspeed: Fix the broken ast2[56]00_evb_sdk tests
MAINTAINERS: Covertests/avocado/machine_aspeed.py
Quoted 5 lines
MAINTAINERS | 2 +- tests/avocado/avocado_qemu/__init__.py | 8 +++++++ tests/avocado/linux_ssh_mips_malta.py | 8 ------- tests/avocado/machine_aspeed.py | 31 +++++++++++++++----------- 4 files changed, 27 insertions(+), 22 deletions(-)
Queued to testing/next, thanks.
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
Quoted 12 lines · Alex Bennée
Quoted 9 lines · Thomas Huth
Fix the broken
ast2500_evb_sdkandast2600_evb_sdkavocado tests. See the patch description of the second patch for details. Also add the test to theMAINTAINERSfile (third patch).Thomas Huth (3):
tests/avocado: Makessh_command_output_contains()globally available tests/avocado/machine_aspeed: Fix the broken ast2[56]00_evb_sdk tests
MAINTAINERS: Covertests/avocado/machine_aspeed.py
Quoted 7 lines
Quoted 5 lines
MAINTAINERS | 2 +- tests/avocado/avocado_qemu/__init__.py | 8 +++++++ tests/avocado/linux_ssh_mips_malta.py | 8 ------- tests/avocado/machine_aspeed.py | 31 +++++++++++++++----------- 4 files changed, 27 insertions(+), 22 deletions(-)Queued to testing/next, thanks.
Thanks, but could you please remove that "self.log.info('going to starrt *******')" line in the second patch? That was a debugging left-over...
Thomas
Quoted 17 lines · Thomas Huth
Quoted 16 lines · Alex Bennée
Quoted 14 lines · Thomas Huth
Fix the broken
ast2500_evb_sdkandast2600_evb_sdkavocado tests. See the patch description of the second patch for details. Also add the test to theMAINTAINERSfile (third patch).Thomas Huth (3):
tests/avocado: Makessh_command_output_contains()globally available tests/avocado/machine_aspeed: Fix the broken ast2[56]00_evb_sdk tests
MAINTAINERS: Covertests/avocado/machine_aspeed.pyMAINTAINERS | 2 +- tests/avocado/avocado_qemu/__init__.py | 8 +++++++ tests/avocado/linux_ssh_mips_malta.py | 8 ------- tests/avocado/machine_aspeed.py | 31 +++++++++++++++----------- 4 files changed, 27 insertions(+), 22 deletions(-)
Quoted 5 lines
Quoted 1 line
Queued to testing/next, thanks.
Thanks, but could you please remove that "
self.log.info('going to starrt *******')" line in the second patch? That was a debugging left-over...
no problem, done.
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro