tests/acceptance: Add tests for the PReP/40p machine
cover tests/acceptance: Add tests for the PReP/40p machine
Collection of various user cases from different mailing lists.
Quick tests worth to avoid regressions, idea from
https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04177.html
"Maintainers, please tell us how to boot your machines"
Regards,
Phil.
Philippe Mathieu-Daudé (5):
tests/acceptance: Add test that runs NetBSD installer on PRep/40p tests/acceptance: Test Open Firmware on the PReP/40p tests/acceptance: Test OpenBIOS on the PReP/40p tests/acceptance: Test Sandalfoot initrd on the PReP/40p .travis.yml: Let the avocado job run the 40p tests
.travis.yml | 2 +- tests/acceptance/ppc_prep_40p.py | 150 +++++++++++++++++++++++++++++++ 2 files changed, 151 insertions(+), 1 deletion(-) create mode 100644 tests/acceptance/ppc_prep_40p.py
Signature · 1 line
2.19.1
PATCH 1/5 tests/acceptance: Add test that runs NetBSD installer on PRep/40p
User case from:
http://mail-index.netbsd.org/port-prep/2017/04/11/msg000112.html
--- tests/acceptance/ppc_prep_40p.py | 63 ++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) create mode 100644 tests/acceptance/ppc_prep_40p.py
tests/acceptance/ppc_prep_40p.py new file +63 -0
@@ -0,0 +1,63 @@ +# Functional test that boots a PReP/40p machine and checks its serial console. +# +# Copyright (c) Philippe Mathieu-Daudé <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import os +import logging + +from avocado import skipIf +from avocado_qemu import Test + + +class IbmPrep40pMachine(Test): + + timeout = 60 + + # TODO refactor to MachineTest + def wait_for_console_pattern(self, success_message, failure_message=None): + """ + Waits for messages to appear on the console, while logging the content + + :param success_message: if this message appears, test succeeds + :param failure_message: if this message appears, test fails + """ + console = self.vm.console_socket.makefile() + console_logger = logging.getLogger('console') + while True: + msg = console.readline().strip() + if not msg: + continue + console_logger.debug(msg) + if success_message in msg: + break + if failure_message and failure_message in msg: + fail = 'Failure message found in console: %s' % failure_message + self.fail(fail) + + @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI') + def test_factory_firmware_and_netbsd(self): + """ + :avocado: tags=arch:ppc + :avocado: tags=machine:40p + :avocado: tags=slowness:high + """ + bios_url = ('ftp://ftp.boulder.ibm.com/rs6000/firmware/' + '7020-40p/P12H0456.IMG') + bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03' + bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash) + drive_url = ('https://ftp.netbsd.org/pub/NetBSD/NetBSD-archive/' + 'NetBSD-4.0/prep/installation/floppy/generic_com0.fs') + drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb' + drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash) + + self.vm.set_machine('40p') + self.vm.set_console() + self.vm.add_args('-bios', bios_path, + '-fda', drive_path) + self.vm.launch() + os_banner = 'NetBSD 4.0 (GENERIC) #0: Sun Dec 16 00:49:40 PST 2007' + self.wait_for_console_pattern(os_banner) + self.wait_for_console_pattern('Model: IBM PPS Model 6015')
Signature · 1 line
2.19.1
Re: [Qemu-devel] [PATCH 1/5] tests/acceptance: Add test that runs NetBSD installer on PRep/40p
typo "PRep" -> "PReP" in patch subject
Quoted 8 lines · Philippe Mathieu-Daudé
User case from:
http://mail-index.netbsd.org/port-prep/2017/04/11/msg000112.htmlSigned-off-by Philippe Mathieu-Daudé <[email protected]>--- tests/acceptance/ppc_prep_40p.py | 63 ++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) create mode 100644 tests/acceptance/ppc_prep_40p.py
Hervé, in v2 I plan to rename it machine_ppc_prep_40p.py and add an entry in MAINTAINERS:
-- >8 --
MAINTAINERS +1 -1
@@ -1051,6 +1051,7 @@ F: hw/timer/m48t59-isa.c F: include/hw/isa/pc87312.h F: include/hw/timer/m48t59.h F: pc-bios/ppc_rom.bin +F: tests/acceptance/machine_ppc_prep_40p.py ---
Quoted 71 lines
tests/acceptance/ppc_prep_40p.py new file +63
-0@@ -0,0 +1,63 @@ +# Functional test that boots a PReP/40p machine and checks its serial console. +# +# Copyright (c) Philippe Mathieu-Daudé <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import os +import logging + +from avocado import skipIf +from avocado_qemu import Test + + +class IbmPrep40pMachine(Test): + + timeout = 60 + + # TODO refactor to MachineTest + def wait_for_console_pattern(self, success_message, failure_message=None): + """ + Waits for messages to appear on the console, while logging the content + + :param success_message: if this message appears, test succeeds + :param failure_message: if this message appears, test fails + """ + console = self.vm.console_socket.makefile() + console_logger = logging.getLogger('console') + while True: + msg = console.readline().strip() + if not msg: + continue + console_logger.debug(msg) + if success_message in msg: + break + if failure_message and failure_message in msg: + fail = 'Failure message found in console: %s' % failure_message + self.fail(fail) + + @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI') + def test_factory_firmware_and_netbsd(self): + """ + :avocado: tags=arch:ppc + :avocado: tags=machine:40p + :avocado: tags=slowness:high + """ + bios_url = ('ftp://ftp.boulder.ibm.com/rs6000/firmware/' + '7020-40p/P12H0456.IMG') + bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03' + bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash) + drive_url = ('https://ftp.netbsd.org/pub/NetBSD/NetBSD-archive/' + 'NetBSD-4.0/prep/installation/floppy/generic_com0.fs') + drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb' + drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash) + + self.vm.set_machine('40p') + self.vm.set_console() + self.vm.add_args('-bios', bios_path, + '-fda', drive_path) + self.vm.launch() + os_banner = 'NetBSD 4.0 (GENERIC) #0: Sun Dec 16 00:49:40 PST 2007' + self.wait_for_console_pattern(os_banner) + self.wait_for_console_pattern('Model: IBM PPS Model 6015')
Hervé Poussineau
Reviewed-byRe: [Qemu-devel] [PATCH 1/5] tests/acceptance: Add test that runs NetBSD installer on PRep/40p
Le 27/06/2019 à 14:00, Philippe Mathieu-Daudé a écrit :
Quoted 6 lines
typo "PRep" -> "PReP" in patch subject
Quoted 3 lines · Philippe Mathieu-Daudé
User case from:
http://mail-index.netbsd.org/port-prep/2017/04/11/msg000112.html
Quoted 8 lines
Quoted 5 lines
Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- tests/acceptance/ppc_prep_40p.py | 63 ++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+)create mode 100644
tests/acceptance/ppc_prep_40p.pyHervé, in v2 I plan to rename it
machine_ppc_prep_40p.pyand add an entry inMAINTAINERS:
With the change you want to do:
Quoted 83 lines
-- >8 --
MAINTAINERS +1
-1@@ -1051,6 +1051,7 @@ F: hw/timer/m48t59-isa.c F: include/hw/isa/pc87312.h F: include/hw/timer/m48t59.h F: pc-bios/ppc_rom.bin +F: tests/acceptance/machine_ppc_prep_40p.py ---Quoted 71 lines
tests/acceptance/ppc_prep_40p.py new file +63
-0@@ -0,0 +1,63 @@ +# Functional test that boots a PReP/40p machine and checks its serial console. +# +# Copyright (c) Philippe Mathieu-Daudé <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import os +import logging + +from avocado import skipIf +from avocado_qemu import Test + + +class IbmPrep40pMachine(Test): + + timeout = 60 + + # TODO refactor to MachineTest + def wait_for_console_pattern(self, success_message, failure_message=None): + """ + Waits for messages to appear on the console, while logging the content + + :param success_message: if this message appears, test succeeds + :param failure_message: if this message appears, test fails + """ + console = self.vm.console_socket.makefile() + console_logger = logging.getLogger('console') + while True: + msg = console.readline().strip() + if not msg: + continue + console_logger.debug(msg) + if success_message in msg: + break + if failure_message and failure_message in msg: + fail = 'Failure message found in console: %s' % failure_message + self.fail(fail) + + @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI') + def test_factory_firmware_and_netbsd(self): + """ + :avocado: tags=arch:ppc + :avocado: tags=machine:40p + :avocado: tags=slowness:high + """ + bios_url = ('ftp://ftp.boulder.ibm.com/rs6000/firmware/' + '7020-40p/P12H0456.IMG') + bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03' + bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash) + drive_url = ('https://ftp.netbsd.org/pub/NetBSD/NetBSD-archive/' + 'NetBSD-4.0/prep/installation/floppy/generic_com0.fs') + drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb' + drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash) + + self.vm.set_machine('40p') + self.vm.set_console() + self.vm.add_args('-bios', bios_path, + '-fda', drive_path) + self.vm.launch() + os_banner = 'NetBSD 4.0 (GENERIC) #0: Sun Dec 16 00:49:40 PST 2007' + self.wait_for_console_pattern(os_banner) + self.wait_for_console_pattern('Model: IBM PPS Model 6015')
Re: [Qemu-devel] [PATCH 1/5] tests/acceptance: Add test that runs NetBSD installer on PRep/40p
In reply to Philippe Mathieu-DaudéQuoted 5 lines · Philippe Mathieu-Daudé
tests/acceptance/ppc_prep_40p.py new file +5
-0+ bios_url = ('ftp://ftp.boulder.ibm.com/rs6000/firmware/' + '7020-40p/P12H0456.IMG') + bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03' + bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash) + drive_url = ('https://ftp.netbsd.org/pub/NetBSD/NetBSD-archive/'
tests/acceptance/ppc_prep_40p.py line 61
Quoted 10 lines
tests/acceptance/ppc_prep_40p.py new file +10
-0+ 'NetBSD-4.0/prep/installation/floppy/generic_com0.fs') + drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb' + drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash) + + self.vm.set_machine('40p') + self.vm.set_console() + self.vm.add_args('-bios', bios_path, + '-fda', drive_path) + self.vm.launch() + os_banner = 'NetBSD 4.0 (GENERIC) #0: Sun Dec 16 00:49:40 PST 2007'
Is there a specific reason to use NetBSD 4.0? It's a very old release, the newest one is 8.1.
Re: [Qemu-devel] [PATCH 1/5] tests/acceptance: Add test that runs NetBSD installer on PRep/40p
Hi Kamil,
Quoted 17 lines · Kamil Rytarowski
Quoted 5 lines · Philippe Mathieu-Daudé
tests/acceptance/ppc_prep_40p.py new file +5
-0+ bios_url = ('ftp://ftp.boulder.ibm.com/rs6000/firmware/' + '7020-40p/P12H0456.IMG') + bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03' + bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash) + drive_url = ('https://ftp.netbsd.org/pub/NetBSD/NetBSD-archive/'tests/acceptance/ppc_prep_40p.py line 61
Quoted 10 lines
tests/acceptance/ppc_prep_40p.py new file +10
-0+ 'NetBSD-4.0/prep/installation/floppy/generic_com0.fs') + drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb' + drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash) + + self.vm.set_machine('40p') + self.vm.set_console() + self.vm.add_args('-bios', bios_path, + '-fda', drive_path) + self.vm.launch() + os_banner = 'NetBSD 4.0 (GENERIC) #0: Sun Dec 16 00:49:40 PST 2007'
Quoted 2 lines
Is there a specific reason to use NetBSD 4.0? It's a very old release, the newest one is 8.1.
The goal of these integration tests is to verify a specific configuration that worked in the past still works with today codebase.
It is particularly useful for machines that have not a big quantity of users.
This test does not intent to test NetBSD, but that the PReP/40p machine is still working, as it used to work in the post saved in the commit description:
http://mail-index.netbsd.org/port-prep/2017/04/11/msg000112.html
So this test is useful to avoid the PReP machine code to bitrot.
I'd like to know what other from the QEMU community think/expect about these tests.
FWIW I tested newer versions and they don't boot.
Regards,
Phil.
Re: [Qemu-devel] [PATCH 1/5] tests/acceptance: Add test that runs NetBSD installer on PRep/40p
Quoted 33 lines · Philippe Mathieu-Daudé
Hi Kamil,
Quoted 17 lines · Kamil Rytarowski
Quoted 5 lines · Philippe Mathieu-Daudé
tests/acceptance/ppc_prep_40p.py new file +5
-0+ bios_url = ('ftp://ftp.boulder.ibm.com/rs6000/firmware/' + '7020-40p/P12H0456.IMG') + bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03' + bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash) + drive_url = ('https://ftp.netbsd.org/pub/NetBSD/NetBSD-archive/'tests/acceptance/ppc_prep_40p.py line 61
Quoted 10 lines
tests/acceptance/ppc_prep_40p.py new file +10
-0+ 'NetBSD-4.0/prep/installation/floppy/generic_com0.fs') + drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb' + drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash) + + self.vm.set_machine('40p') + self.vm.set_console() + self.vm.add_args('-bios', bios_path, + '-fda', drive_path) + self.vm.launch() + os_banner = 'NetBSD 4.0 (GENERIC) #0: Sun Dec 16 00:49:40 PST 2007'Quoted 2 lines
Is there a specific reason to use NetBSD 4.0? It's a very old release, the newest one is 8.1.
The goal of these integration tests is to verify a specific configuration that worked in the past still works with today codebase.
It is particularly useful for machines that have not a big quantity of users.
This test does not intent to test NetBSD, but that the PReP/40p machine is still working, as it used to work in the post saved in the commit description:
Quoted 7 lines
http://mail-index.netbsd.org/port-prep/2017/04/11/msg000112.html
So this test is useful to avoid the PReP machine code to bitrot.
I'd like to know what other from the QEMU community think/expect about these tests.
This is understood and appreciated as NetBSD still formally supports the prep port.
Quoted 2 lines
FWIW I tested newer versions and they don't boot.
This is what I wanted to know, whether this was an accident in using an older version or not. I will try to report it to other developers and maybe someone could debug and fix it.
Quoted 4 lines
Regards,
Phil.
PATCH 2/5 tests/acceptance: Test Open Firmware on the PReP/40p
In reply to Philippe Mathieu-DaudéUser case from:
https://tyom.blogspot.com/2019/04/aixprep-under-qemu-how-to.html
--- tests/acceptance/ppc_prep_40p.py | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
tests/acceptance/ppc_prep_40p.py +21 -0
@@ -61,3 +61,24 @@ class IbmPrep40pMachine(Test): os_banner = 'NetBSD 4.0 (GENERIC) #0: Sun Dec 16 00:49:40 PST 2007' self.wait_for_console_pattern(os_banner) self.wait_for_console_pattern('Model: IBM PPS Model 6015') + + def test_openfirmware(self): + """ + :avocado: tags=arch:ppc + :avocado: tags=machine:40p + """ + bios_url = ('https://github.com/artyom-tarasenko/openfirmware/' + 'releases/download/40p-20190413/q40pofw-serial.rom') + bios_hash = '880c80172ea5b2247c0ac2a8bf36bbe385192c72' + bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash) + + self.vm.set_machine('40p') + self.vm.set_console() + self.vm.add_args('-bios', bios_path) + + self.vm.launch() + self.wait_for_console_pattern('QEMU PReP/40p') + fw_banner = 'Open Firmware, built April 13, 2019 09:29:23' + self.wait_for_console_pattern(fw_banner) + prompt_msg = 'Type any key to interrupt automatic startup' + self.wait_for_console_pattern(prompt_msg)
Signature · 1 line
2.19.1
PATCH 3/5 tests/acceptance: Test OpenBIOS on the PReP/40p
In reply to Philippe Mathieu-DaudéUser case from:
https://mail.coreboot.org/pipermail/openbios/2018-May/010360.html
--- tests/acceptance/ppc_prep_40p.py | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)
tests/acceptance/ppc_prep_40p.py +32 -0
@@ -82,3 +82,35 @@ class IbmPrep40pMachine(Test): self.wait_for_console_pattern(fw_banner) prompt_msg = 'Type any key to interrupt automatic startup' self.wait_for_console_pattern(prompt_msg) + + def test_openbios_192m(self): + """ + :avocado: tags=arch:ppc + :avocado: tags=machine:40p + """ + self.vm.set_machine('40p') + self.vm.set_console() + self.vm.add_args('-m', '192') + + self.vm.launch() + self.wait_for_console_pattern('>> OpenBIOS') + self.wait_for_console_pattern('>> Memory: 192M') + self.wait_for_console_pattern('>> CPU type PowerPC,604') + + def test_openbios_and_netbsd(self): + """ + :avocado: tags=arch:ppc + :avocado: tags=machine:40p + """ + drive_url = ('https://ftp.netbsd.org/pub/NetBSD/iso/7.1.2/' + 'NetBSD-7.1.2-prep.iso') + drive_hash = '78734c1bdda79778f0b6f391969ad2458ed8981c' + drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash) + + self.vm.set_machine('40p') + self.vm.set_console() + self.vm.add_args('-cdrom', drive_path, + '-boot', 'd') + + self.vm.launch() + self.wait_for_console_pattern('NetBSD/prep BOOT, Revision 1.9')
Signature · 1 line
2.19.1
PATCH 4/5 tests/acceptance: Test Sandalfoot initrd on the PReP/40p
In reply to Philippe Mathieu-DaudéUser case from:
https://mail.coreboot.org/pipermail/openbios/2018-May/010360.html
Sandalfoot info:
http://www.juneau-lug.org/sandalfoot.php
--- tests/acceptance/ppc_prep_40p.py | 34 ++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
tests/acceptance/ppc_prep_40p.py +34 -0
@@ -37,6 +37,11 @@ class IbmPrep40pMachine(Test): fail = 'Failure message found in console: %s' % failure_message self.fail(fail) + def exec_command_and_wait_for_pattern(self, command, success_message): + command += '\n' + self.vm.console_socket.sendall(command.encode()) + self.wait_for_console_pattern(success_message) + @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI') def test_factory_firmware_and_netbsd(self): """
@@ -114,3 +119,32 @@ class IbmPrep40pMachine(Test): self.vm.launch() self.wait_for_console_pattern('NetBSD/prep BOOT, Revision 1.9') + + def test_sandalfoot_busybox(self): + """ + :avocado: tags=arch:ppc + :avocado: tags=machine:40p + """ + drive_url = ('http://www.juneau-lug.org/zImage.initrd.sandalfoot') + drive_hash = 'dacacfc4085ea51d34d99ef70e972b849e2c6949' + drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash) + + self.vm.set_machine('40p') + self.vm.set_console() + self.vm.add_args('-cdrom', drive_path, + '-boot', 'd') + + self.vm.launch() + self.wait_for_console_pattern('Now booting the kernel') + + msg = 'Please press Enter to activate this console.' + self.wait_for_console_pattern(msg) + + version = 'BusyBox v0.60.0 (2001.08.19-09:26+0000) Built-in shell (ash)' + self.exec_command_and_wait_for_pattern('', version) + + uname = 'Linux ppc 2.4.18 #5 Wed May 21 23:50:43 AKDT 2003 ppc unknown' + self.exec_command_and_wait_for_pattern('uname -a', uname) + + cpu = 'PReP IBM 6015/7020 (Sandalfoot/Sandalbow)' + self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo', cpu)
Signature · 1 line
2.19.1
PATCH 5/5 .travis.yml: Let the avocado job run the 40p tests
In reply to Philippe Mathieu-Daudé---
If this list continues to grow we can
- split it (as other jobs)
- move them to GitLab where we can have multi-stage jobs, avocado tests run on top of build jobs.
--- .travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
.travis.yml +1 -1
@@ -231,7 +231,7 @@ matrix: # Acceptance (Functional) tests - env: - - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu" + - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu,ppc-softmmu" - TEST_CMD="make check-acceptance" after_failure: - cat tests/results/latest/job.log
Signature · 1 line
2.19.1