mips: Add more Avocado tests
v3 cover mips: Add more Avocado tests
Hi,
It was a rainy week-end here, so I invested it to automatize some of my MIPS tests.
The BootLinuxSshTest is not Global warming friendly, it is not meant to run on a CI system but rather on a workstation previous to post a pull request.
It can surely be improved, but it is a good starting point.
v3: Addressed issues reported by Cleber (only posting 3/4 and 4/4)
v2: Decorate slow tests with skipIf(CONTINUOUS_INTEGRATION):
VENV /home/travis/build/philmd/qemu/tests/venv PIP /home/travis/build/philmd/qemu/tests/requirements.txt MKDIR /home/travis/build/philmd/qemu/tests/results
AVOCADO tests/acceptance
JOB ID : 02a2b1c585c12aebfbd8db2f330ed9730a547ae0
JOB LOG : /home/travis/build/philmd/qemu/tests/results/job-2019-05-23T16.05-02a2b1c/job.log
(01/22) acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_pc: PASS (5.41 s)
(02/22) acceptance/boot_linux_console.py:BootLinuxConsole.test_mips_malta: PASS (3.33 s)
(03/22) acceptance/boot_linux_console.py:BootLinuxConsole.test_mips64el_malta: PASS (3.47 s) (04/22) acceptance/boot_linux_console.py:BootLinuxConsole.test_mips_malta_cpio: PASS (15.09 s) (05/22) acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_virt: PASS (2.48 s)
(06/22) acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_virt: PASS (7.25 s)
(07/22) acceptance/boot_linux_console.py:BootLinuxConsole.test_s390x_s390_ccw_virtio: PASS (4.66 s) (08/22) acceptance/boot_linux_console.py:BootLinuxConsole.test_alpha_clipper: PASS (3.37 s)
(09/22) acceptance/cpu_queries.py:QueryCPUModelExpansion.test: PASS (0.16 s)
(10/22) acceptance/empty_cpu_model.py:EmptyCPUModel.test: PASS (0.02 s)
(11/22) acceptance/linux_initrd.py:LinuxInitrd.test_with_2gib_file_should_exit_error_msg_with_linux_v3_6: PASS (3.89 s) (12/22) acceptance/linux_initrd.py:LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16: PASS (15.14 s) (13/22) acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta32eb_kernel3_2_0: SKIP: Running on Travis-CI (14/22) acceptance/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta64el_kernel3_2_0: SKIP: Running on Travis-CI (15/22) acceptance/migration.py:Migration.test_migration_with_tcp_localhost: PASS (0.23 s)
(16/22) acceptance/version.py:Version.test_qmp_human_info_version: PASS (0.04 s)
(17/22) acceptance/virtio_version.py:VirtioVersionCheck.test_conventional_devs: PASS (1.20 s) (18/22) acceptance/virtio_version.py:VirtioVersionCheck.test_modern_only_devs: PASS (0.41 s) (19/22) acceptance/vnc.py:Vnc.test_no_vnc: PASS (0.03 s)
(20/22) acceptance/vnc.py:Vnc.test_no_vnc_change_password: PASS (0.04 s)
(21/22) acceptance/vnc.py:Vnc.test_vnc_change_password_requires_a_password: PASS (0.04 s)
(22/22) acceptance/vnc.py:Vnc.test_vnc_change_password: PASS (0.04 s)
RESULTS : PASS 20 | ERROR 0 | FAIL 0 | SKIP 2 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 68.64 s
Job output here:
https://travis-ci.org/philmd/qemu/jobs/536369687
"Ran for 16 min 26 sec"
Regards,
Phil.
Philippe Mathieu-Daudé (2):
BootLinuxConsoleTest: Test nanoMIPS kernels on the I7200 CPU
BootLinuxConsoleTest: Run kerneltests BusyBox on Malta
tests/acceptance/boot_linux_console.py | 102 +++++++++++++++++++++++++ 1 file changed, 102 insertions(+)
Signature · 1 line
2.20.1
PATCH v3 1/2 BootLinuxConsoleTest: Test nanoMIPS kernels on the I7200 CPU
From: Philippe Mathieu-Daudé <[email protected]>
Similar to the x86_64/pc test, it boots a Linux kernel on a Malta machine and verify the serial is working.
Use the documentation added in commit f7d257cb4a17 to test nanoMIPS kernels and the I7200 CPU.
This test can be run using:
$ avocado --show=console run -t arch:mipsel tests/acceptance/boot_linux_console.py console: [ 0.000000] Linux version 4.15.18-00432-gb2eb9a8b (emubuild@mipscs563) (gcc version 6.3.0 (Codescape GNU Tools 2018.04-02 for nanoMIPS Linux)) #1 SMP Wed Jun 27 11:10:08 PDT 2018 console: [ 0.000000] GCRs appear to have been moved (expected them at 0x1fbf8000)! console: [ 0.000000] GCRs appear to have been moved (expected them at 0x1fbf8000)! console: [ 0.000000] CPU0 revision is: 00010000 (MIPS GENERIC QEMU) console: [ 0.000000] MIPS: machine is mti,malta console: [ 0.000000] Determined physical RAM map: console: [ 0.000000] memory: 08000000 @ 00000000 (usable) console: [ 0.000000] earlycon: ns16550a0 at I/O port 0x3f8 (options '38400n8') console: [ 0.000000] bootconsole [ns16550a0] enabled console: [ 0.000000] User-defined physical RAM map: console: [ 0.000000] memory: 10000000 @ 00000000 (usable) console: [ 0.000000] Initrd not found or empty - disabling initrd console: [ 0.000000] MIPS CPS SMP unable to proceed without a CM console: [ 0.000000] Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes. console: [ 0.000000] Primary data cache 32kB, 4-way, VIPT, cache aliases, linesize 32 bytes console: [ 0.000000] This processor doesn't support highmem. -262144k highmem ignored console: [ 0.000000] Zone ranges: console: [ 0.000000] Normal [mem 0x0000000000000000-0x000000000fffffff] console: [ 0.000000] HighMem empty console: [ 0.000000] Movable zone start for each node console: [ 0.000000] Early memory node ranges console: [ 0.000000] node 0: [mem 0x0000000000000000-0x000000000fffffff] console: [ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff] console: [ 0.000000] random: get_random_bytes called from start_kernel+0x60/0x2f0 with crng_init=0 console: [ 0.000000] percpu: Embedded 16 pages/cpu @(ptrval) s36620 r8192 d20724 u65536 console: [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 64960 console: [ 0.000000] Kernel command line: printk.time=0 mem=256m@@0x0 console=ttyS0 earlycon
---
v3: No change
--- tests/acceptance/boot_linux_console.py | 58 ++++++++++++++++++++++++++ 1 file changed, 58 insertions(+)
tests/acceptance/boot_linux_console.py +58 -0
@@ -10,6 +10,8 @@ import os import logging +import lzma +import shutil from avocado_qemu import Test from avocado.utils import process
@@ -138,6 +140,62 @@ class BootLinuxConsole(Test): console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) + def do_test_mips_malta32el_nanomips(self, kernel_url, kernel_hash): + kernel_path_xz = self.fetch_asset(kernel_url, asset_hash=kernel_hash) + kernel_path = self.workdir + "kernel" + with lzma.open(kernel_path_xz, 'rb') as f_in: + with open(kernel_path, 'wb') as f_out: + shutil.copyfileobj(f_in, f_out) + + self.vm.set_machine('malta') + self.vm.set_console() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'mem=256m@@0x0 ' + + 'console=ttyS0') + self.vm.add_args('-no-reboot', + '-cpu', 'I7200', + '-kernel', kernel_path, + '-append', kernel_command_line) + self.vm.launch() + console_pattern = 'Kernel command line: %s' % kernel_command_line + self.wait_for_console_pattern(console_pattern) + + def test_mips_malta32el_nanomips_4k(self): + """ + :avocado: tags=arch:mipsel + :avocado: tags=machine:malta + :avocado: tags=endian:little + """ + kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/' + 'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/' + 'generic_nano32r6el_page4k.xz') + kernel_hash = '477456aafd2a0f1ddc9482727f20fe9575565dd6' + self.do_test_mips_malta32el_nanomips(kernel_url, kernel_hash) + + def test_mips_malta32el_nanomips_16k_up(self): + """ + :avocado: tags=arch:mipsel + :avocado: tags=machine:malta + :avocado: tags=endian:little + """ + kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/' + 'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/' + 'generic_nano32r6el_page16k_up.xz') + kernel_hash = 'e882868f944c71c816e832e2303b7874d044a7bc' + self.do_test_mips_malta32el_nanomips(kernel_url, kernel_hash) + + def test_mips_malta32el_nanomips_64k_dbg(self): + """ + :avocado: tags=arch:mipsel + :avocado: tags=machine:malta + :avocado: tags=endian:little + """ + kernel_url = ('https://mipsdistros.mips.com/LinuxDistro/nanomips/' + 'kernels/v4.15.18-432-gb2eb9a8b07a1-20180627102142/' + 'generic_nano32r6el_page64k_dbg.xz') + kernel_hash = '18d1c68f2e23429e266ca39ba5349ccd0aeb7180' + self.do_test_mips_malta32el_nanomips(kernel_url, kernel_hash) + def test_aarch64_virt(self): """ :avocado: tags=arch:aarch64
Signature · 1 line
2.20.1
PATCH v3 2/2 BootLinuxConsoleTest: Run kerneltests BusyBox on Malta
In reply to Philippe Mathieu-DaudéFrom: Philippe Mathieu-Daudé <[email protected]>
This tests boots a Linux kernel on a Malta machine up to a busybox shell on the serial console. Few commands are executed before halting the machine (via reboot).
We use the initrd cpio image from the kerneltests project:
https://kerneltests.org/
If MIPS is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.
Alternatively, this test can be run using:
$ avocado --show=console run -t arch:mips tests/acceptance/boot_linux_console.py [...] console: Boot successful. [...] console: / # uname -a console: Linux buildroot 4.5.0-2-4kc-malta #1 Debian 4.5.5-1 (2016-05-29) mips GNU/Linux console: / # reboot console: / # reboot: Restarting system
---
v3: Use archive.extract() instead of shutil+gzip (Cleber)
--- tests/acceptance/boot_linux_console.py | 44 ++++++++++++++++++++++++++ 1 file changed, 44 insertions(+)
tests/acceptance/boot_linux_console.py +44 -0
@@ -49,6 +49,11 @@ class BootLinuxConsole(Test): fail = 'Failure message found in console: %s' % failure_message self.fail(fail) + def exec_command_and_wait_for_pattern(self, command, success_message): + command += '\n' + self.vm.console_socket.sendall(command.encode()) + self.wait_for_console_pattern(success_message) + def extract_from_deb(self, deb, path): """ Extracts a file from a deb package into the test workdir
@@ -140,6 +145,45 @@ class BootLinuxConsole(Test): console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) + def test_mips_malta_cpio(self): + """ + :avocado: tags=arch:mips + :avocado: tags=machine:malta + :avocado: tags=endian:big + """ + deb_url = ('http://snapshot.debian.org/archive/debian/' + '20160601T041800Z/pool/main/l/linux/' + 'linux-image-4.5.0-2-4kc-malta_4.5.5-1_mips.deb') + deb_hash = 'a3c84f3e88b54e06107d65a410d1d1e8e0f340f8' + deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash) + kernel_path = self.extract_from_deb(deb_path, + '/boot/vmlinux-4.5.0-2-4kc-malta') + initrd_url = ('https://github.com/groeck/linux-build-test/raw/' + '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/' + 'mips/rootfs.cpio.gz') + initrd_hash = 'bf806e17009360a866bf537f6de66590de349a99' + initrd_path_gz = self.fetch_asset(initrd_url, asset_hash=initrd_hash) + initrd_path = archive.extract(initrd_path_gz, + os.path.join(self.workdir, 'rootfs.cpio')) + self.vm.set_machine('malta') + self.vm.set_console() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 console=tty ' + + 'rdinit=/sbin/init noreboot') + self.vm.add_args('-kernel', kernel_path, + '-initrd', initrd_path, + '-append', kernel_command_line, + '-no-reboot') + self.vm.launch() + self.wait_for_console_pattern('Boot successful.') + + self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo', + 'BogoMIPS') + self.exec_command_and_wait_for_pattern('uname -a', + 'Debian') + self.exec_command_and_wait_for_pattern('reboot', + 'reboot: Restarting system') + def do_test_mips_malta32el_nanomips(self, kernel_url, kernel_hash): kernel_path_xz = self.fetch_asset(kernel_url, asset_hash=kernel_hash) kernel_path = self.workdir + "kernel"
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v3 2/2] BootLinuxConsoleTest: Run kerneltests BusyBox on Malta
Quoted 97 lines · Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <[email protected]>
This tests boots a Linux kernel on a Malta machine up to a busybox shell on the serial console. Few commands are executed before halting the machine (via reboot).
We use the initrd cpio image from the kerneltests project:
https://kerneltests.org/If MIPS is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.
Alternatively, this test can be run using:
$ avocado --show=console run -t arch:mips tests/acceptance/boot_linux_console.py [...] console: Boot successful. [...] console: / # uname -a console: Linux buildroot 4.5.0-2-4kc-malta #1 Debian 4.5.5-1 (2016-05-29) mips GNU/Linux console: / # reboot console: / # reboot: Restarting systemSigned-off-by Philippe Mathieu-Daudé <[email protected]>Tested-by Cleber Rosa <[email protected]>---
v3: Usearchive.extract()instead of shutil+gzip (Cleber)--- tests/acceptance/boot_linux_console.py | 44 ++++++++++++++++++++++++++ 1 file changed, 44 insertions(+)tests/acceptance/boot_linux_console.py +44
-0@@ -49,6 +49,11 @@ class BootLinuxConsole(Test): fail = 'Failure message found in console: %s' % failure_message self.fail(fail) + def exec_command_and_wait_for_pattern(self, command, success_message): + command += '\n' + self.vm.console_socket.sendall(command.encode()) + self.wait_for_console_pattern(success_message) + def extract_from_deb(self, deb, path): """ Extracts a file from a deb package into the test workdir@@ -140,6 +145,45 @@ class BootLinuxConsole(Test): console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) + def test_mips_malta_cpio(self): + """ + :avocado: tags=arch:mips + :avocado: tags=machine:malta + :avocado: tags=endian:big + """ + deb_url = ('http://snapshot.debian.org/archive/debian/' + '20160601T041800Z/pool/main/l/linux/' + 'linux-image-4.5.0-2-4kc-malta_4.5.5-1_mips.deb') + deb_hash = 'a3c84f3e88b54e06107d65a410d1d1e8e0f340f8' + deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash) + kernel_path = self.extract_from_deb(deb_path, + '/boot/vmlinux-4.5.0-2-4kc-malta') + initrd_url = ('https://github.com/groeck/linux-build-test/raw/' + '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/' + 'mips/rootfs.cpio.gz') + initrd_hash = 'bf806e17009360a866bf537f6de66590de349a99' + initrd_path_gz = self.fetch_asset(initrd_url, asset_hash=initrd_hash) + initrd_path = archive.extract(initrd_path_gz, + os.path.join(self.workdir, 'rootfs.cpio')) + self.vm.set_machine('malta') + self.vm.set_console() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 console=tty ' + + 'rdinit=/sbin/init noreboot') + self.vm.add_args('-kernel', kernel_path, + '-initrd', initrd_path, + '-append', kernel_command_line, + '-no-reboot') + self.vm.launch() + self.wait_for_console_pattern('Boot successful.') + + self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo', + 'BogoMIPS') + self.exec_command_and_wait_for_pattern('uname -a', + 'Debian') + self.exec_command_and_wait_for_pattern('reboot', + 'reboot: Restarting system') + def do_test_mips_malta32el_nanomips(self, kernel_url, kernel_hash): kernel_path_xz = self.fetch_asset(kernel_url, asset_hash=kernel_hash) kernel_path = self.workdir + "kernel"Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v3 2/2] BootLinuxConsoleTest: Run kerneltests BusyBox on Malta
In reply to Philippe Mathieu-DaudéQuoted 10 lines · Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <[email protected]>
This tests boots a Linux kernel on a Malta machine up to a busybox shell on the serial console. Few commands are executed before halting the machine (via reboot).
We use the initrd cpio image from the kerneltests project:
https://kerneltests.org/
Quoted 6 lines
If MIPS is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:mips" tags.
Alternatively, this test can be run using:
$ avocado --show=console run -t arch:mips
tests/acceptance/boot_linux_console.py
Quoted 5 lines
[...]
console: Boot successful.
[...]
console: / # uname -a
console: Linux buildroot 4.5.0-2-4kc-malta #1 Debian 4.5.5-1
(2016-05-29) mips GNU/Linux
Quoted 7 lines
console: / # reboot
console: / # reboot: Restarting systemSigned-off-by Philippe Mathieu-Daudé <[email protected]>Tested-by Cleber Rosa <[email protected]>---
Excellent test, and commit message content too. May I suggest the test group inclusion of the text of this commit message (or similar text) to the documentation on QEMU Testing web page, as an example of the acceptance test usage?
Yours, Aleksandar
Review
Quoted 6 lines
v3: Use
archive.extract()instead of shutil+gzip (Cleber)--- tests/acceptance/boot_linux_console.py | 44 ++++++++++++++++++++++++++ 1 file changed, 44 insertions(+)(quoted diff) +0
-0
b/tests/acceptance/boot_linux_console.py
tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 49
Quoted 5 lines
index
a7a735c90d..751e3bff86100644tests/acceptance/boot_linux_console.py +0
-0@@ -49,6 +49,11 @@ class BootLinuxConsole(Test): fail = 'Failure message found in console: %s' %
failure_message
tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test):
Quoted 3 lines
tests/acceptance/boot_linux_console.py +1
-0self.fail(fail) + def exec_command_and_wait_for_pattern(self, command,
success_message):
Quoted 14 lines
tests/acceptance/boot_linux_console.py +7
-0+ command += '\n' + self.vm.console_socket.sendall(command.encode()) + self.wait_for_console_pattern(success_message) + def extract_from_deb(self, deb, path): """ Extracts a file from a deb package into the test workdir@@ -140,6 +145,45 @@ class BootLinuxConsole(Test): console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) + def test_mips_malta_cpio(self): + """ + :avocado: tags=arch:mips
tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 160
Quoted 10 lines
tests/acceptance/boot_linux_console.py +10
-0+ :avocado: tags=machine:malta + :avocado: tags=endian:big + """ + deb_url = ('http://snapshot.debian.org/archive/debian/' + '20160601T041800Z/pool/main/l/linux/' + 'linux-image-4.5.0-2-4kc-malta_4.5.5-1_mips.deb') + deb_hash = 'a3c84f3e88b54e06107d65a410d1d1e8e0f340f8' + deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash) + kernel_path = self.extract_from_deb(deb_path, +
'/boot/vmlinux-4.5.0-2-4kc-malta')
tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test):
Quoted 5 lines
tests/acceptance/boot_linux_console.py +5
-0+ initrd_url = ('https://github.com/groeck/linux-build-test/raw/' + '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/' + 'mips/rootfs.cpio.gz') + initrd_hash = 'bf806e17009360a866bf537f6de66590de349a99' + initrd_path_gz = self.fetch_asset(initrd_url,
asset_hash=initrd_hash)
tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 167
Quoted 2 lines
tests/acceptance/boot_linux_console.py +2
-0+ initrd_path = archive.extract(initrd_path_gz, + os.path.join(self.workdir,
'rootfs.cpio'))
Quoted 8 lines
tests/acceptance/boot_linux_console.py +8
-0+ self.vm.set_machine('malta') + self.vm.set_console() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyS0 console=tty ' + + 'rdinit=/sbin/init noreboot') + self.vm.add_args('-kernel', kernel_path, + '-initrd', initrd_path, + '-append', kernel_command_line,
tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test):
Quoted 10 lines
tests/acceptance/boot_linux_console.py +10
-0+ '-no-reboot') + self.vm.launch() + self.wait_for_console_pattern('Boot successful.') + + self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo', + 'BogoMIPS') + self.exec_command_and_wait_for_pattern('uname -a', + 'Debian') + self.exec_command_and_wait_for_pattern('reboot', + 'reboot: Restarting
system')
tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test):
Quoted 3 lines
tests/acceptance/boot_linux_console.py +1
-0+ def do_test_mips_malta32el_nanomips(self, kernel_url, kernel_hash): kernel_path_xz = self.fetch_asset(kernel_url,
asset_hash=kernel_hash)
Quoted 5 lines
tests/acceptance/boot_linux_console.py +0
-1kernel_path = self.workdir + "kernel" --2.20.1