Acceptance Tests: Test the Raspberry Pi 2 board
v2 cover Acceptance Tests: Test the Raspberry Pi 2 board
Add the raspi2 to the list of boards tested by the boot_linux_console Avocado test.
Based on "Acceptance Tests: target architecture support" v5:
https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04111.html
Based-on: <[email protected]>
Philippe Mathieu-Daudé (2):
BootLinuxConsoleTest: Let extract_from_deb handle various compressions Boot Linux Console Test: add a test for the Raspberry Pi 2
tests/acceptance/boot_linux_console.py | 34 ++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-)
Signature · 1 line
2.20.1
PATCH v2 1/2 BootLinuxConsoleTest: Let extract_from_deb handle various compressions
Debian binary package format supports various compressions.
Per man deb(5):
NAME
deb - Debian binary package format
FORMAT
...
The third, last required member is named data.tar. It contains the filesystem as a tar archive, either not compressed (supported since dpkg 1.10.24), or compressed with gzip (with .gz extension), xz (with .xz extension, supported since dpkg 1.15.6), bzip2 (with .bz2 extension, supported since dpkg 1.10.24) or lzma (with .lzma extension, supported since dpkg 1.13.25).
List the archive files to have the 3rd name with the correct extension.
The function avocado.utils.archive.extract() will handle the different compression format for us.
---
v2: Use process.run() (Cleber)
--- tests/acceptance/boot_linux_console.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tests/acceptance/boot_linux_console.py +3 -2
@@ -55,8 +55,9 @@ class BootLinuxConsole(Test): """ cwd = os.getcwd() os.chdir(self.workdir) - process.run("ar x %s data.tar.gz" % deb) - archive.extract("data.tar.gz", self.workdir) + file_path = process.run("ar t %s" % deb).stdout_text.split()[2] + process.run("ar x %s %s" % (deb, file_path)) + archive.extract(file_path, self.workdir) os.chdir(cwd) return self.workdir + path
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v2 1/2] BootLinuxConsoleTest: Let extract_from_deb handle various compressions
Quoted 16 lines · Philippe Mathieu-Daudé
Debian binary package format supports various compressions.
Per man deb(5):
NAME
deb - Debian binary package formatFORMAT
...
The third, last required member is named data.tar. It contains the filesystem as a tar archive, either not compressed (supported since dpkg 1.10.24), or compressed with gzip (with .gz extension), xz (with .xz extension, supported since dpkg 1.15.6), bzip2 (with .bz2 extension, supported since dpkg 1.10.24) or lzma (with .lzma extension, supported since dpkg 1.13.25).
Quoted 6 lines
List the archive files to have the 3rd name with the correct extension.
The function
avocado.utils.archive.extract()will handle the different compression format for us.Signed-off-by Philippe Mathieu-Daudé <[email protected]>
PATCH v2 2/2 Boot Linux Console Test: add a test for the Raspberry Pi 2
In reply to Philippe Mathieu-DaudéFrom: Philippe Mathieu-Daudé <[email protected]>
Similar to the x86_64/pc test, it boots a Linux kernel on a raspi2 board and verify the serial is working.
If ARM is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:arm" tags.
Alternatively, this test can be run using:
$ avocado run -t arch:arm tests/acceptance
$ avocado run -t machine:raspi2 tests/acceptance--- tests/acceptance/boot_linux_console.py | 29 ++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)
tests/acceptance/boot_linux_console.py +29 -0
@@ -197,6 +197,35 @@ class BootLinuxConsole(Test): console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) + def test_arm_raspi2(self): + """ + The kernel can be rebuilt using the kernel source referenced + and following the instructions on the on: + https://www.raspberrypi.org/documentation/linux/kernel/building.md + + :avocado: tags=arch:arm + :avocado: tags=machine:raspi2 + """ + deb_url = ('http://archive.raspberrypi.org/debian/' + 'pool/main/r/raspberrypi-firmware/' + 'raspberrypi-kernel_1.20190215-1_armhf.deb') + deb_hash = 'cd284220b32128c5084037553db3c482426f3972' + deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash) + print("deb_path: ", deb_path) + kernel_path = self.extract_from_deb(deb_path, '/boot/kernel7.img') + dtb_path = self.extract_from_deb(deb_path, '/boot/bcm2709-rpi-2-b.dtb') + + self.vm.set_machine('raspi2') + self.vm.set_console() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyAMA0') + self.vm.add_args('-kernel', kernel_path, + '-dtb', dtb_path, + '-append', kernel_command_line) + self.vm.launch() + console_pattern = 'Kernel command line: %s' % kernel_command_line + self.wait_for_console_pattern(console_pattern) + def test_s390x_s390_ccw_virtio(self): """ :avocado: tags=arch:s390x
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v2 2/2] Boot Linux Console Test: add a test for the Raspberry Pi 2
Quoted 31 lines · Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <[email protected]>
Similar to the x86_64/pc test, it boots a Linux kernel on a raspi2 board and verify the serial is working.
If ARM is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:arm" tags.
Alternatively, this test can be run using:
$ avocado run -t arch:arm tests/acceptance $ avocado run -t machine:raspi2 tests/acceptanceSigned-off-by Philippe Mathieu-Daudé <[email protected]>--- tests/acceptance/boot_linux_console.py | 29 ++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)tests/acceptance/boot_linux_console.py +5
-0@@ -197,6 +197,35 @@ class BootLinuxConsole(Test): console_pattern = 'Kernel command line: %s' % kernel_command_line self.wait_for_console_pattern(console_pattern) + def test_arm_raspi2(self): + """ + The kernel can be rebuilt using the kernel source referenced + and following the instructions on the on: + https://www.raspberrypi.org/documentation/linux/kernel/building.md
tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 214
Quoted 10 lines
tests/acceptance/boot_linux_console.py +10
-0+ + :avocado: tags=arch:arm + :avocado: tags=machine:raspi2 + """ + deb_url = ('http://archive.raspberrypi.org/debian/' + 'pool/main/r/raspberrypi-firmware/' + 'raspberrypi-kernel_1.20190215-1_armhf.deb') + deb_hash = 'cd284220b32128c5084037553db3c482426f3972' + deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash) + print("deb_path: ", deb_path)
This looks like a left over you forgot to remove.
Quoted 20 lines
tests/acceptance/boot_linux_console.py +14
-0+ kernel_path = self.extract_from_deb(deb_path, '/boot/kernel7.img') + dtb_path = self.extract_from_deb(deb_path, '/boot/bcm2709-rpi-2-b.dtb') + + self.vm.set_machine('raspi2') + self.vm.set_console() + kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE + + 'console=ttyAMA0') + self.vm.add_args('-kernel', kernel_path, + '-dtb', dtb_path, + '-append', kernel_command_line) + self.vm.launch() + console_pattern = 'Kernel command line: %s' % kernel_command_line + self.wait_for_console_pattern(console_pattern) + def test_s390x_s390_ccw_virtio(self): """ :avocado: tags=arch:s390xSignature · 1 line
2.20.1
Other than that, it looks good.
- Cleber.
Re: [Qemu-arm] [Qemu-devel] [PATCH v2 2/2] Boot Linux Console Test: add a test for the Raspberry Pi 2
In reply to Philippe Mathieu-DaudéQuoted 8 lines · Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <[email protected]>
Similar to the x86_64/pc test, it boots a Linux kernel on a raspi2 board and verify the serial is working.
If ARM is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:arm" tags.
Quoted 6 lines
Alternatively, this test can be run using:
$ avocado run -t arch:arm tests/acceptance $ avocado run -t machine:raspi2 tests/acceptanceSigned-off-by Philippe Mathieu-Daudé <[email protected]>
We're getting timeouts on travis-ci:
https://travis-ci.org/ehabkost/qemu/jobs/535468057#L3289
I don't know yet if the guest is hanging, or if we just need to increase the timeout. I could reproduce the timeout locally, too.
Signature · 1 line
Eduardo
Re: [Qemu-devel] [PATCH v2 2/2] Boot Linux Console Test: add a test for the Raspberry Pi 2
----- Original Message -----
Quoted 26 lines
From: "Eduardo Habkost" <[email protected]>
To: "Philippe Mathieu-Daudé" <[email protected]>
Cc: [email protected], "Cleber Rosa" <[email protected]>, "Peter Maydell" <[email protected]>, [email protected], "Philippe Mathieu-Daudé" <[email protected]>, "Andrew Baumann" <[email protected]>
Sent: Wednesday, May 22, 2019 4:52:34 PM
Subject: Re: [Qemu-devel] [PATCH v2 2/2] Boot Linux Console Test: add a test for the Raspberry Pi 2Quoted 8 lines · Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <[email protected]>
Similar to the x86_64/pc test, it boots a Linux kernel on a raspi2 board and verify the serial is working.
If ARM is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:arm" tags.
Quoted 6 lines
Alternatively, this test can be run using:
$ avocado run -t arch:arm tests/acceptance $ avocado run -t machine:raspi2 tests/acceptanceSigned-off-by Philippe Mathieu-Daudé <[email protected]>We're getting timeouts on travis-ci:
https://travis-ci.org/ehabkost/qemu/jobs/535468057#L3289
Quoted 7 lines
I don't know yet if the guest is hanging, or if we just need to increase the timeout. I could reproduce the timeout locally, too.
--
Eduardo
It may be related to:
https://bugs.launchpad.net/qemu/+bug/1829779
And this is a proof that we urgently need to have a better way of presenting/storing test artifacts. The brief output is nice when everything goes well, but makes the test results close to useless once a failure happens.
Philippe showed us how GitLab allows CI jobs to preserve artifacts, so maybe the solution is to move the loads there.
- Cleber.
Re: [Qemu-arm] [Qemu-devel] [PATCH v2 2/2] Boot Linux Console Test: add a test for the Raspberry Pi 2
Quoted 24 lines · Cleber Rosa
----- Original Message -----
Quoted 23 lines
From: "Eduardo Habkost" <[email protected]>
Quoted 8 lines · Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <[email protected]>
Similar to the x86_64/pc test, it boots a Linux kernel on a raspi2 board and verify the serial is working.
If ARM is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:arm" tags.
Quoted 6 lines
Alternatively, this test can be run using:
$ avocado run -t arch:arm tests/acceptance $ avocado run -t machine:raspi2 tests/acceptanceSigned-off-by Philippe Mathieu-Daudé <[email protected]>We're getting timeouts on travis-ci:
https://travis-ci.org/ehabkost/qemu/jobs/535468057#L3289I don't know yet if the guest is hanging, or if we just need to increase the timeout. I could reproduce the timeout locally, too.
That's odd, I can't reproduce (this test is quicker than the following test_arm_emcraft_sf2 which start u-boot then timeouts and start the kernel).
My guess is network issues, since this test use a different mirror:
archive.raspberrypi.org
Gerd already raised this problem (timeout reached while fetching artifacts) to Cleber.
Cleber, can we add test_setup() methods that use different timeouts?
Regards,
Phil.
Quoted 14 lines
It may be related to:
https://bugs.launchpad.net/qemu/+bug/1829779
And this is a proof that we urgently need to have a better way of presenting/storing test artifacts. The brief output is nice when everything goes well, but makes the test results close to useless once a failure happens.
Philippe showed us how GitLab allows CI jobs to preserve artifacts, so maybe the solution is to move the loads there.
- Cleber.
Re: [Qemu-arm] [Qemu-devel] [PATCH v2 2/2] Boot Linux Console Test: add a test for the Raspberry Pi 2
----- Original Message -----
Quoted 33 lines
From: "Philippe Mathieu-Daudé" <[email protected]>
To: "Cleber Rosa" <[email protected]>, "Eduardo Habkost" <[email protected]>
Cc: [email protected], "Peter Maydell" <[email protected]>, [email protected], "Philippe Mathieu-Daudé" <[email protected]>, "Andrew Baumann" <[email protected]>, "Gerd Hoffmann" <[email protected]>
Sent: Thursday, May 23, 2019 5:29:22 AM
Subject: Re: [Qemu-devel] [PATCH v2 2/2] Boot Linux Console Test: add a test for the Raspberry Pi 2Quoted 24 lines · Cleber Rosa
----- Original Message -----
Quoted 23 lines
From: "Eduardo Habkost" <[email protected]>
Quoted 8 lines · Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <[email protected]>
Similar to the x86_64/pc test, it boots a Linux kernel on a raspi2 board and verify the serial is working.
If ARM is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:arm" tags.
Quoted 6 lines
Alternatively, this test can be run using:
$ avocado run -t arch:arm tests/acceptance $ avocado run -t machine:raspi2 tests/acceptanceSigned-off-by Philippe Mathieu-Daudé <[email protected]>We're getting timeouts on travis-ci:
https://travis-ci.org/ehabkost/qemu/jobs/535468057#L3289I don't know yet if the guest is hanging, or if we just need to increase the timeout. I could reproduce the timeout locally, too.
Quoted 6 lines
That's odd, I can't reproduce (this test is quicker than the following
test_arm_emcraft_sf2which start u-boot then timeouts and start the kernel).My guess is network issues, since this test use a different mirror:
archive.raspberrypi.org
It could be a network issue, it could be something else. I think the very first step, and I'd urge us to get that on master ASAP, is to show the entire logs in CI, I mean:
---
.travis.yml +1 -1
@@ -225,7 +225,7 @@ matrix: # Acceptance (Functional) tests - env: - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu" - - TEST_CMD="make check-acceptance" + - TEST_CMD="make AVOCADO_SHOW=test check-acceptance" addons: apt: packages:
---
That way we can know for sure what's going on.
Quoted 4 lines
Gerd already raised this problem (timeout reached while fetching artifacts) to Cleber.
Cleber, can we addtest_setup()methods that use different timeouts?
Not in a released Avocado version. Interestingly enough, I wrote a PoC that adds different timeouts to tearDown()[1], that can be used the same way for setUp(), and it looks like Intel DAOS is already using those patches[2].
So, I guess I can work on a non-PoC version for that.
- Cleber.
[1] - https://github.com/avocado-framework/avocado/pull/3076
[2] - https://github.com/daos-stack/daos/commit/084ec23461e7bd9b997d4b6f5e8095a4eaffc685
Quoted 19 lines
Regards,
Phil.
Quoted 14 lines
It may be related to:
https://bugs.launchpad.net/qemu/+bug/1829779
And this is a proof that we urgently need to have a better way of presenting/storing test artifacts. The brief output is nice when everything goes well, but makes the test results close to useless once a failure happens.
Philippe showed us how GitLab allows CI jobs to preserve artifacts, so maybe the solution is to move the loads there.
- Cleber.
Re: [Qemu-arm] [Qemu-devel] [PATCH v2 2/2] Boot Linux Console Test: add a test for the Raspberry Pi 2
Quoted 52 lines · Cleber Rosa
----- Original Message -----
Quoted 33 lines
From: "Philippe Mathieu-Daudé" <[email protected]>
To: "Cleber Rosa" <[email protected]>, "Eduardo Habkost" <[email protected]>
Cc: [email protected], "Peter Maydell" <[email protected]>, [email protected], "Philippe Mathieu-Daudé" <[email protected]>, "Andrew Baumann" <[email protected]>, "Gerd Hoffmann" <[email protected]>
Sent: Thursday, May 23, 2019 5:29:22 AM
Subject: Re: [Qemu-devel] [PATCH v2 2/2] Boot Linux Console Test: add a test for the Raspberry Pi 2Quoted 24 lines · Cleber Rosa
----- Original Message -----
Quoted 23 lines
From: "Eduardo Habkost" <[email protected]>
Quoted 8 lines · Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <[email protected]>
Similar to the x86_64/pc test, it boots a Linux kernel on a raspi2 board and verify the serial is working.
If ARM is a target being built, "make check-acceptance" will automatically include this test by the use of the "arch:arm" tags.
Quoted 6 lines
Alternatively, this test can be run using:
$ avocado run -t arch:arm tests/acceptance $ avocado run -t machine:raspi2 tests/acceptanceSigned-off-by Philippe Mathieu-Daudé <[email protected]>We're getting timeouts on travis-ci:
https://travis-ci.org/ehabkost/qemu/jobs/535468057#L3289I don't know yet if the guest is hanging, or if we just need to increase the timeout. I could reproduce the timeout locally, too.
Quoted 6 lines
That's odd, I can't reproduce (this test is quicker than the following
test_arm_emcraft_sf2which start u-boot then timeouts and start the kernel).My guess is network issues, since this test use a different mirror:
archive.raspberrypi.orgIt could be a network issue, it could be something else. I think the very first step, and I'd urge us to get that on master ASAP, is to show the entire logs in CI, I mean:
---
.travis.yml +0
-0
.travis.yml matrix: · line 231
Quoted 8 lines
.travis.yml +1
-1@@ -225,7 +225,7 @@ matrix: # Acceptance (Functional) tests - env: - CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu" - - TEST_CMD="make check-acceptance" + - TEST_CMD="make AVOCADO_SHOW=test check-acceptance" addons: apt: packages:
I have applied this change on python-next. The failure can be seen here:
https://travis-ci.org/ehabkost/qemu/jobs/541758418#L4033
While we don't fix this, I'm removing this test case from the queue, so I can send a pull request. Sorry for taking so long to act on this.
Signature · 1 line
Eduardo
Quoted 7 lines · Philippe Mathieu-Daudé
Add the raspi2 to the list of boards tested by the
boot_linux_consoleAvocado test.Based on "Acceptance Tests: target architecture support" v5:
https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04111.html
Based-on: <[email protected]>
Quoted 6 lines
Philippe Mathieu-Daudé (2):
BootLinuxConsoleTest: Letextract_from_debhandle various compressions Boot Linux Console Test: add a test for the Raspberry Pi 2tests/acceptance/boot_linux_console.py | 34 ++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-)
I was hoping this series would be included in "Acceptance Tests: target architecture support" by Eduardo (removing the 'print' debug line) but I just realized I forgot to Cc him :S
Quoted 12 lines · Philippe Mathieu-Daudé
Quoted 11 lines · Philippe Mathieu-Daudé
Add the raspi2 to the list of boards tested by the
boot_linux_consoleAvocado test.Based on "Acceptance Tests: target architecture support" v5:
https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04111.html
Based-on: <[email protected]>Philippe Mathieu-Daudé (2):
BootLinuxConsoleTest: Letextract_from_debhandle various compressions Boot Linux Console Test: add a test for the Raspberry Pi 2
Quoted 6 lines
Quoted 2 lines
tests/acceptance/boot_linux_console.py | 34 ++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-)I was hoping this series would be included in "Acceptance Tests: target architecture support" by Eduardo (removing the 'print' debug line) but I just realized I forgot to Cc him :S
Problem solved: queued on python-next (after removing the print statement from patch 2/2). :)
Signature · 1 line
Eduardo
Quoted 16 lines · Eduardo Habkost
Quoted 15 lines · Philippe Mathieu-Daudé
Quoted 13 lines · Philippe Mathieu-Daudé
Add the raspi2 to the list of boards tested by the
boot_linux_consoleAvocado test.Based on "Acceptance Tests: target architecture support" v5:
https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04111.html
Based-on: <[email protected]>Philippe Mathieu-Daudé (2):
BootLinuxConsoleTest: Letextract_from_debhandle various compressions Boot Linux Console Test: add a test for the Raspberry Pi 2tests/acceptance/boot_linux_console.py | 34 ++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-)
Quoted 6 lines
Quoted 3 lines
I was hoping this series would be included in "Acceptance Tests: target architecture support" by Eduardo (removing the 'print' debug line) but I just realized I forgot to Cc him :S
Problem solved: queued on python-next (after removing the print statement from patch 2/2). :)
Thanks a lot!