selftest improvements to respect IMAGE_LINK_NAME
cover selftest improvements to respect IMAGE_LINK_NAME
These changes are preparation for the actual changes from [YOCTO #12937], but without changing anything in current artifacts naming schema.
Only change outside selftest was moving of INITRAMFS_IMAGE_NAME variable and adding IMAGE_MACHINE_SUFFIX variable to make it easier to change both.
I've run complete selftest with my other changes for [YOCTO #12937] which highlighted where the issues are and then again with only these changes to make sure I didn't break it for current mickledore.
The most dangerous is probably:
git-submodule-test: disable upstream version check
because I cannot explain why distrodata.Distrodata.test_checkpkg isn't failing on AB
but maybe it's just because many (maybe all AB runs disable this check?), e.g.
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/4948/steps/14/logs/stdio
shows it skipped:
ERROR: Command . ./oe-init-build-env; ${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=/srv/autobuilder/autobuilder.yocto.io/pub/repro-fail/ DISPLAY=:1 oe-selftest -a --skip-tests distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T machine -T toolchain-user -T toolchain-system -j 15 failed with exit code 1, see errors above. (1678628531.8: 4253.0)
There were some tests which were skipped (even without --skip-tests) or failed in my oe-selftest -a run:
== SKIPPED ==
A) INFO - ... skipped 'Test needs meta-oe for nodejs' due to testing without meta-oe
2023-03-12 12:08:44,526 - oe-selftest - INFO - RESULTS - devtool.DevtoolAddTests.test_devtool_add_npm: SKIPPED (2.22s) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolCreateTests.test_recipetool_create_npm: SKIPPED (2.83s)
B) INFO - ... skipped 'No "dash" found on test system.' no dash on gentoo
2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: SKIPPED (0.00s)
C) INFO - ... skipped 'Target does not use libssp' not testing on mingw32 with libssp:
2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - gcc.GccLibSspSelfTest.test_libssp: SKIPPED (3.47s) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - gcc.GccLibSspSelfTestSystemEmulated.test_libssp: SKIPPED (4.23s)
D) INFO - ... skipped 'You must set up tap devices with scripts/runqemu-gen-tapdevs before running this test' even when the tapdevs were correctly configured with runqemu-gen-tapdevs before oe-selftest was started later resolved by "touch /etc/runqemu-nosudo" and it PASSED,
https://git.openembedded.org/openembedded-core/commit/?id=2258345e19efff7717fe19a5026ec55f1b6f90b6
looks suspicious and will try to improve it (to also ignore
skipped taps like /tmp/qemu-tap-locks/tap0.skip which I'm using
because tap0 is already used by openvpn running on my builder)
2023-03-12 12:08:44,526 - oe-selftest - INFO - RESULTS - devtool.DevtoolExtractTests.test_devtool_deploy_target: SKIPPED (2.33s)
== FAILED ==
1) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - distrodata.Distrodata.test_checkpkg: FAILED (146.68s)
2023-03-13 08:57:10,817 - oe-selftest - INFO - Traceback (most recent call last):
File "/OE/build/poky/meta/lib/oeqa/selftest/cases/distrodata.py", line 40, in test_checkpkg
self.assertTrue(len(regressed_failures) == 0 and len(regressed_successes) == 0, msg)
AssertionError: False is not true :
The following packages failed upstream version checks. Please fix them using UPSTREAM_CHECK_URI/UPSTREAM_CHECK_REGEX
(when using tarballs) or UPSTREAM_CHECK_GITTAGREGEX (when using git). If an upstream version check cannot be performed
(for example, if upstream does not use git tags), you can set UPSTREAM_VERSION_UNKNOWN to '1' in the recipe to acknowledge
that the check cannot be performed.
binutilsused to show git-submodule-test as well (without the first patch here) I have no explanation why this isn't triggered on AB, when I've added more debug in this text, the binutils disappeared as well (so I was assuming it was just temporary network glitch, but today it returned again - with debug already removed).
2) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - fetch.Fetch.test_git_mirrors: FAILED (24.34s)
This is a limitation of current selftest as discussed with RP on IRC.
easy to explain, but more difficult to fix, it sets (PRE)MIRRORS:forcevariable = "" and expects to trigger fetch failure from dbus-wait, but PREMIRRORS:prepend in my local.conf defeats it and dbus-wait is fetched successfully.
I can switch from prepend to =+ in my local.conf but that will solve it only in my environment.
RP suggested to change selftest to ignore user's local.conf or pick only some specific key things from it, but current state is what it is and after removing PREMIRRORS:prepend this test PASSED as well.
3) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline_weston: FAILED (2043.13s)
2023-03-13 09:01:25,976 - oe-selftest - INFO - ... FAIL
2023-03-13 09:01:25,977 - oe-selftest - INFO - Traceback (most recent call last):
File "/OE/build/poky/meta/lib/oeqa/selftest/cases/incompatible_lic.py", line 161, in test_core_image_full_cmdline_weston
bitbake('-c testimage core-image-full-cmdline core-image-weston')
File "/OE/build/poky/meta/lib/oeqa/utils/commands.py", line 236, in bitbake
return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/OE/build/poky/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'bitbake -c testimage core-image-full-cmdline core-image-weston' returned non-zero exit status 1:seems to run many tests succesfully with the test image:
core-image-weston () - Ran 71 tests in 34.038s
core-image-weston - OK - All required tests passed (successes=14, skipped=57, failures=0, errors=0)
NOTE: recipe core-image-weston-1.0-r0: task do_testimage: Succeeded
NOTE: Tasks Summary: Attempted 1141 tasks of which 1139 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/OE/build/poky/meta/recipes-extended/images/core-image-full-cmdline.bb:do_testimage
Summary: There were 2 WARNING messages.
Summary: There were 3 ERROR messages, returning a non-zero exit code.
but there is also:
runqemu - INFO - Found /tmp/qemu-tap-locks/tap0.skip, skipping tap0
runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap1.lock failed: [Errno 11] Resource temporarily unavailable
I need to debug it a bit more, probably outside selftest as well, will try to run it when I don't need tap0 for openvpn.
But it fails the same with current master without any of my changes as well.
4) runqemu.RunqemuTests still fail with my other [YOCTO #12937] changes but work fine with this pull request, I have some WIP changes locally but the testimage+runqemu interaction is far from simple and will need more time to debug to cover all possible use-cases for runqemu
runqemu.RunqemuTests.test_boot_machine_ext4.log:2023-03-13 09:03:12,008 - oe-selftest - INFO - RESULTS - runqemu.RunqemuTests.test_boot_machine_ext4: FAILED (63.37s) runqemu.RunqemuTests.test_boot_recipe_image.log:2023-03-13 08:31:06,400 - oe-selftest - INFO - RESULTS - runqemu.RunqemuTests.test_boot_recipe_image: ERROR (47.56s) runqemu.RunqemuTests.test_boot_recipe_image_vdi.log:2023-03-13 08:32:06,489 - oe-selftest - INFO - RESULTS - runqemu.RunqemuTests.test_boot_recipe_image_vdi: ERROR (46.82s) runqemu.RunqemuTests.test_boot_recipe_image_vmdk.log:2023-03-13 08:33:07,020 - oe-selftest - INFO - RESULTS - runqemu.RunqemuTests.test_boot_recipe_image_vmdk: ERROR (47.30s)
The following changes since commit 65ca2c4b7349a4f7dcfcc580d926a99c673deb60:
dhcpcd: Fix install conflict when enable multilib. (2023-03-11 00:08:38 +0000)
are available in the Git repository at:
https://git.openembedded.org/openembedded-core-contrib jansa/pull
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=jansa/pull
Martin Jansa (14):
git-submodule-test: disable upstream version check
selftest: devtool: set BB_HASHSERVE_UPSTREAM when setting
SSTATE_MIRROR
selftest: wic: respect IMAGE_LINK_NAME
selftest: wic: respect IMAGE_LINK_NAME also in
test_rawcopy_plugin_qemu
selftest: runqemu: respect IMAGE_LINK_NAME
selftest: multiconfig-image-packager: try to respect IMAGE_LINK_NAME image-artifact-names.bbclass: add INITRAMFS_IMAGE_NAME from
kernel.bbclass
selftest: fitimage.py: respect INITRAMFS_IMAGE_NAME and
KERNEL_FIT_LINK_NAME
image-artifact-names: add IMAGE_MACHINE_SUFFIX variable
selftest: gdbserver.py: respect IMAGE_LINK_NAME
selftest: minidebuginfo.py respect IMAGE_LINK_NAME
selftest: efibootpartition.py: fix QEMU_USE_KVM usage
runqemu: get_first_file() rename cmd* to glob*
selftest: imagefeatures.py: respect IMAGE_LINK_NAME for debugfs and
manifest as well
.../git-submodule-test/git-submodule-test.bb | 2 + .../multiconfig-image-packager_0.1.bb | 18 +++- meta-selftest/wic/test_rawcopy_plugin.wks.in | 2 +- .../image-artifact-names.bbclass | 16 +++- .../kernel-artifact-names.bbclass | 2 +- meta/classes-recipe/kernel.bbclass | 1 - meta/lib/oeqa/selftest/cases/devtool.py | 1 + .../oeqa/selftest/cases/efibootpartition.py | 4 +- meta/lib/oeqa/selftest/cases/fitimage.py | 34 ++++---- meta/lib/oeqa/selftest/cases/gdbserver.py | 9 +- meta/lib/oeqa/selftest/cases/imagefeatures.py | 82 +++++++++---------- meta/lib/oeqa/selftest/cases/minidebuginfo.py | 15 ++-- meta/lib/oeqa/selftest/cases/runqemu.py | 10 ++- meta/lib/oeqa/selftest/cases/wic.py | 44 +++++----- scripts/runqemu | 36 ++++---- 15 files changed, 155 insertions(+), 121 deletions(-)
Signature · 1 line
2.39.2
Tests in distrodata that contact upstream servers had to be disabled because it's not possible to tell what is a real failure, and what is a transient upstream issue (with expired certificates or other network issues). To ensure upstream version checks don't regress, I run them manually every now and then.
Alex
Quoted 182 lines · Martin Jansa
These changes are preparation for the actual changes from [YOCTO #12937], but without changing anything in current artifacts naming schema.
Only change outside selftest was moving of
INITRAMFS_IMAGE_NAMEvariable and addingIMAGE_MACHINE_SUFFIXvariable to make it easier to change both.I've run complete selftest with my other changes for [YOCTO #12937] which highlighted where the issues are and then again with only these changes to make sure I didn't break it for current mickledore.
The most dangerous is probably:
git-submodule-test: disable upstream version check
because I cannot explain why distrodata.Distrodata.test_checkpkg isn't failing on ABbut maybe it's just because many (maybe all AB runs disable this check?), e.g.
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/4948/steps/14/logs/stdio
shows it skipped:
ERROR: Command . ./oe-init-build-env; ${SCRIPTSDIR}/checkvnc; OEQA_DEBUGGING_SAVED_OUTPUT=/srv/autobuilder/autobuilder.yocto.io/pub/repro-fail/ DISPLAY=:1 oe-selftest -a --skip-tests distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T machine -T toolchain-user -T toolchain-system -j 15 failed with exit code 1, see errors above. (1678628531.8: 4253.0)There were some tests which were skipped (even without --skip-tests) or failed in my oe-selftest -a run:
== SKIPPED ==
A) INFO - ... skipped 'Test needs meta-oe for nodejs' due to testing without meta-oe2023-03-12 12:08:44,526 - oe-selftest - INFO - RESULTS - devtool.DevtoolAddTests.test_devtool_add_npm: SKIPPED (2.22s) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolCreateTests.test_recipetool_create_npm: SKIPPED (2.83s)B) INFO - ... skipped 'No "dash" found on test system.' no dash on gentoo
2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: SKIPPED (0.00s)C) INFO - ... skipped 'Target does not use libssp' not testing on mingw32 with libssp:
2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - gcc.GccLibSspSelfTest.test_libssp: SKIPPED (3.47s) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - gcc.GccLibSspSelfTestSystemEmulated.test_libssp: SKIPPED (4.23s)D) INFO - ... skipped 'You must set up tap devices with
scripts/runqemu-gen-tapdevsbefore running this test' even when the tapdevs were correctly configured with runqemu-gen-tapdevs before oe-selftest was started later resolved by "touch/etc/runqemu-nosudo" and it PASSED,
https://git.openembedded.org/openembedded-core/commit/?id=2258345e19efff7717fe19a5026ec55f1b6f90b6
looks suspicious and will try to improve it (to also ignore
skipped taps like/tmp/qemu-tap-locks/tap0.skipwhich I'm using
because tap0 is already used by openvpn running on my builder)2023-03-12 12:08:44,526 - oe-selftest - INFO - RESULTS - devtool.DevtoolExtractTests.test_devtool_deploy_target: SKIPPED (2.33s)== FAILED ==
1) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - distrodata.Distrodata.test_checkpkg: FAILED (146.68s)2023-03-13 08:57:10,817 - oe-selftest - INFO - Traceback (most recent call last): File "/OE/build/poky/meta/lib/oeqa/selftest/cases/distrodata.py", line 40, in test_checkpkg self.assertTrue(len(regressed_failures) == 0 and len(regressed_successes) == 0, msg) AssertionError: False is not true : The following packages failed upstream version checks. Please fix them using UPSTREAM_CHECK_URI/UPSTREAM_CHECK_REGEX (when using tarballs) or UPSTREAM_CHECK_GITTAGREGEX (when using git). If an upstream version check cannot be performed (for example, if upstream does not use git tags), you can set UPSTREAM_VERSION_UNKNOWN to '1' in the recipe to acknowledge that the check cannot be performed. binutilsused to show git-submodule-test as well (without the first patch here) I have no explanation why this isn't triggered on AB, when I've added more debug in this text, the binutils disappeared as well (so I was assuming it was just temporary network glitch, but today it returned again - with debug already removed).
2) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - fetch.Fetch.test_git_mirrors: FAILED (24.34s)
This is a limitation of current selftest as discussed with RP on IRC.
easy to explain, but more difficult to fix, it sets (PRE)MIRRORS:forcevariable = "" and expects to trigger fetch failure from dbus-wait, but PREMIRRORS:prepend in my local.conf defeats it and dbus-wait is fetched successfully.
I can switch from prepend to =+ in my local.conf but that will solve it only in my environment.
RP suggested to change selftest to ignore user's local.conf or pick only some specific key things from it, but current state is what it is and after removing PREMIRRORS:prepend this test PASSED as well.
3) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS - incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline_weston: FAILED (2043.13s)
2023-03-13 09:01:25,976 - oe-selftest - INFO - ... FAIL 2023-03-13 09:01:25,977 - oe-selftest - INFO - Traceback (most recent call last): File "/OE/build/poky/meta/lib/oeqa/selftest/cases/incompatible_lic.py", line 161, in test_core_image_full_cmdline_weston bitbake('-c testimage core-image-full-cmdline core-image-weston') File "/OE/build/poky/meta/lib/oeqa/utils/commands.py", line 236, in bitbake return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/OE/build/poky/meta/lib/oeqa/utils/commands.py", line 214, in runCmd raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output)) AssertionError: Command 'bitbake -c testimage core-image-full-cmdline core-image-weston' returned non-zero exit status 1:seems to run many tests succesfully with the test image:
core-image-weston () - Ran 71 tests in 34.038s
core-image-weston - OK - All required tests passed (successes=14, skipped=57, failures=0, errors=0)
NOTE: recipe core-image-weston-1.0-r0: taskdo_testimage: Succeeded
NOTE: Tasks Summary: Attempted 1141 tasks of which 1139 didn't need to be rerun and 1 failed.Summary: 1 task failed:
/OE/build/poky/meta/recipes-extended/images/core-image-full-cmdline.bb:do_testimage
Summary: There were 2 WARNING messages.
Summary: There were 3 ERROR messages, returning a non-zero exit code.but there is also:
runqemu - INFO - Found/tmp/qemu-tap-locks/tap0.skip, skipping tap0
runqemu - INFO - Acquiring lockfile/tmp/qemu-tap-locks/tap1.lockfailed: [Errno 11] Resource temporarily unavailableI need to debug it a bit more, probably outside selftest as well, will try to run it when I don't need tap0 for openvpn.
But it fails the same with current master without any of my changes as well.4) runqemu.RunqemuTests still fail with my other [YOCTO #12937] changes but work fine with this pull request, I have some WIP changes locally but the testimage+runqemu interaction is far from simple and will need more time to debug to cover all possible use-cases for runqemu
runqemu.RunqemuTests.test_boot_machine_ext4.log:2023-03-13 09:03:12,008 - oe-selftest - INFO - RESULTS - runqemu.RunqemuTests.test_boot_machine_ext4: FAILED (63.37s) runqemu.RunqemuTests.test_boot_recipe_image.log:2023-03-13 08:31:06,400 - oe-selftest - INFO - RESULTS - runqemu.RunqemuTests.test_boot_recipe_image: ERROR (47.56s) runqemu.RunqemuTests.test_boot_recipe_image_vdi.log:2023-03-13 08:32:06,489 - oe-selftest - INFO - RESULTS - runqemu.RunqemuTests.test_boot_recipe_image_vdi: ERROR (46.82s) runqemu.RunqemuTests.test_boot_recipe_image_vmdk.log:2023-03-13 08:33:07,020 - oe-selftest - INFO - RESULTS - runqemu.RunqemuTests.test_boot_recipe_image_vmdk: ERROR (47.30s)
The following changes since commit
65ca2c4b7349a4f7dcfcc580d926a99c673deb60:dhcpcd: Fix install conflict when enable multilib. (2023-03-11 00:08:38 +0000)
are available in the Git repository at:
https://git.openembedded.org/openembedded-core-contrib jansa/pull
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=jansa/pullMartin Jansa (14):
git-submodule-test: disable upstream version check
selftest: devtool: setBB_HASHSERVE_UPSTREAMwhen setting
SSTATE_MIRROR
selftest: wic: respectIMAGE_LINK_NAME
selftest: wic: respectIMAGE_LINK_NAMEalso in
test_rawcopy_plugin_qemu
selftest: runqemu: respectIMAGE_LINK_NAME
selftest: multiconfig-image-packager: try to respectIMAGE_LINK_NAMEimage-artifact-names.bbclass: addINITRAMFS_IMAGE_NAMEfrom
kernel.bbclass
selftest:fitimage.py: respectINITRAMFS_IMAGE_NAMEand
KERNEL_FIT_LINK_NAME
image-artifact-names: addIMAGE_MACHINE_SUFFIXvariable
selftest:gdbserver.py: respectIMAGE_LINK_NAME
selftest:minidebuginfo.pyrespectIMAGE_LINK_NAME
selftest:efibootpartition.py: fixQEMU_USE_KVMusage
runqemu:get_first_file()rename cmd* to glob*
selftest:imagefeatures.py: respectIMAGE_LINK_NAMEfor debugfs and
manifest as well.../git-submodule-test/git-submodule-test.bb | 2 + .../multiconfig-image-packager_0.1.bb | 18 +++- meta-selftest/wic/test_rawcopy_plugin.wks.in | 2 +- .../image-artifact-names.bbclass | 16 +++- .../kernel-artifact-names.bbclass | 2 +- meta/classes-recipe/kernel.bbclass | 1 - meta/lib/oeqa/selftest/cases/devtool.py | 1 + .../oeqa/selftest/cases/efibootpartition.py | 4 +- meta/lib/oeqa/selftest/cases/fitimage.py | 34 ++++---- meta/lib/oeqa/selftest/cases/gdbserver.py | 9 +- meta/lib/oeqa/selftest/cases/imagefeatures.py | 82 +++++++++---------- meta/lib/oeqa/selftest/cases/minidebuginfo.py | 15 ++-- meta/lib/oeqa/selftest/cases/runqemu.py | 10 ++- meta/lib/oeqa/selftest/cases/wic.py | 44 +++++----- scripts/runqemu | 36 ++++---- 15 files changed, 155 insertions(+), 121 deletions(-)--
2.39.2-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178432): https://lists.openembedded.org/g/openembedded-core/message/178432
Mute This Topic: https://lists.openembedded.org/mt/97578950/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 6 lines · Alexander Kanavin
Tests in distrodata that contact upstream servers had to be disabled because it's not possible to tell what is a real failure, and what is a transient upstream issue (with expired certificates or other network issues). To ensure upstream version checks don't regress, I run them manually every now and then.
Thanks, can you confirm that git-submodule-test is currently failing for you (and probably was for long time) and
that binutils is often reporting failure?
Quoted 245 lines · Martin Jansa
Quoted 6 lines
These changes are preparation for the actual changes from [YOCTO #12937], but without changing anything in current artifacts naming schema.
Only change outside selftest was moving of
INITRAMFS_IMAGE_NAMEvariable and addingIMAGE_MACHINE_SUFFIXvariable to make it easier to changeboth.
Quoted 8 lines
I've run complete selftest with my other changes for [YOCTO #12937] which highlighted where the issues are and then again with only these changes to make sure I didn't break it for current mickledore.
The most dangerous is probably:
git-submodule-test: disable upstream version check
because I cannot explain why distrodata.Distrodata.test_checkpkg isn'tfailing on AB
Quoted 2 lines
but maybe it's just because many (maybe all AB runs disable this
check?), e.g.
Quoted 1 line
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/4948/steps/14/logs/stdio
Quoted 2 lines
shows it skipped:
ERROR: Command . ./oe-init-build-env; ${SCRIPTSDIR}/checkvnc;OEQA_DEBUGGING_SAVED_OUTPUT=/srv/autobuilder/ autobuilder.yocto.io/pub/repro-fail/ DISPLAY=:1 oe-selftest -a --skip-tests distrodata.Distrodata.test_checkpkg buildoptions.SourceMirroring.test_yocto_source_mirror reproducible -T machine -T toolchain-user -T toolchain-system -j 15 failed with exit code 1, see errors above. (1678628531.8: 4253.0)
Quoted 2 lines
There were some tests which were skipped (even without --skip-tests) or
failed in my oe-selftest -a run:
Quoted 5 lines
== SKIPPED ==
A) INFO - ... skipped 'Test needs meta-oe for nodejs' due to testing without meta-oe2023-03-12 12:08:44,526 - oe-selftest - INFO - RESULTS -devtool.DevtoolAddTests.test_devtool_add_npm: SKIPPED (2.22s)
Quoted 1 line
2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS -recipetool.RecipetoolCreateTests.test_recipetool_create_npm: SKIPPED (2.83s)
Quoted 4 lines
B) INFO - ... skipped 'No "dash" found on test system.' no dash on gentoo
2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS -oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: SKIPPED (0.00s)
Quoted 4 lines
C) INFO - ... skipped 'Target does not use libssp' not testing on mingw32 with libssp:
2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS -gcc.GccLibSspSelfTest.test_libssp: SKIPPED (3.47s)
Quoted 1 line
2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS -gcc.GccLibSspSelfTestSystemEmulated.test_libssp: SKIPPED (4.23s)
Quoted 2 lines
D) INFO - ... skipped 'You must set up tap devices with
scripts/runqemu-gen-tapdevsbefore running this test'Quoted 1 line
even when the tapdevs were correctly configured with
runqemu-gen-tapdevs before oe-selftest was started
Quoted 2 lines
later resolved by "touch
/etc/runqemu-nosudo" and it PASSED,https://git.openembedded.org/openembedded-core/commit/?id=2258345e19efff7717fe19a5026ec55f1b6f90b6
Quoted 4 lines
looks suspicious and will try to improve it (to also ignore skipped taps like
/tmp/qemu-tap-locks/tap0.skipwhich I'm using because tap0 is already used by openvpn running on my builder)2023-03-12 12:08:44,526 - oe-selftest - INFO - RESULTS -devtool.DevtoolExtractTests.test_devtool_deploy_target: SKIPPED (2.33s)
Quoted 3 lines
== FAILED ==
1) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS -distrodata.Distrodata.test_checkpkg: FAILED (146.68s)
Quoted 2 lines
2023-03-13 08:57:10,817 - oe-selftest - INFO - Traceback (most recentcall last):
Quoted 1 line
File "/OE/build/poky/meta/lib/oeqa/selftest/cases/distrodata.py",
line 40, in
test_checkpkgQuoted 1 line
self.assertTrue(len(regressed_failures) == 0 and
len(regressed_successes)== 0, msg)Quoted 2 lines
AssertionError: False is not true :
The following packages failed upstream version checks. Please fix themusing UPSTREAM_CHECK_URI/UPSTREAM_CHECK_REGEX
Quoted 1 line
(when using tarballs) or
UPSTREAM_CHECK_GITTAGREGEX(when using git).If an upstream version check cannot be performed
Quoted 1 line
(for example, if upstream does not use git tags), you can set
UPSTREAM_VERSION_UNKNOWNto '1' in the recipe to acknowledgeQuoted 10 lines
that the check cannot be performed.
binutilsused to show git-submodule-test as well (without the first patch here) I have no explanation why this isn't triggered on AB, when I've added more debug in this text, the binutils disappeared as well (so I was assuming it was just temporary network glitch, but today it returned again - with debug already removed).
2) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS -
fetch.Fetch.test_git_mirrors: FAILED (24.34s)
Quoted 4 lines
This is a limitation of current selftest as discussed with RP on IRC.
easy to explain, but more difficult to fix, it sets
(PRE)MIRRORS:forcevariable = ""
Quoted 1 line
and expects to trigger fetch failure from dbus-wait, but
PREMIRRORS:prepend in my
Quoted 3 lines
local.conf defeats it and dbus-wait is fetched successfully.
I can switch from prepend to =+ in my local.conf but that will solve
it only in my environment.
Quoted 7 lines
RP suggested to change selftest to ignore user's local.conf or pick only some specific key things from it, but current state is what it is and after removing PREMIRRORS:prepend this test PASSED as well.
3) 2023-03-12 12:08:44,527 - oe-selftest - INFO - RESULTS -
incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline_weston:
FAILED (2043.13s)Quoted 3 lines
2023-03-13 09:01:25,976 - oe-selftest - INFO - ... FAIL 2023-03-13 09:01:25,977 - oe-selftest - INFO - Traceback (most recentcall last):
Quoted 1 line
File
"/OE/build/poky/meta/lib/oeqa/selftest/cases/incompatible_lic.py", line 161, in
test_core_image_full_cmdline_westonQuoted 2 lines
bitbake('-c testimage core-image-full-cmdline core-image-weston')
File "/OE/build/poky/meta/lib/oeqa/utils/commands.py", line 236, inbitbake
Quoted 1 line
return runCmd(cmd,
ignore_status, timeout, output_log=output_log,**options)
Quoted 1 line
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 1 line
File "/OE/build/poky/meta/lib/oeqa/utils/commands.py", line 214, in
runCmd
Quoted 1 line
raise AssertionError("Command '%s' returned non-zero exit status
%d:\n%s" % (command, result.status,
exc_output))Quoted 1 line
AssertionError: Command 'bitbake -c testimage core-image-full-cmdline
core-image-weston' returned non-zero exit status 1:
Quoted 5 lines
seems to run many tests succesfully with the test image:
core-image-weston () - Ran 71 tests in 34.038s core-image-weston - OK - All required tests passed (successes=14,
skipped=57, failures=0, errors=0)
Quoted 2 lines
NOTE: recipe core-image-weston-1.0-r0: task
do_testimage: Succeeded
NOTE: Tasks Summary: Attempted 1141 tasks of which 1139 didn't need tobe rerun and 1 failed.
Quoted 3 lines
Summary: 1 task failed:
/OE/build/poky/meta/recipes-extended/images/core-image-full-cmdline.bb:
do_testimageQuoted 6 lines
Summary: There were 2 WARNING messages.
Summary: There were 3 ERROR messages, returning a non-zero exit code.but there is also:
runqemu - INFO - Found/tmp/qemu-tap-locks/tap0.skip, skipping tap0 runqemu - INFO - Acquiring lockfile/tmp/qemu-tap-locks/tap1.lockfailed: [Errno 11] Resource temporarily unavailable
Quoted 2 lines
I need to debug it a bit more, probably outside selftest as well, will
try
Quoted 2 lines
to run it when I don't need tap0 for openvpn.
But it fails the same with current master without any of my changes aswell.
Quoted 7 lines
4) runqemu.RunqemuTests still fail with my other [YOCTO #12937] changes but work fine with this pull request, I have some WIP changes locally but the testimage+runqemu interaction is far from simple and will need more time to debug to cover all possible use-cases for runqemu
runqemu.RunqemuTests.test_boot_machine_ext4.log:2023-03-13 09:03:12,008
- oe-selftest - INFO - RESULTS - runqemu.RunqemuTests.test_boot_machine_ext4: FAILED (63.37s)
Quoted 1 line
runqemu.RunqemuTests.test_boot_recipe_image.log:2023-03-13 08:31:06,400
- oe-selftest - INFO - RESULTS - runqemu.RunqemuTests.test_boot_recipe_image: ERROR (47.56s)
Quoted 1 line
runqemu.RunqemuTests.test_boot_recipe_image_vdi.log:2023-03-13
08:32:06,489 - oe-selftest - INFO - RESULTS - runqemu.RunqemuTests.test_boot_recipe_image_vdi: ERROR (46.82s)
Quoted 1 line
runqemu.RunqemuTests.test_boot_recipe_image_vmdk.log:2023-03-13
08:33:07,020 - oe-selftest - INFO - RESULTS - runqemu.RunqemuTests.test_boot_recipe_image_vmdk: ERROR (47.30s)
Quoted 2 lines
The following changes since commit
65ca2c4b7349a4f7dcfcc580d926a99c673deb60:Quoted 2 lines
dhcpcd: Fix install conflict when enable multilib. (2023-03-11
00:08:38 +0000)
Quoted 5 lines
are available in the Git repository at:
https://git.openembedded.org/openembedded-core-contrib jansa/pull
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=jansa/pull
Quoted 39 lines
Martin Jansa (14):
git-submodule-test: disable upstream version check
selftest: devtool: setBB_HASHSERVE_UPSTREAMwhen setting
SSTATE_MIRROR
selftest: wic: respectIMAGE_LINK_NAME
selftest: wic: respectIMAGE_LINK_NAMEalso in
test_rawcopy_plugin_qemu
selftest: runqemu: respectIMAGE_LINK_NAME
selftest: multiconfig-image-packager: try to respectIMAGE_LINK_NAMEimage-artifact-names.bbclass: addINITRAMFS_IMAGE_NAMEfrom
kernel.bbclass
selftest:fitimage.py: respectINITRAMFS_IMAGE_NAMEand
KERNEL_FIT_LINK_NAME
image-artifact-names: addIMAGE_MACHINE_SUFFIXvariable
selftest:gdbserver.py: respectIMAGE_LINK_NAME
selftest:minidebuginfo.pyrespectIMAGE_LINK_NAME
selftest:efibootpartition.py: fixQEMU_USE_KVMusage
runqemu:get_first_file()rename cmd* to glob*
selftest:imagefeatures.py: respectIMAGE_LINK_NAMEfor debugfs and
manifest as well.../git-submodule-test/git-submodule-test.bb | 2 + .../multiconfig-image-packager_0.1.bb | 18 +++- meta-selftest/wic/test_rawcopy_plugin.wks.in | 2 +- .../image-artifact-names.bbclass | 16 +++- .../kernel-artifact-names.bbclass | 2 +- meta/classes-recipe/kernel.bbclass | 1 - meta/lib/oeqa/selftest/cases/devtool.py | 1 + .../oeqa/selftest/cases/efibootpartition.py | 4 +- meta/lib/oeqa/selftest/cases/fitimage.py | 34 ++++---- meta/lib/oeqa/selftest/cases/gdbserver.py | 9 +- meta/lib/oeqa/selftest/cases/imagefeatures.py | 82 +++++++++---------- meta/lib/oeqa/selftest/cases/minidebuginfo.py | 15 ++-- meta/lib/oeqa/selftest/cases/runqemu.py | 10 ++- meta/lib/oeqa/selftest/cases/wic.py | 44 +++++----- scripts/runqemu | 36 ++++---- 15 files changed, 155 insertions(+), 121 deletions(-)Quoted 7 lines
--
2.39.2-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#178432):https://lists.openembedded.org/g/openembedded-core/message/178432
Quoted 3 lines
Mute This Topic: https://lists.openembedded.org/mt/97578950/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Quoted 2 lines
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 10 lines · Martin Jansa
Quoted 6 lines · Alexander Kanavin
Tests in distrodata that contact upstream servers had to be disabled because it's not possible to tell what is a real failure, and what is a transient upstream issue (with expired certificates or other network issues). To ensure upstream version checks don't regress, I run them manually every now and then.
Quoted 2 lines
Thanks, can you confirm that git-submodule-test is currently failing for you (and probably was for long time) and that binutils is often reporting failure?
I run the check (basically 'devtool check-upgrade-status' with a manual review of what it prints) without meta-selftest included into the build, but I can try that if you want.
binutils is flaky, yes. The server is contacted mutliple times for various flavours of it, and some kind of silly rate limiting kicks in. If you run a standaline 'devtool check-upgrade-status binutils', it will work.
Alex
Quoted 16 lines · Alexander Kanavin
Quoted 2 lines · Martin Jansa
On Mon, Mar 13, 2023 at 6:26 PM Alexander Kanavin <
Quoted 9 lines · [email protected]> wrote:
Quoted 6 lines
Tests in distrodata that contact upstream servers had to be disabled because it's not possible to tell what is a real failure, and what is a transient upstream issue (with expired certificates or other network issues). To ensure upstream version checks don't regress, I run them manually every now and then.
Thanks, can you confirm that git-submodule-test is currently failing for
you (and probably was for long time) and
Quoted 1 line
that binutils is often reporting failure?
Quoted 9 lines
I run the check (basically 'devtool check-upgrade-status' with a manual review of what it prints) without meta-selftest included into the build,
but I can try that if you want.
No need, "without meta-selftest included" explains why you haven't fixed it already. I was reading "I run them
manually every now and then." as that you're triggering some special job on AB manually, but with meta-selftest and
through OESelftestTestCase.test_checkpkg.
binutils is flaky, yes. The server is contaced mutliple times for
Quoted 4 lines
various flavours of it, and some kind of silly rate limiting kicks in. If you run a standaline 'devtool check-upgrade-status binutils', it will work.
Yes, that's what I was seeing yesterday as well, but then it was failing quite reliably in OESelftestTestCase.test_checkpkg until I've added some debug into it.
Cheers,