python3: skip tests requiring tools-sdk
cover python3: skip tests requiring tools-sdk
When running python3-ptest for any release since Python 3.8.8 with a minimal target (e.g. python3-ptest), two test cases will fail:
https://github.com/python/cpython/blob/main/Lib/ctypes/test/test_find.py#L116
https://github.com/python/cpython/blob/main/Lib/ctypes/test/test_find.py#L120
For the autobuilder, these tests have always been running on an image which set IMAGE_FEATURES to include 'tools-sdk', which includes the required packagegroup-core-buildessential. Here we add a patch to test for 'tools-sdk' and skip the tests if it is not present.
The following changes since commit b8723c114b3d9b58d2da1a86d728d176dcbeb4b1:
nfs-utils: upgrade 2.5.3 -> 2.5.4 (2021-06-17 22:18:41 +0100)
are available in the Git repository at:
git://push.openembedded.org/openembedded-core-contrib timo/python3-skip-test-find
Tim Orling (1):
python3: skip tests requiring tools-sdk
...pes.test_find-skip-without-tools-sdk.patch | 33 +++++++++++++++++++ meta/recipes-devtools/python/python3_3.9.5.bb | 1 + 2 files changed, 34 insertions(+) create mode 100644 meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch
Signature · 1 line
2.30.2
PATCH 1/1 python3: skip tests requiring tools-sdk
Conditionally skip test_ctypes.test_find unless IMAGE_FEATURES contains 'tools-sdk' as these test cases require full packagegroup-core-buildessential
Fixes:
AssertionError: Failed ptests:
{'python3': ['test_find_library_with_gcc', 'test_find_library_with_ld']}
--- ...pes.test_find-skip-without-tools-sdk.patch | 33 +++++++++++++++++++ meta/recipes-devtools/python/python3_3.9.5.bb | 1 + 2 files changed, 34 insertions(+) create mode 100644 meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch
meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +33 -0
@@ -0,0 +1,33 @@ +From 7a2bddfa437be633bb6945d0e6b7d6f27da870ad Mon Sep 17 00:00:00 2001 +From: Tim Orling <[email protected]> +Date: Fri, 18 Jun 2021 11:56:50 -0700 +Subject: [PATCH] test_ctypes.test_find: skip without tools-sdk + +These tests need full packagegroup-core-buildessential, the +easiest way to dynamically check for that is looking for +'tools-sdk' in IMAGE_FEATURES. + +Upstream-Status: Inappropriate [oe-specific] + +Signed-off-by: Tim Orling <[email protected]> +--- + Lib/ctypes/test/test_find.py | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/Lib/ctypes/test/test_find.py b/Lib/ctypes/test/test_find.py +index 92ac184..0d009d1 100644 +--- a/Lib/ctypes/test/test_find.py ++++ b/Lib/ctypes/test/test_find.py +@@ -112,10 +112,12 @@ class FindLibraryLinux(unittest.TestCase): + # LD_LIBRARY_PATH) + self.assertEqual(find_library(libname), 'lib%s.so' % libname) + ++ @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"") + def test_find_library_with_gcc(self): + with unittest.mock.patch("ctypes.util._findSoname_ldconfig", lambda *args: None): + self.assertNotEqual(find_library('c'), None) + ++ @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"") + def test_find_library_with_ld(self): + with unittest.mock.patch("ctypes.util._findSoname_ldconfig", lambda *args: None), \ + unittest.mock.patch("ctypes.util._findLib_gcc", lambda *args: None):
meta/recipes-devtools/python/python3_3.9.5.bb +1 -0
@@ -31,6 +31,7 @@ SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \ file://0020-configure.ac-setup.py-do-not-add-a-curses-include-pa.patch \ file://0001-Lib-sysconfig.py-use-libdir-values-from-configuratio.patch \ file://0001-Skip-failing-tests-due-to-load-variability-on-YP-AB.patch \ + ${@bb.utils.contains('IMAGE_FEATURES', 'tools-sdk', '', 'file://0001-test_ctypes.test_find-skip-without-tools-sdk.patch', d)} \ " SRC_URI_append_class-native = " \
Signature · 1 line
2.30.2
Wait. How does this work, if IMAGE_FEATURES are set from image recipes, and not globally?
I'm afraid this may have to be reverted, as it just made it to master.
Alex
Quoted 87 lines · Tim Orling
Conditionally skip test_ctypes.test_find unless
IMAGE_FEATUREScontains 'tools-sdk' as these test cases require full packagegroup-core-buildessentialFixes:
AssertionError: Failed ptests:
{'python3': ['test_find_library_with_gcc', 'test_find_library_with_ld']}Signed-off-by Tim Orling <[email protected]>--- ...pes.test_find-skip-without-tools-sdk.patch | 33 +++++++++++++++++++ meta/recipes-devtools/python/python3_3.9.5.bb | 1 + 2 files changed, 34 insertions(+) create mode 100644
meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patchdiff --git
a/meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patchb/meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patchnew file mode 100644
index 00000000000..7d684b3c43c
---/dev/null
+++b/meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patchmeta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +23
-0@@ -0,0 +1,33 @@ +From 7a2bddfa437be633bb6945d0e6b7d6f27da870ad Mon Sep 17 00:00:00 2001 +From: Tim Orling <[email protected]> +Date: Fri, 18 Jun 2021 11:56:50 -0700 +Subject: [PATCH] test_ctypes.test_find: skip without tools-sdk + +These tests need full packagegroup-core-buildessential, the +easiest way to dynamically check for that is looking for +'tools-sdk' in IMAGE_FEATURES. + +Upstream-Status: Inappropriate [oe-specific] + +Signed-off-by: Tim Orling <[email protected]> +--- + Lib/ctypes/test/test_find.py | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/Lib/ctypes/test/test_find.py b/Lib/ctypes/test/test_find.py +index 92ac184..0d009d1 100644 +--- a/Lib/ctypes/test/test_find.py ++++ b/Lib/ctypes/test/test_find.py +@@ -112,10 +112,12 @@ class FindLibraryLinux(unittest.TestCase): + # LD_LIBRARY_PATH) + self.assertEqual(find_library(libname), 'lib%s.so' %libname)
meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +4
-0+ ++ @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"") + def test_find_library_with_gcc(self): + with unittest.mock.patch("ctypes.util._findSoname_ldconfig",lambda *args: None):
meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +5
-0+ self.assertNotEqual(find_library('c'), None) + ++ @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"") + def test_find_library_with_ld(self): + with unittest.mock.patch("ctypes.util._findSoname_ldconfig",lambda *args: None), \
+ unittest.mock.patch("ctypes.util._findLib_gcc", lambda
*args: None):meta/recipes-devtools/python/python3_3.9.5.bb +0
-0@@ -31,6 +31,7 @@ SRC_URI = " http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \ file://0020-configure.ac-setup.py-do-not-add-a-curses-include-pa.patch \ file://0001-Lib-sysconfig.py-use-libdir-values-from-configuratio.patch \file://0001-Skip-failing-tests-due-to-load-variability-on-YP-AB.patch \
+ ${@bb.utils.contains('IMAGE_FEATURES', 'tools-sdk', '','file://0001-test_ctypes.test_find-skip-without-tools-sdk.patch', d)} \ "SRC_URI_append_class-native = " \
--
2.30.2
Quoted 6 lines · Alexander Kanavin
Wait. How does this work, if
IMAGE_FEATURESare set from image recipes, and not globally?I'm afraid this may have to be reverted, as it just made it to master.
These tests have only been passing (since.python3 3.9.0 upgrade) because the AB ptest images in one way or another have included packagegroup-core-buildessential.
I asked RP about adding packagegroup-core-buildessential as RDEPENDS for python3-ptest and he was concerned about build performance. I tried to narrow the scope to gcc-symlinks and binutils-symlinks, but that was not enough to make the tests pass.
As I see it, we have two options, this patch which skips the tests conditionally, or we UNCONDITIONALLY skip the tests.
If you run python3-ptest (and nothing else) on core-image-minimal, these test cases will fail, which means we do not have proper RDEPENDS for python3-ptest.
Open to other ideas.
Quoted 91 lines
Alex
Quoted 87 lines · Tim Orling
Conditionally skip test_ctypes.test_find unless
IMAGE_FEATUREScontains 'tools-sdk' as these test cases require full packagegroup-core-buildessentialFixes:
AssertionError: Failed ptests:
{'python3': ['test_find_library_with_gcc', 'test_find_library_with_ld']}Signed-off-by Tim Orling <[email protected]>--- ...pes.test_find-skip-without-tools-sdk.patch | 33 +++++++++++++++++++ meta/recipes-devtools/python/python3_3.9.5.bb | 1 + 2 files changed, 34 insertions(+) create mode 100644
meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patchdiff --git
a/meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patchb/meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patchnew file mode 100644
index 00000000000..7d684b3c43c
---/dev/null
+++b/meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patchmeta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +23
-0@@ -0,0 +1,33 @@ +From 7a2bddfa437be633bb6945d0e6b7d6f27da870ad Mon Sep 17 00:00:00 2001 +From: Tim Orling <[email protected]> +Date: Fri, 18 Jun 2021 11:56:50 -0700 +Subject: [PATCH] test_ctypes.test_find: skip without tools-sdk + +These tests need full packagegroup-core-buildessential, the +easiest way to dynamically check for that is looking for +'tools-sdk' in IMAGE_FEATURES. + +Upstream-Status: Inappropriate [oe-specific] + +Signed-off-by: Tim Orling <[email protected]> +--- + Lib/ctypes/test/test_find.py | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/Lib/ctypes/test/test_find.py b/Lib/ctypes/test/test_find.py +index 92ac184..0d009d1 100644 +--- a/Lib/ctypes/test/test_find.py ++++ b/Lib/ctypes/test/test_find.py +@@ -112,10 +112,12 @@ class FindLibraryLinux(unittest.TestCase): + # LD_LIBRARY_PATH) + self.assertEqual(find_library(libname), 'lib%s.so' %libname)
meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +4
-0+ ++ @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"") + def test_find_library_with_gcc(self): + with unittest.mock.patch("ctypes.util._findSoname_ldconfig",lambda *args: None):
meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +5
-0+ self.assertNotEqual(find_library('c'), None) + ++ @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"") + def test_find_library_with_ld(self): + with unittest.mock.patch("ctypes.util._findSoname_ldconfig",lambda *args: None), \
+ unittest.mock.patch("ctypes.util._findLib_gcc", lambda
*args: None):meta/recipes-devtools/python/python3_3.9.5.bb +0
-0@@ -31,6 +31,7 @@ SRC_URI = " http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \ file://0020-configure.ac-setup.py-do-not-add-a-curses-include-pa.patch \ file://0001-Lib-sysconfig.py-use-libdir-values-from-configuratio.patch \file://0001-Skip-failing-tests-due-to-load-variability-on-YP-AB.patch \
+ ${@bb.utils.contains('IMAGE_FEATURES', 'tools-sdk', '','file://0001-test_ctypes.test_find-skip-without-tools-sdk.patch', d)} \ "SRC_URI_append_class-native = " \
--
2.30.2
Quoted 20 lines · Tim Orling
Quoted 6 lines · Alexander Kanavin
Wait. How does this work, if
IMAGE_FEATURESare set from image recipes, and not globally?I'm afraid this may have to be reverted, as it just made it to master.
These tests have only been passing (since.python3 3.9.0 upgrade) because the AB ptest images in one way or another have included packagegroup-core-buildessential.
I asked RP about adding packagegroup-core-buildessential as RDEPENDS for python3-ptest and he was concerned about build performance. I tried to narrow the scope to gcc-symlinks and binutils-symlinks, but that was not enough to make the tests pass.
Quoted 6 lines
As I see it, we have two options, this patch which skips the tests conditionally, or we UNCONDITIONALLY skip the tests.
If you run python3-ptest (and nothing else) on core-image-minimal, these test cases will fail, which means we do not have proper RDEPENDS for python3-ptest.
I think the right way out of this is to modify the tests so that they check for the presence of items that are used by the tests and skip themselfes if those items (gcc, etc) are not present (turn a hard dependency into a soft one). Then simply apply the patch without any conditions in the recipe.
There are many other tests which self-skip in a similar way already, we do not necessarily maintain the full soft dependency set for the fullest ptest coverage, not should we try to.
Alex
Quoted 29 lines · Alexander Kanavin
Quoted 20 lines · Tim Orling
Quoted 6 lines · Alexander Kanavin
Wait. How does this work, if
IMAGE_FEATURESare set from image recipes, and not globally?I'm afraid this may have to be reverted, as it just made it to master.
These tests have only been passing (since.python3 3.9.0 upgrade) because the AB ptest images in one way or another have included packagegroup-core-buildessential.
I asked RP about adding packagegroup-core-buildessential as RDEPENDS for python3-ptest and he was concerned about build performance. I tried to narrow the scope to gcc-symlinks and binutils-symlinks, but that was not enough to make the tests pass.
Quoted 6 lines
As I see it, we have two options, this patch which skips the tests conditionally, or we UNCONDITIONALLY skip the tests.
If you run python3-ptest (and nothing else) on core-image-minimal, these test cases will fail, which means we do not have proper RDEPENDS for python3-ptest.
Quoted 7 lines
I think the right way out of this is to modify the tests so that they check for the presence of items that are used by the tests and skip themselfes if those items (gcc, etc) are not present (turn a hard dependency into a soft one). Then simply apply the patch without any conditions in the recipe.
I don't disagree, but a simple check for gcc or ld (and ldconfig) is not enough to pass the tests. I proved this by adding gcc-symlinks and binutils-symlinks to the RDEPENDS.
It's a bit convoluted in the code base.
There are many other tests which self-skip in a similar way already, we do
Quoted 5 lines
not necessarily maintain the full soft dependency set for the fullest ptest coverage, not should we try to.
Alex
Quoted 17 lines · Tim Orling
Quoted 5 lines · Alexander Kanavin
Wait. How does this work, if
IMAGE_FEATURESare set from image recipes, and not globally?I'm afraid this may have to be reverted, as it just made it to master.
These tests have only been passing (since.python3 3.9.0 upgrade) because the AB ptest images in one way or another have included packagegroup-core-buildessential.
I asked RP about adding packagegroup-core-buildessential as RDEPENDS for python3-ptest and he was concerned about build performance. I tried to narrow the scope to gcc-symlinks and binutils-symlinks, but that was not enough to make the tests pass.
Quoted 6 lines
As I see it, we have two options, this patch which skips the tests conditionally, or we UNCONDITIONALLY skip the tests.
If you run python3-ptest (and nothing else) on core-image-minimal, these test cases will fail, which means we do not have proper RDEPENDS for python3-ptest.Open to other ideas.
I think we're all fine with disabling these tests. The issue is that IMAGE_FEATURES isn't available in a recipe. As such we may as well unconditionally disable them as that is effectively what the current change does.
Cheers,
Richard
Quoted 5 lines · Richard Purdie
I think we're all fine with disabling these tests. The issue is that
IMAGE_FEATURESisn't available in a recipe. As such we may as well unconditionally disable them as that is effectively what the current change does.
I'm ok with this too, although I'd prefer a check with a fallback to a skip in the test.
Alex