Skip to contentSkip to conversation

python3: skip tests requiring tools-sdk

PATCH In review Tim Orling 3 participants 8 messages Jun 19, 2021 – Jun 21, 2021

lore
Share & tools
lore

Tim Orling

author
#0 lore raw

cover python3: skip tests requiring tools-sdk

When running python3-ptest for any release since Python 3.8.8 with a minimal target (e.g. python3-ptest), two test cases will fail:

https://github.com/python/cpython/blob/main/Lib/ctypes/test/test_find.py#L116
https://github.com/python/cpython/blob/main/Lib/ctypes/test/test_find.py#L120

For the autobuilder, these tests have always been running on an image which set IMAGE_FEATURES to include 'tools-sdk', which includes the required packagegroup-core-buildessential. Here we add a patch to test for 'tools-sdk' and skip the tests if it is not present.

The following changes since commit b8723c114b3d9b58d2da1a86d728d176dcbeb4b1:

nfs-utils: upgrade 2.5.3 -> 2.5.4 (2021-06-17 22:18:41 +0100)

are available in the Git repository at:

git://push.openembedded.org/openembedded-core-contrib timo/python3-skip-test-find

Tim Orling (1):
python3: skip tests requiring tools-sdk

 ...pes.test_find-skip-without-tools-sdk.patch | 33 +++++++++++++++++++
 meta/recipes-devtools/python/python3_3.9.5.bb |  1 +
 2 files changed, 34 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch
Signature · 1 line
2.30.2
author
#1 lore raw

PATCH 1/1 python3: skip tests requiring tools-sdk

Conditionally skip test_ctypes.test_find unless IMAGE_FEATURES contains 'tools-sdk' as these test cases require full packagegroup-core-buildessential

Fixes:
AssertionError: Failed ptests:
{'python3': ['test_find_library_with_gcc', 'test_find_library_with_ld']}

---
 ...pes.test_find-skip-without-tools-sdk.patch | 33 +++++++++++++++++++
 meta/recipes-devtools/python/python3_3.9.5.bb |  1 +
 2 files changed, 34 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch
meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +33 -0
new file mode 100644
index 00000000000..7d684b3c43c
@@ -0,0 +1,33 @@
+From 7a2bddfa437be633bb6945d0e6b7d6f27da870ad Mon Sep 17 00:00:00 2001
+From: Tim Orling <[email protected]>
+Date: Fri, 18 Jun 2021 11:56:50 -0700
+Subject: [PATCH] test_ctypes.test_find: skip without tools-sdk
+
+These tests need full packagegroup-core-buildessential, the
+easiest way to dynamically check for that is looking for
+'tools-sdk' in IMAGE_FEATURES.
+
+Upstream-Status: Inappropriate [oe-specific]
+
+Signed-off-by: Tim Orling <[email protected]>
+---
+ Lib/ctypes/test/test_find.py | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/Lib/ctypes/test/test_find.py b/Lib/ctypes/test/test_find.py
+index 92ac184..0d009d1 100644
+--- a/Lib/ctypes/test/test_find.py
++++ b/Lib/ctypes/test/test_find.py
+@@ -112,10 +112,12 @@ class FindLibraryLinux(unittest.TestCase):
+                 # LD_LIBRARY_PATH)
+                 self.assertEqual(find_library(libname), 'lib%s.so' % libname)
+ 
++    @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"")
+     def test_find_library_with_gcc(self):
+         with unittest.mock.patch("ctypes.util._findSoname_ldconfig", lambda *args: None):
+             self.assertNotEqual(find_library('c'), None)
+ 
++    @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"")
+     def test_find_library_with_ld(self):
+         with unittest.mock.patch("ctypes.util._findSoname_ldconfig", lambda *args: None), \
+              unittest.mock.patch("ctypes.util._findLib_gcc", lambda *args: None):
meta/recipes-devtools/python/python3_3.9.5.bb +1 -0
index f4002f2516f..1717ff737d4 100644
@@ -31,6 +31,7 @@ SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
            file://0020-configure.ac-setup.py-do-not-add-a-curses-include-pa.patch \
            file://0001-Lib-sysconfig.py-use-libdir-values-from-configuratio.patch \
            file://0001-Skip-failing-tests-due-to-load-variability-on-YP-AB.patch \
+           ${@bb.utils.contains('IMAGE_FEATURES', 'tools-sdk', '', 'file://0001-test_ctypes.test_find-skip-without-tools-sdk.patch', d)} \
            "
 
 SRC_URI_append_class-native = " \
Signature · 1 line
2.30.2
#2 lore raw

Wait. How does this work, if IMAGE_FEATURES are set from image recipes, and not globally?

I'm afraid this may have to be reverted, as it just made it to master.

Alex

Quoted 87 lines · Tim Orling

Conditionally skip test_ctypes.test_find unless IMAGE_FEATURES contains 'tools-sdk' as these test cases require full packagegroup-core-buildessential

Fixes:
AssertionError: Failed ptests:
{'python3': ['test_find_library_with_gcc', 'test_find_library_with_ld']}

---
 ...pes.test_find-skip-without-tools-sdk.patch | 33 +++++++++++++++++++
 meta/recipes-devtools/python/python3_3.9.5.bb |  1 +
 2 files changed, 34 insertions(+)
 create mode 100644

meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch

diff --git a/meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch b/meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file mode 100644
index 00000000000..7d684b3c43c
--- /dev/null
+++
b/meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch

meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +23 -0
@@ -0,0 +1,33 @@
+From 7a2bddfa437be633bb6945d0e6b7d6f27da870ad Mon Sep 17 00:00:00 2001
+From: Tim Orling <[email protected]>
+Date: Fri, 18 Jun 2021 11:56:50 -0700
+Subject: [PATCH] test_ctypes.test_find: skip without tools-sdk
+
+These tests need full packagegroup-core-buildessential, the
+easiest way to dynamically check for that is looking for
+'tools-sdk' in IMAGE_FEATURES.
+
+Upstream-Status: Inappropriate [oe-specific]
+
+Signed-off-by: Tim Orling <[email protected]>
+---
+ Lib/ctypes/test/test_find.py | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/Lib/ctypes/test/test_find.py b/Lib/ctypes/test/test_find.py
+index 92ac184..0d009d1 100644
+--- a/Lib/ctypes/test/test_find.py
++++ b/Lib/ctypes/test/test_find.py
+@@ -112,10 +112,12 @@ class FindLibraryLinux(unittest.TestCase):
+                 # LD_LIBRARY_PATH)
+                 self.assertEqual(find_library(libname), 'lib%s.so' %

libname)

meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +4 -0
+
++    @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"")
+     def test_find_library_with_gcc(self):
+         with unittest.mock.patch("ctypes.util._findSoname_ldconfig",

lambda *args: None):

meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +5 -0
+             self.assertNotEqual(find_library('c'), None)
+
++    @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"")
+     def test_find_library_with_ld(self):
+         with unittest.mock.patch("ctypes.util._findSoname_ldconfig",

lambda *args: None), \
+ unittest.mock.patch("ctypes.util._findLib_gcc", lambda
*args: None):

meta/recipes-devtools/python/python3_3.9.5.bb +0 -0
index f4002f2516f..1717ff737d4 100644
@@ -31,6 +31,7 @@ SRC_URI = "
 http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
 
 file://0020-configure.ac-setup.py-do-not-add-a-curses-include-pa.patch \
 
 file://0001-Lib-sysconfig.py-use-libdir-values-from-configuratio.patch \
 

file://0001-Skip-failing-tests-due-to-load-variability-on-YP-AB.patch \
+ ${@bb.utils.contains('IMAGE_FEATURES', 'tools-sdk', '',

'file://0001-test_ctypes.test_find-skip-without-tools-sdk.patch', d)} \
            "

SRC_URI_append_class-native = " \
--
2.30.2

author
#3 lore raw
Quoted 6 lines · Alexander Kanavin

Wait. How does this work, if IMAGE_FEATURES are set from image recipes, and not globally?

I'm afraid this may have to be reverted, as it just made it to master.

These tests have only been passing (since.python3 3.9.0 upgrade) because the AB ptest images in one way or another have included packagegroup-core-buildessential.

I asked RP about adding packagegroup-core-buildessential as RDEPENDS for python3-ptest and he was concerned about build performance. I tried to narrow the scope to gcc-symlinks and binutils-symlinks, but that was not enough to make the tests pass.

As I see it, we have two options, this patch which skips the tests conditionally, or we UNCONDITIONALLY skip the tests.
If you run python3-ptest (and nothing else) on core-image-minimal, these test cases will fail, which means we do not have proper RDEPENDS for python3-ptest.

Open to other ideas.

Quoted 91 lines

Alex

Quoted 87 lines · Tim Orling

Conditionally skip test_ctypes.test_find unless IMAGE_FEATURES contains 'tools-sdk' as these test cases require full packagegroup-core-buildessential

Fixes:
AssertionError: Failed ptests:
{'python3': ['test_find_library_with_gcc', 'test_find_library_with_ld']}

---
 ...pes.test_find-skip-without-tools-sdk.patch | 33 +++++++++++++++++++
 meta/recipes-devtools/python/python3_3.9.5.bb |  1 +
 2 files changed, 34 insertions(+)
 create mode 100644

meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch

diff --git a/meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch b/meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file mode 100644
index 00000000000..7d684b3c43c
--- /dev/null
+++
b/meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch

meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +23 -0
@@ -0,0 +1,33 @@
+From 7a2bddfa437be633bb6945d0e6b7d6f27da870ad Mon Sep 17 00:00:00 2001
+From: Tim Orling <[email protected]>
+Date: Fri, 18 Jun 2021 11:56:50 -0700
+Subject: [PATCH] test_ctypes.test_find: skip without tools-sdk
+
+These tests need full packagegroup-core-buildessential, the
+easiest way to dynamically check for that is looking for
+'tools-sdk' in IMAGE_FEATURES.
+
+Upstream-Status: Inappropriate [oe-specific]
+
+Signed-off-by: Tim Orling <[email protected]>
+---
+ Lib/ctypes/test/test_find.py | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/Lib/ctypes/test/test_find.py b/Lib/ctypes/test/test_find.py
+index 92ac184..0d009d1 100644
+--- a/Lib/ctypes/test/test_find.py
++++ b/Lib/ctypes/test/test_find.py
+@@ -112,10 +112,12 @@ class FindLibraryLinux(unittest.TestCase):
+                 # LD_LIBRARY_PATH)
+                 self.assertEqual(find_library(libname), 'lib%s.so' %

libname)

meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +4 -0
+
++    @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"")
+     def test_find_library_with_gcc(self):
+         with unittest.mock.patch("ctypes.util._findSoname_ldconfig",

lambda *args: None):

meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch new file +5 -0
+             self.assertNotEqual(find_library('c'), None)
+
++    @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"")
+     def test_find_library_with_ld(self):
+         with unittest.mock.patch("ctypes.util._findSoname_ldconfig",

lambda *args: None), \
+ unittest.mock.patch("ctypes.util._findLib_gcc", lambda
*args: None):

meta/recipes-devtools/python/python3_3.9.5.bb +0 -0
index f4002f2516f..1717ff737d4 100644
@@ -31,6 +31,7 @@ SRC_URI = "
 http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
 
 file://0020-configure.ac-setup.py-do-not-add-a-curses-include-pa.patch \
 
 file://0001-Lib-sysconfig.py-use-libdir-values-from-configuratio.patch \
 

file://0001-Skip-failing-tests-due-to-load-variability-on-YP-AB.patch \
+ ${@bb.utils.contains('IMAGE_FEATURES', 'tools-sdk', '',

'file://0001-test_ctypes.test_find-skip-without-tools-sdk.patch', d)} \
            "

SRC_URI_append_class-native = " \
--
2.30.2

#4 lore raw
Quoted 20 lines · Tim Orling
Quoted 6 lines · Alexander Kanavin

Wait. How does this work, if IMAGE_FEATURES are set from image recipes, and not globally?

I'm afraid this may have to be reverted, as it just made it to master.

These tests have only been passing (since.python3 3.9.0 upgrade) because the AB ptest images in one way or another have included packagegroup-core-buildessential.

I asked RP about adding packagegroup-core-buildessential as RDEPENDS for python3-ptest and he was concerned about build performance. I tried to narrow the scope to gcc-symlinks and binutils-symlinks, but that was not enough to make the tests pass.

Quoted 6 lines

As I see it, we have two options, this patch which skips the tests conditionally, or we UNCONDITIONALLY skip the tests.
If you run python3-ptest (and nothing else) on core-image-minimal, these test cases will fail, which means we do not have proper RDEPENDS for python3-ptest.

I think the right way out of this is to modify the tests so that they check for the presence of items that are used by the tests and skip themselfes if those items (gcc, etc) are not present (turn a hard dependency into a soft one). Then simply apply the patch without any conditions in the recipe.

There are many other tests which self-skip in a similar way already, we do not necessarily maintain the full soft dependency set for the fullest ptest coverage, not should we try to.

Alex

author
#5 lore raw
Quoted 29 lines · Alexander Kanavin
Quoted 20 lines · Tim Orling
Quoted 6 lines · Alexander Kanavin

Wait. How does this work, if IMAGE_FEATURES are set from image recipes, and not globally?

I'm afraid this may have to be reverted, as it just made it to master.

These tests have only been passing (since.python3 3.9.0 upgrade) because the AB ptest images in one way or another have included packagegroup-core-buildessential.

I asked RP about adding packagegroup-core-buildessential as RDEPENDS for python3-ptest and he was concerned about build performance. I tried to narrow the scope to gcc-symlinks and binutils-symlinks, but that was not enough to make the tests pass.

Quoted 6 lines

As I see it, we have two options, this patch which skips the tests conditionally, or we UNCONDITIONALLY skip the tests.
If you run python3-ptest (and nothing else) on core-image-minimal, these test cases will fail, which means we do not have proper RDEPENDS for python3-ptest.

Quoted 7 lines

I think the right way out of this is to modify the tests so that they check for the presence of items that are used by the tests and skip themselfes if those items (gcc, etc) are not present (turn a hard dependency into a soft one). Then simply apply the patch without any conditions in the recipe.

I don't disagree, but a simple check for gcc or ld (and ldconfig) is not enough to pass the tests. I proved this by adding gcc-symlinks and binutils-symlinks to the RDEPENDS.
It's a bit convoluted in the code base.

There are many other tests which self-skip in a similar way already, we do

Quoted 5 lines

not necessarily maintain the full soft dependency set for the fullest ptest coverage, not should we try to.

Alex

#6 lore raw
In reply to Tim Orling
Quoted 17 lines · Tim Orling
Quoted 5 lines · Alexander Kanavin

Wait. How does this work, if IMAGE_FEATURES are set from image recipes, and not globally?

I'm afraid this may have to be reverted, as it just made it to master.

These tests have only been passing (since.python3 3.9.0 upgrade) because the AB ptest images in one way or another have included packagegroup-core-buildessential.

I asked RP about adding packagegroup-core-buildessential as RDEPENDS for python3-ptest and he was concerned about build performance. I tried to narrow the scope to gcc-symlinks and binutils-symlinks, but that was not enough to make the tests pass.

Quoted 6 lines

As I see it, we have two options, this patch which skips the tests conditionally, or we UNCONDITIONALLY skip the tests.
If you run python3-ptest (and nothing else) on core-image-minimal, these test cases will fail, which means we do not have proper RDEPENDS for python3-ptest.

Open to other ideas.

I think we're all fine with disabling these tests. The issue is that IMAGE_FEATURES isn't available in a recipe. As such we may as well unconditionally disable them as that is effectively what the current change does.

Cheers,

Richard

#7 lore raw
Quoted 5 lines · Richard Purdie

I think we're all fine with disabling these tests. The issue is that IMAGE_FEATURES isn't available in a recipe. As such we may as well unconditionally disable them as that is effectively what the current change does.

I'm ok with this too, although I'd prefer a check with a fallback to a skip in the test.

Alex