Skip to contentSkip to conversation

python3-packaging: Ignore old glibc tests on musl

PATCH In review Khem Raj 2 participants 6 messages Oct 19, 2024 – Oct 21, 2024

lore
Share & tools
lore

Khem Raj

author
#0 lore raw

PATCH 1/2 python3-packaging: Ignore old glibc tests on musl

These tests expect glibc to be the system C library ignore them for musl

---
 meta/recipes-devtools/python/python3-packaging_24.1.bb | 4 ++++
 1 file changed, 4 insertions(+)
meta/recipes-devtools/python/python3-packaging_24.1.bb +4 -0
index 20bcd55fbad..f8465d9fdb7 100644
@@ -25,6 +25,10 @@ do_install_ptest() {
     rm -f ${D}${PTEST_PATH}/tests/manylinux/build.sh
 }
 
+do_install_ptest:append:libc-musl() {
+    sed -i -e '/^pytest/ s|$| --deselect tests/test_manylinux.py::test_is_manylinux_compatible_old --ignore=tests/test_tags.py|' ${D}${PTEST_PATH}/run-ptest
+}
+
 RDEPENDS:${PN}-ptest = "\
     python3-ctypes \
     python3-pretend \
author
#1 lore raw

PATCH 2/2 python3-libarchive-c: Avoid using find_library python API

find_library API depends on platform to provide ldconfig, ld, gcc, objdump etc, so either we add these dependencies or avoid them by computing the libarchive library name during build, which we can be done.

This ensures that ffi module works with musl as well as glibc equally as musl does not provide ldconfig like glibc does

---
 meta/recipes-devtools/python/python3-libarchive-c_5.1.bb | 8 ++++++++
 1 file changed, 8 insertions(+)
meta/recipes-devtools/python/python3-libarchive-c_5.1.bb +8 -0
index 3116c6b62ec..510e2d9159c 100644
@@ -16,6 +16,14 @@ inherit pypi setuptools3 ptest
 
 SRC_URI[sha256sum] = "7bcce24ea6c0fa3bc62468476c6d2f6264156db2f04878a372027c10615a2721"
 
+DEPENDS += "patchelf-native libarchive"
+# Avoid using find_library API which needs ldconfig and ld/objdump
+# https://docs.python.org/3/library/ctypes.html#ctypes-reference
+#
+do_configure:append() {
+    sed -i -e "s|find_library('archive')|'${libdir}/$(patchelf --print-soname ${STAGING_LIBDIR}/libarchive.so)'|" ${S}/libarchive/ffi.py
+}
+
 RDEPENDS:${PN} += "\
   libarchive \
   python3-ctypes \
#2 lore raw

Re: [OE-core] [PATCH 2/2] python3-libarchive-c: Avoid using find_library python API

Can this be a proper patch please?

Alex

Quoted 41 lines · Khem Raj via lists.openembedded.org

find_library API depends on platform to provide ldconfig, ld, gcc, objdump etc, so either we add these dependencies or avoid them by computing the libarchive library name during build, which we can be done.

This ensures that ffi module works with musl as well as glibc equally as musl does not provide ldconfig like glibc does

---
 meta/recipes-devtools/python/python3-libarchive-c_5.1.bb | 8 ++++++++
 1 file changed, 8 insertions(+)
meta/recipes-devtools/python/python3-libarchive-c_5.1.bb +8 -1
index 3116c6b62ec..510e2d9159c 100644
@@ -16,6 +16,14 @@ inherit pypi setuptools3 ptest
 
 SRC_URI[sha256sum] = "7bcce24ea6c0fa3bc62468476c6d2f6264156db2f04878a372027c10615a2721"
 
+DEPENDS += "patchelf-native libarchive"
+# Avoid using find_library API which needs ldconfig and ld/objdump
+# https://docs.python.org/3/library/ctypes.html#ctypes-reference
+#
+do_configure:append() {
+    sed -i -e "s|find_library('archive')|'${libdir}/$(patchelf --print-soname ${STAGING_LIBDIR}/libarchive.so)'|" ${S}/libarchive/ffi.py
+}
+
 RDEPENDS:${PN} += "\
   libarchive \
   python3-ctypes \
 
-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group.
View/Reply Online (#206086): https://lists.openembedded.org/g/openembedded-core/message/206086
Mute This Topic: https://lists.openembedded.org/mt/109104795/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

author
#3 lore raw

Re: [OE-core] [PATCH 2/2] python3-libarchive-c: Avoid using find_library python API

Quoted 1 line · Alexander Kanavin

Can this be a proper patch please?

I have thought of it. sadly It can not be patched with a single patch as libdir is not uniform across all combos we may build

Quoted 54 lines

Alex

Quoted 2 lines · Khem Raj via lists.openembedded.org

find_library API depends on platform to provide ldconfig, ld, gcc,

objdump

Quoted 6 lines

etc, so either we add these dependencies or avoid them by computing the libarchive library name during build, which we can be done.

This ensures that ffi module works with musl as well as glibc equally as musl does not provide ldconfig like glibc does

Review

Quoted 6 lines
---
 meta/recipes-devtools/python/python3-libarchive-c_5.1.bb | 8 ++++++++
 1 file changed, 8 insertions(+)
(quoted diff) +0 -0

b/meta/recipes-devtools/python/python3-libarchive-c_5.1.bb

meta/recipes-devtools/python/python3-libarchive-c_5.1.bb inherit pypi setuptools3 ptest · line 17

Quoted 6 lines

index 3116c6b62ec..510e2d9159c 100644

meta/recipes-devtools/python/python3-libarchive-c_5.1.bb +0 -0
@@ -16,6 +16,14 @@ inherit pypi setuptools3 ptest
 
 SRC_URI[sha256sum] =

"7bcce24ea6c0fa3bc62468476c6d2f6264156db2f04878a372027c10615a2721"

meta/recipes-devtools/python/python3-libarchive-c_5.1.bb inherit pypi setuptools3 ptest

Quoted 7 lines
meta/recipes-devtools/python/python3-libarchive-c_5.1.bb +6 -0
+DEPENDS += "patchelf-native libarchive"
+# Avoid using find_library API which needs ldconfig and ld/objdump
+# https://docs.python.org/3/library/ctypes.html#ctypes-reference
+#
+do_configure:append() {
+    sed -i -e "s|find_library('archive')|'${libdir}/$(patchelf

--print-soname ${STAGING_LIBDIR}/libarchive.so)'|" ${S}/libarchive/ffi.py

Quoted 9 lines
meta/recipes-devtools/python/python3-libarchive-c_5.1.bb +2 -1
+}
+
 RDEPENDS:${PN} += "\
   libarchive \
   python3-ctypes \
 
-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group. View/Reply Online (#206086):

https://lists.openembedded.org/g/openembedded-core/message/206086

Quoted 3 lines

Mute This Topic: https://lists.openembedded.org/mt/109104795/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [

[email protected]]

Quoted 2 lines

-=-=-=-=-=-=-=-=-=-=-=-

#4 lore raw
In reply to Khem Raj

Can run-ptest be rewritten to load the list of tests to skip from a file, and then the recipe would add to that file instead? These sed hacks are brittle and barely readable.

Alex

Quoted 33 lines · Khem Raj via lists.openembedded.org

These tests expect glibc to be the system C library ignore them for musl

---
 meta/recipes-devtools/python/python3-packaging_24.1.bb | 4 ++++
 1 file changed, 4 insertions(+)
meta/recipes-devtools/python/python3-packaging_24.1.bb +4 -1
index 20bcd55fbad..f8465d9fdb7 100644
@@ -25,6 +25,10 @@ do_install_ptest() {
     rm -f ${D}${PTEST_PATH}/tests/manylinux/build.sh
 }
 
+do_install_ptest:append:libc-musl() {
+    sed -i -e '/^pytest/ s|$| --deselect tests/test_manylinux.py::test_is_manylinux_compatible_old --ignore=tests/test_tags.py|' ${D}${PTEST_PATH}/run-ptest
+}
+
 RDEPENDS:${PN}-ptest = "\
     python3-ctypes \
     python3-pretend \
 
-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group.
View/Reply Online (#206085): https://lists.openembedded.org/g/openembedded-core/message/206085
Mute This Topic: https://lists.openembedded.org/mt/109104794/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

author
#5 lore raw
Quoted 3 lines · Alexander Kanavin

Can run-ptest be rewritten to load the list of tests to skip from a file, and then the recipe would add to that file instead? These sed hacks are brittle and barely readable.

Yeah IIRC there is also some pre existing case of doing so I will look into it

Quoted 45 lines

Alex

Review

Quoted 10 lines · Khem Raj via lists.openembedded.org

These tests expect glibc to be the system C library ignore them for musl

---
 meta/recipes-devtools/python/python3-packaging_24.1.bb | 4 ++++
 1 file changed, 4 insertions(+)
(quoted diff) +0 -0

b/meta/recipes-devtools/python/python3-packaging_24.1.bb

meta/recipes-devtools/python/python3-packaging_24.1.bb do_install_ptest() { · line 29

Quoted 9 lines

index 20bcd55fbad..f8465d9fdb7 100644

meta/recipes-devtools/python/python3-packaging_24.1.bb +2 -0
@@ -25,6 +25,10 @@ do_install_ptest() {
     rm -f ${D}${PTEST_PATH}/tests/manylinux/build.sh
 }
 
+do_install_ptest:append:libc-musl() {
+    sed -i -e '/^pytest/ s|$| --deselect

tests/test_manylinux.py::test_is_manylinux_compatible_old --ignore=tests/test_tags.py|' ${D}${PTEST_PATH}/run-ptest

Quoted 9 lines
meta/recipes-devtools/python/python3-packaging_24.1.bb +2 -1
+}
+
 RDEPENDS:${PN}-ptest = "\
     python3-ctypes \
     python3-pretend \
 
-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group. View/Reply Online (#206085):

https://lists.openembedded.org/g/openembedded-core/message/206085

Quoted 3 lines

Mute This Topic: https://lists.openembedded.org/mt/109104794/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [

[email protected]]

Quoted 2 lines

-=-=-=-=-=-=-=-=-=-=-=-