New: Python ctypes find_library() not working correctly.
https://bugs.busybox.net/show_bug.cgi?id=14886
Bug ID: 14886
Summary: Python ctypes find_library() not working correctly.
Product: buildroot
Version: 2022.02.3
Hardware: PC
OS: All
Status: NEW
Severity: normal
Priority: P5
Component: Other
Assignee: [email protected]
Reporter: [email protected]
CC: [email protected]Target Milestone: ---
The find_library() function in the ctypes Python standard library is not working properly.
https://docs.python.org/3/library/ctypes.html#finding-shared-libraries
You can reproduce the issue by trying the following in the python shell...
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
from ctypes.util import
find_libraryfind_library("c")
This should return the string "libc.so.0" but returns nothing, this is the same for any library name I tried even though you can see the libraries are installed and "ldconfig -v" lists the libraries.
This has the effect of causing libraries like pyusb to not work properly because they reply on ctypes & find_library() to find libusb.
Digging a bit deeper on how find_library() works it calls either ldconfig -p, ld, or gcc. buildroot doesn't have ld or gcc installed so I assume that means it is using "ldconfig -p". ldconfig is part of the uclibc library and uclibc has caching turned off in its configuration so ldconfig -p just prints a message says, "Library cache disabled" instead of printing a list of libraries and their absolute paths.
https://github.com/python/cpython/blob/main/Lib/ctypes/util.py
There is an option in the uclibc menuconfig to enable the cache (LDSO_CACHE_SUPPORT) but when I did that it didn't seem to change anything. Any ideas how to fix this?
I'm using the pc_x86_64_efi_defconfig with Python enabled. No other changes.
Signature · 6 lines
You are receiving this mail because: You are on the CC list for the bug. _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
[Buildroot] [Bug 14886] Python ctypes find_library() not working correctly.
https://bugs.busybox.net/show_bug.cgi?id=14886
--- Comment #1 from Peter Scheie <[email protected]> ---
I have seen this bug, too, in building an image for the Beaglebone Black. It appears in the iio package in the python binding: In iio.py, CDLL calls find_library with the argument "iio" to get the name of the libiio.so.0 library, but fails. Some folks on #beagle said using find_library() this way was a bad idea because it produces unpredictable results (which I can attest to). Instead, it was recommended that CDLL be called with libiio.so.0 directly, removing find_libary(). This works and I am using a patch to implement it. BR currently uses version 0.19 of libiio but even in the most current version, 0.24, it still uses find_library().
I tried find_library() on a few other libs and got the same None result.
Signature · 6 lines
You are receiving this mail because: You are on the CC list for the bug. _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
[Buildroot] [Bug 14886] Python ctypes find_library() not working correctly.
In reply to [email protected]https://bugs.busybox.net/show_bug.cgi?id=14886
--- Comment #2 from Pete Morici <[email protected]> ---
The problem is a bunch of dependencies rely on the find_library function so I don't really have a choice short of patching every library.
I think the right way to fix this would be to enable the ldconfig cache mechanism in uclibc or mark libraries that depend on ctypes as dependent on glibc tthat does support the caching mechanism required for find_library
Signature · 6 lines
You are receiving this mail because: You are on the CC list for the bug. _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
[Buildroot] [Bug 14886] Python ctypes find_library() not working correctly.
In reply to [email protected]https://bugs.busybox.net/show_bug.cgi?id=14886
--- Comment #3 from Pete Morici <[email protected]> ---
I was able to solve this by enabling the LDSO_CACHE_SUPPORT option in
uclibc via a configuration fragment and then patching the file Lib/ctypes/util.py in the python3 package to change the mapping of "x86_64-64" from "libc6,x86_64" to "libc0". I wasn't able to find a way to generate the cache file during the build process so you have to run ldconfig on first boot, you could use and init script, before using Python to generate /etc/ls.so.cache and then ctypes will work correctly.
The crux of the issue is that, on Linux, Python's ctypes module just isn't
compatible with anything except glibc because is explicitly looks for "libc6" and other glibc specific items in the output of ldconfig -p which is uses to resolve paths to libraries loaded with ctypes. If you are using uclibc, of musl you are going to run into this.
Signature · 6 lines
You are receiving this mail because: You are on the CC list for the bug. _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
[Buildroot] [Bug 14886] Python ctypes find_library() not working correctly.
In reply to [email protected]https://bugs.busybox.net/show_bug.cgi?id=14886
--- Comment #4 from Thomas Petazzoni <[email protected]> --- This issue is indeed tricky. I think the right solution would be to patch Python to reimplement the find_library() function in a way that is compatible with Buildroot:
(1) Walk the library directories to find the matching library.
(2) Retrieve its SONAME. This might be a bit annoying to do in pure Python
without invoking a separate Python library to read ELF files.
A bit annoying to do, but probably the solution that is going to require the least amount of dependencies in Buildroot.
Signature · 6 lines
You are receiving this mail because: You are on the CC list for the bug. _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
[Buildroot] [Bug 14886] Python ctypes find_library() not working correctly.
In reply to [email protected]https://bugs.busybox.net/show_bug.cgi?id=14886
--- Comment #5 from Pete Morici <[email protected]> --- Created attachment 9341
--> https://bugs.busybox.net/attachment.cgi?id=9341&action=edit
Python patch to support ctypes when using uclibc
Also set LDSO_CACHE_SUPPORT=y un uclibc configuration. Then run ldconfig at boot prior to using Python / ctypes if ld.so.cache doesn't exist.
if [[ ! -e "/etc/ld.so.cache" ]]; then ldconfig; fi
Signature · 6 lines
You are receiving this mail because: You are on the CC list for the bug. _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
[Buildroot] [Bug 14886] Python ctypes find_library() not working correctly.
In reply to [email protected]https://bugs.busybox.net/show_bug.cgi?id=14886
--- Comment #6 from Pete Morici <[email protected]> ---
My understanding from reading some of the other posts is that the reason the ld.so.cache is disabled is because you can't reliably run it in the fakeroot environment without a bunch of patches.
Signature · 6 lines
You are receiving this mail because: You are on the CC list for the bug. _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
[Buildroot] [Bug 14886] Python ctypes find_library() not working correctly.
In reply to [email protected]https://bugs.busybox.net/show_bug.cgi?id=14886
--- Comment #7 from Marcus Hoffmann <[email protected]> --- Created attachment 9566
--> https://bugs.busybox.net/attachment.cgi?id=9566&action=edit
libiio patch to fix python bindings
I've just run into this problem with libiio python bindings as well. (On a glibc system though), the attached (libiio) patch fixes it, though I'm not sure if that's a desired solution.
Signature · 6 lines
You are receiving this mail because: You are on the CC list for the bug. _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
[Buildroot] [Bug 14886] Python ctypes find_library() not working correctly.
In reply to [email protected]https://bugs.busybox.net/show_bug.cgi?id=14886
Yann E. MORIN <[email protected]> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |MOVED
CC| |[email protected]
Status|NEW |RESOLVED--- Comment #8 from Yann E. MORIN <[email protected]> --- Thank you for your report.
The issue tracker for the Buildroot project has been moved to the Gitlab.com issue tracker:
https://gitlab.com/buildroot.org/buildroot/-/issues
We are taking this opportunity to close old issues in this old tracker. If you believe your issue is still relevant, please open one in the new issue tracker.
Thank you!
Signature · 6 lines
You are receiving this mail because: You are on the CC list for the bug. _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot