bindings: python: standalone build tooling for tests
v2 cover bindings: python: standalone build tooling for tests
After much delibration about how in-place test builds should work, I have pulled out all of the test extension code from setup.py into a new build_tests.py and updated Makefile.am to invoke it.
This replaces the old python3 setup.py build_ext --inplace with enough setuptools tooling to replicate that behaviour, while also putting it in a better place to avoid problems arising from the deprecation of direct setup.py invocation [1].
Since receipients of the gpiod Python bindings (in tar.gz source or .whl binary format) won't need the test C extensions (and can't run them anyway without the accompanying Python modules) the test extension code has been removed from setup.py rather than duplicated into this new file.
The new build_tests.py creates a temporary directory and redirects all build-time output into it. A final step - handled by build_ext and internal to setuptools - then moves the built _ext.<python_version>-<arch>-<os>.so C extensions into gpiod, tests/gpiosim and tests/procname where they can be imported.
Since release packages and tests no longer share the build/ directory I have removed the "rmtree" hack from the "build_ext" override in setup.py. There should be no cases where test extensions can accidentally end up being installed into the system/user Python environment.
To build and run tests:
- make python-tests
- (sudo) make python-tests-run (I need root for the gpiosim module)
Tested with setuptools==68.2.2 (Sep 2023), setuptools==59.8.0 (Dec 2021) and setuptools==42.0.2 (Dec 2019).
Changes v1 -> v2:
- Add the gpiod ext module to build_tests.py (using glob for sources)
- Formalize the command for running tests by adding it to Makefile.am
- Add build_tests.py output artefacts to Makefile.am's clean-local target
- Sort distutils.log imports (DEBUG before set_verbosity)
Phil Howard (1):
bindings: python: standalone build tooling for tests
bindings/python/Makefile.am | 33 +++++++++--- bindings/python/build_tests.py | 95 ++++++++++++++++++++++++++++++++++ bindings/python/setup.py | 28 +--------- 3 files changed, 121 insertions(+), 35 deletions(-) create mode 100644 bindings/python/build_tests.py
Signature · 1 line
2.34.1
PATCH v2 1/1 bindings: python: standalone build tooling for tests
Move extension definitions and tooling for building tests into build_tests.py and update Makefile.am to call it with appropriate path prefixes.
build_tests.py will perform a standalone build of the text extensions, keeping any build noise in a temporary directory and copying the final built modules automatically out to tests/gpiosim and tests/procname.
Add "python-tests-run" to Makefile.am so it's clear how to run the tests.
Add .so object files generated by build_test.py to Makefile.am's clean-local.
--- bindings/python/Makefile.am | 33 +++++++++--- bindings/python/build_tests.py | 95 ++++++++++++++++++++++++++++++++++ bindings/python/setup.py | 28 +--------- 3 files changed, 121 insertions(+), 35 deletions(-) create mode 100644 bindings/python/build_tests.py
bindings/python/Makefile.am +25 -8
@@ -7,19 +7,36 @@ EXTRA_DIST = \ if WITH_TESTS -BUILD_TESTS = 1 +python-tests: + TOP_SRCDIR=$(abs_top_builddir) \ + TOP_BUILDDIR=$(abs_top_builddir) \ + $(PYTHON) build_tests.py + +python-tests-run: + PYTHONPATH=$(abs_top_srcdir)/bindings/python + LD_LIBRARY_PATH=$(abs_top_builddir)/lib/.libs/:\ +$(abs_top_builddir)/tests/gpiosim/.libs/ \ + python3 -B -m tests + +else + +python-tests: endif -all-local: - GPIOD_WITH_TESTS=$(BUILD_TESTS) \ - $(PYTHON) setup.py build_ext --inplace \ - --include-dirs=$(top_srcdir)/include/:$(top_srcdir)/tests/gpiosim/ \ - --library-dirs=$(top_builddir)/lib/.libs/:$(top_srcdir)/tests/gpiosim/.libs/ +clean-local: + rm -rf dist + rm -f gpiod/_ext.*.so + rm -f tests/*/_ext.*.so + +all-local: python-tests + CFLAGS="-I$(abs_top_srcdir)/include/ -I$(abs_top_srcdir)/tests/gpiosim \ + -L$(abs_top_builddir)/lib/.libs/ -L$(abs_top_builddir)/tests/gpiosim/.libs/" \ + $(PYTHON) -m build install-exec-local: - GPIOD_WITH_TESTS= \ - $(PYTHON) setup.py install --prefix=$(DESTDIR)$(prefix) + $(PYTHON) -m pip install dist/*.whl \ + --prefix=$(DESTDIR)$(prefix) SUBDIRS = gpiod
bindings/python/build_tests.py new file +95 -0
95 lines not on this page · raw message
bindings/python/setup.py +1 -27
@@ -13,7 +13,6 @@ from setuptools.errors import BaseError LINK_SYSTEM_LIBGPIOD = getenv("LINK_SYSTEM_LIBGPIOD") == "1" LIBGPIOD_MINIMUM_VERSION = "2.1" LIBGPIOD_VERSION = getenv("LIBGPIOD_VERSION") -GPIOD_WITH_TESTS = getenv("GPIOD_WITH_TESTS") == "1" SRC_BASE_URL = "https://mirrors.edge.kernel.org/pub/software/libs/libgpiod/" TAR_FILENAME = "libgpiod-{version}.tar.gz" ASC_FILENAME = "sha256sums.asc"
@@ -189,11 +188,6 @@ class build_ext(orig_build_ext): super().run() - # We don't ever want the module tests directory in our package - # since this might include gpiosim._ext or procname._ext from a - # previous dirty build tree. - rmtree(path.join(self.build_lib, "tests"), ignore_errors=True) - class sdist(orig_sdist): """
@@ -226,32 +220,12 @@ gpiod_ext = Extension( extra_compile_args=["-Wall", "-Wextra"], ) -gpiosim_ext = Extension( - "tests.gpiosim._ext", - sources=["tests/gpiosim/ext.c"], - define_macros=[("_GNU_SOURCE", "1")], - libraries=["gpiosim"], - extra_compile_args=["-Wall", "-Wextra"], -) - -procname_ext = Extension( - "tests.procname._ext", - sources=["tests/procname/ext.c"], - define_macros=[("_GNU_SOURCE", "1")], - extra_compile_args=["-Wall", "-Wextra"], -) - -extensions = [gpiod_ext] -if GPIOD_WITH_TESTS: - extensions.append(gpiosim_ext) - extensions.append(procname_ext) - setup( name="gpiod", url="https://git.kernel.org/pub/scm/libs/libgpiod/libgpiod.git", packages=find_packages(exclude=["tests", "tests.*"]), python_requires=">=3.9.0", - ext_modules=extensions, + ext_modules=[gpiod_ext], cmdclass={"build_ext": build_ext, "sdist": sdist}, version=__version__, author="Bartosz Golaszewski",
Signature · 1 line
2.34.1
Quoted 9 lines · Phil Howard
Move extension definitions and tooling for building tests into
build_tests.pyand update Makefile.am to call it with appropriate path prefixes.
build_tests.pywill perform a standalone build of the text extensions, keeping any build noise in a temporary directory and copying the final built modules automatically out totests/gpiosimandtests/procname.
Quoted 7 lines
Add "python-tests-run" to Makefile.am so it's clear how to run the tests.
Add .so object files generated by
build_test.pyto Makefile.am's clean-local.Signed-off-by Phil Howard <[email protected]>---
Thanks, this works quite nicely. I applied it and will release python bindings v2.1.2 to pypi.
Bart