bindings: python: add support for free-threaded Python
Subsystems (2)
cover bindings: python: add support for free-threaded Python
This short series adds support for free-threaded Python and adds a new test suite to provide some confidence in the suitability of the bindings being used within a multi-threaded context.
A changelog is also added to keep track of changes between releases.
Note:
The new threaded test suite was run on multiple machines in excess of 1M+ iterations in an attempt to make sure they were bug free but due to the nature of the problem, they may not be absolutely error free.
Vincent Fazio (3):
bindings: python: tests: add multi-threaded tests
bindings: python: support free-threaded CPython
bindings: python: add a changelog
bindings/python/CHANGELOG.md | 44 ++ bindings/python/MANIFEST.in | 1 + bindings/python/README.md | 8 +- bindings/python/gpiod/ext/module.c | 5 +- bindings/python/pyproject.toml | 3 +- bindings/python/tests/__main__.py | 1 + bindings/python/tests/gpiosim/ext.c | 3 + bindings/python/tests/helpers.py | 5 + bindings/python/tests/meson.build | 1 + bindings/python/tests/system/ext.c | 3 + bindings/python/tests/tests_threading.py | 742 +++++++++++++++++++++++ docs/python_api.rst | 6 + 12 files changed, 819 insertions(+), 3 deletions(-) create mode 100644 bindings/python/CHANGELOG.md create mode 100644 bindings/python/tests/tests_threading.py
Signature · 1 line
2.43.0
PATCH 1/3 bindings: python: tests: add multi-threaded tests
Add new multi-threaded test cases to ensure the bindings operate as expected and do not lead to crashes or data corruption.
--- bindings/python/tests/__main__.py | 1 + bindings/python/tests/helpers.py | 5 + bindings/python/tests/meson.build | 1 + bindings/python/tests/tests_threading.py | 739 +++++++++++++++++++++++ 4 files changed, 746 insertions(+) create mode 100644 bindings/python/tests/tests_threading.py
bindings/python/tests/__main__.py +1 -0
@@ -14,6 +14,7 @@ from .tests_line_info import * from .tests_line_request import * from .tests_line_settings import * from .tests_module import * +from .tests_threading import * set_process_name("python-gpiod")
bindings/python/tests/helpers.py +5 -0
@@ -4,6 +4,7 @@ from __future__ import annotations import os +import sys from typing import TYPE_CHECKING if TYPE_CHECKING:
@@ -25,3 +26,7 @@ class LinkGuard: tb: TracebackType | None, ) -> None: os.unlink(self.dst) + + +def is_free_threaded() -> bool: + return hasattr(sys, "_is_gil_enabled") and not sys._is_gil_enabled()
bindings/python/tests/meson.build +1 -0
@@ -19,6 +19,7 @@ foreach f : [ 'tests_line_request.py', 'tests_line_settings.py', 'tests_module.py', + 'tests_threading.py', ] fs.copyfile(f) endforeach
bindings/python/tests/tests_threading.py new file +739 -0
739 lines not on this page · raw message
Signature · 1 line
2.43.0
Bartosz Golaszewski
maintainerRe: [libgpiod][PATCH 1/3] bindings: python: tests: add multi-threaded tests
Quoted 3 lines · Vincent Fazio
Add new multi-threaded test cases to ensure the bindings operate as expected and do not lead to crashes or data corruption.
Awesome work, thanks for doing this!
I assume these tests effectively don't test anything in single-threaded python?
Quoted 2 lines
Signed-off-by Vincent Fazio <[email protected]>---
...
bindings/python/tests/tests_threading.py line 99
Quoted 10 lines
bindings/python/tests/tests_threading.py new file +10
-0+ + +class ThreadedTestCase(TestCase): + NUM_THREADS: ClassVar[int] + ITERATIONS: ClassVar[int] + TIMEOUT: ClassVar[int] + + def shortDescription(self) -> None: + return None +
Is this needed? Tests seem to work without it.
Overall looks solid, I will queue it shortly.
Bart
Re: [libgpiod][PATCH 1/3] bindings: python: tests: add multi-threaded tests
Quoted 10 lines · Bartosz Golaszewski
Quoted 3 lines · Vincent Fazio
Add new multi-threaded test cases to ensure the bindings operate as expected and do not lead to crashes or data corruption.
Awesome work, thanks for doing this!
I assume these tests effectively don't test anything in single-threaded python?
The tests still run since there's a small chance that the Python interpreter could switch threads at an inopportune time and cause issues. We could try to bring this out with sys.setswitchinterval [0][1].
It didn't seem like something we needed to test explicitly because if we're passing on a free-threaded build, we're more than likely going to pass on a standard GIL-enabled build if proper synchronization is used.
Quoted 5 lines
bindings/python/tests/tests_threading.py line 99
Quoted 3 lines
bindings/python/tests/tests_threading.py new file +3
-0+ def shortDescription(self) -> None: + return None +Is this needed? Tests seem to work without it.
It's a short-cut to stop unittest from printing out the first line of the docstring as the test description [2] which I found irritating while debugging in verbose mode. Instead of trying to make the docstrings more succinct, I just neutered the feature by stubbing this out.
-Vincent
[0]: https://docs.python.org/3/library/sys.html#sys.setswitchinterval
[1]: https://py-free-threading.github.io/testing/#validating-thread-safety-with-testing
[2]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.shortDescription
Bartosz Golaszewski
maintainerRe: [libgpiod][PATCH 1/3] bindings: python: tests: add multi-threaded tests
Quoted 24 lines · Vincent Fazio
Quoted 10 lines · Bartosz Golaszewski
Quoted 3 lines · Vincent Fazio
Add new multi-threaded test cases to ensure the bindings operate as expected and do not lead to crashes or data corruption.
Awesome work, thanks for doing this!
I assume these tests effectively don't test anything in single-threaded python?
The tests still run since there's a small chance that the Python interpreter could switch threads at an inopportune time and cause issues. We could try to bring this out with
sys.setswitchinterval[0][1].It didn't seem like something we needed to test explicitly because if we're passing on a free-threaded build, we're more than likely going to pass on a standard GIL-enabled build if proper synchronization is used.
Quoted 4 lines
Quoted 3 lines
bindings/python/tests/tests_threading.py new file +3
-0+ def shortDescription(self) -> None: + return None +
Quoted 8 lines
Quoted 1 line
Is this needed? Tests seem to work without it.
It's a short-cut to stop unittest from printing out the first line of the docstring as the test description [2] which I found irritating while debugging in verbose mode. Instead of trying to make the docstrings more succinct, I just neutered the feature by stubbing this out.
Would you mind just commenting on it here since you're already sending v2 anyway?
Bart
PATCH 2/3 bindings: python: support free-threaded CPython
In reply to Vincent FazioPEP 703 [0] discusses making the GIL optional in certain builds of the CPython interpreter.
This build option was available experiementally in Python 3.13 but has since been stabilized in Python 3.14 per PEP 779 [1].
According to the porting guide [2], there is no strict requirement that C extensions must be thread-safe.
Experiments have shown that no logic changes are required if callers use sychronization mechanisms provided by the Python standard library.
The documentation has been updated to call this out specifically using terminology from the porting guide [3].
[0]: https://peps.python.org/pep-0703/
[1]: https://peps.python.org/pep-0779/
[2]: https://py-free-threading.github.io/porting/#define-and-document-thread-safety-guarantees
[3]: https://py-free-threading.github.io/documentation-principles/#free-threading-terminology
--- bindings/python/README.md | 8 +++++++- bindings/python/gpiod/ext/module.c | 5 ++++- bindings/python/pyproject.toml | 2 +- bindings/python/tests/gpiosim/ext.c | 3 +++ bindings/python/tests/system/ext.c | 3 +++ bindings/python/tests/tests_threading.py | 3 +++ docs/python_api.rst | 6 ++++++ 7 files changed, 27 insertions(+), 3 deletions(-)
bindings/python/README.md +7 -1
@@ -5,10 +5,16 @@ These are the official Python bindings for [libgpiod](https://git.kernel.org/pub/scm/libs/libgpiod/libgpiod.git/about/). +Both GIL-enabled and free-threaded CPython are supported. + +The Python bindings, much like the C API they wrap, are not thread-safe and do +require external synchronization by the caller to serialize access to objects +shared across threads. + The gpiod library has been vendored into this package for your convenience and this version of gpiod is independent from your system package. -Binary wheels are not provided. The source package requires python3-dev. +Binary wheels are provided for some platforms. The source package requires python3-dev. ## Rationale
bindings/python/gpiod/ext/module.c +4 -1
@@ -188,7 +188,10 @@ static int module_exec(PyObject* module)static struct PyModuleDef_Slot module_slots[] = { { Py_mod_exec, module_exec }, - { }, +#if PY_VERSION_HEX >= 0x030E0000 && defined(Py_GIL_DISABLED) + {Py_mod_gil, Py_MOD_GIL_NOT_USED}, +#endif + {0 ,NULL }, };static PyModuleDef module_def = {
bindings/python/pyproject.toml +1 -1
@@ -27,6 +27,7 @@ classifiers = [ "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.14", + "Programming Language :: Python :: Free Threading :: 3 - Stable", ] [project.urls]
@@ -70,5 +71,4 @@ ignore=[ [tool.cibuildwheel] build = "cp*" -skip = "cp31?t-*" # Do not build free-threaded wheels archs = ["x86_64", "aarch64"]
bindings/python/tests/gpiosim/ext.c +3 -0
@@ -367,6 +367,9 @@ static void free_module_state(void *mod)static struct PyModuleDef_Slot module_slots[] = { { Py_mod_exec, module_exec }, +#if PY_VERSION_HEX >= 0x030E0000 && defined(Py_GIL_DISABLED) + {Py_mod_gil, Py_MOD_GIL_NOT_USED}, +#endif {0 ,NULL }, };
bindings/python/tests/system/ext.c +3 -0
@@ -68,6 +68,9 @@ static PyMethodDef module_methods[] = { };static struct PyModuleDef_Slot module_slots[] = { +#if PY_VERSION_HEX >= 0x030E0000 && defined(Py_GIL_DISABLED) + {Py_mod_gil, Py_MOD_GIL_NOT_USED}, +#endif {0 ,NULL }, };
bindings/python/tests/tests_threading.py +3 -0
@@ -1,3 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# SPDX-FileCopyrightText: 2026 Vincent Fazio <[email protected]> + import errno import fcntl import os
docs/python_api.rst +6 -0
@@ -17,6 +17,12 @@ easily through Python scripts, enabling tasks such as reading input values, setting outputs, monitoring events, and configuring more fine-grained pin options. +The Python bindings, much like the C API they wrap, are not thread-safe and do +require external synchronization by the caller to serialize access to objects +shared across threads. + +The bindings support both GIL-enabled and free-threaded CPython interpreters. + .. note:: Python bindings require python3 support and libpython development files for building from sources.
Signature · 1 line
2.43.0
Bartosz Golaszewski
maintainerRe: [libgpiod][PATCH 2/3] bindings: python: support free-threaded CPython
Quoted 36 lines · Vincent Fazio
PEP 703 [0] discusses making the GIL optional in certain builds of the CPython interpreter.
This build option was available experiementally in Python 3.13 but has since been stabilized in Python 3.14 per PEP 779 [1].
According to the porting guide [2], there is no strict requirement that C extensions must be thread-safe.
Experiments have shown that no logic changes are required if callers use sychronization mechanisms provided by the Python standard library.
The documentation has been updated to call this out specifically using terminology from the porting guide [3].
[0]: https://peps.python.org/pep-0703/
[1]: https://peps.python.org/pep-0779/
[2]: https://py-free-threading.github.io/porting/#define-and-document-thread-safety-guarantees
[3]: https://py-free-threading.github.io/documentation-principles/#free-threading-terminologySigned-off-by Vincent Fazio <[email protected]>--- bindings/python/README.md | 8 +++++++- bindings/python/gpiod/ext/module.c | 5 ++++- bindings/python/pyproject.toml | 2 +- bindings/python/tests/gpiosim/ext.c | 3 +++ bindings/python/tests/system/ext.c | 3 +++ bindings/python/tests/tests_threading.py | 3 +++ docs/python_api.rst | 6 ++++++ 7 files changed, 27 insertions(+), 3 deletions(-)bindings/python/README.md +0
-0
bindings/python/README.md line 12
Quoted 8 lines
bindings/python/README.md +5
-0@@ -5,10 +5,16 @@ These are the official Python bindings for [libgpiod](https://git.kernel.org/pub/scm/libs/libgpiod/libgpiod.git/about/). +Both GIL-enabled and free-threaded CPython are supported. + +The Python bindings, much like the C API they wrap, are not thread-safe and do +require external synchronization by the caller to serialize access to objects +shared across threads.
Do we need to document which classes and methods need synchronization?
Bart
Re: [libgpiod][PATCH 2/3] bindings: python: support free-threaded CPython
Quoted 6 lines · Bartosz Golaszewski
bindings/python/README.md line 12
Quoted 4 lines
bindings/python/README.md +3
-0+The Python bindings, much like the C API they wrap, are not thread-safe and do +require external synchronization by the caller to serialize access to objects +shared across threads.Do we need to document which classes and methods need synchronization?
This seems fair. I already did some of that documentation in the threaded unit test so I can copy some of that into the docs API consumers actually read.
I'll fixup this patch unless you want other fixes folded into the series.
Bartosz Golaszewski
maintainerRe: [libgpiod][PATCH 2/3] bindings: python: support free-threaded CPython
Quoted 8 lines · Vincent Fazio
Quoted 6 lines · Bartosz Golaszewski
bindings/python/README.md line 12
Quoted 4 lines
bindings/python/README.md +3
-0+The Python bindings, much like the C API they wrap, are not thread-safe and do +require external synchronization by the caller to serialize access to objects +shared across threads.Do we need to document which classes and methods need synchronization?
Quoted 5 lines
This seems fair. I already did some of that documentation in the threaded unit test so I can copy some of that into the docs API consumers actually read.
I'll fixup this patch unless you want other fixes folded into the series.
No, I don't think there are any other issues I noticed. Thanks!
Bart
PATCH 3/3 bindings: python: add a changelog
In reply to Vincent FazioAdd a changelog to summarize changes to the bindings between releases.
--- bindings/python/CHANGELOG.md | 44 ++++++++++++++++++++++++++++++++++ bindings/python/MANIFEST.in | 1 + bindings/python/pyproject.toml | 1 + 3 files changed, 46 insertions(+) create mode 100644 bindings/python/CHANGELOG.md
bindings/python/CHANGELOG.md new file +44 -0
@@ -0,0 +1,44 @@ +# Changelog + +All notable changes to this project will be documented in this file. + +The format is loosely based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). + +## [Unreleased] + +### Added + +- Add free-threaded CPython support +- Generate wheels for CPython 3.14t (free-threaded) +- Include the license file into the source and binary distributions + +### Changed + +- Remove `wheel` from build dependencies +- Allow `Chip.close` to be called multiple times without raising an error +- Allow `LineRequest.release` to be called multiple times without raising an error +- (Internal) Add more patterns to `.gitignore` +- (Internal) Update linter configuration +- (Internal) Refactor code to conform to updated linter configuration +- (Internal) Refactor code to use Python 3.10 type annotations +- (Internal) Introduce a dependency group for linting +- (Internal) Simplify Chip finalization +- (Internal) Simplify LineRequest finalization +- (Internal) Use newer functions/macros in the C extension. +- (Internal) Migrate to multi-phase module initialization +- (Internal) Removed duplicate call to `gpiod_line_settings_set_edge_detection` +- (Internal) Add multi-threaded tests + +### Fixed + +- Fix refcount imbalance due to missing `Py_INCREF` when querying line names +- Fix refcount imbalance by not calling `Py_DECREF` when `PyList_SetItem` fails +- Require `setuptools` v77 or higher to avoid build errors + +### Security + +- Fix possible buffer overflows when setting/getting `LineRequest` values + +### Removed + +- Support for Python 3.9 has been dropped
bindings/python/MANIFEST.in +1 -0
@@ -5,6 +5,7 @@ include setup.py include build_tests.py include README.md include libgpiod-version.txt +include CHANGELOG.md recursive-include gpiod *.py recursive-include tests *.py
bindings/python/pyproject.toml +1 -0
@@ -33,6 +33,7 @@ classifiers = [ [project.urls] Homepage = "https://git.kernel.org/pub/scm/libs/libgpiod/libgpiod.git" Issues = "https://github.com/brgl/libgpiod/issues/" +Changelog = "https://github.com/brgl/libgpiod/blob/master/bindings/python/CHANGELOG.md" [dependency-groups] lint = ["mypy>=1.18.1", "ruff~=0.15.0", "types-setuptools", "packaging"]
Signature · 1 line
2.43.0
Quoted 5 lines · Vincent Fazio
This short series adds support for free-threaded Python and adds a new test suite to provide some confidence in the suitability of the bindings being used within a multi-threaded context.
Quoted 7 lines
A changelog is also added to keep track of changes between releases.
Note:
The new threaded test suite was run on multiple machines in excess of 1M+ iterations in an attempt to make sure they were bug free but due to the nature of the problem, they may not be absolutely error free.
I forgot to link test logs, but the latest pipeline [0] shows all passes and includes an extra commit (not submitted) to run the free-threaded test for 100k iterations for warm and fuzzies.
[0]: https://github.com/vfazio/libgpiod/actions/runs/26309499084
-Vincent