Skip to contentSkip to conversation

bindings: python: add support for free-threaded Python

PATCH Superseded Vincent Fazio 2 participants 11 messages May 22 – May 27

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover bindings: python: add support for free-threaded Python

This short series adds support for free-threaded Python and adds a new test suite to provide some confidence in the suitability of the bindings being used within a multi-threaded context.

A changelog is also added to keep track of changes between releases.

Note:
The new threaded test suite was run on multiple machines in excess of 1M+ iterations in an attempt to make sure they were bug free but due to the nature of the problem, they may not be absolutely error free.

Vincent Fazio (3):
bindings: python: tests: add multi-threaded tests
bindings: python: support free-threaded CPython
bindings: python: add a changelog

 bindings/python/CHANGELOG.md             |  44 ++
 bindings/python/MANIFEST.in              |   1 +
 bindings/python/README.md                |   8 +-
 bindings/python/gpiod/ext/module.c       |   5 +-
 bindings/python/pyproject.toml           |   3 +-
 bindings/python/tests/__main__.py        |   1 +
 bindings/python/tests/gpiosim/ext.c      |   3 +
 bindings/python/tests/helpers.py         |   5 +
 bindings/python/tests/meson.build        |   1 +
 bindings/python/tests/system/ext.c       |   3 +
 bindings/python/tests/tests_threading.py | 742 +++++++++++++++++++++++
 docs/python_api.rst                      |   6 +
 12 files changed, 819 insertions(+), 3 deletions(-)
 create mode 100644 bindings/python/CHANGELOG.md
 create mode 100644 bindings/python/tests/tests_threading.py
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/3 bindings: python: tests: add multi-threaded tests

Add new multi-threaded test cases to ensure the bindings operate as expected and do not lead to crashes or data corruption.

---
 bindings/python/tests/__main__.py        |   1 +
 bindings/python/tests/helpers.py         |   5 +
 bindings/python/tests/meson.build        |   1 +
 bindings/python/tests/tests_threading.py | 739 +++++++++++++++++++++++
 4 files changed, 746 insertions(+)
 create mode 100644 bindings/python/tests/tests_threading.py
bindings/python/tests/__main__.py +1 -0
index 318e0df..591e20d 100644
@@ -14,6 +14,7 @@ from .tests_line_info import *
 from .tests_line_request import *
 from .tests_line_settings import *
 from .tests_module import *
+from .tests_threading import *
 
 set_process_name("python-gpiod")
 
bindings/python/tests/helpers.py +5 -0
index 4abd8b2..a6c02de 100644
@@ -4,6 +4,7 @@
 from __future__ import annotations
 
 import os
+import sys
 from typing import TYPE_CHECKING
 
 if TYPE_CHECKING:
@@ -25,3 +26,7 @@ class LinkGuard:
         tb: TracebackType | None,
     ) -> None:
         os.unlink(self.dst)
+
+
+def is_free_threaded() -> bool:
+    return hasattr(sys, "_is_gil_enabled") and not sys._is_gil_enabled()
bindings/python/tests/meson.build +1 -0
index 16d84cc..3ae36a4 100644
@@ -19,6 +19,7 @@ foreach f : [
   'tests_line_request.py',
   'tests_line_settings.py',
   'tests_module.py',
+  'tests_threading.py',
 ]
   fs.copyfile(f)
 endforeach
bindings/python/tests/tests_threading.py new file +739 -0

739 lines not on this page · raw message

Signature · 1 line
2.43.0
#2 lore raw

Re: [libgpiod][PATCH 1/3] bindings: python: tests: add multi-threaded tests

Quoted 3 lines · Vincent Fazio

Add new multi-threaded test cases to ensure the bindings operate as expected and do not lead to crashes or data corruption.

Awesome work, thanks for doing this!

I assume these tests effectively don't test anything in single-threaded python?

Quoted 2 lines

---

...

bindings/python/tests/tests_threading.py line 99

Quoted 10 lines
bindings/python/tests/tests_threading.py new file +10 -0
+
+
+class ThreadedTestCase(TestCase):
+    NUM_THREADS: ClassVar[int]
+    ITERATIONS: ClassVar[int]
+    TIMEOUT: ClassVar[int]
+
+    def shortDescription(self) -> None:
+        return None
+

Is this needed? Tests seem to work without it.

Overall looks solid, I will queue it shortly.

Bart

#3 lore raw

Re: [libgpiod][PATCH 1/3] bindings: python: tests: add multi-threaded tests

Quoted 10 lines · Bartosz Golaszewski
Quoted 3 lines · Vincent Fazio

Add new multi-threaded test cases to ensure the bindings operate as expected and do not lead to crashes or data corruption.

Awesome work, thanks for doing this!

I assume these tests effectively don't test anything in single-threaded python?

The tests still run since there's a small chance that the Python interpreter could switch threads at an inopportune time and cause issues. We could try to bring this out with sys.setswitchinterval [0][1].

It didn't seem like something we needed to test explicitly because if we're passing on a free-threaded build, we're more than likely going to pass on a standard GIL-enabled build if proper synchronization is used.

Quoted 5 lines

bindings/python/tests/tests_threading.py line 99

Quoted 3 lines
bindings/python/tests/tests_threading.py new file +3 -0
+    def shortDescription(self) -> None:
+        return None
+

Is this needed? Tests seem to work without it.

It's a short-cut to stop unittest from printing out the first line of the docstring as the test description [2] which I found irritating while debugging in verbose mode. Instead of trying to make the docstrings more succinct, I just neutered the feature by stubbing this out.

-Vincent

[0]: https://docs.python.org/3/library/sys.html#sys.setswitchinterval
[1]: https://py-free-threading.github.io/testing/#validating-thread-safety-with-testing
[2]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.shortDescription

#4 lore raw

Re: [libgpiod][PATCH 1/3] bindings: python: tests: add multi-threaded tests

Quoted 24 lines · Vincent Fazio
Quoted 10 lines · Bartosz Golaszewski
Quoted 3 lines · Vincent Fazio

Add new multi-threaded test cases to ensure the bindings operate as expected and do not lead to crashes or data corruption.

Awesome work, thanks for doing this!

I assume these tests effectively don't test anything in single-threaded python?

The tests still run since there's a small chance that the Python interpreter could switch threads at an inopportune time and cause issues. We could try to bring this out with sys.setswitchinterval [0][1].

It didn't seem like something we needed to test explicitly because if we're passing on a free-threaded build, we're more than likely going to pass on a standard GIL-enabled build if proper synchronization is used.

Quoted 4 lines
Quoted 3 lines
bindings/python/tests/tests_threading.py new file +3 -0
+    def shortDescription(self) -> None:
+        return None
+
Quoted 8 lines
Quoted 1 line

Is this needed? Tests seem to work without it.

It's a short-cut to stop unittest from printing out the first line of the docstring as the test description [2] which I found irritating while debugging in verbose mode. Instead of trying to make the docstrings more succinct, I just neutered the feature by stubbing this out.

Would you mind just commenting on it here since you're already sending v2 anyway?

Bart

Quoted 6 lines

-Vincent

[0]: https://docs.python.org/3/library/sys.html#sys.setswitchinterval
[1]: https://py-free-threading.github.io/testing/#validating-thread-safety-with-testing
[2]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.shortDescription

#5 lore raw

PATCH 2/3 bindings: python: support free-threaded CPython

In reply to Vincent Fazio

PEP 703 [0] discusses making the GIL optional in certain builds of the CPython interpreter.

This build option was available experiementally in Python 3.13 but has since been stabilized in Python 3.14 per PEP 779 [1].

According to the porting guide [2], there is no strict requirement that C extensions must be thread-safe.

Experiments have shown that no logic changes are required if callers use sychronization mechanisms provided by the Python standard library.

The documentation has been updated to call this out specifically using terminology from the porting guide [3].

[0]: https://peps.python.org/pep-0703/
[1]: https://peps.python.org/pep-0779/
[2]: https://py-free-threading.github.io/porting/#define-and-document-thread-safety-guarantees
[3]: https://py-free-threading.github.io/documentation-principles/#free-threading-terminology

---
 bindings/python/README.md                | 8 +++++++-
 bindings/python/gpiod/ext/module.c       | 5 ++++-
 bindings/python/pyproject.toml           | 2 +-
 bindings/python/tests/gpiosim/ext.c      | 3 +++
 bindings/python/tests/system/ext.c       | 3 +++
 bindings/python/tests/tests_threading.py | 3 +++
 docs/python_api.rst                      | 6 ++++++
 7 files changed, 27 insertions(+), 3 deletions(-)
bindings/python/README.md +7 -1
index 2faa6f4..f3cd77a 100644
@@ -5,10 +5,16 @@
 
 These are the official Python bindings for [libgpiod](https://git.kernel.org/pub/scm/libs/libgpiod/libgpiod.git/about/).
 
+Both GIL-enabled and free-threaded CPython are supported.
+
+The Python bindings, much like the C API they wrap, are not thread-safe and do
+require external synchronization by the caller to serialize access to objects
+shared across threads.
+
 The gpiod library has been vendored into this package for your convenience and
 this version of gpiod is independent from your system package.
 
-Binary wheels are not provided. The source package requires python3-dev.
+Binary wheels are provided for some platforms. The source package requires python3-dev.
 
 ## Rationale
 
bindings/python/gpiod/ext/module.c +4 -1
index e567f07..21ea519 100644
@@ -188,7 +188,10 @@ static int module_exec(PyObject* module)
 
 static struct PyModuleDef_Slot module_slots[] = {
 	{ Py_mod_exec, module_exec },
-	{ },
+#if PY_VERSION_HEX >= 0x030E0000 && defined(Py_GIL_DISABLED)
+	{Py_mod_gil, Py_MOD_GIL_NOT_USED},
+#endif
+	{0, NULL},
 };
 
 static PyModuleDef module_def = {
bindings/python/pyproject.toml +1 -1
index 98bb44c..7c4474a 100644
@@ -27,6 +27,7 @@ classifiers = [
   "Programming Language :: Python :: 3.12",
   "Programming Language :: Python :: 3.13",
   "Programming Language :: Python :: 3.14",
+  "Programming Language :: Python :: Free Threading :: 3 - Stable",
 ]
 
 [project.urls]
@@ -70,5 +71,4 @@ ignore=[
 
 [tool.cibuildwheel]
 build = "cp*"
-skip = "cp31?t-*"  # Do not build free-threaded wheels
 archs = ["x86_64", "aarch64"]
bindings/python/tests/gpiosim/ext.c +3 -0
index c15ebf5..cc7bd37 100644
@@ -367,6 +367,9 @@ static void free_module_state(void *mod)
 
 static struct PyModuleDef_Slot module_slots[] = {
 	{ Py_mod_exec, module_exec },
+#if PY_VERSION_HEX >= 0x030E0000 && defined(Py_GIL_DISABLED)
+	{Py_mod_gil, Py_MOD_GIL_NOT_USED},
+#endif
 	{ 0, NULL },
 };
 
bindings/python/tests/system/ext.c +3 -0
index 8f451fc..8b307f6 100644
@@ -68,6 +68,9 @@ static PyMethodDef module_methods[] = {
 };
 
 static struct PyModuleDef_Slot module_slots[] = {
+#if PY_VERSION_HEX >= 0x030E0000 && defined(Py_GIL_DISABLED)
+	{Py_mod_gil, Py_MOD_GIL_NOT_USED},
+#endif
 	{ 0, NULL },
 };
 
bindings/python/tests/tests_threading.py +3 -0
index e9f551f..faaead0 100644
@@ -1,3 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+# SPDX-FileCopyrightText: 2026 Vincent Fazio <[email protected]>
+
 import errno
 import fcntl
 import os
docs/python_api.rst +6 -0
index 2c4f59d..57d45ff 100644
@@ -17,6 +17,12 @@ easily through Python scripts, enabling tasks such as reading input values,
 setting outputs, monitoring events, and configuring more fine-grained pin
 options.
 
+The Python bindings, much like the C API they wrap, are not thread-safe and do
+require external synchronization by the caller to serialize access to objects
+shared across threads.
+
+The bindings support both GIL-enabled and free-threaded CPython interpreters.
+
 .. note::
    Python bindings require python3 support and libpython development files for
    building from sources.
Signature · 1 line
2.43.0
#6 lore raw

Re: [libgpiod][PATCH 2/3] bindings: python: support free-threaded CPython

Quoted 36 lines · Vincent Fazio

PEP 703 [0] discusses making the GIL optional in certain builds of the CPython interpreter.

This build option was available experiementally in Python 3.13 but has since been stabilized in Python 3.14 per PEP 779 [1].

According to the porting guide [2], there is no strict requirement that C extensions must be thread-safe.

Experiments have shown that no logic changes are required if callers use sychronization mechanisms provided by the Python standard library.

The documentation has been updated to call this out specifically using terminology from the porting guide [3].

[0]: https://peps.python.org/pep-0703/
[1]: https://peps.python.org/pep-0779/
[2]: https://py-free-threading.github.io/porting/#define-and-document-thread-safety-guarantees
[3]: https://py-free-threading.github.io/documentation-principles/#free-threading-terminology

---
 bindings/python/README.md                | 8 +++++++-
 bindings/python/gpiod/ext/module.c       | 5 ++++-
 bindings/python/pyproject.toml           | 2 +-
 bindings/python/tests/gpiosim/ext.c      | 3 +++
 bindings/python/tests/system/ext.c       | 3 +++
 bindings/python/tests/tests_threading.py | 3 +++
 docs/python_api.rst                      | 6 ++++++
 7 files changed, 27 insertions(+), 3 deletions(-)
bindings/python/README.md +0 -0
index 2faa6f4..f3cd77a 100644

bindings/python/README.md line 12

Quoted 8 lines
bindings/python/README.md +5 -0
@@ -5,10 +5,16 @@
 
 These are the official Python bindings for [libgpiod](https://git.kernel.org/pub/scm/libs/libgpiod/libgpiod.git/about/).
 
+Both GIL-enabled and free-threaded CPython are supported.
+
+The Python bindings, much like the C API they wrap, are not thread-safe and do
+require external synchronization by the caller to serialize access to objects
+shared across threads.

Do we need to document which classes and methods need synchronization?

Bart

#7 lore raw

Re: [libgpiod][PATCH 2/3] bindings: python: support free-threaded CPython

Quoted 6 lines · Bartosz Golaszewski

bindings/python/README.md line 12

Quoted 4 lines
bindings/python/README.md +3 -0
+The Python bindings, much like the C API they wrap, are not thread-safe and do
+require external synchronization by the caller to serialize access to objects
+shared across threads.

Do we need to document which classes and methods need synchronization?

This seems fair. I already did some of that documentation in the threaded unit test so I can copy some of that into the docs API consumers actually read.

I'll fixup this patch unless you want other fixes folded into the series.

#8 lore raw

Re: [libgpiod][PATCH 2/3] bindings: python: support free-threaded CPython

Quoted 8 lines · Vincent Fazio
Quoted 6 lines · Bartosz Golaszewski

bindings/python/README.md line 12

Quoted 4 lines
bindings/python/README.md +3 -0
+The Python bindings, much like the C API they wrap, are not thread-safe and do
+require external synchronization by the caller to serialize access to objects
+shared across threads.

Do we need to document which classes and methods need synchronization?

Quoted 5 lines

This seems fair. I already did some of that documentation in the threaded unit test so I can copy some of that into the docs API consumers actually read.

I'll fixup this patch unless you want other fixes folded into the series.

No, I don't think there are any other issues I noticed. Thanks!

Bart

#9 lore raw

PATCH 3/3 bindings: python: add a changelog

In reply to Vincent Fazio

Add a changelog to summarize changes to the bindings between releases.

---
 bindings/python/CHANGELOG.md   | 44 ++++++++++++++++++++++++++++++++++
 bindings/python/MANIFEST.in    |  1 +
 bindings/python/pyproject.toml |  1 +
 3 files changed, 46 insertions(+)
 create mode 100644 bindings/python/CHANGELOG.md
bindings/python/CHANGELOG.md new file +44 -0
new file mode 100644
index 0000000..0f5dce3
@@ -0,0 +1,44 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is loosely based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
+
+## [Unreleased]
+
+### Added
+
+- Add free-threaded CPython support
+- Generate wheels for CPython 3.14t (free-threaded)
+- Include the license file into the source and binary distributions
+
+### Changed
+
+- Remove `wheel` from build dependencies
+- Allow `Chip.close` to be called multiple times without raising an error
+- Allow `LineRequest.release` to be called multiple times without raising an error
+- (Internal) Add more patterns to `.gitignore`
+- (Internal) Update linter configuration
+- (Internal) Refactor code to conform to updated linter configuration
+- (Internal) Refactor code to use Python 3.10 type annotations
+- (Internal) Introduce a dependency group for linting
+- (Internal) Simplify Chip finalization
+- (Internal) Simplify LineRequest finalization
+- (Internal) Use newer functions/macros in the C extension.
+- (Internal) Migrate to multi-phase module initialization
+- (Internal) Removed duplicate call to `gpiod_line_settings_set_edge_detection`
+- (Internal) Add multi-threaded tests
+
+### Fixed
+
+- Fix refcount imbalance due to missing `Py_INCREF` when querying line names
+- Fix refcount imbalance by not calling `Py_DECREF` when `PyList_SetItem` fails
+- Require `setuptools` v77 or higher to avoid build errors
+
+### Security
+
+- Fix possible buffer overflows when setting/getting `LineRequest` values
+
+### Removed
+
+- Support for Python 3.9 has been dropped
bindings/python/MANIFEST.in +1 -0
index dc7daee..e59387a 100644
@@ -5,6 +5,7 @@ include setup.py
 include build_tests.py
 include README.md
 include libgpiod-version.txt
+include CHANGELOG.md
 
 recursive-include gpiod *.py
 recursive-include tests *.py
bindings/python/pyproject.toml +1 -0
index 7c4474a..3337306 100644
@@ -33,6 +33,7 @@ classifiers = [
 [project.urls]
 Homepage = "https://git.kernel.org/pub/scm/libs/libgpiod/libgpiod.git"
 Issues = "https://github.com/brgl/libgpiod/issues/"
+Changelog = "https://github.com/brgl/libgpiod/blob/master/bindings/python/CHANGELOG.md"
 
 [dependency-groups]
 lint = ["mypy>=1.18.1", "ruff~=0.15.0", "types-setuptools", "packaging"]
Signature · 1 line
2.43.0
#10 lore raw
In reply to Vincent Fazio
Quoted 5 lines · Vincent Fazio

This short series adds support for free-threaded Python and adds a new test suite to provide some confidence in the suitability of the bindings being used within a multi-threaded context.

Quoted 7 lines

A changelog is also added to keep track of changes between releases.

Note:
The new threaded test suite was run on multiple machines in excess of 1M+ iterations in an attempt to make sure they were bug free but due to the nature of the problem, they may not be absolutely error free.

I forgot to link test logs, but the latest pipeline [0] shows all passes and includes an extra commit (not submitted) to run the free-threaded test for 100k iterations for warm and fuzzies.

[0]: https://github.com/vfazio/libgpiod/actions/runs/26309499084

-Vincent