Skip to contentSkip to conversation

bindings: python: add support for free-threaded Python

PATCH In review Vincent Fazio 2 participants 5 messages May 27 – May 28

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover bindings: python: add support for free-threaded Python

This short series adds support for free-threaded Python and adds a new test suite to provide some confidence in the suitability of the bindings being used within a multi-threaded context.

A changelog is also added to keep track of changes between releases.

Note:
The new threaded test suite was run on multiple machines in excess of 1M+ iterations in an attempt to make sure they were bug free but due to the nature of the problem, they may not be absolutely error free.

---
Changes in v2:
- add documentation around the shortDescription stub (Bart)
- add synchronization info to Python class docstrings (Bart)
- Link to v1: https://lore.kernel.org/all/[email protected]/
---
Vincent Fazio (3):
bindings: python: tests: add multi-threaded tests
bindings: python: support free-threaded CPython
bindings: python: add a changelog

 bindings/python/CHANGELOG.md             |  44 ++
 bindings/python/MANIFEST.in              |   1 +
 bindings/python/README.md                |   8 +-
 bindings/python/gpiod/chip.py            |   3 +
 bindings/python/gpiod/chip_info.py       |   4 +-
 bindings/python/gpiod/edge_event.py      |   2 +
 bindings/python/gpiod/ext/module.c       |   5 +-
 bindings/python/gpiod/info_event.py      |   2 +
 bindings/python/gpiod/line_info.py       |   4 +-
 bindings/python/gpiod/line_request.py    |   3 +
 bindings/python/gpiod/line_settings.py   |   2 +
 bindings/python/pyproject.toml           |   3 +-
 bindings/python/tests/__main__.py        |   1 +
 bindings/python/tests/gpiosim/ext.c      |   3 +
 bindings/python/tests/helpers.py         |   5 +
 bindings/python/tests/meson.build        |   1 +
 bindings/python/tests/system/ext.c       |   3 +
 bindings/python/tests/tests_threading.py | 745 +++++++++++++++++++++++
 docs/python_api.rst                      |   6 +
 19 files changed, 840 insertions(+), 5 deletions(-)
 create mode 100644 bindings/python/CHANGELOG.md
 create mode 100644 bindings/python/tests/tests_threading.py
Signature · 1 line
2.43.0
#1 lore raw

PATCH v2 1/3 bindings: python: tests: add multi-threaded tests

Add new multi-threaded test cases to ensure the bindings operate as expected and do not lead to crashes or data corruption.

---
 bindings/python/tests/__main__.py        |   1 +
 bindings/python/tests/helpers.py         |   5 +
 bindings/python/tests/meson.build        |   1 +
 bindings/python/tests/tests_threading.py | 742 +++++++++++++++++++++++
 4 files changed, 749 insertions(+)
 create mode 100644 bindings/python/tests/tests_threading.py
bindings/python/tests/__main__.py +1 -0
index 318e0df..591e20d 100644
@@ -14,6 +14,7 @@ from .tests_line_info import *
 from .tests_line_request import *
 from .tests_line_settings import *
 from .tests_module import *
+from .tests_threading import *
 
 set_process_name("python-gpiod")
 
bindings/python/tests/helpers.py +5 -0
index 4abd8b2..a6c02de 100644
@@ -4,6 +4,7 @@
 from __future__ import annotations
 
 import os
+import sys
 from typing import TYPE_CHECKING
 
 if TYPE_CHECKING:
@@ -25,3 +26,7 @@ class LinkGuard:
         tb: TracebackType | None,
     ) -> None:
         os.unlink(self.dst)
+
+
+def is_free_threaded() -> bool:
+    return hasattr(sys, "_is_gil_enabled") and not sys._is_gil_enabled()
bindings/python/tests/meson.build +1 -0
index 16d84cc..3ae36a4 100644
@@ -19,6 +19,7 @@ foreach f : [
   'tests_line_request.py',
   'tests_line_settings.py',
   'tests_module.py',
+  'tests_threading.py',
 ]
   fs.copyfile(f)
 endforeach
bindings/python/tests/tests_threading.py new file +742 -0

742 lines not on this page · raw message

Signature · 1 line
2.43.0
#2 lore raw

PATCH v2 2/3 bindings: python: support free-threaded CPython

In reply to Vincent Fazio

PEP 703 [0] discusses making the GIL optional in certain builds of the CPython interpreter.

This build option was available experiementally in Python 3.13 but has since been stabilized in Python 3.14 per PEP 779 [1].

According to the porting guide [2], there is no strict requirement that C extensions must be thread-safe.

Experiments have shown that no logic changes are required if callers use sychronization mechanisms provided by the Python standard library.

The documentation has been updated to call this out specifically using terminology from the porting guide [3].

[0]: https://peps.python.org/pep-0703/
[1]: https://peps.python.org/pep-0779/
[2]: https://py-free-threading.github.io/porting/#define-and-document-thread-safety-guarantees
[3]: https://py-free-threading.github.io/documentation-principles/#free-threading-terminology

---
 bindings/python/README.md                | 8 +++++++-
 bindings/python/gpiod/chip.py            | 3 +++
 bindings/python/gpiod/chip_info.py       | 4 +++-
 bindings/python/gpiod/edge_event.py      | 2 ++
 bindings/python/gpiod/ext/module.c       | 5 ++++-
 bindings/python/gpiod/info_event.py      | 2 ++
 bindings/python/gpiod/line_info.py       | 4 +++-
 bindings/python/gpiod/line_request.py    | 3 +++
 bindings/python/gpiod/line_settings.py   | 2 ++
 bindings/python/pyproject.toml           | 2 +-
 bindings/python/tests/gpiosim/ext.c      | 3 +++
 bindings/python/tests/system/ext.c       | 3 +++
 bindings/python/tests/tests_threading.py | 3 +++
 docs/python_api.rst                      | 6 ++++++
 14 files changed, 45 insertions(+), 5 deletions(-)
bindings/python/README.md +7 -1
index 2faa6f4..f3cd77a 100644
@@ -5,10 +5,16 @@
 
 These are the official Python bindings for [libgpiod](https://git.kernel.org/pub/scm/libs/libgpiod/libgpiod.git/about/).
 
+Both GIL-enabled and free-threaded CPython are supported.
+
+The Python bindings, much like the C API they wrap, are not thread-safe and do
+require external synchronization by the caller to serialize access to objects
+shared across threads.
+
 The gpiod library has been vendored into this package for your convenience and
 this version of gpiod is independent from your system package.
 
-Binary wheels are not provided. The source package requires python3-dev.
+Binary wheels are provided for some platforms. The source package requires python3-dev.
 
 ## Rationale
 
bindings/python/gpiod/chip.py +3 -0
index 6b7b32b..2acb872 100644
@@ -50,6 +50,9 @@ class Chip:
 
         with gpiod.Chip(path="/dev/gpiochip0") as chip:
             do_something(chip)
+
+    Synchronization: objects of this class require external synchronization.
+    Protect calls with a lock when sharing an instance across threads.
     """
 
     def __init__(self, path: str):
bindings/python/gpiod/chip_info.py +3 -1
index 737a45e..1a81a9d 100644
@@ -10,7 +10,9 @@ __all__ = ["ChipInfo"]
 @dataclass(frozen=True, repr=False)
 class ChipInfo:
     """
-    Snapshot of a chip's status.
+    Immutable snapshot of a chip's status.
+
+    Synchronization: none required
     """
 
     name: str
bindings/python/gpiod/edge_event.py +2 -0
index c888cb2..99bb83d 100644
@@ -13,6 +13,8 @@ __all__ = ["EdgeEvent"]
 class EdgeEvent:
     """
     Immutable object containing data about a single edge event.
+
+    Synchronization: none required
     """
 
     class Type(Enum):
bindings/python/gpiod/ext/module.c +4 -1
index e567f07..21ea519 100644
@@ -188,7 +188,10 @@ static int module_exec(PyObject* module)
 
 static struct PyModuleDef_Slot module_slots[] = {
 	{ Py_mod_exec, module_exec },
-	{ },
+#if PY_VERSION_HEX >= 0x030E0000 && defined(Py_GIL_DISABLED)
+	{Py_mod_gil, Py_MOD_GIL_NOT_USED},
+#endif
+	{0, NULL},
 };
 
 static PyModuleDef module_def = {
bindings/python/gpiod/info_event.py +2 -0
index cd2785e..e3dd582 100644
@@ -14,6 +14,8 @@ __all__ = ["InfoEvent"]
 class InfoEvent:
     """
     Immutable object containing data about a single line info event.
+
+    Synchronization: none required
     """
 
     class Type(Enum):
bindings/python/gpiod/line_info.py +3 -1
index d31565e..b11ed42 100644
@@ -12,7 +12,9 @@ __all__ = ["LineInfo"]
 @dataclass(frozen=True, init=False, repr=False)
 class LineInfo:
     """
-    Snapshot of a line's status.
+    Immutable snapshot of a line's status.
+
+    Synchronization: none required
     """
 
     offset: int
bindings/python/gpiod/line_request.py +3 -0
index a271080..dc7cd22 100644
@@ -26,6 +26,9 @@ __all__ = ["LineRequest"]
 class LineRequest:
     """
     Stores the context of a set of requested GPIO lines.
+
+    Synchronization: objects of this class require external synchronization.
+    Protect calls with a lock when sharing an instance across threads.
     """
 
     def __init__(self, req: _ext.Request):
bindings/python/gpiod/line_settings.py +2 -0
index 3752acd..e54bc80 100644
@@ -14,6 +14,8 @@ __all__ = ["LineSettings"]
 class LineSettings:
     """
     Stores a set of line properties.
+
+    Synchronization: none required
     """
 
     direction: Direction = Direction.AS_IS
bindings/python/pyproject.toml +1 -1
index 98bb44c..7c4474a 100644
@@ -27,6 +27,7 @@ classifiers = [
   "Programming Language :: Python :: 3.12",
   "Programming Language :: Python :: 3.13",
   "Programming Language :: Python :: 3.14",
+  "Programming Language :: Python :: Free Threading :: 3 - Stable",
 ]
 
 [project.urls]
@@ -70,5 +71,4 @@ ignore=[
 
 [tool.cibuildwheel]
 build = "cp*"
-skip = "cp31?t-*"  # Do not build free-threaded wheels
 archs = ["x86_64", "aarch64"]
bindings/python/tests/gpiosim/ext.c +3 -0
index c15ebf5..cc7bd37 100644
@@ -367,6 +367,9 @@ static void free_module_state(void *mod)
 
 static struct PyModuleDef_Slot module_slots[] = {
 	{ Py_mod_exec, module_exec },
+#if PY_VERSION_HEX >= 0x030E0000 && defined(Py_GIL_DISABLED)
+	{Py_mod_gil, Py_MOD_GIL_NOT_USED},
+#endif
 	{ 0, NULL },
 };
 
bindings/python/tests/system/ext.c +3 -0
index 8f451fc..8b307f6 100644
@@ -68,6 +68,9 @@ static PyMethodDef module_methods[] = {
 };
 
 static struct PyModuleDef_Slot module_slots[] = {
+#if PY_VERSION_HEX >= 0x030E0000 && defined(Py_GIL_DISABLED)
+	{Py_mod_gil, Py_MOD_GIL_NOT_USED},
+#endif
 	{ 0, NULL },
 };
 
bindings/python/tests/tests_threading.py +3 -0
index a61af5b..6fbf8be 100644
@@ -1,3 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+# SPDX-FileCopyrightText: 2026 Vincent Fazio <[email protected]>
+
 import errno
 import fcntl
 import os
docs/python_api.rst +6 -0
index 2c4f59d..57d45ff 100644
@@ -17,6 +17,12 @@ easily through Python scripts, enabling tasks such as reading input values,
 setting outputs, monitoring events, and configuring more fine-grained pin
 options.
 
+The Python bindings, much like the C API they wrap, are not thread-safe and do
+require external synchronization by the caller to serialize access to objects
+shared across threads.
+
+The bindings support both GIL-enabled and free-threaded CPython interpreters.
+
 .. note::
    Python bindings require python3 support and libpython development files for
    building from sources.
Signature · 1 line
2.43.0
#3 lore raw

PATCH v2 3/3 bindings: python: add a changelog

In reply to Vincent Fazio

Add a changelog to summarize changes to the bindings between releases.

---
 bindings/python/CHANGELOG.md   | 44 ++++++++++++++++++++++++++++++++++
 bindings/python/MANIFEST.in    |  1 +
 bindings/python/pyproject.toml |  1 +
 3 files changed, 46 insertions(+)
 create mode 100644 bindings/python/CHANGELOG.md
bindings/python/CHANGELOG.md new file +44 -0
new file mode 100644
index 0000000..0f5dce3
@@ -0,0 +1,44 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is loosely based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
+
+## [Unreleased]
+
+### Added
+
+- Add free-threaded CPython support
+- Generate wheels for CPython 3.14t (free-threaded)
+- Include the license file into the source and binary distributions
+
+### Changed
+
+- Remove `wheel` from build dependencies
+- Allow `Chip.close` to be called multiple times without raising an error
+- Allow `LineRequest.release` to be called multiple times without raising an error
+- (Internal) Add more patterns to `.gitignore`
+- (Internal) Update linter configuration
+- (Internal) Refactor code to conform to updated linter configuration
+- (Internal) Refactor code to use Python 3.10 type annotations
+- (Internal) Introduce a dependency group for linting
+- (Internal) Simplify Chip finalization
+- (Internal) Simplify LineRequest finalization
+- (Internal) Use newer functions/macros in the C extension.
+- (Internal) Migrate to multi-phase module initialization
+- (Internal) Removed duplicate call to `gpiod_line_settings_set_edge_detection`
+- (Internal) Add multi-threaded tests
+
+### Fixed
+
+- Fix refcount imbalance due to missing `Py_INCREF` when querying line names
+- Fix refcount imbalance by not calling `Py_DECREF` when `PyList_SetItem` fails
+- Require `setuptools` v77 or higher to avoid build errors
+
+### Security
+
+- Fix possible buffer overflows when setting/getting `LineRequest` values
+
+### Removed
+
+- Support for Python 3.9 has been dropped
bindings/python/MANIFEST.in +1 -0
index dc7daee..e59387a 100644
@@ -5,6 +5,7 @@ include setup.py
 include build_tests.py
 include README.md
 include libgpiod-version.txt
+include CHANGELOG.md
 
 recursive-include gpiod *.py
 recursive-include tests *.py
bindings/python/pyproject.toml +1 -0
index 7c4474a..3337306 100644
@@ -33,6 +33,7 @@ classifiers = [
 [project.urls]
 Homepage = "https://git.kernel.org/pub/scm/libs/libgpiod/libgpiod.git"
 Issues = "https://github.com/brgl/libgpiod/issues/"
+Changelog = "https://github.com/brgl/libgpiod/blob/master/bindings/python/CHANGELOG.md"
 
 [dependency-groups]
 lint = ["mypy>=1.18.1", "ruff~=0.15.0", "types-setuptools", "packaging"]
Signature · 1 line
2.43.0
#4 lore raw
In reply to Vincent Fazio
Quoted 4 lines · Vincent Fazio

This short series adds support for free-threaded Python and adds a new test suite to provide some confidence in the suitability of the bindings being used within a multi-threaded context.

Quoted 8 lines

A changelog is also added to keep track of changes between releases.

Note:
The new threaded test suite was run on multiple machines in excess of 1M+ iterations in an attempt to make sure they were bug free but due to the nature of the problem, they may not be absolutely error free.

[...]

Thanks Vincent, I queued the series. I noticed what I suppose is a rebasing issue where the copyright header in the tests_threading.py file was added in patch 2/3 instead of along the implementation itself so I moved it around. The changelog also misses the SPDX header so I allowed myself to add your copyright in a comment block similar to the top-level README.md.

[1/3] bindings: python: tests: add multi-threaded tests

      https://git.kernel.org/brgl/c/9d8940f38f6e858073e7f144664ec99061c71831
[2/3] bindings: python: support free-threaded CPython
      https://git.kernel.org/brgl/c/c9378d6c636abe8e5eb28dc71d49f8dc45978396
[3/3] bindings: python: add a changelog
      https://git.kernel.org/brgl/c/c0650edd4417bcb74e294ee2a1dafdc6ea594e9a

Best regards,

Signature · 1 line
Bartosz Golaszewski <[email protected]>