Skip to contentSkip to conversation

bindings: python: modernize C extensions

PATCH Superseded Vincent Fazio 2 participants 13 messages Apr 22 – Apr 23 GPIO

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover bindings: python: modernize C extensions

This series performs some minor cleanup of the C extension modules and migrates the module defintions to multi-phase intitialization (PEP 489).

Patch 1 makes chip_get_line_name consistent with other functions.

Patches 2 & 3 avoid calling back into python to perform C level object cleanup.

Patch 4 introduces no real functional change but simplifies the code to get the same result.

Patch 5 applies suggestions from from a utility maintained by a CPython core developer to help modernize macro/function usage.

Patch 6 conditionally compiles support for using a standard CPython function over a backported version bundled within the C extension.

Patches 7-9 migrate the C extensions to use multi-phase initialization as described in PEP 489 [0]. While not strictly necessary for enabling free-threaded builds, it makes adding support more straighforward and sets the stage for both PEP 793 [1] which soft-deprecates PyInit_* and PEP 803 [2] which could simplify wheel builds.

Patches have been tested against the full matrix of supported versions:
https://github.com/vfazio/libgpiod/actions/runs/24752901690

[0]: https://peps.python.org/pep-0489/
[1]: https://peps.python.org/pep-0793/
[2]: https://peps.python.org/pep-0803/

---
Vincent Fazio (9):
bindings: python: use Py_RETURN_NONE in chip_get_line_name
bindings: python: avoid PyObject_CallMethod during chip finalize
bindings: python: avoid PyObject_CallMethod during request finalize
bindings: python: simplify disallowing _ext.Request from being created
bindings: python: use suggestions from upgrade_pythoncapi.py
bindings: python: use PyImport_ImportModuleAttrString when available
bindings: python: migrate the gpiod._ext module to multi-phase init
bindings: python: tests: migrate the system module to multi-phase init
bindings: python: tests: migrate the gpiosim module to multi-phase
init

 bindings/python/gpiod/ext/chip.c     |  35 +++---
 bindings/python/gpiod/ext/common.c   |   5 +-
 bindings/python/gpiod/ext/internal.h |  10 +-
 bindings/python/gpiod/ext/module.c   |  68 ++++++------
 bindings/python/gpiod/ext/request.c  |  26 ++---
 bindings/python/tests/gpiosim/ext.c  | 153 +++++++++++++++++----------
 bindings/python/tests/system/ext.c   |   7 +-
 7 files changed, 176 insertions(+), 128 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/9 bindings: python: use Py_RETURN_NONE in chip_get_line_name

When commit 0c0993569c54 introduced chip_get_line_name, it failed to utilize the standard Py_RETURN_NONE macro used throughout the rest of the extension.

While commit fbc0a4471215 fixed the underlying reference counting issue, utilizing the macro is the standard way of managing reference counts, works across all CPython versions, and brings the function in line with others in the extension.

---
 bindings/python/gpiod/ext/chip.c | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)
bindings/python/gpiod/ext/chip.c +6 -7
index 53c4b80..19b3a4f 100644
@@ -152,7 +152,7 @@ static PyObject *chip_get_line_name(chip_object *self, PyObject *args)
 	int ret;
 	unsigned int offset;
 	struct gpiod_line_info *info;
-	PyObject *line_name;
+	PyObject *line_name = NULL;
 	const char *name;
 
 	ret = PyArg_ParseTuple(args, "I", &offset);
@@ -166,16 +166,15 @@ static PyObject *chip_get_line_name(chip_object *self, PyObject *args)
 		return Py_gpiod_SetErrFromErrno();
 
 	name = gpiod_line_info_get_name(info);
-	if (!name) {
-		Py_INCREF(Py_None);
-		line_name = Py_None;
-	} else {
+	if (name)
 		line_name = PyUnicode_FromString(name);
-	}
 
 	gpiod_line_info_free(info);
 
-	return line_name;
+	if (line_name)
+		return line_name;
+
+	Py_RETURN_NONE;
 }
 
 static PyObject *
Signature · 1 line
2.43.0
#2 lore raw

Re: [libgpiod][PATCH 1/9] bindings: python: use Py_RETURN_NONE in chip_get_line_name

Quoted 29 lines · Vincent Fazio

When commit 0c0993569c54 introduced chip_get_line_name, it failed to utilize the standard Py_RETURN_NONE macro used throughout the rest of the extension.

While commit fbc0a4471215 fixed the underlying reference counting issue, utilizing the macro is the standard way of managing reference counts, works across all CPython versions, and brings the function in line with others in the extension.

---
 bindings/python/gpiod/ext/chip.c | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)
bindings/python/gpiod/ext/chip.c +1 -1
index 53c4b80..19b3a4f 100644
@@ -152,7 +152,7 @@ static PyObject *chip_get_line_name(chip_object *self, PyObject *args)
        int ret;
        unsigned int offset;
        struct gpiod_line_info *info;
-       PyObject *line_name;
+       PyObject *line_name = NULL;
        const char *name;
 
        ret = PyArg_ParseTuple(args, "I", &offset);

bindings/python/gpiod/ext/chip.c static PyObject *chip_get_line_name(chip_object *self, PyObject *args) · line 170

Quoted 9 lines
bindings/python/gpiod/ext/chip.c +1 -4
@@ -166,16 +166,15 @@ static PyObject *chip_get_line_name(chip_object *self, PyObject *args)
                return Py_gpiod_SetErrFromErrno();
 
        name = gpiod_line_info_get_name(info);
-       if (!name) {
-               Py_INCREF(Py_None);
-               line_name = Py_None;
-       } else {
+       if (name)
                line_name = PyUnicode_FromString(name);

This can fail and return NULL. An exception will be set...

bindings/python/gpiod/ext/chip.c static PyObject *chip_get_line_name(chip_object *self, PyObject *args) · line 175

Quoted 7 lines
bindings/python/gpiod/ext/chip.c +2 -2
-       }
 
        gpiod_line_info_free(info);
 
-       return line_name;
+       if (line_name)
+               return line_name;

... and since line_name is NULL you skip this...

bindings/python/gpiod/ext/chip.c static PyObject *chip_get_line_name(chip_object *self, PyObject *args) · line 177

Quoted 2 lines
bindings/python/gpiod/ext/chip.c +2 -0
+
+       Py_RETURN_NONE;

... returning None and ignoring the exception whose reference count will not be subsequently decreased.

I think you need to check the return value of PyUnicode_FromString() right after the call.

Bart

#3 lore raw

Re: [libgpiod][PATCH 1/9] bindings: python: use Py_RETURN_NONE in chip_get_line_name

Quoted 3 lines · Bartosz Golaszewski

I think you need to check the return value of PyUnicode_FromString() right after the call.

Or I can drop the patch completely.

There's no excuse for trying to be too clever here, which I often lose sight of, just for the sake of using the standard macro.

I'll update the series to drop this, but I will add one more entry into patch 5 to transition to Py_NewRef

-Vincent

#4 lore raw

PATCH 2/9 bindings: python: avoid PyObject_CallMethod during chip finalize

In reply to Vincent Fazio

The PyObject_CallMethod function jumps through hoops to find the method to call and to build a stack (if necessary) to call a given method.

In the case of finalizing the chip object, that overhead can be avoided by simply sharing a function with chip_close to close the underlying gpiod_chip object.

---
 bindings/python/gpiod/ext/chip.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)
bindings/python/gpiod/ext/chip.c +12 -6
index 19b3a4f..9061d23 100644
@@ -32,10 +32,19 @@ chip_init(chip_object *self, PyObject *args, PyObject *Py_UNUSED(ignored))
 	return 0;
 }
 
+static void internal_chip_close(chip_object *self)
+{
+	if (self->chip) {
+		Py_BEGIN_ALLOW_THREADS;
+		gpiod_chip_close(self->chip);
+		Py_END_ALLOW_THREADS;
+		self->chip = NULL;
+	}
+}
+
 static void chip_finalize(chip_object *self)
 {
-	if (self->chip)
-		PyObject_CallMethod((PyObject *)self, "close", "");
+	internal_chip_close(self);
 }
 
 static PyObject *chip_path(chip_object *self, void *Py_UNUSED(ignored))
@@ -62,10 +71,7 @@ static PyGetSetDef chip_getset[] = {
 
 static PyObject *chip_close(chip_object *self, PyObject *Py_UNUSED(ignored))
 {
-	Py_BEGIN_ALLOW_THREADS;
-	gpiod_chip_close(self->chip);
-	Py_END_ALLOW_THREADS;
-	self->chip = NULL;
+	internal_chip_close(self);
 
 	Py_RETURN_NONE;
 }
Signature · 1 line
2.43.0
#5 lore raw

PATCH 3/9 bindings: python: avoid PyObject_CallMethod during request finalize

In reply to Vincent Fazio

The PyObject_CallMethod function jumps through hoops to find the method to call and to build a stack (if necessary) to call a given method.

In the case of finalizing the request object, that overhead can be avoided by simply sharing a function with request_release to close the underlying gpiod_line_request object.

---
 bindings/python/gpiod/ext/request.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)
bindings/python/gpiod/ext/request.c +12 -6
index 9acf828..1e1f65a 100644
@@ -22,10 +22,19 @@ static int request_init(PyObject *Py_UNUSED(ignored0),
 	return -1;
 }
 
+static void internal_request_release(request_object *self)
+{
+	if (self->request){
+		Py_BEGIN_ALLOW_THREADS;
+		gpiod_line_request_release(self->request);
+		Py_END_ALLOW_THREADS;
+		self->request = NULL;
+	}
+}
+
 static void request_finalize(request_object *self)
 {
-	if (self->request)
-		PyObject_CallMethod((PyObject *)self, "release", "");
+	internal_request_release(self);
 
 	if (self->offsets)
 		PyMem_Free(self->offsets);
@@ -121,10 +130,7 @@ static PyGetSetDef request_getset[] = {
 static PyObject *
 request_release(request_object *self, PyObject *Py_UNUSED(ignored))
 {
-	Py_BEGIN_ALLOW_THREADS;
-	gpiod_line_request_release(self->request);
-	Py_END_ALLOW_THREADS;
-	self->request = NULL;
+	internal_request_release(self);
 
 	Py_RETURN_NONE;
 }
Signature · 1 line
2.43.0
#6 lore raw

PATCH 4/9 bindings: python: simplify disallowing _ext.Request from being created

In reply to Vincent Fazio

Previously, tp_new and tp_init functions were defined for Request with the tp_init function raising a NotImplementedError to prevent the object from being initialized (__init__, the stage after __new__).

Similar results can be achieved by removing the tp_new and tp_init functions which raise a slightly different error:

TypeError: cannot create 'gpiod._ext.Request' instances

As this is an internal only detail, there should be impact to users.

---
 bindings/python/gpiod/ext/request.c | 12 ------------
 1 file changed, 12 deletions(-)
bindings/python/gpiod/ext/request.c +0 -12
index 1e1f65a..9940382 100644
@@ -12,16 +12,6 @@ typedef struct {
 	struct gpiod_edge_event_buffer *buffer;
 } request_object;
 
-static int request_init(PyObject *Py_UNUSED(ignored0),
-			PyObject *Py_UNUSED(ignored1),
-			PyObject *Py_UNUSED(ignored2))
-{
-	PyErr_SetString(PyExc_NotImplementedError,
-			"_ext.LineRequest cannot be instantiated");
-
-	return -1;
-}
-
 static void internal_request_release(request_object *self)
 {
 	if (self->request){
@@ -390,8 +380,6 @@ PyTypeObject request_type = {
 	.tp_name = "gpiod._ext.Request",
 	.tp_basicsize = sizeof(request_object),
 	.tp_flags = Py_TPFLAGS_DEFAULT,
-	.tp_new = PyType_GenericNew,
-	.tp_init = (initproc)request_init,
 	.tp_finalize = (destructor)request_finalize,
 	.tp_dealloc = (destructor)Py_gpiod_dealloc,
 	.tp_getset = request_getset,
Signature · 1 line
2.43.0
#7 lore raw

PATCH 5/9 bindings: python: use suggestions from upgrade_pythoncapi.py

In reply to Vincent Fazio

The upgrade_pythoncapi.py script performs transforms on C extension sources to use new syntax [0]. It's generally used with a shim header for compatibility, but the header is not necessary for these changes.

The suggested changes are:

* Use the Py_IsNone function introduced in CPython 3.10 [1] over direct comparison to Py_None.

* Use PyObject_Free instead of PyObject_Del since PyObject_Del has been aliased to PyObject_Free for nearly 24 years [2] and has been announced as deprecated [3].

[0]: https://pythoncapi-compat.readthedocs.io/en/latest/index.html
[1]: https://github.com/python/cpython/pull/25227
[2]: https://github.com/python/cpython/commit/3e7b893899bdaa294a72e894694b099a1d370765
[3]: https://docs.python.org/3/c-api/allocation.html#deprecated-aliases

---
 bindings/python/gpiod/ext/chip.c    | 4 ++--
 bindings/python/gpiod/ext/common.c  | 2 +-
 bindings/python/gpiod/ext/request.c | 2 +-
 bindings/python/tests/gpiosim/ext.c | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)
bindings/python/gpiod/ext/chip.c +2 -2
index 9061d23..498a5ee 100644
@@ -271,7 +271,7 @@ make_request_config(PyObject *consumer_obj, PyObject *event_buffer_size_obj)
 		return NULL;
 	}
 
-	if (consumer_obj != Py_None) {
+	if (!Py_IsNone(consumer_obj)) {
 		consumer = PyUnicode_AsUTF8(consumer_obj);
 		if (!consumer) {
 			gpiod_request_config_free(req_cfg);
@@ -281,7 +281,7 @@ make_request_config(PyObject *consumer_obj, PyObject *event_buffer_size_obj)
 		gpiod_request_config_set_consumer(req_cfg, consumer);
 	}
 
-	if (event_buffer_size_obj != Py_None) {
+	if (!Py_IsNone(event_buffer_size_obj)) {
 		event_buffer_size = PyLong_AsSize_t(event_buffer_size_obj);
 		if (PyErr_Occurred()) {
 			gpiod_request_config_free(req_cfg);
bindings/python/gpiod/ext/common.c +1 -1
index 62201b6..7d2dda7 100644
@@ -12,7 +12,7 @@ void Py_gpiod_dealloc(PyObject *self)
 	if (ret < 0)
 		return;
 
-	PyObject_Del(self);
+	PyObject_Free(self);
 }
 
 PyObject *Py_gpiod_SetErrFromErrno(void)
bindings/python/gpiod/ext/request.c +1 -1
index 9940382..46e07ae 100644
@@ -286,7 +286,7 @@ static PyObject *request_read_edge_events(request_object *self, PyObject *args)
 	if (!ret)
 		return NULL;
 
-	if (max_events_obj != Py_None) {
+	if (!Py_IsNone(max_events_obj)) {
 		max_events = PyLong_AsSize_t(max_events_obj);
 		if (PyErr_Occurred())
 			return NULL;
bindings/python/tests/gpiosim/ext.c +1 -1
index cb5611a..1ebd5af 100644
@@ -116,7 +116,7 @@ static void chip_dealloc(PyObject *self)
 	if (ret < 0)
 		return;
 
-	PyObject_Del(self);
+	PyObject_Free(self);
 }
 
 static PyObject *chip_dev_path(chip_object *self, void *Py_UNUSED(ignored))
Signature · 1 line
2.43.0
#8 lore raw

PATCH 6/9 bindings: python: use PyImport_ImportModuleAttrString when available

In reply to Vincent Fazio

Commit 035ad74fdd introduced Py_gpiod_GetModuleAttrString as a way to get a module's attribute and was inspired by CPython's internal function _PyImport_GetModuleAttrString.

CPython 3.14 added PyImport_ImportModuleAttrString to the public API [0] which allows us to use the standard function when available.

[0]: https://github.com/python/cpython/pull/128912

---
 bindings/python/gpiod/ext/common.c   |  3 +++
 bindings/python/gpiod/ext/internal.h | 10 ++++++++--
 2 files changed, 11 insertions(+), 2 deletions(-)
bindings/python/gpiod/ext/common.c +3 -0
index 7d2dda7..fc28e96 100644
@@ -64,6 +64,8 @@ PyObject *Py_gpiod_SetErrFromErrno(void)
 	return PyErr_SetFromErrno(exc);
 }
 
+/* TODO: remove when 3.14 is the minimum version */
+#if PY_VERSION_HEX < 0x030E0000
 PyObject *Py_gpiod_GetModuleAttrString(const char *modname,
 				       const char *attrname)
 {
@@ -78,6 +80,7 @@ PyObject *Py_gpiod_GetModuleAttrString(const char *modname,
 
 	return attribute;
 }
+#endif
 
 unsigned int Py_gpiod_PyLongAsUnsignedInt(PyObject *pylong)
 {
bindings/python/gpiod/ext/internal.h +8 -2
index 15aedfb..2996884 100644
@@ -8,8 +8,6 @@
 #include <Python.h>
 
 PyObject *Py_gpiod_SetErrFromErrno(void);
-PyObject *Py_gpiod_GetModuleAttrString(const char *modname,
-				       const char *attrname);
 unsigned int Py_gpiod_PyLongAsUnsignedInt(PyObject *pylong);
 void Py_gpiod_dealloc(PyObject *self);
 PyObject *Py_gpiod_MakeRequestObject(struct gpiod_line_request *request,
@@ -17,4 +15,12 @@ PyObject *Py_gpiod_MakeRequestObject(struct gpiod_line_request *request,
 struct gpiod_line_config *Py_gpiod_LineConfigGetData(PyObject *obj);
 struct gpiod_line_settings *Py_gpiod_LineSettingsGetData(PyObject *obj);
 
+#if PY_VERSION_HEX >= 0x030E0000
+/* Alias to standard function available in 3.14 */
+#define Py_gpiod_GetModuleAttrString PyImport_ImportModuleAttrString
+#else
+PyObject *Py_gpiod_GetModuleAttrString(const char *modname,
+				       const char *attrname);
+#endif /* PY_VERSION_HEX >= 0x030E0000 */
+
 #endif /* __LIBGPIOD_PYTHON_MODULE_H__ */
Signature · 1 line
2.43.0
#9 lore raw

PATCH 7/9 bindings: python: migrate the gpiod._ext module to multi-phase init

In reply to Vincent Fazio

Single-phase initialization has been classified as legacy within CPython documentation [0] with multi-phase being its successor [1].

As such, switch to the new methodology.

[0]: https://docs.python.org/3/c-api/extension-modules.html#legacy-single-phase-initialization
[1]: https://docs.python.org/3/c-api/extension-modules.html#multi-phase-initialization

---
 bindings/python/gpiod/ext/module.c | 68 +++++++++++++++---------------
 1 file changed, 33 insertions(+), 35 deletions(-)
bindings/python/gpiod/ext/module.c +33 -35
index 25c252a..71fa3c2 100644
@@ -135,12 +135,6 @@ static PyMethodDef module_methods[] = {
 	{ }
 };
 
-static PyModuleDef module_def = {
-	PyModuleDef_HEAD_INIT,
-	.m_name = "gpiod._ext",
-	.m_methods = module_methods,
-};
-
 extern PyTypeObject chip_type;
 extern PyTypeObject line_config_type;
 extern PyTypeObject line_settings_type;
@@ -154,53 +148,57 @@ static PyTypeObject *types[] = {
 	NULL,
 };
 
-PyMODINIT_FUNC PyInit__ext(void)
+static int module_exec(PyObject* module)
 {
 	const struct module_const *modconst;
-	PyObject *module, *all;
+	PyObject *all;
 	PyTypeObject **type;
 	int ret;
 
-	module = PyModule_Create(&module_def);
-	if (!module)
-		return NULL;
-
 	ret = PyModule_AddStringConstant(module, "api_version",
 					 gpiod_api_version());
-	if (ret) {
-		Py_DECREF(module);
-		return NULL;
-	}
+
+	if (ret < 0)
+		return -1;
 
 	all = PyList_New(0);
-	if (!all) {
-		Py_DECREF(module);
-		return NULL;
-	}
+	if (!all)
+		return -1;
 
 	ret = PyModule_AddObjectRef(module, "__all__", all);
 	Py_DECREF(all);
-	if (ret) {
-		Py_DECREF(module);
-		return NULL;
-	}
+	if (ret)
+		return -1;
 
 	for (type = types; *type; type++) {
 		ret = PyModule_AddType(module, *type);
-		if (ret) {
-			Py_DECREF(module);
-			return NULL;
-		}
+		if (ret < 0)
+			return -1;
 	}
 
 	for (modconst = module_constants; modconst->name; modconst++) {
-		ret = PyModule_AddIntConstant(module,
-					      modconst->name, modconst->val);
-		if (ret) {
-			Py_DECREF(module);
-			return NULL;
-		}
+		ret = PyModule_AddIntConstant(module, modconst->name,
+					      modconst->val);
+		if (ret < 0)
+			return -1;
 	}
 
-	return module;
+	return 0;
+}
+
+static struct PyModuleDef_Slot module_slots[] = {
+	{Py_mod_exec, module_exec},
+	{0, NULL},
+};
+
+static PyModuleDef module_def = {
+	PyModuleDef_HEAD_INIT,
+	.m_name = "gpiod._ext",
+	.m_methods = module_methods,
+	.m_slots = module_slots,
+};
+
+PyMODINIT_FUNC PyInit__ext(void)
+{
+	return PyModuleDef_Init(&module_def);
 }
Signature · 1 line
2.43.0
#10 lore raw

PATCH 8/9 bindings: python: tests: migrate the system module to multi-phase init

In reply to Vincent Fazio

Single-phase initialization has been classified as legacy within CPython documentation [0] with multi-phase being its successor [1].

As such, switch to the new methodology.

[0]: https://docs.python.org/3/c-api/extension-modules.html#legacy-single-phase-initialization
[1]: https://docs.python.org/3/c-api/extension-modules.html#multi-phase-initialization

---
 bindings/python/tests/system/ext.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
bindings/python/tests/system/ext.c +6 -1
index c982fa6..800648e 100644
@@ -67,13 +67,18 @@ static PyMethodDef module_methods[] = {
 	{ }
 };
 
+static struct PyModuleDef_Slot module_slots[] = {
+	{0, NULL},
+};
+
 static PyModuleDef module_def = {
 	PyModuleDef_HEAD_INIT,
 	.m_name = "system._ext",
 	.m_methods = module_methods,
+	.m_slots = module_slots,
 };
 
 PyMODINIT_FUNC PyInit__ext(void)
 {
-	return PyModule_Create(&module_def);
+	return PyModuleDef_Init(&module_def);
 }
Signature · 1 line
2.43.0
#11 lore raw

PATCH 9/9 bindings: python: tests: migrate the gpiosim module to multi-phase init

In reply to Vincent Fazio

Single-phase initialization has been classified as legacy within CPython documentation [0] with multi-phase being its successor [1].

As such, switch to the new methodology.

Switching requires converting gpiosim.Chip to be a heap-type [2] in order to access the module's state [3].

[0]: https://docs.python.org/3/c-api/extension-modules.html#legacy-single-phase-initialization
[1]: https://docs.python.org/3/c-api/extension-modules.html#multi-phase-initialization
[2]: https://docs.python.org/3/c-api/typeobj.html#heap-types
[3]: https://docs.python.org/3/howto/isolating-extensions.html#heap-types

---
 bindings/python/tests/gpiosim/ext.c | 151 +++++++++++++++++-----------
 1 file changed, 94 insertions(+), 57 deletions(-)
bindings/python/tests/gpiosim/ext.c +94 -57
index 1ebd5af..9954dd5 100644
@@ -45,21 +45,6 @@ struct module_state {
 	struct gpiosim_ctx *sim_ctx;
 };
 
-static void free_module_state(void *mod)
-{
-	struct module_state *state = PyModule_GetState((PyObject *)mod);
-
-	if (state->sim_ctx)
-		gpiosim_ctx_unref(state->sim_ctx);
-}
-
-static PyModuleDef module_def = {
-	PyModuleDef_HEAD_INIT,
-	.m_name = "gpiosim._ext",
-	.m_size = sizeof(struct module_state),
-	.m_free = free_module_state,
-};
-
 typedef struct {
 	PyObject_HEAD
 	struct gpiosim_dev *dev;
@@ -71,13 +56,11 @@ static int chip_init(chip_object *self,
 		     PyObject *Py_UNUSED(ignored1))
 {
 	struct module_state *state;
-	PyObject *mod;
 
-	mod = PyState_FindModule(&module_def);
-	if (!mod)
-		return -1;
+	state = PyType_GetModuleState(Py_TYPE(self));
 
-	state = PyModule_GetState(mod);
+	if (!state)
+		return -1;
 
 	self->dev = gpiosim_dev_new(state->sim_ctx);
 	if (!self->dev) {
@@ -111,12 +94,14 @@ static void chip_finalize(chip_object *self)
 static void chip_dealloc(PyObject *self)
 {
 	int ret;
+	PyTypeObject *tp = Py_TYPE(self);
 
 	ret = PyObject_CallFinalizerFromDealloc(self);
 	if (ret < 0)
 		return;
 
 	PyObject_Free(self);
+	Py_DECREF(tp);
 }
 
 static PyObject *chip_dev_path(chip_object *self, void *Py_UNUSED(ignored))
@@ -289,58 +274,110 @@ static PyMethodDef chip_methods[] = {
 	{ }
 };
 
-static PyTypeObject chip_type = {
-	PyVarObject_HEAD_INIT(NULL, 0)
-	.tp_name = "gpiosim.Chip",
-	.tp_basicsize = sizeof(chip_object),
-	.tp_flags = Py_TPFLAGS_DEFAULT,
-	.tp_new = PyType_GenericNew,
-	.tp_init = (initproc)chip_init,
-	.tp_finalize = (destructor)chip_finalize,
-	.tp_dealloc = (destructor)chip_dealloc,
-	.tp_methods = chip_methods,
-	.tp_getset = chip_getset,
+static PyType_Slot chip_type_slots[] = {
+	{Py_tp_new, PyType_GenericNew},
+	{Py_tp_init, (initproc)chip_init},
+	{Py_tp_finalize, (destructor)chip_finalize},
+	{Py_tp_dealloc, (destructor)chip_dealloc},
+	{Py_tp_methods, chip_methods},
+	{Py_tp_getset, chip_getset},
+	{0, 0}
 };
 
-PyMODINIT_FUNC PyInit__ext(void)
+/*
+See xxlimited.c and _bz2module.c for inspiration.
+
+As part of transitioning to multi-phase module initialization, the
+gpiosim.Chip type needs to become heap allocated so that it can access
+module state.
+
+We disallow subclassing by not specifying Py_TPFLAGS_BASETYPE. This
+allows the module to use PyType_GetModuleState() since it may otherwise
+not return the proper module if a subclass is invoking the method.
+
+Note:
+We do not hold PyObject references so no reference cycles should exist. As such,
+we do not set Py_TPFLAGS_HAVE_GC nor define either tp_traverse or tp_clear.
+
+There is still some ongoing debate about this this use case however:
+  https://github.com/python/cpython/issues/116946
+
+Note:
+If we allow subclassing in the future, reconsider use of PyObject_Free vs using
+the function defined in the tp_free slot.
+
+See also:
+  https://github.com/python/cpython/pull/138329#issuecomment-3242079564
+  https://github.com/python/cpython/issues/116946#issuecomment-3242135537
+  https://github.com/python/cpython/pull/139073
+  https://github.com/python/cpython/commit/ec689187957cc80af56b9a63251bbc295bafd781
+*/
+static PyType_Spec chip_type_spec = {
+	.name = "gpiosim.Chip",
+	.basicsize = sizeof(chip_object),
+	.flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_IMMUTABLETYPE),
+	.slots = chip_type_slots,
+};
+
+static int module_exec(PyObject *module)
 {
 	const struct module_const *modconst;
 	struct module_state *state;
-	PyObject *module;
+	PyObject *chip_type_obj;
 	int ret;
 
-	module = PyModule_Create(&module_def);
-	if (!module)
-		return NULL;
-
-	ret = PyState_AddModule(module, &module_def);
-	if (ret) {
-		Py_DECREF(module);
-		return NULL;
-	}
-
 	state = PyModule_GetState(module);
 
 	state->sim_ctx = gpiosim_ctx_new();
 	if (!state->sim_ctx) {
-		Py_DECREF(module);
-		return PyErr_SetFromErrno(PyExc_OSError);
+		PyErr_SetFromErrno(PyExc_OSError);
+		return -1;
 	}
 
-	ret = PyModule_AddType(module, &chip_type);
-	if (ret) {
-		Py_DECREF(module);
-		return NULL;
-	}
+	chip_type_obj = PyType_FromModuleAndSpec(module, &chip_type_spec, NULL);
+
+	if (!chip_type_obj)
+		return -1;
+
+	ret = PyModule_AddType(module, (PyTypeObject*)chip_type_obj);
+	Py_DECREF(chip_type_obj);
+	if (ret < 0)
+		return -1;
 
 	for (modconst = module_constants; modconst->name; modconst++) {
-		ret = PyModule_AddIntConstant(module,
-					      modconst->name, modconst->val);
-		if (ret) {
-			Py_DECREF(module);
-			return NULL;
-		}
+		ret = PyModule_AddIntConstant(module, modconst->name,
+					      modconst->val);
+		if (ret < 0)
+			return -1;
+	}
+
+	return 0;
+}
+
+static void free_module_state(void *mod)
+{
+	struct module_state *state = PyModule_GetState((PyObject *)mod);
+
+	if (state->sim_ctx) {
+		gpiosim_ctx_unref(state->sim_ctx);
+		state->sim_ctx = NULL;
 	}
+}
+
+static struct PyModuleDef_Slot module_slots[] = {
+	{Py_mod_exec, module_exec},
+	{0, NULL},
+};
+
+static PyModuleDef module_def = {
+	PyModuleDef_HEAD_INIT,
+	.m_name = "gpiosim._ext",
+	.m_size = sizeof(struct module_state),
+	.m_free = free_module_state,
+	.m_slots = module_slots,
+};
 
-	return module;
+PyMODINIT_FUNC PyInit__ext(void)
+{
+	return PyModuleDef_Init(&module_def);
 }
Signature · 1 line
2.43.0
#12 lore raw
In reply to Vincent Fazio
Quoted 27 lines · Vincent Fazio

This series performs some minor cleanup of the C extension modules and migrates the module defintions to multi-phase intitialization (PEP 489).

Patch 1 makes chip_get_line_name consistent with other functions.

Patches 2 & 3 avoid calling back into python to perform C level object cleanup.

Patch 4 introduces no real functional change but simplifies the code to get the same result.

Patch 5 applies suggestions from from a utility maintained by a CPython core developer to help modernize macro/function usage.

Patch 6 conditionally compiles support for using a standard CPython function over a backported version bundled within the C extension.

Patches 7-9 migrate the C extensions to use multi-phase initialization as described in PEP 489 [0]. While not strictly necessary for enabling free-threaded builds, it makes adding support more straighforward and sets the stage for both PEP 793 [1] which soft-deprecates PyInit_* and PEP 803 [2] which could simplify wheel builds.

Patches have been tested against the full matrix of supported versions:
https://github.com/vfazio/libgpiod/actions/runs/24752901690

Quoted 6 lines

[0]: https://peps.python.org/pep-0489/
[1]: https://peps.python.org/pep-0793/
[2]: https://peps.python.org/pep-0803/

---

Thanks for doing this. Series looks good to me except for one nit in patch 1/9.

Bart