Skip to contentSkip to conversation

treewide: rework struct gpiod_line_bulk

PATCH Superseded Bartosz Golaszewski 2 participants 6 messages Oct 27, 2020 – Oct 28, 2020

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 treewide: rework struct gpiod_line_bulk

From: Bartosz Golaszewski <[email protected]>

Current implementation of struct gpiod_line_bulk uses stack memory excessively. The structure is big: it's an array 64 pointers + 4 bytes size. That amounts to 260 bytes on 32-bit and 516 bytes on 64-bit architectures respectively. It's also used everywhere as all functions dealing with single lines eventually end up calling bulk counterparts.

The rework addresses it by making the bulk structure opaque and providing appropriate interfaces for library users while retaining a way for internal users to allocate single line bulks on the stack.

The macro-based loop has been removed. In its place we provide a function iterating over all lines held by a bulk and calling the provided callback function for each line as well as a new line bulk iterator which works similarily to chip and line iterators.

Since bulk operations can now fail, a bunch of test-cases has been added to cover the relevant code.

While at it: using the word offset both when referring to line's HW offset in a chip as well as the offset in a bulk leads to confusion. This patch renames the bulk offset to index.

---
v1 -> v2:
- make offset -> index in all bulk-related interfaces and docs
- drop the owner chip reference from struct gpiod_line_bulk and reuse the owner of the first line
- drop bulk_same_chip helper

 bindings/cxx/gpiod.hpp        |  20 ++-
 bindings/cxx/line_bulk.cpp    | 112 +++++++++--------
 bindings/python/gpiodmodule.c | 182 +++++++++++++++++++--------
 include/gpiod.h               | 172 ++++++++++++++------------
 lib/core.c                    | 211 +++++++++++++++++++------------
 lib/ctxless.c                 |  53 +++++---
 lib/helpers.c                 |  61 +++++----
 lib/iter.c                    |  33 +++++
 tests/Makefile.am             |   1 +
 tests/gpiod-test.h            |   5 +
 tests/tests-bulk.c            | 158 ++++++++++++++++++++++++
 tests/tests-chip.c            |  59 +++++----
 tests/tests-event.c           |  73 +++++++----
 tests/tests-iter.c            |  26 ++++
 tests/tests-line.c            | 226 +++++++++++++---------------------
 15 files changed, 894 insertions(+), 498 deletions(-)
 create mode 100644 tests/tests-bulk.c
bindings/cxx/gpiod.hpp +15 -5
index 8dfc172..897d8c0 100644
@@ -626,18 +626,20 @@ public:
 
 	/**
 	 * @brief Get the line at given offset.
-	 * @param offset Offset of the line to get.
+	 * @param index Index of the line to get.
 	 * @return Reference to the line object.
+	 * @note This method will throw if index is equal or greater than the
+	 *       number of lines currently held by this bulk.
 	 */
-	GPIOD_API line& get(unsigned int offset);
+	GPIOD_API line& get(unsigned int index);
 
 	/**
 	 * @brief Get the line at given offset without bounds checking.
-	 * @param offset Offset of the line to get.
+	 * @param index Offset of the line to get.
 	 * @return Reference to the line object.
 	 * @note No bounds checking is performed.
 	 */
-	GPIOD_API line& operator[](unsigned int offset);
+	GPIOD_API line& operator[](unsigned int index);
 
 	/**
 	 * @brief Get the number of lines currently held by this object.
@@ -839,8 +841,16 @@ public:
 
 private:
 
+	struct line_bulk_deleter
+	{
+		void operator()(::gpiod_line_bulk *bulk);
+	};
+
 	void throw_if_empty(void) const;
-	void to_line_bulk(::gpiod_line_bulk* bulk) const;
+
+	using line_bulk_ptr = ::std::unique_ptr<::gpiod_line_bulk, line_bulk_deleter>;
+
+	line_bulk_ptr to_line_bulk(void) const;
 
 	::std::vector<line> _m_bulk;
 };
bindings/cxx/line_bulk.cpp +63 -49

235 lines not on this page · raw message

bindings/python/gpiodmodule.c +132 -50

395 lines not on this page · raw message

include/gpiod.h +91 -81

245 lines not on this page · raw message

lib/core.c +132 -79

380 lines not on this page · raw message

lib/ctxless.c +36 -17

156 lines not on this page · raw message

lib/helpers.c +40 -21

106 lines not on this page · raw message

lib/iter.c +33 -0
index bfd2852..c9a117c 100644
@@ -23,6 +23,12 @@ struct gpiod_line_iter {
 	unsigned int offset;
 };
 
+struct gpiod_line_bulk_iter {
+	struct gpiod_line_bulk *bulk;
+	unsigned int num_lines;
+	unsigned int index;
+};
+
 static int dir_filter(const struct dirent *dir)
 {
 	return !strncmp(dir->d_name, "gpiochip", 8);
@@ -169,3 +175,30 @@ struct gpiod_line *gpiod_line_iter_next(struct gpiod_line_iter *iter)
 	return iter->offset < (iter->num_lines)
 					? iter->lines[iter->offset++] : NULL;
 }
+
+struct gpiod_line_bulk_iter *
+gpiod_line_bulk_iter_new(struct gpiod_line_bulk *bulk)
+{
+	struct gpiod_line_bulk_iter *iter;
+
+	iter = malloc(sizeof(*iter));
+	if (!iter)
+		return NULL;
+
+	iter->bulk = bulk;
+	iter->index = 0;
+	iter->num_lines = gpiod_line_bulk_num_lines(bulk);
+
+	return iter;
+}
+
+void gpiod_line_bulk_iter_free(struct gpiod_line_bulk_iter *iter)
+{
+	free(iter);
+}
+
+struct gpiod_line *gpiod_line_bulk_iter_next(struct gpiod_line_bulk_iter *iter)
+{
+	return iter->index < iter->num_lines ?
+		gpiod_line_bulk_get_line(iter->bulk, iter->index++) : NULL;
+}
tests/Makefile.am +1 -0
index f79f8d5..4a3befd 100644
@@ -22,6 +22,7 @@ bin_PROGRAMS = gpiod-test
 gpiod_test_SOURCES =			\
 		gpiod-test.c		\
 		gpiod-test.h		\
+		tests-bulk.c		\
 		tests-chip.c		\
 		tests-ctxless.c		\
 		tests-event.c		\
tests/gpiod-test.h +5 -0
index 8308547..8813e68 100644
@@ -24,12 +24,17 @@
  * regular 'struct typename' syntax.
  */
 typedef struct gpiod_chip gpiod_chip_struct;
+typedef struct gpiod_line_bulk gpiod_line_bulk_struct;
 typedef struct gpiod_chip_iter gpiod_chip_iter_struct;
 typedef struct gpiod_line_iter gpiod_line_iter_struct;
+typedef struct gpiod_line_bulk_iter gpiod_line_bulk_iter_struct;
 
 G_DEFINE_AUTOPTR_CLEANUP_FUNC(gpiod_chip_struct, gpiod_chip_close);
+G_DEFINE_AUTOPTR_CLEANUP_FUNC(gpiod_line_bulk_struct, gpiod_line_bulk_free);
 G_DEFINE_AUTOPTR_CLEANUP_FUNC(gpiod_chip_iter_struct, gpiod_chip_iter_free);
 G_DEFINE_AUTOPTR_CLEANUP_FUNC(gpiod_line_iter_struct, gpiod_line_iter_free);
+G_DEFINE_AUTOPTR_CLEANUP_FUNC(gpiod_line_bulk_iter_struct,
+			      gpiod_line_bulk_iter_free);
 
 /* These are private definitions and should not be used directly. */
 typedef void (*_gpiod_test_func)(void);
tests/tests-bulk.c new file +158 -0

158 lines not on this page · raw message

tests/tests-chip.c +29 -30

114 lines not on this page · raw message

tests/tests-event.c +47 -26

202 lines not on this page · raw message

tests/tests-iter.c +26 -0
index 8deee8e..18263af 100644
@@ -121,3 +121,29 @@ GPIOD_TEST_CASE(line_iter, 0, { 8 })
 
 	g_assert_cmpuint(i, ==, 8);
 }
+
+GPIOD_TEST_CASE(line_bulk_iter, 0, { 4 })
+{
+	g_autoptr(gpiod_line_bulk_iter_struct) iter = NULL;
+	g_autoptr(gpiod_line_bulk_struct) bulk = NULL;
+	g_autoptr(gpiod_chip_struct) chip = NULL;
+	struct gpiod_line *line;
+	gint i = 0;
+
+	chip = gpiod_chip_open(gpiod_test_chip_path(0));
+	g_assert_nonnull(chip);
+	gpiod_test_return_if_failed();
+
+	bulk = gpiod_chip_get_all_lines(chip);
+	g_assert_nonnull(bulk);
+	gpiod_test_return_if_failed();
+
+	iter = gpiod_line_bulk_iter_new(bulk);
+	g_assert_nonnull(iter);
+	gpiod_test_return_if_failed();
+
+	gpiod_line_bulk_iter_foreach_line(iter, line)
+		g_assert_cmpint(i++, ==, gpiod_line_offset(line));
+
+	g_assert_cmpint(i, ==, 4);
+}
tests/tests-line.c +86 -140

416 lines not on this page · raw message

Signature · 1 line
2.29.1
#1 lore raw
Quoted 2 lines · Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Subject should be prefixed with [libgpiod] according to the README ;).

[snip]

bindings/cxx/line_bulk.cpp const ::std::map<::std::bitset<32>, int, bitset_cmp> reqflag_mapping = { · line 48

Quoted 8 lines
bindings/cxx/line_bulk.cpp +0 -0
index e77baa2..e7bd20e 100644
@@ -46,6 +46,29 @@ const ::std::map<::std::bitset<32>, int, bitset_cmp> reqflag_mapping = {
 	{ line_request::FLAG_BIAS_PULL_UP,	GPIOD_LINE_REQUEST_FLAG_BIAS_PULL_UP, },
 };
 

A - see comment after line_bulk::event_wait()

Quoted 13 lines
bindings/cxx/line_bulk.cpp +13 -0
+struct line_bulk_iter_deleter
+{
+	void operator()(::gpiod_line_bulk_iter *iter)
+	{
+		::gpiod_line_bulk_iter_free(iter);
+	}
+};
+
+using line_bulk_iter_ptr = ::std::unique_ptr<::gpiod_line_bulk_iter,
+					     line_bulk_iter_deleter>;
+
+line_bulk_iter_ptr make_line_bulk_iter(::gpiod_line_bulk *bulk)
+{

bindings/cxx/line_bulk.cpp const ::std::map<::std::bitset<32>, int, bitset_cmp> reqflag_mapping = { · line 71

Quoted 10 lines
bindings/cxx/line_bulk.cpp +10 -0
+	::gpiod_line_bulk_iter *iter;
+
+	iter = ::gpiod_line_bulk_iter_new(bulk);
+	if (!iter)
+		throw ::std::system_error(errno, ::std::system_category(),
+					  "Unable to create new line bulk iterator");
+
+	return line_bulk_iter_ptr(iter);
+}
+

B - see comment after line_bulk::event_wait()

Quoted 2 lines
bindings/cxx/line_bulk.cpp +0 -0
 } /* namespace */
 

[snip]

bindings/cxx/line_bulk.cpp line_bulk line_bulk::event_wait(const ::std::chrono::nanoseconds& timeout) const · line 274

Quoted 7 lines
bindings/cxx/line_bulk.cpp +2 -1
@@ -263,27 +270,26 @@ line_bulk line_bulk::event_wait(const ::std::chrono::nanoseconds& timeout) const
 {
 	this->throw_if_empty();
 
-	::gpiod_line_bulk bulk, event_bulk;
+	line_bulk_ptr ev_bulk(::gpiod_line_bulk_new(this->size()));
+	auto chip = this->_m_bulk[0].get_chip();

Move chip into the block where it is used.

Quoted 19 lines
bindings/cxx/line_bulk.cpp +2 -7
+	auto bulk = this->to_line_bulk();
 	::timespec ts;
 	line_bulk ret;
 	int rv;
 
-	this->to_line_bulk(::std::addressof(bulk));
-
-	::gpiod_line_bulk_init(::std::addressof(event_bulk));
-
 	ts.tv_sec = timeout.count() / 1000000000ULL;
 	ts.tv_nsec = timeout.count() % 1000000000ULL;
 
-	rv = ::gpiod_line_event_wait_bulk(::std::addressof(bulk),
-					  ::std::addressof(ts),
-					  ::std::addressof(event_bulk));
+	rv = ::gpiod_line_event_wait_bulk(bulk.get(), ::std::addressof(ts), ev_bulk.get());
 	if (rv < 0) {
 		throw ::std::system_error(errno, ::std::system_category(),
 					  "error polling for events");

bindings/cxx/line_bulk.cpp line_bulk line_bulk::event_wait(const ::std::chrono::nanoseconds& timeout) const · line 294

Quoted 10 lines
bindings/cxx/line_bulk.cpp +5 -2
 	} else if (rv > 0) {
-		for (unsigned int i = 0; i < event_bulk.num_lines; i++)
-			ret.append(line(event_bulk.lines[i], this->_m_bulk[i].get_chip()));
+		auto iter = make_line_bulk_iter(ev_bulk.get());
+		::gpiod_line *curr_line;
+
+		gpiod_line_bulk_iter_foreach_line(iter.get(), curr_line)
+			ret.append(line(curr_line, chip));
 	}
 

If you replace the gpiod_line_bulk_iter_foreach_line() here with manually looping over the bulk lines then everything from A to B above can be dropped.

i.e.

-               auto iter = make_line_bulk_iter(ev_bulk.get());
-               ::gpiod_line *curr_line;
+               auto chip = this->_m_bulk[0].get_chip();
+               auto num_lines = ::gpiod_line_bulk_num_lines(ev_bulk.get());

- gpiod_line_bulk_iter_foreach_line(iter.get(), curr_line)
- ret.append(line(curr_line, chip));
+ for (unsigned int i = 0; i < num_lines; i++)
+ ret.append(line(::gpiod_line_bulk_get_line(ev_bulk.get(), i), chip));

That includes the chip relocation, btw.

[snip]

Quoted 70 lines
bindings/cxx/line_bulk.cpp +32 -9
 }
@@ -1715,9 +1751,10 @@ static PyObject *gpiod_LineBulk_event_wait(gpiod_LineBulkObject *self,
 {
 	static char *kwlist[] = { "sec", "nsec", NULL };
 
-	struct gpiod_line_bulk bulk, ev_bulk;
-	struct gpiod_line *line, **line_ptr;
+	struct gpiod_line_bulk *bulk, *ev_bulk;
+	struct gpiod_line_bulk_iter *iter;
 	gpiod_LineObject *line_obj;
+	struct gpiod_line *line;
 	gpiod_ChipObject *owner;
 	long sec = 0, nsec = 0;
 	struct timespec ts;
@@ -1736,37 +1773,64 @@ static PyObject *gpiod_LineBulk_event_wait(gpiod_LineBulkObject *self,
 	ts.tv_sec = sec;
 	ts.tv_nsec = nsec;
 
-	gpiod_LineBulkObjToCLineBulk(self, &bulk);
+	bulk = gpiod_LineBulkObjToCLineBulk(self);
+	if (!bulk)
+		return NULL;
+
+	ev_bulk = gpiod_line_bulk_new(self->num_lines);
+	if (!ev_bulk) {
+		gpiod_line_bulk_free(bulk);
+		return NULL;
+	}
 
 	Py_BEGIN_ALLOW_THREADS;
-	rv = gpiod_line_event_wait_bulk(&bulk, &ts, &ev_bulk);
+	rv = gpiod_line_event_wait_bulk(bulk, &ts, ev_bulk);
+	gpiod_line_bulk_free(bulk);
 	Py_END_ALLOW_THREADS;
-	if (rv < 0)
+	if (rv < 0) {
+		gpiod_line_bulk_free(ev_bulk);
 		return PyErr_SetFromErrno(PyExc_OSError);
-	else if (rv == 0)
+	} else if (rv == 0) {
+		gpiod_line_bulk_free(ev_bulk);
 		Py_RETURN_NONE;
+	}
 
-	ret = PyList_New(gpiod_line_bulk_num_lines(&ev_bulk));
-	if (!ret)
+	ret = PyList_New(gpiod_line_bulk_num_lines(ev_bulk));
+	if (!ret) {
+		gpiod_line_bulk_free(ev_bulk);
 		return NULL;
+	}
 
 	owner = ((gpiod_LineObject *)(self->lines[0]))->owner;
 
+	iter = gpiod_line_bulk_iter_new(ev_bulk);
+	if (!iter) {
+		gpiod_line_bulk_free(ev_bulk);
+		return PyErr_SetFromErrno(PyExc_OSError);
+	}
+
 	i = 0;
-	gpiod_line_bulk_foreach_line(&ev_bulk, line, line_ptr) {
+	gpiod_line_bulk_iter_foreach_line(iter, line) {
 		line_obj = gpiod_MakeLineObject(owner, line);
 		if (!line_obj) {
+			gpiod_line_bulk_iter_free(iter);
+			gpiod_line_bulk_free(ev_bulk);
 			Py_DECREF(ret);
 			return NULL;
 		}

bindings/cxx/line_bulk.cpp static PyObject *gpiod_LineBulk_event_wait(gpiod_LineBulkObject *self, · line 1830

Quoted 10 lines
bindings/cxx/line_bulk.cpp +2 -0
 
 		rv = PyList_SetItem(ret, i++, (PyObject *)line_obj);
 		if (rv < 0) {
+			gpiod_line_bulk_iter_free(iter);
+			gpiod_line_bulk_free(ev_bulk);
 			Py_DECREF(ret);
 			return NULL;
 		}
 	}
 

This function can be simplified by looping over the bulk manually rather than using the line_bulk_iter.

Quoted 47 lines
bindings/cxx/line_bulk.cpp +19 -7
+	gpiod_line_bulk_iter_free(iter);
+	gpiod_line_bulk_free(ev_bulk);
+
 	return ret;
 }
 
@@ -2241,41 +2305,59 @@ PyDoc_STRVAR(gpiod_Chip_get_all_lines_doc,
 static gpiod_LineBulkObject *
 gpiod_Chip_get_all_lines(gpiod_ChipObject *self, PyObject *Py_UNUSED(ignored))
 {
+	struct gpiod_line_bulk_iter *iter;
 	gpiod_LineBulkObject *bulk_obj;
-	struct gpiod_line_bulk bulk;
+	struct gpiod_line_bulk *bulk;
 	gpiod_LineObject *line_obj;
 	struct gpiod_line *line;
-	unsigned int offset;
+	unsigned int index = 0;
 	PyObject *list;
 	int rv;
 
 	if (gpiod_ChipIsClosed(self))
 		return NULL;
 
-	rv = gpiod_chip_get_all_lines(self->chip, &bulk);
-	if (rv)
+	bulk = gpiod_chip_get_all_lines(self->chip);
+	if (!bulk)
 		return (gpiod_LineBulkObject *)PyErr_SetFromErrno(
 							PyExc_OSError);
 
-	list = PyList_New(gpiod_line_bulk_num_lines(&bulk));
-	if (!list)
+	list = PyList_New(gpiod_line_bulk_num_lines(bulk));
+	if (!list) {
+		gpiod_line_bulk_free(bulk);
 		return NULL;
+	}
 
-	gpiod_line_bulk_foreach_line_off(&bulk, line, offset) {
+	iter = gpiod_line_bulk_iter_new(bulk);
+	if (!iter) {
+		gpiod_line_bulk_free(bulk);
+		Py_DECREF(list);
+		return (gpiod_LineBulkObject *)PyErr_SetFromErrno(
+							PyExc_OSError);
+	}

bindings/cxx/line_bulk.cpp PyDoc_STRVAR(gpiod_Chip_get_all_lines_doc, · line 2347

Quoted 10 lines
bindings/cxx/line_bulk.cpp +4 -0
+
+	gpiod_line_bulk_iter_foreach_line(iter, line) {
 		line_obj = gpiod_MakeLineObject(self, line);
 		if (!line_obj) {
+			gpiod_line_bulk_iter_free(iter);
+			gpiod_line_bulk_free(bulk);
 			Py_DECREF(list);
 			return NULL;
 		}
 

Again, NOT using the line_bulk_iter here, and manually looping instead, is actually simpler and cleaner.

Cheers,
Kent.

#2 lore raw
Quoted 7 lines · Kent Gibson
Quoted 2 lines · Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Subject should be prefixed with [libgpiod] according to the README ;).

Oh come on! :)

[snip]

Quoted 5 lines

If you replace the gpiod_line_bulk_iter_foreach_line() here with manually looping over the bulk lines then everything from A to B above can be dropped.

Something is telling me that dedicated iterators are more elegant. But I guess we can do everything manually for now and see about that later. Fair enough, I'll drop it.

Bartosz

[snip]

#3 lore raw
Quoted 14 lines · Bartosz Golaszewski
Quoted 7 lines · Kent Gibson
Quoted 2 lines · Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Subject should be prefixed with [libgpiod] according to the README ;).

Oh come on! :)

[snip]

Quoted 1 line
Quoted 9 lines
Quoted 4 lines

If you replace the gpiod_line_bulk_iter_foreach_line() here with manually looping over the bulk lines then everything from A to B above can be dropped.

Something is telling me that dedicated iterators are more elegant. But I guess we can do everything manually for now and see about that later. Fair enough, I'll drop it.

Good thing I'm here to let you know that little voice is wrong :).

I've got a couple of commits in my working tree[1] that apply to your patch and drop the iters as I've suggested if you'd like to compare.

Cheers,
Kent.

[1] https://github.com/warthog618/libgpiod/commits/for-v2

#4 lore raw
Quoted 28 lines · Kent Gibson
Quoted 14 lines · Bartosz Golaszewski
Quoted 7 lines · Kent Gibson
Quoted 2 lines · Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Subject should be prefixed with [libgpiod] according to the README ;).

Oh come on! :)

[snip]

Quoted 1 line
Quoted 9 lines
Quoted 4 lines

If you replace the gpiod_line_bulk_iter_foreach_line() here with manually looping over the bulk lines then everything from A to B above can be dropped.

Something is telling me that dedicated iterators are more elegant. But I guess we can do everything manually for now and see about that later. Fair enough, I'll drop it.

Good thing I'm here to let you know that little voice is wrong :).

Quoted 7 lines

I've got a couple of commits in my working tree[1] that apply to your patch and drop the iters as I've suggested if you'd like to compare.

Cheers,
Kent.

[1] https://github.com/warthog618/libgpiod/commits/for-v2

Would you mind if I squashed those changes into my patch?

Bartosz

#5 lore raw
Quoted 32 lines · Bartosz Golaszewski
Quoted 31 lines · Kent Gibson
Quoted 14 lines · Bartosz Golaszewski
Quoted 7 lines · Kent Gibson
Quoted 2 lines · Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Subject should be prefixed with [libgpiod] according to the README ;).

Oh come on! :)

[snip]

Quoted 1 line
Quoted 9 lines
Quoted 4 lines

If you replace the gpiod_line_bulk_iter_foreach_line() here with manually looping over the bulk lines then everything from A to B above can be dropped.

Something is telling me that dedicated iterators are more elegant. But I guess we can do everything manually for now and see about that later. Fair enough, I'll drop it.

Good thing I'm here to let you know that little voice is wrong :).

I've got a couple of commits in my working tree[1] that apply to your patch and drop the iters as I've suggested if you'd like to compare.

Quoted 7 lines
Quoted 4 lines

Cheers,
Kent.

[1] https://github.com/warthog618/libgpiod/commits/for-v2

Would you mind if I squashed those changes into my patch?

Not at all.

Cheers,
Kent.