Skip to contentSkip to conversation

dma-buf: heaps: Add deferred-free-helper library code

PATCH New John Stultz 1 participant 3 messages Feb 2, 2021

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v4 1/3 dma-buf: heaps: Add deferred-free-helper library code

This patch provides infrastructure for deferring buffer frees.

This is a feature ION provided which when used with some form of a page pool, provides a nice performance boost in an allocation microbenchmark. The reason it helps is it allows the page-zeroing to be done out of the normal allocation/free path, and pushed off to a kthread.

As not all heaps will find this useful, its implemented as a optional helper library that heaps can utilize.

Cc Daniel Vetter <[email protected]>
Cc Sumit Semwal <[email protected]>
Cc Liam Mark <[email protected]>
Cc Chris Goldsworthy <[email protected]>
Cc Laura Abbott <[email protected]>
Cc Brian Starkey <[email protected]>
Cc Hridya Valsaraju <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
Cc Sandeep Patil <[email protected]>
Cc Daniel Mentz <[email protected]>
Cc Ørjan Eide <[email protected]>
Cc Robin Murphy <[email protected]>
Cc Ezequiel Garcia <[email protected]>
Cc Simon Ser <[email protected]>
Cc James Jones <[email protected]>

---
v2:
* Fix sleep in atomic issue from using a mutex, by switching to a spinlock as Reported-by: kernel test robot <[email protected]>
* Cleanup API to use a reason enum for clarity and add some documentation comments as suggested by Suren Baghdasaryan.

v3:
* Minor tweaks so it can be built as a module
* A few small fixups suggested by Daniel Mentz

v4:
* Tweak from Daniel Mentz to make sure the shrinker count/freed values are tracked in pages not bytes

---
 drivers/dma-buf/heaps/Kconfig                |   3 +
 drivers/dma-buf/heaps/Makefile               |   1 +
 drivers/dma-buf/heaps/deferred-free-helper.c | 138 +++++++++++++++++++
 drivers/dma-buf/heaps/deferred-free-helper.h |  55 ++++++++
 4 files changed, 197 insertions(+)
 create mode 100644 drivers/dma-buf/heaps/deferred-free-helper.c
 create mode 100644 drivers/dma-buf/heaps/deferred-free-helper.h
drivers/dma-buf/heaps/Kconfig +3 -0
index a5eef06c4226..f7aef8bc7119 100644
@@ -1,3 +1,6 @@
+config DMABUF_HEAPS_DEFERRED_FREE
+	tristate
+
 config DMABUF_HEAPS_SYSTEM
 	bool "DMA-BUF System Heap"
 	depends on DMABUF_HEAPS
drivers/dma-buf/heaps/Makefile +1 -0
index 974467791032..4e7839875615 100644
@@ -1,3 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_DMABUF_HEAPS_DEFERRED_FREE) += deferred-free-helper.o
 obj-$(CONFIG_DMABUF_HEAPS_SYSTEM)	+= system_heap.o
 obj-$(CONFIG_DMABUF_HEAPS_CMA)		+= cma_heap.o
drivers/dma-buf/heaps/deferred-free-helper.c new file +138 -0

138 lines not on this page · raw message

drivers/dma-buf/heaps/deferred-free-helper.h new file +55 -0

55 lines not on this page · raw message

Signature · 6 lines
2.25.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#1 lore raw

PATCH v4 2/3 dma-buf: system_heap: Add pagepool support to system heap

Reuse/abuse the pagepool code from the network code to speed up allocation performance.

This is similar to the ION pagepool usage, but tries to utilize generic code instead of a custom implementation.

Cc Daniel Vetter <[email protected]>
Cc Sumit Semwal <[email protected]>
Cc Liam Mark <[email protected]>
Cc Chris Goldsworthy <[email protected]>
Cc Laura Abbott <[email protected]>
Cc Brian Starkey <[email protected]>
Cc Hridya Valsaraju <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
Cc Sandeep Patil <[email protected]>
Cc Daniel Mentz <[email protected]>
Cc Ørjan Eide <[email protected]>
Cc Robin Murphy <[email protected]>
Cc Ezequiel Garcia <[email protected]>
Cc Simon Ser <[email protected]>
Cc James Jones <[email protected]>

---
v2:
* Fix build issue caused by selecting PAGE_POOL w/o NET as Reported-by: kernel test robot <[email protected]>
v3:
* Simplify the page zeroing logic a bit by using kmap_atomic instead of vmap as suggested by Daniel Mentz

---
 drivers/dma-buf/heaps/Kconfig       |  2 ++
 drivers/dma-buf/heaps/system_heap.c | 52 ++++++++++++++++++++++++++---
 2 files changed, 50 insertions(+), 4 deletions(-)
drivers/dma-buf/heaps/Kconfig +2 -0
index f7aef8bc7119..45c7d277448b 100644
@@ -4,6 +4,8 @@ config DMABUF_HEAPS_DEFERRED_FREE
 config DMABUF_HEAPS_SYSTEM
 	bool "DMA-BUF System Heap"
 	depends on DMABUF_HEAPS
+	select NET
+	select PAGE_POOL
 	help
 	  Choose this option to enable the system dmabuf heap. The system heap
 	  is backed by pages from the buddy allocator. If in doubt, say Y.
drivers/dma-buf/heaps/system_heap.c +48 -4

92 lines not on this page · raw message

Signature · 6 lines
2.25.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#2 lore raw

PATCH v4 3/3 dma-buf: system_heap: Add deferred freeing to the system heap

In reply to John Stultz

Utilize the deferred free helper library in the system heap.

This provides a nice performance bump and puts the system heap performance on par with ION.

Cc Daniel Vetter <[email protected]>
Cc Sumit Semwal <[email protected]>
Cc Liam Mark <[email protected]>
Cc Chris Goldsworthy <[email protected]>
Cc Laura Abbott <[email protected]>
Cc Brian Starkey <[email protected]>
Cc Hridya Valsaraju <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
Cc Sandeep Patil <[email protected]>
Cc Daniel Mentz <[email protected]>
Cc Ørjan Eide <[email protected]>
Cc Robin Murphy <[email protected]>
Cc Ezequiel Garcia <[email protected]>
Cc Simon Ser <[email protected]>
Cc James Jones <[email protected]>

---
v2:
* Rework deferred-free api to use reason enum as suggested by Suren Baghdasaryan

---
 drivers/dma-buf/heaps/Kconfig       |  1 +
 drivers/dma-buf/heaps/system_heap.c | 32 ++++++++++++++++++++++-------
 2 files changed, 26 insertions(+), 7 deletions(-)
drivers/dma-buf/heaps/Kconfig +1 -0
index 45c7d277448b..2276420ae905 100644
@@ -6,6 +6,7 @@ config DMABUF_HEAPS_SYSTEM
 	depends on DMABUF_HEAPS
 	select NET
 	select PAGE_POOL
+	select DMABUF_HEAPS_DEFERRED_FREE
 	help
 	  Choose this option to enable the system dmabuf heap. The system heap
 	  is backed by pages from the buddy allocator. If in doubt, say Y.
drivers/dma-buf/heaps/system_heap.c +25 -7
index 2addb6d832e0..fe122b5eff10 100644
@@ -22,6 +22,8 @@
 #include <linux/vmalloc.h>
 #include <net/page_pool.h>
 
+#include "deferred-free-helper.h"
+
 static struct dma_heap *sys_heap;
 
 struct system_heap_buffer {
@@ -32,6 +34,7 @@ struct system_heap_buffer {
 	struct sg_table sg_table;
 	int vmap_cnt;
 	void *vaddr;
+	struct deferred_freelist_item deferred_free;
 };
 
 struct dma_heap_attachment {
@@ -301,30 +304,45 @@ static int system_heap_zero_buffer(struct system_heap_buffer *buffer)
 	return ret;
 }
 
-static void system_heap_dma_buf_release(struct dma_buf *dmabuf)
+static void system_heap_buf_free(struct deferred_freelist_item *item,
+				 enum df_reason reason)
 {
-	struct system_heap_buffer *buffer = dmabuf->priv;
+	struct system_heap_buffer *buffer;
 	struct sg_table *table;
 	struct scatterlist *sg;
 	int i, j;
 
+	buffer = container_of(item, struct system_heap_buffer, deferred_free);
 	/* Zero the buffer pages before adding back to the pool */
-	system_heap_zero_buffer(buffer);
+	if (reason == DF_NORMAL)
+		if (system_heap_zero_buffer(buffer))
+			reason = DF_UNDER_PRESSURE; // On failure, just free
 
 	table = &buffer->sg_table;
 	for_each_sg(table->sgl, sg, table->nents, i) {
 		struct page *page = sg_page(sg);
 
-		for (j = 0; j < NUM_ORDERS; j++) {
-			if (compound_order(page) == orders[j])
-				break;
+		if (reason == DF_UNDER_PRESSURE) {
+			__free_pages(page, compound_order(page));
+		} else {
+			for (j = 0; j < NUM_ORDERS; j++) {
+				if (compound_order(page) == orders[j])
+					break;
+			}
+			page_pool_put_full_page(pools[j], page, false);
 		}
-		page_pool_put_full_page(pools[j], page, false);
 	}
 	sg_free_table(table);
 	kfree(buffer);
 }
 
+static void system_heap_dma_buf_release(struct dma_buf *dmabuf)
+{
+	struct system_heap_buffer *buffer = dmabuf->priv;
+
+	deferred_free(&buffer->deferred_free, system_heap_buf_free, buffer->len);
+}
+
 static const struct dma_buf_ops system_heap_buf_ops = {
 	.attach = system_heap_attach,
 	.detach = system_heap_detach,
Signature · 6 lines
2.25.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel