Skip to contentSkip to conversation

iio: buffer-dma: write() and new DMABUF based API

PATCH Superseded Paul Cercueil 8 participants 59 messages Nov 15, 2021 – Nov 28, 2021

Subsystems (5)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover iio: buffer-dma: write() and new DMABUF based API

Hi Jonathan,

This patchset introduces a new userspace interface based on DMABUF objects, to complement the existing fileio based API.

The advantage of this DMABUF based interface vs. the fileio interface, is that it avoids an extra copy of the data between the kernel and userspace. This is particularly userful for high-speed devices which produce several megabytes or even gigabytes of data per second.

The first few patches [01/15] to [03/15] are not really related, but allow to reduce the size of the patches that introduce the new API.

Patch [04/15] to [06/15] enables write() support to the buffer-dma implementation of the buffer API, to continue the work done by Mihail Chindris.

Patches [07/15] to [12/15] introduce the new DMABUF based API.

Patches [13/15] and [14/15] add support for cyclic buffers, only through the new API. A cyclic buffer will be repeated on the output until the buffer is disabled.

Patch [15/15] adds documentation about the new API.

For now, the API allows you to alloc DMABUF objects and mmap() them to read or write the samples. It does not yet allow to import DMABUFs parented to other subsystems, but that should eventually be possible once it's wired.

This patchset is inspired by the "mmap interface" that was previously submitted by Alexandru Ardelean and Lars-Peter Clausen, so it would be great if I could get a review from you guys. Alexandru's commit was signed with his @analog.com address but he doesn't work at ADI anymore, so I believe I'll need him to sign with a new email.

Cheers,
-Paul

Alexandru Ardelean (1):
iio: buffer-dma: split iio_dma_buffer_fileio_free() function

Paul Cercueil (14):
iio: buffer-dma: Get rid of incoming/outgoing queues
iio: buffer-dma: Remove unused iio_buffer_block struct
iio: buffer-dma: Use round_down() instead of rounddown()
iio: buffer-dma: Enable buffer write support
iio: buffer-dmaengine: Support specifying buffer direction
iio: buffer-dmaengine: Enable write support
iio: core: Add new DMABUF interface infrastructure
iio: buffer-dma: Use DMABUFs instead of custom solution
iio: buffer-dma: Implement new DMABUF based userspace API
iio: buffer-dma: Boost performance using write-combine cache setting
iio: buffer-dmaengine: Support new DMABUF based userspace API
iio: core: Add support for cyclic buffers
iio: buffer-dmaengine: Add support for cyclic buffers
Documentation: iio: Document high-speed DMABUF based API

 Documentation/driver-api/dma-buf.rst          |   2 +
 Documentation/iio/dmabuf_api.rst              |  94 +++
 Documentation/iio/index.rst                   |   2 +
 drivers/iio/adc/adi-axi-adc.c                 |   3 +-
 drivers/iio/buffer/industrialio-buffer-dma.c  | 670 ++++++++++++++----
 .../buffer/industrialio-buffer-dmaengine.c    |  42 +-
 drivers/iio/industrialio-buffer.c             |  49 ++
 include/linux/iio/buffer-dma.h                |  43 +-
 include/linux/iio/buffer-dmaengine.h          |   5 +-
 include/linux/iio/buffer_impl.h               |   8 +
 include/uapi/linux/iio/buffer.h               |  30 +
 11 files changed, 783 insertions(+), 165 deletions(-)
 create mode 100644 Documentation/iio/dmabuf_api.rst
Signature · 1 line
2.33.0
#1 lore raw

PATCH 1/15 iio: buffer-dma: Get rid of incoming/outgoing queues

The buffer-dma code was using two queues, incoming and outgoing, to manage the state of the blocks in use.

While this totally works, it adds some complexity to the code, especially since the code only manages 2 blocks. It is much easier to just check each block's state manually, and keep a counter for the next block to dequeue.

Since the new DMABUF based API wouldn't use these incoming and outgoing queues anyway, getting rid of them now makes the upcoming changes simpler.

---
 drivers/iio/buffer/industrialio-buffer-dma.c | 68 ++++++++++----------
 include/linux/iio/buffer-dma.h               |  7 +-
 2 files changed, 37 insertions(+), 38 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +34 -34
index d348af8b9705..abac88f20104 100644
@@ -191,16 +191,8 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 
 static void _iio_dma_buffer_block_done(struct iio_dma_buffer_block *block)
 {
-	struct iio_dma_buffer_queue *queue = block->queue;
-
-	/*
-	 * The buffer has already been freed by the application, just drop the
-	 * reference.
-	 */
-	if (block->state != IIO_BLOCK_STATE_DEAD) {
+	if (block->state != IIO_BLOCK_STATE_DEAD)
 		block->state = IIO_BLOCK_STATE_DONE;
-		list_add_tail(&block->head, &queue->outgoing);
-	}
 }
 
 /**
@@ -317,11 +309,8 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
 	 * dead. This means we can reset the lists without having to fear
 	 * corrution.
 	 */
-	INIT_LIST_HEAD(&queue->outgoing);
 	spin_unlock_irq(&queue->list_lock);
 
-	INIT_LIST_HEAD(&queue->incoming);
-
 	for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
 		if (queue->fileio.blocks[i]) {
 			block = queue->fileio.blocks[i];
@@ -346,7 +335,6 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
 		}
 
 		block->state = IIO_BLOCK_STATE_QUEUED;
-		list_add_tail(&block->head, &queue->incoming);
 	}
 
 out_unlock:
@@ -401,13 +389,18 @@ int iio_dma_buffer_enable(struct iio_buffer *buffer,
 	struct iio_dev *indio_dev)
 {
 	struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer);
-	struct iio_dma_buffer_block *block, *_block;
+	struct iio_dma_buffer_block *block;
+	unsigned int i;
 
 	mutex_lock(&queue->lock);
 	queue->active = true;
-	list_for_each_entry_safe(block, _block, &queue->incoming, head) {
-		list_del(&block->head);
-		iio_dma_buffer_submit_block(queue, block);
+	queue->fileio.next_dequeue = 0;
+
+	for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
+		block = queue->fileio.blocks[i];
+
+		if (block->state == IIO_BLOCK_STATE_QUEUED)
+			iio_dma_buffer_submit_block(queue, block);
 	}
 	mutex_unlock(&queue->lock);
 
@@ -442,28 +435,33 @@ EXPORT_SYMBOL_GPL(iio_dma_buffer_disable);
 static void iio_dma_buffer_enqueue(struct iio_dma_buffer_queue *queue,
 	struct iio_dma_buffer_block *block)
 {
-	if (block->state == IIO_BLOCK_STATE_DEAD) {
+	if (block->state == IIO_BLOCK_STATE_DEAD)
 		iio_buffer_block_put(block);
-	} else if (queue->active) {
+	else if (queue->active)
 		iio_dma_buffer_submit_block(queue, block);
-	} else {
+	else
 		block->state = IIO_BLOCK_STATE_QUEUED;
-		list_add_tail(&block->head, &queue->incoming);
-	}
 }
 
 static struct iio_dma_buffer_block *iio_dma_buffer_dequeue(
 	struct iio_dma_buffer_queue *queue)
 {
 	struct iio_dma_buffer_block *block;
+	unsigned int idx;
 
 	spin_lock_irq(&queue->list_lock);
-	block = list_first_entry_or_null(&queue->outgoing, struct
-		iio_dma_buffer_block, head);
-	if (block != NULL) {
-		list_del(&block->head);
+
+	idx = queue->fileio.next_dequeue;
+	block = queue->fileio.blocks[idx];
+
+	if (block->state == IIO_BLOCK_STATE_DONE) {
 		block->state = IIO_BLOCK_STATE_DEQUEUED;
+		idx = (idx + 1) % ARRAY_SIZE(queue->fileio.blocks);
+		queue->fileio.next_dequeue = idx;
+	} else {
+		block = NULL;
 	}
+
 	spin_unlock_irq(&queue->list_lock);
 
 	return block;
@@ -539,6 +537,7 @@ size_t iio_dma_buffer_data_available(struct iio_buffer *buf)
 	struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buf);
 	struct iio_dma_buffer_block *block;
 	size_t data_available = 0;
+	unsigned int i;
 
 	/*
 	 * For counting the available bytes we'll use the size of the block not
@@ -552,8 +551,15 @@ size_t iio_dma_buffer_data_available(struct iio_buffer *buf)
 		data_available += queue->fileio.active_block->size;
 
 	spin_lock_irq(&queue->list_lock);
-	list_for_each_entry(block, &queue->outgoing, head)
-		data_available += block->size;
+
+	for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
+		block = queue->fileio.blocks[i];
+
+		if (block != queue->fileio.active_block
+		    && block->state == IIO_BLOCK_STATE_DONE)
+			data_available += block->size;
+	}
+
 	spin_unlock_irq(&queue->list_lock);
 	mutex_unlock(&queue->lock);
 
@@ -616,9 +622,6 @@ int iio_dma_buffer_init(struct iio_dma_buffer_queue *queue,
 	queue->dev = dev;
 	queue->ops = ops;
 
-	INIT_LIST_HEAD(&queue->incoming);
-	INIT_LIST_HEAD(&queue->outgoing);
-
 	mutex_init(&queue->lock);
 	spin_lock_init(&queue->list_lock);
 
@@ -645,11 +648,8 @@ void iio_dma_buffer_exit(struct iio_dma_buffer_queue *queue)
 			continue;
 		queue->fileio.blocks[i]->state = IIO_BLOCK_STATE_DEAD;
 	}
-	INIT_LIST_HEAD(&queue->outgoing);
 	spin_unlock_irq(&queue->list_lock);
 
-	INIT_LIST_HEAD(&queue->incoming);
-
 	for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
 		if (!queue->fileio.blocks[i])
 			continue;
include/linux/iio/buffer-dma.h +3 -4
index ff15c61bf319..d4ed5ff39d44 100644
@@ -78,12 +78,15 @@ struct iio_dma_buffer_block {
  * @active_block: Block being used in read()
  * @pos: Read offset in the active block
  * @block_size: Size of each block
+ * @next_dequeue: index of next block that will be dequeued
  */
 struct iio_dma_buffer_queue_fileio {
 	struct iio_dma_buffer_block *blocks[2];
 	struct iio_dma_buffer_block *active_block;
 	size_t pos;
 	size_t block_size;
+
+	unsigned int next_dequeue;
 };
 
 /**
@@ -97,8 +100,6 @@ struct iio_dma_buffer_queue_fileio {
  *   atomic context as well as blocks on those lists. This is the outgoing queue
  *   list and typically also a list of active blocks in the part that handles
  *   the DMA controller
- * @incoming: List of buffers on the incoming queue
- * @outgoing: List of buffers on the outgoing queue
  * @active: Whether the buffer is currently active
  * @fileio: FileIO state
  */
@@ -109,8 +110,6 @@ struct iio_dma_buffer_queue {
 
 	struct mutex lock;
 	spinlock_t list_lock;
-	struct list_head incoming;
-	struct list_head outgoing;
 
 	bool active;
 
Signature · 1 line
2.33.0
Reviewed-by on v1 1/15
#2 lore raw

Re: [PATCH 01/15] iio: buffer-dma: Get rid of incoming/outgoing queues

Quoted 4 lines · Paul Cercueil

The buffer-dma code was using two queues, incoming and outgoing, to manage the state of the blocks in use.

Quoted 9 lines

While this totally works, it adds some complexity to the code, especially since the code only manages 2 blocks. It is much easier to just check each block's state manually, and keep a counter for the next block to dequeue.

Since the new DMABUF based API wouldn't use these incoming and outgoing queues anyway, getting rid of them now makes the upcoming changes simpler.

Quoted 203 lines
---
 drivers/iio/buffer/industrialio-buffer-dma.c | 68 ++++++++++----------
 include/linux/iio/buffer-dma.h               |  7 +-
 2 files changed, 37 insertions(+), 38 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +34 -34
index d348af8b9705..abac88f20104 100644
@@ -191,16 +191,8 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 
 static void _iio_dma_buffer_block_done(struct iio_dma_buffer_block *block)
 {
-       struct iio_dma_buffer_queue *queue = block->queue;
-
-       /*
-        * The buffer has already been freed by the application, just drop the
-        * reference.
-        */
-       if (block->state != IIO_BLOCK_STATE_DEAD) {
+       if (block->state != IIO_BLOCK_STATE_DEAD)
                block->state = IIO_BLOCK_STATE_DONE;
-               list_add_tail(&block->head, &queue->outgoing);
-       }
 }
 
 /**
@@ -317,11 +309,8 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
         * dead. This means we can reset the lists without having to fear
         * corrution.
         */
-       INIT_LIST_HEAD(&queue->outgoing);
        spin_unlock_irq(&queue->list_lock);
 
-       INIT_LIST_HEAD(&queue->incoming);
-
        for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
                if (queue->fileio.blocks[i]) {
                        block = queue->fileio.blocks[i];
@@ -346,7 +335,6 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
                }
 
                block->state = IIO_BLOCK_STATE_QUEUED;
-               list_add_tail(&block->head, &queue->incoming);
        }
 
 out_unlock:
@@ -401,13 +389,18 @@ int iio_dma_buffer_enable(struct iio_buffer *buffer,
        struct iio_dev *indio_dev)
 {
        struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer);
-       struct iio_dma_buffer_block *block, *_block;
+       struct iio_dma_buffer_block *block;
+       unsigned int i;
 
        mutex_lock(&queue->lock);
        queue->active = true;
-       list_for_each_entry_safe(block, _block, &queue->incoming, head) {
-               list_del(&block->head);
-               iio_dma_buffer_submit_block(queue, block);
+       queue->fileio.next_dequeue = 0;
+
+       for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
+               block = queue->fileio.blocks[i];
+
+               if (block->state == IIO_BLOCK_STATE_QUEUED)
+                       iio_dma_buffer_submit_block(queue, block);
        }
        mutex_unlock(&queue->lock);
 
@@ -442,28 +435,33 @@ EXPORT_SYMBOL_GPL(iio_dma_buffer_disable);
 static void iio_dma_buffer_enqueue(struct iio_dma_buffer_queue *queue,
        struct iio_dma_buffer_block *block)
 {
-       if (block->state == IIO_BLOCK_STATE_DEAD) {
+       if (block->state == IIO_BLOCK_STATE_DEAD)
                iio_buffer_block_put(block);
-       } else if (queue->active) {
+       else if (queue->active)
                iio_dma_buffer_submit_block(queue, block);
-       } else {
+       else
                block->state = IIO_BLOCK_STATE_QUEUED;
-               list_add_tail(&block->head, &queue->incoming);
-       }
 }
 
 static struct iio_dma_buffer_block *iio_dma_buffer_dequeue(
        struct iio_dma_buffer_queue *queue)
 {
        struct iio_dma_buffer_block *block;
+       unsigned int idx;
 
        spin_lock_irq(&queue->list_lock);
-       block = list_first_entry_or_null(&queue->outgoing, struct
-               iio_dma_buffer_block, head);
-       if (block != NULL) {
-               list_del(&block->head);
+
+       idx = queue->fileio.next_dequeue;
+       block = queue->fileio.blocks[idx];
+
+       if (block->state == IIO_BLOCK_STATE_DONE) {
                block->state = IIO_BLOCK_STATE_DEQUEUED;
+               idx = (idx + 1) % ARRAY_SIZE(queue->fileio.blocks);
+               queue->fileio.next_dequeue = idx;
+       } else {
+               block = NULL;
        }
+
        spin_unlock_irq(&queue->list_lock);
 
        return block;
@@ -539,6 +537,7 @@ size_t iio_dma_buffer_data_available(struct iio_buffer *buf)
        struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buf);
        struct iio_dma_buffer_block *block;
        size_t data_available = 0;
+       unsigned int i;
 
        /*
         * For counting the available bytes we'll use the size of the block not
@@ -552,8 +551,15 @@ size_t iio_dma_buffer_data_available(struct iio_buffer *buf)
                data_available += queue->fileio.active_block->size;
 
        spin_lock_irq(&queue->list_lock);
-       list_for_each_entry(block, &queue->outgoing, head)
-               data_available += block->size;
+
+       for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
+               block = queue->fileio.blocks[i];
+
+               if (block != queue->fileio.active_block
+                   && block->state == IIO_BLOCK_STATE_DONE)
+                       data_available += block->size;
+       }
+
        spin_unlock_irq(&queue->list_lock);
        mutex_unlock(&queue->lock);
 
@@ -616,9 +622,6 @@ int iio_dma_buffer_init(struct iio_dma_buffer_queue *queue,
        queue->dev = dev;
        queue->ops = ops;
 
-       INIT_LIST_HEAD(&queue->incoming);
-       INIT_LIST_HEAD(&queue->outgoing);
-
        mutex_init(&queue->lock);
        spin_lock_init(&queue->list_lock);
 
@@ -645,11 +648,8 @@ void iio_dma_buffer_exit(struct iio_dma_buffer_queue *queue)
                        continue;
                queue->fileio.blocks[i]->state = IIO_BLOCK_STATE_DEAD;
        }
-       INIT_LIST_HEAD(&queue->outgoing);
        spin_unlock_irq(&queue->list_lock);
 
-       INIT_LIST_HEAD(&queue->incoming);
-
        for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
                if (!queue->fileio.blocks[i])
                        continue;
include/linux/iio/buffer-dma.h +3 -5
index ff15c61bf319..d4ed5ff39d44 100644
@@ -78,12 +78,15 @@ struct iio_dma_buffer_block {
  * @active_block: Block being used in read()
  * @pos: Read offset in the active block
  * @block_size: Size of each block
+ * @next_dequeue: index of next block that will be dequeued
  */
 struct iio_dma_buffer_queue_fileio {
        struct iio_dma_buffer_block *blocks[2];
        struct iio_dma_buffer_block *active_block;
        size_t pos;
        size_t block_size;
+
+       unsigned int next_dequeue;
 };
 
 /**
@@ -97,8 +100,6 @@ struct iio_dma_buffer_queue_fileio {
  *   atomic context as well as blocks on those lists. This is the outgoing queue
  *   list and typically also a list of active blocks in the part that handles
  *   the DMA controller
- * @incoming: List of buffers on the incoming queue
- * @outgoing: List of buffers on the outgoing queue
  * @active: Whether the buffer is currently active
  * @fileio: FileIO state
  */
@@ -109,8 +110,6 @@ struct iio_dma_buffer_queue {
 
        struct mutex lock;
        spinlock_t list_lock;
-       struct list_head incoming;
-       struct list_head outgoing;
 
        bool active;
 
--

2.33.0

maintainer
#3 lore raw

Re: [PATCH 01/15] iio: buffer-dma: Get rid of incoming/outgoing queues

In reply to Paul Cercueil
Quoted 8 lines · On Mon, 15 Nov 2021 14:19:11 +0000 Paul Cercueil <[email protected]> wrote:

The buffer-dma code was using two queues, incoming and outgoing, to manage the state of the blocks in use.

While this totally works, it adds some complexity to the code, especially since the code only manages 2 blocks. It is much easier to just check each block's state manually, and keep a counter for the next block to dequeue.

Quoted 5 lines

Since the new DMABUF based API wouldn't use these incoming and outgoing queues anyway, getting rid of them now makes the upcoming changes simpler.

This looks to be a nice little cleanup on it's own.

Let me know if you'd like me to pick this up before the main discussion comes to any conclusion.

Jonathan

Quoted 199 lines
---
 drivers/iio/buffer/industrialio-buffer-dma.c | 68 ++++++++++----------
 include/linux/iio/buffer-dma.h               |  7 +-
 2 files changed, 37 insertions(+), 38 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +34 -34
index d348af8b9705..abac88f20104 100644
@@ -191,16 +191,8 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 
 static void _iio_dma_buffer_block_done(struct iio_dma_buffer_block *block)
 {
-	struct iio_dma_buffer_queue *queue = block->queue;
-
-	/*
-	 * The buffer has already been freed by the application, just drop the
-	 * reference.
-	 */
-	if (block->state != IIO_BLOCK_STATE_DEAD) {
+	if (block->state != IIO_BLOCK_STATE_DEAD)
 		block->state = IIO_BLOCK_STATE_DONE;
-		list_add_tail(&block->head, &queue->outgoing);
-	}
 }
 
 /**
@@ -317,11 +309,8 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
 	 * dead. This means we can reset the lists without having to fear
 	 * corrution.
 	 */
-	INIT_LIST_HEAD(&queue->outgoing);
 	spin_unlock_irq(&queue->list_lock);
 
-	INIT_LIST_HEAD(&queue->incoming);
-
 	for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
 		if (queue->fileio.blocks[i]) {
 			block = queue->fileio.blocks[i];
@@ -346,7 +335,6 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
 		}
 
 		block->state = IIO_BLOCK_STATE_QUEUED;
-		list_add_tail(&block->head, &queue->incoming);
 	}
 
 out_unlock:
@@ -401,13 +389,18 @@ int iio_dma_buffer_enable(struct iio_buffer *buffer,
 	struct iio_dev *indio_dev)
 {
 	struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer);
-	struct iio_dma_buffer_block *block, *_block;
+	struct iio_dma_buffer_block *block;
+	unsigned int i;
 
 	mutex_lock(&queue->lock);
 	queue->active = true;
-	list_for_each_entry_safe(block, _block, &queue->incoming, head) {
-		list_del(&block->head);
-		iio_dma_buffer_submit_block(queue, block);
+	queue->fileio.next_dequeue = 0;
+
+	for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
+		block = queue->fileio.blocks[i];
+
+		if (block->state == IIO_BLOCK_STATE_QUEUED)
+			iio_dma_buffer_submit_block(queue, block);
 	}
 	mutex_unlock(&queue->lock);
 
@@ -442,28 +435,33 @@ EXPORT_SYMBOL_GPL(iio_dma_buffer_disable);
 static void iio_dma_buffer_enqueue(struct iio_dma_buffer_queue *queue,
 	struct iio_dma_buffer_block *block)
 {
-	if (block->state == IIO_BLOCK_STATE_DEAD) {
+	if (block->state == IIO_BLOCK_STATE_DEAD)
 		iio_buffer_block_put(block);
-	} else if (queue->active) {
+	else if (queue->active)
 		iio_dma_buffer_submit_block(queue, block);
-	} else {
+	else
 		block->state = IIO_BLOCK_STATE_QUEUED;
-		list_add_tail(&block->head, &queue->incoming);
-	}
 }
 
 static struct iio_dma_buffer_block *iio_dma_buffer_dequeue(
 	struct iio_dma_buffer_queue *queue)
 {
 	struct iio_dma_buffer_block *block;
+	unsigned int idx;
 
 	spin_lock_irq(&queue->list_lock);
-	block = list_first_entry_or_null(&queue->outgoing, struct
-		iio_dma_buffer_block, head);
-	if (block != NULL) {
-		list_del(&block->head);
+
+	idx = queue->fileio.next_dequeue;
+	block = queue->fileio.blocks[idx];
+
+	if (block->state == IIO_BLOCK_STATE_DONE) {
 		block->state = IIO_BLOCK_STATE_DEQUEUED;
+		idx = (idx + 1) % ARRAY_SIZE(queue->fileio.blocks);
+		queue->fileio.next_dequeue = idx;
+	} else {
+		block = NULL;
 	}
+
 	spin_unlock_irq(&queue->list_lock);
 
 	return block;
@@ -539,6 +537,7 @@ size_t iio_dma_buffer_data_available(struct iio_buffer *buf)
 	struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buf);
 	struct iio_dma_buffer_block *block;
 	size_t data_available = 0;
+	unsigned int i;
 
 	/*
 	 * For counting the available bytes we'll use the size of the block not
@@ -552,8 +551,15 @@ size_t iio_dma_buffer_data_available(struct iio_buffer *buf)
 		data_available += queue->fileio.active_block->size;
 
 	spin_lock_irq(&queue->list_lock);
-	list_for_each_entry(block, &queue->outgoing, head)
-		data_available += block->size;
+
+	for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
+		block = queue->fileio.blocks[i];
+
+		if (block != queue->fileio.active_block
+		    && block->state == IIO_BLOCK_STATE_DONE)
+			data_available += block->size;
+	}
+
 	spin_unlock_irq(&queue->list_lock);
 	mutex_unlock(&queue->lock);
 
@@ -616,9 +622,6 @@ int iio_dma_buffer_init(struct iio_dma_buffer_queue *queue,
 	queue->dev = dev;
 	queue->ops = ops;
 
-	INIT_LIST_HEAD(&queue->incoming);
-	INIT_LIST_HEAD(&queue->outgoing);
-
 	mutex_init(&queue->lock);
 	spin_lock_init(&queue->list_lock);
 
@@ -645,11 +648,8 @@ void iio_dma_buffer_exit(struct iio_dma_buffer_queue *queue)
 			continue;
 		queue->fileio.blocks[i]->state = IIO_BLOCK_STATE_DEAD;
 	}
-	INIT_LIST_HEAD(&queue->outgoing);
 	spin_unlock_irq(&queue->list_lock);
 
-	INIT_LIST_HEAD(&queue->incoming);
-
 	for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
 		if (!queue->fileio.blocks[i])
 			continue;
include/linux/iio/buffer-dma.h +3 -4
index ff15c61bf319..d4ed5ff39d44 100644
@@ -78,12 +78,15 @@ struct iio_dma_buffer_block {
  * @active_block: Block being used in read()
  * @pos: Read offset in the active block
  * @block_size: Size of each block
+ * @next_dequeue: index of next block that will be dequeued
  */
 struct iio_dma_buffer_queue_fileio {
 	struct iio_dma_buffer_block *blocks[2];
 	struct iio_dma_buffer_block *active_block;
 	size_t pos;
 	size_t block_size;
+
+	unsigned int next_dequeue;
 };
 
 /**
@@ -97,8 +100,6 @@ struct iio_dma_buffer_queue_fileio {
  *   atomic context as well as blocks on those lists. This is the outgoing queue
  *   list and typically also a list of active blocks in the part that handles
  *   the DMA controller
- * @incoming: List of buffers on the incoming queue
- * @outgoing: List of buffers on the outgoing queue
  * @active: Whether the buffer is currently active
  * @fileio: FileIO state
  */
@@ -109,8 +110,6 @@ struct iio_dma_buffer_queue {
 
 	struct mutex lock;
 	spinlock_t list_lock;
-	struct list_head incoming;
-	struct list_head outgoing;
 
 	bool active;
 
#4 lore raw

Re: [PATCH 01/15] iio: buffer-dma: Get rid of incoming/outgoing queues

In reply to Paul Cercueil
Quoted 8 lines · Paul Cercueil

The buffer-dma code was using two queues, incoming and outgoing, to manage the state of the blocks in use.

While this totally works, it adds some complexity to the code, especially since the code only manages 2 blocks. It is much easier to just check each block's state manually, and keep a counter for the next block to dequeue.

Quoted 5 lines

Since the new DMABUF based API wouldn't use these incoming and outgoing queues anyway, getting rid of them now makes the upcoming changes simpler.

The outgoing queue is going to be replaced by fences, but I think we need to keep the incoming queue.

Quoted 5 lines

[...]

drivers/iio/buffer/industrialio-buffer-dma.c +0 -0
@@ -442,28 +435,33 @@ EXPORT_SYMBOL_GPL(iio_dma_buffer_disable);
  static void iio_dma_buffer_enqueue(struct iio_dma_buffer_queue *queue,
  	struct iio_dma_buffer_block *block)
  {

drivers/iio/buffer/industrialio-buffer-dma.c EXPORT_SYMBOL_GPL(iio_dma_buffer_disable); · line 451

Quoted 10 lines
drivers/iio/buffer/industrialio-buffer-dma.c +3 -4
-	if (block->state == IIO_BLOCK_STATE_DEAD) {
+	if (block->state == IIO_BLOCK_STATE_DEAD)
  		iio_buffer_block_put(block);
-	} else if (queue->active) {
+	else if (queue->active)
  		iio_dma_buffer_submit_block(queue, block);
-	} else {
+	else
  		block->state = IIO_BLOCK_STATE_QUEUED;
-		list_add_tail(&block->head, &queue->incoming);

If iio_dma_buffer_enqueue() is called with a dmabuf and the buffer is not active, it will be marked as queued, but we don't actually keep a reference to it anywhere. It will never be submitted to the DMA, and it will never be signaled as completed.

#5 lore raw

Re: [PATCH 01/15] iio: buffer-dma: Get rid of incoming/outgoing queues

Hi Lars,

Quoted 9 lines · Lars-Peter Clausen
Quoted 8 lines · Paul Cercueil

The buffer-dma code was using two queues, incoming and outgoing, to manage the state of the blocks in use.

While this totally works, it adds some complexity to the code, especially since the code only manages 2 blocks. It is much easier to just check each block's state manually, and keep a counter for the next block to dequeue.

Quoted 6 lines
Quoted 5 lines

Since the new DMABUF based API wouldn't use these incoming and outgoing queues anyway, getting rid of them now makes the upcoming changes simpler.

The outgoing queue is going to be replaced by fences, but I think we need to keep the incoming queue.

Blocks are always accessed in sequential order, so we now have a "queue->next_dequeue" that cycles between the buffers allocated for fileio.

Quoted 9 lines
Quoted 9 lines

[...]

drivers/iio/buffer/industrialio-buffer-dma.c +1 -1
@@ -442,28 +435,33 @@ EXPORT_SYMBOL_GPL(iio_dma_buffer_disable);
  static void iio_dma_buffer_enqueue(struct iio_dma_buffer_queue 
 *queue,
  	struct iio_dma_buffer_block *block)
  {
-	if (block->state == IIO_BLOCK_STATE_DEAD) {
+	if (block->state == IIO_BLOCK_STATE_DEAD)
  		iio_buffer_block_put(block);
Quoted 8 lines

drivers/iio/buffer/industrialio-buffer-dma.c EXPORT_SYMBOL_GPL(iio_dma_buffer_disable); · line 451

Quoted 7 lines
drivers/iio/buffer/industrialio-buffer-dma.c +2 -3
-	} else if (queue->active) {
+	else if (queue->active)
  		iio_dma_buffer_submit_block(queue, block);
-	} else {
+	else
  		block->state = IIO_BLOCK_STATE_QUEUED;
-		list_add_tail(&block->head, &queue->incoming);

If iio_dma_buffer_enqueue() is called with a dmabuf and the buffer is not active, it will be marked as queued, but we don't actually keep a reference to it anywhere. It will never be submitted to the DMA, and it will never be signaled as completed.

We do keep a reference to the buffers, in the queue->fileio.blocks array. When the buffer is enabled, all the blocks in that array that are in the "queued" state will be submitted to the DMA.

Cheers,
-Paul

#6 lore raw

Re: [PATCH 01/15] iio: buffer-dma: Get rid of incoming/outgoing queues

Quoted 38 lines · Paul Cercueil

Hi Lars,

Quoted 9 lines · Lars-Peter Clausen
Quoted 8 lines · Paul Cercueil

The buffer-dma code was using two queues, incoming and outgoing, to manage the state of the blocks in use.

While this totally works, it adds some complexity to the code, especially since the code only manages 2 blocks. It is much easier to just check each block's state manually, and keep a counter for the next block to dequeue.

Quoted 6 lines
Quoted 5 lines

Since the new DMABUF based API wouldn't use these incoming and outgoing queues anyway, getting rid of them now makes the upcoming changes simpler.

The outgoing queue is going to be replaced by fences, but I think we need to keep the incoming queue.

Blocks are always accessed in sequential order, so we now have a "queue->next_dequeue" that cycles between the buffers allocated for fileio.

Quoted 8 lines
Quoted 8 lines

[...]

drivers/iio/buffer/industrialio-buffer-dma.c +1 -1
@@ -442,28 +435,33 @@ EXPORT_SYMBOL_GPL(iio_dma_buffer_disable);
   static void iio_dma_buffer_enqueue(struct iio_dma_buffer_queue *queue,
       struct iio_dma_buffer_block *block)
   {
-    if (block->state == IIO_BLOCK_STATE_DEAD) {
+    if (block->state == IIO_BLOCK_STATE_DEAD)
           iio_buffer_block_put(block);
Quoted 8 lines

drivers/iio/buffer/industrialio-buffer-dma.c EXPORT_SYMBOL_GPL(iio_dma_buffer_disable); · line 451

Quoted 7 lines
drivers/iio/buffer/industrialio-buffer-dma.c +2 -3
-    } else if (queue->active) {
+    else if (queue->active)
           iio_dma_buffer_submit_block(queue, block);
-    } else {
+    else
           block->state = IIO_BLOCK_STATE_QUEUED;
-        list_add_tail(&block->head, &queue->incoming);

If iio_dma_buffer_enqueue() is called with a dmabuf and the buffer is not active, it will be marked as queued, but we don't actually keep a reference to it anywhere. It will never be submitted to the DMA, and it will never be signaled as completed.

Quoted 2 lines

We do keep a reference to the buffers, in the queue->fileio.blocks array. When the buffer is enabled, all the blocks in that array that are in the "queued" state will be submitted to the DMA.

But not when used in combination with the DMA buf changes later in this series.

#7 lore raw

Re: [PATCH 01/15] iio: buffer-dma: Get rid of incoming/outgoing queues

Quoted 40 lines · Lars-Peter Clausen
Quoted 39 lines · Paul Cercueil

Hi Lars,

Quoted 9 lines · Lars-Peter Clausen 
Quoted 8 lines · Paul Cercueil

The buffer-dma code was using two queues, incoming and outgoing, to manage the state of the blocks in use.

While this totally works, it adds some complexity to the code, especially since the code only manages 2 blocks. It is much easier to just check each block's state manually, and keep a counter for the next block to dequeue.

Quoted 6 lines
Quoted 5 lines

Since the new DMABUF based API wouldn't use these incoming and outgoing queues anyway, getting rid of them now makes the upcoming changes simpler.

The outgoing queue is going to be replaced by fences, but I think we need to keep the incoming queue.

Blocks are always accessed in sequential order, so we now have a "queue->next_dequeue" that cycles between the buffers allocated for fileio.

Quoted 17 lines
Quoted 6 lines

[...]

drivers/iio/buffer/industrialio-buffer-dma.c +0 -0
@@ -442,28 +435,33 @@ EXPORT_SYMBOL_GPL(iio_dma_buffer_disable);
  static void iio_dma_buffer_enqueue(struct iio_dma_buffer_queue 
 *queue,
      struct iio_dma_buffer_block *block)
  {

drivers/iio/buffer/industrialio-buffer-dma.c EXPORT_SYMBOL_GPL(iio_dma_buffer_disable); · line 452

Quoted 10 lines
drivers/iio/buffer/industrialio-buffer-dma.c +3 -4
-    if (block->state == IIO_BLOCK_STATE_DEAD) {
+    if (block->state == IIO_BLOCK_STATE_DEAD)
          iio_buffer_block_put(block);
-    } else if (queue->active) {
+    else if (queue->active)
          iio_dma_buffer_submit_block(queue, block);
-    } else {
+    else
          block->state = IIO_BLOCK_STATE_QUEUED;
-        list_add_tail(&block->head, &queue->incoming);

If iio_dma_buffer_enqueue() is called with a dmabuf and the buffer is not active, it will be marked as queued, but we don't actually keep a reference to it anywhere. It will never be submitted to the DMA, and it will never be signaled as completed.

Quoted 4 lines
Quoted 2 lines

We do keep a reference to the buffers, in the queue->fileio.blocks array. When the buffer is enabled, all the blocks in that array that are in the "queued" state will be submitted to the DMA.

But not when used in combination with the DMA buf changes later in this series.

That's still the case after the DMABUF changes of the series. Or can you point me exactly what you think is broken?

-Paul

#8 lore raw

Re: [PATCH 01/15] iio: buffer-dma: Get rid of incoming/outgoing queues

Quoted 42 lines · Paul Cercueil
Quoted 39 lines · Lars-Peter Clausen
Quoted 38 lines · Paul Cercueil

Hi Lars,

Quoted 9 lines · Lars-Peter Clausen 
Quoted 8 lines · Paul Cercueil

The buffer-dma code was using two queues, incoming and outgoing, to manage the state of the blocks in use.

While this totally works, it adds some complexity to the code, especially since the code only manages 2 blocks. It is much easier to just check each block's state manually, and keep a counter for the next block to dequeue.

Quoted 6 lines
Quoted 5 lines

Since the new DMABUF based API wouldn't use these incoming and outgoing queues anyway, getting rid of them now makes the upcoming changes simpler.

The outgoing queue is going to be replaced by fences, but I think we need to keep the incoming queue.

Blocks are always accessed in sequential order, so we now have a "queue->next_dequeue" that cycles between the buffers allocated for fileio.

Quoted 16 lines
Quoted 5 lines

[...]

drivers/iio/buffer/industrialio-buffer-dma.c +0 -0
@@ -442,28 +435,33 @@ EXPORT_SYMBOL_GPL(iio_dma_buffer_disable);
   static void iio_dma_buffer_enqueue(struct iio_dma_buffer_queue *queue,
       struct iio_dma_buffer_block *block)
   {

drivers/iio/buffer/industrialio-buffer-dma.c EXPORT_SYMBOL_GPL(iio_dma_buffer_disable); · line 451

Quoted 10 lines
drivers/iio/buffer/industrialio-buffer-dma.c +3 -4
-    if (block->state == IIO_BLOCK_STATE_DEAD) {
+    if (block->state == IIO_BLOCK_STATE_DEAD)
           iio_buffer_block_put(block);
-    } else if (queue->active) {
+    else if (queue->active)
           iio_dma_buffer_submit_block(queue, block);
-    } else {
+    else
           block->state = IIO_BLOCK_STATE_QUEUED;
-        list_add_tail(&block->head, &queue->incoming);

If iio_dma_buffer_enqueue() is called with a dmabuf and the buffer is not active, it will be marked as queued, but we don't actually keep a reference to it anywhere. It will never be submitted to the DMA, and it will never be signaled as completed.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

We do keep a reference to the buffers, in the queue->fileio.blocks array. When the buffer is enabled, all the blocks in that array that are in the "queued" state will be submitted to the DMA.

But not when used in combination with the DMA buf changes later in this series.

That's still the case after the DMABUF changes of the series. Or can you point me exactly what you think is broken?

When you allocate a DMABUF with the allocate IOCTL and then submit it with the enqueue IOCTL before the buffer is enabled it will end up marked as queued, but not actually be queued anywhere.

#9 lore raw

Re: [PATCH 01/15] iio: buffer-dma: Get rid of incoming/outgoing queues

Hi Lars,

Quoted 44 lines · Lars-Peter Clausen
Quoted 43 lines · Paul Cercueil
Quoted 40 lines · Lars-Peter Clausen 
Quoted 39 lines · Paul Cercueil

Hi Lars,

Quoted 9 lines · Lars-Peter Clausen 
Quoted 8 lines · Paul Cercueil

The buffer-dma code was using two queues, incoming and outgoing, to manage the state of the blocks in use.

While this totally works, it adds some complexity to the code, especially since the code only manages 2 blocks. It is much easier to just check each block's state manually, and keep a counter for the next block to dequeue.

Quoted 6 lines
Quoted 5 lines

Since the new DMABUF based API wouldn't use these incoming and outgoing queues anyway, getting rid of them now makes the upcoming changes simpler.

The outgoing queue is going to be replaced by fences, but I think we need to keep the incoming queue.

Blocks are always accessed in sequential order, so we now have a "queue->next_dequeue" that cycles between the buffers allocated for fileio.

Quoted 17 lines
Quoted 6 lines

[...]

drivers/iio/buffer/industrialio-buffer-dma.c +0 -0
@@ -442,28 +435,33 @@ EXPORT_SYMBOL_GPL(iio_dma_buffer_disable);
  static void iio_dma_buffer_enqueue(struct iio_dma_buffer_queue 
 *queue,
      struct iio_dma_buffer_block *block)
  {

drivers/iio/buffer/industrialio-buffer-dma.c EXPORT_SYMBOL_GPL(iio_dma_buffer_disable); · line 452

Quoted 10 lines
drivers/iio/buffer/industrialio-buffer-dma.c +3 -4
-    if (block->state == IIO_BLOCK_STATE_DEAD) {
+    if (block->state == IIO_BLOCK_STATE_DEAD)
          iio_buffer_block_put(block);
-    } else if (queue->active) {
+    else if (queue->active)
          iio_dma_buffer_submit_block(queue, block);
-    } else {
+    else
          block->state = IIO_BLOCK_STATE_QUEUED;
-        list_add_tail(&block->head, &queue->incoming);

If iio_dma_buffer_enqueue() is called with a dmabuf and the buffer is not active, it will be marked as queued, but we don't actually keep a reference to it anywhere. It will never be submitted to the DMA, and it will never be signaled as completed.

Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

We do keep a reference to the buffers, in the queue->fileio.blocks array. When the buffer is enabled, all the blocks in that array that are in the "queued" state will be submitted to the DMA.

But not when used in combination with the DMA buf changes later in this series.

That's still the case after the DMABUF changes of the series. Or can you point me exactly what you think is broken?

When you allocate a DMABUF with the allocate IOCTL and then submit it with the enqueue IOCTL before the buffer is enabled it will end up marked as queued, but not actually be queued anywhere.

Ok, it works for me because I never enqueue blocks before enabling the buffer. I can add a requirement that blocks must be enqueued only after the buffer is enabled.

Cheers,
-Paul

#10 lore raw

Re: [PATCH 01/15] iio: buffer-dma: Get rid of incoming/outgoing queues

Quoted 51 lines · Paul Cercueil

Hi Lars,

Quoted 48 lines · Lars-Peter Clausen
Quoted 47 lines · Paul Cercueil
Quoted 43 lines · Lars-Peter Clausen 
Quoted 38 lines · Paul Cercueil

Hi Lars,

Quoted 9 lines · Lars-Peter Clausen 
Quoted 8 lines · Paul Cercueil

The buffer-dma code was using two queues, incoming and outgoing, to manage the state of the blocks in use.

While this totally works, it adds some complexity to the code, especially since the code only manages 2 blocks. It is much easier to just check each block's state manually, and keep a counter for the next block to dequeue.

Quoted 6 lines
Quoted 5 lines

Since the new DMABUF based API wouldn't use these incoming and outgoing queues anyway, getting rid of them now makes the upcoming changes simpler.

The outgoing queue is going to be replaced by fences, but I think we need to keep the incoming queue.

Blocks are always accessed in sequential order, so we now have a "queue->next_dequeue" that cycles between the buffers allocated for fileio.

Quoted 8 lines
Quoted 8 lines

[...]

(quoted diff) +1 -1
@@ -442,28 +435,33 @@ EXPORT_SYMBOL_GPL(iio_dma_buffer_disable);
   static void iio_dma_buffer_enqueue(struct iio_dma_buffer_queue *queue,
       struct iio_dma_buffer_block *block)
   {
-    if (block->state == IIO_BLOCK_STATE_DEAD) {
+    if (block->state == IIO_BLOCK_STATE_DEAD)
           iio_buffer_block_put(block);
Quoted 8 lines

Review EXPORT_SYMBOL_GPL(iio_dma_buffer_disable); · line 451

Quoted 7 lines
(quoted diff) +2 -3
-    } else if (queue->active) {
+    else if (queue->active)
           iio_dma_buffer_submit_block(queue, block);
-    } else {
+    else
           block->state = IIO_BLOCK_STATE_QUEUED;
-        list_add_tail(&block->head, &queue->incoming);

If iio_dma_buffer_enqueue() is called with a dmabuf and the buffer is not active, it will be marked as queued, but we don't actually keep a reference to it anywhere. It will never be submitted to the DMA, and it will never be signaled as completed.

Quoted 2 lines

We do keep a reference to the buffers, in the queue->fileio.blocks array. When the buffer is enabled, all the blocks in that array that are in the "queued" state will be submitted to the DMA.

But not when used in combination with the DMA buf changes later in this series.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

That's still the case after the DMABUF changes of the series. Or can you point me exactly what you think is broken?

When you allocate a DMABUF with the allocate IOCTL and then submit it with the enqueue IOCTL before the buffer is enabled it will end up marked as queued, but not actually be queued anywhere.

Ok, it works for me because I never enqueue blocks before enabling the buffer. I can add a requirement that blocks must be enqueued only after the buffer is enabled.

I don't think that is a good idea. This way you are going to potentially drop data at the begining of your stream when the DMA isn't ready yet.

#11 lore raw

Re: [PATCH 01/15] iio: buffer-dma: Get rid of incoming/outgoing queues

Quoted 54 lines · Lars-Peter Clausen
Quoted 53 lines · Paul Cercueil

Hi Lars,

Quoted 50 lines · Lars-Peter Clausen 
Quoted 49 lines · Paul Cercueil
Quoted 45 lines · Lars-Peter Clausen 
Quoted 40 lines · Paul Cercueil

Hi Lars,

Quoted 9 lines · Lars-Peter Clausen 
Quoted 8 lines · Paul Cercueil

The buffer-dma code was using two queues, incoming and outgoing, to manage the state of the blocks in use.

While this totally works, it adds some complexity to the code, especially since the code only manages 2 blocks. It is much easier to just check each block's state manually, and keep a counter for the next block to dequeue.

Quoted 6 lines
Quoted 5 lines

Since the new DMABUF based API wouldn't use these incoming and outgoing queues anyway, getting rid of them now makes the upcoming changes simpler.

The outgoing queue is going to be replaced by fences, but I think we need to keep the incoming queue.

Blocks are always accessed in sequential order, so we now have a "queue->next_dequeue" that cycles between the buffers allocated for fileio.

Quoted 10 lines
Quoted 10 lines

[...]

(quoted diff) +1 -1
@@ -442,28 +435,33 @@ 
 EXPORT_SYMBOL_GPL(iio_dma_buffer_disable);
  static void iio_dma_buffer_enqueue(struct 
 iio_dma_buffer_queue *queue,
      struct iio_dma_buffer_block *block)
  {
-    if (block->state == IIO_BLOCK_STATE_DEAD) {
+    if (block->state == IIO_BLOCK_STATE_DEAD)
          iio_buffer_block_put(block);
Quoted 8 lines

Review line 453

Quoted 7 lines
(quoted diff) +2 -3
-    } else if (queue->active) {
+    else if (queue->active)
          iio_dma_buffer_submit_block(queue, block);
-    } else {
+    else
          block->state = IIO_BLOCK_STATE_QUEUED;
-        list_add_tail(&block->head, &queue->incoming);

If iio_dma_buffer_enqueue() is called with a dmabuf and the buffer is not active, it will be marked as queued, but we don't actually keep a reference to it anywhere. It will never be submitted to the DMA, and it will never be signaled as completed.

Quoted 2 lines

We do keep a reference to the buffers, in the queue->fileio.blocks array. When the buffer is enabled, all the blocks in that array that are in the "queued" state will be submitted to the DMA.

But not when used in combination with the DMA buf changes later in this series.

Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

That's still the case after the DMABUF changes of the series. Or can you point me exactly what you think is broken?

When you allocate a DMABUF with the allocate IOCTL and then submit it with the enqueue IOCTL before the buffer is enabled it will end up marked as queued, but not actually be queued anywhere.

Ok, it works for me because I never enqueue blocks before enabling the buffer. I can add a requirement that blocks must be enqueued only after the buffer is enabled.

I don't think that is a good idea. This way you are going to potentially drop data at the begining of your stream when the DMA isn't ready yet.

You wouldn't drop data, but it could cause an underrun, yes. Is it such a big deal, knowing that the buffer was just enabled? I don't think you can disable then enable the buffer without causing a discontinuity in the stream.

-Paul

#12 lore raw

PATCH 2/15 iio: buffer-dma: Remove unused iio_buffer_block struct

In reply to Paul Cercueil

This structure was never used anywhere, so it can safely be dropped.

It will later be re-introduced as a different structure in a different header.

---
 include/linux/iio/buffer-dma.h | 5 -----
 1 file changed, 5 deletions(-)
include/linux/iio/buffer-dma.h +0 -5
index d4ed5ff39d44..a65a005c4a19 100644
@@ -17,11 +17,6 @@ struct iio_dma_buffer_queue;
 struct iio_dma_buffer_ops;
 struct device;
 
-struct iio_buffer_block {
-	u32 size;
-	u32 bytes_used;
-};
-
 /**
  * enum iio_block_state - State of a struct iio_dma_buffer_block
  * @IIO_BLOCK_STATE_DEQUEUED: Block is not queued
Signature · 1 line
2.33.0
Reviewed-by on v1 2/15
#13 lore raw

Re: [PATCH 02/15] iio: buffer-dma: Remove unused iio_buffer_block struct

Quoted 5 lines · Paul Cercueil

This structure was never used anywhere, so it can safely be dropped.

It will later be re-introduced as a different structure in a different header.

Quoted 25 lines
---
 include/linux/iio/buffer-dma.h | 5 -----
 1 file changed, 5 deletions(-)
include/linux/iio/buffer-dma.h +0 -6
index d4ed5ff39d44..a65a005c4a19 100644
@@ -17,11 +17,6 @@ struct iio_dma_buffer_queue;
 struct iio_dma_buffer_ops;
 struct device;
 
-struct iio_buffer_block {
-       u32 size;
-       u32 bytes_used;
-};
-
 /**
  * enum iio_block_state - State of a struct iio_dma_buffer_block
  * @IIO_BLOCK_STATE_DEQUEUED: Block is not queued
--

2.33.0

#14 lore raw

PATCH 3/15 iio: buffer-dma: Use round_down() instead of rounddown()

In reply to Paul Cercueil

We know that the buffer's alignment will always be a power of two; therefore, we can use the faster round_down() macro.

---
 drivers/iio/buffer/industrialio-buffer-dmaengine.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iio/buffer/industrialio-buffer-dmaengine.c +1 -1
index 1ac94c4e9792..f8ce26a24c57 100644
@@ -67,7 +67,7 @@ static int iio_dmaengine_buffer_submit_block(struct iio_dma_buffer_queue *queue,
 	dma_cookie_t cookie;
 
 	block->bytes_used = min(block->size, dmaengine_buffer->max_size);
-	block->bytes_used = rounddown(block->bytes_used,
+	block->bytes_used = round_down(block->bytes_used,
 			dmaengine_buffer->align);
 
 	desc = dmaengine_prep_slave_single(dmaengine_buffer->chan,
Signature · 1 line
2.33.0
Reviewed-by on v1 3/15
#15 lore raw

Re: [PATCH 03/15] iio: buffer-dma: Use round_down() instead of rounddown()

Quoted 4 lines · Paul Cercueil

We know that the buffer's alignment will always be a power of two; therefore, we can use the faster round_down() macro.

Quoted 21 lines
---
 drivers/iio/buffer/industrialio-buffer-dmaengine.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iio/buffer/industrialio-buffer-dmaengine.c +1 -2
index 1ac94c4e9792..f8ce26a24c57 100644
@@ -67,7 +67,7 @@ static int iio_dmaengine_buffer_submit_block(struct iio_dma_buffer_queue *queue,
        dma_cookie_t cookie;
 
        block->bytes_used = min(block->size, dmaengine_buffer->max_size);
-       block->bytes_used = rounddown(block->bytes_used,
+       block->bytes_used = round_down(block->bytes_used,
                        dmaengine_buffer->align);
 
        desc = dmaengine_prep_slave_single(dmaengine_buffer->chan,
--

2.33.0

maintainer
#16 lore raw

Re: [PATCH 03/15] iio: buffer-dma: Use round_down() instead of rounddown()

In reply to Paul Cercueil
Quoted 4 lines · On Mon, 15 Nov 2021 14:19:13 +0000 Paul Cercueil <[email protected]> wrote:

We know that the buffer's alignment will always be a power of two; therefore, we can use the faster round_down() macro.

*groan*. I don't want to know where the naming of these two came from but that is spectacular...

Anyhow, happy to pick up 1-3 now if you like as all are good cleanup of existing code.

Jonathan

Quoted 17 lines
---
 drivers/iio/buffer/industrialio-buffer-dmaengine.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iio/buffer/industrialio-buffer-dmaengine.c +1 -1
index 1ac94c4e9792..f8ce26a24c57 100644
@@ -67,7 +67,7 @@ static int iio_dmaengine_buffer_submit_block(struct iio_dma_buffer_queue *queue,
 	dma_cookie_t cookie;
 
 	block->bytes_used = min(block->size, dmaengine_buffer->max_size);
-	block->bytes_used = rounddown(block->bytes_used,
+	block->bytes_used = round_down(block->bytes_used,
 			dmaengine_buffer->align);
 
 	desc = dmaengine_prep_slave_single(dmaengine_buffer->chan,
#17 lore raw

Re: [PATCH 03/15] iio: buffer-dma: Use round_down() instead of rounddown()

Hi Jonathan,

Quoted 6 lines · Jonathan Cameron
Quoted 3 lines · On Mon, 15 Nov 2021 14:19:13 +0000 Paul Cercueil <[email protected]> wrote:

We know that the buffer's alignment will always be a power of two; therefore, we can use the faster round_down() macro.

Quoted 6 lines
Quoted 1 line

*groan*. I don't want to know where the naming of these two came from but that is spectacular...

Anyhow, happy to pick up 1-3 now if you like as all are good cleanup of existing code.

I think you can pick 2-3 now; I will do some changes to patch [01/15] in the V2.

Cheers,
-Paul

maintainer
#18 lore raw

Re: [PATCH 03/15] iio: buffer-dma: Use round_down() instead of rounddown()

Quoted 15 lines · On Mon, 22 Nov 2021 10:00:19 +0000 Paul Cercueil <[email protected]> wrote:

Hi Jonathan,

Quoted 11 lines · Jonathan Cameron
Quoted 4 lines · On Mon, 15 Nov 2021 14:19:13 +0000 Paul Cercueil <[email protected]> wrote:

We know that the buffer's alignment will always be a power of two; therefore, we can use the faster round_down() macro.

*groan*. I don't want to know where the naming of these two came from but that
is spectacular...

Quoted 6 lines
Quoted 3 lines

Anyhow, happy to pick up 1-3 now if you like as all are good cleanup of
existing code.

I think you can pick 2-3 now; I will do some changes to patch [01/15] in the V2.

Applied, 2+3 to the togreg branch of iio.git and pushed out as testing for all the normal reasons.

Thanks,

Jonathan

Quoted 5 lines

Cheers,
-Paul

#19 lore raw

PATCH 4/15 iio: buffer-dma: Enable buffer write support

In reply to Paul Cercueil

Adding write support to the buffer-dma code is easy - the write() function basically needs to do the exact same thing as the read()
function: dequeue a block, read or write the data, enqueue the block when entirely processed.

Therefore, the iio_buffer_dma_read() and the new iio_buffer_dma_write() now both call a function iio_buffer_dma_io(), which will perform this task.

The .space_available() callback can return the exact same value as the .data_available() callback for input buffers, since in both cases we count the exact same thing (the number of bytes in each available block).

---
 drivers/iio/buffer/industrialio-buffer-dma.c | 75 +++++++++++++++-----
 include/linux/iio/buffer-dma.h               |  7 ++
 2 files changed, 66 insertions(+), 16 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +59 -16
index abac88f20104..eeeed6b2e0cf 100644
@@ -179,7 +179,8 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 	}
 
 	block->size = size;
-	block->state = IIO_BLOCK_STATE_DEQUEUED;
+	block->bytes_used = size;
+	block->state = IIO_BLOCK_STATE_DONE;
 	block->queue = queue;
 	INIT_LIST_HEAD(&block->head);
 	kref_init(&block->kref);
@@ -195,6 +196,18 @@ static void _iio_dma_buffer_block_done(struct iio_dma_buffer_block *block)
 		block->state = IIO_BLOCK_STATE_DONE;
 }
 
+static void iio_dma_buffer_queue_wake(struct iio_dma_buffer_queue *queue)
+{
+	__poll_t flags;
+
+	if (queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+		flags = EPOLLIN | EPOLLRDNORM;
+	else
+		flags = EPOLLOUT | EPOLLWRNORM;
+
+	wake_up_interruptible_poll(&queue->buffer.pollq, flags);
+}
+
 /**
  * iio_dma_buffer_block_done() - Indicate that a block has been completed
  * @block: The completed block
@@ -212,7 +225,7 @@ void iio_dma_buffer_block_done(struct iio_dma_buffer_block *block)
 	spin_unlock_irqrestore(&queue->list_lock, flags);
 
 	iio_buffer_block_put_atomic(block);
-	wake_up_interruptible_poll(&queue->buffer.pollq, EPOLLIN | EPOLLRDNORM);
+	iio_dma_buffer_queue_wake(queue);
 }
 EXPORT_SYMBOL_GPL(iio_dma_buffer_block_done);
 
@@ -241,7 +254,7 @@ void iio_dma_buffer_block_list_abort(struct iio_dma_buffer_queue *queue,
 	}
 	spin_unlock_irqrestore(&queue->list_lock, flags);
 
-	wake_up_interruptible_poll(&queue->buffer.pollq, EPOLLIN | EPOLLRDNORM);
+	iio_dma_buffer_queue_wake(queue);
 }
 EXPORT_SYMBOL_GPL(iio_dma_buffer_block_list_abort);
 
@@ -334,7 +347,8 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
 			queue->fileio.blocks[i] = block;
 		}
 
-		block->state = IIO_BLOCK_STATE_QUEUED;
+		if (queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+			block->state = IIO_BLOCK_STATE_QUEUED;
 	}
 
 out_unlock:
@@ -467,20 +481,12 @@ static struct iio_dma_buffer_block *iio_dma_buffer_dequeue(
 	return block;
 }
 
-/**
- * iio_dma_buffer_read() - DMA buffer read callback
- * @buffer: Buffer to read form
- * @n: Number of bytes to read
- * @user_buffer: Userspace buffer to copy the data to
- *
- * Should be used as the read callback for iio_buffer_access_ops
- * struct for DMA buffers.
- */
-int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
-	char __user *user_buffer)
+static int iio_dma_buffer_io(struct iio_buffer *buffer,
+			     size_t n, char __user *user_buffer, bool is_write)
 {
 	struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer);
 	struct iio_dma_buffer_block *block;
+	void *addr;
 	int ret;
 
 	if (n < buffer->bytes_per_datum)
@@ -503,8 +509,13 @@ int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
 	n = rounddown(n, buffer->bytes_per_datum);
 	if (n > block->bytes_used - queue->fileio.pos)
 		n = block->bytes_used - queue->fileio.pos;
+	addr = block->vaddr + queue->fileio.pos;
 
-	if (copy_to_user(user_buffer, block->vaddr + queue->fileio.pos, n)) {
+	if (is_write)
+		ret = !!copy_from_user(addr, user_buffer, n);
+	else
+		ret = !!copy_to_user(user_buffer, addr, n);
+	if (ret) {
 		ret = -EFAULT;
 		goto out_unlock;
 	}
@@ -513,6 +524,7 @@ int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
 
 	if (queue->fileio.pos == block->bytes_used) {
 		queue->fileio.active_block = NULL;
+		block->bytes_used = block->size;
 		iio_dma_buffer_enqueue(queue, block);
 	}
 
@@ -523,8 +535,39 @@ int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
 
 	return ret;
 }
+
+/**
+ * iio_dma_buffer_read() - DMA buffer read callback
+ * @buffer: Buffer to read form
+ * @n: Number of bytes to read
+ * @user_buffer: Userspace buffer to copy the data to
+ *
+ * Should be used as the read callback for iio_buffer_access_ops
+ * struct for DMA buffers.
+ */
+int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
+	char __user *user_buffer)
+{
+	return iio_dma_buffer_io(buffer, n, user_buffer, false);
+}
 EXPORT_SYMBOL_GPL(iio_dma_buffer_read);
 
+/**
+ * iio_dma_buffer_write() - DMA buffer write callback
+ * @buffer: Buffer to read form
+ * @n: Number of bytes to read
+ * @user_buffer: Userspace buffer to copy the data from
+ *
+ * Should be used as the write callback for iio_buffer_access_ops
+ * struct for DMA buffers.
+ */
+int iio_dma_buffer_write(struct iio_buffer *buffer, size_t n,
+			 const char __user *user_buffer)
+{
+	return iio_dma_buffer_io(buffer, n, (__force char *)user_buffer, true);
+}
+EXPORT_SYMBOL_GPL(iio_dma_buffer_write);
+
 /**
  * iio_dma_buffer_data_available() - DMA buffer data_available callback
  * @buf: Buffer to check for data availability
include/linux/iio/buffer-dma.h +7 -0
index a65a005c4a19..09c07d5563c0 100644
@@ -132,6 +132,8 @@ int iio_dma_buffer_disable(struct iio_buffer *buffer,
 	struct iio_dev *indio_dev);
 int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
 	char __user *user_buffer);
+int iio_dma_buffer_write(struct iio_buffer *buffer, size_t n,
+			 const char __user *user_buffer);
 size_t iio_dma_buffer_data_available(struct iio_buffer *buffer);
 int iio_dma_buffer_set_bytes_per_datum(struct iio_buffer *buffer, size_t bpd);
 int iio_dma_buffer_set_length(struct iio_buffer *buffer, unsigned int length);
@@ -142,4 +144,9 @@ int iio_dma_buffer_init(struct iio_dma_buffer_queue *queue,
 void iio_dma_buffer_exit(struct iio_dma_buffer_queue *queue);
 void iio_dma_buffer_release(struct iio_dma_buffer_queue *queue);
 
+static inline size_t iio_dma_buffer_space_available(struct iio_buffer *buffer)
+{
+	return iio_dma_buffer_data_available(buffer);
+}
+
 #endif
Signature · 1 line
2.33.0
Reviewed-by on v1 4/15
#20 lore raw

Re: [PATCH 04/15] iio: buffer-dma: Enable buffer write support

Quoted 6 lines · Paul Cercueil

Adding write support to the buffer-dma code is easy - the write() function basically needs to do the exact same thing as the read()
function: dequeue a block, read or write the data, enqueue the block when entirely processed.

Quoted 9 lines

Therefore, the iio_buffer_dma_read() and the new iio_buffer_dma_write() now both call a function iio_buffer_dma_io(), which will perform this task.

The .space_available() callback can return the exact same value as the .data_available() callback for input buffers, since in both cases we count the exact same thing (the number of bytes in each available block).

A few comments inline from me.
Otherwise fine from me.

Quoted 11 lines
---
 drivers/iio/buffer/industrialio-buffer-dma.c | 75 +++++++++++++++-----
 include/linux/iio/buffer-dma.h               |  7 ++
 2 files changed, 66 insertions(+), 16 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +0 -0
index abac88f20104..eeeed6b2e0cf 100644

drivers/iio/buffer/industrialio-buffer-dma.c static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block( · line 183

Quoted 6 lines
drivers/iio/buffer/industrialio-buffer-dma.c +2 -1
@@ -179,7 +179,8 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
        }
 
        block->size = size;
-       block->state = IIO_BLOCK_STATE_DEQUEUED;
+       block->bytes_used = size;
+       block->state = IIO_BLOCK_STATE_DONE;

This initial state change looks like it could use a comment in the git log. Maybe it's fine, but I can't remember 100% about it.

Quoted 151 lines
drivers/iio/buffer/industrialio-buffer-dma.c +57 -15
        block->queue = queue;
        INIT_LIST_HEAD(&block->head);
        kref_init(&block->kref);
@@ -195,6 +196,18 @@ static void _iio_dma_buffer_block_done(struct iio_dma_buffer_block *block)
                block->state = IIO_BLOCK_STATE_DONE;
 }
 
+static void iio_dma_buffer_queue_wake(struct iio_dma_buffer_queue *queue)
+{
+       __poll_t flags;
+
+       if (queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+               flags = EPOLLIN | EPOLLRDNORM;
+       else
+               flags = EPOLLOUT | EPOLLWRNORM;
+
+       wake_up_interruptible_poll(&queue->buffer.pollq, flags);
+}
+
 /**
  * iio_dma_buffer_block_done() - Indicate that a block has been completed
  * @block: The completed block
@@ -212,7 +225,7 @@ void iio_dma_buffer_block_done(struct iio_dma_buffer_block *block)
        spin_unlock_irqrestore(&queue->list_lock, flags);
 
        iio_buffer_block_put_atomic(block);
-       wake_up_interruptible_poll(&queue->buffer.pollq, EPOLLIN | EPOLLRDNORM);
+       iio_dma_buffer_queue_wake(queue);
 }
 EXPORT_SYMBOL_GPL(iio_dma_buffer_block_done);
 
@@ -241,7 +254,7 @@ void iio_dma_buffer_block_list_abort(struct iio_dma_buffer_queue *queue,
        }
        spin_unlock_irqrestore(&queue->list_lock, flags);
 
-       wake_up_interruptible_poll(&queue->buffer.pollq, EPOLLIN | EPOLLRDNORM);
+       iio_dma_buffer_queue_wake(queue);
 }
 EXPORT_SYMBOL_GPL(iio_dma_buffer_block_list_abort);
 
@@ -334,7 +347,8 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
                        queue->fileio.blocks[i] = block;
                }
 
-               block->state = IIO_BLOCK_STATE_QUEUED;
+               if (queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+                       block->state = IIO_BLOCK_STATE_QUEUED;
        }
 
 out_unlock:
@@ -467,20 +481,12 @@ static struct iio_dma_buffer_block *iio_dma_buffer_dequeue(
        return block;
 }
 
-/**
- * iio_dma_buffer_read() - DMA buffer read callback
- * @buffer: Buffer to read form
- * @n: Number of bytes to read
- * @user_buffer: Userspace buffer to copy the data to
- *
- * Should be used as the read callback for iio_buffer_access_ops
- * struct for DMA buffers.
- */
-int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
-       char __user *user_buffer)
+static int iio_dma_buffer_io(struct iio_buffer *buffer,
+                            size_t n, char __user *user_buffer, bool is_write)
 {
        struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer);
        struct iio_dma_buffer_block *block;
+       void *addr;
        int ret;
 
        if (n < buffer->bytes_per_datum)
@@ -503,8 +509,13 @@ int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
        n = rounddown(n, buffer->bytes_per_datum);
        if (n > block->bytes_used - queue->fileio.pos)
                n = block->bytes_used - queue->fileio.pos;
+       addr = block->vaddr + queue->fileio.pos;
 
-       if (copy_to_user(user_buffer, block->vaddr + queue->fileio.pos, n)) {
+       if (is_write)
+               ret = !!copy_from_user(addr, user_buffer, n);
+       else
+               ret = !!copy_to_user(user_buffer, addr, n);
+       if (ret) {
                ret = -EFAULT;
                goto out_unlock;
        }
@@ -513,6 +524,7 @@ int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
 
        if (queue->fileio.pos == block->bytes_used) {
                queue->fileio.active_block = NULL;
+               block->bytes_used = block->size;
                iio_dma_buffer_enqueue(queue, block);
        }
 
@@ -523,8 +535,39 @@ int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
 
        return ret;
 }
+
+/**
+ * iio_dma_buffer_read() - DMA buffer read callback
+ * @buffer: Buffer to read form
+ * @n: Number of bytes to read
+ * @user_buffer: Userspace buffer to copy the data to
+ *
+ * Should be used as the read callback for iio_buffer_access_ops
+ * struct for DMA buffers.
+ */
+int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
+       char __user *user_buffer)
+{
+       return iio_dma_buffer_io(buffer, n, user_buffer, false);
+}
 EXPORT_SYMBOL_GPL(iio_dma_buffer_read);
 
+/**
+ * iio_dma_buffer_write() - DMA buffer write callback
+ * @buffer: Buffer to read form
+ * @n: Number of bytes to read
+ * @user_buffer: Userspace buffer to copy the data from
+ *
+ * Should be used as the write callback for iio_buffer_access_ops
+ * struct for DMA buffers.
+ */
+int iio_dma_buffer_write(struct iio_buffer *buffer, size_t n,
+                        const char __user *user_buffer)
+{
+       return iio_dma_buffer_io(buffer, n, (__force char *)user_buffer, true);
+}
+EXPORT_SYMBOL_GPL(iio_dma_buffer_write);
+
 /**
  * iio_dma_buffer_data_available() - DMA buffer data_available callback
  * @buf: Buffer to check for data availability
include/linux/iio/buffer-dma.h +2 -0
index a65a005c4a19..09c07d5563c0 100644
@@ -132,6 +132,8 @@ int iio_dma_buffer_disable(struct iio_buffer *buffer,
        struct iio_dev *indio_dev);
 int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
        char __user *user_buffer);
+int iio_dma_buffer_write(struct iio_buffer *buffer, size_t n,
+                        const char __user *user_buffer);
 size_t iio_dma_buffer_data_available(struct iio_buffer *buffer);
 int iio_dma_buffer_set_bytes_per_datum(struct iio_buffer *buffer, size_t bpd);
 int iio_dma_buffer_set_length(struct iio_buffer *buffer, unsigned int length);

include/linux/iio/buffer-dma.h int iio_dma_buffer_init(struct iio_dma_buffer_queue *queue, · line 150

Quoted 7 lines
include/linux/iio/buffer-dma.h +4 -0
@@ -142,4 +144,9 @@ int iio_dma_buffer_init(struct iio_dma_buffer_queue *queue,
 void iio_dma_buffer_exit(struct iio_dma_buffer_queue *queue);
 void iio_dma_buffer_release(struct iio_dma_buffer_queue *queue);
 
+static inline size_t iio_dma_buffer_space_available(struct iio_buffer *buffer)
+{
+       return iio_dma_buffer_data_available(buffer);
+}

Maybe re-using iio_dma_buffer_data_available() (and updating the return type) would have been a little neater than wrapping this. Also fine like this though.

Quoted 5 lines
include/linux/iio/buffer-dma.h +1 -1
+
 #endif
--

2.33.0

maintainer
#21 lore raw

Re: [PATCH 04/15] iio: buffer-dma: Enable buffer write support

In reply to Paul Cercueil
Quoted 9 lines · On Mon, 15 Nov 2021 14:19:14 +0000 Paul Cercueil <[email protected]> wrote:

Adding write support to the buffer-dma code is easy - the write() function basically needs to do the exact same thing as the read()
function: dequeue a block, read or write the data, enqueue the block when entirely processed.

Therefore, the iio_buffer_dma_read() and the new iio_buffer_dma_write() now both call a function iio_buffer_dma_io(), which will perform this task.

Quoted 6 lines

The .space_available() callback can return the exact same value as the .data_available() callback for input buffers, since in both cases we count the exact same thing (the number of bytes in each available block).

Hi Paul,

There are a few changes in here, such as the bytes_used value being set that I'm not following the reasoning behind. More info on those?
Also good to provide something about those in this patch description.

Thanks,

Jonathan

Quoted 10 lines
---
 drivers/iio/buffer/industrialio-buffer-dma.c | 75 +++++++++++++++-----
 include/linux/iio/buffer-dma.h               |  7 ++
 2 files changed, 66 insertions(+), 16 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +0 -0
index abac88f20104..eeeed6b2e0cf 100644

drivers/iio/buffer/industrialio-buffer-dma.c static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block( · line 183

Quoted 6 lines
drivers/iio/buffer/industrialio-buffer-dma.c +2 -1
@@ -179,7 +179,8 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 	}
 
 	block->size = size;
-	block->state = IIO_BLOCK_STATE_DEQUEUED;
+	block->bytes_used = size;
+	block->state = IIO_BLOCK_STATE_DONE;

I don't know why these are here - some more info?

Quoted 41 lines
drivers/iio/buffer/industrialio-buffer-dma.c +14 -2
 	block->queue = queue;
 	INIT_LIST_HEAD(&block->head);
 	kref_init(&block->kref);
@@ -195,6 +196,18 @@ static void _iio_dma_buffer_block_done(struct iio_dma_buffer_block *block)
 		block->state = IIO_BLOCK_STATE_DONE;
 }
 
+static void iio_dma_buffer_queue_wake(struct iio_dma_buffer_queue *queue)
+{
+	__poll_t flags;
+
+	if (queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+		flags = EPOLLIN | EPOLLRDNORM;
+	else
+		flags = EPOLLOUT | EPOLLWRNORM;
+
+	wake_up_interruptible_poll(&queue->buffer.pollq, flags);
+}
+
 /**
  * iio_dma_buffer_block_done() - Indicate that a block has been completed
  * @block: The completed block
@@ -212,7 +225,7 @@ void iio_dma_buffer_block_done(struct iio_dma_buffer_block *block)
 	spin_unlock_irqrestore(&queue->list_lock, flags);
 
 	iio_buffer_block_put_atomic(block);
-	wake_up_interruptible_poll(&queue->buffer.pollq, EPOLLIN | EPOLLRDNORM);
+	iio_dma_buffer_queue_wake(queue);
 }
 EXPORT_SYMBOL_GPL(iio_dma_buffer_block_done);
 
@@ -241,7 +254,7 @@ void iio_dma_buffer_block_list_abort(struct iio_dma_buffer_queue *queue,
 	}
 	spin_unlock_irqrestore(&queue->list_lock, flags);
 
-	wake_up_interruptible_poll(&queue->buffer.pollq, EPOLLIN | EPOLLRDNORM);
+	iio_dma_buffer_queue_wake(queue);
 }
 EXPORT_SYMBOL_GPL(iio_dma_buffer_block_list_abort);
 

drivers/iio/buffer/industrialio-buffer-dma.c int iio_dma_buffer_request_update(struct iio_buffer *buffer) · line 351

Quoted 6 lines
drivers/iio/buffer/industrialio-buffer-dma.c +2 -1
@@ -334,7 +347,8 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
 			queue->fileio.blocks[i] = block;
 		}
 
-		block->state = IIO_BLOCK_STATE_QUEUED;
+		if (queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+			block->state = IIO_BLOCK_STATE_QUEUED;

Possibly worth a comment on the state being set here. I figured it out, but might save some brain cells in future if it's stated in the code.

Quoted 28 lines
drivers/iio/buffer/industrialio-buffer-dma.c +3 -11
 	}
 
 out_unlock:
@@ -467,20 +481,12 @@ static struct iio_dma_buffer_block *iio_dma_buffer_dequeue(
 	return block;
 }
 
-/**
- * iio_dma_buffer_read() - DMA buffer read callback
- * @buffer: Buffer to read form
- * @n: Number of bytes to read
- * @user_buffer: Userspace buffer to copy the data to
- *
- * Should be used as the read callback for iio_buffer_access_ops
- * struct for DMA buffers.
- */
-int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
-	char __user *user_buffer)
+static int iio_dma_buffer_io(struct iio_buffer *buffer,
+			     size_t n, char __user *user_buffer, bool is_write)
 {
 	struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer);
 	struct iio_dma_buffer_block *block;
+	void *addr;
 	int ret;
 
 	if (n < buffer->bytes_per_datum)

drivers/iio/buffer/industrialio-buffer-dma.c int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n, · line 517

Quoted 10 lines
drivers/iio/buffer/industrialio-buffer-dma.c +5 -1
@@ -503,8 +509,13 @@ int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
 	n = rounddown(n, buffer->bytes_per_datum);
 	if (n > block->bytes_used - queue->fileio.pos)
 		n = block->bytes_used - queue->fileio.pos;
+	addr = block->vaddr + queue->fileio.pos;
 
-	if (copy_to_user(user_buffer, block->vaddr + queue->fileio.pos, n)) {
+	if (is_write)
+		ret = !!copy_from_user(addr, user_buffer, n);
+	else
+		ret = !!copy_to_user(user_buffer, addr, n);

What is the !! gaining us here? We only care about == 0 vs != 0 so forcing it to be 0 or 1 isn't useful.

drivers/iio/buffer/industrialio-buffer-dma.c int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n, · line 527

Quoted 9 lines
drivers/iio/buffer/industrialio-buffer-dma.c +2 -0
+	if (ret) {
 		ret = -EFAULT;
 		goto out_unlock;
 	}
@@ -513,6 +524,7 @@ int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
 
 	if (queue->fileio.pos == block->bytes_used) {
 		queue->fileio.active_block = NULL;
+		block->bytes_used = block->size;

This seems to be a functional change that isn't called out in the patch description.

Quoted 66 lines
drivers/iio/buffer/industrialio-buffer-dma.c +31 -0
 		iio_dma_buffer_enqueue(queue, block);
 	}
 
@@ -523,8 +535,39 @@ int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
 
 	return ret;
 }
+
+/**
+ * iio_dma_buffer_read() - DMA buffer read callback
+ * @buffer: Buffer to read form
+ * @n: Number of bytes to read
+ * @user_buffer: Userspace buffer to copy the data to
+ *
+ * Should be used as the read callback for iio_buffer_access_ops
+ * struct for DMA buffers.
+ */
+int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
+	char __user *user_buffer)
+{
+	return iio_dma_buffer_io(buffer, n, user_buffer, false);
+}
 EXPORT_SYMBOL_GPL(iio_dma_buffer_read);
 
+/**
+ * iio_dma_buffer_write() - DMA buffer write callback
+ * @buffer: Buffer to read form
+ * @n: Number of bytes to read
+ * @user_buffer: Userspace buffer to copy the data from
+ *
+ * Should be used as the write callback for iio_buffer_access_ops
+ * struct for DMA buffers.
+ */
+int iio_dma_buffer_write(struct iio_buffer *buffer, size_t n,
+			 const char __user *user_buffer)
+{
+	return iio_dma_buffer_io(buffer, n, (__force char *)user_buffer, true);
+}
+EXPORT_SYMBOL_GPL(iio_dma_buffer_write);
+
 /**
  * iio_dma_buffer_data_available() - DMA buffer data_available callback
  * @buf: Buffer to check for data availability
include/linux/iio/buffer-dma.h +7 -0
index a65a005c4a19..09c07d5563c0 100644
@@ -132,6 +132,8 @@ int iio_dma_buffer_disable(struct iio_buffer *buffer,
 	struct iio_dev *indio_dev);
 int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
 	char __user *user_buffer);
+int iio_dma_buffer_write(struct iio_buffer *buffer, size_t n,
+			 const char __user *user_buffer);
 size_t iio_dma_buffer_data_available(struct iio_buffer *buffer);
 int iio_dma_buffer_set_bytes_per_datum(struct iio_buffer *buffer, size_t bpd);
 int iio_dma_buffer_set_length(struct iio_buffer *buffer, unsigned int length);
@@ -142,4 +144,9 @@ int iio_dma_buffer_init(struct iio_dma_buffer_queue *queue,
 void iio_dma_buffer_exit(struct iio_dma_buffer_queue *queue);
 void iio_dma_buffer_release(struct iio_dma_buffer_queue *queue);
 
+static inline size_t iio_dma_buffer_space_available(struct iio_buffer *buffer)
+{
+	return iio_dma_buffer_data_available(buffer);
+}
+
 #endif
#22 lore raw

Re: [PATCH 04/15] iio: buffer-dma: Enable buffer write support

Hi Jonathan,

Quoted 47 lines · Jonathan Cameron
Quoted 11 lines · On Mon, 15 Nov 2021 14:19:14 +0000 Paul Cercueil <[email protected]> wrote:

Adding write support to the buffer-dma code is easy - the write() function basically needs to do the exact same thing as the read()
function: dequeue a block, read or write the data, enqueue the block when entirely processed.

Therefore, the iio_buffer_dma_read() and the new
iio_buffer_dma_write()
now both call a function iio_buffer_dma_io(), which will perform
this
task.

Quoted 7 lines

The .space_available() callback can return the exact same value as
the
.data_available() callback for input buffers, since in both cases we count the exact same thing (the number of bytes in each available block).

Hi Paul,

There are a few changes in here, such as the bytes_used value being set that I'm not following the reasoning behind. More info on those?
Also good to provide something about those in this patch description.

Thanks,

Jonathan

Quoted 15 lines

---

  drivers/iio/buffer/industrialio-buffer-dma.c | 75 
+++++++++++++++-----
  include/linux/iio/buffer-dma.h               |  7 ++
  2 files changed, 66 insertions(+), 16 deletions(-)

diff --git a/drivers/iio/buffer/industrialio-buffer-dma.c
b/drivers/iio/buffer/industrialio-buffer-dma.c
index abac88f20104..eeeed6b2e0cf 100644
--- a/drivers/iio/buffer/industrialio-buffer-dma.c +++ b/drivers/iio/buffer/industrialio-buffer-dma.c @@ -179,7 +179,8 @@ static struct iio_dma_buffer_block
*iio_dma_buffer_alloc_block(
}

Quoted 6 lines
Quoted 4 lines

block->size = size;
- block->state = IIO_BLOCK_STATE_DEQUEUED;
+ block->bytes_used = size;
+ block->state = IIO_BLOCK_STATE_DONE;

I don't know why these are here - some more info?

When using an input buffer the block->bytes_used is unconditionally reset in iio_dmaengine_buffer_submit_block(), so this was fine until now.

When using an output buffer the block->bytes_used can actually (with the new API) be specified by the user, so we don't want iio_dmaengine_buffer_submit_block() to unconditionally override it. Which means that in the case where we have an output buffer in fileio mode, we do need block->bytes_used to be initialized to the buffer's size since it won't be set anywhere else.

About the change in block->state: in patch [01/15] we removed the incoming/outgoing queues, and while the "enqueued" state is still useful to know which buffers have to be submitted when the buffer is enabled, the "dequeued" state is not useful anymore since there is no more distinction vs. the "done" state.

I believe this change should be moved to patch [01/15] then, and I should go further and remove the IIO_BLOCK_STATE_DEQUEUED completely.

Quoted 52 lines
Quoted 52 lines

block->queue = queue; INIT_LIST_HEAD(&block->head); kref_init(&block->kref);
@@ -195,6 +196,18 @@ static void _iio_dma_buffer_block_done(struct
iio_dma_buffer_block *block)
block->state = IIO_BLOCK_STATE_DONE;
}

+static void iio_dma_buffer_queue_wake(struct iio_dma_buffer_queue
*queue)
+{
+ __poll_t flags;
+
+ if (queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+ flags = EPOLLIN | EPOLLRDNORM;
+ else
+ flags = EPOLLOUT | EPOLLWRNORM;
+
+ wake_up_interruptible_poll(&queue->buffer.pollq, flags);
+}
+
/**
* iio_dma_buffer_block_done() - Indicate that a block has been
completed
* @block: The completed block
@@ -212,7 +225,7 @@ void iio_dma_buffer_block_done(struct
iio_dma_buffer_block *block)
spin_unlock_irqrestore(&queue->list_lock, flags);

iio_buffer_block_put_atomic(block);
- wake_up_interruptible_poll(&queue->buffer.pollq, EPOLLIN |
EPOLLRDNORM);
+ iio_dma_buffer_queue_wake(queue);
}
EXPORT_SYMBOL_GPL(iio_dma_buffer_block_done);

@@ -241,7 +254,7 @@ void iio_dma_buffer_block_list_abort(struct
iio_dma_buffer_queue *queue,
}
spin_unlock_irqrestore(&queue->list_lock, flags);

- wake_up_interruptible_poll(&queue->buffer.pollq, EPOLLIN |
EPOLLRDNORM);
+ iio_dma_buffer_queue_wake(queue);
} EXPORT_SYMBOL_GPL(iio_dma_buffer_block_list_abort);

@@ -334,7 +347,8 @@ int iio_dma_buffer_request_update(struct
iio_buffer *buffer)
queue->fileio.blocks[i] = block;
}

Quoted 6 lines
Quoted 3 lines

- block->state = IIO_BLOCK_STATE_QUEUED;
+ if (queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+ block->state = IIO_BLOCK_STATE_QUEUED;

Possibly worth a comment on the state being set here. I figured it out, but might save some brain cells in future if it's stated in the code.

Ok.

Quoted 42 lines
Quoted 35 lines
  	}

out_unlock:
@@ -467,20 +481,12 @@ static struct iio_dma_buffer_block
*iio_dma_buffer_dequeue(
return block;
}

-/**
- * iio_dma_buffer_read() - DMA buffer read callback
- * @buffer: Buffer to read form
- * @n: Number of bytes to read
- * @user_buffer: Userspace buffer to copy the data to
- *
- * Should be used as the read callback for iio_buffer_access_ops
- * struct for DMA buffers.
- */
-int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
- char __user *user_buffer)
+static int iio_dma_buffer_io(struct iio_buffer *buffer,
+ size_t n, char __user *user_buffer, bool is_write)
{
struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer); struct iio_dma_buffer_block *block;
+ void *addr;
int ret;

if (n < buffer->bytes_per_datum)
@@ -503,8 +509,13 @@ int iio_dma_buffer_read(struct iio_buffer
*buffer, size_t n,
n = rounddown(n, buffer->bytes_per_datum);
if (n > block->bytes_used - queue->fileio.pos)
n = block->bytes_used - queue->fileio.pos;
+ addr = block->vaddr + queue->fileio.pos;

Quoted 6 lines

- if (copy_to_user(user_buffer, block->vaddr + queue->fileio.pos,
n)) {
+ if (is_write)
+ ret = !!copy_from_user(addr, user_buffer, n);
+ else
+ ret = !!copy_to_user(user_buffer, addr, n);

Quoted 2 lines

What is the !! gaining us here? We only care about == 0 vs != 0 so forcing it to be 0 or 1 isn't useful.

Right.

Quoted 7 lines
Quoted 7 lines

+ if (ret) {
ret = -EFAULT;
goto out_unlock;
}
@@ -513,6 +524,7 @@ int iio_dma_buffer_read(struct iio_buffer
*buffer, size_t n,

Quoted 5 lines
Quoted 3 lines

if (queue->fileio.pos == block->bytes_used) {
queue->fileio.active_block = NULL;
+ block->bytes_used = block->size;

This seems to be a functional change that isn't called out in the patch description.

See the explanation above. Although I most likely don't need to set it at two different spots... I'll check that in detail next week.

Cheers,
-Paul

Quoted 76 lines
Quoted 75 lines
  		iio_dma_buffer_enqueue(queue, block);
  	}

@@ -523,8 +535,39 @@ int iio_dma_buffer_read(struct iio_buffer
*buffer, size_t n,

return ret;
}
+
+/**
+ * iio_dma_buffer_read() - DMA buffer read callback
+ * @buffer: Buffer to read form
+ * @n: Number of bytes to read
+ * @user_buffer: Userspace buffer to copy the data to
+ *
+ * Should be used as the read callback for iio_buffer_access_ops
+ * struct for DMA buffers.
+ */
+int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
+ char __user *user_buffer)
+{
+ return iio_dma_buffer_io(buffer, n, user_buffer, false);
+}
EXPORT_SYMBOL_GPL(iio_dma_buffer_read);

+/**
+ * iio_dma_buffer_write() - DMA buffer write callback
+ * @buffer: Buffer to read form
+ * @n: Number of bytes to read
+ * @user_buffer: Userspace buffer to copy the data from
+ *
+ * Should be used as the write callback for iio_buffer_access_ops
+ * struct for DMA buffers.
+ */
+int iio_dma_buffer_write(struct iio_buffer *buffer, size_t n,
+ const char __user *user_buffer)
+{
+ return iio_dma_buffer_io(buffer, n, (__force char *)user_buffer,
true);
+}
+EXPORT_SYMBOL_GPL(iio_dma_buffer_write);
+
/**
* iio_dma_buffer_data_available() - DMA buffer data_available
callback
* @buf: Buffer to check for data availability
diff --git a/include/linux/iio/buffer-dma.h
b/include/linux/iio/buffer-dma.h
index a65a005c4a19..09c07d5563c0 100644
--- a/include/linux/iio/buffer-dma.h
+++ b/include/linux/iio/buffer-dma.h
@@ -132,6 +132,8 @@ int iio_dma_buffer_disable(struct iio_buffer
*buffer,
struct iio_dev *indio_dev);
int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
char __user *user_buffer);
+int iio_dma_buffer_write(struct iio_buffer *buffer, size_t n,
+ const char __user *user_buffer);
size_t iio_dma_buffer_data_available(struct iio_buffer *buffer); int iio_dma_buffer_set_bytes_per_datum(struct iio_buffer *buffer,
size_t bpd);
int iio_dma_buffer_set_length(struct iio_buffer *buffer, unsigned
int length);
@@ -142,4 +144,9 @@ int iio_dma_buffer_init(struct
iio_dma_buffer_queue *queue,
void iio_dma_buffer_exit(struct iio_dma_buffer_queue *queue); void iio_dma_buffer_release(struct iio_dma_buffer_queue *queue);

+static inline size_t iio_dma_buffer_space_available(struct
iio_buffer *buffer)
+{
+ return iio_dma_buffer_data_available(buffer);
+}
+
#endif

maintainer
#23 lore raw

Re: [PATCH 04/15] iio: buffer-dma: Enable buffer write support

Quoted 63 lines · On Sun, 21 Nov 2021 17:19:32 +0000 Paul Cercueil <[email protected]> wrote:

Hi Jonathan,

Quoted 48 lines · Jonathan Cameron
Quoted 11 lines · On Mon, 15 Nov 2021 14:19:14 +0000 Paul Cercueil <[email protected]> wrote:

Adding write support to the buffer-dma code is easy - the write() function basically needs to do the exact same thing as the read()
function: dequeue a block, read or write the data, enqueue the block when entirely processed.

Therefore, the iio_buffer_dma_read() and the new
iio_buffer_dma_write()
now both call a function iio_buffer_dma_io(), which will perform
this
task.

Quoted 7 lines

The .space_available() callback can return the exact same value as
the
.data_available() callback for input buffers, since in both cases we count the exact same thing (the number of bytes in each available block).

Hi Paul,

There are a few changes in here, such as the bytes_used value being set that
I'm not following the reasoning behind. More info on those? Also good to provide something about those in this patch description.

Thanks,

Jonathan

Quoted 15 lines

---

  drivers/iio/buffer/industrialio-buffer-dma.c | 75 
+++++++++++++++-----
  include/linux/iio/buffer-dma.h               |  7 ++
  2 files changed, 66 insertions(+), 16 deletions(-)

diff --git a/drivers/iio/buffer/industrialio-buffer-dma.c
b/drivers/iio/buffer/industrialio-buffer-dma.c
index abac88f20104..eeeed6b2e0cf 100644
--- a/drivers/iio/buffer/industrialio-buffer-dma.c +++ b/drivers/iio/buffer/industrialio-buffer-dma.c @@ -179,7 +179,8 @@ static struct iio_dma_buffer_block
*iio_dma_buffer_alloc_block(
}

Quoted 6 lines
Quoted 4 lines

block->size = size;
- block->state = IIO_BLOCK_STATE_DEQUEUED;
+ block->bytes_used = size;
+ block->state = IIO_BLOCK_STATE_DONE;

I don't know why these are here - some more info?

When using an input buffer the block->bytes_used is unconditionally reset in iio_dmaengine_buffer_submit_block(), so this was fine until now.

Quoted 6 lines

When using an output buffer the block->bytes_used can actually (with the new API) be specified by the user, so we don't want iio_dmaengine_buffer_submit_block() to unconditionally override it. Which means that in the case where we have an output buffer in fileio mode, we do need block->bytes_used to be initialized to the buffer's size since it won't be set anywhere else.

Makes sense. Thanks for the explanation.

Quoted 9 lines

About the change in block->state: in patch [01/15] we removed the incoming/outgoing queues, and while the "enqueued" state is still useful to know which buffers have to be submitted when the buffer is enabled, the "dequeued" state is not useful anymore since there is no more distinction vs. the "done" state.

I believe this change should be moved to patch [01/15] then, and I should go further and remove the IIO_BLOCK_STATE_DEQUEUED completely.

Complete removal would indeed be a more 'obvious' change, so I'd support that assuming now disadvantages we havent' thought of yet.

Quoted 208 lines
Quoted 53 lines
Quoted 53 lines

block->queue = queue; INIT_LIST_HEAD(&block->head); kref_init(&block->kref);
@@ -195,6 +196,18 @@ static void _iio_dma_buffer_block_done(struct
iio_dma_buffer_block *block)
block->state = IIO_BLOCK_STATE_DONE;
}

+static void iio_dma_buffer_queue_wake(struct iio_dma_buffer_queue
*queue)
+{
+ __poll_t flags;
+
+ if (queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+ flags = EPOLLIN | EPOLLRDNORM;
+ else
+ flags = EPOLLOUT | EPOLLWRNORM;
+
+ wake_up_interruptible_poll(&queue->buffer.pollq, flags);
+}
+
/**
* iio_dma_buffer_block_done() - Indicate that a block has been
completed
* @block: The completed block
@@ -212,7 +225,7 @@ void iio_dma_buffer_block_done(struct
iio_dma_buffer_block *block)
spin_unlock_irqrestore(&queue->list_lock, flags);

iio_buffer_block_put_atomic(block);
- wake_up_interruptible_poll(&queue->buffer.pollq, EPOLLIN |
EPOLLRDNORM);
+ iio_dma_buffer_queue_wake(queue);
}
EXPORT_SYMBOL_GPL(iio_dma_buffer_block_done);

@@ -241,7 +254,7 @@ void iio_dma_buffer_block_list_abort(struct
iio_dma_buffer_queue *queue,
}
spin_unlock_irqrestore(&queue->list_lock, flags);

- wake_up_interruptible_poll(&queue->buffer.pollq, EPOLLIN |
EPOLLRDNORM);
+ iio_dma_buffer_queue_wake(queue);
} EXPORT_SYMBOL_GPL(iio_dma_buffer_block_list_abort);

@@ -334,7 +347,8 @@ int iio_dma_buffer_request_update(struct
iio_buffer *buffer)
queue->fileio.blocks[i] = block;
}

Quoted 7 lines
Quoted 3 lines

- block->state = IIO_BLOCK_STATE_QUEUED;
+ if (queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+ block->state = IIO_BLOCK_STATE_QUEUED;

Possibly worth a comment on the state being set here. I figured it out, but might
save some brain cells in future if it's stated in the code.

Ok.

Quoted 42 lines
Quoted 35 lines
  	}

out_unlock:
@@ -467,20 +481,12 @@ static struct iio_dma_buffer_block
*iio_dma_buffer_dequeue(
return block;
}

-/**
- * iio_dma_buffer_read() - DMA buffer read callback
- * @buffer: Buffer to read form
- * @n: Number of bytes to read
- * @user_buffer: Userspace buffer to copy the data to
- *
- * Should be used as the read callback for iio_buffer_access_ops
- * struct for DMA buffers.
- */
-int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
- char __user *user_buffer)
+static int iio_dma_buffer_io(struct iio_buffer *buffer,
+ size_t n, char __user *user_buffer, bool is_write)
{
struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer); struct iio_dma_buffer_block *block;
+ void *addr;
int ret;

if (n < buffer->bytes_per_datum)
@@ -503,8 +509,13 @@ int iio_dma_buffer_read(struct iio_buffer
*buffer, size_t n,
n = rounddown(n, buffer->bytes_per_datum);
if (n > block->bytes_used - queue->fileio.pos)
n = block->bytes_used - queue->fileio.pos;
+ addr = block->vaddr + queue->fileio.pos;

Quoted 6 lines

- if (copy_to_user(user_buffer, block->vaddr + queue->fileio.pos,
n)) {
+ if (is_write)
+ ret = !!copy_from_user(addr, user_buffer, n);
+ else
+ ret = !!copy_to_user(user_buffer, addr, n);

Quoted 2 lines

What is the !! gaining us here? We only care about == 0 vs != 0 so forcing it to be 0 or 1 isn't useful.

Right.

Quoted 7 lines
Quoted 7 lines

+ if (ret) {
ret = -EFAULT;
goto out_unlock;
}
@@ -513,6 +524,7 @@ int iio_dma_buffer_read(struct iio_buffer
*buffer, size_t n,

Quoted 6 lines
Quoted 3 lines

if (queue->fileio.pos == block->bytes_used) {
queue->fileio.active_block = NULL;
+ block->bytes_used = block->size;

This seems to be a functional change that isn't called out in the patch description.

See the explanation above. Although I most likely don't need to set it at two different spots... I'll check that in detail next week.

Cheers,
-Paul

Quoted 76 lines
Quoted 75 lines
  		iio_dma_buffer_enqueue(queue, block);
  	}

@@ -523,8 +535,39 @@ int iio_dma_buffer_read(struct iio_buffer
*buffer, size_t n,

return ret;
}
+
+/**
+ * iio_dma_buffer_read() - DMA buffer read callback
+ * @buffer: Buffer to read form
+ * @n: Number of bytes to read
+ * @user_buffer: Userspace buffer to copy the data to
+ *
+ * Should be used as the read callback for iio_buffer_access_ops
+ * struct for DMA buffers.
+ */
+int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
+ char __user *user_buffer)
+{
+ return iio_dma_buffer_io(buffer, n, user_buffer, false);
+}
EXPORT_SYMBOL_GPL(iio_dma_buffer_read);

+/**
+ * iio_dma_buffer_write() - DMA buffer write callback
+ * @buffer: Buffer to read form
+ * @n: Number of bytes to read
+ * @user_buffer: Userspace buffer to copy the data from
+ *
+ * Should be used as the write callback for iio_buffer_access_ops
+ * struct for DMA buffers.
+ */
+int iio_dma_buffer_write(struct iio_buffer *buffer, size_t n,
+ const char __user *user_buffer)
+{
+ return iio_dma_buffer_io(buffer, n, (__force char *)user_buffer,
true);
+}
+EXPORT_SYMBOL_GPL(iio_dma_buffer_write);
+
/**
* iio_dma_buffer_data_available() - DMA buffer data_available
callback
* @buf: Buffer to check for data availability
diff --git a/include/linux/iio/buffer-dma.h
b/include/linux/iio/buffer-dma.h
index a65a005c4a19..09c07d5563c0 100644
--- a/include/linux/iio/buffer-dma.h
+++ b/include/linux/iio/buffer-dma.h
@@ -132,6 +132,8 @@ int iio_dma_buffer_disable(struct iio_buffer
*buffer,
struct iio_dev *indio_dev);
int iio_dma_buffer_read(struct iio_buffer *buffer, size_t n,
char __user *user_buffer);
+int iio_dma_buffer_write(struct iio_buffer *buffer, size_t n,
+ const char __user *user_buffer);
size_t iio_dma_buffer_data_available(struct iio_buffer *buffer); int iio_dma_buffer_set_bytes_per_datum(struct iio_buffer *buffer,
size_t bpd);
int iio_dma_buffer_set_length(struct iio_buffer *buffer, unsigned
int length);
@@ -142,4 +144,9 @@ int iio_dma_buffer_init(struct
iio_dma_buffer_queue *queue,
void iio_dma_buffer_exit(struct iio_dma_buffer_queue *queue); void iio_dma_buffer_release(struct iio_dma_buffer_queue *queue);

+static inline size_t iio_dma_buffer_space_available(struct
iio_buffer *buffer)
+{
+ return iio_dma_buffer_data_available(buffer);
+}
+
#endif

#24 lore raw

PATCH 5/15 iio: buffer-dmaengine: Support specifying buffer direction

In reply to Paul Cercueil

Update the devm_iio_dmaengine_buffer_setup() function to support specifying the buffer direction.

Update the iio_dmaengine_buffer_submit() function to handle input buffers as well as output buffers.

---
 drivers/iio/adc/adi-axi-adc.c                 |  3 ++-
 .../buffer/industrialio-buffer-dmaengine.c    | 24 +++++++++++++++----
 include/linux/iio/buffer-dmaengine.h          |  5 +++-
 3 files changed, 25 insertions(+), 7 deletions(-)
drivers/iio/adc/adi-axi-adc.c +2 -1
index a73e3c2d212f..0a6f2c32b1b9 100644
@@ -113,7 +113,8 @@ static int adi_axi_adc_config_dma_buffer(struct device *dev,
 		dma_name = "rx";
 
 	return devm_iio_dmaengine_buffer_setup(indio_dev->dev.parent,
-					       indio_dev, dma_name);
+					       indio_dev, dma_name,
+					       IIO_BUFFER_DIRECTION_IN);
 }
 
 static int adi_axi_adc_read_raw(struct iio_dev *indio_dev,
drivers/iio/buffer/industrialio-buffer-dmaengine.c +19 -5
index f8ce26a24c57..ac26b04aa4a9 100644
@@ -64,14 +64,25 @@ static int iio_dmaengine_buffer_submit_block(struct iio_dma_buffer_queue *queue,
 	struct dmaengine_buffer *dmaengine_buffer =
 		iio_buffer_to_dmaengine_buffer(&queue->buffer);
 	struct dma_async_tx_descriptor *desc;
+	enum dma_transfer_direction dma_dir;
+	size_t max_size;
 	dma_cookie_t cookie;
 
-	block->bytes_used = min(block->size, dmaengine_buffer->max_size);
-	block->bytes_used = round_down(block->bytes_used,
-			dmaengine_buffer->align);
+	max_size = min(block->size, dmaengine_buffer->max_size);
+	max_size = round_down(max_size, dmaengine_buffer->align);
+
+	if (queue->buffer.direction == IIO_BUFFER_DIRECTION_IN) {
+		block->bytes_used = max_size;
+		dma_dir = DMA_DEV_TO_MEM;
+	} else {
+		dma_dir = DMA_MEM_TO_DEV;
+	}
+
+	if (!block->bytes_used || block->bytes_used > max_size)
+		return -EINVAL;
 
 	desc = dmaengine_prep_slave_single(dmaengine_buffer->chan,
-		block->phys_addr, block->bytes_used, DMA_DEV_TO_MEM,
+		block->phys_addr, block->bytes_used, dma_dir,
 		DMA_PREP_INTERRUPT);
 	if (!desc)
 		return -ENOMEM;
@@ -275,7 +286,8 @@ static struct iio_buffer *devm_iio_dmaengine_buffer_alloc(struct device *dev,
  */
 int devm_iio_dmaengine_buffer_setup(struct device *dev,
 				    struct iio_dev *indio_dev,
-				    const char *channel)
+				    const char *channel,
+				    enum iio_buffer_direction dir)
 {
 	struct iio_buffer *buffer;
 
@@ -286,6 +298,8 @@ int devm_iio_dmaengine_buffer_setup(struct device *dev,
 
 	indio_dev->modes |= INDIO_BUFFER_HARDWARE;
 
+	buffer->direction = dir;
+
 	return iio_device_attach_buffer(indio_dev, buffer);
 }
 EXPORT_SYMBOL_GPL(devm_iio_dmaengine_buffer_setup);
include/linux/iio/buffer-dmaengine.h +4 -1
index 5c355be89814..538d0479cdd6 100644
@@ -7,11 +7,14 @@
 #ifndef __IIO_DMAENGINE_H__
 #define __IIO_DMAENGINE_H__
 
+#include <linux/iio/buffer.h>
+
 struct iio_dev;
 struct device;
 
 int devm_iio_dmaengine_buffer_setup(struct device *dev,
 				    struct iio_dev *indio_dev,
-				    const char *channel);
+				    const char *channel,
+				    enum iio_buffer_direction dir);
 
 #endif
Signature · 1 line
2.33.0
Reviewed-by on v1 5/15
#25 lore raw

Re: [PATCH 05/15] iio: buffer-dmaengine: Support specifying buffer direction

Quoted 7 lines · Paul Cercueil

Update the devm_iio_dmaengine_buffer_setup() function to support specifying the buffer direction.

Update the iio_dmaengine_buffer_submit() function to handle input buffers as well as output buffers.

Quoted 97 lines
---
 drivers/iio/adc/adi-axi-adc.c                 |  3 ++-
 .../buffer/industrialio-buffer-dmaengine.c    | 24 +++++++++++++++----
 include/linux/iio/buffer-dmaengine.h          |  5 +++-
 3 files changed, 25 insertions(+), 7 deletions(-)
drivers/iio/adc/adi-axi-adc.c +2 -1
index a73e3c2d212f..0a6f2c32b1b9 100644
@@ -113,7 +113,8 @@ static int adi_axi_adc_config_dma_buffer(struct device *dev,
                dma_name = "rx";
 
        return devm_iio_dmaengine_buffer_setup(indio_dev->dev.parent,
-                                              indio_dev, dma_name);
+                                              indio_dev, dma_name,
+                                              IIO_BUFFER_DIRECTION_IN);
 }
 
 static int adi_axi_adc_read_raw(struct iio_dev *indio_dev,
drivers/iio/buffer/industrialio-buffer-dmaengine.c +19 -5
index f8ce26a24c57..ac26b04aa4a9 100644
@@ -64,14 +64,25 @@ static int iio_dmaengine_buffer_submit_block(struct iio_dma_buffer_queue *queue,
        struct dmaengine_buffer *dmaengine_buffer =
                iio_buffer_to_dmaengine_buffer(&queue->buffer);
        struct dma_async_tx_descriptor *desc;
+       enum dma_transfer_direction dma_dir;
+       size_t max_size;
        dma_cookie_t cookie;
 
-       block->bytes_used = min(block->size, dmaengine_buffer->max_size);
-       block->bytes_used = round_down(block->bytes_used,
-                       dmaengine_buffer->align);
+       max_size = min(block->size, dmaengine_buffer->max_size);
+       max_size = round_down(max_size, dmaengine_buffer->align);
+
+       if (queue->buffer.direction == IIO_BUFFER_DIRECTION_IN) {
+               block->bytes_used = max_size;
+               dma_dir = DMA_DEV_TO_MEM;
+       } else {
+               dma_dir = DMA_MEM_TO_DEV;
+       }
+
+       if (!block->bytes_used || block->bytes_used > max_size)
+               return -EINVAL;
 
        desc = dmaengine_prep_slave_single(dmaengine_buffer->chan,
-               block->phys_addr, block->bytes_used, DMA_DEV_TO_MEM,
+               block->phys_addr, block->bytes_used, dma_dir,
                DMA_PREP_INTERRUPT);
        if (!desc)
                return -ENOMEM;
@@ -275,7 +286,8 @@ static struct iio_buffer *devm_iio_dmaengine_buffer_alloc(struct device *dev,
  */
 int devm_iio_dmaengine_buffer_setup(struct device *dev,
                                    struct iio_dev *indio_dev,
-                                   const char *channel)
+                                   const char *channel,
+                                   enum iio_buffer_direction dir)
 {
        struct iio_buffer *buffer;
 
@@ -286,6 +298,8 @@ int devm_iio_dmaengine_buffer_setup(struct device *dev,
 
        indio_dev->modes |= INDIO_BUFFER_HARDWARE;
 
+       buffer->direction = dir;
+
        return iio_device_attach_buffer(indio_dev, buffer);
 }
 EXPORT_SYMBOL_GPL(devm_iio_dmaengine_buffer_setup);
include/linux/iio/buffer-dmaengine.h +4 -2
index 5c355be89814..538d0479cdd6 100644
@@ -7,11 +7,14 @@
 #ifndef __IIO_DMAENGINE_H__
 #define __IIO_DMAENGINE_H__
 
+#include <linux/iio/buffer.h>
+
 struct iio_dev;
 struct device;
 
 int devm_iio_dmaengine_buffer_setup(struct device *dev,
                                    struct iio_dev *indio_dev,
-                                   const char *channel);
+                                   const char *channel,
+                                   enum iio_buffer_direction dir);
 
 #endif
--

2.33.0

#26 lore raw

PATCH 6/15 iio: buffer-dmaengine: Enable write support

In reply to Paul Cercueil

Use the iio_dma_buffer_write() and iio_dma_buffer_space_available() functions provided by the buffer-dma core, to enable write support in the buffer-dmaengine code.

---
 drivers/iio/buffer/industrialio-buffer-dmaengine.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/iio/buffer/industrialio-buffer-dmaengine.c +2 -0
index ac26b04aa4a9..5cde8fd81c7f 100644
@@ -123,12 +123,14 @@ static void iio_dmaengine_buffer_release(struct iio_buffer *buf)
 
 static const struct iio_buffer_access_funcs iio_dmaengine_buffer_ops = {
 	.read = iio_dma_buffer_read,
+	.write = iio_dma_buffer_write,
 	.set_bytes_per_datum = iio_dma_buffer_set_bytes_per_datum,
 	.set_length = iio_dma_buffer_set_length,
 	.request_update = iio_dma_buffer_request_update,
 	.enable = iio_dma_buffer_enable,
 	.disable = iio_dma_buffer_disable,
 	.data_available = iio_dma_buffer_data_available,
+	.space_available = iio_dma_buffer_space_available,
 	.release = iio_dmaengine_buffer_release,
 
 	.modes = INDIO_BUFFER_HARDWARE,
Signature · 1 line
2.33.0
Reviewed-by on v1 6/15
#27 lore raw

Re: [PATCH 06/15] iio: buffer-dmaengine: Enable write support

Quoted 5 lines · Paul Cercueil

Use the iio_dma_buffer_write() and iio_dma_buffer_space_available() functions provided by the buffer-dma core, to enable write support in the buffer-dmaengine code.

This is a bit related to the comment about
iio_dma_buffer_space_available() in patch 4

But otherwise:

Quoted 27 lines
---
 drivers/iio/buffer/industrialio-buffer-dmaengine.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/iio/buffer/industrialio-buffer-dmaengine.c +2 -1
index ac26b04aa4a9..5cde8fd81c7f 100644
@@ -123,12 +123,14 @@ static void iio_dmaengine_buffer_release(struct iio_buffer *buf)
 
 static const struct iio_buffer_access_funcs iio_dmaengine_buffer_ops = {
        .read = iio_dma_buffer_read,
+       .write = iio_dma_buffer_write,
        .set_bytes_per_datum = iio_dma_buffer_set_bytes_per_datum,
        .set_length = iio_dma_buffer_set_length,
        .request_update = iio_dma_buffer_request_update,
        .enable = iio_dma_buffer_enable,
        .disable = iio_dma_buffer_disable,
        .data_available = iio_dma_buffer_data_available,
+       .space_available = iio_dma_buffer_space_available,
        .release = iio_dmaengine_buffer_release,
 
        .modes = INDIO_BUFFER_HARDWARE,
--

2.33.0

#28 lore raw

PATCH 7/15 iio: core: Add new DMABUF interface infrastructure

In reply to Paul Cercueil

Add the necessary infrastructure to the IIO core to support a new DMABUF based interface.

The advantage of this new DMABUF based interface vs. the read() interface, is that it avoids an extra copy of the data between the kernel and userspace. This is particularly userful for high-speed devices which produce several megabytes or even gigabytes of data per second.

The data in this new DMABUF interface is managed at the granularity of DMABUF objects. Reducing the granularity from byte level to block level is done to reduce the userspace-kernelspace synchronization overhead since performing syscalls for each byte at a few Mbps is just not feasible.

This of course leads to a slightly increased latency. For this reason an application can choose the size of the DMABUFs as well as how many it allocates. E.g. two DMABUFs would be a traditional double buffering scheme. But using a higher number might be necessary to avoid underflow/overflow situations in the presence of scheduling latencies.

As part of the interface, 2 new IOCTLs have been added:

IIO_BUFFER_DMABUF_ALLOC_IOCTL(struct iio_dmabuf_alloc_req *):
Each call will allocate a new DMABUF object. The return value (if not a negative errno value as error) will be the file descriptor of the new DMABUF.

IIO_BUFFER_DMABUF_ENQUEUE_IOCTL(struct iio_dmabuf *):
Place the DMABUF object into the queue pending for hardware process.

These two IOCTLs have to be performed on the IIO buffer's file descriptor (either opened from the corresponding /dev/iio:deviceX, or obtained using the IIO_BUFFER_GET_FD_IOCTL() ioctl).

To access the data stored in a block by userspace the block must be mapped to the process's memory. This is done by calling mmap() on the DMABUF's file descriptor.

Before accessing the data through the map, you must use the DMA_BUF_IOCTL_SYNC(struct dma_buf_sync *) ioctl, with the DMA_BUF_SYNC_START flag, to make sure that the data is available. This call may block until the hardware is done with this block. Once you are done reading or writing the data, you must use this ioctl again with the DMA_BUF_SYNC_END flag, before enqueueing the DMABUF to the kernel's queue.

If you need to know when the hardware is done with a DMABUF, you can poll its file descriptor for the EPOLLOUT event.

Finally, to destroy a DMABUF object, simply call close() on its file descriptor.

A typical workflow for the new interface is:

for block in blocks:
DMABUF_ALLOC block
mmap block

enable buffer

while !done
for block in blocks:
DMABUF_ENQUEUE block

DMABUF_SYNC_START block
process data
DMABUF_SYNC_END block

disable buffer

for block in blocks:
close block

---
 drivers/iio/industrialio-buffer.c | 44 +++++++++++++++++++++++++++++++
 include/linux/iio/buffer_impl.h   |  8 ++++++
 include/uapi/linux/iio/buffer.h   | 29 ++++++++++++++++++++
 3 files changed, 81 insertions(+)
drivers/iio/industrialio-buffer.c +44 -0
index e180728914c0..30910e6c2346 100644
@@ -17,6 +17,7 @@
 #include <linux/fs.h>
 #include <linux/cdev.h>
 #include <linux/slab.h>
+#include <linux/mm.h>
 #include <linux/poll.h>
 #include <linux/sched/signal.h>
 
@@ -1585,12 +1586,55 @@ static long iio_device_buffer_getfd(struct iio_dev *indio_dev, unsigned long arg
 	return ret;
 }
 
+static int iio_buffer_enqueue_dmabuf(struct iio_buffer *buffer,
+				     struct iio_dmabuf __user *user_buf)
+{
+	struct iio_dmabuf dmabuf;
+
+	if (!buffer->access->enqueue_dmabuf)
+		return -EPERM;
+
+	if (copy_from_user(&dmabuf, user_buf, sizeof(dmabuf)))
+		return -EFAULT;
+
+	if (dmabuf.flags & ~IIO_BUFFER_DMABUF_SUPPORTED_FLAGS)
+		return -EINVAL;
+
+	return buffer->access->enqueue_dmabuf(buffer, &dmabuf);
+}
+
+static int iio_buffer_alloc_dmabuf(struct iio_buffer *buffer,
+				   struct iio_dmabuf_alloc_req __user *user_req)
+{
+	struct iio_dmabuf_alloc_req req;
+
+	if (!buffer->access->alloc_dmabuf)
+		return -EPERM;
+
+	if (copy_from_user(&req, user_req, sizeof(req)))
+		return -EFAULT;
+
+	if (req.resv)
+		return -EINVAL;
+
+	return buffer->access->alloc_dmabuf(buffer, &req);
+}
+
 static long iio_device_buffer_ioctl(struct iio_dev *indio_dev, struct file *filp,
 				    unsigned int cmd, unsigned long arg)
 {
+	struct iio_dev_buffer_pair *ib = filp->private_data;
+	struct iio_buffer *buffer = ib->buffer;
+	void __user *_arg = (void __user *)arg;
+
 	switch (cmd) {
 	case IIO_BUFFER_GET_FD_IOCTL:
 		return iio_device_buffer_getfd(indio_dev, arg);
+	case IIO_BUFFER_DMABUF_ALLOC_IOCTL:
+		return iio_buffer_alloc_dmabuf(buffer, _arg);
+	case IIO_BUFFER_DMABUF_ENQUEUE_IOCTL:
+		/* TODO: support non-blocking enqueue operation */
+		return iio_buffer_enqueue_dmabuf(buffer, _arg);
 	default:
 		return IIO_IOCTL_UNHANDLED;
 	}
include/linux/iio/buffer_impl.h +8 -0
index e2ca8ea23e19..728541bc2c63 100644
@@ -39,6 +39,9 @@ struct iio_buffer;
  *                      device stops sampling. Calles are balanced with @enable.
  * @release:		called when the last reference to the buffer is dropped,
  *			should free all resources allocated by the buffer.
+ * @alloc_dmabuf:	called from userspace via ioctl to allocate one DMABUF.
+ * @enqueue_dmabuf:	called from userspace via ioctl to queue this DMABUF
+ *			object to this buffer. Requires a valid DMABUF fd.
  * @modes:		Supported operating modes by this buffer type
  * @flags:		A bitmask combination of INDIO_BUFFER_FLAG_*
  *
@@ -68,6 +71,11 @@ struct iio_buffer_access_funcs {
 
 	void (*release)(struct iio_buffer *buffer);
 
+	int (*alloc_dmabuf)(struct iio_buffer *buffer,
+			    struct iio_dmabuf_alloc_req *req);
+	int (*enqueue_dmabuf)(struct iio_buffer *buffer,
+			      struct iio_dmabuf *block);
+
 	unsigned int modes;
 	unsigned int flags;
 };
include/uapi/linux/iio/buffer.h +29 -0
index 13939032b3f6..e4621b926262 100644
@@ -5,6 +5,35 @@
 #ifndef _UAPI_IIO_BUFFER_H_
 #define _UAPI_IIO_BUFFER_H_
 
+#include <linux/types.h>
+
+#define IIO_BUFFER_DMABUF_SUPPORTED_FLAGS	0x00000000
+
+/**
+ * struct iio_dmabuf_alloc_req - Descriptor for allocating IIO DMABUFs
+ * @size:	the size of a single DMABUF
+ * @resv:	reserved
+ */
+struct iio_dmabuf_alloc_req {
+	__u64 size;
+	__u64 resv;
+};
+
+/**
+ * struct iio_dmabuf - Descriptor for a single IIO DMABUF object
+ * @fd:		file descriptor of the DMABUF object
+ * @flags:	one or more IIO_BUFFER_DMABUF_* flags
+ * @bytes_used:	number of bytes used in this DMABUF for the data transfer.
+ *		If zero, the full buffer is used.
+ */
+struct iio_dmabuf {
+	__u32 fd;
+	__u32 flags;
+	__u64 bytes_used;
+};
+
 #define IIO_BUFFER_GET_FD_IOCTL			_IOWR('i', 0x91, int)
+#define IIO_BUFFER_DMABUF_ALLOC_IOCTL		_IOW('i', 0x92, struct iio_dmabuf_alloc_req)
+#define IIO_BUFFER_DMABUF_ENQUEUE_IOCTL		_IOW('i', 0x93, struct iio_dmabuf)
 
 #endif /* _UAPI_IIO_BUFFER_H_ */
Signature · 1 line
2.33.0
maintainer
#29 lore raw

Re: [PATCH 07/15] iio: core: Add new DMABUF interface infrastructure

Quoted 28 lines · On Mon, 15 Nov 2021 14:19:17 +0000 Paul Cercueil <[email protected]> wrote:

Add the necessary infrastructure to the IIO core to support a new DMABUF based interface.

The advantage of this new DMABUF based interface vs. the read() interface, is that it avoids an extra copy of the data between the kernel and userspace. This is particularly userful for high-speed devices which produce several megabytes or even gigabytes of data per second.

The data in this new DMABUF interface is managed at the granularity of DMABUF objects. Reducing the granularity from byte level to block level is done to reduce the userspace-kernelspace synchronization overhead since performing syscalls for each byte at a few Mbps is just not feasible.

This of course leads to a slightly increased latency. For this reason an application can choose the size of the DMABUFs as well as how many it allocates. E.g. two DMABUFs would be a traditional double buffering scheme. But using a higher number might be necessary to avoid underflow/overflow situations in the presence of scheduling latencies.

As part of the interface, 2 new IOCTLs have been added:

IIO_BUFFER_DMABUF_ALLOC_IOCTL(struct iio_dmabuf_alloc_req *):
Each call will allocate a new DMABUF object. The return value (if not a negative errno value as error) will be the file descriptor of the new DMABUF.

Quoted 6 lines

IIO_BUFFER_DMABUF_ENQUEUE_IOCTL(struct iio_dmabuf *):
Place the DMABUF object into the queue pending for hardware process.

These two IOCTLs have to be performed on the IIO buffer's file descriptor (either opened from the corresponding /dev/iio:deviceX, or obtained using the IIO_BUFFER_GET_FD_IOCTL() ioctl).

Unrelated to this patch except tangentially. Maybe we should enable new buffer features only on the IIO_BUFFER_GET_FD_IOCTL() route as we probably want to deprecate the old interfaces due to the it only supporting a single buffer / datastream per device.

Possibly something for another day...

Nothing to add on actual code...

Jonathan

Quoted 181 lines

To access the data stored in a block by userspace the block must be mapped to the process's memory. This is done by calling mmap() on the DMABUF's file descriptor.

Before accessing the data through the map, you must use the DMA_BUF_IOCTL_SYNC(struct dma_buf_sync *) ioctl, with the DMA_BUF_SYNC_START flag, to make sure that the data is available. This call may block until the hardware is done with this block. Once you are done reading or writing the data, you must use this ioctl again with the DMA_BUF_SYNC_END flag, before enqueueing the DMABUF to the kernel's queue.

If you need to know when the hardware is done with a DMABUF, you can poll its file descriptor for the EPOLLOUT event.

Finally, to destroy a DMABUF object, simply call close() on its file descriptor.

A typical workflow for the new interface is:

for block in blocks:
DMABUF_ALLOC block
mmap block

enable buffer

while !done
for block in blocks:
DMABUF_ENQUEUE block

DMABUF_SYNC_START block
process data
DMABUF_SYNC_END block

disable buffer

for block in blocks:
close block

---
 drivers/iio/industrialio-buffer.c | 44 +++++++++++++++++++++++++++++++
 include/linux/iio/buffer_impl.h   |  8 ++++++
 include/uapi/linux/iio/buffer.h   | 29 ++++++++++++++++++++
 3 files changed, 81 insertions(+)
drivers/iio/industrialio-buffer.c +44 -0
index e180728914c0..30910e6c2346 100644
@@ -17,6 +17,7 @@
 #include <linux/fs.h>
 #include <linux/cdev.h>
 #include <linux/slab.h>
+#include <linux/mm.h>
 #include <linux/poll.h>
 #include <linux/sched/signal.h>
 
@@ -1585,12 +1586,55 @@ static long iio_device_buffer_getfd(struct iio_dev *indio_dev, unsigned long arg
 	return ret;
 }
 
+static int iio_buffer_enqueue_dmabuf(struct iio_buffer *buffer,
+				     struct iio_dmabuf __user *user_buf)
+{
+	struct iio_dmabuf dmabuf;
+
+	if (!buffer->access->enqueue_dmabuf)
+		return -EPERM;
+
+	if (copy_from_user(&dmabuf, user_buf, sizeof(dmabuf)))
+		return -EFAULT;
+
+	if (dmabuf.flags & ~IIO_BUFFER_DMABUF_SUPPORTED_FLAGS)
+		return -EINVAL;
+
+	return buffer->access->enqueue_dmabuf(buffer, &dmabuf);
+}
+
+static int iio_buffer_alloc_dmabuf(struct iio_buffer *buffer,
+				   struct iio_dmabuf_alloc_req __user *user_req)
+{
+	struct iio_dmabuf_alloc_req req;
+
+	if (!buffer->access->alloc_dmabuf)
+		return -EPERM;
+
+	if (copy_from_user(&req, user_req, sizeof(req)))
+		return -EFAULT;
+
+	if (req.resv)
+		return -EINVAL;
+
+	return buffer->access->alloc_dmabuf(buffer, &req);
+}
+
 static long iio_device_buffer_ioctl(struct iio_dev *indio_dev, struct file *filp,
 				    unsigned int cmd, unsigned long arg)
 {
+	struct iio_dev_buffer_pair *ib = filp->private_data;
+	struct iio_buffer *buffer = ib->buffer;
+	void __user *_arg = (void __user *)arg;
+
 	switch (cmd) {
 	case IIO_BUFFER_GET_FD_IOCTL:
 		return iio_device_buffer_getfd(indio_dev, arg);
+	case IIO_BUFFER_DMABUF_ALLOC_IOCTL:
+		return iio_buffer_alloc_dmabuf(buffer, _arg);
+	case IIO_BUFFER_DMABUF_ENQUEUE_IOCTL:
+		/* TODO: support non-blocking enqueue operation */
+		return iio_buffer_enqueue_dmabuf(buffer, _arg);
 	default:
 		return IIO_IOCTL_UNHANDLED;
 	}
include/linux/iio/buffer_impl.h +8 -0
index e2ca8ea23e19..728541bc2c63 100644
@@ -39,6 +39,9 @@ struct iio_buffer;
  *                      device stops sampling. Calles are balanced with @enable.
  * @release:		called when the last reference to the buffer is dropped,
  *			should free all resources allocated by the buffer.
+ * @alloc_dmabuf:	called from userspace via ioctl to allocate one DMABUF.
+ * @enqueue_dmabuf:	called from userspace via ioctl to queue this DMABUF
+ *			object to this buffer. Requires a valid DMABUF fd.
  * @modes:		Supported operating modes by this buffer type
  * @flags:		A bitmask combination of INDIO_BUFFER_FLAG_*
  *
@@ -68,6 +71,11 @@ struct iio_buffer_access_funcs {
 
 	void (*release)(struct iio_buffer *buffer);
 
+	int (*alloc_dmabuf)(struct iio_buffer *buffer,
+			    struct iio_dmabuf_alloc_req *req);
+	int (*enqueue_dmabuf)(struct iio_buffer *buffer,
+			      struct iio_dmabuf *block);
+
 	unsigned int modes;
 	unsigned int flags;
 };
include/uapi/linux/iio/buffer.h +29 -0
index 13939032b3f6..e4621b926262 100644
@@ -5,6 +5,35 @@
 #ifndef _UAPI_IIO_BUFFER_H_
 #define _UAPI_IIO_BUFFER_H_
 
+#include <linux/types.h>
+
+#define IIO_BUFFER_DMABUF_SUPPORTED_FLAGS	0x00000000
+
+/**
+ * struct iio_dmabuf_alloc_req - Descriptor for allocating IIO DMABUFs
+ * @size:	the size of a single DMABUF
+ * @resv:	reserved
+ */
+struct iio_dmabuf_alloc_req {
+	__u64 size;
+	__u64 resv;
+};
+
+/**
+ * struct iio_dmabuf - Descriptor for a single IIO DMABUF object
+ * @fd:		file descriptor of the DMABUF object
+ * @flags:	one or more IIO_BUFFER_DMABUF_* flags
+ * @bytes_used:	number of bytes used in this DMABUF for the data transfer.
+ *		If zero, the full buffer is used.
+ */
+struct iio_dmabuf {
+	__u32 fd;
+	__u32 flags;
+	__u64 bytes_used;
+};
+
 #define IIO_BUFFER_GET_FD_IOCTL			_IOWR('i', 0x91, int)
+#define IIO_BUFFER_DMABUF_ALLOC_IOCTL		_IOW('i', 0x92, struct iio_dmabuf_alloc_req)
+#define IIO_BUFFER_DMABUF_ENQUEUE_IOCTL		_IOW('i', 0x93, struct iio_dmabuf)
 
 #endif /* _UAPI_IIO_BUFFER_H_ */
#30 lore raw

PATCH 8/15 iio: buffer-dma: split iio_dma_buffer_fileio_free() function

In reply to Paul Cercueil

From: Alexandru Ardelean <[email protected]>

A part of the logic in the iio_dma_buffer_exit() is required for the change to add mmap support to IIO buffers.
This change splits the logic into a separate function, which will be re-used later.

---
 drivers/iio/buffer/industrialio-buffer-dma.c | 39 +++++++++++---------
 1 file changed, 22 insertions(+), 17 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +22 -17
index eeeed6b2e0cf..eb8cfd3af030 100644
@@ -358,6 +358,27 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
 }
 EXPORT_SYMBOL_GPL(iio_dma_buffer_request_update);
 
+static void iio_dma_buffer_fileio_free(struct iio_dma_buffer_queue *queue)
+{
+	unsigned int i;
+
+	spin_lock_irq(&queue->list_lock);
+	for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
+		if (!queue->fileio.blocks[i])
+			continue;
+		queue->fileio.blocks[i]->state = IIO_BLOCK_STATE_DEAD;
+	}
+	spin_unlock_irq(&queue->list_lock);
+
+	for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
+		if (!queue->fileio.blocks[i])
+			continue;
+		iio_buffer_block_put(queue->fileio.blocks[i]);
+		queue->fileio.blocks[i] = NULL;
+	}
+	queue->fileio.active_block = NULL;
+}
+
 static void iio_dma_buffer_submit_block(struct iio_dma_buffer_queue *queue,
 	struct iio_dma_buffer_block *block)
 {
@@ -681,25 +702,9 @@ EXPORT_SYMBOL_GPL(iio_dma_buffer_init);
  */
 void iio_dma_buffer_exit(struct iio_dma_buffer_queue *queue)
 {
-	unsigned int i;
-
 	mutex_lock(&queue->lock);
 
-	spin_lock_irq(&queue->list_lock);
-	for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
-		if (!queue->fileio.blocks[i])
-			continue;
-		queue->fileio.blocks[i]->state = IIO_BLOCK_STATE_DEAD;
-	}
-	spin_unlock_irq(&queue->list_lock);
-
-	for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
-		if (!queue->fileio.blocks[i])
-			continue;
-		iio_buffer_block_put(queue->fileio.blocks[i]);
-		queue->fileio.blocks[i] = NULL;
-	}
-	queue->fileio.active_block = NULL;
+	iio_dma_buffer_fileio_free(queue);
 	queue->ops = NULL;
 
 	mutex_unlock(&queue->lock);
Signature · 1 line
2.33.0
#31 lore raw

Re: [PATCH 08/15] iio: buffer-dma: split iio_dma_buffer_fileio_free() function

Quoted 8 lines · Paul Cercueil

From: Alexandru Ardelean <[email protected]>

A part of the logic in the iio_dma_buffer_exit() is required for the change to add mmap support to IIO buffers.
This change splits the logic into a separate function, which will be re-used later.

Not sure how the protocol is here, since my old @analog.com email isn't working anymore.

But:

Thanks :)
Alex

Quoted 68 lines
---
 drivers/iio/buffer/industrialio-buffer-dma.c | 39 +++++++++++---------
 1 file changed, 22 insertions(+), 17 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +22 -18
index eeeed6b2e0cf..eb8cfd3af030 100644
@@ -358,6 +358,27 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
 }
 EXPORT_SYMBOL_GPL(iio_dma_buffer_request_update);
 
+static void iio_dma_buffer_fileio_free(struct iio_dma_buffer_queue *queue)
+{
+       unsigned int i;
+
+       spin_lock_irq(&queue->list_lock);
+       for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
+               if (!queue->fileio.blocks[i])
+                       continue;
+               queue->fileio.blocks[i]->state = IIO_BLOCK_STATE_DEAD;
+       }
+       spin_unlock_irq(&queue->list_lock);
+
+       for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
+               if (!queue->fileio.blocks[i])
+                       continue;
+               iio_buffer_block_put(queue->fileio.blocks[i]);
+               queue->fileio.blocks[i] = NULL;
+       }
+       queue->fileio.active_block = NULL;
+}
+
 static void iio_dma_buffer_submit_block(struct iio_dma_buffer_queue *queue,
        struct iio_dma_buffer_block *block)
 {
@@ -681,25 +702,9 @@ EXPORT_SYMBOL_GPL(iio_dma_buffer_init);
  */
 void iio_dma_buffer_exit(struct iio_dma_buffer_queue *queue)
 {
-       unsigned int i;
-
        mutex_lock(&queue->lock);
 
-       spin_lock_irq(&queue->list_lock);
-       for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
-               if (!queue->fileio.blocks[i])
-                       continue;
-               queue->fileio.blocks[i]->state = IIO_BLOCK_STATE_DEAD;
-       }
-       spin_unlock_irq(&queue->list_lock);
-
-       for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
-               if (!queue->fileio.blocks[i])
-                       continue;
-               iio_buffer_block_put(queue->fileio.blocks[i]);
-               queue->fileio.blocks[i] = NULL;
-       }
-       queue->fileio.active_block = NULL;
+       iio_dma_buffer_fileio_free(queue);
        queue->ops = NULL;
 
        mutex_unlock(&queue->lock);
--

2.33.0

maintainer
#32 lore raw

Re: [PATCH 08/15] iio: buffer-dma: split iio_dma_buffer_fileio_free() function

Quoted 10 lines · On Tue, 16 Nov 2021 12:59:30 +0200 Alexandru Ardelean <[email protected]>…
Quoted 8 lines · Paul Cercueil

From: Alexandru Ardelean <[email protected]>

A part of the logic in the iio_dma_buffer_exit() is required for the change to add mmap support to IIO buffers.
This change splits the logic into a separate function, which will be re-used later.

Quoted 6 lines

Not sure how the protocol is here, since my old @analog.com email isn't working anymore.

But:

If you want to both document your email address change and make various scripts find you more easily, send a patch for .mailmap

In meantime we can probably leave your original sign off in place but add a Cc: with an up to date email address. That will reflect that the sign off for this was (I assume) given back when you were at Analog.

Quoted 72 lines

Thanks :)
Alex

Quoted 68 lines
---
 drivers/iio/buffer/industrialio-buffer-dma.c | 39 +++++++++++---------
 1 file changed, 22 insertions(+), 17 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +22 -18
index eeeed6b2e0cf..eb8cfd3af030 100644
@@ -358,6 +358,27 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
 }
 EXPORT_SYMBOL_GPL(iio_dma_buffer_request_update);
 
+static void iio_dma_buffer_fileio_free(struct iio_dma_buffer_queue *queue)
+{
+       unsigned int i;
+
+       spin_lock_irq(&queue->list_lock);
+       for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
+               if (!queue->fileio.blocks[i])
+                       continue;
+               queue->fileio.blocks[i]->state = IIO_BLOCK_STATE_DEAD;
+       }
+       spin_unlock_irq(&queue->list_lock);
+
+       for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
+               if (!queue->fileio.blocks[i])
+                       continue;
+               iio_buffer_block_put(queue->fileio.blocks[i]);
+               queue->fileio.blocks[i] = NULL;
+       }
+       queue->fileio.active_block = NULL;
+}
+
 static void iio_dma_buffer_submit_block(struct iio_dma_buffer_queue *queue,
        struct iio_dma_buffer_block *block)
 {
@@ -681,25 +702,9 @@ EXPORT_SYMBOL_GPL(iio_dma_buffer_init);
  */
 void iio_dma_buffer_exit(struct iio_dma_buffer_queue *queue)
 {
-       unsigned int i;
-
        mutex_lock(&queue->lock);
 
-       spin_lock_irq(&queue->list_lock);
-       for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
-               if (!queue->fileio.blocks[i])
-                       continue;
-               queue->fileio.blocks[i]->state = IIO_BLOCK_STATE_DEAD;
-       }
-       spin_unlock_irq(&queue->list_lock);
-
-       for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
-               if (!queue->fileio.blocks[i])
-                       continue;
-               iio_buffer_block_put(queue->fileio.blocks[i]);
-               queue->fileio.blocks[i] = NULL;
-       }
-       queue->fileio.active_block = NULL;
+       iio_dma_buffer_fileio_free(queue);
        queue->ops = NULL;
 
        mutex_unlock(&queue->lock);
--

2.33.0

#33 lore raw

PATCH 9/15 iio: buffer-dma: Use DMABUFs instead of custom solution

In reply to Paul Cercueil

Enhance the current fileio code by using DMABUF objects instead of custom buffers.

This adds more code than it removes, but:
- a lot of the complexity can be dropped, e.g. custom kref and iio_buffer_block_put_atomic() are not needed anymore;
- it will be much easier to introduce an API to export these DMABUF objects to userspace in a following patch.

---
 drivers/iio/buffer/industrialio-buffer-dma.c | 196 ++++++++++++-------
 include/linux/iio/buffer-dma.h               |   8 +-
 2 files changed, 127 insertions(+), 77 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +123 -73
index eb8cfd3af030..adb20434f2d2 100644
@@ -14,6 +14,7 @@
 #include <linux/poll.h>
 #include <linux/iio/buffer_impl.h>
 #include <linux/iio/buffer-dma.h>
+#include <linux/dma-buf.h>
 #include <linux/dma-mapping.h>
 #include <linux/sizes.h>
 
@@ -90,104 +91,150 @@
  * callback is called from within the custom callback.
  */
 
-static void iio_buffer_block_release(struct kref *kref)
-{
-	struct iio_dma_buffer_block *block = container_of(kref,
-		struct iio_dma_buffer_block, kref);
-
-	WARN_ON(block->state != IIO_BLOCK_STATE_DEAD);
-
-	dma_free_coherent(block->queue->dev, PAGE_ALIGN(block->size),
-					block->vaddr, block->phys_addr);
-
-	iio_buffer_put(&block->queue->buffer);
-	kfree(block);
-}
-
-static void iio_buffer_block_get(struct iio_dma_buffer_block *block)
-{
-	kref_get(&block->kref);
-}
-
-static void iio_buffer_block_put(struct iio_dma_buffer_block *block)
-{
-	kref_put(&block->kref, iio_buffer_block_release);
-}
-
-/*
- * dma_free_coherent can sleep, hence we need to take some special care to be
- * able to drop a reference from an atomic context.
- */
-static LIST_HEAD(iio_dma_buffer_dead_blocks);
-static DEFINE_SPINLOCK(iio_dma_buffer_dead_blocks_lock);
-
-static void iio_dma_buffer_cleanup_worker(struct work_struct *work)
-{
-	struct iio_dma_buffer_block *block, *_block;
-	LIST_HEAD(block_list);
-
-	spin_lock_irq(&iio_dma_buffer_dead_blocks_lock);
-	list_splice_tail_init(&iio_dma_buffer_dead_blocks, &block_list);
-	spin_unlock_irq(&iio_dma_buffer_dead_blocks_lock);
-
-	list_for_each_entry_safe(block, _block, &block_list, head)
-		iio_buffer_block_release(&block->kref);
-}
-static DECLARE_WORK(iio_dma_buffer_cleanup_work, iio_dma_buffer_cleanup_worker);
-
-static void iio_buffer_block_release_atomic(struct kref *kref)
-{
+struct iio_buffer_dma_buf_attachment {
+	struct scatterlist sgl;
+	struct sg_table sg_table;
 	struct iio_dma_buffer_block *block;
-	unsigned long flags;
-
-	block = container_of(kref, struct iio_dma_buffer_block, kref);
-
-	spin_lock_irqsave(&iio_dma_buffer_dead_blocks_lock, flags);
-	list_add_tail(&block->head, &iio_dma_buffer_dead_blocks);
-	spin_unlock_irqrestore(&iio_dma_buffer_dead_blocks_lock, flags);
-
-	schedule_work(&iio_dma_buffer_cleanup_work);
-}
-
-/*
- * Version of iio_buffer_block_put() that can be called from atomic context
- */
-static void iio_buffer_block_put_atomic(struct iio_dma_buffer_block *block)
-{
-	kref_put(&block->kref, iio_buffer_block_release_atomic);
-}
+};
 
 static struct iio_dma_buffer_queue *iio_buffer_to_queue(struct iio_buffer *buf)
 {
 	return container_of(buf, struct iio_dma_buffer_queue, buffer);
 }
 
+static struct iio_buffer_dma_buf_attachment *
+to_iio_buffer_dma_buf_attachment(struct sg_table *table)
+{
+	return container_of(table, struct iio_buffer_dma_buf_attachment, sg_table);
+}
+
+static void iio_buffer_block_get(struct iio_dma_buffer_block *block)
+{
+	get_dma_buf(block->dmabuf);
+}
+
+static void iio_buffer_block_put(struct iio_dma_buffer_block *block)
+{
+	dma_buf_put(block->dmabuf);
+}
+
+static int iio_buffer_dma_buf_attach(struct dma_buf *dbuf,
+				     struct dma_buf_attachment *at)
+{
+	at->priv = dbuf->priv;
+
+	return 0;
+}
+
+static struct sg_table *iio_buffer_dma_buf_map(struct dma_buf_attachment *at,
+					       enum dma_data_direction dma_dir)
+{
+	struct iio_dma_buffer_block *block = at->priv;
+	struct iio_buffer_dma_buf_attachment *dba;
+	int ret;
+
+	dba = kzalloc(sizeof(*dba), GFP_KERNEL);
+	if (!dba)
+		return ERR_PTR(-ENOMEM);
+
+	sg_init_one(&dba->sgl, block->vaddr, PAGE_ALIGN(block->size));
+	dba->sg_table.sgl = &dba->sgl;
+	dba->sg_table.nents = 1;
+	dba->block = block;
+
+	ret = dma_map_sgtable(at->dev, &dba->sg_table, dma_dir, 0);
+	if (ret) {
+		kfree(dba);
+		return ERR_PTR(ret);
+	}
+
+	return &dba->sg_table;
+}
+
+static void iio_buffer_dma_buf_unmap(struct dma_buf_attachment *at,
+				     struct sg_table *sg_table,
+				     enum dma_data_direction dma_dir)
+{
+	struct iio_buffer_dma_buf_attachment *dba =
+		to_iio_buffer_dma_buf_attachment(sg_table);
+
+	dma_unmap_sgtable(at->dev, &dba->sg_table, dma_dir, 0);
+	kfree(dba);
+}
+
+static void iio_buffer_dma_buf_release(struct dma_buf *dbuf)
+{
+	struct iio_dma_buffer_block *block = dbuf->priv;
+	struct iio_dma_buffer_queue *queue = block->queue;
+
+	WARN_ON(block->state != IIO_BLOCK_STATE_DEAD);
+
+	mutex_lock(&queue->lock);
+
+	dma_free_coherent(queue->dev, PAGE_ALIGN(block->size),
+			  block->vaddr, block->phys_addr);
+
+	kfree(block);
+
+	mutex_unlock(&queue->lock);
+	iio_buffer_put(&queue->buffer);
+}
+
+
+static const struct dma_buf_ops iio_dma_buffer_dmabuf_ops = {
+	.attach			= iio_buffer_dma_buf_attach,
+	.map_dma_buf		= iio_buffer_dma_buf_map,
+	.unmap_dma_buf		= iio_buffer_dma_buf_unmap,
+	.release		= iio_buffer_dma_buf_release,
+};
+
 static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 	struct iio_dma_buffer_queue *queue, size_t size)
 {
 	struct iio_dma_buffer_block *block;
+	DEFINE_DMA_BUF_EXPORT_INFO(einfo);
+	struct dma_buf *dmabuf;
+	int err;
 
 	block = kzalloc(sizeof(*block), GFP_KERNEL);
 	if (!block)
-		return NULL;
+		return ERR_PTR(-ENOMEM);
 
 	block->vaddr = dma_alloc_coherent(queue->dev, PAGE_ALIGN(size),
 		&block->phys_addr, GFP_KERNEL);
 	if (!block->vaddr) {
-		kfree(block);
-		return NULL;
+		err = -ENOMEM;
+		goto err_free_block;
 	}
 
+	einfo.ops = &iio_dma_buffer_dmabuf_ops;
+	einfo.size = PAGE_ALIGN(size);
+	einfo.priv = block;
+	einfo.flags = O_RDWR;
+
+	dmabuf = dma_buf_export(&einfo);
+	if (IS_ERR(dmabuf)) {
+		err = PTR_ERR(dmabuf);
+		goto err_free_dma;
+	}
+
+	block->dmabuf = dmabuf;
 	block->size = size;
 	block->bytes_used = size;
 	block->state = IIO_BLOCK_STATE_DONE;
 	block->queue = queue;
 	INIT_LIST_HEAD(&block->head);
-	kref_init(&block->kref);
 
 	iio_buffer_get(&queue->buffer);
 
 	return block;
+
+err_free_dma:
+	dma_free_coherent(queue->dev, PAGE_ALIGN(size),
+			  block->vaddr, block->phys_addr);
+err_free_block:
+	kfree(block);
+	return ERR_PTR(err);
 }
 
 static void _iio_dma_buffer_block_done(struct iio_dma_buffer_block *block)
@@ -224,7 +271,7 @@ void iio_dma_buffer_block_done(struct iio_dma_buffer_block *block)
 	_iio_dma_buffer_block_done(block);
 	spin_unlock_irqrestore(&queue->list_lock, flags);
 
-	iio_buffer_block_put_atomic(block);
+	iio_buffer_block_put(block);
 	iio_dma_buffer_queue_wake(queue);
 }
 EXPORT_SYMBOL_GPL(iio_dma_buffer_block_done);
@@ -250,7 +297,8 @@ void iio_dma_buffer_block_list_abort(struct iio_dma_buffer_queue *queue,
 		list_del(&block->head);
 		block->bytes_used = 0;
 		_iio_dma_buffer_block_done(block);
-		iio_buffer_block_put_atomic(block);
+
+		iio_buffer_block_put(block);
 	}
 	spin_unlock_irqrestore(&queue->list_lock, flags);
 
@@ -340,11 +388,13 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
 
 		if (!block) {
 			block = iio_dma_buffer_alloc_block(queue, size);
-			if (!block) {
-				ret = -ENOMEM;
+			if (IS_ERR(block)) {
+				ret = PTR_ERR(block);
 				goto out_unlock;
 			}
 			queue->fileio.blocks[i] = block;
+
+			iio_buffer_block_get(block);
 		}
 
 		if (queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
include/linux/iio/buffer-dma.h +4 -4
index 09c07d5563c0..22effd6cfbb6 100644
@@ -8,7 +8,6 @@
 #define __INDUSTRIALIO_DMA_BUFFER_H__
 
 #include <linux/list.h>
-#include <linux/kref.h>
 #include <linux/spinlock.h>
 #include <linux/mutex.h>
 #include <linux/iio/buffer_impl.h>
@@ -16,6 +15,7 @@
 struct iio_dma_buffer_queue;
 struct iio_dma_buffer_ops;
 struct device;
+struct dma_buf;
 
 /**
  * enum iio_block_state - State of a struct iio_dma_buffer_block
@@ -41,8 +41,8 @@ enum iio_block_state {
  * @vaddr: Virutal address of the blocks memory
  * @phys_addr: Physical address of the blocks memory
  * @queue: Parent DMA buffer queue
- * @kref: kref used to manage the lifetime of block
  * @state: Current state of the block
+ * @dmabuf: Underlying DMABUF object
  */
 struct iio_dma_buffer_block {
 	/* May only be accessed by the owner of the block */
@@ -58,13 +58,13 @@ struct iio_dma_buffer_block {
 	size_t size;
 	struct iio_dma_buffer_queue *queue;
 
-	/* Must not be accessed outside the core. */
-	struct kref kref;
 	/*
 	 * Must not be accessed outside the core. Access needs to hold
 	 * queue->list_lock if the block is not owned by the core.
 	 */
 	enum iio_block_state state;
+
+	struct dma_buf *dmabuf;
 };
 
 /**
Signature · 1 line
2.33.0
#34 lore raw

PATCH 10/15 iio: buffer-dma: Implement new DMABUF based userspace API

In reply to Paul Cercueil

Implement the two functions iio_dma_buffer_alloc_dmabuf() and iio_dma_buffer_enqueue_dmabuf(), as well as all the necesary bits to enable userspace access to the DMABUF objects.

These two functions are exported as GPL symbols so that IIO buffer implementations can support the new DMABUF based userspace API.

---
 drivers/iio/buffer/industrialio-buffer-dma.c | 273 ++++++++++++++++++-
 include/linux/iio/buffer-dma.h               |  13 +
 2 files changed, 279 insertions(+), 7 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +266 -7
index adb20434f2d2..92356ee02f30 100644
@@ -15,7 +15,9 @@
 #include <linux/iio/buffer_impl.h>
 #include <linux/iio/buffer-dma.h>
 #include <linux/dma-buf.h>
+#include <linux/dma-fence.h>
 #include <linux/dma-mapping.h>
+#include <linux/dma-resv.h>
 #include <linux/sizes.h>
 
 /*
@@ -97,6 +99,18 @@ struct iio_buffer_dma_buf_attachment {
 	struct iio_dma_buffer_block *block;
 };
 
+struct iio_buffer_dma_fence {
+	struct dma_fence base;
+	struct iio_dma_buffer_block *block;
+	spinlock_t lock;
+};
+
+static struct iio_buffer_dma_fence *
+to_iio_buffer_dma_fence(struct dma_fence *fence)
+{
+	return container_of(fence, struct iio_buffer_dma_fence, base);
+}
+
 static struct iio_dma_buffer_queue *iio_buffer_to_queue(struct iio_buffer *buf)
 {
 	return container_of(buf, struct iio_dma_buffer_queue, buffer);
@@ -118,6 +132,48 @@ static void iio_buffer_block_put(struct iio_dma_buffer_block *block)
 	dma_buf_put(block->dmabuf);
 }
 
+static const char *
+iio_buffer_dma_fence_get_driver_name(struct dma_fence *fence)
+{
+	struct iio_buffer_dma_fence *iio_fence = to_iio_buffer_dma_fence(fence);
+
+	return dev_name(iio_fence->block->queue->dev);
+}
+
+static void iio_buffer_dma_fence_release(struct dma_fence *fence)
+{
+	struct iio_buffer_dma_fence *iio_fence = to_iio_buffer_dma_fence(fence);
+
+	kfree(iio_fence);
+}
+
+static const struct dma_fence_ops iio_buffer_dma_fence_ops = {
+	.get_driver_name	= iio_buffer_dma_fence_get_driver_name,
+	.get_timeline_name	= iio_buffer_dma_fence_get_driver_name,
+	.release		= iio_buffer_dma_fence_release,
+};
+
+static struct dma_fence *
+iio_dma_buffer_create_dma_fence(struct iio_dma_buffer_block *block)
+{
+	struct iio_buffer_dma_fence *fence;
+	u64 ctx;
+
+	fence = kzalloc(sizeof(*fence), GFP_KERNEL);
+	if (!fence)
+		return ERR_PTR(-ENOMEM);
+
+	fence->block = block;
+	spin_lock_init(&fence->lock);
+
+	ctx = dma_fence_context_alloc(1);
+
+	dma_fence_init(&fence->base, &iio_buffer_dma_fence_ops,
+		       &fence->lock, ctx, 0);
+
+	return &fence->base;
+}
+
 static int iio_buffer_dma_buf_attach(struct dma_buf *dbuf,
 				     struct dma_buf_attachment *at)
 {
@@ -162,10 +218,26 @@ static void iio_buffer_dma_buf_unmap(struct dma_buf_attachment *at,
 	kfree(dba);
 }
 
+static int iio_buffer_dma_buf_mmap(struct dma_buf *dbuf,
+				   struct vm_area_struct *vma)
+{
+	struct iio_dma_buffer_block *block = dbuf->priv;
+	struct device *dev = block->queue->dev;
+
+	vma->vm_flags |= VM_DONTEXPAND | VM_DONTDUMP;
+
+	if (vma->vm_ops->open)
+		vma->vm_ops->open(vma);
+
+	return dma_mmap_pages(dev, vma, vma->vm_end - vma->vm_start,
+			      virt_to_page(block->vaddr));
+}
+
 static void iio_buffer_dma_buf_release(struct dma_buf *dbuf)
 {
 	struct iio_dma_buffer_block *block = dbuf->priv;
 	struct iio_dma_buffer_queue *queue = block->queue;
+	bool is_fileio = block->fileio;
 
 	WARN_ON(block->state != IIO_BLOCK_STATE_DEAD);
 
@@ -176,20 +248,51 @@ static void iio_buffer_dma_buf_release(struct dma_buf *dbuf)
 
 	kfree(block);
 
+	queue->num_blocks--;
+	if (is_fileio)
+		queue->num_fileio_blocks--;
 	mutex_unlock(&queue->lock);
 	iio_buffer_put(&queue->buffer);
 }
 
+static int iio_buffer_dma_buf_begin_cpu_access(struct dma_buf *dbuf,
+					       enum dma_data_direction dma_dir)
+{
+	struct iio_dma_buffer_block *block = dbuf->priv;
+	struct device *dev = block->queue->dev;
+
+	/* We only need to invalidate the cache for input buffers */
+	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+		dma_sync_single_for_cpu(dev, block->phys_addr, block->size, dma_dir);
+
+	return 0;
+}
+
+static int iio_buffer_dma_buf_end_cpu_access(struct dma_buf *dbuf,
+					     enum dma_data_direction dma_dir)
+{
+	struct iio_dma_buffer_block *block = dbuf->priv;
+	struct device *dev = block->queue->dev;
+
+	/* We only need to sync the cache for output buffers */
+	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_OUT)
+		dma_sync_single_for_device(dev, block->phys_addr, block->size, dma_dir);
+
+	return 0;
+}
 
 static const struct dma_buf_ops iio_dma_buffer_dmabuf_ops = {
 	.attach			= iio_buffer_dma_buf_attach,
 	.map_dma_buf		= iio_buffer_dma_buf_map,
 	.unmap_dma_buf		= iio_buffer_dma_buf_unmap,
+	.mmap			= iio_buffer_dma_buf_mmap,
 	.release		= iio_buffer_dma_buf_release,
+	.begin_cpu_access	= iio_buffer_dma_buf_begin_cpu_access,
+	.end_cpu_access		= iio_buffer_dma_buf_end_cpu_access,
 };
 
 static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
-	struct iio_dma_buffer_queue *queue, size_t size)
+	struct iio_dma_buffer_queue *queue, size_t size, bool fileio)
 {
 	struct iio_dma_buffer_block *block;
 	DEFINE_DMA_BUF_EXPORT_INFO(einfo);
@@ -223,10 +326,15 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 	block->bytes_used = size;
 	block->state = IIO_BLOCK_STATE_DONE;
 	block->queue = queue;
+	block->fileio = fileio;
 	INIT_LIST_HEAD(&block->head);
 
 	iio_buffer_get(&queue->buffer);
 
+	queue->num_blocks++;
+	if (fileio)
+		queue->num_fileio_blocks++;
+
 	return block;
 
 err_free_dma:
@@ -265,14 +373,22 @@ static void iio_dma_buffer_queue_wake(struct iio_dma_buffer_queue *queue)
 void iio_dma_buffer_block_done(struct iio_dma_buffer_block *block)
 {
 	struct iio_dma_buffer_queue *queue = block->queue;
+	struct dma_resv *resv = block->dmabuf->resv;
+	struct dma_fence *fence;
 	unsigned long flags;
 
 	spin_lock_irqsave(&queue->list_lock, flags);
 	_iio_dma_buffer_block_done(block);
 	spin_unlock_irqrestore(&queue->list_lock, flags);
 
+	fence = dma_resv_excl_fence(resv);
+	if (fence)
+		dma_fence_signal(fence);
+	dma_resv_unlock(resv);
+
 	iio_buffer_block_put(block);
-	iio_dma_buffer_queue_wake(queue);
+	if (queue->fileio.enabled)
+		iio_dma_buffer_queue_wake(queue);
 }
 EXPORT_SYMBOL_GPL(iio_dma_buffer_block_done);
 
@@ -298,6 +414,8 @@ void iio_dma_buffer_block_list_abort(struct iio_dma_buffer_queue *queue,
 		block->bytes_used = 0;
 		_iio_dma_buffer_block_done(block);
 
+		if (dma_resv_is_locked(block->dmabuf->resv))
+			dma_resv_unlock(block->dmabuf->resv);
 		iio_buffer_block_put(block);
 	}
 	spin_unlock_irqrestore(&queue->list_lock, flags);
@@ -323,6 +441,12 @@ static bool iio_dma_block_reusable(struct iio_dma_buffer_block *block)
 	}
 }
 
+static bool iio_dma_buffer_fileio_mode(struct iio_dma_buffer_queue *queue)
+{
+	return queue->fileio.enabled ||
+		queue->num_blocks == queue->num_fileio_blocks;
+}
+
 /**
  * iio_dma_buffer_request_update() - DMA buffer request_update callback
  * @buffer: The buffer which to request an update
@@ -349,6 +473,12 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
 
 	mutex_lock(&queue->lock);
 
+	queue->fileio.enabled = iio_dma_buffer_fileio_mode(queue);
+
+	/* If DMABUFs were created, disable fileio interface */
+	if (!queue->fileio.enabled)
+		goto out_unlock;
+
 	/* Allocations are page aligned */
 	if (PAGE_ALIGN(queue->fileio.block_size) == PAGE_ALIGN(size))
 		try_reuse = true;
@@ -387,7 +517,7 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer)
 		}
 
 		if (!block) {
-			block = iio_dma_buffer_alloc_block(queue, size);
+			block = iio_dma_buffer_alloc_block(queue, size, true);
 			if (IS_ERR(block)) {
 				ret = PTR_ERR(block);
 				goto out_unlock;
@@ -444,6 +574,8 @@ static void iio_dma_buffer_submit_block(struct iio_dma_buffer_queue *queue,
 
 	block->state = IIO_BLOCK_STATE_ACTIVE;
 	iio_buffer_block_get(block);
+	dma_resv_lock(block->dmabuf->resv, NULL);
+
 	ret = queue->ops->submit(queue, block);
 	if (ret) {
 		/*
@@ -480,12 +612,18 @@ int iio_dma_buffer_enable(struct iio_buffer *buffer,
 	mutex_lock(&queue->lock);
 	queue->active = true;
 	queue->fileio.next_dequeue = 0;
+	queue->fileio.enabled = iio_dma_buffer_fileio_mode(queue);
 
-	for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
-		block = queue->fileio.blocks[i];
+	dev_dbg(queue->dev, "Buffer enabled in %s mode\n",
+		queue->fileio.enabled ? "fileio" : "dmabuf");
 
-		if (block->state == IIO_BLOCK_STATE_QUEUED)
-			iio_dma_buffer_submit_block(queue, block);
+	if (queue->fileio.enabled) {
+		for (i = 0; i < ARRAY_SIZE(queue->fileio.blocks); i++) {
+			block = queue->fileio.blocks[i];
+
+			if (block->state == IIO_BLOCK_STATE_QUEUED)
+				iio_dma_buffer_submit_block(queue, block);
+		}
 	}
 	mutex_unlock(&queue->lock);
 
@@ -507,6 +645,7 @@ int iio_dma_buffer_disable(struct iio_buffer *buffer,
 	struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer);
 
 	mutex_lock(&queue->lock);
+	queue->fileio.enabled = false;
 	queue->active = false;
 
 	if (queue->ops && queue->ops->abort)
@@ -565,6 +704,11 @@ static int iio_dma_buffer_io(struct iio_buffer *buffer,
 
 	mutex_lock(&queue->lock);
 
+	if (!queue->fileio.enabled) {
+		ret = -EBUSY;
+		goto out_unlock;
+	}
+
 	if (!queue->fileio.active_block) {
 		block = iio_dma_buffer_dequeue(queue);
 		if (block == NULL) {
@@ -681,6 +825,121 @@ size_t iio_dma_buffer_data_available(struct iio_buffer *buf)
 }
 EXPORT_SYMBOL_GPL(iio_dma_buffer_data_available);
 
+int iio_dma_buffer_alloc_dmabuf(struct iio_buffer *buffer,
+				struct iio_dmabuf_alloc_req *req)
+{
+	struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer);
+	struct iio_dma_buffer_block *block;
+	int ret = 0;
+
+	mutex_lock(&queue->lock);
+
+	/*
+	 * If the buffer is enabled and in fileio mode new blocks can't be
+	 * allocated.
+	 */
+	if (queue->fileio.enabled) {
+		ret = -EBUSY;
+		goto out_unlock;
+	}
+
+	if (!req->size || req->size > SIZE_MAX) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+
+	/* Free memory that might be in use for fileio mode */
+	iio_dma_buffer_fileio_free(queue);
+
+	block = iio_dma_buffer_alloc_block(queue, req->size, false);
+	if (IS_ERR(block)) {
+		ret = PTR_ERR(block);
+		goto out_unlock;
+	}
+
+	ret = dma_buf_fd(block->dmabuf, O_CLOEXEC);
+	if (ret < 0) {
+		dma_buf_put(block->dmabuf);
+		goto out_unlock;
+	}
+
+out_unlock:
+	mutex_unlock(&queue->lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(iio_dma_buffer_alloc_dmabuf);
+
+int iio_dma_buffer_enqueue_dmabuf(struct iio_buffer *buffer,
+				  struct iio_dmabuf *iio_dmabuf)
+{
+	struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer);
+	struct iio_dma_buffer_block *dma_block;
+	struct dma_fence *fence;
+	struct dma_buf *dmabuf;
+	int ret = 0;
+
+	mutex_lock(&queue->lock);
+
+	/* If in fileio mode buffers can't be enqueued. */
+	if (queue->fileio.enabled) {
+		ret = -EBUSY;
+		goto out_unlock;
+	}
+
+	dmabuf = dma_buf_get(iio_dmabuf->fd);
+	if (IS_ERR(dmabuf)) {
+		ret = PTR_ERR(dmabuf);
+		goto out_unlock;
+	}
+
+	if (dmabuf->ops != &iio_dma_buffer_dmabuf_ops) {
+		dev_err(queue->dev, "importing DMABUFs from other drivers is not yet supported.\n");
+		ret = -EINVAL;
+		goto out_dma_buf_put;
+	}
+
+	dma_block = dmabuf->priv;
+
+	if (iio_dmabuf->bytes_used > dma_block->size) {
+		ret = -EINVAL;
+		goto out_dma_buf_put;
+	}
+
+	dma_block->bytes_used = iio_dmabuf->bytes_used ?: dma_block->size;
+
+	switch (dma_block->state) {
+	case IIO_BLOCK_STATE_QUEUED:
+		/* Nothing to do */
+		goto out_unlock;
+	case IIO_BLOCK_STATE_DONE:
+		break;
+	default:
+		ret = -EBUSY;
+		goto out_dma_buf_put;
+	}
+
+	fence = iio_dma_buffer_create_dma_fence(dma_block);
+	if (IS_ERR(fence)) {
+		ret = PTR_ERR(fence);
+		goto out_dma_buf_put;
+	}
+
+	dma_resv_lock(dmabuf->resv, NULL);
+	dma_resv_add_excl_fence(dmabuf->resv, fence);
+	dma_resv_unlock(dmabuf->resv);
+
+	iio_dma_buffer_enqueue(queue, dma_block);
+
+out_dma_buf_put:
+	dma_buf_put(dmabuf);
+out_unlock:
+	mutex_unlock(&queue->lock);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(iio_dma_buffer_enqueue_dmabuf);
+
 /**
  * iio_dma_buffer_set_bytes_per_datum() - DMA buffer set_bytes_per_datum callback
  * @buffer: Buffer to set the bytes-per-datum for
include/linux/iio/buffer-dma.h +13 -0
index 22effd6cfbb6..85e55fe35282 100644
@@ -42,6 +42,7 @@ enum iio_block_state {
  * @phys_addr: Physical address of the blocks memory
  * @queue: Parent DMA buffer queue
  * @state: Current state of the block
+ * @fileio: True if this buffer is used for fileio mode
  * @dmabuf: Underlying DMABUF object
  */
 struct iio_dma_buffer_block {
@@ -64,6 +65,7 @@ struct iio_dma_buffer_block {
 	 */
 	enum iio_block_state state;
 
+	bool fileio;
 	struct dma_buf *dmabuf;
 };
 
@@ -74,6 +76,7 @@ struct iio_dma_buffer_block {
  * @pos: Read offset in the active block
  * @block_size: Size of each block
  * @next_dequeue: index of next block that will be dequeued
+ * @enabled: Whether the buffer is operating in fileio mode
  */
 struct iio_dma_buffer_queue_fileio {
 	struct iio_dma_buffer_block *blocks[2];
@@ -82,6 +85,7 @@ struct iio_dma_buffer_queue_fileio {
 	size_t block_size;
 
 	unsigned int next_dequeue;
+	bool enabled;
 };
 
 /**
@@ -96,6 +100,8 @@ struct iio_dma_buffer_queue_fileio {
  *   list and typically also a list of active blocks in the part that handles
  *   the DMA controller
  * @active: Whether the buffer is currently active
+ * @num_blocks: Total number of blocks in the queue
+ * @num_fileio_blocks: Number of blocks used for fileio interface
  * @fileio: FileIO state
  */
 struct iio_dma_buffer_queue {
@@ -107,6 +113,8 @@ struct iio_dma_buffer_queue {
 	spinlock_t list_lock;
 
 	bool active;
+	unsigned int num_blocks;
+	unsigned int num_fileio_blocks;
 
 	struct iio_dma_buffer_queue_fileio fileio;
 };
@@ -149,4 +157,9 @@ static inline size_t iio_dma_buffer_space_available(struct iio_buffer *buffer)
 	return iio_dma_buffer_data_available(buffer);
 }
 
+int iio_dma_buffer_alloc_dmabuf(struct iio_buffer *buffer,
+				struct iio_dmabuf_alloc_req *req);
+int iio_dma_buffer_enqueue_dmabuf(struct iio_buffer *buffer,
+				  struct iio_dmabuf *dmabuf);
+
 #endif
Signature · 1 line
2.33.0
#35 lore raw

PATCH 11/15 iio: buffer-dma: Boost performance using write-combine cache setting

In reply to Paul Cercueil

We can be certain that the input buffers will only be accessed by userspace for reading, and output buffers will mostly be accessed by userspace for writing.

Therefore, it makes more sense to use only fully cached input buffers, and to use the write-combine cache coherency setting for output buffers.

This boosts performance, as the data written to the output buffers does not have to be sync'd for coherency. It will halve performance if the userspace application tries to read from the output buffer, but this should never happen.

Since we don't need to sync the cache when disabling CPU access either for input buffers or output buffers, the .end_cpu_access() callback can be dropped completely.

---
 drivers/iio/buffer/industrialio-buffer-dma.c | 82 +++++++++++++-------
 1 file changed, 54 insertions(+), 28 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +54 -28
index 92356ee02f30..fb39054d8c15 100644
@@ -229,8 +229,33 @@ static int iio_buffer_dma_buf_mmap(struct dma_buf *dbuf,
 	if (vma->vm_ops->open)
 		vma->vm_ops->open(vma);
 
-	return dma_mmap_pages(dev, vma, vma->vm_end - vma->vm_start,
-			      virt_to_page(block->vaddr));
+	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN) {
+		/*
+		 * With an input buffer, userspace will only read the data and
+		 * never write. We can mmap the buffer fully cached.
+		 */
+		return dma_mmap_pages(dev, vma, vma->vm_end - vma->vm_start,
+				      virt_to_page(block->vaddr));
+	} else {
+		/*
+		 * With an output buffer, userspace will only write the data
+		 * and should rarely (if never) read from it. It is better to
+		 * use write-combine in this case.
+		 */
+		return dma_mmap_wc(dev, vma, block->vaddr, block->phys_addr,
+				   vma->vm_end - vma->vm_start);
+	}
+}
+
+static void iio_dma_buffer_free_dmamem(struct iio_dma_buffer_block *block)
+{
+	struct device *dev = block->queue->dev;
+	size_t size = PAGE_ALIGN(block->size);
+
+	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+		dma_free_coherent(dev, size, block->vaddr, block->phys_addr);
+	else
+		dma_free_wc(dev, size, block->vaddr, block->phys_addr);
 }
 
 static void iio_buffer_dma_buf_release(struct dma_buf *dbuf)
@@ -243,9 +268,7 @@ static void iio_buffer_dma_buf_release(struct dma_buf *dbuf)
 
 	mutex_lock(&queue->lock);
 
-	dma_free_coherent(queue->dev, PAGE_ALIGN(block->size),
-			  block->vaddr, block->phys_addr);
-
+	iio_dma_buffer_free_dmamem(block);
 	kfree(block);
 
 	queue->num_blocks--;
@@ -268,19 +291,6 @@ static int iio_buffer_dma_buf_begin_cpu_access(struct dma_buf *dbuf,
 	return 0;
 }
 
-static int iio_buffer_dma_buf_end_cpu_access(struct dma_buf *dbuf,
-					     enum dma_data_direction dma_dir)
-{
-	struct iio_dma_buffer_block *block = dbuf->priv;
-	struct device *dev = block->queue->dev;
-
-	/* We only need to sync the cache for output buffers */
-	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_OUT)
-		dma_sync_single_for_device(dev, block->phys_addr, block->size, dma_dir);
-
-	return 0;
-}
-
 static const struct dma_buf_ops iio_dma_buffer_dmabuf_ops = {
 	.attach			= iio_buffer_dma_buf_attach,
 	.map_dma_buf		= iio_buffer_dma_buf_map,
@@ -288,9 +298,28 @@ static const struct dma_buf_ops iio_dma_buffer_dmabuf_ops = {
 	.mmap			= iio_buffer_dma_buf_mmap,
 	.release		= iio_buffer_dma_buf_release,
 	.begin_cpu_access	= iio_buffer_dma_buf_begin_cpu_access,
-	.end_cpu_access		= iio_buffer_dma_buf_end_cpu_access,
 };
 
+static int iio_dma_buffer_alloc_dmamem(struct iio_dma_buffer_block *block)
+{
+	struct device *dev = block->queue->dev;
+	size_t size = PAGE_ALIGN(block->size);
+
+	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN) {
+		block->vaddr = dma_alloc_coherent(dev, size,
+						  &block->phys_addr,
+						  GFP_KERNEL);
+	} else {
+		block->vaddr = dma_alloc_wc(dev, size,
+					    &block->phys_addr,
+					    GFP_KERNEL);
+	}
+	if (!block->vaddr)
+		return -ENOMEM;
+
+	return 0;
+}
+
 static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 	struct iio_dma_buffer_queue *queue, size_t size, bool fileio)
 {
@@ -303,12 +332,12 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 	if (!block)
 		return ERR_PTR(-ENOMEM);
 
-	block->vaddr = dma_alloc_coherent(queue->dev, PAGE_ALIGN(size),
-		&block->phys_addr, GFP_KERNEL);
-	if (!block->vaddr) {
-		err = -ENOMEM;
+	block->size = size;
+	block->queue = queue;
+
+	err = iio_dma_buffer_alloc_dmamem(block);
+	if (err)
 		goto err_free_block;
-	}
 
 	einfo.ops = &iio_dma_buffer_dmabuf_ops;
 	einfo.size = PAGE_ALIGN(size);
@@ -322,10 +351,8 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 	}
 
 	block->dmabuf = dmabuf;
-	block->size = size;
 	block->bytes_used = size;
 	block->state = IIO_BLOCK_STATE_DONE;
-	block->queue = queue;
 	block->fileio = fileio;
 	INIT_LIST_HEAD(&block->head);
 
@@ -338,8 +365,7 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 	return block;
 
 err_free_dma:
-	dma_free_coherent(queue->dev, PAGE_ALIGN(size),
-			  block->vaddr, block->phys_addr);
+	iio_dma_buffer_free_dmamem(block);
 err_free_block:
 	kfree(block);
 	return ERR_PTR(err);
Signature · 1 line
2.33.0
#36 lore raw

Re: [PATCH 11/15] iio: buffer-dma: Boost performance using write-combine cache setting

Hi,

Quoted 106 lines · Paul Cercueil

We can be certain that the input buffers will only be accessed by userspace for reading, and output buffers will mostly be accessed by userspace for writing.

Therefore, it makes more sense to use only fully cached input buffers, and to use the write-combine cache coherency setting for output buffers.

This boosts performance, as the data written to the output buffers does not have to be sync'd for coherency. It will halve performance if the userspace application tries to read from the output buffer, but this should never happen.

Since we don't need to sync the cache when disabling CPU access either for input buffers or output buffers, the .end_cpu_access() callback can be dropped completely.

---
 drivers/iio/buffer/industrialio-buffer-dma.c | 82 
+++++++++++++-------
 1 file changed, 54 insertions(+), 28 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +27 -2
index 92356ee02f30..fb39054d8c15 100644
@@ -229,8 +229,33 @@ static int iio_buffer_dma_buf_mmap(struct 
 dma_buf *dbuf,
 	if (vma->vm_ops->open)
 		vma->vm_ops->open(vma);
 
-	return dma_mmap_pages(dev, vma, vma->vm_end - vma->vm_start,
-			      virt_to_page(block->vaddr));
+	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN) {
+		/*
+		 * With an input buffer, userspace will only read the data and
+		 * never write. We can mmap the buffer fully cached.
+		 */
+		return dma_mmap_pages(dev, vma, vma->vm_end - vma->vm_start,
+				      virt_to_page(block->vaddr));
+	} else {
+		/*
+		 * With an output buffer, userspace will only write the data
+		 * and should rarely (if never) read from it. It is better to
+		 * use write-combine in this case.
+		 */
+		return dma_mmap_wc(dev, vma, block->vaddr, block->phys_addr,
+				   vma->vm_end - vma->vm_start);
+	}
+}
+
+static void iio_dma_buffer_free_dmamem(struct iio_dma_buffer_block 
 *block)
+{
+	struct device *dev = block->queue->dev;
+	size_t size = PAGE_ALIGN(block->size);
+
+	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+		dma_free_coherent(dev, size, block->vaddr, block->phys_addr);
+	else
+		dma_free_wc(dev, size, block->vaddr, block->phys_addr);
 }

static void iio_buffer_dma_buf_release(struct dma_buf *dbuf)

drivers/iio/buffer/industrialio-buffer-dma.c +1 -3
@@ -243,9 +268,7 @@ static void iio_buffer_dma_buf_release(struct 
 dma_buf *dbuf)
 
 	mutex_lock(&queue->lock);
 
-	dma_free_coherent(queue->dev, PAGE_ALIGN(block->size),
-			  block->vaddr, block->phys_addr);
-
+	iio_dma_buffer_free_dmamem(block);
 	kfree(block);
 
drivers/iio/buffer/industrialio-buffer-dma.c +0 -13
 	queue->num_blocks--;
@@ -268,19 +291,6 @@ static int 
 iio_buffer_dma_buf_begin_cpu_access(struct dma_buf *dbuf,
 	return 0;
 }
 
-static int iio_buffer_dma_buf_end_cpu_access(struct dma_buf *dbuf,
-					     enum dma_data_direction dma_dir)
-{
-	struct iio_dma_buffer_block *block = dbuf->priv;
-	struct device *dev = block->queue->dev;
-
-	/* We only need to sync the cache for output buffers */
-	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_OUT)
-		dma_sync_single_for_device(dev, block->phys_addr, block->size, 
 dma_dir);
-
-	return 0;
-}
-
 static const struct dma_buf_ops iio_dma_buffer_dmabuf_ops = {
drivers/iio/buffer/industrialio-buffer-dma.c +0 -1
 	.attach			= iio_buffer_dma_buf_attach,
 	.map_dma_buf		= iio_buffer_dma_buf_map,
@@ -288,9 +298,28 @@ static const struct dma_buf_ops 
 iio_dma_buffer_dmabuf_ops = {
 	.mmap			= iio_buffer_dma_buf_mmap,
 	.release		= iio_buffer_dma_buf_release,
 	.begin_cpu_access	= iio_buffer_dma_buf_begin_cpu_access,
-	.end_cpu_access		= iio_buffer_dma_buf_end_cpu_access,
 };
 

drivers/iio/buffer/industrialio-buffer-dma.c static const struct dma_buf_ops · line 313

Quoted 10 lines
drivers/iio/buffer/industrialio-buffer-dma.c +9 -0
+static int iio_dma_buffer_alloc_dmamem(struct iio_dma_buffer_block 
 *block)
+{
+	struct device *dev = block->queue->dev;
+	size_t size = PAGE_ALIGN(block->size);
+
+	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN) {
+		block->vaddr = dma_alloc_coherent(dev, size,
+						  &block->phys_addr,
+						  GFP_KERNEL);

I'm so used to dma_alloc_noncoherent() that I didn't even notice that it was dma_alloc_coherent() here. The code I added meant to work with non-coherent memory - hence the dma_sync_* operations and the use of dma_mmap_pages().

I'll fix that in V2.

Cheers,
-Paul

Quoted 59 lines
drivers/iio/buffer/industrialio-buffer-dma.c +11 -0
+	} else {
+		block->vaddr = dma_alloc_wc(dev, size,
+					    &block->phys_addr,
+					    GFP_KERNEL);
+	}
+	if (!block->vaddr)
+		return -ENOMEM;
+
+	return 0;
+}
+
 static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
drivers/iio/buffer/industrialio-buffer-dma.c +5 -5
 	struct iio_dma_buffer_queue *queue, size_t size, bool fileio)
 {
@@ -303,12 +332,12 @@ static struct iio_dma_buffer_block 
 *iio_dma_buffer_alloc_block(
 	if (!block)
 		return ERR_PTR(-ENOMEM);
 
-	block->vaddr = dma_alloc_coherent(queue->dev, PAGE_ALIGN(size),
-		&block->phys_addr, GFP_KERNEL);
-	if (!block->vaddr) {
-		err = -ENOMEM;
+	block->size = size;
+	block->queue = queue;
+
+	err = iio_dma_buffer_alloc_dmamem(block);
+	if (err)
 		goto err_free_block;
-	}
 
 	einfo.ops = &iio_dma_buffer_dmabuf_ops;
drivers/iio/buffer/industrialio-buffer-dma.c +1 -4
 	einfo.size = PAGE_ALIGN(size);
@@ -322,10 +351,8 @@ static struct iio_dma_buffer_block 
 *iio_dma_buffer_alloc_block(
 	}
 
 	block->dmabuf = dmabuf;
-	block->size = size;
 	block->bytes_used = size;
 	block->state = IIO_BLOCK_STATE_DONE;
-	block->queue = queue;
 	block->fileio = fileio;
 	INIT_LIST_HEAD(&block->head);
@@ -338,8 +365,7 @@ static struct iio_dma_buffer_block 
 *iio_dma_buffer_alloc_block(
 	return block;
 
 err_free_dma:
-	dma_free_coherent(queue->dev, PAGE_ALIGN(size),
-			  block->vaddr, block->phys_addr);
+	iio_dma_buffer_free_dmamem(block);
 err_free_block:
 	kfree(block);

return ERR_PTR(err);
--
2.33.0

maintainer
#37 lore raw

Re: [PATCH 11/15] iio: buffer-dma: Boost performance using write-combine cache setting

In reply to Paul Cercueil
Quoted 3 lines · On Mon, 15 Nov 2021 14:19:21 +0000 Paul Cercueil <[email protected]> wrote:

We can be certain that the input buffers will only be accessed by userspace for reading, and output buffers will mostly be accessed by userspace for writing.

Mostly? Perhaps a little more info on why that's not 'only'.

Quoted 4 lines

Therefore, it makes more sense to use only fully cached input buffers, and to use the write-combine cache coherency setting for output buffers.

Quoted 8 lines

This boosts performance, as the data written to the output buffers does not have to be sync'd for coherency. It will halve performance if the userspace application tries to read from the output buffer, but this should never happen.

Since we don't need to sync the cache when disabling CPU access either for input buffers or output buffers, the .end_cpu_access() callback can be dropped completely.

We have an odd mix of coherent and non coherent DMA in here as you noted, but are you sure this is safe on all platforms?

Quoted 2 lines

Any numbers to support this patch? The mapping types are performance optimisations so nice to know how much of a difference they make.

Quoted 144 lines
---
 drivers/iio/buffer/industrialio-buffer-dma.c | 82 +++++++++++++-------
 1 file changed, 54 insertions(+), 28 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +54 -28
index 92356ee02f30..fb39054d8c15 100644
@@ -229,8 +229,33 @@ static int iio_buffer_dma_buf_mmap(struct dma_buf *dbuf,
 	if (vma->vm_ops->open)
 		vma->vm_ops->open(vma);
 
-	return dma_mmap_pages(dev, vma, vma->vm_end - vma->vm_start,
-			      virt_to_page(block->vaddr));
+	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN) {
+		/*
+		 * With an input buffer, userspace will only read the data and
+		 * never write. We can mmap the buffer fully cached.
+		 */
+		return dma_mmap_pages(dev, vma, vma->vm_end - vma->vm_start,
+				      virt_to_page(block->vaddr));
+	} else {
+		/*
+		 * With an output buffer, userspace will only write the data
+		 * and should rarely (if never) read from it. It is better to
+		 * use write-combine in this case.
+		 */
+		return dma_mmap_wc(dev, vma, block->vaddr, block->phys_addr,
+				   vma->vm_end - vma->vm_start);
+	}
+}
+
+static void iio_dma_buffer_free_dmamem(struct iio_dma_buffer_block *block)
+{
+	struct device *dev = block->queue->dev;
+	size_t size = PAGE_ALIGN(block->size);
+
+	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+		dma_free_coherent(dev, size, block->vaddr, block->phys_addr);
+	else
+		dma_free_wc(dev, size, block->vaddr, block->phys_addr);
 }
 
 static void iio_buffer_dma_buf_release(struct dma_buf *dbuf)
@@ -243,9 +268,7 @@ static void iio_buffer_dma_buf_release(struct dma_buf *dbuf)
 
 	mutex_lock(&queue->lock);
 
-	dma_free_coherent(queue->dev, PAGE_ALIGN(block->size),
-			  block->vaddr, block->phys_addr);
-
+	iio_dma_buffer_free_dmamem(block);
 	kfree(block);
 
 	queue->num_blocks--;
@@ -268,19 +291,6 @@ static int iio_buffer_dma_buf_begin_cpu_access(struct dma_buf *dbuf,
 	return 0;
 }
 
-static int iio_buffer_dma_buf_end_cpu_access(struct dma_buf *dbuf,
-					     enum dma_data_direction dma_dir)
-{
-	struct iio_dma_buffer_block *block = dbuf->priv;
-	struct device *dev = block->queue->dev;
-
-	/* We only need to sync the cache for output buffers */
-	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_OUT)
-		dma_sync_single_for_device(dev, block->phys_addr, block->size, dma_dir);
-
-	return 0;
-}
-
 static const struct dma_buf_ops iio_dma_buffer_dmabuf_ops = {
 	.attach			= iio_buffer_dma_buf_attach,
 	.map_dma_buf		= iio_buffer_dma_buf_map,
@@ -288,9 +298,28 @@ static const struct dma_buf_ops iio_dma_buffer_dmabuf_ops = {
 	.mmap			= iio_buffer_dma_buf_mmap,
 	.release		= iio_buffer_dma_buf_release,
 	.begin_cpu_access	= iio_buffer_dma_buf_begin_cpu_access,
-	.end_cpu_access		= iio_buffer_dma_buf_end_cpu_access,
 };
 
+static int iio_dma_buffer_alloc_dmamem(struct iio_dma_buffer_block *block)
+{
+	struct device *dev = block->queue->dev;
+	size_t size = PAGE_ALIGN(block->size);
+
+	if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN) {
+		block->vaddr = dma_alloc_coherent(dev, size,
+						  &block->phys_addr,
+						  GFP_KERNEL);
+	} else {
+		block->vaddr = dma_alloc_wc(dev, size,
+					    &block->phys_addr,
+					    GFP_KERNEL);
+	}
+	if (!block->vaddr)
+		return -ENOMEM;
+
+	return 0;
+}
+
 static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 	struct iio_dma_buffer_queue *queue, size_t size, bool fileio)
 {
@@ -303,12 +332,12 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 	if (!block)
 		return ERR_PTR(-ENOMEM);
 
-	block->vaddr = dma_alloc_coherent(queue->dev, PAGE_ALIGN(size),
-		&block->phys_addr, GFP_KERNEL);
-	if (!block->vaddr) {
-		err = -ENOMEM;
+	block->size = size;
+	block->queue = queue;
+
+	err = iio_dma_buffer_alloc_dmamem(block);
+	if (err)
 		goto err_free_block;
-	}
 
 	einfo.ops = &iio_dma_buffer_dmabuf_ops;
 	einfo.size = PAGE_ALIGN(size);
@@ -322,10 +351,8 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 	}
 
 	block->dmabuf = dmabuf;
-	block->size = size;
 	block->bytes_used = size;
 	block->state = IIO_BLOCK_STATE_DONE;
-	block->queue = queue;
 	block->fileio = fileio;
 	INIT_LIST_HEAD(&block->head);
 
@@ -338,8 +365,7 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
 	return block;
 
 err_free_dma:
-	dma_free_coherent(queue->dev, PAGE_ALIGN(size),
-			  block->vaddr, block->phys_addr);
+	iio_dma_buffer_free_dmamem(block);
 err_free_block:
 	kfree(block);
 	return ERR_PTR(err);
#38 lore raw

Re: [PATCH 11/15] iio: buffer-dma: Boost performance using write-combine cache setting

Hi Jonathan,

Quoted 8 lines · Jonathan Cameron
Quoted 3 lines · On Mon, 15 Nov 2021 14:19:21 +0000 Paul Cercueil <[email protected]> wrote:

We can be certain that the input buffers will only be accessed by userspace for reading, and output buffers will mostly be accessed by userspace for writing.

Mostly? Perhaps a little more info on why that's not 'only'.

Just like with a framebuffer, it really depends on what the application does. Most of the cases it will just read sequentially an input buffer, or write sequentially an output buffer. But then you get the exotic application that will try to do something like alpha blending, which means read+write. Hence "mostly".

Quoted 13 lines
Quoted 13 lines

Therefore, it makes more sense to use only fully cached input
buffers,
and to use the write-combine cache coherency setting for output
buffers.

This boosts performance, as the data written to the output buffers
does
not have to be sync'd for coherency. It will halve performance if
the
userspace application tries to read from the output buffer, but this should never happen.

Quoted 8 lines
Quoted 5 lines

Since we don't need to sync the cache when disabling CPU access
either
for input buffers or output buffers, the .end_cpu_access() callback
can
be dropped completely.

We have an odd mix of coherent and non coherent DMA in here as you noted, but are you sure this is safe on all platforms?

The mix isn't safe, but using only coherent or only non-coherent should be safe, yes.

Quoted 6 lines
Quoted 3 lines

Any numbers to support this patch? The mapping types are performance optimisations so nice to know how much of a difference they make.

Output buffers are definitely faster in write-combine mode. On a ZedBoard with a AD9361 transceiver set to 66 MSPS, and buffer/size set to 8192, I would get about 185 MiB/s before, 197 MiB/s after.

Input buffers... early results are mixed. On ARM32 it does look like it is slightly faster to read from *uncached* memory than reading from cached memory. The cache sync does take a long time.

Other architectures might have a different result, for instance on MIPS invalidating the cache is a very fast operation, so using cached buffers would be a huge win in performance.

Setups where the DMA operations are coherent also wouldn't require any cache sync and this patch would give a huge win in performance.

I'll run some more tests next week to have some fresh numbers.

Cheers,
-Paul

Quoted 157 lines
Quoted 156 lines

---

  drivers/iio/buffer/industrialio-buffer-dma.c | 82 
+++++++++++++-------
  1 file changed, 54 insertions(+), 28 deletions(-)

diff --git a/drivers/iio/buffer/industrialio-buffer-dma.c
b/drivers/iio/buffer/industrialio-buffer-dma.c
index 92356ee02f30..fb39054d8c15 100644
--- a/drivers/iio/buffer/industrialio-buffer-dma.c +++ b/drivers/iio/buffer/industrialio-buffer-dma.c @@ -229,8 +229,33 @@ static int iio_buffer_dma_buf_mmap(struct
dma_buf *dbuf,
if (vma->vm_ops->open)
vma->vm_ops->open(vma);

- return dma_mmap_pages(dev, vma, vma->vm_end - vma->vm_start,
- virt_to_page(block->vaddr));
+ if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN) {
+ /*
+ * With an input buffer, userspace will only read the data and
+ * never write. We can mmap the buffer fully cached.
+ */
+ return dma_mmap_pages(dev, vma, vma->vm_end - vma->vm_start,
+ virt_to_page(block->vaddr));
+ } else {
+ /*
+ * With an output buffer, userspace will only write the data
+ * and should rarely (if never) read from it. It is better to
+ * use write-combine in this case.
+ */
+ return dma_mmap_wc(dev, vma, block->vaddr, block->phys_addr,
+ vma->vm_end - vma->vm_start);
+ }
+}
+
+static void iio_dma_buffer_free_dmamem(struct iio_dma_buffer_block
*block)
+{
+ struct device *dev = block->queue->dev;
+ size_t size = PAGE_ALIGN(block->size);
+
+ if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+ dma_free_coherent(dev, size, block->vaddr, block->phys_addr);
+ else
+ dma_free_wc(dev, size, block->vaddr, block->phys_addr);
}

static void iio_buffer_dma_buf_release(struct dma_buf *dbuf)
@@ -243,9 +268,7 @@ static void iio_buffer_dma_buf_release(struct
dma_buf *dbuf)

  	mutex_lock(&queue->lock);

- dma_free_coherent(queue->dev, PAGE_ALIGN(block->size),
- block->vaddr, block->phys_addr);
-
+ iio_dma_buffer_free_dmamem(block);
kfree(block);

queue->num_blocks--;
@@ -268,19 +291,6 @@ static int
iio_buffer_dma_buf_begin_cpu_access(struct dma_buf *dbuf,
return 0;
}

-static int iio_buffer_dma_buf_end_cpu_access(struct dma_buf *dbuf,
- enum dma_data_direction dma_dir)
-{
- struct iio_dma_buffer_block *block = dbuf->priv;
- struct device *dev = block->queue->dev;
-
- /* We only need to sync the cache for output buffers */
- if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_OUT)
- dma_sync_single_for_device(dev, block->phys_addr, block->size,
dma_dir);
-
- return 0;
-}
-
static const struct dma_buf_ops iio_dma_buffer_dmabuf_ops = {
.attach = iio_buffer_dma_buf_attach,
.map_dma_buf = iio_buffer_dma_buf_map,
@@ -288,9 +298,28 @@ static const struct dma_buf_ops
iio_dma_buffer_dmabuf_ops = {
.mmap = iio_buffer_dma_buf_mmap,
.release = iio_buffer_dma_buf_release,
.begin_cpu_access = iio_buffer_dma_buf_begin_cpu_access,
- .end_cpu_access = iio_buffer_dma_buf_end_cpu_access,
};

+static int iio_dma_buffer_alloc_dmamem(struct iio_dma_buffer_block
*block)
+{
+ struct device *dev = block->queue->dev;
+ size_t size = PAGE_ALIGN(block->size);
+
+ if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN) {
+ block->vaddr = dma_alloc_coherent(dev, size,
+ &block->phys_addr,
+ GFP_KERNEL);
+ } else {
+ block->vaddr = dma_alloc_wc(dev, size,
+ &block->phys_addr,
+ GFP_KERNEL);
+ }
+ if (!block->vaddr)
+ return -ENOMEM;
+
+ return 0;
+}
+
static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
struct iio_dma_buffer_queue *queue, size_t size, bool fileio)
{
@@ -303,12 +332,12 @@ static struct iio_dma_buffer_block
*iio_dma_buffer_alloc_block(
if (!block)
return ERR_PTR(-ENOMEM);

- block->vaddr = dma_alloc_coherent(queue->dev, PAGE_ALIGN(size),
- &block->phys_addr, GFP_KERNEL);
- if (!block->vaddr) {
- err = -ENOMEM;
+ block->size = size;
+ block->queue = queue;
+
+ err = iio_dma_buffer_alloc_dmamem(block);
+ if (err)
goto err_free_block;
- }

einfo.ops = &iio_dma_buffer_dmabuf_ops; einfo.size = PAGE_ALIGN(size);
@@ -322,10 +351,8 @@ static struct iio_dma_buffer_block
*iio_dma_buffer_alloc_block(
}

block->dmabuf = dmabuf;
- block->size = size;
block->bytes_used = size;
block->state = IIO_BLOCK_STATE_DONE;
- block->queue = queue;
block->fileio = fileio;
INIT_LIST_HEAD(&block->head);

@@ -338,8 +365,7 @@ static struct iio_dma_buffer_block
*iio_dma_buffer_alloc_block(
return block;

err_free_dma:
- dma_free_coherent(queue->dev, PAGE_ALIGN(size),
- block->vaddr, block->phys_addr);
+ iio_dma_buffer_free_dmamem(block);
err_free_block:
kfree(block);
return ERR_PTR(err);

#39 lore raw

Re: [PATCH 11/15] iio: buffer-dma: Boost performance using write-combine cache setting

Hi Jonathan,

Quoted 49 lines · Paul Cercueil

Hi Jonathan,

Quoted 9 lines · Jonathan Cameron
Quoted 4 lines · On Mon, 15 Nov 2021 14:19:21 +0000 Paul Cercueil <[email protected]> wrote:

We can be certain that the input buffers will only be accessed by userspace for reading, and output buffers will mostly be accessed
by
userspace for writing.

Mostly? Perhaps a little more info on why that's not 'only'.

Just like with a framebuffer, it really depends on what the application does. Most of the cases it will just read sequentially an input buffer, or write sequentially an output buffer. But then you get the exotic application that will try to do something like alpha blending, which means read+write. Hence "mostly".

Quoted 14 lines
Quoted 14 lines

Therefore, it makes more sense to use only fully cached input
buffers,
and to use the write-combine cache coherency setting for output
buffers.

This boosts performance, as the data written to the output buffers
does
not have to be sync'd for coherency. It will halve performance if
the
userspace application tries to read from the output buffer, but
this
should never happen.

Quoted 8 lines
Quoted 5 lines

Since we don't need to sync the cache when disabling CPU access
either
for input buffers or output buffers, the .end_cpu_access()
callback can
be dropped completely.

We have an odd mix of coherent and non coherent DMA in here as you noted, but are you sure this is safe on all platforms?

The mix isn't safe, but using only coherent or only non-coherent should be safe, yes.

Quoted 6 lines
Quoted 3 lines

Any numbers to support this patch? The mapping types are performance optimisations so nice to know how much of a difference they make.

Output buffers are definitely faster in write-combine mode. On a ZedBoard with a AD9361 transceiver set to 66 MSPS, and buffer/size set to 8192, I would get about 185 MiB/s before, 197 MiB/s after.

Quoted 7 lines

Input buffers... early results are mixed. On ARM32 it does look like it is slightly faster to read from *uncached* memory than reading from cached memory. The cache sync does take a long time.

Other architectures might have a different result, for instance on MIPS invalidating the cache is a very fast operation, so using cached buffers would be a huge win in performance.

Setups where the DMA operations are coherent also wouldn't require any cache sync and this patch would give a huge win in performance.

I'll run some more tests next week to have some fresh numbers.

I think I mixed things up before, because I get different results now.

Here are some fresh benchmarks, triple-checked, using libiio's iio_readdev and iio_writedev tools, with 64K samples buffers at 61.44 MSPS (max. theorical throughput: 234 MiB/s):
iio_readdev -b 65536 cf-ad9361-lpc voltage0 voltage1 | pv > /dev/null
pv /dev/zero | iio_writedev -b 65536 cf-ad9361-dds-core-lpc voltage0
voltage1

Coherent mapping:
- fileio:
read: 125 MiB/s
write: 141 MiB/s
- dmabuf:
read: 171 MiB/s
write: 210 MiB/s

Coherent reads + Write-combine writes:
- fileio:
read: 125 MiB/s
write: 141 MiB/s
- dmabuf:
read: 171 MiB/s
write: 210 MiB/s

Non-coherent mapping:
- fileio:
read: 119 MiB/s
write: 124 MiB/s
- dmabuf:
read: 159 MiB/s
write: 124 MiB/s

Non-coherent reads + write-combine writes:
- fileio:
read: 119 MiB/s
write: 140 MiB/s
- dmabuf:
read: 159 MiB/s
write: 210 MiB/s

Non-coherent mapping with no cache sync:
- fileio:
read: 156 MiB/s
write: 123 MiB/s
- dmabuf:
read: 234 MiB/s (capped by sample rate)
write: 182 MiB/s

Non-coherent reads with no cache sync + write-combine writes:
- fileio:
read: 156 MiB/s
write: 140 MiB/s
- dmabuf:
read: 234 MiB/s (capped by sample rate)
write: 210 MiB/s

A few things we can deduce from this:

* Write-combine is not available on Zynq/ARM? If it was working, it should give a better performance than the coherent mapping, but it doesn't seem to do anything at all. At least it doesn't harm performance.

* Non-coherent + cache invalidation is definitely a good deal slower than using coherent mapping, at least on ARM32. However, when the cache sync is disabled (e.g. if the DMA operations are coherent) the reads are much faster.

* The new dma-buf based API is a great deal faster than the fileio API.

So in the future we could use coherent reads + write-combine writes, unless we know the DMA operations are coherent, and in this case use non-coherent reads + write-combine writes.

Regarding this patch, unfortunately I cannot prove that write-combine is faster, so I'll just drop this patch for now.

Cheers,
-Paul

maintainer
#40 lore raw

Re: [PATCH 11/15] iio: buffer-dma: Boost performance using write-combine cache setting

Quoted 75 lines · On Thu, 25 Nov 2021 17:29:58 +0000 Paul Cercueil <[email protected]> wrote:

Hi Jonathan,

Quoted 62 lines · Paul Cercueil

Hi Jonathan,

Quoted 9 lines · Jonathan Cameron
Quoted 4 lines · On Mon, 15 Nov 2021 14:19:21 +0000 Paul Cercueil <[email protected]> wrote:

We can be certain that the input buffers will only be accessed by userspace for reading, and output buffers will mostly be accessed
by
userspace for writing.

Mostly? Perhaps a little more info on why that's not 'only'.

Just like with a framebuffer, it really depends on what the application does. Most of the cases it will just read sequentially an input buffer, or write sequentially an output buffer. But then you get the exotic application that will try to do something like alpha blending, which means read+write. Hence "mostly".

Quoted 20 lines
Quoted 14 lines

Therefore, it makes more sense to use only fully cached input
buffers,
and to use the write-combine cache coherency setting for output
buffers.

This boosts performance, as the data written to the output buffers
does
not have to be sync'd for coherency. It will halve performance if
the
userspace application tries to read from the output buffer, but
this
should never happen.

Quoted 5 lines

Since we don't need to sync the cache when disabling CPU access
either
for input buffers or output buffers, the .end_cpu_access()
callback can
be dropped completely.

Quoted 3 lines

We have an odd mix of coherent and non coherent DMA in here as you noted,
but are you sure this is safe on all platforms?

The mix isn't safe, but using only coherent or only non-coherent should be safe, yes.

Quoted 6 lines
Quoted 3 lines

Any numbers to support this patch? The mapping types are performance optimisations so nice to know how much of a difference they make.

Output buffers are definitely faster in write-combine mode. On a ZedBoard with a AD9361 transceiver set to 66 MSPS, and buffer/size set to 8192, I would get about 185 MiB/s before, 197 MiB/s after.

Input buffers... early results are mixed. On ARM32 it does look like it is slightly faster to read from *uncached* memory than reading from cached memory. The cache sync does take a long time.

Quoted 8 lines

Other architectures might have a different result, for instance on MIPS invalidating the cache is a very fast operation, so using cached buffers would be a huge win in performance.

Setups where the DMA operations are coherent also wouldn't require any cache sync and this patch would give a huge win in performance.

I'll run some more tests next week to have some fresh numbers.

Quoted 8 lines

I think I mixed things up before, because I get different results now.

Here are some fresh benchmarks, triple-checked, using libiio's iio_readdev and iio_writedev tools, with 64K samples buffers at 61.44 MSPS (max. theorical throughput: 234 MiB/s):
iio_readdev -b 65536 cf-ad9361-lpc voltage0 voltage1 | pv > /dev/null pv /dev/zero | iio_writedev -b 65536 cf-ad9361-dds-core-lpc voltage0
voltage1

There is a bit of a terminology confusion going on here. I think for the mappings you mean cacheable vs non-cacheable but maybe I'm misunderstanding. That doesn't necessarily correspond to coherency. Non cached memory is always coherent because all caches miss.

Non-cacheable can be related to coherency of course. Also beware that given hardware might not implement non-cacheable if it knows all possible accesses are IO-coherent. Affect is the same and if implemented correctly it will not hurt performance significantly.

firmware should be letting the OS know if the device does coherent DMA or not... dma-coherent in dt. It might be optional for a given piece of DMA engine but I've not seen that..

I'm not sure I see how you can do a mixture of cacheable for reads and write combine (which means uncacheable) for writes...

Quoted 52 lines

Coherent mapping:
- fileio:

    read:	125 MiB/s
    write:	141 MiB/s
- dmabuf:
    read:	171 MiB/s
    write:	210 MiB/s

Coherent reads + Write-combine writes:
- fileio:

    read:	125 MiB/s
    write:	141 MiB/s
- dmabuf:
    read:	171 MiB/s
    write:	210 MiB/s

Non-coherent mapping:
- fileio:

    read:	119 MiB/s
    write:	124 MiB/s
- dmabuf:
    read:	159 MiB/s
    write:	124 MiB/s

Non-coherent reads + write-combine writes:
- fileio:

    read:	119 MiB/s
    write:	140 MiB/s
- dmabuf:
    read:	159 MiB/s
    write:	210 MiB/s

Non-coherent mapping with no cache sync:
- fileio:

    read:	156 MiB/s
    write:	123 MiB/s

- dmabuf:
read: 234 MiB/s (capped by sample rate)
write: 182 MiB/s

Non-coherent reads with no cache sync + write-combine writes:
- fileio:

    read:	156 MiB/s
    write:	140 MiB/s

- dmabuf:
read: 234 MiB/s (capped by sample rate)
write: 210 MiB/s

Quoted 6 lines

A few things we can deduce from this:

* Write-combine is not available on Zynq/ARM? If it was working, it should give a better performance than the coherent mapping, but it doesn't seem to do anything at all. At least it doesn't harm performance.

I'm not sure it's very relevant to this sort of streaming write. If you write a sequence of addresses then nothing stops them getting combined into a single write whether or not it is write-combining.

You may be right that the particular path to memory doesn't support it anyway. Also some cache architectures will rapidly detect streaming writes and elect not to cache them whether coherent or not.

Quoted 5 lines

* Non-coherent + cache invalidation is definitely a good deal slower than using coherent mapping, at least on ARM32. However, when the cache sync is disabled (e.g. if the DMA operations are coherent) the reads are much faster.

If you are running with cache sync then it better not be cached as such it's coherent in the sense of there being no entries in the cache in either direction.

Quoted 2 lines

* The new dma-buf based API is a great deal faster than the fileio API.

:)

Quoted 4 lines

So in the future we could use coherent reads + write-combine writes, unless we know the DMA operations are coherent, and in this case use non-coherent reads + write-combine writes.

Not following this argument at all, but anyway we can revisit when it mattrs.

Quoted 3 lines

Regarding this patch, unfortunately I cannot prove that write-combine is faster, so I'll just drop this patch for now.

Sure, thanks for checking. It's worth noting that WC usage in kernel is vanishingly rare and I suspect that's mostly because it doesn't do anything on many implementations.

Jonathan

Quoted 5 lines

Cheers,
-Paul

#41 lore raw

Re: [PATCH 11/15] iio: buffer-dma: Boost performance using write-combine cache setting

Quoted 19 lines · Jonathan Cameron
Quoted 19 lines

Non-coherent mapping with no cache sync:
- fileio:

     read:	156 MiB/s
     write:	123 MiB/s

- dmabuf:
read: 234 MiB/s (capped by sample rate)
write: 182 MiB/s

Non-coherent reads with no cache sync + write-combine writes:
- fileio:

     read:	156 MiB/s
     write:	140 MiB/s

- dmabuf:
read: 234 MiB/s (capped by sample rate)
write: 210 MiB/s

A few things we can deduce from this:

Quoted 7 lines
Quoted 4 lines

* Write-combine is not available on Zynq/ARM? If it was working, it should give a better performance than the coherent mapping, but it doesn't seem to do anything at all. At least it doesn't harm performance.

I'm not sure it's very relevant to this sort of streaming write. If you write a sequence of addresses then nothing stops them getting combined into a single write whether or not it is write-combining.

There is a difference at which point they can get combined. With write-combine they can be coalesced into a single transaction anywhere in the interconnect, as early as the CPU itself. Without write-cobmine the DDR controller might decide to combine them, but not earlier. This can make a difference especially if the write is a narrow write, i.e. the access size is smaller than the buswidth.

Lets say you do 32-bit writes, but your bus is 64 bits wide. With WC two 32-bits can be combined into a 64-bit write. Without WC that is not possible and you are potentially not using the bus to its fullest capacity. This is especially true if the memory bus is wider than the widest access size of the CPU.

maintainer
#42 lore raw

Re: [PATCH 11/15] iio: buffer-dma: Boost performance using write-combine cache setting

In reply to Paul Cercueil
Quoted 11 lines · On Sun, 21 Nov 2021 17:43:20 +0000 Paul Cercueil <[email protected]> wrote:

Hi Jonathan,

Quoted 7 lines · Jonathan Cameron
Quoted 3 lines · On Mon, 15 Nov 2021 14:19:21 +0000 Paul Cercueil <[email protected]> wrote:

We can be certain that the input buffers will only be accessed by userspace for reading, and output buffers will mostly be accessed by userspace for writing.

Quoted 7 lines
Quoted 1 line

Mostly? Perhaps a little more info on why that's not 'only'.

Just like with a framebuffer, it really depends on what the application does. Most of the cases it will just read sequentially an input buffer, or write sequentially an output buffer. But then you get the exotic application that will try to do something like alpha blending, which means read+write. Hence "mostly".

Ok. That makes sense though I hope no one actually does it, we can't prevent them doing so.

Quoted 20 lines
Quoted 20 lines
Quoted 19 lines

Therefore, it makes more sense to use only fully cached input
buffers,
and to use the write-combine cache coherency setting for output
buffers.

This boosts performance, as the data written to the output buffers
does
not have to be sync'd for coherency. It will halve performance if
the
userspace application tries to read from the output buffer, but this should never happen.

Since we don't need to sync the cache when disabling CPU access
either
for input buffers or output buffers, the .end_cpu_access() callback
can
be dropped completely.

Quoted 6 lines
Quoted 3 lines

We have an odd mix of coherent and non coherent DMA in here as you noted,
but are you sure this is safe on all platforms?

The mix isn't safe, but using only coherent or only non-coherent should be safe, yes.

yup

Quoted 16 lines
Quoted 7 lines
Quoted 4 lines

Any numbers to support this patch? The mapping types are performance optimisations so nice to know how much of a difference they make.

Output buffers are definitely faster in write-combine mode. On a ZedBoard with a AD9361 transceiver set to 66 MSPS, and buffer/size set to 8192, I would get about 185 MiB/s before, 197 MiB/s after.

Input buffers... early results are mixed. On ARM32 it does look like it is slightly faster to read from *uncached* memory than reading from cached memory. The cache sync does take a long time.

Quoted 8 lines

Other architectures might have a different result, for instance on MIPS invalidating the cache is a very fast operation, so using cached buffers would be a huge win in performance.

Setups where the DMA operations are coherent also wouldn't require any cache sync and this patch would give a huge win in performance.

I'll run some more tests next week to have some fresh numbers.

Great.

Thanks,

Jonathan

Quoted 163 lines

Cheers,
-Paul

Quoted 157 lines
Quoted 156 lines

---

  drivers/iio/buffer/industrialio-buffer-dma.c | 82 
+++++++++++++-------
  1 file changed, 54 insertions(+), 28 deletions(-)

diff --git a/drivers/iio/buffer/industrialio-buffer-dma.c
b/drivers/iio/buffer/industrialio-buffer-dma.c
index 92356ee02f30..fb39054d8c15 100644
--- a/drivers/iio/buffer/industrialio-buffer-dma.c +++ b/drivers/iio/buffer/industrialio-buffer-dma.c @@ -229,8 +229,33 @@ static int iio_buffer_dma_buf_mmap(struct
dma_buf *dbuf,
if (vma->vm_ops->open)
vma->vm_ops->open(vma);

- return dma_mmap_pages(dev, vma, vma->vm_end - vma->vm_start,
- virt_to_page(block->vaddr));
+ if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN) {
+ /*
+ * With an input buffer, userspace will only read the data and
+ * never write. We can mmap the buffer fully cached.
+ */
+ return dma_mmap_pages(dev, vma, vma->vm_end - vma->vm_start,
+ virt_to_page(block->vaddr));
+ } else {
+ /*
+ * With an output buffer, userspace will only write the data
+ * and should rarely (if never) read from it. It is better to
+ * use write-combine in this case.
+ */
+ return dma_mmap_wc(dev, vma, block->vaddr, block->phys_addr,
+ vma->vm_end - vma->vm_start);
+ }
+}
+
+static void iio_dma_buffer_free_dmamem(struct iio_dma_buffer_block
*block)
+{
+ struct device *dev = block->queue->dev;
+ size_t size = PAGE_ALIGN(block->size);
+
+ if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN)
+ dma_free_coherent(dev, size, block->vaddr, block->phys_addr);
+ else
+ dma_free_wc(dev, size, block->vaddr, block->phys_addr);
}

static void iio_buffer_dma_buf_release(struct dma_buf *dbuf)
@@ -243,9 +268,7 @@ static void iio_buffer_dma_buf_release(struct
dma_buf *dbuf)

  	mutex_lock(&queue->lock);

- dma_free_coherent(queue->dev, PAGE_ALIGN(block->size),
- block->vaddr, block->phys_addr);
-
+ iio_dma_buffer_free_dmamem(block);
kfree(block);

queue->num_blocks--;
@@ -268,19 +291,6 @@ static int
iio_buffer_dma_buf_begin_cpu_access(struct dma_buf *dbuf,
return 0;
}

-static int iio_buffer_dma_buf_end_cpu_access(struct dma_buf *dbuf,
- enum dma_data_direction dma_dir)
-{
- struct iio_dma_buffer_block *block = dbuf->priv;
- struct device *dev = block->queue->dev;
-
- /* We only need to sync the cache for output buffers */
- if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_OUT)
- dma_sync_single_for_device(dev, block->phys_addr, block->size,
dma_dir);
-
- return 0;
-}
-
static const struct dma_buf_ops iio_dma_buffer_dmabuf_ops = {
.attach = iio_buffer_dma_buf_attach,
.map_dma_buf = iio_buffer_dma_buf_map,
@@ -288,9 +298,28 @@ static const struct dma_buf_ops
iio_dma_buffer_dmabuf_ops = {
.mmap = iio_buffer_dma_buf_mmap,
.release = iio_buffer_dma_buf_release,
.begin_cpu_access = iio_buffer_dma_buf_begin_cpu_access,
- .end_cpu_access = iio_buffer_dma_buf_end_cpu_access,
};

+static int iio_dma_buffer_alloc_dmamem(struct iio_dma_buffer_block
*block)
+{
+ struct device *dev = block->queue->dev;
+ size_t size = PAGE_ALIGN(block->size);
+
+ if (block->queue->buffer.direction == IIO_BUFFER_DIRECTION_IN) {
+ block->vaddr = dma_alloc_coherent(dev, size,
+ &block->phys_addr,
+ GFP_KERNEL);
+ } else {
+ block->vaddr = dma_alloc_wc(dev, size,
+ &block->phys_addr,
+ GFP_KERNEL);
+ }
+ if (!block->vaddr)
+ return -ENOMEM;
+
+ return 0;
+}
+
static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block(
struct iio_dma_buffer_queue *queue, size_t size, bool fileio)
{
@@ -303,12 +332,12 @@ static struct iio_dma_buffer_block
*iio_dma_buffer_alloc_block(
if (!block)
return ERR_PTR(-ENOMEM);

- block->vaddr = dma_alloc_coherent(queue->dev, PAGE_ALIGN(size),
- &block->phys_addr, GFP_KERNEL);
- if (!block->vaddr) {
- err = -ENOMEM;
+ block->size = size;
+ block->queue = queue;
+
+ err = iio_dma_buffer_alloc_dmamem(block);
+ if (err)
goto err_free_block;
- }

einfo.ops = &iio_dma_buffer_dmabuf_ops; einfo.size = PAGE_ALIGN(size);
@@ -322,10 +351,8 @@ static struct iio_dma_buffer_block
*iio_dma_buffer_alloc_block(
}

block->dmabuf = dmabuf;
- block->size = size;
block->bytes_used = size;
block->state = IIO_BLOCK_STATE_DONE;
- block->queue = queue;
block->fileio = fileio;
INIT_LIST_HEAD(&block->head);

@@ -338,8 +365,7 @@ static struct iio_dma_buffer_block
*iio_dma_buffer_alloc_block(
return block;

err_free_dma:
- dma_free_coherent(queue->dev, PAGE_ALIGN(size),
- block->vaddr, block->phys_addr);
+ iio_dma_buffer_free_dmamem(block);
err_free_block:
kfree(block);
return ERR_PTR(err);

#43 lore raw

PATCH 12/15 iio: buffer-dmaengine: Support new DMABUF based userspace API

In reply to Paul Cercueil

Use the functions provided by the buffer-dma core to implement the DMABUF userspace API in the buffer-dmaengine IIO buffer implementation.

---
 drivers/iio/buffer/industrialio-buffer-dmaengine.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/iio/buffer/industrialio-buffer-dmaengine.c +3 -0
index 5cde8fd81c7f..57a8b2e4ba3c 100644
@@ -133,6 +133,9 @@ static const struct iio_buffer_access_funcs iio_dmaengine_buffer_ops = {
 	.space_available = iio_dma_buffer_space_available,
 	.release = iio_dmaengine_buffer_release,
 
+	.alloc_dmabuf = iio_dma_buffer_alloc_dmabuf,
+	.enqueue_dmabuf = iio_dma_buffer_enqueue_dmabuf,
+
 	.modes = INDIO_BUFFER_HARDWARE,
 	.flags = INDIO_BUFFER_FLAG_FIXED_WATERMARK,
 };
Signature · 1 line
2.33.0
#44 lore raw

PATCH 13/15 iio: core: Add support for cyclic buffers

Introduce a new flag IIO_BUFFER_DMABUF_CYCLIC in the "flags" field of the iio_dmabuf uapi structure.

When set, the DMABUF enqueued with the enqueue ioctl will be endlessly repeated on the TX output, until the buffer is disabled.

---
 drivers/iio/industrialio-buffer.c | 5 +++++
 include/uapi/linux/iio/buffer.h   | 3 ++-
 2 files changed, 7 insertions(+), 1 deletion(-)
drivers/iio/industrialio-buffer.c +5 -0
index 30910e6c2346..41bc51c88002 100644
@@ -1600,6 +1600,11 @@ static int iio_buffer_enqueue_dmabuf(struct iio_buffer *buffer,
 	if (dmabuf.flags & ~IIO_BUFFER_DMABUF_SUPPORTED_FLAGS)
 		return -EINVAL;
 
+	/* Cyclic flag is only supported on output buffers */
+	if ((dmabuf.flags & IIO_BUFFER_DMABUF_CYCLIC) &&
+	    buffer->direction != IIO_BUFFER_DIRECTION_OUT)
+		return -EINVAL;
+
 	return buffer->access->enqueue_dmabuf(buffer, &dmabuf);
 }
 
include/uapi/linux/iio/buffer.h +2 -1
index e4621b926262..2d541d038c02 100644
@@ -7,7 +7,8 @@
 
 #include <linux/types.h>
 
-#define IIO_BUFFER_DMABUF_SUPPORTED_FLAGS	0x00000000
+#define IIO_BUFFER_DMABUF_CYCLIC		(1 << 0)
+#define IIO_BUFFER_DMABUF_SUPPORTED_FLAGS	0x00000001
 
 /**
  * struct iio_dmabuf_alloc_req - Descriptor for allocating IIO DMABUFs
Signature · 1 line
2.33.0
Reviewed-by on v1 13/15
#45 lore raw

Re: [PATCH 13/15] iio: core: Add support for cyclic buffers

Quoted 7 lines · Paul Cercueil

Introduce a new flag IIO_BUFFER_DMABUF_CYCLIC in the "flags" field of the iio_dmabuf uapi structure.

When set, the DMABUF enqueued with the enqueue ioctl will be endlessly repeated on the TX output, until the buffer is disabled.

Quoted 39 lines
---
 drivers/iio/industrialio-buffer.c | 5 +++++
 include/uapi/linux/iio/buffer.h   | 3 ++-
 2 files changed, 7 insertions(+), 1 deletion(-)
drivers/iio/industrialio-buffer.c +5 -0
index 30910e6c2346..41bc51c88002 100644
@@ -1600,6 +1600,11 @@ static int iio_buffer_enqueue_dmabuf(struct iio_buffer *buffer,
        if (dmabuf.flags & ~IIO_BUFFER_DMABUF_SUPPORTED_FLAGS)
                return -EINVAL;
 
+       /* Cyclic flag is only supported on output buffers */
+       if ((dmabuf.flags & IIO_BUFFER_DMABUF_CYCLIC) &&
+           buffer->direction != IIO_BUFFER_DIRECTION_OUT)
+               return -EINVAL;
+
        return buffer->access->enqueue_dmabuf(buffer, &dmabuf);
 }
 
include/uapi/linux/iio/buffer.h +2 -2
index e4621b926262..2d541d038c02 100644
@@ -7,7 +7,8 @@
 
 #include <linux/types.h>
 
-#define IIO_BUFFER_DMABUF_SUPPORTED_FLAGS      0x00000000
+#define IIO_BUFFER_DMABUF_CYCLIC               (1 << 0)
+#define IIO_BUFFER_DMABUF_SUPPORTED_FLAGS      0x00000001
 
 /**
  * struct iio_dmabuf_alloc_req - Descriptor for allocating IIO DMABUFs
--

2.33.0

#46 lore raw

PATCH 14/15 iio: buffer-dmaengine: Add support for cyclic buffers

In reply to Paul Cercueil

Handle the IIO_BUFFER_DMABUF_CYCLIC flag to support cyclic buffers.

---
 drivers/iio/buffer/industrialio-buffer-dma.c      |  1 +
 .../iio/buffer/industrialio-buffer-dmaengine.c    | 15 ++++++++++++---
 include/linux/iio/buffer-dma.h                    |  3 +++
 3 files changed, 16 insertions(+), 3 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +1 -0
index fb39054d8c15..6658f103ee17 100644
@@ -933,6 +933,7 @@ int iio_dma_buffer_enqueue_dmabuf(struct iio_buffer *buffer,
 	}
 
 	dma_block->bytes_used = iio_dmabuf->bytes_used ?: dma_block->size;
+	dma_block->cyclic = iio_dmabuf->flags & IIO_BUFFER_DMABUF_CYCLIC;
 
 	switch (dma_block->state) {
 	case IIO_BLOCK_STATE_QUEUED:
drivers/iio/buffer/industrialio-buffer-dmaengine.c +12 -3
index 57a8b2e4ba3c..952e2160a11e 100644
@@ -81,9 +81,18 @@ static int iio_dmaengine_buffer_submit_block(struct iio_dma_buffer_queue *queue,
 	if (!block->bytes_used || block->bytes_used > max_size)
 		return -EINVAL;
 
-	desc = dmaengine_prep_slave_single(dmaengine_buffer->chan,
-		block->phys_addr, block->bytes_used, dma_dir,
-		DMA_PREP_INTERRUPT);
+	if (block->cyclic) {
+		desc = dmaengine_prep_dma_cyclic(dmaengine_buffer->chan,
+						 block->phys_addr,
+						 block->size,
+						 block->bytes_used,
+						 dma_dir, 0);
+	} else {
+		desc = dmaengine_prep_slave_single(dmaengine_buffer->chan,
+						   block->phys_addr,
+						   block->bytes_used, dma_dir,
+						   DMA_PREP_INTERRUPT);
+	}
 	if (!desc)
 		return -ENOMEM;
 
include/linux/iio/buffer-dma.h +3 -0
index 85e55fe35282..27639fdf7b54 100644
@@ -42,6 +42,7 @@ enum iio_block_state {
  * @phys_addr: Physical address of the blocks memory
  * @queue: Parent DMA buffer queue
  * @state: Current state of the block
+ * @cyclic: True if this is a cyclic buffer
  * @fileio: True if this buffer is used for fileio mode
  * @dmabuf: Underlying DMABUF object
  */
@@ -65,6 +66,8 @@ struct iio_dma_buffer_block {
 	 */
 	enum iio_block_state state;
 
+	bool cyclic;
+
 	bool fileio;
 	struct dma_buf *dmabuf;
 };
Signature · 1 line
2.33.0
Reviewed-by on v1 14/15
#47 lore raw

Re: [PATCH 14/15] iio: buffer-dmaengine: Add support for cyclic buffers

Quoted 3 lines · Paul Cercueil

Handle the IIO_BUFFER_DMABUF_CYCLIC flag to support cyclic buffers.

Quoted 69 lines
---
 drivers/iio/buffer/industrialio-buffer-dma.c      |  1 +
 .../iio/buffer/industrialio-buffer-dmaengine.c    | 15 ++++++++++++---
 include/linux/iio/buffer-dma.h                    |  3 +++
 3 files changed, 16 insertions(+), 3 deletions(-)
drivers/iio/buffer/industrialio-buffer-dma.c +1 -0
index fb39054d8c15..6658f103ee17 100644
@@ -933,6 +933,7 @@ int iio_dma_buffer_enqueue_dmabuf(struct iio_buffer *buffer,
        }
 
        dma_block->bytes_used = iio_dmabuf->bytes_used ?: dma_block->size;
+       dma_block->cyclic = iio_dmabuf->flags & IIO_BUFFER_DMABUF_CYCLIC;
 
        switch (dma_block->state) {
        case IIO_BLOCK_STATE_QUEUED:
drivers/iio/buffer/industrialio-buffer-dmaengine.c +12 -3
index 57a8b2e4ba3c..952e2160a11e 100644
@@ -81,9 +81,18 @@ static int iio_dmaengine_buffer_submit_block(struct iio_dma_buffer_queue *queue,
        if (!block->bytes_used || block->bytes_used > max_size)
                return -EINVAL;
 
-       desc = dmaengine_prep_slave_single(dmaengine_buffer->chan,
-               block->phys_addr, block->bytes_used, dma_dir,
-               DMA_PREP_INTERRUPT);
+       if (block->cyclic) {
+               desc = dmaengine_prep_dma_cyclic(dmaengine_buffer->chan,
+                                                block->phys_addr,
+                                                block->size,
+                                                block->bytes_used,
+                                                dma_dir, 0);
+       } else {
+               desc = dmaengine_prep_slave_single(dmaengine_buffer->chan,
+                                                  block->phys_addr,
+                                                  block->bytes_used, dma_dir,
+                                                  DMA_PREP_INTERRUPT);
+       }
        if (!desc)
                return -ENOMEM;
 
include/linux/iio/buffer-dma.h +3 -1
index 85e55fe35282..27639fdf7b54 100644
@@ -42,6 +42,7 @@ enum iio_block_state {
  * @phys_addr: Physical address of the blocks memory
  * @queue: Parent DMA buffer queue
  * @state: Current state of the block
+ * @cyclic: True if this is a cyclic buffer
  * @fileio: True if this buffer is used for fileio mode
  * @dmabuf: Underlying DMABUF object
  */
@@ -65,6 +66,8 @@ struct iio_dma_buffer_block {
         */
        enum iio_block_state state;
 
+       bool cyclic;
+
        bool fileio;
        struct dma_buf *dmabuf;
 };
--

2.33.0

#48 lore raw

PATCH 15/15 Documentation: iio: Document high-speed DMABUF based API

In reply to Paul Cercueil

Document the new DMABUF based API.

---
 Documentation/driver-api/dma-buf.rst |  2 +
 Documentation/iio/dmabuf_api.rst     | 94 ++++++++++++++++++++++++++++
 Documentation/iio/index.rst          |  2 +
 3 files changed, 98 insertions(+)
 create mode 100644 Documentation/iio/dmabuf_api.rst
Documentation/driver-api/dma-buf.rst +2 -0
index 2cd7db82d9fe..d3c9b58d2706 100644
@@ -1,3 +1,5 @@
+.. _dma-buf:
+
 Buffer Sharing and Synchronization
 ==================================
 
Documentation/iio/dmabuf_api.rst new file +94 -0

94 lines not on this page · raw message

Documentation/iio/index.rst +2 -0
index 58b7a4ebac51..9ce799fbf262 100644
@@ -10,3 +10,5 @@ Industrial I/O
    iio_configfs
 
    ep93xx_adc
+
+   dmabuf_api
Signature · 1 line
2.33.0
maintainer
#49 lore raw

Re: [PATCH 15/15] Documentation: iio: Document high-speed DMABUF based API

Quoted 3 lines · On Mon, 15 Nov 2021 14:22:43 +0000 Paul Cercueil <[email protected]> wrote:

Document the new DMABUF based API.

Hi Paul,

A few trivial things inline but looks good to me if we do end up using DMABUF anyway.

Jonathan

Quoted 12 lines
---
 Documentation/driver-api/dma-buf.rst |  2 +
 Documentation/iio/dmabuf_api.rst     | 94 ++++++++++++++++++++++++++++
 Documentation/iio/index.rst          |  2 +
 3 files changed, 98 insertions(+)
 create mode 100644 Documentation/iio/dmabuf_api.rst
Documentation/driver-api/dma-buf.rst +0 -0
index 2cd7db82d9fe..d3c9b58d2706 100644

Documentation/driver-api/dma-buf.rst line 2

Quoted 2 lines
Documentation/driver-api/dma-buf.rst +2 -0
@@ -1,3 +1,5 @@
+.. _dma-buf:
+

Why this change?

Quoted 12 lines
Documentation/driver-api/dma-buf.rst +0 -0
 Buffer Sharing and Synchronization
 ==================================
 
Documentation/iio/dmabuf_api.rst new file +3 -0
new file mode 100644
index 000000000000..b4e120a4ef0c
@@ -0,0 +1,94 @@
+===================================
+High-speed DMABUF interface for IIO
+===================================

Documentation/iio/dmabuf_api.rst line 13

Quoted 10 lines
Documentation/iio/dmabuf_api.rst new file +10 -0
+
+1. Overview
+===========
+
+The Industrial I/O subsystem supports access to buffers through a file-based
+interface, with read() and write() access calls through the IIO device's dev
+node.
+
+It additionally supports a DMABUF based interface, where the userspace
+application can allocate and append DMABUF objects to the buffer's queue.

I would note somewhere that this interface is optional for a given IIO driver. I don't want people to start assuming their i2c ADC will support this and wondering why it doesn't work :)

Quoted 86 lines
Documentation/iio/dmabuf_api.rst new file +81 -0
+
+The advantage of this DMABUF based interface vs. the fileio
+interface, is that it avoids an extra copy of the data between the
+kernel and userspace. This is particularly userful for high-speed
+devices which produce several megabytes or even gigabytes of data per
+second.
+
+The data in this DMABUF interface is managed at the granularity of
+DMABUF objects. Reducing the granularity from byte level to block level
+is done to reduce the userspace-kernelspace synchronization overhead
+since performing syscalls for each byte at a few Mbps is just not
+feasible.
+
+This of course leads to a slightly increased latency. For this reason an
+application can choose the size of the DMABUFs as well as how many it
+allocates. E.g. two DMABUFs would be a traditional double buffering
+scheme. But using a higher number might be necessary to avoid
+underflow/overflow situations in the presence of scheduling latencies.
+
+2. User API
+===========
+
+``IIO_BUFFER_DMABUF_ALLOC_IOCTL(struct iio_dmabuf_alloc_req *)``
+----------------------------------------------------------------
+
+Each call will allocate a new DMABUF object. The return value (if not
+a negative errno value as error) will be the file descriptor of the new
+DMABUF.
+
+``IIO_BUFFER_DMABUF_ENQUEUE_IOCTL(struct iio_dmabuf *)``
+--------------------------------------------------------
+
+Place the DMABUF object into the queue pending for hardware process.
+
+These two IOCTLs have to be performed on the IIO buffer's file
+descriptor (either opened from the corresponding /dev/iio:deviceX, or
+obtained using the `IIO_BUFFER_GET_FD_IOCTL` ioctl).
+
+3. Usage
+========
+
+To access the data stored in a block by userspace the block must be
+mapped to the process's memory. This is done by calling mmap() on the
+DMABUF's file descriptor.
+
+Before accessing the data through the map, you must use the
+DMA_BUF_IOCTL_SYNC(struct dma_buf_sync *) ioctl, with the
+DMA_BUF_SYNC_START flag, to make sure that the data is available.
+This call may block until the hardware is done with this block. Once
+you are done reading or writing the data, you must use this ioctl again
+with the DMA_BUF_SYNC_END flag, before enqueueing the DMABUF to the
+kernel's queue.
+
+If you need to know when the hardware is done with a DMABUF, you can
+poll its file descriptor for the EPOLLOUT event.
+
+Finally, to destroy a DMABUF object, simply call close() on its file
+descriptor.
+
+For more information about manipulating DMABUF objects, see: :ref:`dma-buf`.
+
+A typical workflow for the new interface is:
+
+    for block in blocks:
+      DMABUF_ALLOC block
+      mmap block
+
+    enable buffer
+
+    while !done
+      for block in blocks:
+        DMABUF_ENQUEUE block
+
+        DMABUF_SYNC_START block
+        process data
+        DMABUF_SYNC_END block
+
+    disable buffer
+
+    for block in blocks:
+      close block
Documentation/iio/index.rst +0 -0
index 58b7a4ebac51..9ce799fbf262 100644

Documentation/iio/index.rst Industrial I/O · line 14

Quoted 5 lines
Documentation/iio/index.rst +2 -0
@@ -10,3 +10,5 @@ Industrial I/O
    iio_configfs
 
    ep93xx_adc
+
+   dmabuf_api

Given this is core stuff rather than driver specific, perhaps move it up a few lines?

#50 lore raw

Re: [PATCH 15/15] Documentation: iio: Document high-speed DMABUF based API

Hi Jonathan,

Quoted 31 lines · Jonathan Cameron
Quoted 3 lines · On Mon, 15 Nov 2021 14:22:43 +0000 Paul Cercueil <[email protected]> wrote:

Document the new DMABUF based API.

Hi Paul,

A few trivial things inline but looks good to me if we do end up using DMABUF anyway.

Jonathan

Quoted 8 lines

---

  Documentation/driver-api/dma-buf.rst |  2 +
  Documentation/iio/dmabuf_api.rst     | 94 
++++++++++++++++++++++++++++
  Documentation/iio/index.rst          |  2 +
  3 files changed, 98 insertions(+)

create mode 100644 Documentation/iio/dmabuf_api.rst

Quoted 8 lines

diff --git a/Documentation/driver-api/dma-buf.rst
b/Documentation/driver-api/dma-buf.rst
index 2cd7db82d9fe..d3c9b58d2706 100644 --- a/Documentation/driver-api/dma-buf.rst +++ b/Documentation/driver-api/dma-buf.rst @@ -1,3 +1,5 @@
+.. _dma-buf:
+

Quoted 1 line

Why this change?

I have this line in the file:
For more information about manipulating DMABUF objects, see: :ref:dma-buf.

For the :ref: to work I need a label at the reference point, if I understood correctly.

Quoted 28 lines
Quoted 19 lines

Buffer Sharing and Synchronization
==================================

diff --git a/Documentation/iio/dmabuf_api.rst
b/Documentation/iio/dmabuf_api.rst
new file mode 100644
index 000000000000..b4e120a4ef0c
--- /dev/null
+++ b/Documentation/iio/dmabuf_api.rst
@@ -0,0 +1,94 @@
+===================================
+High-speed DMABUF interface for IIO
+===================================
+
+1. Overview
+===========
+
+The Industrial I/O subsystem supports access to buffers through a
file-based

Quoted 8 lines

+interface, with read() and write() access calls through the IIO
device's dev
+node.
+
+It additionally supports a DMABUF based interface, where the
userspace
+application can allocate and append DMABUF objects to the buffer's
queue.

Quoted 3 lines

I would note somewhere that this interface is optional for a given IIO driver. I don't want people to start assuming their i2c ADC will support this and wondering why it doesn't work :)

Their I2C ADC will support it, as long as the driver supports the dmaengine buffer interface. I can make that explicit, yes.

Quoted 105 lines
Quoted 105 lines

+
+The advantage of this DMABUF based interface vs. the fileio +interface, is that it avoids an extra copy of the data between the +kernel and userspace. This is particularly userful for high-speed +devices which produce several megabytes or even gigabytes of data
per
+second.
+
+The data in this DMABUF interface is managed at the granularity of +DMABUF objects. Reducing the granularity from byte level to block
level
+is done to reduce the userspace-kernelspace synchronization
overhead
+since performing syscalls for each byte at a few Mbps is just not +feasible.
+
+This of course leads to a slightly increased latency. For this
reason an
+application can choose the size of the DMABUFs as well as how many
it
+allocates. E.g. two DMABUFs would be a traditional double buffering +scheme. But using a higher number might be necessary to avoid +underflow/overflow situations in the presence of scheduling
latencies.
+
+2. User API
+===========
+
+``IIO_BUFFER_DMABUF_ALLOC_IOCTL(struct iio_dmabuf_alloc_req *)``
+----------------------------------------------------------------
+
+Each call will allocate a new DMABUF object. The return value (if
not
+a negative errno value as error) will be the file descriptor of
the new
+DMABUF.
+
+``IIO_BUFFER_DMABUF_ENQUEUE_IOCTL(struct iio_dmabuf *)``
+--------------------------------------------------------
+
+Place the DMABUF object into the queue pending for hardware
process.
+
+These two IOCTLs have to be performed on the IIO buffer's file +descriptor (either opened from the corresponding /dev/iio:deviceX,
or
+obtained using the IIO_BUFFER_GET_FD_IOCTL ioctl).
+
+3. Usage
+========
+
+To access the data stored in a block by userspace the block must be +mapped to the process's memory. This is done by calling mmap() on
the
+DMABUF's file descriptor.
+
+Before accessing the data through the map, you must use the +DMA_BUF_IOCTL_SYNC(struct dma_buf_sync *) ioctl, with the +DMA_BUF_SYNC_START flag, to make sure that the data is available. +This call may block until the hardware is done with this block.
Once
+you are done reading or writing the data, you must use this ioctl
again
+with the DMA_BUF_SYNC_END flag, before enqueueing the DMABUF to the +kernel's queue.
+
+If you need to know when the hardware is done with a DMABUF, you
can
+poll its file descriptor for the EPOLLOUT event.
+
+Finally, to destroy a DMABUF object, simply call close() on its
file
+descriptor.
+
+For more information about manipulating DMABUF objects, see:
:ref:dma-buf.
+
+A typical workflow for the new interface is:
+
+ for block in blocks:
+ DMABUF_ALLOC block
+ mmap block
+
+ enable buffer
+
+ while !done
+ for block in blocks:
+ DMABUF_ENQUEUE block
+

 +        DMABUF_SYNC_START block
 +        process data
 +        DMABUF_SYNC_END block

+
+ disable buffer
+
+ for block in blocks:
+ close block
diff --git a/Documentation/iio/index.rst
b/Documentation/iio/index.rst
index 58b7a4ebac51..9ce799fbf262 100644
--- a/Documentation/iio/index.rst
+++ b/Documentation/iio/index.rst
@@ -10,3 +10,5 @@ Industrial I/O
iio_configfs

Quoted 5 lines
Quoted 3 lines
     ep93xx_adc

+
+ dmabuf_api

Given this is core stuff rather than driver specific, perhaps move it up a few lines?

Alright.

Cheers,
-Paul

#51 lore raw
In reply to Paul Cercueil
Quoted 31 lines · Paul Cercueil

Hi Jonathan,

This patchset introduces a new userspace interface based on DMABUF objects, to complement the existing fileio based API.

The advantage of this DMABUF based interface vs. the fileio interface, is that it avoids an extra copy of the data between the kernel and userspace. This is particularly userful for high-speed devices which produce several megabytes or even gigabytes of data per second.

The first few patches [01/15] to [03/15] are not really related, but allow to reduce the size of the patches that introduce the new API.

Patch [04/15] to [06/15] enables write() support to the buffer-dma implementation of the buffer API, to continue the work done by Mihail Chindris.

Patches [07/15] to [12/15] introduce the new DMABUF based API.

Patches [13/15] and [14/15] add support for cyclic buffers, only through the new API. A cyclic buffer will be repeated on the output until the buffer is disabled.

Patch [15/15] adds documentation about the new API.

For now, the API allows you to alloc DMABUF objects and mmap() them to read or write the samples. It does not yet allow to import DMABUFs parented to other subsystems, but that should eventually be possible once it's wired.

Quoted 5 lines

This patchset is inspired by the "mmap interface" that was previously submitted by Alexandru Ardelean and Lars-Peter Clausen, so it would be great if I could get a review from you guys. Alexandru's commit was signed with his @analog.com address but he doesn't work at ADI anymore, so I believe I'll need him to sign with a new email.

Why dma-buf? dma-buf looks like something super generic and useful, until you realize that there's a metric ton of gpu/accelerator bagage piled in. So unless buffer sharing with a gpu/video/accel/whatever device is the goal here, and it's just for a convenient way to get at buffer handles, this doesn't sound like a good idea.

Also if the idea is to this with gpus/accelerators then I'd really like to see the full thing, since most likely at that point you also want dma_fence. And once we talk dma_fence things get truly horrible from a locking pov :-( Or well, just highly constrained and I get to review what iio is doing with these buffers to make sure it all fits.

Cheers, Daniel

Quoted 40 lines

Cheers,
-Paul

Alexandru Ardelean (1):
iio: buffer-dma: split iio_dma_buffer_fileio_free() function

Paul Cercueil (14):
iio: buffer-dma: Get rid of incoming/outgoing queues
iio: buffer-dma: Remove unused iio_buffer_block struct
iio: buffer-dma: Use round_down() instead of rounddown()
iio: buffer-dma: Enable buffer write support
iio: buffer-dmaengine: Support specifying buffer direction
iio: buffer-dmaengine: Enable write support
iio: core: Add new DMABUF interface infrastructure
iio: buffer-dma: Use DMABUFs instead of custom solution
iio: buffer-dma: Implement new DMABUF based userspace API
iio: buffer-dma: Boost performance using write-combine cache setting
iio: buffer-dmaengine: Support new DMABUF based userspace API
iio: core: Add support for cyclic buffers
iio: buffer-dmaengine: Add support for cyclic buffers
Documentation: iio: Document high-speed DMABUF based API

 Documentation/driver-api/dma-buf.rst          |   2 +
 Documentation/iio/dmabuf_api.rst              |  94 +++
 Documentation/iio/index.rst                   |   2 +
 drivers/iio/adc/adi-axi-adc.c                 |   3 +-
 drivers/iio/buffer/industrialio-buffer-dma.c  | 670 ++++++++++++++----
 .../buffer/industrialio-buffer-dmaengine.c    |  42 +-
 drivers/iio/industrialio-buffer.c             |  49 ++
 include/linux/iio/buffer-dma.h                |  43 +-
 include/linux/iio/buffer-dmaengine.h          |   5 +-
 include/linux/iio/buffer_impl.h               |   8 +
 include/uapi/linux/iio/buffer.h               |  30 +
 11 files changed, 783 insertions(+), 165 deletions(-)
 create mode 100644 Documentation/iio/dmabuf_api.rst
Signature · 1 line
2.33.0
Signature · 3 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
#52 lore raw

Hi Daniel,

Quoted 45 lines · Daniel Vetter
Quoted 35 lines · Paul Cercueil

Hi Jonathan,

This patchset introduces a new userspace interface based on DMABUF objects, to complement the existing fileio based API.

The advantage of this DMABUF based interface vs. the fileio interface, is that it avoids an extra copy of the data between the kernel and userspace. This is particularly userful for high-speed devices which produce several megabytes or even gigabytes of data
per
second.

The first few patches [01/15] to [03/15] are not really related, but allow to reduce the size of the patches that introduce the new API.

Patch [04/15] to [06/15] enables write() support to the buffer-dma implementation of the buffer API, to continue the work done by Mihail Chindris.

Patches [07/15] to [12/15] introduce the new DMABUF based API.

Patches [13/15] and [14/15] add support for cyclic buffers, only
through
the new API. A cyclic buffer will be repeated on the output until
the
buffer is disabled.

Patch [15/15] adds documentation about the new API.

For now, the API allows you to alloc DMABUF objects and mmap() them
to
read or write the samples. It does not yet allow to import DMABUFs parented to other subsystems, but that should eventually be possible once it's wired.

Quoted 8 lines

This patchset is inspired by the "mmap interface" that was
previously
submitted by Alexandru Ardelean and Lars-Peter Clausen, so it would
be
great if I could get a review from you guys. Alexandru's commit was signed with his @analog.com address but he doesn't work at ADI
anymore,
so I believe I'll need him to sign with a new email.

Quoted 5 lines

Why dma-buf? dma-buf looks like something super generic and useful, until you realize that there's a metric ton of gpu/accelerator bagage piled in. So unless buffer sharing with a gpu/video/accel/whatever device is the goal here, and it's just for a convenient way to get at buffer handles, this doesn't sound like a good idea.

Good question. The first reason is that a somewhat similar API was intented before[1], but refused upstream as it was kind of re-inventing the wheel.

The second reason, is that we want to be able to share buffers too, not with gpu/video but with the network (zctap) and in the future with USB (functionFS) too.

[1]: https://lore.kernel.org/linux-iio/[email protected]/T/

Quoted 5 lines

Also if the idea is to this with gpus/accelerators then I'd really like to see the full thing, since most likely at that point you also want dma_fence. And once we talk dma_fence things get truly horrible from a locking pov :-( Or well, just highly constrained and I get to review what iio is doing with these buffers to make sure it all fits.

There is some dma_fence action in patch #10, which is enough for the userspace apps to use the API.

What "horribleness" are we talking about here? It doesn't look that scary to me, but I certainly don't have the complete picture.

Cheers,
-Paul

Quoted 49 lines

Cheers, Daniel

Quoted 37 lines

Cheers,
-Paul

Alexandru Ardelean (1):
iio: buffer-dma: split iio_dma_buffer_fileio_free() function

Paul Cercueil (14):
iio: buffer-dma: Get rid of incoming/outgoing queues
iio: buffer-dma: Remove unused iio_buffer_block struct
iio: buffer-dma: Use round_down() instead of rounddown()
iio: buffer-dma: Enable buffer write support
iio: buffer-dmaengine: Support specifying buffer direction
iio: buffer-dmaengine: Enable write support
iio: core: Add new DMABUF interface infrastructure
iio: buffer-dma: Use DMABUFs instead of custom solution
iio: buffer-dma: Implement new DMABUF based userspace API
iio: buffer-dma: Boost performance using write-combine cache
setting
iio: buffer-dmaengine: Support new DMABUF based userspace API
iio: core: Add support for cyclic buffers
iio: buffer-dmaengine: Add support for cyclic buffers
Documentation: iio: Document high-speed DMABUF based API

  Documentation/driver-api/dma-buf.rst          |   2 +
  Documentation/iio/dmabuf_api.rst              |  94 +++
  Documentation/iio/index.rst                   |   2 +
  drivers/iio/adc/adi-axi-adc.c                 |   3 +-
  drivers/iio/buffer/industrialio-buffer-dma.c  | 670 
++++++++++++++----
  .../buffer/industrialio-buffer-dmaengine.c    |  42 +-
  drivers/iio/industrialio-buffer.c             |  49 ++
  include/linux/iio/buffer-dma.h                |  43 +-
  include/linux/iio/buffer-dmaengine.h          |   5 +-
  include/linux/iio/buffer_impl.h               |   8 +
  include/uapi/linux/iio/buffer.h               |  30 +
  11 files changed, 783 insertions(+), 165 deletions(-)
Quoted 5 lines

create mode 100644 Documentation/iio/dmabuf_api.rst

--
2.33.0

--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

#53 lore raw
Quoted 57 lines · Paul Cercueil

Hi Daniel,

Le lun., nov. 15 2021 at 15:37:16 +0100, Daniel Vetter <[email protected]> a écrit :

Quoted 45 lines · Paul Cercueil
Quoted 35 lines

Hi Jonathan,

This patchset introduces a new userspace interface based on DMABUF objects, to complement the existing fileio based API.

The advantage of this DMABUF based interface vs. the fileio interface, is that it avoids an extra copy of the data between the kernel and userspace. This is particularly userful for high-speed devices which produce several megabytes or even gigabytes of data
per
second.

The first few patches [01/15] to [03/15] are not really related, but allow to reduce the size of the patches that introduce the new API.

Patch [04/15] to [06/15] enables write() support to the buffer-dma implementation of the buffer API, to continue the work done by Mihail Chindris.

Patches [07/15] to [12/15] introduce the new DMABUF based API.

Patches [13/15] and [14/15] add support for cyclic buffers, only
through
the new API. A cyclic buffer will be repeated on the output until
the
buffer is disabled.

Patch [15/15] adds documentation about the new API.

For now, the API allows you to alloc DMABUF objects and mmap() them
to
read or write the samples. It does not yet allow to import DMABUFs parented to other subsystems, but that should eventually be possible once it's wired.

Quoted 8 lines

This patchset is inspired by the "mmap interface" that was
previously
submitted by Alexandru Ardelean and Lars-Peter Clausen, so it would
be
great if I could get a review from you guys. Alexandru's commit was signed with his @analog.com address but he doesn't work at ADI
anymore,
so I believe I'll need him to sign with a new email.

Quoted 7 lines

Why dma-buf? dma-buf looks like something super generic and useful, until
you realize that there's a metric ton of gpu/accelerator bagage piled in.
So unless buffer sharing with a gpu/video/accel/whatever device is the goal here, and it's just for a convenient way to get at buffer handles, this doesn't sound like a good idea.

Quoted 8 lines

Good question. The first reason is that a somewhat similar API was intented before[1], but refused upstream as it was kind of re-inventing the wheel.

The second reason, is that we want to be able to share buffers too, not with gpu/video but with the network (zctap) and in the future with USB (functionFS) too.

[1]: https://lore.kernel.org/linux-iio/[email protected]/T/

Hm is that code merged already in upstream already?

I know that dma-buf looks really generic, but like I said if there's no need ever to interface with any of the gpu buffer sharing it might be better to use something else (like get_user_pages maybe, would that work?).

Quoted 8 lines
Quoted 7 lines

Also if the idea is to this with gpus/accelerators then I'd really like to
see the full thing, since most likely at that point you also want dma_fence. And once we talk dma_fence things get truly horrible from a locking pov :-( Or well, just highly constrained and I get to review what
iio is doing with these buffers to make sure it all fits.

Quoted 5 lines

There is some dma_fence action in patch #10, which is enough for the userspace apps to use the API.

What "horribleness" are we talking about here? It doesn't look that scary to me, but I certainly don't have the complete picture.

You need to annotate all the code involved in signalling that dma_fence using dma_fence_begin/end_signalling, and then enable full lockdep and everything.

You can safely assume you'll find bugs, because we even have bugs about this in gpu drivers (where that annotation isn't fully rolled out yet).

The tldr is that you can allocate memory in there. And a pile of other restrictions, but not being able to allocate memory (well GFP_ATOMIC is ok, but that can fail) is a very serious restriction. -Daniel

Quoted 55 lines

Cheers,
-Paul

Quoted 49 lines

Cheers, Daniel

Quoted 37 lines

Cheers,
-Paul

Alexandru Ardelean (1):
iio: buffer-dma: split iio_dma_buffer_fileio_free() function

Paul Cercueil (14):
iio: buffer-dma: Get rid of incoming/outgoing queues
iio: buffer-dma: Remove unused iio_buffer_block struct
iio: buffer-dma: Use round_down() instead of rounddown()
iio: buffer-dma: Enable buffer write support
iio: buffer-dmaengine: Support specifying buffer direction
iio: buffer-dmaengine: Enable write support
iio: core: Add new DMABUF interface infrastructure
iio: buffer-dma: Use DMABUFs instead of custom solution
iio: buffer-dma: Implement new DMABUF based userspace API
iio: buffer-dma: Boost performance using write-combine cache
setting
iio: buffer-dmaengine: Support new DMABUF based userspace API
iio: core: Add support for cyclic buffers
iio: buffer-dmaengine: Add support for cyclic buffers
Documentation: iio: Document high-speed DMABUF based API

  Documentation/driver-api/dma-buf.rst          |   2 +
  Documentation/iio/dmabuf_api.rst              |  94 +++
  Documentation/iio/index.rst                   |   2 +
  drivers/iio/adc/adi-axi-adc.c                 |   3 +-
  drivers/iio/buffer/industrialio-buffer-dma.c  | 670
++++++++++++++----
  .../buffer/industrialio-buffer-dmaengine.c    |  42 +-
  drivers/iio/industrialio-buffer.c             |  49 ++
  include/linux/iio/buffer-dma.h                |  43 +-
  include/linux/iio/buffer-dmaengine.h          |   5 +-
  include/linux/iio/buffer_impl.h               |   8 +
  include/uapi/linux/iio/buffer.h               |  30 +
  11 files changed, 783 insertions(+), 165 deletions(-)
Quoted 5 lines

create mode 100644 Documentation/iio/dmabuf_api.rst

--
2.33.0

--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Signature · 3 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
#54 lore raw
Quoted 41 lines · Daniel Vetter
Quoted 40 lines · Paul Cercueil
Quoted 39 lines · Daniel Vetter
Quoted 37 lines · Paul Cercueil

Hi Jonathan,

This patchset introduces a new userspace interface based on DMABUF objects, to complement the existing fileio based API.

The advantage of this DMABUF based interface vs. the fileio interface, is that it avoids an extra copy of the data between the kernel and userspace. This is particularly userful for high-speed devices which produce several megabytes or even gigabytes of data per second.

The first few patches [01/15] to [03/15] are not really related, but allow to reduce the size of the patches that introduce the new API.

Patch [04/15] to [06/15] enables write() support to the buffer-dma implementation of the buffer API, to continue the work done by Mihail Chindris.

Patches [07/15] to [12/15] introduce the new DMABUF based API.

Patches [13/15] and [14/15] add support for cyclic buffers, only through the new API. A cyclic buffer will be repeated on the output until the buffer is disabled.

Patch [15/15] adds documentation about the new API.

For now, the API allows you to alloc DMABUF objects and mmap() them
to
read or write the samples. It does not yet allow to import DMABUFs parented to other subsystems, but that should eventually be possible once it's wired.

This patchset is inspired by the "mmap interface" that was previously submitted by Alexandru Ardelean and Lars-Peter Clausen, so it would be great if I could get a review from you guys. Alexandru's commit was signed with his @analog.com address but he doesn't work at ADI anymore, so I believe I'll need him to sign with a new email.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Why dma-buf? dma-buf looks like something super generic and useful, until you realize that there's a metric ton of gpu/accelerator bagage piled in. So unless buffer sharing with a gpu/video/accel/whatever device is the

And cameras (maybe they're included in "whatever" ?).

Quoted 10 lines
Quoted 10 lines
Quoted 2 lines

goal here, and it's just for a convenient way to get at buffer handles, this doesn't sound like a good idea.

Good question. The first reason is that a somewhat similar API was intented before[1], but refused upstream as it was kind of re-inventing the wheel.

The second reason, is that we want to be able to share buffers too, not with gpu/video but with the network (zctap) and in the future with USB (functionFS) too.

Quoted 7 lines
Quoted 1 line

[1]: https://lore.kernel.org/linux-iio/[email protected]/T/

Hm is that code merged already in upstream already?

I know that dma-buf looks really generic, but like I said if there's no need ever to interface with any of the gpu buffer sharing it might be better to use something else (like get_user_pages maybe, would that work?).

Not GUP please. That brings another set of issues, especially when dealing with DMA, we've suffered enough from the USERPTR support in V4L2 to avoid repeating this. Let's make dma-buf better instead.

Quoted 56 lines
Quoted 11 lines
Quoted 5 lines

Also if the idea is to this with gpus/accelerators then I'd really like to see the full thing, since most likely at that point you also want dma_fence. And once we talk dma_fence things get truly horrible from a locking pov :-( Or well, just highly constrained and I get to review what iio is doing with these buffers to make sure it all fits.

There is some dma_fence action in patch #10, which is enough for the userspace apps to use the API.

What "horribleness" are we talking about here? It doesn't look that scary to me, but I certainly don't have the complete picture.

You need to annotate all the code involved in signalling that dma_fence using dma_fence_begin/end_signalling, and then enable full lockdep and everything.

You can safely assume you'll find bugs, because we even have bugs about this in gpu drivers (where that annotation isn't fully rolled out yet).

The tldr is that you can allocate memory in there. And a pile of other restrictions, but not being able to allocate memory (well GFP_ATOMIC is ok, but that can fail) is a very serious restriction. -Daniel

Quoted 32 lines
Quoted 32 lines
Quoted 32 lines

Alexandru Ardelean (1):
iio: buffer-dma: split iio_dma_buffer_fileio_free() function

Paul Cercueil (14):
iio: buffer-dma: Get rid of incoming/outgoing queues
iio: buffer-dma: Remove unused iio_buffer_block struct
iio: buffer-dma: Use round_down() instead of rounddown()
iio: buffer-dma: Enable buffer write support
iio: buffer-dmaengine: Support specifying buffer direction
iio: buffer-dmaengine: Enable write support
iio: core: Add new DMABUF interface infrastructure
iio: buffer-dma: Use DMABUFs instead of custom solution
iio: buffer-dma: Implement new DMABUF based userspace API
iio: buffer-dma: Boost performance using write-combine cache setting
iio: buffer-dmaengine: Support new DMABUF based userspace API
iio: core: Add support for cyclic buffers
iio: buffer-dmaengine: Add support for cyclic buffers
Documentation: iio: Document high-speed DMABUF based API

  Documentation/driver-api/dma-buf.rst          |   2 +
  Documentation/iio/dmabuf_api.rst              |  94 +++
  Documentation/iio/index.rst                   |   2 +
  drivers/iio/adc/adi-axi-adc.c                 |   3 +-
  drivers/iio/buffer/industrialio-buffer-dma.c  | 670 ++++++++++++++----
  .../buffer/industrialio-buffer-dmaengine.c    |  42 +-
  drivers/iio/industrialio-buffer.c             |  49 ++
  include/linux/iio/buffer-dma.h                |  43 +-
  include/linux/iio/buffer-dmaengine.h          |   5 +-
  include/linux/iio/buffer_impl.h               |   8 +
  include/uapi/linux/iio/buffer.h               |  30 +
  11 files changed, 783 insertions(+), 165 deletions(-)

create mode 100644 Documentation/iio/dmabuf_api.rst

Signature · 3 lines
Regards,

Laurent Pinchart
maintainer
#55 lore raw

Am 16.11.21 um 17:31 schrieb Laurent Pinchart:

Quoted 56 lines · Daniel Vetter
Quoted 35 lines · Paul Cercueil
Quoted 34 lines · Daniel Vetter
Quoted 33 lines · Paul Cercueil
Quoted 32 lines

Hi Jonathan,

This patchset introduces a new userspace interface based on DMABUF objects, to complement the existing fileio based API.

The advantage of this DMABUF based interface vs. the fileio interface, is that it avoids an extra copy of the data between the kernel and userspace. This is particularly userful for high-speed devices which produce several megabytes or even gigabytes of data per second.

The first few patches [01/15] to [03/15] are not really related, but allow to reduce the size of the patches that introduce the new API.

Patch [04/15] to [06/15] enables write() support to the buffer-dma implementation of the buffer API, to continue the work done by Mihail Chindris.

Patches [07/15] to [12/15] introduce the new DMABUF based API.

Patches [13/15] and [14/15] add support for cyclic buffers, only through the new API. A cyclic buffer will be repeated on the output until the buffer is disabled.

Patch [15/15] adds documentation about the new API.

For now, the API allows you to alloc DMABUF objects and mmap() them
to
read or write the samples. It does not yet allow to import DMABUFs parented to other subsystems, but that should eventually be possible once it's wired.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

This patchset is inspired by the "mmap interface" that was previously submitted by Alexandru Ardelean and Lars-Peter Clausen, so it would be great if I could get a review from you guys. Alexandru's commit was signed with his @analog.com address but he doesn't work at ADI anymore, so I believe I'll need him to sign with a new email.

Why dma-buf? dma-buf looks like something super generic and useful, until you realize that there's a metric ton of gpu/accelerator bagage piled in. So unless buffer sharing with a gpu/video/accel/whatever device is the

And cameras (maybe they're included in "whatever" ?).

Quoted 10 lines
Quoted 10 lines
Quoted 2 lines

goal here, and it's just for a convenient way to get at buffer handles, this doesn't sound like a good idea.

Good question. The first reason is that a somewhat similar API was intented before[1], but refused upstream as it was kind of re-inventing the wheel.

The second reason, is that we want to be able to share buffers too, not with gpu/video but with the network (zctap) and in the future with USB (functionFS) too.

[1]: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Flinux-iio%2F20210217073638.21681-1-alexandru.ardelean%40analog.com%2FT%2F&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C7fffe09e82514577747808d9a91e9dd9%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637726772007336951%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=w1vD8IKz5G7ut%2FlsOyuYXKKQRBV1s8dN7qJBUwo8x9g%3D&amp;reserved=0

Quoted 8 lines
Quoted 5 lines

Hm is that code merged already in upstream already?

I know that dma-buf looks really generic, but like I said if there's no need ever to interface with any of the gpu buffer sharing it might be better to use something else (like get_user_pages maybe, would that work?).

Not GUP please. That brings another set of issues, especially when dealing with DMA, we've suffered enough from the USERPTR support in V4L2 to avoid repeating this. Let's make dma-buf better instead.

Yeah, when comparing GUP and DMA-buf the later is clearly the lesser evil.

DMA-buf indeed has some design issues we need to work on, especially around the async operations and synchronization. But I still think those are solvable.

GUP on the other hand has some hard fundamental problems which you can only solved completely if the hardware is capable of fast and reliable recoverable page faults.

Regards,
Christian.

#56 lore raw
In reply to Daniel Vetter

Hi Daniel,

Quoted 87 lines · Daniel Vetter
Quoted 86 lines · Paul Cercueil

Hi Daniel,

Le lun., nov. 15 2021 at 15:37:16 +0100, Daniel Vetter
<[email protected]> a
écrit :

Quoted 4 lines · Paul Cercueil
Quoted 3 lines

Hi Jonathan,

This patchset introduces a new userspace interface based on

DMABUF

Quoted 4 lines
Quoted 4 lines

objects, to complement the existing fileio based API.

The advantage of this DMABUF based interface vs. the fileio interface, is that it avoids an extra copy of the data between

the

Quoted 1 line
Quoted 1 line

kernel and userspace. This is particularly userful for

high-speed

Quoted 1 line
Quoted 1 line

devices which produce several megabytes or even gigabytes of

data

Quoted 4 lines
Quoted 4 lines

per
second.

The first few patches [01/15] to [03/15] are not really

related, but

Quoted 1 line
Quoted 1 line

allow to reduce the size of the patches that introduce the new

API.

Quoted 2 lines
Quoted 2 lines

Patch [04/15] to [06/15] enables write() support to the

buffer-dma

Quoted 6 lines
Quoted 6 lines

implementation of the buffer API, to continue the work done by Mihail Chindris.

Patches [07/15] to [12/15] introduce the new DMABUF based API.

Patches [13/15] and [14/15] add support for cyclic buffers,

only

Quoted 2 lines
Quoted 2 lines

through
the new API. A cyclic buffer will be repeated on the output

until

Quoted 6 lines
Quoted 6 lines

the
buffer is disabled.

Patch [15/15] adds documentation about the new API.

For now, the API allows you to alloc DMABUF objects and mmap()

them

Quoted 2 lines
Quoted 2 lines

to
read or write the samples. It does not yet allow to import

DMABUFs

Quoted 1 line
Quoted 1 line

parented to other subsystems, but that should eventually be

possible

Quoted 5 lines
Quoted 5 lines

once it's wired.

This patchset is inspired by the "mmap interface" that was
previously
submitted by Alexandru Ardelean and Lars-Peter Clausen, so it

would

Quoted 2 lines
Quoted 2 lines

be
great if I could get a review from you guys. Alexandru's

commit was

Quoted 5 lines
Quoted 3 lines

signed with his @analog.com address but he doesn't work at ADI
anymore,
so I believe I'll need him to sign with a new email.

Why dma-buf? dma-buf looks like something super generic and

useful,

Quoted 2 lines

until
you realize that there's a metric ton of gpu/accelerator bagage

piled

Quoted 2 lines

in.
So unless buffer sharing with a gpu/video/accel/whatever device

is the

Quoted 1 line

goal here, and it's just for a convenient way to get at buffer

handles,

Quoted 1 line

this doesn't sound like a good idea.

Good question. The first reason is that a somewhat similar API was
intented
before[1], but refused upstream as it was kind of re-inventing the
wheel.

The second reason, is that we want to be able to share buffers too,
not with
gpu/video but with the network (zctap) and in the future with USB (functionFS) too.

Quoted 4 lines
Quoted 2 lines

[1]:
https://lore.kernel.org/linux-iio/[email protected]/T/

Hm is that code merged already in upstream already?

No, it was never merged.

Quoted 3 lines

I know that dma-buf looks really generic, but like I said if there's no need ever to interface with any of the gpu buffer sharing it might be better to use something else (like get_user_pages maybe, would that work?).

If it was such a bad idea, why didn't you say it in the first place when you replied to my request for feedback? [1]

I don't think we have any other solution. We can design a custom API to pass buffers between IIO and user space, but that won't allow us to share these buffers with other subsystems. If dma-buf is not a generic solution, then we need a generic solution.

[1]: https://x-lore.kernel.org/io-uring/[email protected]/T/

Quoted 14 lines
Quoted 14 lines
Quoted 1 line

Also if the idea is to this with gpus/accelerators then I'd

really like

Quoted 3 lines

to
see the full thing, since most likely at that point you also want dma_fence. And once we talk dma_fence things get truly horrible

from a

Quoted 1 line

locking pov :-( Or well, just highly constrained and I get to

review

Quoted 2 lines

what
iio is doing with these buffers to make sure it all fits.

There is some dma_fence action in patch #10, which is enough for the userspace apps to use the API.

Quoted 7 lines
Quoted 3 lines

What "horribleness" are we talking about here? It doesn't look that
scary to
me, but I certainly don't have the complete picture.

You need to annotate all the code involved in signalling that dma_fence using dma_fence_begin/end_signalling, and then enable full lockdep and everything.

Doesn't dma_fence_signal() do it for me? Looking at the code, it does call dma_fence_begin/end_signalling.

Cheers,
-Paul

Quoted 69 lines

You can safely assume you'll find bugs, because we even have bugs about this in gpu drivers (where that annotation isn't fully rolled out yet).

The tldr is that you can allocate memory in there. And a pile of other restrictions, but not being able to allocate memory (well GFP_ATOMIC is ok, but that can fail) is a very serious restriction. -Daniel

Quoted 55 lines

Cheers,
-Paul

Quoted 49 lines

Cheers, Daniel

Quoted 37 lines

Cheers,
-Paul

Alexandru Ardelean (1):
iio: buffer-dma: split iio_dma_buffer_fileio_free() function

Paul Cercueil (14):
iio: buffer-dma: Get rid of incoming/outgoing queues
iio: buffer-dma: Remove unused iio_buffer_block struct
iio: buffer-dma: Use round_down() instead of rounddown()
iio: buffer-dma: Enable buffer write support
iio: buffer-dmaengine: Support specifying buffer direction
iio: buffer-dmaengine: Enable write support
iio: core: Add new DMABUF interface infrastructure
iio: buffer-dma: Use DMABUFs instead of custom solution
iio: buffer-dma: Implement new DMABUF based userspace API
iio: buffer-dma: Boost performance using write-combine cache
setting
iio: buffer-dmaengine: Support new DMABUF based userspace API
iio: core: Add support for cyclic buffers
iio: buffer-dmaengine: Add support for cyclic buffers
Documentation: iio: Document high-speed DMABUF based API

  Documentation/driver-api/dma-buf.rst          |   2 +
  Documentation/iio/dmabuf_api.rst              |  94 +++
  Documentation/iio/index.rst                   |   2 +
  drivers/iio/adc/adi-axi-adc.c                 |   3 +-
  drivers/iio/buffer/industrialio-buffer-dma.c  | 670
++++++++++++++----
  .../buffer/industrialio-buffer-dmaengine.c    |  42 +-
  drivers/iio/industrialio-buffer.c             |  49 ++
  include/linux/iio/buffer-dma.h                |  43 +-
  include/linux/iio/buffer-dmaengine.h          |   5 +-
  include/linux/iio/buffer_impl.h               |   8 +
  include/uapi/linux/iio/buffer.h               |  30 +
  11 files changed, 783 insertions(+), 165 deletions(-)
Quoted 5 lines

create mode 100644 Documentation/iio/dmabuf_api.rst

--
2.33.0

--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

#57 lore raw
Quoted 98 lines

-----Original Message-----
From: Paul Cercueil <[email protected]>
Sent: Mittwoch, 17. November 2021 13:50
To: Daniel Vetter <[email protected]>
Cc: Jonathan Cameron <[email protected]>; Hennerich, Michael <[email protected]>; [email protected]; linux- [email protected]; [email protected]; Christian König <[email protected]>; [email protected]; Alexandru Ardelean <[email protected]>; [email protected]
Subject: Re: [PATCH 00/15] iio: buffer-dma: write() and new DMABUF based API

[External]

Hi Daniel,

Le mar., nov. 16 2021 at 17:02:25 +0100, Daniel Vetter <[email protected]> a écrit :

Quoted 56 lines · Paul Cercueil
Quoted 55 lines

Hi Daniel,

Le lun., nov. 15 2021 at 15:37:16 +0100, Daniel Vetter
<[email protected]> a écrit :

Quoted 4 lines · Paul Cercueil
Quoted 3 lines

Hi Jonathan,

This patchset introduces a new userspace interface based on

DMABUF > > objects, to complement the existing fileio based API.

Quoted 4 lines
Quoted 2 lines

The advantage of this DMABUF based interface vs. the fileio >

interface, is that it avoids an extra copy of the data between the

Quoted 1 line

kernel and userspace. This is particularly userful for

high-speed > > devices which produce several megabytes or even gigabytes of data > > per > > second.

Quoted 2 lines
Quoted 2 lines

The first few patches [01/15] to [03/15] are not really

related, but > > allow to reduce the size of the patches that introduce the new API.

Quoted 2 lines
Quoted 2 lines

Patch [04/15] to [06/15] enables write() support to the

buffer-dma > > implementation of the buffer API, to continue the work done by > > Mihail Chindris.

Quoted 4 lines
Quoted 4 lines

Patches [07/15] to [12/15] introduce the new DMABUF based API.

Patches [13/15] and [14/15] add support for cyclic buffers,

only > > through > > the new API. A cyclic buffer will be repeated on the output until > > the > > buffer is disabled.

Quoted 4 lines
Quoted 4 lines

Patch [15/15] adds documentation about the new API.

For now, the API allows you to alloc DMABUF objects and mmap()

them > > to > > read or write the samples. It does not yet allow to import DMABUFs > > parented to other subsystems, but that should eventually be possible > > once it's wired.

Quoted 2 lines
Quoted 2 lines

This patchset is inspired by the "mmap interface" that was > >

previously > > submitted by Alexandru Ardelean and Lars-Peter Clausen, so it would > > be > > great if I could get a review from you guys. Alexandru's commit was > > signed with his @analog.com address but he doesn't work at ADI > > anymore, > > so I believe I'll need him to sign with a new email.

Quoted 2 lines

Why dma-buf? dma-buf looks like something super generic and

useful, > until > you realize that there's a metric ton of gpu/accelerator bagage piled > in.

Quoted 1 line

So unless buffer sharing with a gpu/video/accel/whatever device is

the > goal here, and it's just for a convenient way to get at buffer handles, > this doesn't sound like a good idea.

Good question. The first reason is that a somewhat similar API was
intented before[1], but refused upstream as it was kind of re-inventing the wheel.

Quoted 8 lines
Quoted 8 lines

The second reason, is that we want to be able to share buffers too,
not with gpu/video but with the network (zctap) and in the future with USB
(functionFS) too.

[1]:
https://urldefense.com/v3/__https://lore.kernel.org/linux-iio/2021021
7073638.21681-1-

[email protected]/T/__;!!A3Ni8CS0y2Y!p67g

Quoted 1 line
Quoted 1 line

KdXW2zXUxASbwbCKzXgcwxEo1R3h4AumAE6QHiSPyI3KYaz9TmGpnSIF3wsQ c3gAgQ$

Quoted 2 lines

Hm is that code merged already in upstream already?

No, it was never merged.

Quoted 4 lines

I know that dma-buf looks really generic, but like I said if there's no need ever to interface with any of the gpu buffer sharing it might be better to use something else (like get_user_pages maybe, would that work?).

Quoted 7 lines

If it was such a bad idea, why didn't you say it in the first place when you replied to my request for feedback? [1]

I don't think we have any other solution. We can design a custom API to pass buffers between IIO and user space, but that won't allow us to share these buffers with other subsystems. If dma-buf is not a generic solution, then we need a generic solution.

I don't think we need another generic solution - dma-buf is the solution:

From https://www.kernel.org/doc/html/v5.15/driver-api/dma-buf.html:
" The dma-buf subsystem provides the framework for sharing buffers for hardware (DMA)
access across multiple device drivers and subsystems, and for synchronizing asynchronous hardware access."

That's sounds pretty much like what we need.

It lives under linux/drivers/dma-buf
if it would be a video only shouldn't this be documented somewhere, and perhaps the code should be somewhere else?

Just my 2cents.

Greetings,
Michael

Quoted 107 lines

[1]:
https://urldefense.com/v3/__https://x-lore.kernel.org/io-uring/b0a336c0-
ae2f-e77f-3c5f- [email protected]/T/__;!!A3Ni8CS0y2Y!p67gKdXW2zXUxASbwbCKzXg cwxEo1R3h4AumAE6QHiSPyI3KYaz9TmGpnSIF3wu9jAKL1Q$

Quoted 10 lines
Quoted 10 lines
Quoted 1 line

Also if the idea is to this with gpus/accelerators then I'd really

like > to > see the full thing, since most likely at that point you also want > dma_fence. And once we talk dma_fence things get truly horrible from a > locking pov :-( Or well, just highly constrained and I get to review > what > iio is doing with these buffers to make sure it all fits.

There is some dma_fence action in patch #10, which is enough for the
userspace apps to use the API.

Quoted 6 lines
Quoted 2 lines

What "horribleness" are we talking about here? It doesn't look that
scary to me, but I certainly don't have the complete picture.

You need to annotate all the code involved in signalling that dma_fence using dma_fence_begin/end_signalling, and then enable full lockdep and everything.

Doesn't dma_fence_signal() do it for me? Looking at the code, it does call dma_fence_begin/end_signalling.

Cheers,
-Paul

Quoted 4 lines

You can safely assume you'll find bugs, because we even have bugs about this in gpu drivers (where that annotation isn't fully rolled out yet).

Quoted 60 lines

The tldr is that you can allocate memory in there. And a pile of other restrictions, but not being able to allocate memory (well GFP_ATOMIC is ok, but that can fail) is a very serious restriction. -Daniel

Quoted 54 lines

Cheers,
-Paul

Quoted 41 lines

Cheers, Daniel

Quoted 39 lines

Cheers,
-Paul

Alexandru Ardelean (1):
iio: buffer-dma: split iio_dma_buffer_fileio_free() function

Paul Cercueil (14):
iio: buffer-dma: Get rid of incoming/outgoing queues
iio: buffer-dma: Remove unused iio_buffer_block struct
iio: buffer-dma: Use round_down() instead of rounddown()
iio: buffer-dma: Enable buffer write support
iio: buffer-dmaengine: Support specifying buffer direction
iio: buffer-dmaengine: Enable write support
iio: core: Add new DMABUF interface infrastructure
iio: buffer-dma: Use DMABUFs instead of custom solution
iio: buffer-dma: Implement new DMABUF based userspace API
iio: buffer-dma: Boost performance using write-combine cache
setting
iio: buffer-dmaengine: Support new DMABUF based userspace API
iio: core: Add support for cyclic buffers
iio: buffer-dmaengine: Add support for cyclic buffers
Documentation: iio: Document high-speed DMABUF based API

  Documentation/driver-api/dma-buf.rst          |   2 +
  Documentation/iio/dmabuf_api.rst              |  94 +++
  Documentation/iio/index.rst                   |   2 +
  drivers/iio/adc/adi-axi-adc.c                 |   3 +-
  drivers/iio/buffer/industrialio-buffer-dma.c  | 670
++++++++++++++----
  .../buffer/industrialio-buffer-dmaengine.c    |  42 +-
  drivers/iio/industrialio-buffer.c             |  49 ++
  include/linux/iio/buffer-dma.h                |  43 +-
  include/linux/iio/buffer-dmaengine.h          |   5 +-
  include/linux/iio/buffer_impl.h               |   8 +
  include/uapi/linux/iio/buffer.h               |  30 +
  11 files changed, 783 insertions(+), 165 deletions(-)

create mode 100644 Documentation/iio/dmabuf_api.rst

Quoted 7 lines
Quoted 3 lines

--
2.33.0

--
Daniel Vetter
Software Engineer, Intel Corporation >

https://urldefense.com/v3/__http://blog.ffwll.ch__;!!A3Ni8CS0y2Y!p67g

KdXW2zXUxASbwbCKzXgcwxEo1R3h4AumAE6QHiSPyI3KYaz9TmGpnSIF3ws_ BZIjsg$

Quoted 8 lines
Quoted 2 lines

--
Daniel Vetter
Software Engineer, Intel Corporation
https://urldefense.com/v3/__http://blog.ffwll.ch__;!!A3Ni8CS0y2Y!p67gK

dXW2zXUxASbwbCKzXgcwxEo1R3h4AumAE6QHiSPyI3KYaz9TmGpnSIF3ws_B ZIjsg$

maintainer
#58 lore raw
In reply to Paul Cercueil
Quoted 10 lines · On Mon, 15 Nov 2021 14:19:10 +0000 Paul Cercueil <[email protected]> wrote:

Hi Jonathan,

This patchset introduces a new userspace interface based on DMABUF objects, to complement the existing fileio based API.

The advantage of this DMABUF based interface vs. the fileio interface, is that it avoids an extra copy of the data between the kernel and userspace. This is particularly userful for high-speed devices which produce several megabytes or even gigabytes of data per second.

I'm going to defer to others for the discussion of whether dmabuf is the right way to do this. I look forwards to some conclusions. Much like yourselves, I want to see 'a' way of doing this but I'm not knowledgeable enough about the quirks and corner cases of the different options to offer a strong opinion.

Either way, it's great to get that discussion going and it would never have happened without having actual code so thanks for doing this!

Quoted 21 lines

The first few patches [01/15] to [03/15] are not really related, but allow to reduce the size of the patches that introduce the new API.

Patch [04/15] to [06/15] enables write() support to the buffer-dma implementation of the buffer API, to continue the work done by Mihail Chindris.

Patches [07/15] to [12/15] introduce the new DMABUF based API.

Patches [13/15] and [14/15] add support for cyclic buffers, only through the new API. A cyclic buffer will be repeated on the output until the buffer is disabled.

Patch [15/15] adds documentation about the new API.

For now, the API allows you to alloc DMABUF objects and mmap() them to read or write the samples. It does not yet allow to import DMABUFs parented to other subsystems, but that should eventually be possible once it's wired.

Quoted 5 lines

This patchset is inspired by the "mmap interface" that was previously submitted by Alexandru Ardelean and Lars-Peter Clausen, so it would be great if I could get a review from you guys. Alexandru's commit was signed with his @analog.com address but he doesn't work at ADI anymore, so I believe I'll need him to sign with a new email.

Given it reflects a sign off given back then I think it's fine to leave them as analog.com but maybe he can give an Acked-by: on a different email address to reflect that he is happy for these to go forwards.

It's far from unusual to have out of date email addresses in sign offs given often code is coming out of trees many years after someone originally wrote it for a vendor tree or similar. Whilst not a lawyer I don't think that's a problem.

Jonathan

Quoted 37 lines

Cheers,
-Paul

Alexandru Ardelean (1):
iio: buffer-dma: split iio_dma_buffer_fileio_free() function

Paul Cercueil (14):
iio: buffer-dma: Get rid of incoming/outgoing queues
iio: buffer-dma: Remove unused iio_buffer_block struct
iio: buffer-dma: Use round_down() instead of rounddown()
iio: buffer-dma: Enable buffer write support
iio: buffer-dmaengine: Support specifying buffer direction
iio: buffer-dmaengine: Enable write support
iio: core: Add new DMABUF interface infrastructure
iio: buffer-dma: Use DMABUFs instead of custom solution
iio: buffer-dma: Implement new DMABUF based userspace API
iio: buffer-dma: Boost performance using write-combine cache setting
iio: buffer-dmaengine: Support new DMABUF based userspace API
iio: core: Add support for cyclic buffers
iio: buffer-dmaengine: Add support for cyclic buffers
Documentation: iio: Document high-speed DMABUF based API

 Documentation/driver-api/dma-buf.rst          |   2 +
 Documentation/iio/dmabuf_api.rst              |  94 +++
 Documentation/iio/index.rst                   |   2 +
 drivers/iio/adc/adi-axi-adc.c                 |   3 +-
 drivers/iio/buffer/industrialio-buffer-dma.c  | 670 ++++++++++++++----
 .../buffer/industrialio-buffer-dmaengine.c    |  42 +-
 drivers/iio/industrialio-buffer.c             |  49 ++
 include/linux/iio/buffer-dma.h                |  43 +-
 include/linux/iio/buffer-dmaengine.h          |   5 +-
 include/linux/iio/buffer_impl.h               |   8 +
 include/uapi/linux/iio/buffer.h               |  30 +
 11 files changed, 783 insertions(+), 165 deletions(-)
 create mode 100644 Documentation/iio/dmabuf_api.rst