Skip to contentSkip to conversation

udmbuf bug fix and some improvements

PATCH Superseded Huan Yang 2 participants 20 messages Sep 3, 2024 – Sep 9, 2024

Subsystems (3)

v5 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Huan Yang

author
#0 lore raw

v5 cover udmbuf bug fix and some improvements

This patchset attempts to fix some errors in udmabuf and remove the upin_list structure.

By this patchset, udmabuf remove page struct relay, only use pfn and folios.

Some of this fix just gather the patches which I upload before.

Any patch has passed the udmabuf self-test suite's tests. Suggested by Kasireddy, Vivek <[email protected]> Patch5 modified the unpin function, therefore running the udmabuf self-test program in a loop did not reveal any memory leaks.

Notice: Test item 6 maybe requires running the command:
echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

Changelog
===
v4 -> v5:
Patch1 doc why pre-fault failed not report error.
Patch2 no change
Patch3 update ack tag marked in v4
Patch4 split some change into patch5 and patch7, this only contains
udmabuf pin folio's function
Patch5 add helper init/deini to clean udmabuf struct initialized Patch6 remove unnecessary code
Patch7 remove reuse folio remove v4 patch4

v3 -> v4:
Patch1 fill vma area with pfn only when true access. Patch2 no change
Patch3 select VMAP_PFN
Patch4 use function pin each item.
Patch5 use pinned folio array instead udmabuf_folio struct.
v2 -> v3:
Patch1, avoid use page, instead, use pfn, and use vmf_insert_pfn map suggested-by Kasireddy, Vivek <[email protected]>

Patch2, update acked-by Kasireddy, Vivek <[email protected]> And keep the kvcalloc on the same line.

Patch3, avoid use page, instead, use pfn, then use vmap_pfn map

Patch4, split v2 patch4, single update codestyle to keep review easy.

Patch5, another way to remove udmabuf_folio

---
v1 -> v2:
Patch1, 3 Rectify the improper use of the sg table. suggested-by Christian König <[email protected]>

Patch2 add acked-by Christian K�nig <[email protected]> which marked in v1

Patch4
Modify the data structure to restore the use of pages and correct the misunderstanding of loop conditions such as "pgcnt". make sure pass self test.

remove v1's patch4

v4
https://lore.kernel.org/all/[email protected]/
v3
https://lore.kernel.org/all/[email protected]/
v2
https://lore.kernel.org/all/[email protected]/
v1
https://lore.kernel.org/all/[email protected]/

Huan Yang (7):
udmabuf: pre-fault when first page fault
udmabuf: change folios array from kmalloc to kvmalloc
udmabuf: fix vmap_udmabuf error page set
udmabuf: udmabuf_create pin folio codestyle cleanup
udmabuf: introduce udmabuf init and deinit helper
udmabuf: remove udmabuf_folio
udmabuf: reuse folio array when pin folios

 drivers/dma-buf/Kconfig   |   1 +
 drivers/dma-buf/udmabuf.c | 272 +++++++++++++++++++++++---------------
 2 files changed, 166 insertions(+), 107 deletions(-)

base-commit: c79c85875f1af04040fe4492ed94ce37ad729c4d

Signature · 1 line
2.45.2
author
#1 lore raw

PATCH v5 1/7 udmabuf: pre-fault when first page fault

The current udmabuf mmap uses a page fault to populate the vma.

However, the current udmabuf has already obtained and pinned the folio upon completion of the creation.This means that the physical memory has already been acquired, rather than being accessed dynamically.

As a result, the page fault has lost its purpose as a demanding page. Due to the fact that page fault requires trapping into kernel mode and filling in when accessing the corresponding virtual address in mmap, when creating a large size udmabuf, this represents a considerable overhead.

This patch first the pfn into page table, and then pre-fault each pfn into vma, when first access. Should know, if anything wrong when pre-fault, will not report it's error, else, report when task access it at the first time.

---
 drivers/dma-buf/udmabuf.c | 35 +++++++++++++++++++++++++++++++++--
 1 file changed, 33 insertions(+), 2 deletions(-)
drivers/dma-buf/udmabuf.c +33 -2
index 047c3cd2ceff..0a8c231a36e1 100644
@@ -43,7 +43,8 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
 	struct vm_area_struct *vma = vmf->vma;
 	struct udmabuf *ubuf = vma->vm_private_data;
 	pgoff_t pgoff = vmf->pgoff;
-	unsigned long pfn;
+	unsigned long addr, end, pfn;
+	vm_fault_t ret;
 
 	if (pgoff >= ubuf->pagecount)
 		return VM_FAULT_SIGBUS;
@@ -51,7 +52,37 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
 	pfn = folio_pfn(ubuf->folios[pgoff]);
 	pfn += ubuf->offsets[pgoff] >> PAGE_SHIFT;
 
-	return vmf_insert_pfn(vma, vmf->address, pfn);
+	ret = vmf_insert_pfn(vma, vmf->address, pfn);
+	if (ret & VM_FAULT_ERROR)
+		return ret;
+
+	/* pre fault */
+	pgoff = vma->vm_pgoff;
+	end = vma->vm_end;
+	addr = vma->vm_start;
+
+	for (; addr < end; pgoff++, addr += PAGE_SIZE) {
+		if (addr == vmf->address)
+			continue;
+
+		if (WARN_ON(pgoff >= ubuf->pagecount))
+			break;
+
+		pfn = folio_pfn(ubuf->folios[pgoff]);
+
+		pfn += ubuf->offsets[pgoff] >> PAGE_SHIFT;
+
+		/**
+		 * If something wrong, due to this vm fault success,
+		 * do not report in here, report only when true access
+		 * this addr.
+		 * So, don't update ret here, just break.
+		 */
+		if (vmf_insert_pfn(vma, addr, pfn) & VM_FAULT_ERROR)
+			break;
+	}
+
+	return ret;
 }
 
 static const struct vm_operations_struct udmabuf_vm_ops = {
Signature · 1 line
2.45.2
maintainer Acked-by on v5 1/7
#2 lore raw

RE: [PATCH v5 1/7] udmabuf: pre-fault when first page fault

Hi Huan,

Quoted 43 lines

Subject: [PATCH v5 1/7] udmabuf: pre-fault when first page fault

The current udmabuf mmap uses a page fault to populate the vma.

However, the current udmabuf has already obtained and pinned the folio upon completion of the creation.This means that the physical memory has already been acquired, rather than being accessed dynamically.

As a result, the page fault has lost its purpose as a demanding page. Due to the fact that page fault requires trapping into kernel mode and filling in when accessing the corresponding virtual address in mmap, when creating a large size udmabuf, this represents a considerable overhead.

This patch first the pfn into page table, and then pre-fault each pfn into vma, when first access. Should know, if anything wrong when pre-fault, will not report it's error, else, report when task access it at the first time.

---
 drivers/dma-buf/udmabuf.c | 35 +++++++++++++++++++++++++++++++++--
 1 file changed, 33 insertions(+), 2 deletions(-)
drivers/dma-buf/udmabuf.c +2 -1
index 047c3cd2ceff..0a8c231a36e1 100644
@@ -43,7 +43,8 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault
 *vmf)
 	struct vm_area_struct *vma = vmf->vma;
 	struct udmabuf *ubuf = vma->vm_private_data;
 	pgoff_t pgoff = vmf->pgoff;
-	unsigned long pfn;
+	unsigned long addr, end, pfn;
+	vm_fault_t ret;
 
 	if (pgoff >= ubuf->pagecount)
drivers/dma-buf/udmabuf.c +0 -0
 		return VM_FAULT_SIGBUS;
@@ -51,7 +52,37 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault
 *vmf)
 	pfn = folio_pfn(ubuf->folios[pgoff]);

drivers/dma-buf/udmabuf.c static vm_fault_t udmabuf_vm_fault(struct vm_fault · line 62

Quoted 10 lines
drivers/dma-buf/udmabuf.c +7 -1
 	pfn += ubuf->offsets[pgoff] >> PAGE_SHIFT;
 
-	return vmf_insert_pfn(vma, vmf->address, pfn);
+	ret = vmf_insert_pfn(vma, vmf->address, pfn);
+	if (ret & VM_FAULT_ERROR)
+		return ret;
+
+	/* pre fault */
+	pgoff = vma->vm_pgoff;
+	end = vma->vm_end;

Nit: use vma->vm_end directly in the loop below, as end is used only once.

drivers/dma-buf/udmabuf.c static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf) · line 72

Quoted 11 lines
drivers/dma-buf/udmabuf.c +11 -0
+	addr = vma->vm_start;
+
+	for (; addr < end; pgoff++, addr += PAGE_SIZE) {
+		if (addr == vmf->address)
+			continue;
+
+		if (WARN_ON(pgoff >= ubuf->pagecount))
+			break;
+
+		pfn = folio_pfn(ubuf->folios[pgoff]);
+

Nit: no need for a blank line here.

drivers/dma-buf/udmabuf.c static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf) · line 79

Quoted 7 lines
drivers/dma-buf/udmabuf.c +7 -0
+		pfn += ubuf->offsets[pgoff] >> PAGE_SHIFT;
+
+		/**
+		 * If something wrong, due to this vm fault success,
+		 * do not report in here, report only when true access
+		 * this addr.
+		 * So, don't update ret here, just break.

Please rewrite the above comments as:
* If the below vmf_insert_pfn() fails, we do not return an error here
* during this pre-fault step. However, an error will be returned if the
* failure occurs when the addr is truly accessed.

With that,

Quoted 11 lines
drivers/dma-buf/udmabuf.c +6 -0
+		 */
+		if (vmf_insert_pfn(vma, addr, pfn) & VM_FAULT_ERROR)
+			break;
+	}
+
+	return ret;
 }
 

static const struct vm_operations_struct udmabuf_vm_ops = {
--
2.45.2

author
#3 lore raw

Re: [PATCH v5 1/7] udmabuf: pre-fault when first page fault

Quoted 70 lines · Vivek

Hi Huan,

Quoted 43 lines

Subject: [PATCH v5 1/7] udmabuf: pre-fault when first page fault

The current udmabuf mmap uses a page fault to populate the vma.

However, the current udmabuf has already obtained and pinned the folio upon completion of the creation.This means that the physical memory has already been acquired, rather than being accessed dynamically.

As a result, the page fault has lost its purpose as a demanding page. Due to the fact that page fault requires trapping into kernel mode and filling in when accessing the corresponding virtual address in mmap, when creating a large size udmabuf, this represents a considerable overhead.

This patch first the pfn into page table, and then pre-fault each pfn into vma, when first access. Should know, if anything wrong when pre-fault, will not report it's error, else, report when task access it at the first time.

---
  drivers/dma-buf/udmabuf.c | 35 +++++++++++++++++++++++++++++++++--
  1 file changed, 33 insertions(+), 2 deletions(-)
drivers/dma-buf/udmabuf.c +2 -1
index 047c3cd2ceff..0a8c231a36e1 100644
@@ -43,7 +43,8 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault
 *vmf)
  	struct vm_area_struct *vma = vmf->vma;
  	struct udmabuf *ubuf = vma->vm_private_data;
  	pgoff_t pgoff = vmf->pgoff;
-	unsigned long pfn;
+	unsigned long addr, end, pfn;
+	vm_fault_t ret;
 
  	if (pgoff >= ubuf->pagecount)

return VM_FAULT_SIGBUS;

drivers/dma-buf/udmabuf.c +0 -0
@@ -51,7 +52,37 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault
 *vmf)
  	pfn = folio_pfn(ubuf->folios[pgoff]);

drivers/dma-buf/udmabuf.c static vm_fault_t udmabuf_vm_fault(struct vm_fault · line 62

Quoted 10 lines
drivers/dma-buf/udmabuf.c +7 -1
  	pfn += ubuf->offsets[pgoff] >> PAGE_SHIFT;
 
-	return vmf_insert_pfn(vma, vmf->address, pfn);
+	ret = vmf_insert_pfn(vma, vmf->address, pfn);
+	if (ret & VM_FAULT_ERROR)
+		return ret;
+
+	/* pre fault */
+	pgoff = vma->vm_pgoff;
+	end = vma->vm_end;

Nit: use vma->vm_end directly in the loop below, as end is used only once.

drivers/dma-buf/udmabuf.c static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf) · line 72

Quoted 11 lines
drivers/dma-buf/udmabuf.c +11 -0
+	addr = vma->vm_start;
+
+	for (; addr < end; pgoff++, addr += PAGE_SIZE) {
+		if (addr == vmf->address)
+			continue;
+
+		if (WARN_ON(pgoff >= ubuf->pagecount))
+			break;
+
+		pfn = folio_pfn(ubuf->folios[pgoff]);
+

Nit: no need for a blank line here.

Quoted 11 lines

drivers/dma-buf/udmabuf.c static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf) · line 79

Quoted 7 lines
drivers/dma-buf/udmabuf.c +7 -0
+		pfn += ubuf->offsets[pgoff] >> PAGE_SHIFT;
+
+		/**
+		 * If something wrong, due to this vm fault success,
+		 * do not report in here, report only when true access
+		 * this addr.
+		 * So, don't update ret here, just break.

Please rewrite the above comments as:
* If the below vmf_insert_pfn() fails, we do not return an error here
* during this pre-fault step. However, an error will be returned if the
* failure occurs when the addr is truly accessed.

Thank you

Quoted 2 lines

With that,

OK.

Thanks

Quoted 13 lines
Quoted 11 lines
drivers/dma-buf/udmabuf.c +6 -0
+		 */
+		if (vmf_insert_pfn(vma, addr, pfn) & VM_FAULT_ERROR)
+			break;
+	}
+
+	return ret;
  }
 

static const struct vm_operations_struct udmabuf_vm_ops = {
--
2.45.2

author
#4 lore raw

PATCH v5 2/7 udmabuf: change folios array from kmalloc to kvmalloc

In reply to Huan Yang

When PAGE_SIZE 4096, MAX_PAGE_ORDER 10, 64bit machine, page_alloc only support 4MB.
If above this, trigger this warn and return NULL.

udmabuf can change size limit, if change it to 3072(3GB), and then alloc 3GB udmabuf, will fail create.

[ 4080.876581] ------------[ cut here ]------------
[ 4080.876843] WARNING: CPU: 3 PID: 2015 at mm/page_alloc.c:4556 __alloc_pages+0x2c8/0x350
[ 4080.878839] RIP: 0010:__alloc_pages+0x2c8/0x350
[ 4080.879470] Call Trace:
[ 4080.879473]  <TASK>
[ 4080.879473]  ? __alloc_pages+0x2c8/0x350
[ 4080.879475]  ? __warn.cold+0x8e/0xe8
[ 4080.880647]  ? __alloc_pages+0x2c8/0x350
[ 4080.880909]  ? report_bug+0xff/0x140
[ 4080.881175]  ? handle_bug+0x3c/0x80
[ 4080.881556]  ? exc_invalid_op+0x17/0x70
[ 4080.881559]  ? asm_exc_invalid_op+0x1a/0x20
[ 4080.882077]  ? udmabuf_create+0x131/0x400

Because MAX_PAGE_ORDER, kmalloc can max alloc 4096 * (1 << 10), 4MB memory, each array entry is pointer(8byte), so can save 524288 pages(2GB).

Further more, costly order(order 3) may not be guaranteed that it can be applied for, due to fragmentation.

This patch change udmabuf array use kvmalloc_array, this can fallback alloc into vmalloc, which can guarantee allocation for any size and does not affect the performance of kmalloc allocations.

---
 drivers/dma-buf/udmabuf.c | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)
drivers/dma-buf/udmabuf.c +13 -13
index 0a8c231a36e1..789fba312a68 100644
@@ -111,7 +111,7 @@ static int vmap_udmabuf(struct dma_buf *buf, struct iosys_map *map)
 
 	dma_resv_assert_held(buf->resv);
 
-	pages = kmalloc_array(ubuf->pagecount, sizeof(*pages), GFP_KERNEL);
+	pages = kvmalloc_array(ubuf->pagecount, sizeof(*pages), GFP_KERNEL);
 	if (!pages)
 		return -ENOMEM;
 
@@ -119,7 +119,7 @@ static int vmap_udmabuf(struct dma_buf *buf, struct iosys_map *map)
 		pages[pg] = &ubuf->folios[pg]->page;
 
 	vaddr = vm_map_ram(pages, ubuf->pagecount, -1);
-	kfree(pages);
+	kvfree(pages);
 	if (!vaddr)
 		return -EINVAL;
 
@@ -227,8 +227,8 @@ static void release_udmabuf(struct dma_buf *buf)
 		put_sg_table(dev, ubuf->sg, DMA_BIDIRECTIONAL);
 
 	unpin_all_folios(&ubuf->unpin_list);
-	kfree(ubuf->offsets);
-	kfree(ubuf->folios);
+	kvfree(ubuf->offsets);
+	kvfree(ubuf->folios);
 	kfree(ubuf);
 }
 
@@ -353,14 +353,14 @@ static long udmabuf_create(struct miscdevice *device,
 	if (!ubuf->pagecount)
 		goto err;
 
-	ubuf->folios = kmalloc_array(ubuf->pagecount, sizeof(*ubuf->folios),
-				    GFP_KERNEL);
+	ubuf->folios = kvmalloc_array(ubuf->pagecount, sizeof(*ubuf->folios),
+				      GFP_KERNEL);
 	if (!ubuf->folios) {
 		ret = -ENOMEM;
 		goto err;
 	}
-	ubuf->offsets = kcalloc(ubuf->pagecount, sizeof(*ubuf->offsets),
-				GFP_KERNEL);
+	ubuf->offsets = kvcalloc(ubuf->pagecount, sizeof(*ubuf->offsets),
+				 GFP_KERNEL);
 	if (!ubuf->offsets) {
 		ret = -ENOMEM;
 		goto err;
@@ -374,7 +374,7 @@ static long udmabuf_create(struct miscdevice *device,
 			goto err;
 
 		pgcnt = list[i].size >> PAGE_SHIFT;
-		folios = kmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
+		folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
 		if (!folios) {
 			ret = -ENOMEM;
 			goto err;
@@ -384,7 +384,7 @@ static long udmabuf_create(struct miscdevice *device,
 		ret = memfd_pin_folios(memfd, list[i].offset, end,
 				       folios, pgcnt, &pgoff);
 		if (ret <= 0) {
-			kfree(folios);
+			kvfree(folios);
 			if (!ret)
 				ret = -EINVAL;
 			goto err;
@@ -413,7 +413,7 @@ static long udmabuf_create(struct miscdevice *device,
 			}
 		}
 
-		kfree(folios);
+		kvfree(folios);
 		fput(memfd);
 		memfd = NULL;
 	}
@@ -429,8 +429,8 @@ static long udmabuf_create(struct miscdevice *device,
 	if (memfd)
 		fput(memfd);
 	unpin_all_folios(&ubuf->unpin_list);
-	kfree(ubuf->offsets);
-	kfree(ubuf->folios);
+	kvfree(ubuf->offsets);
+	kvfree(ubuf->folios);
 	kfree(ubuf);
 	return ret;
 }
Signature · 1 line
2.45.2
author
#5 lore raw

PATCH v5 3/7 udmabuf: fix vmap_udmabuf error page set

In reply to Huan Yang

Currently vmap_udmabuf set page's array by each folio. But, ubuf->folios is only contain's the folio's head page.

That mean we repeatedly mapped the folio head page to the vmalloc area.

Due to udmabuf can use hugetlb, if HVO enabled, tail page may not exist, so, we can't use page array to map, instead, use pfn array.

By this, we removed page usage in udmabuf totally.

---
 drivers/dma-buf/Kconfig   |  1 +
 drivers/dma-buf/udmabuf.c | 22 +++++++++++++++-------
 2 files changed, 16 insertions(+), 7 deletions(-)
drivers/dma-buf/Kconfig +1 -0
index b46eb8a552d7..fee04fdb0822 100644
@@ -36,6 +36,7 @@ config UDMABUF
 	depends on DMA_SHARED_BUFFER
 	depends on MEMFD_CREATE || COMPILE_TEST
 	depends on MMU
+	select VMAP_PFN
 	help
 	  A driver to let userspace turn memfd regions into dma-bufs.
 	  Qemu can use this to create host dmabufs for guest framebuffers.
drivers/dma-buf/udmabuf.c +15 -7
index 789fba312a68..456db58446e1 100644
@@ -105,21 +105,29 @@ static int mmap_udmabuf(struct dma_buf *buf, struct vm_area_struct *vma)
 static int vmap_udmabuf(struct dma_buf *buf, struct iosys_map *map)
 {
 	struct udmabuf *ubuf = buf->priv;
-	struct page **pages;
+	unsigned long *pfns;
 	void *vaddr;
 	pgoff_t pg;
 
 	dma_resv_assert_held(buf->resv);
 
-	pages = kvmalloc_array(ubuf->pagecount, sizeof(*pages), GFP_KERNEL);
-	if (!pages)
+	/**
+	 * HVO may free tail pages, so just use pfn to map each folio
+	 * into vmalloc area.
+	 */
+	pfns = kvmalloc_array(ubuf->pagecount, sizeof(*pfns), GFP_KERNEL);
+	if (!pfns)
 		return -ENOMEM;
 
-	for (pg = 0; pg < ubuf->pagecount; pg++)
-		pages[pg] = &ubuf->folios[pg]->page;
+	for (pg = 0; pg < ubuf->pagecount; pg++) {
+		unsigned long pfn = folio_pfn(ubuf->folios[pg]);
 
-	vaddr = vm_map_ram(pages, ubuf->pagecount, -1);
-	kvfree(pages);
+		pfn += ubuf->offsets[pg] >> PAGE_SHIFT;
+		pfns[pg] = pfn;
+	}
+
+	vaddr = vmap_pfn(pfns, ubuf->pagecount, PAGE_KERNEL);
+	kvfree(pfns);
 	if (!vaddr)
 		return -EINVAL;
 
Signature · 1 line
2.45.2
author
#6 lore raw

PATCH v5 4/7 udmabuf: udmabuf_create pin folio codestyle cleanup

In reply to Huan Yang

This patch split pin folios into single function: udmabuf_pin_folios.

When record folio and offset into udmabuf_folio and offsets, the outer loop of this patch iterates through folios, while the inner loop correctly sets the folio and corresponding offset into the udmabuf starting from the offset. if reach to pgcnt or nr_folios, end of loop.

By this, more readable.

---
 drivers/dma-buf/udmabuf.c | 132 ++++++++++++++++++++------------------
 1 file changed, 71 insertions(+), 61 deletions(-)
drivers/dma-buf/udmabuf.c +71 -61
index 456db58446e1..ca2b21c5c57f 100644
@@ -330,17 +330,67 @@ static int export_udmabuf(struct udmabuf *ubuf,
 	return dma_buf_fd(buf, flags);
 }
 
+static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
+			      loff_t start, loff_t size)
+{
+	pgoff_t pgoff, pgcnt, upgcnt = ubuf->pagecount;
+	u32 cur_folio, cur_pgcnt;
+	struct folio **folios = NULL;
+	long nr_folios;
+	loff_t end;
+	int ret = 0;
+
+	pgcnt = size >> PAGE_SHIFT;
+	folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
+	if (!folios)
+		return -ENOMEM;
+
+	end = start + (pgcnt << PAGE_SHIFT) - 1;
+	nr_folios = memfd_pin_folios(memfd, start, end, folios, pgcnt, &pgoff);
+	if (nr_folios <= 0) {
+		ret = nr_folios ? nr_folios : -EINVAL;
+		goto err;
+	}
+
+	cur_pgcnt = 0;
+	for (cur_folio = 0; cur_folio < nr_folios; ++cur_folio) {
+		pgoff_t subpgoff = pgoff;
+		size_t fsize = folio_size(folios[cur_folio]);
+
+		ret = add_to_unpin_list(&ubuf->unpin_list, folios[cur_folio]);
+		if (ret < 0)
+			goto err;
+
+		for (; subpgoff < fsize; subpgoff += PAGE_SIZE) {
+			ubuf->folios[upgcnt] = folios[cur_folio];
+			ubuf->offsets[upgcnt] = subpgoff;
+			++upgcnt;
+
+			if (++cur_pgcnt >= pgcnt)
+				goto end;
+		}
+
+		/**
+		 * The term range may start with offset, so the first folio
+		 * need take care of it. And the remain folio start from 0.
+		 */
+		pgoff = 0;
+	}
+end:
+err:
+	ubuf->pagecount = upgcnt;
+	kvfree(folios);
+	return ret;
+}
+
 static long udmabuf_create(struct miscdevice *device,
 			   struct udmabuf_create_list *head,
 			   struct udmabuf_create_item *list)
 {
-	pgoff_t pgoff, pgcnt, pglimit, pgbuf = 0;
-	long nr_folios, ret = -EINVAL;
-	struct file *memfd = NULL;
-	struct folio **folios;
+	pgoff_t pgcnt = 0, pglimit;
+	long ret = -EINVAL;
 	struct udmabuf *ubuf;
-	u32 i, j, k, flags;
-	loff_t end;
+	u32 i, flags;
 
 	ubuf = kzalloc(sizeof(*ubuf), GFP_KERNEL);
 	if (!ubuf)
@@ -349,81 +399,43 @@ static long udmabuf_create(struct miscdevice *device,
 	INIT_LIST_HEAD(&ubuf->unpin_list);
 	pglimit = (size_limit_mb * 1024 * 1024) >> PAGE_SHIFT;
 	for (i = 0; i < head->count; i++) {
-		if (!IS_ALIGNED(list[i].offset, PAGE_SIZE))
+		if (!PAGE_ALIGNED(list[i].offset))
 			goto err;
-		if (!IS_ALIGNED(list[i].size, PAGE_SIZE))
+		if (!PAGE_ALIGNED(list[i].size))
 			goto err;
-		ubuf->pagecount += list[i].size >> PAGE_SHIFT;
-		if (ubuf->pagecount > pglimit)
+
+		pgcnt += list[i].size >> PAGE_SHIFT;
+		if (pgcnt > pglimit)
 			goto err;
 	}
 
-	if (!ubuf->pagecount)
+	if (!pgcnt)
 		goto err;
 
-	ubuf->folios = kvmalloc_array(ubuf->pagecount, sizeof(*ubuf->folios),
-				      GFP_KERNEL);
+	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios), GFP_KERNEL);
 	if (!ubuf->folios) {
 		ret = -ENOMEM;
 		goto err;
 	}
-	ubuf->offsets = kvcalloc(ubuf->pagecount, sizeof(*ubuf->offsets),
-				 GFP_KERNEL);
+
+	ubuf->offsets = kvcalloc(pgcnt, sizeof(*ubuf->offsets), GFP_KERNEL);
 	if (!ubuf->offsets) {
 		ret = -ENOMEM;
 		goto err;
 	}
 
-	pgbuf = 0;
 	for (i = 0; i < head->count; i++) {
-		memfd = fget(list[i].memfd);
+		struct file *memfd = fget(list[i].memfd);
+
 		ret = check_memfd_seals(memfd);
 		if (ret < 0)
 			goto err;
 
-		pgcnt = list[i].size >> PAGE_SHIFT;
-		folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
-		if (!folios) {
-			ret = -ENOMEM;
-			goto err;
-		}
-
-		end = list[i].offset + (pgcnt << PAGE_SHIFT) - 1;
-		ret = memfd_pin_folios(memfd, list[i].offset, end,
-				       folios, pgcnt, &pgoff);
-		if (ret <= 0) {
-			kvfree(folios);
-			if (!ret)
-				ret = -EINVAL;
-			goto err;
-		}
-
-		nr_folios = ret;
-		pgoff >>= PAGE_SHIFT;
-		for (j = 0, k = 0; j < pgcnt; j++) {
-			ubuf->folios[pgbuf] = folios[k];
-			ubuf->offsets[pgbuf] = pgoff << PAGE_SHIFT;
-
-			if (j == 0 || ubuf->folios[pgbuf-1] != folios[k]) {
-				ret = add_to_unpin_list(&ubuf->unpin_list,
-							folios[k]);
-				if (ret < 0) {
-					kfree(folios);
-					goto err;
-				}
-			}
-
-			pgbuf++;
-			if (++pgoff == folio_nr_pages(folios[k])) {
-				pgoff = 0;
-				if (++k == nr_folios)
-					break;
-			}
-		}
-
-		kvfree(folios);
+		ret = udmabuf_pin_folios(ubuf, memfd, list[i].offset,
+					 list[i].size);
 		fput(memfd);
-		memfd = NULL;
+		if (ret)
+			goto err;
 	}
 
 	flags = head->flags & UDMABUF_FLAGS_CLOEXEC ? O_CLOEXEC : 0;
@@ -434,8 +446,6 @@ static long udmabuf_create(struct miscdevice *device,
 	return ret;
 
 err:
-	if (memfd)
-		fput(memfd);
 	unpin_all_folios(&ubuf->unpin_list);
 	kvfree(ubuf->offsets);
 	kvfree(ubuf->folios);
Signature · 1 line
2.45.2
maintainer
#7 lore raw

RE: [PATCH v5 4/7] udmabuf: udmabuf_create pin folio codestyle cleanup

Hi Huan,

Quoted 24 lines

Subject: [PATCH v5 4/7] udmabuf: udmabuf_create pin folio codestyle cleanup

This patch split pin folios into single function: udmabuf_pin_folios.

When record folio and offset into udmabuf_folio and offsets, the outer loop of this patch iterates through folios, while the inner loop correctly sets the folio and corresponding offset into the udmabuf starting from the offset. if reach to pgcnt or nr_folios, end of loop.

By this, more readable.

---
 drivers/dma-buf/udmabuf.c | 132 ++++++++++++++++++++------------------
 1 file changed, 71 insertions(+), 61 deletions(-)
drivers/dma-buf/udmabuf.c +0 -0
index 456db58446e1..ca2b21c5c57f 100644
@@ -330,17 +330,67 @@ static int export_udmabuf(struct udmabuf *ubuf,
 	return dma_buf_fd(buf, flags);
 }

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 341

Quoted 10 lines
drivers/dma-buf/udmabuf.c +9 -0
 
+static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
+			      loff_t start, loff_t size)
+{
+	pgoff_t pgoff, pgcnt, upgcnt = ubuf->pagecount;
+	u32 cur_folio, cur_pgcnt;
+	struct folio **folios = NULL;
+	long nr_folios;
+	loff_t end;
+	int ret = 0;

Change ret's type and this function's return type to long for consistency.

Quoted 25 lines
drivers/dma-buf/udmabuf.c +24 -0
+
+	pgcnt = size >> PAGE_SHIFT;
+	folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
+	if (!folios)
+		return -ENOMEM;
+
+	end = start + (pgcnt << PAGE_SHIFT) - 1;
+	nr_folios = memfd_pin_folios(memfd, start, end, folios, pgcnt,
 &pgoff);
+	if (nr_folios <= 0) {
+		ret = nr_folios ? nr_folios : -EINVAL;
+		goto err;
+	}
+
+	cur_pgcnt = 0;
+	for (cur_folio = 0; cur_folio < nr_folios; ++cur_folio) {
+		pgoff_t subpgoff = pgoff;
+		size_t fsize = folio_size(folios[cur_folio]);
+
+		ret = add_to_unpin_list(&ubuf->unpin_list, folios[cur_folio]);
+		if (ret < 0)
+			goto err;
+
+		for (; subpgoff < fsize; subpgoff += PAGE_SIZE) {
+			ubuf->folios[upgcnt] = folios[cur_folio];

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 375

Quoted 10 lines
drivers/dma-buf/udmabuf.c +10 -0
+			ubuf->offsets[upgcnt] = subpgoff;
+			++upgcnt;
+
+			if (++cur_pgcnt >= pgcnt)
+				goto end;
+		}
+
+		/**
+		 * The term range may start with offset, so the first folio
+		 * need take care of it. And the remain folio start from 0.

The comments above are not very meaningful. Please rewrite them as:
* In a given range, only the first subpage of the first folio has an offset, that
* is returned by memfd_pin_folios(). The first subpages of other folios (in the
* range) have an offset of 0.

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 380

Quoted 5 lines
drivers/dma-buf/udmabuf.c +5 -0
+		 */
+		pgoff = 0;
+	}
+end:
+err:

No need to have two labels here. Keep end and get rid of err?

Quoted 123 lines
drivers/dma-buf/udmabuf.c +8 -6
+	ubuf->pagecount = upgcnt;
+	kvfree(folios);
+	return ret;
+}
+
 static long udmabuf_create(struct miscdevice *device,
 			   struct udmabuf_create_list *head,
 			   struct udmabuf_create_item *list)
 {
-	pgoff_t pgoff, pgcnt, pglimit, pgbuf = 0;
-	long nr_folios, ret = -EINVAL;
-	struct file *memfd = NULL;
-	struct folio **folios;
+	pgoff_t pgcnt = 0, pglimit;
+	long ret = -EINVAL;
 	struct udmabuf *ubuf;
-	u32 i, j, k, flags;
-	loff_t end;
+	u32 i, flags;
 
 	ubuf = kzalloc(sizeof(*ubuf), GFP_KERNEL);
drivers/dma-buf/udmabuf.c +15 -53
 	if (!ubuf)
@@ -349,81 +399,43 @@ static long udmabuf_create(struct miscdevice
 *device,
 	INIT_LIST_HEAD(&ubuf->unpin_list);
 	pglimit = (size_limit_mb * 1024 * 1024) >> PAGE_SHIFT;
 	for (i = 0; i < head->count; i++) {
-		if (!IS_ALIGNED(list[i].offset, PAGE_SIZE))
+		if (!PAGE_ALIGNED(list[i].offset))
 			goto err;
-		if (!IS_ALIGNED(list[i].size, PAGE_SIZE))
+		if (!PAGE_ALIGNED(list[i].size))
 			goto err;
-		ubuf->pagecount += list[i].size >> PAGE_SHIFT;
-		if (ubuf->pagecount > pglimit)
+
+		pgcnt += list[i].size >> PAGE_SHIFT;
+		if (pgcnt > pglimit)
 			goto err;
 	}
 
-	if (!ubuf->pagecount)
+	if (!pgcnt)
 		goto err;
 
-	ubuf->folios = kvmalloc_array(ubuf->pagecount, sizeof(*ubuf-
 >folios),
-				      GFP_KERNEL);
+	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios),
 GFP_KERNEL);
 	if (!ubuf->folios) {
 		ret = -ENOMEM;
 		goto err;
 	}
-	ubuf->offsets = kvcalloc(ubuf->pagecount, sizeof(*ubuf->offsets),
-				 GFP_KERNEL);
+
+	ubuf->offsets = kvcalloc(pgcnt, sizeof(*ubuf->offsets), GFP_KERNEL);
 	if (!ubuf->offsets) {
 		ret = -ENOMEM;
 		goto err;
 	}
 
-	pgbuf = 0;
 	for (i = 0; i < head->count; i++) {
-		memfd = fget(list[i].memfd);
+		struct file *memfd = fget(list[i].memfd);
+
 		ret = check_memfd_seals(memfd);
 		if (ret < 0)
 			goto err;
 
-		pgcnt = list[i].size >> PAGE_SHIFT;
-		folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
-		if (!folios) {
-			ret = -ENOMEM;
-			goto err;
-		}
-
-		end = list[i].offset + (pgcnt << PAGE_SHIFT) - 1;
-		ret = memfd_pin_folios(memfd, list[i].offset, end,
-				       folios, pgcnt, &pgoff);
-		if (ret <= 0) {
-			kvfree(folios);
-			if (!ret)
-				ret = -EINVAL;
-			goto err;
-		}
-
-		nr_folios = ret;
-		pgoff >>= PAGE_SHIFT;
-		for (j = 0, k = 0; j < pgcnt; j++) {
-			ubuf->folios[pgbuf] = folios[k];
-			ubuf->offsets[pgbuf] = pgoff << PAGE_SHIFT;
-
-			if (j == 0 || ubuf->folios[pgbuf-1] != folios[k]) {
-				ret = add_to_unpin_list(&ubuf->unpin_list,
-							folios[k]);
-				if (ret < 0) {
-					kfree(folios);
-					goto err;
-				}
-			}
-
-			pgbuf++;
-			if (++pgoff == folio_nr_pages(folios[k])) {
-				pgoff = 0;
-				if (++k == nr_folios)
-					break;
-			}
-		}
-
-		kvfree(folios);
+		ret = udmabuf_pin_folios(ubuf, memfd, list[i].offset,
+					 list[i].size);
 		fput(memfd);
-		memfd = NULL;
+		if (ret)
+			goto err;
drivers/dma-buf/udmabuf.c +0 -0
 	}
 
 	flags = head->flags & UDMABUF_FLAGS_CLOEXEC ? O_CLOEXEC : 0;

drivers/dma-buf/udmabuf.c static long udmabuf_create(struct miscdevice · line 439

Quoted 6 lines
drivers/dma-buf/udmabuf.c +0 -2
@@ -434,8 +446,6 @@ static long udmabuf_create(struct miscdevice
 *device,
 	return ret;
 
 err:
-	if (memfd)
-		fput(memfd);

I think this needs to stay because if the seals check fails, then we would not be doing fput(memfd).

Thanks,
Vivek

Quoted 5 lines
drivers/dma-buf/udmabuf.c +0 -0
 	unpin_all_folios(&ubuf->unpin_list);
 	kvfree(ubuf->offsets);

kvfree(ubuf->folios);
--
2.45.2

author
#8 lore raw

Re: [PATCH v5 4/7] udmabuf: udmabuf_create pin folio codestyle cleanup

Quoted 211 lines · Vivek

Hi Huan,

Quoted 24 lines

Subject: [PATCH v5 4/7] udmabuf: udmabuf_create pin folio codestyle cleanup

This patch split pin folios into single function: udmabuf_pin_folios.

When record folio and offset into udmabuf_folio and offsets, the outer loop of this patch iterates through folios, while the inner loop correctly sets the folio and corresponding offset into the udmabuf starting from the offset. if reach to pgcnt or nr_folios, end of loop.

By this, more readable.

---
  drivers/dma-buf/udmabuf.c | 132 ++++++++++++++++++++------------------
  1 file changed, 71 insertions(+), 61 deletions(-)
drivers/dma-buf/udmabuf.c +0 -0
index 456db58446e1..ca2b21c5c57f 100644
@@ -330,17 +330,67 @@ static int export_udmabuf(struct udmabuf *ubuf,
  	return dma_buf_fd(buf, flags);
  }

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 341

Quoted 10 lines
drivers/dma-buf/udmabuf.c +9 -0
 
+static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
+			      loff_t start, loff_t size)
+{
+	pgoff_t pgoff, pgcnt, upgcnt = ubuf->pagecount;
+	u32 cur_folio, cur_pgcnt;
+	struct folio **folios = NULL;
+	long nr_folios;
+	loff_t end;
+	int ret = 0;

Change ret's type and this function's return type to long for consistency.

Quoted 25 lines
drivers/dma-buf/udmabuf.c +24 -0
+
+	pgcnt = size >> PAGE_SHIFT;
+	folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
+	if (!folios)
+		return -ENOMEM;
+
+	end = start + (pgcnt << PAGE_SHIFT) - 1;
+	nr_folios = memfd_pin_folios(memfd, start, end, folios, pgcnt,
 &pgoff);
+	if (nr_folios <= 0) {
+		ret = nr_folios ? nr_folios : -EINVAL;
+		goto err;
+	}
+
+	cur_pgcnt = 0;
+	for (cur_folio = 0; cur_folio < nr_folios; ++cur_folio) {
+		pgoff_t subpgoff = pgoff;
+		size_t fsize = folio_size(folios[cur_folio]);
+
+		ret = add_to_unpin_list(&ubuf->unpin_list, folios[cur_folio]);
+		if (ret < 0)
+			goto err;
+
+		for (; subpgoff < fsize; subpgoff += PAGE_SIZE) {
+			ubuf->folios[upgcnt] = folios[cur_folio];

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 375

Quoted 10 lines
drivers/dma-buf/udmabuf.c +10 -0
+			ubuf->offsets[upgcnt] = subpgoff;
+			++upgcnt;
+
+			if (++cur_pgcnt >= pgcnt)
+				goto end;
+		}
+
+		/**
+		 * The term range may start with offset, so the first folio
+		 * need take care of it. And the remain folio start from 0.

The comments above are not very meaningful. Please rewrite them as:
* In a given range, only the first subpage of the first folio has an offset, that
* is returned by memfd_pin_folios(). The first subpages of other folios (in the
* range) have an offset of 0.

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 380

Quoted 5 lines
drivers/dma-buf/udmabuf.c +5 -0
+		 */
+		pgoff = 0;
+	}
+end:
+err:

No need to have two labels here. Keep end and get rid of err?

Quoted 126 lines
drivers/dma-buf/udmabuf.c +8 -6
+	ubuf->pagecount = upgcnt;
+	kvfree(folios);
+	return ret;
+}
+
  static long udmabuf_create(struct miscdevice *device,
  			   struct udmabuf_create_list *head,
  			   struct udmabuf_create_item *list)
  {
-	pgoff_t pgoff, pgcnt, pglimit, pgbuf = 0;
-	long nr_folios, ret = -EINVAL;
-	struct file *memfd = NULL;
-	struct folio **folios;
+	pgoff_t pgcnt = 0, pglimit;
+	long ret = -EINVAL;
  	struct udmabuf *ubuf;
-	u32 i, j, k, flags;
-	loff_t end;
+	u32 i, flags;
 
  	ubuf = kzalloc(sizeof(*ubuf), GFP_KERNEL);

if (!ubuf)

drivers/dma-buf/udmabuf.c +15 -53
@@ -349,81 +399,43 @@ static long udmabuf_create(struct miscdevice
 *device,
  	INIT_LIST_HEAD(&ubuf->unpin_list);
  	pglimit = (size_limit_mb * 1024 * 1024) >> PAGE_SHIFT;
  	for (i = 0; i < head->count; i++) {
-		if (!IS_ALIGNED(list[i].offset, PAGE_SIZE))
+		if (!PAGE_ALIGNED(list[i].offset))
  			goto err;
-		if (!IS_ALIGNED(list[i].size, PAGE_SIZE))
+		if (!PAGE_ALIGNED(list[i].size))
  			goto err;
-		ubuf->pagecount += list[i].size >> PAGE_SHIFT;
-		if (ubuf->pagecount > pglimit)
+
+		pgcnt += list[i].size >> PAGE_SHIFT;
+		if (pgcnt > pglimit)
  			goto err;
  	}
 
-	if (!ubuf->pagecount)
+	if (!pgcnt)
  		goto err;
 
-	ubuf->folios = kvmalloc_array(ubuf->pagecount, sizeof(*ubuf-
 > folios),
-				      GFP_KERNEL);
+	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios),
 GFP_KERNEL);
  	if (!ubuf->folios) {
  		ret = -ENOMEM;
  		goto err;
  	}
-	ubuf->offsets = kvcalloc(ubuf->pagecount, sizeof(*ubuf->offsets),
-				 GFP_KERNEL);
+
+	ubuf->offsets = kvcalloc(pgcnt, sizeof(*ubuf->offsets), GFP_KERNEL);
  	if (!ubuf->offsets) {
  		ret = -ENOMEM;
  		goto err;
  	}
 
-	pgbuf = 0;
  	for (i = 0; i < head->count; i++) {
-		memfd = fget(list[i].memfd);
+		struct file *memfd = fget(list[i].memfd);
+
  		ret = check_memfd_seals(memfd);
  		if (ret < 0)
  			goto err;
 
-		pgcnt = list[i].size >> PAGE_SHIFT;
-		folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
-		if (!folios) {
-			ret = -ENOMEM;
-			goto err;
-		}
-
-		end = list[i].offset + (pgcnt << PAGE_SHIFT) - 1;
-		ret = memfd_pin_folios(memfd, list[i].offset, end,
-				       folios, pgcnt, &pgoff);
-		if (ret <= 0) {
-			kvfree(folios);
-			if (!ret)
-				ret = -EINVAL;
-			goto err;
-		}
-
-		nr_folios = ret;
-		pgoff >>= PAGE_SHIFT;
-		for (j = 0, k = 0; j < pgcnt; j++) {
-			ubuf->folios[pgbuf] = folios[k];
-			ubuf->offsets[pgbuf] = pgoff << PAGE_SHIFT;
-
-			if (j == 0 || ubuf->folios[pgbuf-1] != folios[k]) {
-				ret = add_to_unpin_list(&ubuf->unpin_list,
-							folios[k]);
-				if (ret < 0) {
-					kfree(folios);
-					goto err;
-				}
-			}
-
-			pgbuf++;
-			if (++pgoff == folio_nr_pages(folios[k])) {
-				pgoff = 0;
-				if (++k == nr_folios)
-					break;
-			}
-		}
-
-		kvfree(folios);
+		ret = udmabuf_pin_folios(ubuf, memfd, list[i].offset,
+					 list[i].size);
  		fput(memfd);
-		memfd = NULL;
+		if (ret)
+			goto err;
  	}

  	flags = head->flags & UDMABUF_FLAGS_CLOEXEC ? O_CLOEXEC : 0;
drivers/dma-buf/udmabuf.c +0 -0
@@ -434,8 +446,6 @@ static long udmabuf_create(struct miscdevice
 *device,
  	return ret;
 
Quoted 5 lines

drivers/dma-buf/udmabuf.c static long udmabuf_create(struct miscdevice *device, · line 438

Quoted 3 lines
drivers/dma-buf/udmabuf.c +0 -2
  err:
-	if (memfd)
-		fput(memfd);

I think this needs to stay because if the seals check fails, then we would not be doing fput(memfd).

Yes, there a mistake, but I'd like set it into here:

ret = check_memfd_seals(memfd);

 		if (ret < 0) {
			fput(memfd);
 			goto err;
		}

due to only in inner look, memfd can get. and memfd change into loop var. Thanks

Quoted 9 lines

Thanks,
Vivek

Quoted 5 lines
drivers/dma-buf/udmabuf.c +0 -0
  	unpin_all_folios(&ubuf->unpin_list);
  	kvfree(ubuf->offsets);

kvfree(ubuf->folios);
--
2.45.2

maintainer
#9 lore raw

RE: [PATCH v5 4/7] udmabuf: udmabuf_create pin folio codestyle cleanup

Hi Huan,

Quoted 222 lines

Subject: Re: [PATCH v5 4/7] udmabuf: udmabuf_create pin folio codestyle cleanup

Quoted 19 lines · Vivek

Hi Huan,

Quoted 17 lines

Subject: [PATCH v5 4/7] udmabuf: udmabuf_create pin folio codestyle cleanup

This patch split pin folios into single function: udmabuf_pin_folios.

When record folio and offset into udmabuf_folio and offsets, the outer loop of this patch iterates through folios, while the inner loop correctly sets the folio and corresponding offset into the udmabuf starting from the offset. if reach to pgcnt or nr_folios, end of loop.

By this, more readable.

---
  drivers/dma-buf/udmabuf.c | 132 ++++++++++++++++++++------------------
  1 file changed, 71 insertions(+), 61 deletions(-)
Quoted 5 lines

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf

Quoted 5 lines
drivers/dma-buf/udmabuf.c +0 -0
index 456db58446e1..ca2b21c5c57f 100644
@@ -330,17 +330,67 @@ static int export_udmabuf(struct udmabuf

*ubuf,

Quoted 44 lines

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 341

Quoted 12 lines
drivers/dma-buf/udmabuf.c +9 -0
  	return dma_buf_fd(buf, flags);
  }
 
+static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
+			      loff_t start, loff_t size)
+{
+	pgoff_t pgoff, pgcnt, upgcnt = ubuf->pagecount;
+	u32 cur_folio, cur_pgcnt;
+	struct folio **folios = NULL;
+	long nr_folios;
+	loff_t end;
+	int ret = 0;

Change ret's type and this function's return type to long for consistency.

Quoted 30 lines
drivers/dma-buf/udmabuf.c +29 -0
+
+	pgcnt = size >> PAGE_SHIFT;
+	folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
+	if (!folios)
+		return -ENOMEM;
+
+	end = start + (pgcnt << PAGE_SHIFT) - 1;
+	nr_folios = memfd_pin_folios(memfd, start, end, folios, pgcnt,
 &pgoff);
+	if (nr_folios <= 0) {
+		ret = nr_folios ? nr_folios : -EINVAL;
+		goto err;
+	}
+
+	cur_pgcnt = 0;
+	for (cur_folio = 0; cur_folio < nr_folios; ++cur_folio) {
+		pgoff_t subpgoff = pgoff;
+		size_t fsize = folio_size(folios[cur_folio]);
+
+		ret = add_to_unpin_list(&ubuf->unpin_list, folios[cur_folio]);
+		if (ret < 0)
+			goto err;
+
+		for (; subpgoff < fsize; subpgoff += PAGE_SIZE) {
+			ubuf->folios[upgcnt] = folios[cur_folio];
+			ubuf->offsets[upgcnt] = subpgoff;
+			++upgcnt;
+
+			if (++cur_pgcnt >= pgcnt)
+				goto end;
Quoted 8 lines

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 375

Quoted 5 lines
drivers/dma-buf/udmabuf.c +5 -0
+		}
+
+		/**
+		 * The term range may start with offset, so the first folio
+		 * need take care of it. And the remain folio start from 0.

The comments above are not very meaningful. Please rewrite them as:
* In a given range, only the first subpage of the first folio has an offset, that
* is returned by memfd_pin_folios(). The first subpages of other folios (in

the

Quoted 135 lines

* range) have an offset of 0.

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 380

Quoted 5 lines
drivers/dma-buf/udmabuf.c +5 -0
+		 */
+		pgoff = 0;
+	}
+end:
+err:

No need to have two labels here. Keep end and get rid of err?

Quoted 126 lines
drivers/dma-buf/udmabuf.c +8 -6
+	ubuf->pagecount = upgcnt;
+	kvfree(folios);
+	return ret;
+}
+
  static long udmabuf_create(struct miscdevice *device,
  			   struct udmabuf_create_list *head,
  			   struct udmabuf_create_item *list)
  {
-	pgoff_t pgoff, pgcnt, pglimit, pgbuf = 0;
-	long nr_folios, ret = -EINVAL;
-	struct file *memfd = NULL;
-	struct folio **folios;
+	pgoff_t pgcnt = 0, pglimit;
+	long ret = -EINVAL;
  	struct udmabuf *ubuf;
-	u32 i, j, k, flags;
-	loff_t end;
+	u32 i, flags;
 
  	ubuf = kzalloc(sizeof(*ubuf), GFP_KERNEL);

if (!ubuf)

drivers/dma-buf/udmabuf.c +15 -53
@@ -349,81 +399,43 @@ static long udmabuf_create(struct miscdevice
 *device,
  	INIT_LIST_HEAD(&ubuf->unpin_list);
  	pglimit = (size_limit_mb * 1024 * 1024) >> PAGE_SHIFT;
  	for (i = 0; i < head->count; i++) {
-		if (!IS_ALIGNED(list[i].offset, PAGE_SIZE))
+		if (!PAGE_ALIGNED(list[i].offset))
  			goto err;
-		if (!IS_ALIGNED(list[i].size, PAGE_SIZE))
+		if (!PAGE_ALIGNED(list[i].size))
  			goto err;
-		ubuf->pagecount += list[i].size >> PAGE_SHIFT;
-		if (ubuf->pagecount > pglimit)
+
+		pgcnt += list[i].size >> PAGE_SHIFT;
+		if (pgcnt > pglimit)
  			goto err;
  	}
 
-	if (!ubuf->pagecount)
+	if (!pgcnt)
  		goto err;
 
-	ubuf->folios = kvmalloc_array(ubuf->pagecount, sizeof(*ubuf-
 > folios),
-				      GFP_KERNEL);
+	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios),
 GFP_KERNEL);
  	if (!ubuf->folios) {
  		ret = -ENOMEM;
  		goto err;
  	}
-	ubuf->offsets = kvcalloc(ubuf->pagecount, sizeof(*ubuf->offsets),
-				 GFP_KERNEL);
+
+	ubuf->offsets = kvcalloc(pgcnt, sizeof(*ubuf->offsets), GFP_KERNEL);
  	if (!ubuf->offsets) {
  		ret = -ENOMEM;
  		goto err;
  	}
 
-	pgbuf = 0;
  	for (i = 0; i < head->count; i++) {
-		memfd = fget(list[i].memfd);
+		struct file *memfd = fget(list[i].memfd);
+
  		ret = check_memfd_seals(memfd);
  		if (ret < 0)
  			goto err;
 
-		pgcnt = list[i].size >> PAGE_SHIFT;
-		folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
-		if (!folios) {
-			ret = -ENOMEM;
-			goto err;
-		}
-
-		end = list[i].offset + (pgcnt << PAGE_SHIFT) - 1;
-		ret = memfd_pin_folios(memfd, list[i].offset, end,
-				       folios, pgcnt, &pgoff);
-		if (ret <= 0) {
-			kvfree(folios);
-			if (!ret)
-				ret = -EINVAL;
-			goto err;
-		}
-
-		nr_folios = ret;
-		pgoff >>= PAGE_SHIFT;
-		for (j = 0, k = 0; j < pgcnt; j++) {
-			ubuf->folios[pgbuf] = folios[k];
-			ubuf->offsets[pgbuf] = pgoff << PAGE_SHIFT;
-
-			if (j == 0 || ubuf->folios[pgbuf-1] != folios[k]) {
-				ret = add_to_unpin_list(&ubuf->unpin_list,
-							folios[k]);
-				if (ret < 0) {
-					kfree(folios);
-					goto err;
-				}
-			}
-
-			pgbuf++;
-			if (++pgoff == folio_nr_pages(folios[k])) {
-				pgoff = 0;
-				if (++k == nr_folios)
-					break;
-			}
-		}
-
-		kvfree(folios);
+		ret = udmabuf_pin_folios(ubuf, memfd, list[i].offset,
+					 list[i].size);
  		fput(memfd);
-		memfd = NULL;
+		if (ret)
+			goto err;
  	}

  	flags = head->flags & UDMABUF_FLAGS_CLOEXEC ? O_CLOEXEC : 0;
drivers/dma-buf/udmabuf.c +0 -0
@@ -434,8 +446,6 @@ static long udmabuf_create(struct miscdevice
 *device,
  	return ret;
 
Quoted 4 lines

drivers/dma-buf/udmabuf.c static long udmabuf_create(struct miscdevice *device, · line 438

Quoted 3 lines
drivers/dma-buf/udmabuf.c +0 -2
  err:
-	if (memfd)
-		fput(memfd);

I think this needs to stay because if the seals check fails, then we would not

Quoted 7 lines

be

Quoted 1 line

doing fput(memfd).

Yes, there a mistake, but I'd like set it into here:

  		ret = check_memfd_seals(memfd);
  		if (ret < 0) {

You still need an if (memfd) check here because check_memfd_seals() might do:

        if (!memfd)
                return -EBADFD;

Thanks,
Vivek

Quoted 15 lines
			fput(memfd);
  			goto err;
		}

due to only in inner look, memfd can get. and memfd change into loop var. Thanks

Quoted 9 lines

Thanks,
Vivek

Quoted 5 lines
drivers/dma-buf/udmabuf.c +0 -0
  	unpin_all_folios(&ubuf->unpin_list);
  	kvfree(ubuf->offsets);

kvfree(ubuf->folios);
--
2.45.2

author
#10 lore raw

Re: [PATCH v5 4/7] udmabuf: udmabuf_create pin folio codestyle cleanup

Quoted 225 lines · Vivek

Hi Huan,

Quoted 223 lines

Subject: Re: [PATCH v5 4/7] udmabuf: udmabuf_create pin folio codestyle cleanup

Quoted 19 lines · Vivek

Hi Huan,

Quoted 17 lines

Subject: [PATCH v5 4/7] udmabuf: udmabuf_create pin folio codestyle cleanup

This patch split pin folios into single function: udmabuf_pin_folios.

When record folio and offset into udmabuf_folio and offsets, the outer loop of this patch iterates through folios, while the inner loop correctly sets the folio and corresponding offset into the udmabuf starting from the offset. if reach to pgcnt or nr_folios, end of loop.

By this, more readable.

---
   drivers/dma-buf/udmabuf.c | 132 ++++++++++++++++++++------------------
   1 file changed, 71 insertions(+), 61 deletions(-)
Quoted 5 lines

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf

Quoted 5 lines
drivers/dma-buf/udmabuf.c +0 -0
index 456db58446e1..ca2b21c5c57f 100644
@@ -330,17 +330,67 @@ static int export_udmabuf(struct udmabuf

*ubuf,

Quoted 44 lines

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 341

Quoted 12 lines
drivers/dma-buf/udmabuf.c +9 -0
   	return dma_buf_fd(buf, flags);
   }
 
+static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
+			      loff_t start, loff_t size)
+{
+	pgoff_t pgoff, pgcnt, upgcnt = ubuf->pagecount;
+	u32 cur_folio, cur_pgcnt;
+	struct folio **folios = NULL;
+	long nr_folios;
+	loff_t end;
+	int ret = 0;

Change ret's type and this function's return type to long for consistency.

Quoted 30 lines
drivers/dma-buf/udmabuf.c +29 -0
+
+	pgcnt = size >> PAGE_SHIFT;
+	folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
+	if (!folios)
+		return -ENOMEM;
+
+	end = start + (pgcnt << PAGE_SHIFT) - 1;
+	nr_folios = memfd_pin_folios(memfd, start, end, folios, pgcnt,
 &pgoff);
+	if (nr_folios <= 0) {
+		ret = nr_folios ? nr_folios : -EINVAL;
+		goto err;
+	}
+
+	cur_pgcnt = 0;
+	for (cur_folio = 0; cur_folio < nr_folios; ++cur_folio) {
+		pgoff_t subpgoff = pgoff;
+		size_t fsize = folio_size(folios[cur_folio]);
+
+		ret = add_to_unpin_list(&ubuf->unpin_list, folios[cur_folio]);
+		if (ret < 0)
+			goto err;
+
+		for (; subpgoff < fsize; subpgoff += PAGE_SIZE) {
+			ubuf->folios[upgcnt] = folios[cur_folio];
+			ubuf->offsets[upgcnt] = subpgoff;
+			++upgcnt;
+
+			if (++cur_pgcnt >= pgcnt)
+				goto end;
Quoted 8 lines

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 375

Quoted 5 lines
drivers/dma-buf/udmabuf.c +5 -0
+		}
+
+		/**
+		 * The term range may start with offset, so the first folio
+		 * need take care of it. And the remain folio start from 0.

The comments above are not very meaningful. Please rewrite them as:
* In a given range, only the first subpage of the first folio has an offset, that
* is returned by memfd_pin_folios(). The first subpages of other folios (in

the

Quoted 135 lines

* range) have an offset of 0.

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 380

Quoted 5 lines
drivers/dma-buf/udmabuf.c +5 -0
+		 */
+		pgoff = 0;
+	}
+end:
+err:

No need to have two labels here. Keep end and get rid of err?

Quoted 126 lines
drivers/dma-buf/udmabuf.c +8 -6
+	ubuf->pagecount = upgcnt;
+	kvfree(folios);
+	return ret;
+}
+
   static long udmabuf_create(struct miscdevice *device,
   			   struct udmabuf_create_list *head,
   			   struct udmabuf_create_item *list)
   {
-	pgoff_t pgoff, pgcnt, pglimit, pgbuf = 0;
-	long nr_folios, ret = -EINVAL;
-	struct file *memfd = NULL;
-	struct folio **folios;
+	pgoff_t pgcnt = 0, pglimit;
+	long ret = -EINVAL;
   	struct udmabuf *ubuf;
-	u32 i, j, k, flags;
-	loff_t end;
+	u32 i, flags;
 
   	ubuf = kzalloc(sizeof(*ubuf), GFP_KERNEL);
   	if (!ubuf)
drivers/dma-buf/udmabuf.c +15 -53
@@ -349,81 +399,43 @@ static long udmabuf_create(struct miscdevice
 *device,
   	INIT_LIST_HEAD(&ubuf->unpin_list);
   	pglimit = (size_limit_mb * 1024 * 1024) >> PAGE_SHIFT;
   	for (i = 0; i < head->count; i++) {
-		if (!IS_ALIGNED(list[i].offset, PAGE_SIZE))
+		if (!PAGE_ALIGNED(list[i].offset))
   			goto err;
-		if (!IS_ALIGNED(list[i].size, PAGE_SIZE))
+		if (!PAGE_ALIGNED(list[i].size))
   			goto err;
-		ubuf->pagecount += list[i].size >> PAGE_SHIFT;
-		if (ubuf->pagecount > pglimit)
+
+		pgcnt += list[i].size >> PAGE_SHIFT;
+		if (pgcnt > pglimit)
   			goto err;
   	}
 
-	if (!ubuf->pagecount)
+	if (!pgcnt)
   		goto err;
 
-	ubuf->folios = kvmalloc_array(ubuf->pagecount, sizeof(*ubuf-
 > folios),
-				      GFP_KERNEL);
+	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios),
 GFP_KERNEL);
   	if (!ubuf->folios) {
   		ret = -ENOMEM;
   		goto err;
   	}
-	ubuf->offsets = kvcalloc(ubuf->pagecount, sizeof(*ubuf->offsets),
-				 GFP_KERNEL);
+
+	ubuf->offsets = kvcalloc(pgcnt, sizeof(*ubuf->offsets), GFP_KERNEL);
   	if (!ubuf->offsets) {
   		ret = -ENOMEM;
   		goto err;
   	}
 
-	pgbuf = 0;
   	for (i = 0; i < head->count; i++) {
-		memfd = fget(list[i].memfd);
+		struct file *memfd = fget(list[i].memfd);
+
   		ret = check_memfd_seals(memfd);
   		if (ret < 0)
   			goto err;
 
-		pgcnt = list[i].size >> PAGE_SHIFT;
-		folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
-		if (!folios) {
-			ret = -ENOMEM;
-			goto err;
-		}
-
-		end = list[i].offset + (pgcnt << PAGE_SHIFT) - 1;
-		ret = memfd_pin_folios(memfd, list[i].offset, end,
-				       folios, pgcnt, &pgoff);
-		if (ret <= 0) {
-			kvfree(folios);
-			if (!ret)
-				ret = -EINVAL;
-			goto err;
-		}
-
-		nr_folios = ret;
-		pgoff >>= PAGE_SHIFT;
-		for (j = 0, k = 0; j < pgcnt; j++) {
-			ubuf->folios[pgbuf] = folios[k];
-			ubuf->offsets[pgbuf] = pgoff << PAGE_SHIFT;
-
-			if (j == 0 || ubuf->folios[pgbuf-1] != folios[k]) {
-				ret = add_to_unpin_list(&ubuf->unpin_list,
-							folios[k]);
-				if (ret < 0) {
-					kfree(folios);
-					goto err;
-				}
-			}
-
-			pgbuf++;
-			if (++pgoff == folio_nr_pages(folios[k])) {
-				pgoff = 0;
-				if (++k == nr_folios)
-					break;
-			}
-		}
-
-		kvfree(folios);
+		ret = udmabuf_pin_folios(ubuf, memfd, list[i].offset,
+					 list[i].size);
   		fput(memfd);
-		memfd = NULL;
+		if (ret)
+			goto err;
   	}

   	flags = head->flags & UDMABUF_FLAGS_CLOEXEC ? O_CLOEXEC : 0;
drivers/dma-buf/udmabuf.c +0 -0
@@ -434,8 +446,6 @@ static long udmabuf_create(struct miscdevice
 *device,
   	return ret;
 
Quoted 4 lines

drivers/dma-buf/udmabuf.c static long udmabuf_create(struct miscdevice *device, · line 438

Quoted 3 lines
drivers/dma-buf/udmabuf.c +0 -2
   err:
-	if (memfd)
-		fput(memfd);

I think this needs to stay because if the seals check fails, then we would not

be

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

doing fput(memfd).

Yes, there a mistake, but I'd like set it into here:

   		ret = check_memfd_seals(memfd);
   		if (ret < 0) {

You still need an if (memfd) check here because check_memfd_seals() might do:

         if (!memfd)
                 return -EBADFD;

Yes, fput do not check if pointer is NULL.

Thanks.

Quoted 18 lines

Thanks,
Vivek

Quoted 14 lines
			fput(memfd);
   			goto err;
		}

due to only in inner look, memfd can get. and memfd change into loop var. Thanks

Quoted 8 lines

Thanks,
Vivek

Quoted 5 lines
drivers/dma-buf/udmabuf.c +0 -0
   	unpin_all_folios(&ubuf->unpin_list);
   	kvfree(ubuf->offsets);
   	kvfree(ubuf->folios);

--
2.45.2

author
#11 lore raw

PATCH v5 5/7 udmabuf: introduce udmabuf init and deinit helper

In reply to Huan Yang

After udmabuf is allocated, its resources need to be initialized, including various array structures. The current array structure has already been greatly expanded.

Also, before udmabuf needs to be kfree, the occupied resources need to be released.

This part is repetitive and maybe overlooked.

This patch give a helper function when init and deinit, by this, deduce duplicate code.

---
 drivers/dma-buf/udmabuf.c | 52 +++++++++++++++++++++++----------------
 1 file changed, 31 insertions(+), 21 deletions(-)
drivers/dma-buf/udmabuf.c +31 -21
index ca2b21c5c57f..254d9ec3d9f3 100644
@@ -226,6 +226,28 @@ static int add_to_unpin_list(struct list_head *unpin_list,
 	return 0;
 }
 
+static __always_inline int init_udmabuf(struct udmabuf *ubuf, pgoff_t pgcnt)
+{
+	INIT_LIST_HEAD(&ubuf->unpin_list);
+
+	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios), GFP_KERNEL);
+	if (!ubuf->folios)
+		return -ENOMEM;
+
+	ubuf->offsets = kvcalloc(pgcnt, sizeof(*ubuf->offsets), GFP_KERNEL);
+	if (!ubuf->offsets)
+		return -ENOMEM;
+
+	return 0;
+}
+
+static __always_inline void deinit_udmabuf(struct udmabuf *ubuf)
+{
+	unpin_all_folios(&ubuf->unpin_list);
+	kvfree(ubuf->offsets);
+	kvfree(ubuf->folios);
+}
+
 static void release_udmabuf(struct dma_buf *buf)
 {
 	struct udmabuf *ubuf = buf->priv;
@@ -234,9 +256,7 @@ static void release_udmabuf(struct dma_buf *buf)
 	if (ubuf->sg)
 		put_sg_table(dev, ubuf->sg, DMA_BIDIRECTIONAL);
 
-	unpin_all_folios(&ubuf->unpin_list);
-	kvfree(ubuf->offsets);
-	kvfree(ubuf->folios);
+	deinit_udmabuf(ubuf);
 	kfree(ubuf);
 }
 
@@ -396,33 +416,24 @@ static long udmabuf_create(struct miscdevice *device,
 	if (!ubuf)
 		return -ENOMEM;
 
-	INIT_LIST_HEAD(&ubuf->unpin_list);
 	pglimit = (size_limit_mb * 1024 * 1024) >> PAGE_SHIFT;
 	for (i = 0; i < head->count; i++) {
 		if (!PAGE_ALIGNED(list[i].offset))
-			goto err;
+			goto err_noinit;
 		if (!PAGE_ALIGNED(list[i].size))
-			goto err;
+			goto err_noinit;
 
 		pgcnt += list[i].size >> PAGE_SHIFT;
 		if (pgcnt > pglimit)
-			goto err;
+			goto err_noinit;
 	}
 
 	if (!pgcnt)
-		goto err;
+		goto err_noinit;
 
-	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios), GFP_KERNEL);
-	if (!ubuf->folios) {
-		ret = -ENOMEM;
+	ret = init_udmabuf(ubuf, pgcnt);
+	if (ret)
 		goto err;
-	}
-
-	ubuf->offsets = kvcalloc(pgcnt, sizeof(*ubuf->offsets), GFP_KERNEL);
-	if (!ubuf->offsets) {
-		ret = -ENOMEM;
-		goto err;
-	}
 
 	for (i = 0; i < head->count; i++) {
 		struct file *memfd = fget(list[i].memfd);
@@ -446,9 +457,8 @@ static long udmabuf_create(struct miscdevice *device,
 	return ret;
 
 err:
-	unpin_all_folios(&ubuf->unpin_list);
-	kvfree(ubuf->offsets);
-	kvfree(ubuf->folios);
+	deinit_udmabuf(ubuf);
+err_noinit:
 	kfree(ubuf);
 	return ret;
 }
Signature · 1 line
2.45.2
maintainer Acked-by on v5 5/7
#12 lore raw

RE: [PATCH v5 5/7] udmabuf: introduce udmabuf init and deinit helper

Hi Huan,

Quoted 6 lines

Subject: [PATCH v5 5/7] udmabuf: introduce udmabuf init and deinit helper

After udmabuf is allocated, its resources need to be initialized, including various array structures. The current array structure has already been greatly expanded.

Quoted 7 lines

Also, before udmabuf needs to be kfree, the occupied resources need to be released.

This part is repetitive and maybe overlooked.

This patch give a helper function when init and deinit, by this, deduce duplicate code.

*reduce

If possible, please try to improve the wording and grammatical correctness in the commit messages of other patches as well.

Quoted 96 lines
---
 drivers/dma-buf/udmabuf.c | 52 +++++++++++++++++++++++----------------
 1 file changed, 31 insertions(+), 21 deletions(-)
drivers/dma-buf/udmabuf.c +22 -0
index ca2b21c5c57f..254d9ec3d9f3 100644
@@ -226,6 +226,28 @@ static int add_to_unpin_list(struct list_head
 *unpin_list,
 	return 0;
 }
 
+static __always_inline int init_udmabuf(struct udmabuf *ubuf, pgoff_t
 pgcnt)
+{
+	INIT_LIST_HEAD(&ubuf->unpin_list);
+
+	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios),
 GFP_KERNEL);
+	if (!ubuf->folios)
+		return -ENOMEM;
+
+	ubuf->offsets = kvcalloc(pgcnt, sizeof(*ubuf->offsets), GFP_KERNEL);
+	if (!ubuf->offsets)
+		return -ENOMEM;
+
+	return 0;
+}
+
+static __always_inline void deinit_udmabuf(struct udmabuf *ubuf)
+{
+	unpin_all_folios(&ubuf->unpin_list);
+	kvfree(ubuf->offsets);
+	kvfree(ubuf->folios);
+}
+

static void release_udmabuf(struct dma_buf *buf)
{

drivers/dma-buf/udmabuf.c +7 -18
 	struct udmabuf *ubuf = buf->priv;
@@ -234,9 +256,7 @@ static void release_udmabuf(struct dma_buf *buf)
 	if (ubuf->sg)
 		put_sg_table(dev, ubuf->sg, DMA_BIDIRECTIONAL);
 
-	unpin_all_folios(&ubuf->unpin_list);
-	kvfree(ubuf->offsets);
-	kvfree(ubuf->folios);
+	deinit_udmabuf(ubuf);
 	kfree(ubuf);
 }
 
@@ -396,33 +416,24 @@ static long udmabuf_create(struct miscdevice
 *device,
 	if (!ubuf)
 		return -ENOMEM;
 
-	INIT_LIST_HEAD(&ubuf->unpin_list);
 	pglimit = (size_limit_mb * 1024 * 1024) >> PAGE_SHIFT;
 	for (i = 0; i < head->count; i++) {
 		if (!PAGE_ALIGNED(list[i].offset))
-			goto err;
+			goto err_noinit;
 		if (!PAGE_ALIGNED(list[i].size))
-			goto err;
+			goto err_noinit;
 
 		pgcnt += list[i].size >> PAGE_SHIFT;
 		if (pgcnt > pglimit)
-			goto err;
+			goto err_noinit;
 	}
 
 	if (!pgcnt)
-		goto err;
+		goto err_noinit;
 
-	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios),
 GFP_KERNEL);
-	if (!ubuf->folios) {
-		ret = -ENOMEM;
+	ret = init_udmabuf(ubuf, pgcnt);
+	if (ret)
 		goto err;
-	}
-
-	ubuf->offsets = kvcalloc(pgcnt, sizeof(*ubuf->offsets), GFP_KERNEL);
-	if (!ubuf->offsets) {
-		ret = -ENOMEM;
-		goto err;
-	}
 
drivers/dma-buf/udmabuf.c +0 -0
 	for (i = 0; i < head->count; i++) {
 		struct file *memfd = fget(list[i].memfd);

drivers/dma-buf/udmabuf.c static long udmabuf_create(struct miscdevice · line 462

Quoted 9 lines
drivers/dma-buf/udmabuf.c +2 -3
@@ -446,9 +457,8 @@ static long udmabuf_create(struct miscdevice
 *device,
 	return ret;
 
 err:
-	unpin_all_folios(&ubuf->unpin_list);
-	kvfree(ubuf->offsets);
-	kvfree(ubuf->folios);
+	deinit_udmabuf(ubuf);
+err_noinit:

I don't really see the need for this new label, but I guess it makes things a bit clear.

Quoted 5 lines
drivers/dma-buf/udmabuf.c +0 -0
 	kfree(ubuf);
 	return ret;

}
--
2.45.2

author
#13 lore raw

Re: [PATCH v5 5/7] udmabuf: introduce udmabuf init and deinit helper

Quoted 11 lines · Vivek

Hi Huan,

Quoted 9 lines

Subject: [PATCH v5 5/7] udmabuf: introduce udmabuf init and deinit helper

After udmabuf is allocated, its resources need to be initialized, including various array structures. The current array structure has already been greatly expanded.

Also, before udmabuf needs to be kfree, the occupied resources need to be released.

Quoted 8 lines
Quoted 4 lines

This part is repetitive and maybe overlooked.

This patch give a helper function when init and deinit, by this, deduce duplicate code.

*reduce

If possible, please try to improve the wording and grammatical correctness in the commit messages of other patches as well.

I'll fix it in next-version

Quoted 99 lines
Quoted 99 lines
---
  drivers/dma-buf/udmabuf.c | 52 +++++++++++++++++++++++----------------
  1 file changed, 31 insertions(+), 21 deletions(-)
drivers/dma-buf/udmabuf.c +22 -0
index ca2b21c5c57f..254d9ec3d9f3 100644
@@ -226,6 +226,28 @@ static int add_to_unpin_list(struct list_head
 *unpin_list,
  	return 0;
  }
 
+static __always_inline int init_udmabuf(struct udmabuf *ubuf, pgoff_t
 pgcnt)
+{
+	INIT_LIST_HEAD(&ubuf->unpin_list);
+
+	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios),
 GFP_KERNEL);
+	if (!ubuf->folios)
+		return -ENOMEM;
+
+	ubuf->offsets = kvcalloc(pgcnt, sizeof(*ubuf->offsets), GFP_KERNEL);
+	if (!ubuf->offsets)
+		return -ENOMEM;
+
+	return 0;
+}
+
+static __always_inline void deinit_udmabuf(struct udmabuf *ubuf)
+{
+	unpin_all_folios(&ubuf->unpin_list);
+	kvfree(ubuf->offsets);
+	kvfree(ubuf->folios);
+}
+
  static void release_udmabuf(struct dma_buf *buf)
  {
  	struct udmabuf *ubuf = buf->priv;
drivers/dma-buf/udmabuf.c +7 -18
@@ -234,9 +256,7 @@ static void release_udmabuf(struct dma_buf *buf)
  	if (ubuf->sg)
  		put_sg_table(dev, ubuf->sg, DMA_BIDIRECTIONAL);
 
-	unpin_all_folios(&ubuf->unpin_list);
-	kvfree(ubuf->offsets);
-	kvfree(ubuf->folios);
+	deinit_udmabuf(ubuf);
  	kfree(ubuf);
  }
 
@@ -396,33 +416,24 @@ static long udmabuf_create(struct miscdevice
 *device,
  	if (!ubuf)
  		return -ENOMEM;
 
-	INIT_LIST_HEAD(&ubuf->unpin_list);
  	pglimit = (size_limit_mb * 1024 * 1024) >> PAGE_SHIFT;
  	for (i = 0; i < head->count; i++) {
  		if (!PAGE_ALIGNED(list[i].offset))
-			goto err;
+			goto err_noinit;
  		if (!PAGE_ALIGNED(list[i].size))
-			goto err;
+			goto err_noinit;
 
  		pgcnt += list[i].size >> PAGE_SHIFT;
  		if (pgcnt > pglimit)
-			goto err;
+			goto err_noinit;
  	}
 
  	if (!pgcnt)
-		goto err;
+		goto err_noinit;
 
-	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios),
 GFP_KERNEL);
-	if (!ubuf->folios) {
-		ret = -ENOMEM;
+	ret = init_udmabuf(ubuf, pgcnt);
+	if (ret)
  		goto err;
-	}
-
-	ubuf->offsets = kvcalloc(pgcnt, sizeof(*ubuf->offsets), GFP_KERNEL);
-	if (!ubuf->offsets) {
-		ret = -ENOMEM;
-		goto err;
-	}
 
  	for (i = 0; i < head->count; i++) {
  		struct file *memfd = fget(list[i].memfd);
drivers/dma-buf/udmabuf.c +0 -0
@@ -446,9 +457,8 @@ static long udmabuf_create(struct miscdevice
 *device,
  	return ret;
 
Quoted 8 lines

drivers/dma-buf/udmabuf.c static long udmabuf_create(struct miscdevice *device, · line 461

Quoted 6 lines
drivers/dma-buf/udmabuf.c +2 -3
  err:
-	unpin_all_folios(&ubuf->unpin_list);
-	kvfree(ubuf->offsets);
-	kvfree(ubuf->folios);
+	deinit_udmabuf(ubuf);
+err_noinit:

I don't really see the need for this new label, but I guess it makes things a bit clear.

If not this, each list err will need kfree, I think use this more clear.

Thank you. :)

Quoted 8 lines
Quoted 5 lines
drivers/dma-buf/udmabuf.c +0 -0
  	kfree(ubuf);
  	return ret;

}
--
2.45.2

author
#14 lore raw

PATCH v5 6/7 udmabuf: remove udmabuf_folio

In reply to Huan Yang

Currently, udmabuf handles folio by creating an unpin list to record each folio obtained from the list and unpinning them when released. To maintain this approach, many data structures have been established.

However, maintaining this type of data structure requires a significant amount of memory and traversing the list is a substantial overhead, which is not friendly to the CPU cache.

Considering that during creation, we arranged the folio array in the order of pin and set the offset according to pgcnt.

We actually don't need to use unpin_list to unpin during release. Instead, we can iterate through the folios array during release and unpin any folio that is different from the ones previously accessed.

By this, not only saves the overhead of the udmabuf_folio data structure but also makes array access more cache-friendly.

---
 drivers/dma-buf/udmabuf.c | 65 +++++++++++++++++----------------------
 1 file changed, 29 insertions(+), 36 deletions(-)
drivers/dma-buf/udmabuf.c +29 -36
index 254d9ec3d9f3..d449c1fd67a5 100644
@@ -27,15 +27,21 @@ MODULE_PARM_DESC(size_limit_mb, "Max size of a dmabuf, in megabytes. Default is
 struct udmabuf {
 	pgoff_t pagecount;
 	struct folio **folios;
+
+	/**
+	 * Unlike folios, pinned_folios is only used for unpin.
+	 * So, nr_pinned is not the same to pagecount, the pinned_folios
+	 * only set each folio which already pinned when udmabuf_create.
+	 * Note that, since a folio may be pinned multiple times, each folio
+	 * can be added to pinned_folios multiple times, depending on how many
+	 * times the folio has been pinned when create.
+	 */
+	pgoff_t nr_pinned;
+	struct folio **pinned_folios;
+
 	struct sg_table *sg;
 	struct miscdevice *device;
 	pgoff_t *offsets;
-	struct list_head unpin_list;
-};
-
-struct udmabuf_folio {
-	struct folio *folio;
-	struct list_head list;
 };
 
 static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
@@ -198,38 +204,18 @@ static void unmap_udmabuf(struct dma_buf_attachment *at,
 	return put_sg_table(at->dev, sg, direction);
 }
 
-static void unpin_all_folios(struct list_head *unpin_list)
+static void unpin_all_folios(struct udmabuf *ubuf)
 {
-	struct udmabuf_folio *ubuf_folio;
+	pgoff_t i;
 
-	while (!list_empty(unpin_list)) {
-		ubuf_folio = list_first_entry(unpin_list,
-					      struct udmabuf_folio, list);
-		unpin_folio(ubuf_folio->folio);
+	for (i = 0; i < ubuf->nr_pinned; ++i)
+		unpin_folio(ubuf->pinned_folios[i]);
 
-		list_del(&ubuf_folio->list);
-		kfree(ubuf_folio);
-	}
-}
-
-static int add_to_unpin_list(struct list_head *unpin_list,
-			     struct folio *folio)
-{
-	struct udmabuf_folio *ubuf_folio;
-
-	ubuf_folio = kzalloc(sizeof(*ubuf_folio), GFP_KERNEL);
-	if (!ubuf_folio)
-		return -ENOMEM;
-
-	ubuf_folio->folio = folio;
-	list_add_tail(&ubuf_folio->list, unpin_list);
-	return 0;
+	kvfree(ubuf->pinned_folios);
 }
 
 static __always_inline int init_udmabuf(struct udmabuf *ubuf, pgoff_t pgcnt)
 {
-	INIT_LIST_HEAD(&ubuf->unpin_list);
-
 	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios), GFP_KERNEL);
 	if (!ubuf->folios)
 		return -ENOMEM;
@@ -238,12 +224,18 @@ static __always_inline int init_udmabuf(struct udmabuf *ubuf, pgoff_t pgcnt)
 	if (!ubuf->offsets)
 		return -ENOMEM;
 
+	ubuf->pinned_folios = kvmalloc_array(pgcnt,
+					     sizeof(*ubuf->pinned_folios),
+					     GFP_KERNEL);
+	if (!ubuf->pinned_folios)
+		return -ENOMEM;
+
 	return 0;
 }
 
 static __always_inline void deinit_udmabuf(struct udmabuf *ubuf)
 {
-	unpin_all_folios(&ubuf->unpin_list);
+	unpin_all_folios(ubuf);
 	kvfree(ubuf->offsets);
 	kvfree(ubuf->folios);
 }
@@ -353,7 +345,9 @@ static int export_udmabuf(struct udmabuf *ubuf,
 static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
 			      loff_t start, loff_t size)
 {
-	pgoff_t pgoff, pgcnt, upgcnt = ubuf->pagecount;
+	pgoff_t pgoff, pgcnt;
+	pgoff_t upgcnt = ubuf->pagecount;
+	pgoff_t nr_pinned = ubuf->nr_pinned;
 	u32 cur_folio, cur_pgcnt;
 	struct folio **folios = NULL;
 	long nr_folios;
@@ -377,9 +371,7 @@ static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
 		pgoff_t subpgoff = pgoff;
 		size_t fsize = folio_size(folios[cur_folio]);
 
-		ret = add_to_unpin_list(&ubuf->unpin_list, folios[cur_folio]);
-		if (ret < 0)
-			goto err;
+		ubuf->pinned_folios[nr_pinned++] = folios[cur_folio];
 
 		for (; subpgoff < fsize; subpgoff += PAGE_SIZE) {
 			ubuf->folios[upgcnt] = folios[cur_folio];
@@ -399,6 +391,7 @@ static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
 end:
 err:
 	ubuf->pagecount = upgcnt;
+	ubuf->nr_pinned = nr_pinned;
 	kvfree(folios);
 	return ret;
 }
Signature · 1 line
2.45.2
maintainer Acked-by on v5 6/7
#15 lore raw

RE: [PATCH v5 6/7] udmabuf: remove udmabuf_folio

Hi Huan,

Quoted 128 lines

Subject: [PATCH v5 6/7] udmabuf: remove udmabuf_folio

Currently, udmabuf handles folio by creating an unpin list to record each folio obtained from the list and unpinning them when released. To maintain this approach, many data structures have been established.

However, maintaining this type of data structure requires a significant amount of memory and traversing the list is a substantial overhead, which is not friendly to the CPU cache.

Considering that during creation, we arranged the folio array in the order of pin and set the offset according to pgcnt.

We actually don't need to use unpin_list to unpin during release. Instead, we can iterate through the folios array during release and unpin any folio that is different from the ones previously accessed.

By this, not only saves the overhead of the udmabuf_folio data structure but also makes array access more cache-friendly.

---
 drivers/dma-buf/udmabuf.c | 65 +++++++++++++++++----------------------
 1 file changed, 29 insertions(+), 36 deletions(-)
drivers/dma-buf/udmabuf.c +0 -0
index 254d9ec3d9f3..d449c1fd67a5 100644
@@ -27,15 +27,21 @@ MODULE_PARM_DESC(size_limit_mb, "Max size of a

dmabuf, in megabytes. Default is
struct udmabuf {

drivers/dma-buf/udmabuf.c +7 -0
 	pgoff_t pagecount;
 	struct folio **folios;
+
+	/**
+	 * Unlike folios, pinned_folios is only used for unpin.
+	 * So, nr_pinned is not the same to pagecount, the pinned_folios
+	 * only set each folio which already pinned when udmabuf_create.
+	 * Note that, since a folio may be pinned multiple times, each folio
+	 * can be added to pinned_folios multiple times, depending on how

many

drivers/dma-buf/udmabuf.c +10 -31
+	 * times the folio has been pinned when create.
+	 */
+	pgoff_t nr_pinned;
+	struct folio **pinned_folios;
+
 	struct sg_table *sg;
 	struct miscdevice *device;
 	pgoff_t *offsets;
-	struct list_head unpin_list;
-};
-
-struct udmabuf_folio {
-	struct folio *folio;
-	struct list_head list;
 };
 
 static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
@@ -198,38 +204,18 @@ static void unmap_udmabuf(struct
 dma_buf_attachment *at,
 	return put_sg_table(at->dev, sg, direction);
 }
 
-static void unpin_all_folios(struct list_head *unpin_list)
+static void unpin_all_folios(struct udmabuf *ubuf)
 {
-	struct udmabuf_folio *ubuf_folio;
+	pgoff_t i;
 
-	while (!list_empty(unpin_list)) {
-		ubuf_folio = list_first_entry(unpin_list,
-					      struct udmabuf_folio, list);
-		unpin_folio(ubuf_folio->folio);
+	for (i = 0; i < ubuf->nr_pinned; ++i)
+		unpin_folio(ubuf->pinned_folios[i]);
 
-		list_del(&ubuf_folio->list);
-		kfree(ubuf_folio);
-	}
-}
-
-static int add_to_unpin_list(struct list_head *unpin_list,
-			     struct folio *folio)
-{
-	struct udmabuf_folio *ubuf_folio;
-
-	ubuf_folio = kzalloc(sizeof(*ubuf_folio), GFP_KERNEL);
-	if (!ubuf_folio)
-		return -ENOMEM;
-
-	ubuf_folio->folio = folio;
-	list_add_tail(&ubuf_folio->list, unpin_list);
-	return 0;
+	kvfree(ubuf->pinned_folios);
 }
 
 static __always_inline int init_udmabuf(struct udmabuf *ubuf, pgoff_t
 pgcnt)
 {
-	INIT_LIST_HEAD(&ubuf->unpin_list);
-
 	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios),

GFP_KERNEL);

drivers/dma-buf/udmabuf.c +7 -1
 	if (!ubuf->folios)
 		return -ENOMEM;
@@ -238,12 +224,18 @@ static __always_inline int init_udmabuf(struct
 udmabuf *ubuf, pgoff_t pgcnt)
 	if (!ubuf->offsets)
 		return -ENOMEM;
 
+	ubuf->pinned_folios = kvmalloc_array(pgcnt,
+					     sizeof(*ubuf->pinned_folios),
+					     GFP_KERNEL);
+	if (!ubuf->pinned_folios)
+		return -ENOMEM;
+
 	return 0;
 }
 
 static __always_inline void deinit_udmabuf(struct udmabuf *ubuf)
 {
-	unpin_all_folios(&ubuf->unpin_list);
+	unpin_all_folios(ubuf);
 	kvfree(ubuf->offsets);
 	kvfree(ubuf->folios);

}

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 350

Quoted 7 lines
drivers/dma-buf/udmabuf.c +3 -1
@@ -353,7 +345,9 @@ static int export_udmabuf(struct udmabuf *ubuf,
 static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
 			      loff_t start, loff_t size)
 {
-	pgoff_t pgoff, pgcnt, upgcnt = ubuf->pagecount;
+	pgoff_t pgoff, pgcnt;
+	pgoff_t upgcnt = ubuf->pagecount;
+	pgoff_t nr_pinned = ubuf->nr_pinned;

Reverse xmas style is preferred; please try to use that.

Quoted 16 lines
drivers/dma-buf/udmabuf.c +1 -3
 	u32 cur_folio, cur_pgcnt;
 	struct folio **folios = NULL;
 	long nr_folios;
@@ -377,9 +371,7 @@ static int udmabuf_pin_folios(struct udmabuf *ubuf,
 struct file *memfd,
 		pgoff_t subpgoff = pgoff;
 		size_t fsize = folio_size(folios[cur_folio]);
 
-		ret = add_to_unpin_list(&ubuf->unpin_list, folios[cur_folio]);
-		if (ret < 0)
-			goto err;
+		ubuf->pinned_folios[nr_pinned++] = folios[cur_folio];
 
 		for (; subpgoff < fsize; subpgoff += PAGE_SIZE) {
drivers/dma-buf/udmabuf.c +0 -0
 			ubuf->folios[upgcnt] = folios[cur_folio];

drivers/dma-buf/udmabuf.c static int udmabuf_pin_folios(struct udmabuf *ubuf, · line 395

Quoted 5 lines
drivers/dma-buf/udmabuf.c +1 -0
@@ -399,6 +391,7 @@ static int udmabuf_pin_folios(struct udmabuf *ubuf,
 struct file *memfd,
 end:
 err:
 	ubuf->pagecount = upgcnt;
+	ubuf->nr_pinned = nr_pinned;
Quoted 5 lines
drivers/dma-buf/udmabuf.c +0 -0
 	kvfree(folios);
 	return ret;

}
--
2.45.2

author
#16 lore raw

Re: [PATCH v5 6/7] udmabuf: remove udmabuf_folio

Quoted 122 lines · Vivek

Hi Huan,

Quoted 120 lines

Subject: [PATCH v5 6/7] udmabuf: remove udmabuf_folio

Currently, udmabuf handles folio by creating an unpin list to record each folio obtained from the list and unpinning them when released. To maintain this approach, many data structures have been established.

However, maintaining this type of data structure requires a significant amount of memory and traversing the list is a substantial overhead, which is not friendly to the CPU cache.

Considering that during creation, we arranged the folio array in the order of pin and set the offset according to pgcnt.

We actually don't need to use unpin_list to unpin during release. Instead, we can iterate through the folios array during release and unpin any folio that is different from the ones previously accessed.

By this, not only saves the overhead of the udmabuf_folio data structure but also makes array access more cache-friendly.

---
  drivers/dma-buf/udmabuf.c | 65 +++++++++++++++++----------------------
  1 file changed, 29 insertions(+), 36 deletions(-)
drivers/dma-buf/udmabuf.c +0 -0
index 254d9ec3d9f3..d449c1fd67a5 100644
@@ -27,15 +27,21 @@ MODULE_PARM_DESC(size_limit_mb, "Max size of a

dmabuf, in megabytes. Default is
struct udmabuf {
pgoff_t pagecount;
struct folio **folios;

drivers/dma-buf/udmabuf.c +7 -0
+
+	/**
+	 * Unlike folios, pinned_folios is only used for unpin.
+	 * So, nr_pinned is not the same to pagecount, the pinned_folios
+	 * only set each folio which already pinned when udmabuf_create.
+	 * Note that, since a folio may be pinned multiple times, each folio
+	 * can be added to pinned_folios multiple times, depending on how

many

drivers/dma-buf/udmabuf.c +10 -31
+	 * times the folio has been pinned when create.
+	 */
+	pgoff_t nr_pinned;
+	struct folio **pinned_folios;
+
  	struct sg_table *sg;
  	struct miscdevice *device;
  	pgoff_t *offsets;
-	struct list_head unpin_list;
-};
-
-struct udmabuf_folio {
-	struct folio *folio;
-	struct list_head list;
  };
 
  static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
@@ -198,38 +204,18 @@ static void unmap_udmabuf(struct
 dma_buf_attachment *at,
  	return put_sg_table(at->dev, sg, direction);
  }
 
-static void unpin_all_folios(struct list_head *unpin_list)
+static void unpin_all_folios(struct udmabuf *ubuf)
  {
-	struct udmabuf_folio *ubuf_folio;
+	pgoff_t i;
 
-	while (!list_empty(unpin_list)) {
-		ubuf_folio = list_first_entry(unpin_list,
-					      struct udmabuf_folio, list);
-		unpin_folio(ubuf_folio->folio);
+	for (i = 0; i < ubuf->nr_pinned; ++i)
+		unpin_folio(ubuf->pinned_folios[i]);
 
-		list_del(&ubuf_folio->list);
-		kfree(ubuf_folio);
-	}
-}
-
-static int add_to_unpin_list(struct list_head *unpin_list,
-			     struct folio *folio)
-{
-	struct udmabuf_folio *ubuf_folio;
-
-	ubuf_folio = kzalloc(sizeof(*ubuf_folio), GFP_KERNEL);
-	if (!ubuf_folio)
-		return -ENOMEM;
-
-	ubuf_folio->folio = folio;
-	list_add_tail(&ubuf_folio->list, unpin_list);
-	return 0;
+	kvfree(ubuf->pinned_folios);
  }
 
  static __always_inline int init_udmabuf(struct udmabuf *ubuf, pgoff_t
 pgcnt)
  {
-	INIT_LIST_HEAD(&ubuf->unpin_list);
-
  	ubuf->folios = kvmalloc_array(pgcnt, sizeof(*ubuf->folios),

GFP_KERNEL);
if (!ubuf->folios)
return -ENOMEM;

drivers/dma-buf/udmabuf.c +6 -0
@@ -238,12 +224,18 @@ static __always_inline int init_udmabuf(struct
 udmabuf *ubuf, pgoff_t pgcnt)
  	if (!ubuf->offsets)
  		return -ENOMEM;
 
+	ubuf->pinned_folios = kvmalloc_array(pgcnt,
+					     sizeof(*ubuf->pinned_folios),
+					     GFP_KERNEL);
+	if (!ubuf->pinned_folios)
+		return -ENOMEM;
+
  	return 0;
  }
 
Quoted 16 lines
Quoted 8 lines
drivers/dma-buf/udmabuf.c +1 -1
  static __always_inline void deinit_udmabuf(struct udmabuf *ubuf)
  {
-	unpin_all_folios(&ubuf->unpin_list);
+	unpin_all_folios(ubuf);
  	kvfree(ubuf->offsets);
  	kvfree(ubuf->folios);
  }

drivers/dma-buf/udmabuf.c static int export_udmabuf(struct udmabuf *ubuf, · line 350

Quoted 7 lines
drivers/dma-buf/udmabuf.c +3 -1
@@ -353,7 +345,9 @@ static int export_udmabuf(struct udmabuf *ubuf,
  static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
  			      loff_t start, loff_t size)
  {
-	pgoff_t pgoff, pgcnt, upgcnt = ubuf->pagecount;
+	pgoff_t pgoff, pgcnt;
+	pgoff_t upgcnt = ubuf->pagecount;
+	pgoff_t nr_pinned = ubuf->nr_pinned;

Reverse xmas style is preferred; please try to use that.

OK, I'll take care of it, and other patch. :)

Quoted 29 lines
Quoted 17 lines
drivers/dma-buf/udmabuf.c +1 -3
  	u32 cur_folio, cur_pgcnt;
  	struct folio **folios = NULL;
  	long nr_folios;
@@ -377,9 +371,7 @@ static int udmabuf_pin_folios(struct udmabuf *ubuf,
 struct file *memfd,
  		pgoff_t subpgoff = pgoff;
  		size_t fsize = folio_size(folios[cur_folio]);
 
-		ret = add_to_unpin_list(&ubuf->unpin_list, folios[cur_folio]);
-		if (ret < 0)
-			goto err;
+		ubuf->pinned_folios[nr_pinned++] = folios[cur_folio];
 
  		for (; subpgoff < fsize; subpgoff += PAGE_SIZE) {
  			ubuf->folios[upgcnt] = folios[cur_folio];

drivers/dma-buf/udmabuf.c static int udmabuf_pin_folios(struct udmabuf *ubuf, · line 395

Quoted 5 lines
drivers/dma-buf/udmabuf.c +1 -0
@@ -399,6 +391,7 @@ static int udmabuf_pin_folios(struct udmabuf *ubuf,
 struct file *memfd,
  end:
  err:
  	ubuf->pagecount = upgcnt;
+	ubuf->nr_pinned = nr_pinned;
Quoted 5 lines
drivers/dma-buf/udmabuf.c +0 -0
  	kvfree(folios);
  	return ret;

}
--
2.45.2

author
#17 lore raw

PATCH v5 7/7 udmabuf: reuse folio array when pin folios

In reply to Huan Yang

When invoke memfd_pin_folios, we need offer an array to save each folio which we pinned.

The currently way is dynamic alloc an array, get folios, save into udmabuf and then free.

If the size is tiny, alloc from slab, is ok due to slab can cache it. Or, just PCP order can cover, also ok.

But if size is huge, need fallback into vmalloc, then, not well, due to each page will iter alloc, and map into vmalloc area. Too heavy.

Now that we need to iter each udmabuf item, then pin it's range folios, we can reuse the maximum size range's folios array.

---
 drivers/dma-buf/udmabuf.c | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)
drivers/dma-buf/udmabuf.c +20 -14
index d449c1fd67a5..d70e45c33442 100644
@@ -343,28 +343,21 @@ static int export_udmabuf(struct udmabuf *ubuf,
 }
 
 static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
-			      loff_t start, loff_t size)
+			      loff_t start, loff_t size, struct folio **folios)
 {
 	pgoff_t pgoff, pgcnt;
 	pgoff_t upgcnt = ubuf->pagecount;
 	pgoff_t nr_pinned = ubuf->nr_pinned;
 	u32 cur_folio, cur_pgcnt;
-	struct folio **folios = NULL;
 	long nr_folios;
 	loff_t end;
 	int ret = 0;
 
 	pgcnt = size >> PAGE_SHIFT;
-	folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
-	if (!folios)
-		return -ENOMEM;
-
 	end = start + (pgcnt << PAGE_SHIFT) - 1;
 	nr_folios = memfd_pin_folios(memfd, start, end, folios, pgcnt, &pgoff);
-	if (nr_folios <= 0) {
-		ret = nr_folios ? nr_folios : -EINVAL;
-		goto err;
-	}
+	if (nr_folios <= 0)
+		return nr_folios ? nr_folios : -EINVAL;
 
 	cur_pgcnt = 0;
 	for (cur_folio = 0; cur_folio < nr_folios; ++cur_folio) {
@@ -389,10 +382,8 @@ static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
 		pgoff = 0;
 	}
 end:
-err:
 	ubuf->pagecount = upgcnt;
 	ubuf->nr_pinned = nr_pinned;
-	kvfree(folios);
 	return ret;
 }
 
@@ -403,6 +394,8 @@ static long udmabuf_create(struct miscdevice *device,
 	pgoff_t pgcnt = 0, pglimit;
 	long ret = -EINVAL;
 	struct udmabuf *ubuf;
+	struct folio **folios = NULL;
+	unsigned long max_nr_folios = 0;
 	u32 i, flags;
 
 	ubuf = kzalloc(sizeof(*ubuf), GFP_KERNEL);
@@ -411,14 +404,19 @@ static long udmabuf_create(struct miscdevice *device,
 
 	pglimit = (size_limit_mb * 1024 * 1024) >> PAGE_SHIFT;
 	for (i = 0; i < head->count; i++) {
+		pgoff_t subpgcnt;
+
 		if (!PAGE_ALIGNED(list[i].offset))
 			goto err_noinit;
 		if (!PAGE_ALIGNED(list[i].size))
 			goto err_noinit;
 
-		pgcnt += list[i].size >> PAGE_SHIFT;
+		subpgcnt = list[i].size >> PAGE_SHIFT;
+		pgcnt += subpgcnt;
 		if (pgcnt > pglimit)
 			goto err_noinit;
+
+		max_nr_folios = max_t(unsigned long, subpgcnt, max_nr_folios);
 	}
 
 	if (!pgcnt)
@@ -428,6 +426,12 @@ static long udmabuf_create(struct miscdevice *device,
 	if (ret)
 		goto err;
 
+	folios = kvmalloc_array(max_nr_folios, sizeof(*folios), GFP_KERNEL);
+	if (!folios) {
+		ret = -ENOMEM;
+		goto err;
+	}
+
 	for (i = 0; i < head->count; i++) {
 		struct file *memfd = fget(list[i].memfd);
 
@@ -436,7 +440,7 @@ static long udmabuf_create(struct miscdevice *device,
 			goto err;
 
 		ret = udmabuf_pin_folios(ubuf, memfd, list[i].offset,
-					 list[i].size);
+					 list[i].size, folios);
 		fput(memfd);
 		if (ret)
 			goto err;
@@ -447,12 +451,14 @@ static long udmabuf_create(struct miscdevice *device,
 	if (ret < 0)
 		goto err;
 
+	kvfree(folios);
 	return ret;
 
 err:
 	deinit_udmabuf(ubuf);
 err_noinit:
 	kfree(ubuf);
+	kvfree(folios);
 	return ret;
 }
 
Signature · 1 line
2.45.2
maintainer Acked-by on v5 7/7
#18 lore raw

RE: [PATCH v5 7/7] udmabuf: reuse folio array when pin folios

Hi Huan,

Quoted 60 lines

Subject: [PATCH v5 7/7] udmabuf: reuse folio array when pin folios

When invoke memfd_pin_folios, we need offer an array to save each folio which we pinned.

The currently way is dynamic alloc an array, get folios, save into udmabuf and then free.

If the size is tiny, alloc from slab, is ok due to slab can cache it. Or, just PCP order can cover, also ok.

But if size is huge, need fallback into vmalloc, then, not well, due to each page will iter alloc, and map into vmalloc area. Too heavy.

Now that we need to iter each udmabuf item, then pin it's range folios, we can reuse the maximum size range's folios array.

---
 drivers/dma-buf/udmabuf.c | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)
drivers/dma-buf/udmabuf.c +3 -10
index d449c1fd67a5..d70e45c33442 100644
@@ -343,28 +343,21 @@ static int export_udmabuf(struct udmabuf *ubuf,
 }
 
 static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
-			      loff_t start, loff_t size)
+			      loff_t start, loff_t size, struct folio **folios)
 {
 	pgoff_t pgoff, pgcnt;
 	pgoff_t upgcnt = ubuf->pagecount;
 	pgoff_t nr_pinned = ubuf->nr_pinned;
 	u32 cur_folio, cur_pgcnt;
-	struct folio **folios = NULL;
 	long nr_folios;
 	loff_t end;
 	int ret = 0;
 
 	pgcnt = size >> PAGE_SHIFT;
-	folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
-	if (!folios)
-		return -ENOMEM;
-
 	end = start + (pgcnt << PAGE_SHIFT) - 1;
 	nr_folios = memfd_pin_folios(memfd, start, end, folios, pgcnt,
 &pgoff);
-	if (nr_folios <= 0) {
-		ret = nr_folios ? nr_folios : -EINVAL;
-		goto err;
-	}
+	if (nr_folios <= 0)
+		return nr_folios ? nr_folios : -EINVAL;
 
 	cur_pgcnt = 0;
drivers/dma-buf/udmabuf.c +0 -0
 	for (cur_folio = 0; cur_folio < nr_folios; ++cur_folio) {

drivers/dma-buf/udmabuf.c static int udmabuf_pin_folios(struct udmabuf · line 388

Quoted 9 lines
drivers/dma-buf/udmabuf.c +0 -2
@@ -389,10 +382,8 @@ static int udmabuf_pin_folios(struct udmabuf
 *ubuf, struct file *memfd,
 		pgoff = 0;
 	}
 end:
-err:
 	ubuf->pagecount = upgcnt;
 	ubuf->nr_pinned = nr_pinned;
-	kvfree(folios);
 	return ret;

The variable ret is now unused in this function. Remove it and just return 0 at the end.

Quoted 60 lines
drivers/dma-buf/udmabuf.c +2 -0
 }
@@ -403,6 +394,8 @@ static long udmabuf_create(struct miscdevice
 *device,
 	pgoff_t pgcnt = 0, pglimit;
 	long ret = -EINVAL;
 	struct udmabuf *ubuf;
+	struct folio **folios = NULL;
+	unsigned long max_nr_folios = 0;
 	u32 i, flags;
 
drivers/dma-buf/udmabuf.c +6 -1
 	ubuf = kzalloc(sizeof(*ubuf), GFP_KERNEL);
@@ -411,14 +404,19 @@ static long udmabuf_create(struct miscdevice
 *device,
 
 	pglimit = (size_limit_mb * 1024 * 1024) >> PAGE_SHIFT;
 	for (i = 0; i < head->count; i++) {
+		pgoff_t subpgcnt;
+
 		if (!PAGE_ALIGNED(list[i].offset))
 			goto err_noinit;
 		if (!PAGE_ALIGNED(list[i].size))
 			goto err_noinit;
 
-		pgcnt += list[i].size >> PAGE_SHIFT;
+		subpgcnt = list[i].size >> PAGE_SHIFT;
+		pgcnt += subpgcnt;
 		if (pgcnt > pglimit)
 			goto err_noinit;
+
+		max_nr_folios = max_t(unsigned long, subpgcnt,
 max_nr_folios);
 	}
drivers/dma-buf/udmabuf.c +7 -1
 	if (!pgcnt)
@@ -428,6 +426,12 @@ static long udmabuf_create(struct miscdevice
 *device,
 	if (ret)
 		goto err;
 
+	folios = kvmalloc_array(max_nr_folios, sizeof(*folios), GFP_KERNEL);
+	if (!folios) {
+		ret = -ENOMEM;
+		goto err;
+	}
+
 	for (i = 0; i < head->count; i++) {
 		struct file *memfd = fget(list[i].memfd);
@@ -436,7 +440,7 @@ static long udmabuf_create(struct miscdevice
 *device,
 			goto err;
 
 		ret = udmabuf_pin_folios(ubuf, memfd, list[i].offset,
-					 list[i].size);
+					 list[i].size, folios);
 		fput(memfd);
 		if (ret)
drivers/dma-buf/udmabuf.c +0 -0
 			goto err;

drivers/dma-buf/udmabuf.c static long udmabuf_create(struct miscdevice · line 455

Quoted 5 lines
drivers/dma-buf/udmabuf.c +1 -0
@@ -447,12 +451,14 @@ static long udmabuf_create(struct miscdevice
 *device,
 	if (ret < 0)
 		goto err;
 
+	kvfree(folios);
Quoted 12 lines
drivers/dma-buf/udmabuf.c +1 -1
 	return ret;
 
 err:
 	deinit_udmabuf(ubuf);
 err_noinit:
 	kfree(ubuf);
+	kvfree(folios);
 	return ret;
 }
 
--

2.45.2

author
#19 lore raw

Re: [PATCH v5 7/7] udmabuf: reuse folio array when pin folios

Quoted 59 lines · Vivek

Hi Huan,

Quoted 57 lines

Subject: [PATCH v5 7/7] udmabuf: reuse folio array when pin folios

When invoke memfd_pin_folios, we need offer an array to save each folio which we pinned.

The currently way is dynamic alloc an array, get folios, save into udmabuf and then free.

If the size is tiny, alloc from slab, is ok due to slab can cache it. Or, just PCP order can cover, also ok.

But if size is huge, need fallback into vmalloc, then, not well, due to each page will iter alloc, and map into vmalloc area. Too heavy.

Now that we need to iter each udmabuf item, then pin it's range folios, we can reuse the maximum size range's folios array.

---
  drivers/dma-buf/udmabuf.c | 34 ++++++++++++++++++++--------------
  1 file changed, 20 insertions(+), 14 deletions(-)
drivers/dma-buf/udmabuf.c +3 -10
index d449c1fd67a5..d70e45c33442 100644
@@ -343,28 +343,21 @@ static int export_udmabuf(struct udmabuf *ubuf,
  }
 
  static int udmabuf_pin_folios(struct udmabuf *ubuf, struct file *memfd,
-			      loff_t start, loff_t size)
+			      loff_t start, loff_t size, struct folio **folios)
  {
  	pgoff_t pgoff, pgcnt;
  	pgoff_t upgcnt = ubuf->pagecount;
  	pgoff_t nr_pinned = ubuf->nr_pinned;
  	u32 cur_folio, cur_pgcnt;
-	struct folio **folios = NULL;
  	long nr_folios;
  	loff_t end;
  	int ret = 0;
 
  	pgcnt = size >> PAGE_SHIFT;
-	folios = kvmalloc_array(pgcnt, sizeof(*folios), GFP_KERNEL);
-	if (!folios)
-		return -ENOMEM;
-
  	end = start + (pgcnt << PAGE_SHIFT) - 1;
  	nr_folios = memfd_pin_folios(memfd, start, end, folios, pgcnt,
 &pgoff);
-	if (nr_folios <= 0) {
-		ret = nr_folios ? nr_folios : -EINVAL;
-		goto err;
-	}
+	if (nr_folios <= 0)
+		return nr_folios ? nr_folios : -EINVAL;
 
Quoted 14 lines

drivers/dma-buf/udmabuf.c static int udmabuf_pin_folios(struct udmabuf · line 388

Quoted 12 lines
drivers/dma-buf/udmabuf.c +0 -0
  	cur_pgcnt = 0;
  	for (cur_folio = 0; cur_folio < nr_folios; ++cur_folio) {
drivers/dma-buf/udmabuf.c +0 -2
@@ -389,10 +382,8 @@ static int udmabuf_pin_folios(struct udmabuf
 *ubuf, struct file *memfd,
  		pgoff = 0;
  	}
  end:
-err:
  	ubuf->pagecount = upgcnt;
  	ubuf->nr_pinned = nr_pinned;
-	kvfree(folios);
  	return ret;

The variable ret is now unused in this function. Remove it and just return 0 at the end.

Oh, nice catch.

Thanks.

Quoted 80 lines
Quoted 61 lines
drivers/dma-buf/udmabuf.c +2 -0
  }
@@ -403,6 +394,8 @@ static long udmabuf_create(struct miscdevice
 *device,
  	pgoff_t pgcnt = 0, pglimit;
  	long ret = -EINVAL;
  	struct udmabuf *ubuf;
+	struct folio **folios = NULL;
+	unsigned long max_nr_folios = 0;
  	u32 i, flags;
 
  	ubuf = kzalloc(sizeof(*ubuf), GFP_KERNEL);
drivers/dma-buf/udmabuf.c +6 -1
@@ -411,14 +404,19 @@ static long udmabuf_create(struct miscdevice
 *device,
 
  	pglimit = (size_limit_mb * 1024 * 1024) >> PAGE_SHIFT;
  	for (i = 0; i < head->count; i++) {
+		pgoff_t subpgcnt;
+
  		if (!PAGE_ALIGNED(list[i].offset))
  			goto err_noinit;
  		if (!PAGE_ALIGNED(list[i].size))
  			goto err_noinit;
 
-		pgcnt += list[i].size >> PAGE_SHIFT;
+		subpgcnt = list[i].size >> PAGE_SHIFT;
+		pgcnt += subpgcnt;
  		if (pgcnt > pglimit)
  			goto err_noinit;
+
+		max_nr_folios = max_t(unsigned long, subpgcnt,
 max_nr_folios);
  	}

if (!pgcnt)

drivers/dma-buf/udmabuf.c +7 -1
@@ -428,6 +426,12 @@ static long udmabuf_create(struct miscdevice
 *device,
  	if (ret)
  		goto err;
 
+	folios = kvmalloc_array(max_nr_folios, sizeof(*folios), GFP_KERNEL);
+	if (!folios) {
+		ret = -ENOMEM;
+		goto err;
+	}
+
  	for (i = 0; i < head->count; i++) {
  		struct file *memfd = fget(list[i].memfd);
@@ -436,7 +440,7 @@ static long udmabuf_create(struct miscdevice
 *device,
  			goto err;
 
  		ret = udmabuf_pin_folios(ubuf, memfd, list[i].offset,
-					 list[i].size);
+					 list[i].size, folios);
  		fput(memfd);
  		if (ret)

goto err;

drivers/dma-buf/udmabuf.c static long udmabuf_create(struct miscdevice · line 455

Quoted 5 lines
drivers/dma-buf/udmabuf.c +1 -0
@@ -447,12 +451,14 @@ static long udmabuf_create(struct miscdevice
 *device,
  	if (ret < 0)
  		goto err;
 
+	kvfree(folios);
Quoted 12 lines
drivers/dma-buf/udmabuf.c +1 -1
  	return ret;
 
  err:
  	deinit_udmabuf(ubuf);
  err_noinit:
  	kfree(ubuf);
+	kvfree(folios);
  	return ret;
  }
 
--

2.45.2