RDMA/umem: add back hugepage sg list
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH 1/1 RDMA/umem: add back hugepage sg list
From: Zhu Yanjun <[email protected]>
After the commit ("RDMA/umem: Move to allocate SG table from pages"), the sg list from ib_ume_get is like the following:
"
sg_dma_address(sg):0x4b3c1ce000 sg_dma_address(sg):0x4c3c1cd000 sg_dma_address(sg):0x4d3c1cc000 sg_dma_address(sg):0x4e3c1cb000
"
But sometimes, we need sg list like the following:
"
sg_dma_address(sg):0x203b400000 sg_dma_address(sg):0x213b200000 sg_dma_address(sg):0x223b000000 sg_dma_address(sg):0x233ae00000 sg_dma_address(sg):0x243ac00000
"
The function ib_umem_add_sg_table can provide the sg list like the second. And this function is removed in the commit ("RDMA/umem: Move to allocate SG table from pages"). Now I add it back.
The new function is ib_umem_get to ib_umem_hugepage_get that calls ib_umem_add_sg_table.
This function ib_umem_huagepage_get can get 4K, 2M sg list dma address.
--- drivers/infiniband/core/umem.c | 197 +++++++++++++++++++++++++++++++++ include/rdma/ib_umem.h | 3 + 2 files changed, 200 insertions(+)
drivers/infiniband/core/umem.c +0 -0
@@ -62,6 +62,203 @@ static void __ib_umem_release(struct ib_device
*dev, struct ib_umem *umem, int d
sg_free_table(&umem->sg_head);
}
+/* ib_umem_add_sg_table - Add N contiguous pages to scatter table
+ *
+ * sg: current scatterlist entry
+ * page_list: array of npage struct page pointers
+ * npages: number of pages in page_list
+ * max_seg_sz: maximum segment size in bytes
+ * nents: [out] number of entries in the scatterlist
+ *
+ * Return new end of scatterlist
+ */
+static struct scatterlist *ib_umem_add_sg_table(struct scatterlist *sg,
+ struct page **page_list,
+ unsigned long npages,
+ unsigned int max_seg_sz,
+ int *nents)
+{
+ unsigned long first_pfn;
+ unsigned long i = 0;
+ bool update_cur_sg = false;
+ bool first = !sg_page(sg);+
+ /* Check if new page_list is contiguous with end of previous page_list.
+ * sg->length here is a multiple of PAGE_SIZE and sg->offset is 0.
+ */
+ if (!first && (page_to_pfn(sg_page(sg)) + (sg->length >> PAGE_SHIFT) ==
+ page_to_pfn(page_list[0])))
+ update_cur_sg = true;
+
+ while (i != npages) {
+ unsigned long len;
+ struct page *first_page = page_list[i];
+
+ first_pfn = page_to_pfn(first_page);
+
+ /* Compute the number of contiguous pages we have starting
+ * at i
+ */
+ for (len = 0; i != npages &&
+ first_pfn + len == page_to_pfn(page_list[i]) &&
+ len < (max_seg_sz >> PAGE_SHIFT);
+ len++)
+ i++;
+
+ /* Squash N contiguous pages from page_list into current sge */
+ if (update_cur_sg) {
+ if ((max_seg_sz - sg->length) >= (len << PAGE_SHIFT)) {
+ sg_set_page(sg, sg_page(sg), + sg->length + (len << PAGE_SHIFT), + 0); + update_cur_sg = false; + continue;
+ }
+ update_cur_sg = false;
+ }
+
+ /* Squash N contiguous pages into next sge or first sge */
+ if (!first)
+ sg = sg_next(sg);
+
+ (*nents)++; + sg_set_page(sg, first_page, len << PAGE_SHIFT, 0); + first = false;
+ }
+
+ return sg;
+}
+
+/**
+ * ib_umem_hugepage_get - Pin and DMA map userspace memory.
+ *
+ * @device: IB device to connect UMEM
+ * @addr: userspace virtual address to start at
+ * @size: length of region to pin
+ * @access: IB_ACCESS_xxx flags for memory being pinned
+ */
+struct ib_umem *ib_umem_hugepage_get(struct ib_device *device,
+ unsigned long addr,
+ size_t size, int access)
+{
+ struct ib_umem *umem; + struct page **page_list; + unsigned long lock_limit; + unsigned long new_pinned; + unsigned long cur_base; + unsigned long dma_attr = 0; + struct mm_struct *mm; + unsigned long npages; + int ret; + struct scatterlist *sg; + unsigned int gup_flags = FOLL_WRITE; + + /* + * If the combination of the addr and size requested for this memory + * region causes an integer overflow, return error. + */
+ if (((addr + size) < addr) ||
+ PAGE_ALIGN(addr + size) < (addr + size))
+ return ERR_PTR(-EINVAL);
+
+ if (!can_do_mlock())
+ return ERR_PTR(-EPERM);
+
+ if (access & IB_ACCESS_ON_DEMAND)
+ return ERR_PTR(-EOPNOTSUPP);
+
+ umem = kzalloc(sizeof(*umem), GFP_KERNEL);
+ if (!umem)
+ return ERR_PTR(-ENOMEM);
+ umem->ibdev = device; + umem->length = size; + umem->address = addr; + umem->writable = ib_access_writable(access); + umem->owning_mm = mm = current->mm; + mmgrab(mm);
+
+ page_list = (struct page **) __get_free_page(GFP_KERNEL);
+ if (!page_list) {
+ ret = -ENOMEM;
+ goto umem_kfree;
+ }
+
+ npages = ib_umem_num_pages(umem);
+ if (npages == 0 || npages > UINT_MAX) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
+
+ new_pinned = atomic64_add_return(npages, &mm->pinned_vm);
+ if (new_pinned > lock_limit && !capable(CAP_IPC_LOCK)) {
+ atomic64_sub(npages, &mm->pinned_vm); + ret = -ENOMEM; + goto out;
+ }
+
+ cur_base = addr & PAGE_MASK;
+
+ ret = sg_alloc_table(&umem->sg_head, npages, GFP_KERNEL);
+ if (ret)
+ goto vma;
+
+ if (!umem->writable)
+ gup_flags |= FOLL_FORCE;
+
+ sg = umem->sg_head.sgl;
+
+ while (npages) {
+ cond_resched();
+ ret = pin_user_pages_fast(cur_base,
+ min_t(unsigned long, npages,
+ PAGE_SIZE /
+ sizeof(struct page *)),
+ gup_flags | FOLL_LONGTERM, page_list);
+ if (ret < 0)
+ goto umem_release;
+
+ cur_base += ret * PAGE_SIZE;
+ npages -= ret;
+
+ sg = ib_umem_add_sg_table(sg, page_list, ret,
+ dma_get_max_seg_size(device->dma_device),
+ &umem->sg_nents);
+ }
+
+ sg_mark_end(sg);
+
+ if (access & IB_ACCESS_RELAXED_ORDERING)
+ dma_attr |= DMA_ATTR_WEAK_ORDERING;
+
+ umem->nmap =
+ ib_dma_map_sg_attrs(device, umem->sg_head.sgl, umem->sg_nents,
+ DMA_BIDIRECTIONAL, dma_attr);
+
+ if (!umem->nmap) {
+ ret = -ENOMEM;
+ goto umem_release;
+ }
+
+ ret = 0;
+ goto out;
+
+umem_release:
+ __ib_umem_release(device, umem, 0);
+vma:
+ atomic64_sub(ib_umem_num_pages(umem), &mm->pinned_vm);
+out:
+ free_page((unsigned long) page_list);
+umem_kfree:
+ if (ret) {
+ mmdrop(umem->owning_mm);
+ kfree(umem);
+ }
+ return ret ? ERR_PTR(ret) : umem;
+}
+EXPORT_SYMBOL(ib_umem_hugepage_get);
+
/**
* ib_umem_find_best_pgsz - Find best HW page size to use for this MR *
include/rdma/ib_umem.h +0 -0
@@ -96,6 +96,9 @@ static inline void
__rdma_umem_block_iter_start(struct ib_block_iter *biter,
__rdma_block_iter_next(biter);)
#ifdef CONFIG_INFINIBAND_USER_MEM
+struct ib_umem *ib_umem_hugepage_get(struct ib_device *device,
+ unsigned long addr,
+ size_t size, int access);
struct ib_umem *ib_umem_get(struct ib_device *device, unsigned long addr,
size_t size, int access);
--
2.25.1
Please use git send-email to send patches.
Also don't write 1/1 for single patch.
Thanks
Quoted 261 lines · Zhu Yanjun
From: Zhu Yanjun <[email protected]>
After the commit ("RDMA/umem: Move to allocate SG table from pages"), the sg list from
ib_ume_getis like the following:
"
sg_dma_address(sg):0x4b3c1ce000 sg_dma_address(sg):0x4c3c1cd000 sg_dma_address(sg):0x4d3c1cc000 sg_dma_address(sg):0x4e3c1cb000
"But sometimes, we need sg list like the following:
"
sg_dma_address(sg):0x203b400000 sg_dma_address(sg):0x213b200000 sg_dma_address(sg):0x223b000000 sg_dma_address(sg):0x233ae00000 sg_dma_address(sg):0x243ac00000
"
The functionib_umem_add_sg_tablecan provide the sg list like the second. And this function is removed in the commit ("RDMA/umem: Move to allocate SG table from pages"). Now I add it back.The new function is
ib_umem_gettoib_umem_hugepage_getthat callsib_umem_add_sg_table.This function
ib_umem_huagepage_getcan get 4K, 2M sg list dma address.Fixes 0c16d9635e3a ("RDMA/umem: Move to allocate SG table from pages")Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/core/umem.c | 197 +++++++++++++++++++++++++++++++++ include/rdma/ib_umem.h | 3 + 2 files changed, 200 insertions(+)drivers/infiniband/core/umem.c +197
-0@@ -62,6 +62,203 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int dsg_free_table (&umem->sg_head); } +/* ib_umem_add_sg_table - Add N contiguous pages to scatter table +* +* sg: current scatterlist entry +* page_list: array of npage struct page pointers +* npages: number of pages in page_list +* max_seg_sz: maximum segment size in bytes +* nents: [out] number of entries in the scatterlist +* +* Return new end of scatterlist +*/ +static struct scatterlist *ib_umem_add_sg_table (struct scatterlist *sg, +struct page **page_list, +unsigned long npages, +unsigned int max_seg_sz, +int *nents) +{ +unsigned long first_pfn; +unsigned long i =0 ; +bool update_cur_sg = false; +bool first = !sg_page (sg); + +/* Check if new page_list is contiguous with end of previous page_list. +* sg->length here is a multiple of PAGE_SIZE and sg->offset is 0. +*/ +if (!first && (page_to_pfn (sg_page (sg)) + (sg->length >>PAGE_SHIFT ) == +page_to_pfn (page_list[0 ]))) + update_cur_sg = true; + +while (i != npages) { +unsigned long len; +struct page *first_page = page_list[i]; + + first_pfn =page_to_pfn (first_page); + +/* Compute the number of contiguous pages we have starting +* at i +*/ +for (len =0 ; i != npages && + first_pfn + len ==page_to_pfn (page_list[i]) && + len < (max_seg_sz >>PAGE_SHIFT ); + len++) + i++; + +/* Squash N contiguous pages from page_list into current sge */ +if (update_cur_sg) { +if ((max_seg_sz - sg->length) >= (len <<PAGE_SHIFT )) { +sg_set_page (sg,sg_page (sg), + sg->length + (len <<PAGE_SHIFT ), +0 ); + update_cur_sg = false; +continue ; + } + update_cur_sg = false; + } + +/* Squash N contiguous pages into next sge or first sge */ +if (!first) + sg =sg_next (sg); + + (*nents)++; +sg_set_page (sg, first_page, len <<PAGE_SHIFT ,0 ); + first = false; + } + +return sg; +} + +/** +* ib_umem_hugepage_get - Pin and DMA map userspace memory. +* +* @device: IB device to connect UMEM +* @addr: userspace virtual address to start at +* @size: length of region to pin +* @access: IB_ACCESS_xxx flags for memory being pinned +*/ +struct ib_umem *ib_umem_hugepage_get (struct ib_device *device, +unsigned long addr, +size_t size,int access) +{ +struct ib_umem *umem; +struct page **page_list; +unsigned long lock_limit; +unsigned long new_pinned; +unsigned long cur_base; +unsigned long dma_attr =0 ; +struct mm_struct *mm; +unsigned long npages; +int ret; +struct scatterlist *sg; +unsigned int gup_flags =FOLL_WRITE ; + +/* +* If the combination of the addr and size requested for this memory +* region causes an integer overflow, return error. +*/ +if (((addr + size) < addr) || +PAGE_ALIGN (addr + size) < (addr + size)) +return ERR_PTR (-EINVAL ); + +if (!can_do_mlock ()) +return ERR_PTR (-EPERM ); + +if (access &IB_ACCESS_ON_DEMAND ) +return ERR_PTR (-EOPNOTSUPP ); + + umem =kzalloc (sizeof (*umem),GFP_KERNEL ); +if (!umem) +return ERR_PTR (-ENOMEM ); + umem->ibdev = device; + umem->length = size; + umem->address = addr; + umem->writable =ib_access_writable (access); + umem->owning_mm = mm = current->mm; +mmgrab (mm); + + page_list = (struct page **)__get_free_page (GFP_KERNEL ); +if (!page_list) { + ret = -ENOMEM ; + goto umem_kfree; + } + + npages =ib_umem_num_pages (umem); +if (npages ==0 || npages >UINT_MAX ) { + ret = -EINVAL ; + goto out; + } + + lock_limit =rlimit (RLIMIT_MEMLOCK ) >>PAGE_SHIFT ; + + new_pinned =atomic64_add_return (npages, &mm->pinned_vm); +if (new_pinned > lock_limit && !capable (CAP_IPC_LOCK )) { +atomic64_sub (npages, &mm->pinned_vm); + ret = -ENOMEM ; + goto out; + } + + cur_base = addr &PAGE_MASK ; + + ret =sg_alloc_table (&umem->sg_head, npages,GFP_KERNEL ); +if (ret) + goto vma; + +if (!umem->writable) + gup_flags |=FOLL_FORCE ; + + sg = umem->sg_head.sgl; + +while (npages) { +cond_resched (); + ret =pin_user_pages_fast (cur_base, + min_t(unsigned long, npages, +PAGE_SIZE / +sizeof (struct page *)), + gup_flags |FOLL_LONGTERM , page_list); +if (ret <0 ) + goto umem_release; + + cur_base += ret *PAGE_SIZE ; + npages -= ret; + + sg =ib_umem_add_sg_table (sg, page_list, ret, +dma_get_max_seg_size (device->dma_device), + &umem->sg_nents); + } + +sg_mark_end (sg); + +if (access &IB_ACCESS_RELAXED_ORDERING ) + dma_attr |=DMA_ATTR_WEAK_ORDERING ; + + umem->nmap = +ib_dma_map_sg_attrs (device, umem->sg_head.sgl, umem->sg_nents, +DMA_BIDIRECTIONAL , dma_attr); + +if (!umem->nmap) { + ret = -ENOMEM ; + goto umem_release; + } + + ret =0 ; + goto out; + +umem_release: +__ib_umem_release (device, umem,0 ); +vma: +atomic64_sub (ib_umem_num_pages (umem), &mm->pinned_vm); +out: +free_page ((unsigned long ) page_list); +umem_kfree: +if (ret) { +mmdrop (umem->owning_mm); +kfree (umem); + } +return ret ?ERR_PTR (ret) : umem; +} +EXPORT_SYMBOL (ib_umem_hugepage_get); +/** * ib_umem_find_best_pgsz - Find best HW page size to use for this MR *
include/rdma/ib_umem.h +3
-0@@ -96,6 +96,9 @@ static inline void __rdma_umem_block_iter_start(struct ib_block_iter *biter, __rdma_block_iter_next(biter);)#ifdef CONFIG_INFINIBAND_USER_MEM +struct ib_umem *ib_umem_hugepage_get (struct ib_device *device, +unsigned long addr, +size_t size,int access); struct ib_umem *ib_umem_get(struct ib_device *device, unsigned long addr,size_t size, int access);--
2.25.1
Quoted 26 lines · Zhu Yanjun
From: Zhu Yanjun <[email protected]>
After the commit ("RDMA/umem: Move to allocate SG table from pages"), the sg list from
ib_ume_getis like the following:
"
sg_dma_address(sg):0x4b3c1ce000 sg_dma_address(sg):0x4c3c1cd000 sg_dma_address(sg):0x4d3c1cc000 sg_dma_address(sg):0x4e3c1cb000
"But sometimes, we need sg list like the following:
"
sg_dma_address(sg):0x203b400000 sg_dma_address(sg):0x213b200000 sg_dma_address(sg):0x223b000000 sg_dma_address(sg):0x233ae00000 sg_dma_address(sg):0x243ac00000
"
The functionib_umem_add_sg_tablecan provide the sg list like the second. And this function is removed in the commit ("RDMA/umem: Move to allocate SG table from pages"). Now I add it back.The new function is
ib_umem_gettoib_umem_hugepage_getthat callsib_umem_add_sg_table.
Quoted 8 lines
This function
ib_umem_huagepage_getcan get 4K, 2M sg list dma address.Fixes 0c16d9635e3a ("RDMA/umem: Move to allocate SG table from pages")Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/core/umem.c | 197 +++++++++++++++++++++++++++++++++ include/rdma/ib_umem.h | 3 + 2 files changed, 200 insertions(+)
You didn't use newly introduced function and didn't really explain why it is needed.
Thanks
Quoted 37 lines · Leon Romanovsky
Quoted 26 lines · Zhu Yanjun
From: Zhu Yanjun <[email protected]>
After the commit ("RDMA/umem: Move to allocate SG table from pages"), the sg list from
ib_ume_getis like the following:
"
sg_dma_address(sg):0x4b3c1ce000 sg_dma_address(sg):0x4c3c1cd000 sg_dma_address(sg):0x4d3c1cc000 sg_dma_address(sg):0x4e3c1cb000
"But sometimes, we need sg list like the following:
"
sg_dma_address(sg):0x203b400000 sg_dma_address(sg):0x213b200000 sg_dma_address(sg):0x223b000000 sg_dma_address(sg):0x233ae00000 sg_dma_address(sg):0x243ac00000
"
The functionib_umem_add_sg_tablecan provide the sg list like the second. And this function is removed in the commit ("RDMA/umem: Move to allocate SG table from pages"). Now I add it back.The new function is
ib_umem_gettoib_umem_hugepage_getthat callsib_umem_add_sg_table.Quoted 8 lines
This function
ib_umem_huagepage_getcan get 4K, 2M sg list dma address.Fixes 0c16d9635e3a ("RDMA/umem: Move to allocate SG table from pages")Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/core/umem.c | 197 +++++++++++++++++++++++++++++++++ include/rdma/ib_umem.h | 3 + 2 files changed, 200 insertions(+)
Quoted 2 lines
You didn't use newly introduced function and didn't really explain why it is needed.
OK. I will explain later.
Zhu Yanjun
Quoted 2 lines
Thanks
Quoted 30 lines · Leon Romanovsky
Quoted 28 lines · Zhu Yanjun
From: Zhu Yanjun <[email protected]>
After the commit ("RDMA/umem: Move to allocate SG table from pages"), the sg list from
ib_ume_getis like the following:
"
sg_dma_address(sg):0x4b3c1ce000 sg_dma_address(sg):0x4c3c1cd000 sg_dma_address(sg):0x4d3c1cc000 sg_dma_address(sg):0x4e3c1cb000
"But sometimes, we need sg list like the following:
"
sg_dma_address(sg):0x203b400000 sg_dma_address(sg):0x213b200000 sg_dma_address(sg):0x223b000000 sg_dma_address(sg):0x233ae00000 sg_dma_address(sg):0x243ac00000
"
The functionib_umem_add_sg_tablecan provide the sg list like the second. And this function is removed in the commit ("RDMA/umem: Move to allocate SG table from pages"). Now I add it back.The new function is
ib_umem_gettoib_umem_hugepage_getthat callsib_umem_add_sg_table.This function
ib_umem_huagepage_getcan get 4K, 2M sg list dma address.
Quoted 8 lines
Quoted 6 lines
Fixes 0c16d9635e3a ("RDMA/umem: Move to allocate SG table from pages")Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/core/umem.c | 197 +++++++++++++++++++++++++++++++++ include/rdma/ib_umem.h | 3 + 2 files changed, 200 insertions(+)You didn't use newly introduced function and didn't really explain why
Thanks Leon and Jason.
Now I set ib_dma_max_seg_size to SZ_2M. Then this problem is fixed. This problem is caused by the capacity parameter of our device.
Originally the ib_dma_max_seg_size is set to UINT_MAX. Then the sg dma address is different from
the ones before the commit ("RDMA/umem: Move to allocate SG table from pages").
And I delved into the source code of __sg_alloc_table_from_pages. I found that this function is
related with ib_dma_max_seg_size. So when ib_dma_max_seg_size is set to UINT_MAX, the sg dma
address is 4K (one page). When ib_dma_max_seg_size is set to SZ_2M, the sg dma address is 2M now.
The function ib_umem_add_sg_table is not related with ib_dma_max_seg_size. So the value of ib_dma_max_seg_size does not affect sg dma address.
Anyway, this problem is not caused by the function __sg_alloc_table_from_pages.
Please ignore this commit.
Zhu Yanjun
Quoted 3 lines
it is needed.
Thanks
Jason Gunthorpe
maintainerQuoted 5 lines · Zhu Yanjun
And I delved into the source code of
__sg_alloc_table_from_pages. I found that this function is related withib_dma_max_seg_size. So whenib_dma_max_seg_sizeis set toUINT_MAX, the sg dma address is 4K (one page). Whenib_dma_max_seg_sizeis set toSZ_2M, the sg dma address is 2M now.
That seems like a bug, you should fix it
Jason
Quoted 10 lines · Jason Gunthorpe
Quoted 5 lines · Zhu Yanjun
And I delved into the source code of
__sg_alloc_table_from_pages. I found that this function is related withib_dma_max_seg_size. So whenib_dma_max_seg_sizeis set toUINT_MAX, the sg dma address is 4K (one page). Whenib_dma_max_seg_sizeis set toSZ_2M, the sg dma address is 2M now.That seems like a bug, you should fix it
Hi, Jason && Leon
I compared the function __sg_alloc_table_from_pages with ib_umem_add_sg_table. In __sg_alloc_table_from_pages:
"
449 if (prv) {
450 unsigned long paddr = (page_to_pfn(sg_page(prv))
* PAGE_SIZE +
451 prv->offset + prv->length) /
452 PAGE_SIZE;
453
454 if (WARN_ON(offset))
455 return ERR_PTR(-EINVAL);
456
457 /* Merge contiguous pages into the last SG */
458 prv_len = prv->length;
459 while (n_pages && page_to_pfn(pages[0]) == paddr) {
460 if (prv->length + PAGE_SIZE > max_segment)
461 break;
462 prv->length += PAGE_SIZE;
463 paddr++;
464 pages++;
465 n_pages--; 466 }
467 if (!n_pages)
468 goto out;
469 }
"
if prv->length + PAGE_SIZE > max_segment, then set another sg. In the commit "RDMA/umem: Move to allocate SG table from pages", max_segment is dma_get_max_seg_size.
Normally it is UINT_MAX. So in my host, prv->length + PAGE_SIZE is usually less than max_segment
since length is unsigned int.
So the following has very few chance to execute.
"
485 for (i = 0; i < chunks; i++) {
...
490 for (j = cur_page + 1; j < n_pages; j++) {
491 seg_len += PAGE_SIZE;
492 if (seg_len >= max_segment ||
493 page_to_pfn(pages[j]) !=
494 page_to_pfn(pages[j - 1]) + 1)
495 break;
496 }
497
498 /* Pass how many chunks might be left */
499 s = get_next_sg(sgt, s, chunks - i + left_pages, gfp_mask);
...
510 sg_set_page(s, pages[cur_page],
511 min_t(unsigned long, size,chunk_size), offset);
512 added_nents++;
...
516 }
"
In the function ib_umem_add_sg_table, max_segment is used like the below:
"
/* Squash N contiguous pages from page_list into current sge */
if (update_cur_sg) {
if ((max_seg_sz - sg->length) >= (len << PAGE_SHIFT)) {
sg_set_page(sg, sg_page(sg),
sg->length + (len << PAGE_SHIFT),
0);
update_cur_sg = false;
continue;
}
update_cur_sg = false;
}
/* Squash N contiguous pages into next sge or first sge */
if (!first)
sg = sg_next(sg);
(*nents)++;
sg_set_page(sg, first_page, len << PAGE_SHIFT, 0);"
if (max_seg_sz - sg->length) >= (len << PAGE_SHIFT), the function sg_next and sg_set_page are called several times.
The different usage of max_seg_sz/max_segment is the root cause that sg_dma_addresses
are different from the function __sg_alloc_table_from_pages and ib_umem_add_sg_table.
The following commit tries to fix this problem, please comment on it.
From 65472ef21146b0fb72d5eb3e2fe1277380d29446 Mon Sep 17 00:00:00 2001
From: Zhu Yanjun <[email protected]>
Date: Thu, 11 Mar 2021 12:35:40 -0500
Subject: [PATCH 1/1] RDMA/umem: fix different sg_dma_address problem
After the commit 0c16d9635e3a ("RDMA/umem: Move to allocate SG table from pages"), the returned sg list has the different dma addresses compared with the removed the function ib_umem_add_sg_table.
The root cause is that max_seg_sz/max_segment has different usage in the function ib_umem_add_sg_table and __sg_alloc_table_from_pages.
--- drivers/infiniband/core/umem.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/infiniband/core/umem.c +0 -0
@@ -232,7 +232,9 @@ struct ib_umem *ib_umem_get(struct ib_device
*device, unsigned long addr,
npages -= ret;
sg = __sg_alloc_table_from_pages(&umem->sg_head, page_list, ret,
0, ret << PAGE_SHIFT,
- ib_dma_max_seg_size(device), sg, npages,
+ min_t(unsigned int,
+ ib_dma_max_seg_size(device), ret * PAGE_SIZE),
+ sg, npages,
GFP_KERNEL);
umem->sg_nents = umem->sg_head.nents;
if (IS_ERR(sg)) {--
2.27.0
Quoted 2 lines
Jason
Jason Gunthorpe
maintainerQuoted 41 lines · Zhu Yanjun
Quoted 10 lines · Jason Gunthorpe
Quoted 5 lines · Zhu Yanjun
And I delved into the source code of
__sg_alloc_table_from_pages. I found that this function is related withib_dma_max_seg_size. So whenib_dma_max_seg_sizeis set toUINT_MAX, the sg dma address is 4K (one page). Whenib_dma_max_seg_sizeis set toSZ_2M, the sg dma address is 2M now.That seems like a bug, you should fix it
Hi, Jason && Leon
I compared the function
__sg_alloc_table_from_pageswithib_umem_add_sg_table. In__sg_alloc_table_from_pages:"
449 if (prv) {
450 unsigned long paddr = (page_to_pfn(sg_page(prv))
*PAGE_SIZE+
451prv->offset+prv->length) /
452PAGE_SIZE;
453
454 if (WARN_ON(offset))
455 returnERR_PTR(-EINVAL);
456457 /* Merge contiguous pages into the last SG */ 458 prv_len = prv->length; 459 while (n_pages && page_to_pfn(pages[0]) == paddr) { 460 if (prv->length + PAGE_SIZE > max_segment) 461 break; 462 prv->length += PAGE_SIZE; 463 paddr++; 464 pages++; 465 n_pages--;466 }
467 if (!n_pages)
468 goto out;
469 }
Quoted 7 lines
"
ifprv->length+PAGE_SIZE>max_segment, then set another sg. In the commit "RDMA/umem: Move to allocate SG table from pages",max_segmentisdma_get_max_seg_size.
Normally it isUINT_MAX. So in my host,prv->length+PAGE_SIZEis usually less thanmax_segment
since length is unsigned int.
I don't understand what you are trying to say
460 if (prv->length + PAGE_SIZE > max_segment)
max_segment should be a very big number and "prv->length + PAGE_SIZE" should always be < max_segment so it should always be increasing the size of prv->length and 'rpv' here is the sgl.
The other loops are the same.
Jason
Quoted 51 lines · Jason Gunthorpe
Quoted 41 lines · Zhu Yanjun
Quoted 10 lines · Jason Gunthorpe
Quoted 5 lines · Zhu Yanjun
And I delved into the source code of
__sg_alloc_table_from_pages. I found that this function is related withib_dma_max_seg_size. So whenib_dma_max_seg_sizeis set toUINT_MAX, the sg dma address is 4K (one page). Whenib_dma_max_seg_sizeis set toSZ_2M, the sg dma address is 2M now.That seems like a bug, you should fix it
Hi, Jason && Leon
I compared the function
__sg_alloc_table_from_pageswithib_umem_add_sg_table. In__sg_alloc_table_from_pages:"
449 if (prv) {
450 unsigned long paddr = (page_to_pfn(sg_page(prv))
*PAGE_SIZE+
451prv->offset+prv->length) /
452PAGE_SIZE;
453
454 if (WARN_ON(offset))
455 returnERR_PTR(-EINVAL);
456457 /* Merge contiguous pages into the last SG */ 458 prv_len = prv->length; 459 while (n_pages && page_to_pfn(pages[0]) == paddr) { 460 if (prv->length + PAGE_SIZE > max_segment) 461 break; 462 prv->length += PAGE_SIZE; 463 paddr++; 464 pages++; 465 n_pages--;466 }
467 if (!n_pages)
468 goto out;
469 }Quoted 7 lines
"
ifprv->length+PAGE_SIZE>max_segment, then set another sg. In the commit "RDMA/umem: Move to allocate SG table from pages",max_segmentisdma_get_max_seg_size.
Normally it isUINT_MAX. So in my host,prv->length+PAGE_SIZEis usually less thanmax_segment
since length is unsigned int.
Quoted 7 lines
I don't understand what you are trying to say
460 if (
prv->length+PAGE_SIZE>max_segment)
max_segmentshould be a very big number and "prv->length+PAGE_SIZE" should always be <max_segmentso it should always be increasing the size ofprv->lengthand 'rpv' here is the sgl.
Sure.
When max_segment is UINT_MAX, prv->length is UINT_MAX - PAGE_SIZE. It is big. That is, segment size is UINT_MAX - PAGE_SIZE.
But from the function ib_umem_add_sg_table, the prv->length is SZ_2M. That is, segment size if SZ_2M.
It is the difference between the 2 functions.
Zhu Yanjun
Quoted 4 lines
The other loops are the same.
Jason
Quoted 63 lines · Zhu Yanjun
Quoted 51 lines · Jason Gunthorpe
Quoted 41 lines · Zhu Yanjun
Quoted 10 lines · Jason Gunthorpe
Quoted 5 lines · Zhu Yanjun
And I delved into the source code of
__sg_alloc_table_from_pages. I found that this function is related withib_dma_max_seg_size. So whenib_dma_max_seg_sizeis set toUINT_MAX, the sg dma address is 4K (one page). Whenib_dma_max_seg_sizeis set toSZ_2M, the sg dma address is 2M now.That seems like a bug, you should fix it
Hi, Jason && Leon
I compared the function
__sg_alloc_table_from_pageswithib_umem_add_sg_table. In__sg_alloc_table_from_pages:"
449 if (prv) {
450 unsigned long paddr = (page_to_pfn(sg_page(prv))
*PAGE_SIZE+
451prv->offset+prv->length) /
452PAGE_SIZE;
453
454 if (WARN_ON(offset))
455 returnERR_PTR(-EINVAL);
456457 /* Merge contiguous pages into the last SG */ 458 prv_len = prv->length; 459 while (n_pages && page_to_pfn(pages[0]) == paddr) { 460 if (prv->length + PAGE_SIZE > max_segment) 461 break; 462 prv->length += PAGE_SIZE; 463 paddr++; 464 pages++; 465 n_pages--;466 }
467 if (!n_pages)
468 goto out;
469 }Quoted 7 lines
"
ifprv->length+PAGE_SIZE>max_segment, then set another sg. In the commit "RDMA/umem: Move to allocate SG table from pages",max_segmentisdma_get_max_seg_size.
Normally it isUINT_MAX. So in my host,prv->length+PAGE_SIZEis usually less thanmax_segment
since length is unsigned int.Quoted 7 lines
I don't understand what you are trying to say
460 if (
prv->length+PAGE_SIZE>max_segment)
max_segmentshould be a very big number and "prv->length+PAGE_SIZE" should always be <max_segmentso it should always be increasing the size ofprv->lengthand 'rpv' here is the sgl.Sure.
Quoted 7 lines
When
max_segmentisUINT_MAX,prv->lengthisUINT_MAX-PAGE_SIZE. It is big. That is, segment size isUINT_MAX-PAGE_SIZE.But from the function
ib_umem_add_sg_table, theprv->lengthisSZ_2M. That is, segment size ifSZ_2M.It is the difference between the 2 functions.
BTW, my commit is to fix this difference.
Zhu Yanjun
Quoted 8 lines
Zhu Yanjun
Quoted 4 lines
The other loops are the same.
Jason
Jason Gunthorpe
maintainerQuoted 62 lines · Zhu Yanjun
Quoted 51 lines · Jason Gunthorpe
Quoted 41 lines · Zhu Yanjun
Quoted 10 lines · Jason Gunthorpe
Quoted 5 lines · Zhu Yanjun
And I delved into the source code of
__sg_alloc_table_from_pages. I found that this function is related withib_dma_max_seg_size. So whenib_dma_max_seg_sizeis set toUINT_MAX, the sg dma address is 4K (one page). Whenib_dma_max_seg_sizeis set toSZ_2M, the sg dma address is 2M now.That seems like a bug, you should fix it
Hi, Jason && Leon
I compared the function
__sg_alloc_table_from_pageswithib_umem_add_sg_table. In__sg_alloc_table_from_pages:"
449 if (prv) {
450 unsigned long paddr = (page_to_pfn(sg_page(prv))
*PAGE_SIZE+
451prv->offset+prv->length) /
452PAGE_SIZE;
453
454 if (WARN_ON(offset))
455 returnERR_PTR(-EINVAL);
456457 /* Merge contiguous pages into the last SG */ 458 prv_len = prv->length; 459 while (n_pages && page_to_pfn(pages[0]) == paddr) { 460 if (prv->length + PAGE_SIZE > max_segment) 461 break; 462 prv->length += PAGE_SIZE; 463 paddr++; 464 pages++; 465 n_pages--;466 }
467 if (!n_pages)
468 goto out;
469 }Quoted 7 lines
"
ifprv->length+PAGE_SIZE>max_segment, then set another sg. In the commit "RDMA/umem: Move to allocate SG table from pages",max_segmentisdma_get_max_seg_size.
Normally it isUINT_MAX. So in my host,prv->length+PAGE_SIZEis usually less thanmax_segment
since length is unsigned int.Quoted 7 lines
I don't understand what you are trying to say
460 if (
prv->length+PAGE_SIZE>max_segment)
max_segmentshould be a very big number and "prv->length+PAGE_SIZE" should always be <max_segmentso it should always be increasing the size ofprv->lengthand 'rpv' here is the sgl.Sure.
Quoted 7 lines
When
max_segmentisUINT_MAX,prv->lengthisUINT_MAX-PAGE_SIZE. It is big. That is, segment size isUINT_MAX-PAGE_SIZE.But from the function
ib_umem_add_sg_table, theprv->lengthisSZ_2M. That is, segment size ifSZ_2M.It is the difference between the 2 functions.
I still have no idea what you are trying to say. Why would prv->length be 'UINT - PAGE_SIZE'? That sounds like max_segment
Jason
Quoted 67 lines · Jason Gunthorpe
Quoted 65 lines · Zhu Yanjun
Quoted 51 lines · Jason Gunthorpe
Quoted 41 lines · Zhu Yanjun
Quoted 10 lines · Jason Gunthorpe
Quoted 5 lines · Zhu Yanjun
And I delved into the source code of
__sg_alloc_table_from_pages. I found that this function is related withib_dma_max_seg_size. So whenib_dma_max_seg_sizeis set toUINT_MAX, the sg dma address is 4K (one page). Whenib_dma_max_seg_sizeis set toSZ_2M, the sg dma address is 2M now.That seems like a bug, you should fix it
Hi, Jason && Leon
I compared the function
__sg_alloc_table_from_pageswithib_umem_add_sg_table. In__sg_alloc_table_from_pages:"
449 if (prv) {
450 unsigned long paddr = (page_to_pfn(sg_page(prv))
*PAGE_SIZE+
451prv->offset+prv->length) /
452PAGE_SIZE;
453
454 if (WARN_ON(offset))
455 returnERR_PTR(-EINVAL);
456457 /* Merge contiguous pages into the last SG */ 458 prv_len = prv->length; 459 while (n_pages && page_to_pfn(pages[0]) == paddr) { 460 if (prv->length + PAGE_SIZE > max_segment) 461 break; 462 prv->length += PAGE_SIZE; 463 paddr++; 464 pages++; 465 n_pages--;466 }
467 if (!n_pages)
468 goto out;
469 }Quoted 7 lines
"
ifprv->length+PAGE_SIZE>max_segment, then set another sg. In the commit "RDMA/umem: Move to allocate SG table from pages",max_segmentisdma_get_max_seg_size.
Normally it isUINT_MAX. So in my host,prv->length+PAGE_SIZEis usually less thanmax_segment
since length is unsigned int.Quoted 7 lines
I don't understand what you are trying to say
460 if (
prv->length+PAGE_SIZE>max_segment)
max_segmentshould be a very big number and "prv->length+PAGE_SIZE" should always be <max_segmentso it should always be increasing the size ofprv->lengthand 'rpv' here is the sgl.Sure.
When
max_segmentisUINT_MAX,prv->lengthisUINT_MAX-PAGE_SIZE. It is big. That is, segment size isUINT_MAX-PAGE_SIZE.
Quoted 7 lines
Quoted 4 lines
But from the function
ib_umem_add_sg_table, theprv->lengthisSZ_2M. That is, segment size ifSZ_2M.It is the difference between the 2 functions.
I still have no idea what you are trying to say. Why would
prv->lengthbe 'UINT -PAGE_SIZE'? That sounds likemax_segment
Sure. whatever, this prv->length from __sg_alloc_table_from_pages is greater than sg->length from the function ib_umem_add_sg_table.
__sg_alloc_table_from_pages:
"
457 /* Merge contiguous pages into the last SG */
458 prv_len = prv->length;
459 while (n_pages && page_to_pfn(pages[0]) == paddr) {
460 if (prv->length + PAGE_SIZE >
max_segment) <-------------- prv->length > max_segment - PAGE_SIZE
461 break;
462 prv->length += PAGE_SIZE;
463 paddr++;
464 pages++;
465 n_pages--; 466 }
467 if (!n_pages)
468 goto out;
"
Zhu Yanjun
Quoted 2 lines
Jason
Quoted 72 lines · Zhu Yanjun
Quoted 70 lines · Jason Gunthorpe
Quoted 68 lines · Zhu Yanjun
Quoted 51 lines · Jason Gunthorpe
Quoted 41 lines · Zhu Yanjun
Quoted 10 lines · Jason Gunthorpe
Quoted 5 lines · Zhu Yanjun
And I delved into the source code of
__sg_alloc_table_from_pages. I found that this function is related withib_dma_max_seg_size. So whenib_dma_max_seg_sizeis set toUINT_MAX, the sg dma address is 4K (one page). Whenib_dma_max_seg_sizeis set toSZ_2M, the sg dma address is 2M now.That seems like a bug, you should fix it
Hi, Jason && Leon
I compared the function
__sg_alloc_table_from_pageswithib_umem_add_sg_table. In__sg_alloc_table_from_pages:"
449 if (prv) {
450 unsigned long paddr = (page_to_pfn(sg_page(prv))
*PAGE_SIZE+
451prv->offset+prv->length) /
452PAGE_SIZE;
453
454 if (WARN_ON(offset))
455 returnERR_PTR(-EINVAL);
456457 /* Merge contiguous pages into the last SG */ 458 prv_len = prv->length; 459 while (n_pages && page_to_pfn(pages[0]) == paddr) { 460 if (prv->length + PAGE_SIZE > max_segment) 461 break; 462 prv->length += PAGE_SIZE; 463 paddr++; 464 pages++; 465 n_pages--;466 }
467 if (!n_pages)
468 goto out;
469 }Quoted 7 lines
"
ifprv->length+PAGE_SIZE>max_segment, then set another sg. In the commit "RDMA/umem: Move to allocate SG table from pages",max_segmentisdma_get_max_seg_size.
Normally it isUINT_MAX. So in my host,prv->length+PAGE_SIZEis usually less thanmax_segment
since length is unsigned int.Quoted 7 lines
I don't understand what you are trying to say
460 if (
prv->length+PAGE_SIZE>max_segment)
max_segmentshould be a very big number and "prv->length+PAGE_SIZE" should always be <max_segmentso it should always be increasing the size ofprv->lengthand 'rpv' here is the sgl.Sure.
When
max_segmentisUINT_MAX,prv->lengthisUINT_MAX-PAGE_SIZE. It is big. That is, segment size isUINT_MAX-PAGE_SIZE.But from the function
ib_umem_add_sg_table, theprv->lengthisSZ_2M. That is, segment size ifSZ_2M.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
It is the difference between the 2 functions.
I still have no idea what you are trying to say. Why would
prv->lengthbe 'UINT -PAGE_SIZE'? That sounds likemax_segmentSure. whatever, this
prv->lengthfrom__sg_alloc_table_from_pagesis greater thansg->lengthfrom the functionib_umem_add_sg_table.
In short, the sg list from __sg_alloc_table_from_pages is different from the sg list from ib_umem_add_sg_table.
This difference will make difference on the later behavior.
Zhu Yanjun
Quoted 21 lines
__sg_alloc_table_from_pages:
"457 /* Merge contiguous pages into the last SG */ 458 prv_len = prv->length; 459 while (n_pages && page_to_pfn(pages[0]) == paddr) { 460 if (prv->length + PAGE_SIZE > max_segment) <-------------- prv->length > max_segment - PAGE_SIZE 461 break; 462 prv->length += PAGE_SIZE; 463 paddr++; 464 pages++; 465 n_pages--;466 }
467 if (!n_pages)
468 goto out;
"Zhu Yanjun
Quoted 2 lines
Jason
Jason Gunthorpe
maintainerQuoted 2 lines · Zhu Yanjun
In short, the sg list from
__sg_alloc_table_from_pagesis different from the sg list fromib_umem_add_sg_table.
I don't care about different. Tell me what is wrong with what we have today.
I thought your first message said the sgl's were too small, but now you seem to say they are too big?
I still have no idea what this problem is
Jason
Quoted 10 lines · Jason Gunthorpe
Quoted 2 lines · Zhu Yanjun
In short, the sg list from
__sg_alloc_table_from_pagesis different from the sg list fromib_umem_add_sg_table.I don't care about different. Tell me what is wrong with what we have today.
I thought your first message said the sgl's were too small, but now you seem to say they are too big?
Sure.
The sg list from __sg_alloc_table_from_pages, length of sg is too big. And the dma address is like the followings:
"
sg_dma_address(sg):0x4b3c1ce000
sg_dma_address(sg):0x4c3c1cd000
sg_dma_address(sg):0x4d3c1cc000
sg_dma_address(sg):0x4e3c1cb000
"
The sg list from ib_umem_add_sg_table, length of sg is 2M.
And the dma address is like the followings:
"
sg_dma_address(sg):0x203b400000
sg_dma_address(sg):0x213b200000
sg_dma_address(sg):0x223b000000
sg_dma_address(sg):0x233ae00000
sg_dma_address(sg):0x243ac00000
"
The above 2 sg lists will be handled in this function ib_umem_find_best_pgsz. The returned values are different.
Zhu Yanjun
Quoted 4 lines
I still have no idea what this problem is
Jason
Jason Gunthorpe
maintainerQuoted 17 lines · Zhu Yanjun
Quoted 10 lines · Jason Gunthorpe
Quoted 2 lines · Zhu Yanjun
In short, the sg list from
__sg_alloc_table_from_pagesis different from the sg list fromib_umem_add_sg_table.I don't care about different. Tell me what is wrong with what we have today.
I thought your first message said the sgl's were too small, but now you seem to say they are too big?
Sure.
The sg list from
__sg_alloc_table_from_pages, length of sg is too big. And the dma address is like the followings:
Quoted 6 lines
"
sg_dma_address(sg):0x4b3c1ce000
sg_dma_address(sg):0x4c3c1cd000
sg_dma_address(sg):0x4d3c1cc000
sg_dma_address(sg):0x4e3c1cb000
"
Ok, so how does too big a dma segment side cause __sg_alloc_table_from_pages() to return sg elements that are too small?
I assume there is some kind of maths overflow here?
Jason
Quoted 26 lines · Jason Gunthorpe
Quoted 17 lines · Zhu Yanjun
Quoted 10 lines · Jason Gunthorpe
Quoted 2 lines · Zhu Yanjun
In short, the sg list from
__sg_alloc_table_from_pagesis different from the sg list fromib_umem_add_sg_table.I don't care about different. Tell me what is wrong with what we have today.
I thought your first message said the sgl's were too small, but now you seem to say they are too big?
Sure.
The sg list from
__sg_alloc_table_from_pages, length of sg is too big. And the dma address is like the followings:Quoted 6 lines
"
sg_dma_address(sg):0x4b3c1ce000
sg_dma_address(sg):0x4c3c1cd000
sg_dma_address(sg):0x4d3c1cc000
sg_dma_address(sg):0x4e3c1cb000
"
Quoted 5 lines
Ok, so how does too big a dma segment side cause
__sg_alloc_table_from_pages()to return sg elements that are too small?I assume there is some kind of maths overflow here?
Please check this function __sg_alloc_table_from_pages "
...
457 /* Merge contiguous pages into the last SG */
458 prv_len = prv->length;
459 while (n_pages && page_to_pfn(pages[0]) == paddr) { 460 if (prv->length + PAGE_SIZE >max_segment) <--max_segment is too big. So n_pages will be 0. Then
the function will goto out to exit.
461 break;
462 prv->length += PAGE_SIZE; 463 paddr++; 464 pages++; 465 n_pages--;
466 }
467 if (!n_pages)
468 goto out;
...
"
Quoted 2 lines
Jason
Jason Gunthorpe
maintainerQuoted 35 lines · Zhu Yanjun
Quoted 26 lines · Jason Gunthorpe
Quoted 17 lines · Zhu Yanjun
Quoted 10 lines · Jason Gunthorpe
Quoted 2 lines · Zhu Yanjun
In short, the sg list from
__sg_alloc_table_from_pagesis different from the sg list fromib_umem_add_sg_table.I don't care about different. Tell me what is wrong with what we have today.
I thought your first message said the sgl's were too small, but now you seem to say they are too big?
Sure.
The sg list from
__sg_alloc_table_from_pages, length of sg is too big. And the dma address is like the followings:Quoted 6 lines
"
sg_dma_address(sg):0x4b3c1ce000
sg_dma_address(sg):0x4c3c1cd000
sg_dma_address(sg):0x4d3c1cc000
sg_dma_address(sg):0x4e3c1cb000
"Quoted 5 lines
Ok, so how does too big a dma segment side cause
__sg_alloc_table_from_pages()to return sg elements that are too small?I assume there is some kind of maths overflow here?
Please check this function
__sg_alloc_table_from_pages"
...
Quoted 6 lines
457 /* Merge contiguous pages into the last SG */ 458 prv_len = prv->length; 459 while (n_pages && page_to_pfn(pages[0]) == paddr) {460 if (
prv->length+PAGE_SIZE>max_segment) <--max_segment is too big. Son_pageswill be 0. Then
the function will goto out to exit.
You already said this.
You are reporting 4k pages, if max_segment is larger than 4k there is no such thing as "too big"
I assume it is "too small" because of some maths overflow.
You should add some prints and find out what is going on.
Jason
Quoted 44 lines · Jason Gunthorpe
Quoted 35 lines · Zhu Yanjun
Quoted 26 lines · Jason Gunthorpe
Quoted 17 lines · Zhu Yanjun
Quoted 10 lines · Jason Gunthorpe
Quoted 2 lines · Zhu Yanjun
In short, the sg list from
__sg_alloc_table_from_pagesis different from the sg list fromib_umem_add_sg_table.I don't care about different. Tell me what is wrong with what we have today.
I thought your first message said the sgl's were too small, but now you seem to say they are too big?
Sure.
The sg list from
__sg_alloc_table_from_pages, length of sg is too big. And the dma address is like the followings:Quoted 6 lines
"
sg_dma_address(sg):0x4b3c1ce000
sg_dma_address(sg):0x4c3c1cd000
sg_dma_address(sg):0x4d3c1cc000
sg_dma_address(sg):0x4e3c1cb000
"Quoted 5 lines
Ok, so how does too big a dma segment side cause
__sg_alloc_table_from_pages()to return sg elements that are too small?I assume there is some kind of maths overflow here?
Please check this function
__sg_alloc_table_from_pages"
...Quoted 6 lines
457 /* Merge contiguous pages into the last SG */ 458 prv_len = prv->length; 459 while (n_pages && page_to_pfn(pages[0]) == paddr) {460 if (
prv->length+PAGE_SIZE>max_segment) <--max_segment is too big. Son_pageswill be 0. Then
the function will goto out to exit.
Quoted 6 lines
You already said this.
You are reporting 4k pages, if
max_segmentis larger than 4k there is no such thing as "too big"I assume it is "too small" because of some maths overflow.
459 while (n_pages && page_to_pfn(pages[0]) == paddr) {
460 if (prv->length + PAGE_SIZE >max_segment) <--it max_segment is big, n_pages is zero.
461 break;
462 prv->length += PAGE_SIZE; 463 paddr++; 464 pages++; 465 n_pages--;
466 }
467 if (!n_pages) <---here, this function will goto out.
468 goto out;
...
509 chunk_size = ((j - cur_page) << PAGE_SHIFT) - offset;
510 sg_set_page(s, pages[cur_page],
511 min_t(unsigned long, size,chunk_size), offset); <----this function will not have many chance to be called if max_segment is big.
512 added_nents++;
513 size -= chunk_size;
If the max_segment is not big enough, for example it is SZ-2M, sg_set_page will be called every SZ_2M.
To now, I do not find any math overflow.
Zhu Yanjun
Quoted 4 lines
You should add some prints and find out what is going on.
Jason
Quoted 73 lines · Zhu Yanjun
Quoted 44 lines · Jason Gunthorpe
Quoted 35 lines · Zhu Yanjun
Quoted 26 lines · Jason Gunthorpe
Quoted 17 lines · Zhu Yanjun
Quoted 10 lines · Jason Gunthorpe
Quoted 2 lines · Zhu Yanjun
In short, the sg list from
__sg_alloc_table_from_pagesis different from the sg list fromib_umem_add_sg_table.I don't care about different. Tell me what is wrong with what we have today.
I thought your first message said the sgl's were too small, but now you seem to say they are too big?
Sure.
The sg list from
__sg_alloc_table_from_pages, length of sg is too big. And the dma address is like the followings:Quoted 6 lines
"
sg_dma_address(sg):0x4b3c1ce000
sg_dma_address(sg):0x4c3c1cd000
sg_dma_address(sg):0x4d3c1cc000
sg_dma_address(sg):0x4e3c1cb000
"Quoted 5 lines
Ok, so how does too big a dma segment side cause
__sg_alloc_table_from_pages()to return sg elements that are too small?I assume there is some kind of maths overflow here?
Please check this function
__sg_alloc_table_from_pages"
...Quoted 6 lines
457 /* Merge contiguous pages into the last SG */ 458 prv_len = prv->length; 459 while (n_pages && page_to_pfn(pages[0]) == paddr) {460 if (
prv->length+PAGE_SIZE>max_segment) <--max_segment is too big. Son_pageswill be 0. Then
the function will goto out to exit.Quoted 6 lines
You already said this.
You are reporting 4k pages, if
max_segmentis larger than 4k there is no such thing as "too big"I assume it is "too small" because of some maths overflow.
459 while (
n_pages&&page_to_pfn(pages[0])== paddr) {
460 if (prv->length+PAGE_SIZE>max_segment) <--itmax_segmentis big,n_pagesis zero.
461 break;462 prv->length += PAGE_SIZE; 463 paddr++; 464 pages++; 465 n_pages--;466 }
467 if (!n_pages) <---here, this function will goto out.
468 goto out;
...
509chunk_size= ((j -cur_page) <<PAGE_SHIFT) - offset;
510 sg_set_page(s, pages[cur_page],
511 min_t(unsigned long, size,chunk_size), offset); <----this function will not have many chance to be called ifmax_segmentis big.
512 added_nents++;
513 size -=chunk_size;
Quoted 3 lines
If the
max_segmentis not big enough, for example it is SZ-2M,sg_set_pagewill be called everySZ_2M.
To now, I do not find any math overflow.
147 struct ib_umem *ib_umem_get(struct ib_device *device, unsigned long addr,
148 size_t size, int access)
149 {
...
244 umem->nmap =
245 ib_dma_map_sg_attrs(device, umem->sg_head.sgl,umem->sg_nents,
246 DMA_BIDIRECTIONAL, dma_attr);
...
And after the function ib_dma_map_sg_attrs, dma address is set. To now, I can not find maths overflow.
Zhu Yanjun
Quoted 6 lines
Zhu Yanjun
Quoted 4 lines
You should add some prints and find out what is going on.
Jason
Jason Gunthorpe
maintainerQuoted 47 lines · Zhu Yanjun
Quoted 46 lines · Jason Gunthorpe
Quoted 35 lines · Zhu Yanjun
Quoted 26 lines · Jason Gunthorpe
Quoted 17 lines · Zhu Yanjun
Quoted 10 lines · Jason Gunthorpe
Quoted 2 lines · Zhu Yanjun
In short, the sg list from
__sg_alloc_table_from_pagesis different from the sg list fromib_umem_add_sg_table.I don't care about different. Tell me what is wrong with what we have today.
I thought your first message said the sgl's were too small, but now you seem to say they are too big?
Sure.
The sg list from
__sg_alloc_table_from_pages, length of sg is too big. And the dma address is like the followings:Quoted 6 lines
"
sg_dma_address(sg):0x4b3c1ce000
sg_dma_address(sg):0x4c3c1cd000
sg_dma_address(sg):0x4d3c1cc000
sg_dma_address(sg):0x4e3c1cb000
"Quoted 5 lines
Ok, so how does too big a dma segment side cause
__sg_alloc_table_from_pages()to return sg elements that are too small?I assume there is some kind of maths overflow here?
Please check this function
__sg_alloc_table_from_pages"
...Quoted 6 lines
457 /* Merge contiguous pages into the last SG */ 458 prv_len = prv->length; 459 while (n_pages && page_to_pfn(pages[0]) == paddr) {460 if (
prv->length+PAGE_SIZE>max_segment) <--max_segment is too big. Son_pageswill be 0. Then
the function will goto out to exit.You already said this.
Quoted 8 lines
Quoted 4 lines
You are reporting 4k pages, if
max_segmentis larger than 4k there is no such thing as "too big"I assume it is "too small" because of some maths overflow.
459 while (
n_pages&&page_to_pfn(pages[0])== paddr) {
460 if (prv->length+PAGE_SIZE>max_segment) <--itmax_segmentis big,n_pagesis zero.
What does n_pages have to do with max_segment?
Please try to explain clearly.
Jason
Quoted 52 lines · Jason Gunthorpe
Quoted 50 lines · Zhu Yanjun
Quoted 49 lines · Jason Gunthorpe
Quoted 35 lines · Zhu Yanjun
Quoted 26 lines · Jason Gunthorpe
Quoted 17 lines · Zhu Yanjun
Quoted 10 lines · Jason Gunthorpe
Quoted 2 lines · Zhu Yanjun
In short, the sg list from
__sg_alloc_table_from_pagesis different from the sg list fromib_umem_add_sg_table.I don't care about different. Tell me what is wrong with what we have today.
I thought your first message said the sgl's were too small, but now you seem to say they are too big?
Sure.
The sg list from
__sg_alloc_table_from_pages, length of sg is too big. And the dma address is like the followings:Quoted 6 lines
"
sg_dma_address(sg):0x4b3c1ce000
sg_dma_address(sg):0x4c3c1cd000
sg_dma_address(sg):0x4d3c1cc000
sg_dma_address(sg):0x4e3c1cb000
"Quoted 5 lines
Ok, so how does too big a dma segment side cause
__sg_alloc_table_from_pages()to return sg elements that are too small?I assume there is some kind of maths overflow here?
Please check this function
__sg_alloc_table_from_pages"
...Quoted 6 lines
457 /* Merge contiguous pages into the last SG */ 458 prv_len = prv->length; 459 while (n_pages && page_to_pfn(pages[0]) == paddr) {460 if (
prv->length+PAGE_SIZE>max_segment) <--max_segment is too big. Son_pageswill be 0. Then
the function will goto out to exit.You already said this.
You are reporting 4k pages, if
max_segmentis larger than 4k there is no such thing as "too big"
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
I assume it is "too small" because of some maths overflow.
459 while (
n_pages&&page_to_pfn(pages[0])== paddr) {
460 if (prv->length+PAGE_SIZE>max_segment) <--itmax_segmentis big,n_pagesis zero.What does
n_pageshave to do withmax_segment?
With the following snippet
"
struct ib_umem *region;
region = ib_umem_get(pd->device, start, len, access);
page_size = ib_umem_find_best_pgsz(region,
SZ_4K | SZ_2M | SZ_1G,
virt);"
Before the commit 0c16d9635e3a ("RDMA/umem: Move to allocate SG table from pages"),
the variable page_size is SZ_2M.
After the commit 0c16d9635e3a ("RDMA/umem: Move to allocate SG table from pages"),
the variable page_size is SZ_4K.
IMHO, you can reproduce this problem in your local host.
Zhu Yanjun
Quoted 4 lines
Please try to explain clearly.
Jason
Jason Gunthorpe
maintainerQuoted 13 lines · Zhu Yanjun
Quoted 10 lines
Quoted 8 lines
Quoted 4 lines
You are reporting 4k pages, if
max_segmentis larger than 4k there is no such thing as "too big"I assume it is "too small" because of some maths overflow.
459 while (
n_pages&&page_to_pfn(pages[0])== paddr) {
460 if (prv->length+PAGE_SIZE>max_segment) <--itmax_segmentis big,n_pagesis zero.What does
n_pageshave to do withmax_segment?With the following snippet
"
Quoted 7 lines
struct ib_umem *region; region = ib_umem_get(pd->device, start, len, access); page_size = ib_umem_find_best_pgsz(region, SZ_4K | SZ_2M | SZ_1G, virt);"
Can you please stop posting random bits of code that do nothing to answer the question?
Quoted 1 line
IMHO, you can reproduce this problem in your local host.
Uh, no, you need to communicate effectively or stop posting please.
Jason
Quoted 23 lines · Jason Gunthorpe
Quoted 13 lines · Zhu Yanjun
Quoted 10 lines
Quoted 8 lines
Quoted 4 lines
You are reporting 4k pages, if
max_segmentis larger than 4k there is no such thing as "too big"I assume it is "too small" because of some maths overflow.
459 while (
n_pages&&page_to_pfn(pages[0])== paddr) {
460 if (prv->length+PAGE_SIZE>max_segment) <--itmax_segmentis big,n_pagesis zero.What does
n_pageshave to do withmax_segment?With the following snippet
"Quoted 7 lines
struct ib_umem *region; region = ib_umem_get(pd->device, start, len, access); page_size = ib_umem_find_best_pgsz(region, SZ_4K | SZ_2M | SZ_1G, virt);"
Quoted 6 lines
Can you please stop posting random bits of code that do nothing to answer the question?
Quoted 1 line
IMHO, you can reproduce this problem in your local host.
Uh, no, you need to communicate effectively or stop posting please.
can you read the following link again
In this link, I explained it in details.
Since the max_segment is very big, so normally the prv->length + PAGE_SIZE is less than max_segment, it will loop over and over again, until n_pages is zero,
then the function __sg_alloc_table_from_pages will exit. So the function __sg_alloc_table_from_pages has few chances to call the function sg_set_page.
This behavior is not like the function ib_umem_add_sg_table. In ib_umem_add_sg_table, the "max_seg_sz - sg->length" has more chances to be greater than (len << PAGE_SHIFT).
So the function sg_set_page is called more times than __sg_alloc_table_from_pages.
The above causes the different sg lists from the 2 functions __sg_alloc_table_from_pages and ib_umem_add_sg_table.
The most important thing is "if (prv->length + PAGE_SIZE > max_segment)" in __sg_alloc_table_from_pages and "if ((max_seg_sz - sg->length) >= (len << PAGE_SHIFT))" in the function ib_umem_add_sg_table.
When max_segment is UINT_MAX, the different test in the 2 functions causes the different sg lists.
Zhu Yanjun
Quoted 2 lines
Jason
Jason Gunthorpe
maintainerQuoted 29 lines · Zhu Yanjun
Quoted 28 lines · Jason Gunthorpe
Quoted 13 lines · Zhu Yanjun
Quoted 10 lines
Quoted 8 lines
Quoted 4 lines
You are reporting 4k pages, if
max_segmentis larger than 4k there is no such thing as "too big"I assume it is "too small" because of some maths overflow.
459 while (
n_pages&&page_to_pfn(pages[0])== paddr) {
460 if (prv->length+PAGE_SIZE>max_segment) <--itmax_segmentis big,n_pagesis zero.What does
n_pageshave to do withmax_segment?With the following snippet
"Quoted 7 lines
struct ib_umem *region; region = ib_umem_get(pd->device, start, len, access); page_size = ib_umem_find_best_pgsz(region, SZ_4K | SZ_2M | SZ_1G, virt);"
Can you please stop posting random bits of code that do nothing to answer the question?
Quoted 1 line
IMHO, you can reproduce this problem in your local host.
Quoted 7 lines
Quoted 1 line
Uh, no, you need to communicate effectively or stop posting please.
can you read the following link again
In this link, I explained it in details.
No, this is all nonsense
Quoted 3 lines
Since the
max_segmentis very big, so normally theprv->length+PAGE_SIZEis less thanmax_segment, it will loop over and over again, untiln_pagesis zero,
And each iteration increases
prv->length += PAGE_SIZE; n_pages--;
Where prv is the last sgl entry, so decreasing n_pages cannot also yield a 4k sgl entry like you are reporting.
You get 4k sgl entries if max_segment is *too small* not too big!
Jason
Quoted 45 lines · Jason Gunthorpe
Quoted 29 lines · Zhu Yanjun
Quoted 28 lines · Jason Gunthorpe
Quoted 13 lines · Zhu Yanjun
Quoted 10 lines
Quoted 8 lines
Quoted 4 lines
You are reporting 4k pages, if
max_segmentis larger than 4k there is no such thing as "too big"I assume it is "too small" because of some maths overflow.
459 while (
n_pages&&page_to_pfn(pages[0])== paddr) {
460 if (prv->length+PAGE_SIZE>max_segment) <--itmax_segmentis big,n_pagesis zero.What does
n_pageshave to do withmax_segment?With the following snippet
"Quoted 7 lines
struct ib_umem *region; region = ib_umem_get(pd->device, start, len, access); page_size = ib_umem_find_best_pgsz(region, SZ_4K | SZ_2M | SZ_1G, virt);"
Can you please stop posting random bits of code that do nothing to answer the question?
Quoted 1 line
IMHO, you can reproduce this problem in your local host.
Quoted 7 lines
Quoted 1 line
Uh, no, you need to communicate effectively or stop posting please.
can you read the following link again
In this link, I explained it in details.
No, this is all nonsense
Quoted 3 lines
Since the
max_segmentis very big, so normally theprv->length+PAGE_SIZEis less thanmax_segment, it will loop over and over again, untiln_pagesis zero,
Quoted 7 lines
And each iteration increases
prv->length += PAGE_SIZE; n_pages--;Where prv is the last sgl entry, so decreasing
n_pagescannot also yield a 4k sgl entry like you are reporting.
No. You suppose n_pages is very big, then prv->length will increase to reach max_segment.
If n_pages is very small, for example, we suppose n_pages is 1, then prv->length will
not reach max_segment.
In fact, in my test case, n_pages is very small.
Zhu Yanjun
Quoted 4 lines
You get 4k sgl entries if
max_segmentis *too small* not too big!Jason
Jason Gunthorpe
maintainerQuoted 7 lines · Zhu Yanjun
No. You suppose
n_pagesis very big, thenprv->lengthwill increase to reachmax_segment.
Ifn_pagesis very small, for example, we supposen_pagesis 1, thenprv->lengthwill
not reachmax_segment.In fact, in my test case,
n_pagesis very small.
n_pages is an *input* it is the number of pages in the VA range, if it is very small it is not a problem with this algorithm
Jason
Got it. Thanks a lot for your help.
Zhu Yanjun
Quoted 15 lines · Jason Gunthorpe
Quoted 7 lines · Zhu Yanjun
No. You suppose
n_pagesis very big, thenprv->lengthwill increase to reachmax_segment.
Ifn_pagesis very small, for example, we supposen_pagesis 1, thenprv->lengthwill
not reachmax_segment.In fact, in my test case,
n_pagesis very small.
n_pagesis an *input* it is the number of pages in the VA range, if it is very small it is not a problem with this algorithmJason
Jason Gunthorpe
maintainerQuoted 4 lines · Zhu Yanjun
From: Zhu Yanjun <[email protected]>
After the commit ("RDMA/umem: Move to allocate SG table from pages"), the sg list from
ib_ume_getis like the following:
If you are not getting huge pages then you need to fix __sg_alloc_table_from_pages()
But as far as I know it is not buggy like you are describing.
Quoted 2 lines
The new function is
ib_umem_gettoib_umem_hugepage_getthat callsib_umem_add_sg_table.
Nope
Jason
Quoted 11 lines · Jason Gunthorpe
Quoted 4 lines · Zhu Yanjun
From: Zhu Yanjun <[email protected]>
After the commit ("RDMA/umem: Move to allocate SG table from pages"), the sg list from
ib_ume_getis like the following:If you are not getting huge pages then you need to fix
__sg_alloc_table_from_pages()But as far as I know it is not buggy like you are describing.
In the same host and the same test, both __sg_alloc_table_from_pages and ib_umem_add_sg_table
are called at the same time, but the returned sg lists are different. It is weird.
Zhu Yanjun
Quoted 7 lines
Quoted 3 lines
The new function is
ib_umem_gettoib_umem_hugepage_getthat callsib_umem_add_sg_table.Nope
Jason