hmm_range_fault related fixes and legacy API removal v3
Subsystems (8)
Hi Jérôme, Ben, Felxi and Jason,
below is a series against the hmm tree which cleans up various minor bits and allows HMM_MIRROR to be built on all architectures.
Diffstat:
7 files changed, 81 insertions(+), 171 deletions(-)
A git tree is also available at:
git://git.infradead.org/users/hch/misc.git hmm-cleanups
Gitweb:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/hmm-cleanups
PATCH 1/13 amdgpu: remove -EAGAIN handling for hmm_range_fault
hmm_range_fault can only return -EAGAIN if called with the block argument set to false, so remove the special handling for it.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 23 +++-------------------- 1 file changed, 3 insertions(+), 20 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +3 -20
@@ -778,7 +778,6 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages)struct hmm_range *range;unsigned long i;uint64_t *pfns; -int retry =0 ;int r =0 ;if (!mm)/* Happens during process shutdown */
@@ -822,7 +821,6 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages)hmm_range_register (range, mirror, start, start + ttm->num_pages *PAGE_SIZE ,PAGE_SHIFT ); -retry:/* * Just wait for range to be valid, safe to ignore return value as we * will use the return value of hmm_range_fault() below under the
@@ -831,24 +829,12 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages)hmm_range_wait_until_valid (range,HMM_RANGE_DEFAULT_TIMEOUT );down_read (&mm->mmap_sem); - r =hmm_range_fault (range,0 ); -if (unlikely (r <0 )) { -if (likely (r == -EAGAIN )) { -/* -* return -EAGAIN, mmap_sem is dropped -*/ -if (retry++ <MAX_RETRY_HMM_RANGE_FAULT ) - goto retry; -else -pr_err ("Retry hmm fault too many times\n" ); - } - - goto out_up_read; - } -up_read (&mm->mmap_sem); +if (unlikely (r <0 )) + goto out_free_pfns; +for (i =0 ; i < ttm->num_pages; i++) { pages[i] =hmm_device_entry_to_page (range, pfns[i]);if (unlikely (!pages[i])) {
@@ -864,9 +850,6 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages)return 0 ; -out_up_read: -if (likely (r != -EAGAIN )) -up_read (&mm->mmap_sem); out_free_pfns:hmm_range_unregister (range);kvfree (pfns);
Signature · 1 line
2.20.1
Re: [PATCH 01/13] amdgpu: remove -EAGAIN handling for hmm_range_fault
Quoted 6 lines · Christoph Hellwig
hmm_range_faultcan only return-EAGAINif called with the block argument set to false, so remove the special handling for it.Signed-off-by Christoph Hellwig <[email protected]>drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 23 +++-------------------- 1 file changed, 3 insertions(+), 20 deletions(-)
Jason
Re: [PATCH 01/13] amdgpu: remove -EAGAIN handling for hmm_range_fault
In reply to Christoph HellwigQuoted 2 lines · Christoph Hellwig
hmm_range_faultcan only return-EAGAINif called with the block argument set to false, so remove the special handling for it.
The block argument no longer exists. You replaced that with the HMM_FAULT_ALLOW_RETRY with opposite logic. So this should read "hmm_range_fault can only return -EAGAIN if called with the HMM_FAULT_ALLOW_RETRY flag set, so remove the special handling for it.
With that fixed, this commit is Reviewed-by: Felix Kuehling <[email protected]>
Quoted 64 lines
Signed-off-by Christoph Hellwig <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 23 +++-------------------- 1 file changed, 3 insertions(+), 20 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +3
-20@@ -778,7 +778,6 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages)struct hmm_range *range;unsigned long i;uint64_t *pfns; -int retry =0 ;int r =0 ;if (!mm)/* Happens during process shutdown */ @@ -822,7 +821,6 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages)hmm_range_register (range, mirror, start, start + ttm->num_pages *PAGE_SIZE ,PAGE_SHIFT ); -retry:/* * Just wait for range to be valid, safe to ignore return value as we * will use the return value of hmm_range_fault() below under the @@ -831,24 +829,12 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages)hmm_range_wait_until_valid (range,HMM_RANGE_DEFAULT_TIMEOUT );down_read (&mm->mmap_sem); - r =hmm_range_fault (range,0 ); -if (unlikely (r <0 )) { -if (likely (r == -EAGAIN )) { -/* -* return -EAGAIN, mmap_sem is dropped -*/ -if (retry++ <MAX_RETRY_HMM_RANGE_FAULT ) - goto retry; -else -pr_err ("Retry hmm fault too many times\n" ); - } - - goto out_up_read; - } -up_read (&mm->mmap_sem); +if (unlikely (r <0 )) + goto out_free_pfns; +for (i =0 ; i < ttm->num_pages; i++) { pages[i] =hmm_device_entry_to_page (range, pfns[i]);if (unlikely (!pages[i])) {@@ -864,9 +850,6 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages)return 0 ; -out_up_read: -if (likely (r != -EAGAIN )) -up_read (&mm->mmap_sem); out_free_pfns:hmm_range_unregister (range);kvfree (pfns);
PATCH 2/13 amdgpu: don't initialize range->list in amdgpu_hmm_init_range
In reply to Christoph HellwigThe list is used to add the range to another list as an entry in the core hmm code, so there is no need to initialize it in a driver.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 1 - 1 file changed, 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +0 -1
@@ -484,6 +484,5 @@ void amdgpu_hmm_init_range(struct hmm_range *range) range->flags = hmm_range_flags; range->values = hmm_range_values; range->pfn_shift =PAGE_SHIFT ; -INIT_LIST_HEAD (&range->list); } }
Signature · 6 lines
2.20.1 _______________________________________________ Nouveau mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/nouveau
Re: [PATCH 02/13] amdgpu: don't initialize range->list in amdgpu_hmm_init_range
Quoted 7 lines · Christoph Hellwig
The list is used to add the range to another list as an entry in the core hmm code, so there is no need to initialize it in a driver.
Signed-off-by Christoph Hellwig <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 1 - 1 file changed, 1 deletion(-)
Jason
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
Re: [PATCH 02/13] amdgpu: don't initialize range->list in amdgpu_hmm_init_range
In reply to Christoph HellwigQuoted 2 lines · Christoph Hellwig
The list is used to add the range to another list as an entry in the core hmm code, so there is no need to initialize it in a driver.
I've seen code that uses list_empty to check whether a list head has been added to a list or not. For that to work, the list head needs to be initialized, and it has to be removed with list_del_init. If HMM doesn't ever do that with range->list, then this patch is Reviewed-by: Felix Kuehling <[email protected]>
Quoted 17 lines
Signed-off-by Christoph Hellwig <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 1 - 1 file changed, 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +0
-1@@ -484,6 +484,5 @@ void amdgpu_hmm_init_range(struct hmm_range *range) range->flags = hmm_range_flags; range->values = hmm_range_values; range->pfn_shift =PAGE_SHIFT ; -INIT_LIST_HEAD (&range->list); } }
Re: [PATCH 02/13] amdgpu: don't initialize range->list in amdgpu_hmm_init_range
Quoted 7 lines · Felix
Quoted 2 lines · Christoph Hellwig
The list is used to add the range to another list as an entry in the core hmm code, so there is no need to initialize it in a driver.
I've seen code that uses
list_emptyto check whether a list head has been added to a list or not. For that to work, the list head needs to be initialized, and it has to be removed withlist_del_init.
I think the ida is that 'list' is a private member of range and drivers shouldn't touch it at all.
Quoted 2 lines
ever do that with
range->list, then this patch is Reviewed-by: Felix Kuehling <[email protected]>
Please put tags on their own empty line so that patchworks will collect them automatically..
Jason
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
PATCH 3/13 nouveau: pass struct nouveau_svmm to nouveau_range_fault
In reply to Christoph HellwigThis avoid having to abuse the vma field in struct hmm_range to unlock the mmap_sem.
--- drivers/gpu/drm/nouveau/nouveau_svm.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/nouveau/nouveau_svm.c +3 -3
@@ -485,14 +485,14 @@ nouveau_range_done(struct hmm_range *range) }static int -nouveau_range_fault (struct hmm_mirror *mirror,struct hmm_range *range) +nouveau_range_fault (struct nouveau_svmm *svmm,struct hmm_range *range) {long ret; range->default_flags =0 ; range->pfn_flags_mask =-1UL ; - ret =hmm_range_register (range, mirror, + ret =hmm_range_register (range, &svmm->mirror, range->start, range->end,PAGE_SHIFT );if (ret) {
@@ -689,7 +689,7 @@ nouveau_svm_fault(struct nvif_notify *notify) range.values = nouveau_svm_pfn_values; range.pfn_shift =NVIF_VMM_PFNMAP_V0_ADDR_SHIFT ; again: - ret =nouveau_range_fault (&svmm->mirror, &range); + ret =nouveau_range_fault (svmm, &range);if (ret ==0 ) {mutex_lock (&svmm->mutex);if (!nouveau_range_done (&range)) {
Signature · 6 lines
2.20.1 _______________________________________________ Nouveau mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/nouveau
Re: [PATCH 03/13] nouveau: pass struct nouveau_svmm to nouveau_range_fault
Quoted 2 lines · Christoph Hellwig
This avoid having to abuse the vma field in struct
hmm_rangeto unlock themmap_sem.
I think the change inside hmm_range_fault got lost on rebase, it is now using:
up_read(&range->hmm->mm->mmap_sem);
But, yes, lets change it to use svmm->mm and try to keep struct hmm opaque to drivers
Jason
Re: [PATCH 03/13] nouveau: pass struct nouveau_svmm to nouveau_range_fault
Quoted 11 lines · Jason Gunthorpe
Quoted 2 lines · Christoph Hellwig
This avoid having to abuse the vma field in struct
hmm_rangeto unlock themmap_sem.I think the change inside
hmm_range_faultgot lost on rebase, it is now using:up_read(&range->hmm->mm->mmap_sem);But, yes, lets change it to use
svmm->mmand try to keep struct hmm opaque to drivers
It got lost somewhat intentionally as I didn't want the churn, but I forgot to update the changelog. But if you are fine with changing it over I can bring it back.
Re: [PATCH 03/13] nouveau: pass struct nouveau_svmm to nouveau_range_fault
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Jason Gunthorpe
Quoted 2 lines · Christoph Hellwig
This avoid having to abuse the vma field in struct
hmm_rangeto unlock themmap_sem.I think the change inside
hmm_range_faultgot lost on rebase, it is now using:
Quoted 8 lines
Quoted 4 lines
up_read(&range->hmm->mm->mmap_sem);But, yes, lets change it to use
svmm->mmand try to keep struct hmm opaque to driversIt got lost somewhat intentionally as I didn't want the churn, but I forgot to update the changelog. But if you are fine with changing it over I can bring it back.
I have a patch deleting hmm->mm, so using svmm seems cleaner churn here, we could defer and I can fold this into that patch?
Jason
Re: [PATCH 03/13] nouveau: pass struct nouveau_svmm to nouveau_range_fault
Quoted 2 lines · Jason Gunthorpe
I have a patch deleting
hmm->mm, so using svmm seems cleaner churn here, we could defer and I can fold this into that patch?
Sounds good. If I don't need to resend feel fee to fold it, otherwise I'll fix it up.
PATCH 4/13 mm: remove the pgmap field from struct hmm_vma_walk
In reply to Christoph HellwigThere is only a single place where the pgmap is passed over a function call, so replace it with local variables in the places where we deal with the pgmap.
--- mm/hmm.c | 62 ++++++++++++++++++++++++-------------------------------- 1 file changed, 27 insertions(+), 35 deletions(-)
mm/hmm.c +27 -35
@@ -278,7 +278,6 @@ EXPORT_SYMBOL(hmm_mirror_unregister);struct hmm_vma_walk {struct hmm_range *range; -struct dev_pagemap *pgmap;unsigned long last;unsigned int flags; };
@@ -475,6 +474,7 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk,#ifdef CONFIG_TRANSPARENT_HUGEPAGE struct hmm_vma_walk *hmm_vma_walk = walk->private;struct hmm_range *range = hmm_vma_walk->range; +struct dev_pagemap *pgmap =NULL ;unsigned long pfn, npages, i;bool fault, write_fault;uint64_t cpu_flags;
@@ -490,17 +490,14 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk, pfn =pmd_pfn (pmd) +pte_index (addr);for (i =0 ; addr < end; addr +=PAGE_SIZE , i++, pfn++) {if (pmd_devmap (pmd)) { - hmm_vma_walk->pgmap =get_dev_pagemap (pfn, - hmm_vma_walk->pgmap); -if (unlikely (!hmm_vma_walk->pgmap)) + pgmap =get_dev_pagemap (pfn, pgmap); +if (unlikely (!pgmap))return -EBUSY ; } pfns[i] =hmm_device_entry_from_pfn (range, pfn) | cpu_flags; } -if (hmm_vma_walk->pgmap) { -put_dev_pagemap (hmm_vma_walk->pgmap); - hmm_vma_walk->pgmap =NULL ; - } +if (pgmap) +put_dev_pagemap (pgmap); hmm_vma_walk->last = end;return 0 ;#else
@@ -520,7 +517,7 @@ static inline uint64_t pte_to_hmm_pfn_flags(struct hmm_range *range, pte_t pte)static int hmm_vma_handle_pte (struct mm_walk *walk,unsigned long addr,unsigned long end,pmd_t *pmdp,pte_t *ptep, -uint64_t *pfn) +uint64_t *pfn,struct dev_pagemap **pgmap) {struct hmm_vma_walk *hmm_vma_walk = walk->private;struct hmm_range *range = hmm_vma_walk->range;
@@ -591,9 +588,8 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, goto fault;if (pte_devmap (pte)) { - hmm_vma_walk->pgmap =get_dev_pagemap (pte_pfn (pte), - hmm_vma_walk->pgmap); -if (unlikely (!hmm_vma_walk->pgmap)) + *pgmap =get_dev_pagemap (pte_pfn (pte), *pgmap); +if (unlikely (!*pgmap))return -EBUSY ; }else if (IS_ENABLED (CONFIG_ARCH_HAS_PTE_SPECIAL ) &&pte_special (pte)) { *pfn = range->values[HMM_PFN_SPECIAL ];
@@ -604,10 +600,10 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr,return 0 ; fault: -if (hmm_vma_walk->pgmap) { -put_dev_pagemap (hmm_vma_walk->pgmap); - hmm_vma_walk->pgmap =NULL ; - } +if (*pgmap) +put_dev_pagemap (*pgmap); + *pgmap =NULL ; +pte_unmap (ptep);/* Fault any virtual address we were asked to fault */ return hmm_vma_walk_hole_ (addr, end, fault, write_fault, walk);
@@ -620,6 +616,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, {struct hmm_vma_walk *hmm_vma_walk = walk->private;struct hmm_range *range = hmm_vma_walk->range; +struct dev_pagemap *pgmap =NULL ;uint64_t *pfns = range->pfns;unsigned long addr = start, i;pte_t *ptep;
@@ -683,23 +680,21 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp,for (; addr < end; addr +=PAGE_SIZE , ptep++, i++) {int r; - r =hmm_vma_handle_pte (walk, addr, end, pmdp, ptep, &pfns[i]); + r =hmm_vma_handle_pte (walk, addr, end, pmdp, ptep, &pfns[i], + &pgmap);if (r) {/* hmm_vma_handle_pte() did unmap pte directory */ hmm_vma_walk->last = addr;return r; } } -if (hmm_vma_walk->pgmap) { -/* -* We do put_dev_pagemap() here and not in hmm_vma_handle_pte() -* so that we can leverage get_dev_pagemap() optimization which -* will not re-take a reference on a pgmap if we already have -* one. -*/ -put_dev_pagemap (hmm_vma_walk->pgmap); - hmm_vma_walk->pgmap =NULL ; - } +/* +* We do put_dev_pagemap() here and not in hmm_vma_handle_pte() so that +* we can leverage the get_dev_pagemap() optimization which will not +* re-take a reference on a pgmap if we already have one. +*/ +if (pgmap) +put_dev_pagemap (pgmap);pte_unmap (ptep -1 ); hmm_vma_walk->last = addr;
@@ -714,6 +709,7 @@ static int hmm_vma_walk_pud(pud_t *pudp,struct hmm_vma_walk *hmm_vma_walk = walk->private;struct hmm_range *range = hmm_vma_walk->range;unsigned long addr = start, next; +struct dev_pagemap *pgmap =NULL ;pmd_t *pmdp;pud_t pud;int ret;
@@ -744,17 +740,14 @@ static int hmm_vma_walk_pud(pud_t *pudp, pfn =pud_pfn (pud) + ((addr & ~PUD_MASK ) >>PAGE_SHIFT );for (i =0 ; i < npages; ++i, ++pfn) { - hmm_vma_walk->pgmap =get_dev_pagemap (pfn, - hmm_vma_walk->pgmap); -if (unlikely (!hmm_vma_walk->pgmap)) + pgmap =get_dev_pagemap (pfn, pgmap); +if (unlikely (!pgmap))return -EBUSY ; pfns[i] =hmm_device_entry_from_pfn (range, pfn) | cpu_flags; } -if (hmm_vma_walk->pgmap) { -put_dev_pagemap (hmm_vma_walk->pgmap); - hmm_vma_walk->pgmap =NULL ; - } +if (pgmap) +put_dev_pagemap (pgmap); hmm_vma_walk->last = end;return 0 ; }
@@ -1002,7 +995,6 @@ long hmm_range_fault(struct hmm_range *range, unsigned int flags)return -EPERM ; } - hmm_vma_walk.pgmap =NULL ; hmm_vma_walk.last = start; hmm_vma_walk.flags = flags; hmm_vma_walk.range = range;
Signature · 6 lines
2.20.1 _______________________________________________ Nouveau mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/nouveau
PATCH 5/13 mm: remove the unused vma argument to hmm_range_dma_unmap
In reply to Christoph Hellwig--- include/linux/hmm.h | 1 - mm/hmm.c | 2 -- 2 files changed, 3 deletions(-)
include/linux/hmm.h +0 -1
@@ -422,7 +422,6 @@ long hmm_range_dma_map(struct hmm_range *range,dma_addr_t *daddrs, unsigned int flags);long hmm_range_dma_unmap (struct hmm_range *range, -struct vm_area_struct *vma,struct device *device,dma_addr_t *daddrs,bool dirty);
mm/hmm.c +0 -2
@@ -1121,7 +1121,6 @@ EXPORT_SYMBOL(hmm_range_dma_map);/** * hmm_range_dma_unmap() - unmap range of that was map with hmm_range_dma_map() * @range: range being unmapped -* @vma: the vma against which the range (optional) * @device: device against which dma map was done * @daddrs: dma address of mapped pages * @dirty: dirty page if it had the write flag set
@@ -1133,7 +1132,6 @@ EXPORT_SYMBOL(hmm_range_dma_map);* concurrent mmu notifier or sync_cpu_device_pagetables() to make progress. */ long hmm_range_dma_unmap (struct hmm_range *range, -struct vm_area_struct *vma,struct device *device,dma_addr_t *daddrs,bool dirty)
Signature · 6 lines
2.20.1 _______________________________________________ Nouveau mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/nouveau
Re: [PATCH 05/13] mm: remove the unused vma argument to hmm_range_dma_unmap
Quoted 4 lines · Christoph Hellwig
Signed-off-by Christoph Hellwig <[email protected]>include/linux/hmm.h | 1 - mm/hmm.c | 2 -- 2 files changed, 3 deletions(-)
Unclear what this was intended for, but I think if we need to carry information from the dma_map to unmap it should be done in some opaque way.
The driver should not have to care about VMAs, beyond perhaps using VMAs to guide what VA ranges to mirror.
Jason
PATCH 6/13 mm: remove superflous arguments from hmm_range_register
In reply to Christoph HellwigThe start, end and page_shift values are all saved in the range structure, so we might as well use that for argument passing.
--- Documentation/vm/hmm.rst | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 7 +++++-- drivers/gpu/drm/nouveau/nouveau_svm.c | 5 ++--- include/linux/hmm.h | 6 +----- mm/hmm.c | 20 +++++--------------- 5 files changed, 14 insertions(+), 26 deletions(-)
Documentation/vm/hmm.rst +1 -1
@@ -222,7 +222,7 @@ The usage pattern is:: range.flags = ...; range.values = ...; range.pfn_shift = ...; - hmm_range_register(&range); + hmm_range_register(&range, mirror); /* * Just wait for range to be valid, safe to ignore return value as we
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +5 -2
@@ -818,8 +818,11 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages) 0 : range->flags[HMM_PFN_WRITE]; range->pfn_flags_mask =0 ; range->pfns = pfns; -hmm_range_register (range, mirror, start, - start + ttm->num_pages *PAGE_SIZE ,PAGE_SHIFT ); + range->page_shift =PAGE_SHIFT ; + range->start = start; + range->end = start + ttm->num_pages *PAGE_SIZE ; + +hmm_range_register (range, mirror);/* * Just wait for range to be valid, safe to ignore return value as we
drivers/gpu/drm/nouveau/nouveau_svm.c +2 -3
@@ -492,9 +492,7 @@ nouveau_range_fault(struct nouveau_svmm *svmm, struct hmm_range *range) range->default_flags =0 ; range->pfn_flags_mask =-1UL ; - ret =hmm_range_register (range, &svmm->mirror, - range->start, range->end, -PAGE_SHIFT ); + ret =hmm_range_register (range, &svmm->mirror);if (ret) {up_read (&range->hmm->mm->mmap_sem);return (int )ret;
@@ -682,6 +680,7 @@ nouveau_svm_fault(struct nvif_notify *notify) args.i.p.addr + args.i.p.size, fn - fi);/* Have HMM fault pages within the fault window to the GPU. */ + range.page_shift =PAGE_SHIFT ; range.start = args.i.p.addr; range.end = args.i.p.addr + args.i.p.size; range.pfns = args.phys;
include/linux/hmm.h +1 -5
@@ -400,11 +400,7 @@ void hmm_mirror_unregister(struct hmm_mirror *mirror);/* * Please see Documentation/vm/hmm.rst for how to use the range API. */ -int hmm_range_register (struct hmm_range *range, -struct hmm_mirror *mirror, -unsigned long start, -unsigned long end, -unsigned page_shift); +int hmm_range_register (struct hmm_range *range,struct hmm_mirror *mirror);void hmm_range_unregister (struct hmm_range *range);/*
mm/hmm.c +5 -15
@@ -843,35 +843,25 @@ static void hmm_pfns_clear(struct hmm_range *range,* hmm_range_register() - start tracking change to CPU page table over a range * @range: range * @mm: the mm struct for the range of virtual address -* @start: start virtual address (inclusive) -* @end: end virtual address (exclusive) -* @page_shift: expect page shift for the range +* * Return: 0 on success, -EFAULT if the address space is no longer valid * * Track updates to the CPU page table see include/linux/hmm.h */ -int hmm_range_register (struct hmm_range *range, -struct hmm_mirror *mirror, -unsigned long start, -unsigned long end, -unsigned page_shift) +int hmm_range_register (struct hmm_range *range,struct hmm_mirror *mirror) { -unsigned long mask = ((1UL << page_shift) -1UL ); +unsigned long mask = ((1UL << range->page_shift) -1UL );struct hmm *hmm = mirror->hmm;unsigned long flags; range->valid = false; range->hmm =NULL ; -if ((start & mask) || (end & mask)) +if ((range->start & mask) || (range->end & mask))return -EINVAL ; -if (start >= end) +if (range->start >= range->end)return -EINVAL ; - range->page_shift = page_shift; - range->start = start; - range->end = end; -/* Prevent hmm_release() from running while the range is valid */ if (!mmget_not_zero (hmm->mm))return -EFAULT ;
Signature · 1 line
2.20.1
Re: [PATCH 06/13] mm: remove superflous arguments from hmm_range_register
Quoted 11 lines · Christoph Hellwig
The start, end and
page_shiftvalues are all saved in the range structure, so we might as well use that for argument passing.Signed-off-by Christoph Hellwig <[email protected]>--- Documentation/vm/hmm.rst | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 7 +++++-- drivers/gpu/drm/nouveau/nouveau_svm.c | 5 ++--- include/linux/hmm.h | 6 +----- mm/hmm.c | 20 +++++--------------- 5 files changed, 14 insertions(+), 26 deletions(-)
I also don't really like how the API sets up some things in the struct and some things via arguments, so:
Jason
Re: [PATCH 06/13] mm: remove superflous arguments from hmm_range_register
In reply to Christoph HellwigQuoted 4 lines · Christoph Hellwig
The start, end and
page_shiftvalues are all saved in the range structure, so we might as well use that for argument passing.Signed-off-by Christoph Hellwig <[email protected]>
Quoted 114 lines
--- Documentation/vm/hmm.rst | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 7 +++++-- drivers/gpu/drm/nouveau/nouveau_svm.c | 5 ++--- include/linux/hmm.h | 6 +----- mm/hmm.c | 20 +++++--------------- 5 files changed, 14 insertions(+), 26 deletions(-)Documentation/vm/hmm.rst +1
-1@@ -222,7 +222,7 @@ The usage pattern is:: range.flags = ...; range.values = ...; range.pfn_shift = ...; - hmm_range_register(&range); + hmm_range_register(&range, mirror); /* * Just wait for range to be valid, safe to ignore return value as wedrivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +5
-2@@ -818,8 +818,11 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages) 0 : range->flags[HMM_PFN_WRITE]; range->pfn_flags_mask =0 ; range->pfns = pfns; -hmm_range_register (range, mirror, start, - start + ttm->num_pages *PAGE_SIZE ,PAGE_SHIFT ); + range->page_shift =PAGE_SHIFT ; + range->start = start; + range->end = start + ttm->num_pages *PAGE_SIZE ; + +hmm_range_register (range, mirror);/* * Just wait for range to be valid, safe to ignore return value as we drivers/gpu/drm/nouveau/nouveau_svm.c +2
-3@@ -492,9 +492,7 @@ nouveau_range_fault(struct nouveau_svmm *svmm, struct hmm_range *range) range->default_flags =0 ; range->pfn_flags_mask =-1UL ; - ret =hmm_range_register (range, &svmm->mirror, - range->start, range->end, -PAGE_SHIFT ); + ret =hmm_range_register (range, &svmm->mirror);if (ret) {up_read (&range->hmm->mm->mmap_sem);return (int )ret;@@ -682,6 +680,7 @@ nouveau_svm_fault(struct nvif_notify *notify) args.i.p.addr + args.i.p.size, fn - fi);/* Have HMM fault pages within the fault window to the GPU. */ + range.page_shift =PAGE_SHIFT ; range.start = args.i.p.addr; range.end = args.i.p.addr + args.i.p.size; range.pfns = args.phys;include/linux/hmm.h +1
-5@@ -400,11 +400,7 @@ void hmm_mirror_unregister(struct hmm_mirror *mirror);/* * Please see Documentation/vm/hmm.rst for how to use the range API. */ -int hmm_range_register (struct hmm_range *range, -struct hmm_mirror *mirror, -unsigned long start, -unsigned long end, -unsigned page_shift); +int hmm_range_register (struct hmm_range *range,struct hmm_mirror *mirror);void hmm_range_unregister (struct hmm_range *range);/* mm/hmm.c +4
-11@@ -843,35 +843,25 @@ static void hmm_pfns_clear(struct hmm_range *range,* hmm_range_register() - start tracking change to CPU page table over a range * @range: range * @mm: the mm struct for the range of virtual address -* @start: start virtual address (inclusive) -* @end: end virtual address (exclusive) -* @page_shift: expect page shift for the range +* * Return: 0 on success, -EFAULT if the address space is no longer valid * * Track updates to the CPU page table see include/linux/hmm.h */ -int hmm_range_register (struct hmm_range *range, -struct hmm_mirror *mirror, -unsigned long start, -unsigned long end, -unsigned page_shift) +int hmm_range_register (struct hmm_range *range,struct hmm_mirror *mirror) { -unsigned long mask = ((1UL << page_shift) -1UL ); +unsigned long mask = ((1UL << range->page_shift) -1UL );struct hmm *hmm = mirror->hmm;unsigned long flags; range->valid = false; range->hmm =NULL ; -if ((start & mask) || (end & mask)) +if ((range->start & mask) || (range->end & mask))return -EINVAL ; -if (start >= end)
mm/hmm.c static void hmm_pfns_clear(struct hmm_range *range, · line 867
Quoted 10 lines
mm/hmm.c +1
-4+if (range->start >= range->end)return -EINVAL ; - range->page_shift = page_shift; - range->start = start; - range->end = end; -/* Prevent hmm_release() from running while the range is valid */ if (!mmget_not_zero (hmm->mm))return -EFAULT ;
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
PATCH 7/13 mm: remove the page_shift member from struct hmm_range
In reply to Christoph HellwigAll users pass PAGE_SIZE here, and if we wanted to support single entries for huge pages we should really just add a HMM_FAULT_HUGEPAGE flag instead that uses the huge page size instead of having the caller calculate that size once, just for the hmm code to verify it.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 1 - drivers/gpu/drm/nouveau/nouveau_svm.c | 1 - include/linux/hmm.h | 22 ------------- mm/hmm.c | 42 ++++++------------------- 4 files changed, 9 insertions(+), 57 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +0 -1
@@ -818,7 +818,6 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages) 0 : range->flags[HMM_PFN_WRITE]; range->pfn_flags_mask =0 ; range->pfns = pfns; - range->page_shift =PAGE_SHIFT ; range->start = start; range->end = start + ttm->num_pages *PAGE_SIZE ;
drivers/gpu/drm/nouveau/nouveau_svm.c +0 -1
@@ -680,7 +680,6 @@ nouveau_svm_fault(struct nvif_notify *notify) args.i.p.addr + args.i.p.size, fn - fi);/* Have HMM fault pages within the fault window to the GPU. */ - range.page_shift =PAGE_SHIFT ; range.start = args.i.p.addr; range.end = args.i.p.addr + args.i.p.size; range.pfns = args.phys;
include/linux/hmm.h +0 -22
@@ -158,7 +158,6 @@ enum hmm_pfn_value_e {* @values: pfn value for some special case (none, special, error, ...) * @default_flags: default flags for the range (write, read, ... see hmm doc) * @pfn_flags_mask: allows to mask pfn flags so that only default_flags matter -* @page_shift: device virtual address shift value (should be >= PAGE_SHIFT) * @pfn_shifts: pfn shift value (should be <= PAGE_SHIFT) * @valid: pfns array did not change since it has been fill by an HMM function */
@@ -172,31 +171,10 @@ struct hmm_range {const uint64_t *values;uint64_t default_flags;uint64_t pfn_flags_mask; -uint8_t page_shift;uint8_t pfn_shift;bool valid; }; -/* -* hmm_range_page_shift() - return the page shift for the range -* @range: range being queried -* Return: page shift (page size = 1 << page shift) for the range -*/ -static inline unsigned hmm_range_page_shift (const struct hmm_range *range) -{ -return range->page_shift; -} - -/* -* hmm_range_page_size() - return the page size for the range -* @range: range being queried -* Return: page size for the range in bytes -*/ -static inline unsigned long hmm_range_page_size (const struct hmm_range *range) -{ -return 1UL <<hmm_range_page_shift (range); -} -/* * hmm_range_wait_until_valid() - wait for range to be valid * @range: range affected by invalidation to wait on
mm/hmm.c +9 -33
91 lines not on this page · raw message
Signature · 6 lines
2.20.1 _______________________________________________ Nouveau mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/nouveau
Re: [PATCH 07/13] mm: remove the page_shift member from struct hmm_range
Quoted 4 lines · Christoph Hellwig
All users pass
PAGE_SIZEhere, and if we wanted to support single entries for huge pages we should really just add aHMM_FAULT_HUGEPAGEflag instead that uses the huge page size instead of having the caller calculate that size once, just for the hmm code to verify it.
I suspect this was added for the ODP conversion that does use both page sizes. I think the ODP code for this is kind of broken, but I haven't delved into that..
The challenge is that the driver needs to know what page size to configure the hardware before it does any range stuff.
The other challenge is that the HW is configured to do only one page size, and if the underlying CPU page side changes it goes south.
What I would prefer is if the driver could somehow dynamically adjust the the page size after each dma map, but I don't know if ODP HW can do that.
Since this is all driving toward making ODP use this maybe we should keep this API?
I'm not sure I can loose the crappy huge page support in ODP.
Jason
Re: [PATCH 07/13] mm: remove the page_shift member from struct hmm_range
Quoted 10 lines · Jason Gunthorpe
I suspect this was added for the ODP conversion that does use both page sizes. I think the ODP code for this is kind of broken, but I haven't delved into that..
The challenge is that the driver needs to know what page size to configure the hardware before it does any range stuff.
The other challenge is that the HW is configured to do only one page size, and if the underlying CPU page side changes it goes south.
Quoted 8 lines
What I would prefer is if the driver could somehow dynamically adjust the the page size after each dma map, but I don't know if ODP HW can do that.
Since this is all driving toward making ODP use this maybe we should keep this API?
I'm not sure I can loose the crappy huge page support in ODP.
The problem is that I see no way how to use the current API. To know the huge page size you need to have the vma, and the current API doesn't require a vma to be passed in.
That's why I suggested an api where we pass in a flag that huge pages are ok into hmm_range_fault, and it then could pass the shift out, and limits itself to a single vma (which it normally doesn't, that is an additional complication). But all this seems really awkward in terms of an API still. AFAIK ODP is only used by mlx5, and mlx5 unlike other IB HCAs can use scatterlist style MRs with variable length per entry, so even if we pass multiple pages per entry from hmm it could coalesce them. The best API for mlx4 would of course be to pass a biovec-style variable length structure that hmm_fault could fill out, but that would be a major restructure.
Re: [PATCH 07/13] mm: remove the page_shift member from struct hmm_range
Quoted 15 lines · Christoph Hellwig
Quoted 14 lines · Jason Gunthorpe
I suspect this was added for the ODP conversion that does use both page sizes. I think the ODP code for this is kind of broken, but I haven't delved into that..
The challenge is that the driver needs to know what page size to configure the hardware before it does any range stuff.
The other challenge is that the HW is configured to do only one page size, and if the underlying CPU page side changes it goes south.
What I would prefer is if the driver could somehow dynamically adjust the the page size after each dma map, but I don't know if ODP HW can do that.
Quoted 8 lines
Quoted 4 lines
Since this is all driving toward making ODP use this maybe we should keep this API?
I'm not sure I can loose the crappy huge page support in ODP.
The problem is that I see no way how to use the current API. To know the huge page size you need to have the vma, and the current API doesn't require a vma to be passed in.
The way ODP seems to work is once in hugetlb mode the dma addresses must give huge pages or the page fault will be failed. I think that is a terrible design, but this is how the driver is ..
So, from this HMM perspective if the caller asked for huge pages then the results have to be all huge pages or a hard failure.
It is not negotiated as an optimization like you are thinking.
[note, I haven't yet checked carefully how this works in ODP, every
time I look at parts of it the thing seems crazy]
Quoted 8 lines
That's why I suggested an api where we pass in a flag that huge pages are ok into
hmm_range_fault, and it then could pass the shift out, and limits itself to a single vma (which it normally doesn't, that is an additional complication). But all this seems really awkward in terms of an API still. AFAIK ODP is only used by mlx5, and mlx5 unlike other IB HCAs can use scatterlist style MRs with variable length per entry, so even if we pass multiple pages per entry from hmm it could coalesce them.
When the driver takes faults it has to repair the MR mapping, and fixing a page in the middle of a variable length SGL would be pretty complicated. Even so, I don't think the SG_GAPs feature and ODP are compatible - I'm pretty sure ODP has to be page lists not SGL..
However, what ODP can maybe do is represent a full multi-level page table, so we could have 2M entries that map to a single DMA or to another page table w/ 4k pages (have to check on this)
But the driver isn't set up to do that right now.
Quoted 3 lines
The best API for mlx4 would of course be to pass a biovec-style variable length structure that
hmm_faultcould fill out, but that would be a major restructure.
It would work, but the driver has to expand that into a page list right awayhow.
We can't even dma map the biovec with today's dma API as it needs the ability to remap on a page granularity.
Jason
Re: [PATCH 07/13] mm: remove the page_shift member from struct hmm_range
Quoted 6 lines · Jason Gunthorpe
The way ODP seems to work is once in hugetlb mode the dma addresses must give huge pages or the page fault will be failed. I think that is a terrible design, but this is how the driver is ..
So, from this HMM perspective if the caller asked for huge pages then the results have to be all huge pages or a hard failure.
Which isn't how the page_shift member works at moment. It still allows non-hugetlb mappings even with the member.
Quoted 4 lines
It is not negotiated as an optimization like you are thinking.
[note, I haven't yet checked carefully how this works in ODP, every
time I look at parts of it the thing seems crazy]
This seems pretty crazy. Especially as hugetlb use in applications seems to fade in favour of THP, for which this ODP scheme does not seem to work at all.
Quoted 9 lines
Quoted 3 lines
The best API for mlx4 would of course be to pass a biovec-style variable length structure that
hmm_faultcould fill out, but that would be a major restructure.It would work, but the driver has to expand that into a page list right awayhow.
We can't even dma map the biovec with today's dma API as it needs the ability to remap on a page granularity.
We can do dma_map_page loops over each biovec entry pretty trivially, and that won't be any worse than the current loop over each page in the hmm dma helpers. Once I get around the work to have a better API for iommu mappings for bio_vecs we could coalesce it similar to how we do it with scatterlist (but without all the mess of a new structure). That work is going to take a little longer through, as it needs the amd and intell iommu drivers to be convered to dma-iommu which isn't making progress as far as I hoped.
Let me know if you want to keep this code for now despite the issues, or if we'd rather reimplement it once you've made sense of the ODP code.
Re: [PATCH 07/13] mm: remove the page_shift member from struct hmm_range
In reply to Christoph HellwigQuoted 4 lines · Christoph Hellwig
All users pass
PAGE_SIZEhere, and if we wanted to support single entries for huge pages we should really just add aHMM_FAULT_HUGEPAGEflag instead that uses the huge page size instead of having the caller calculate that size once, just for the hmm code to verify it.
Maybe this was meant to support device page size != native page size? Anyway, looks like we didn't use it that way.
Quoted 178 lines
Signed-off-by Christoph Hellwig <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 1 - drivers/gpu/drm/nouveau/nouveau_svm.c | 1 - include/linux/hmm.h | 22 ------------- mm/hmm.c | 42 ++++++------------------- 4 files changed, 9 insertions(+), 57 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +0
-1@@ -818,7 +818,6 @@ int amdgpu_ttm_tt_get_user_pages(struct amdgpu_bo *bo, struct page **pages) 0 : range->flags[HMM_PFN_WRITE]; range->pfn_flags_mask =0 ; range->pfns = pfns; - range->page_shift =PAGE_SHIFT ; range->start = start; range->end = start + ttm->num_pages *PAGE_SIZE ;drivers/gpu/drm/nouveau/nouveau_svm.c +0
-1@@ -680,7 +680,6 @@ nouveau_svm_fault(struct nvif_notify *notify) args.i.p.addr + args.i.p.size, fn - fi);/* Have HMM fault pages within the fault window to the GPU. */ - range.page_shift =PAGE_SHIFT ; range.start = args.i.p.addr; range.end = args.i.p.addr + args.i.p.size; range.pfns = args.phys;include/linux/hmm.h +0
-22@@ -158,7 +158,6 @@ enum hmm_pfn_value_e {* @values: pfn value for some special case (none, special, error, ...) * @default_flags: default flags for the range (write, read, ... see hmm doc) * @pfn_flags_mask: allows to mask pfn flags so that only default_flags matter -* @page_shift: device virtual address shift value (should be >= PAGE_SHIFT) * @pfn_shifts: pfn shift value (should be <= PAGE_SHIFT) * @valid: pfns array did not change since it has been fill by an HMM function */ @@ -172,31 +171,10 @@ struct hmm_range {const uint64_t *values;uint64_t default_flags;uint64_t pfn_flags_mask; -uint8_t page_shift;uint8_t pfn_shift;bool valid; }; -/* -* hmm_range_page_shift() - return the page shift for the range -* @range: range being queried -* Return: page shift (page size = 1 << page shift) for the range -*/ -static inline unsigned hmm_range_page_shift (const struct hmm_range *range) -{ -return range->page_shift; -} - -/* -* hmm_range_page_size() - return the page size for the range -* @range: range being queried -* Return: page size for the range in bytes -*/ -static inline unsigned long hmm_range_page_size (const struct hmm_range *range) -{ -return 1UL <<hmm_range_page_shift (range); -} -/* * hmm_range_wait_until_valid() - wait for range to be valid * @range: range affected by invalidation to wait on mm/hmm.c +9
-33@@ -344,13 +344,12 @@ static int hmm_vma_walk_hole_(unsigned long addr, unsigned long end,struct hmm_vma_walk *hmm_vma_walk = walk->private;struct hmm_range *range = hmm_vma_walk->range;uint64_t *pfns = range->pfns; -unsigned long i, page_size; +unsigned long i; hmm_vma_walk->last = addr; - page_size =hmm_range_page_size (range); - i = (addr - range->start) >> range->page_shift; + i = (addr - range->start) >>PAGE_SHIFT ; -for (; addr < end; addr += page_size, i++) { +for (; addr < end; addr +=PAGE_SIZE , i++) { pfns[i] = range->values[HMM_PFN_NONE ];if (fault || write_fault) {int ret;@@ -772,7 +771,7 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, struct mm_walk *walk) {#ifdef CONFIG_HUGETLB_PAGE -unsigned long addr = start, i, pfn, mask, size, pfn_inc; +unsigned long addr = start, i, pfn, mask;struct hmm_vma_walk *hmm_vma_walk = walk->private;struct hmm_range *range = hmm_vma_walk->range;struct vm_area_struct *vma = walk->vma;@@ -783,24 +782,12 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask,pte_t entry;int ret =0 ; - size =huge_page_size (h); - mask = size -1 ; -if (range->page_shift !=PAGE_SHIFT ) { -/* Make sure we are looking at a full page. */ -if (start & mask) -return -EINVAL ; -if (end < (start + size)) -return -EINVAL ; - pfn_inc = size >>PAGE_SHIFT ; - }else { - pfn_inc =1 ; - size =PAGE_SIZE ; - } + mask =huge_page_size (h) -1 ; ptl =huge_pte_lock (hstate_vma (vma), walk->mm, pte); entry =huge_ptep_get (pte); - i = (start - range->start) >> range->page_shift; + i = (start - range->start) >>PAGE_SHIFT ; orig_pfn = range->pfns[i]; range->pfns[i] = range->values[HMM_PFN_NONE ]; cpu_flags =pte_to_hmm_pfn_flags (range, entry);@@ -812,8 +799,8 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, goto unlock; } - pfn =pte_pfn (entry) + ((start & mask) >> range->page_shift); -for (; addr < end; addr += size, i++, pfn += pfn_inc) + pfn =pte_pfn (entry) + ((start & mask) >>PAGE_SHIFT ); +for (; addr < end; addr +=PAGE_SIZE , i++, pfn++) range->pfns[i] =hmm_device_entry_from_pfn (range, pfn) | cpu_flags; hmm_vma_walk->last = end;@@ -850,14 +837,13 @@ static void hmm_pfns_clear(struct hmm_range *range,*/ int hmm_range_register (struct hmm_range *range,struct hmm_mirror *mirror) { -unsigned long mask = ((1UL << range->page_shift) -1UL );struct hmm *hmm = mirror->hmm;unsigned long flags; range->valid = false; range->hmm =NULL ; -if ((range->start & mask) || (range->end & mask)) +if ((range->start & (PAGE_SIZE -1 )) || (range->end & (PAGE_SIZE -1 )))return -EINVAL ;if (range->start >= range->end)return -EINVAL ;@@ -964,16 +950,6 @@ long hmm_range_fault(struct hmm_range *range, unsigned int flags)if (vma ==NULL || (vma->vm_flags & device_vma))return -EFAULT ; -if (is_vm_hugetlb_page (vma)) { -if (huge_page_shift (hstate_vma (vma)) != - range->page_shift && - range->page_shift !=PAGE_SHIFT ) -return -EINVAL ; - }else { -if (range->page_shift !=PAGE_SHIFT ) -return -EINVAL ; - } -if (!(vma->vm_flags &VM_READ )) {/* * If vma do not allow read access, then assume that it
PATCH 8/13 mm: remove the mask variable in hmm_vma_walk_hugetlb_entry
In reply to Christoph HellwigThe pagewalk code already passes the value as the hmask parameter.
--- mm/hmm.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
mm/hmm.c +2 -5
@@ -771,19 +771,16 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, struct mm_walk *walk) {#ifdef CONFIG_HUGETLB_PAGE -unsigned long addr = start, i, pfn, mask; +unsigned long addr = start, i, pfn;struct hmm_vma_walk *hmm_vma_walk = walk->private;struct hmm_range *range = hmm_vma_walk->range;struct vm_area_struct *vma = walk->vma; -struct hstate *h =hstate_vma (vma);uint64_t orig_pfn, cpu_flags;bool fault, write_fault;spinlock_t *ptl;pte_t entry;int ret =0 ; - mask =huge_page_size (h) -1 ; - ptl =huge_pte_lock (hstate_vma (vma), walk->mm, pte); entry =huge_ptep_get (pte);
@@ -799,7 +796,7 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, goto unlock; } - pfn =pte_pfn (entry) + ((start & mask) >>PAGE_SHIFT ); + pfn =pte_pfn (entry) + ((start & hmask) >>PAGE_SHIFT );for (; addr < end; addr +=PAGE_SIZE , i++, pfn++) range->pfns[i] =hmm_device_entry_from_pfn (range, pfn) | cpu_flags;
Signature · 6 lines
2.20.1 _______________________________________________ Nouveau mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/nouveau
Re: [PATCH 08/13] mm: remove the mask variable in hmm_vma_walk_hugetlb_entry
Quoted 31 lines · Christoph Hellwig
The pagewalk code already passes the value as the hmask parameter.
Signed-off-by Christoph Hellwig <[email protected]>mm/hmm.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)mm/hmm.c +2
-4+++ b/mm/hmm.c@@ -771,19 +771,16 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, struct mm_walk *walk) {#ifdef CONFIG_HUGETLB_PAGE -unsigned long addr = start, i, pfn, mask; +unsigned long addr = start, i, pfn;struct hmm_vma_walk *hmm_vma_walk = walk->private;struct hmm_range *range = hmm_vma_walk->range;struct vm_area_struct *vma = walk->vma; -struct hstate *h =hstate_vma (vma);uint64_t orig_pfn, cpu_flags;bool fault, write_fault;spinlock_t *ptl;pte_t entry;int ret =0 ; - mask =huge_page_size (h) -1 ; - ptl =huge_pte_lock (hstate_vma (vma), walk->mm, pte); entry =huge_ptep_get (pte);
mm/hmm.c static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, · line 799
Quoted 5 lines
mm/hmm.c +1
-1@@ -799,7 +796,7 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, goto unlock; } - pfn =pte_pfn (entry) + ((start & mask) >>PAGE_SHIFT ); + pfn =pte_pfn (entry) + ((start & hmask) >>PAGE_SHIFT );
I don't know this hstate stuff, but this doesn't look the same?
static int walk_hugetlb_range(unsigned long addr, unsigned long end, {
struct hstate *h = hstate_vma(vma);
unsigned long hmask = huge_page_mask(h); // aka h->mask
err = walk->hugetlb_entry(pte, hmask, addr, next, walk);And the first place I found setting h->mask is:
void __init hugetlb_add_hstate(unsigned int order) {
h->mask = ~((1ULL << (order + PAGE_SHIFT)) - 1);Compared with
mask = huge_page_size(h) - 1;
= ((unsigned long)PAGE_SIZE << h->order) - 1Looks like hmask == ~mask
?
Jason
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
Re: [PATCH 08/13] mm: remove the mask variable in hmm_vma_walk_hugetlb_entry
In reply to Christoph HellwigQuoted 33 lines · Christoph Hellwig
The pagewalk code already passes the value as the hmask parameter.
Signed-off-by Christoph Hellwig <[email protected]>--- mm/hmm.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)mm/hmm.c +1
-4@@ -771,19 +771,16 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, struct mm_walk *walk) {#ifdef CONFIG_HUGETLB_PAGE -unsigned long addr = start, i, pfn, mask; +unsigned long addr = start, i, pfn;struct hmm_vma_walk *hmm_vma_walk = walk->private;struct hmm_range *range = hmm_vma_walk->range;struct vm_area_struct *vma = walk->vma; -struct hstate *h =hstate_vma (vma);uint64_t orig_pfn, cpu_flags;bool fault, write_fault;spinlock_t *ptl;pte_t entry;int ret =0 ; - mask =huge_page_size (h) -1 ; - ptl =huge_pte_lock (hstate_vma (vma), walk->mm, pte); entry =huge_ptep_get (pte);
mm/hmm.c static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, · line 799
Quoted 5 lines
mm/hmm.c +1
-1@@ -799,7 +796,7 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, goto unlock; } - pfn =pte_pfn (entry) + ((start & mask) >>PAGE_SHIFT ); + pfn =pte_pfn (entry) + ((start & hmask) >>PAGE_SHIFT );
This needs to be "~hmask" so that the upper bits of the start address are not added to the pfn. It's the middle bits of the address that offset into the huge page that are needed.
mm/hmm.c static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, · line 802
Quoted 4 lines
mm/hmm.c +0
-0for (; addr < end; addr +=PAGE_SIZE , i++, pfn++) range->pfns[i] =hmm_device_entry_from_pfn (range, pfn) | cpu_flags;
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
PATCH 9/13 mm: don't abuse pte_index() in hmm_vma_handle_pmd
In reply to Christoph Hellwigpte_index is an internal arch helper in various architectures, without consistent semantics. Open code that calculation of a PMD index based on the virtual address instead.
--- mm/hmm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/hmm.c +1 -1
@@ -486,7 +486,7 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk,if (pmd_protnone (pmd) || fault || write_fault)return hmm_vma_walk_hole_ (addr, end, fault, write_fault, walk); - pfn =pmd_pfn (pmd) +pte_index (addr); + pfn =pmd_pfn (pmd) + ((addr & ~PMD_MASK ) >>PAGE_SHIFT );for (i =0 ; addr < end; addr +=PAGE_SIZE , i++, pfn++) {if (pmd_devmap (pmd)) { pgmap =get_dev_pagemap (pfn, pgmap);
Signature · 6 lines
2.20.1 _______________________________________________ Nouveau mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/nouveau
PATCH 10/13 mm: only define hmm_vma_walk_pud if needed
In reply to Christoph HellwigWe only need the special pud_entry walker if PUD-sized hugepages and pte mappings are supported, else the common pagewalk code will take care of the iteration. Not implementing this callback reduced the amount of code compiled for non-x86 platforms, and also fixes compile failures with other architectures when helpers like pud_pfn are not implemented.
--- mm/hmm.c | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-)
mm/hmm.c +16 -13
@@ -455,15 +455,6 @@ static inline uint64_t pmd_to_hmm_pfn_flags(struct hmm_range *range, pmd_t pmd) range->flags[HMM_PFN_VALID ]; } -static inline uint64_t pud_to_hmm_pfn_flags (struct hmm_range *range,pud_t pud) -{ -if (!pud_present (pud)) -return 0 ; -return pud_write (pud) ? range->flags[HMM_PFN_VALID ] | - range->flags[HMM_PFN_WRITE ] : - range->flags[HMM_PFN_VALID ]; -} - static int hmm_vma_handle_pmd(struct mm_walk *walk, unsigned long addr, unsigned long end,
@@ -700,10 +691,19 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp,return 0 ; } -static int hmm_vma_walk_pud (pud_t *pudp, -unsigned long start, -unsigned long end, -struct mm_walk *walk) +#if defined(CONFIG_ARCH_HAS_PTE_DEVMAP ) && \ + defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD ) +static inline uint64_t pud_to_hmm_pfn_flags (struct hmm_range *range,pud_t pud) +{ +if (!pud_present (pud)) +return 0 ; +return pud_write (pud) ? range->flags[HMM_PFN_VALID ] | + range->flags[HMM_PFN_WRITE ] : + range->flags[HMM_PFN_VALID ]; +} + +static int hmm_vma_walk_pud (pud_t *pudp,unsigned long start,unsigned long end, +struct mm_walk *walk) {struct hmm_vma_walk *hmm_vma_walk = walk->private;struct hmm_range *range = hmm_vma_walk->range;
@@ -765,6 +765,9 @@ static int hmm_vma_walk_pud(pud_t *pudp,return 0 ; } +#else +#define hmm_vma_walk_pud NULL +#endif static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask, unsigned long start, unsigned long end,
Signature · 6 lines
2.20.1 _______________________________________________ Nouveau mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/nouveau
Re: [PATCH 10/13] mm: only define hmm_vma_walk_pud if needed
Quoted 11 lines · Christoph Hellwig
We only need the special
pud_entrywalker if PUD-sized hugepages and pte mappings are supported, else the common pagewalk code will take care of the iteration. Not implementing this callback reduced the amount of code compiled for non-x86 platforms, and also fixes compile failures with other architectures when helpers likepud_pfnare not implemented.Signed-off-by Christoph Hellwig <[email protected]>--- mm/hmm.c | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-)
Jason
PATCH 11/13 mm: cleanup the hmm_vma_handle_pmd stub
In reply to Christoph HellwigStub out the whole function when CONFIG_TRANSPARENT_HUGEPAGE is not set to make the function easier to read.
--- mm/hmm.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
mm/hmm.c +9 -9
@@ -455,13 +455,10 @@ static inline uint64_t pmd_to_hmm_pfn_flags(struct hmm_range *range, pmd_t pmd) range->flags[HMM_PFN_VALID ]; } -static int hmm_vma_handle_pmd (struct mm_walk *walk, -unsigned long addr, -unsigned long end, -uint64_t *pfns, -pmd_t pmd) -{#ifdef CONFIG_TRANSPARENT_HUGEPAGE +static int hmm_vma_handle_pmd (struct mm_walk *walk,unsigned long addr, +unsigned long end,uint64_t *pfns,pmd_t pmd) +{struct hmm_vma_walk *hmm_vma_walk = walk->private;struct hmm_range *range = hmm_vma_walk->range;struct dev_pagemap *pgmap =NULL ;
@@ -490,11 +487,14 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk,put_dev_pagemap (pgmap); hmm_vma_walk->last = end; return 0; -#else -/* If THP is not enabled then we should never reach this code ! */ +} +#else /* CONFIG_TRANSPARENT_HUGEPAGE */ +static int hmm_vma_handle_pmd (struct mm_walk *walk,unsigned long addr, +unsigned long end,uint64_t *pfns,pmd_t pmd) +{return -EINVAL ; -#endif } +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ static inline uint64_t pte_to_hmm_pfn_flags (struct hmm_range *range,pte_t pte) {
Signature · 6 lines
2.20.1 _______________________________________________ Nouveau mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/nouveau
Re: [PATCH 11/13] mm: cleanup the hmm_vma_handle_pmd stub
Quoted 28 lines · Christoph Hellwig
Stub out the whole function when
CONFIG_TRANSPARENT_HUGEPAGEis not set to make the function easier to read.Signed-off-by Christoph Hellwig <[email protected]>mm/hmm.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)mm/hmm.c +4
-6+++ b/mm/hmm.c@@ -455,13 +455,10 @@ static inline uint64_t pmd_to_hmm_pfn_flags(struct hmm_range *range, pmd_t pmd) range->flags[HMM_PFN_VALID ]; } -static int hmm_vma_handle_pmd (struct mm_walk *walk, -unsigned long addr, -unsigned long end, -uint64_t *pfns, -pmd_t pmd) -{#ifdef CONFIG_TRANSPARENT_HUGEPAGE +static int hmm_vma_handle_pmd (struct mm_walk *walk,unsigned long addr, +unsigned long end,uint64_t *pfns,pmd_t pmd) +{struct hmm_vma_walk *hmm_vma_walk = walk->private;struct hmm_range *range = hmm_vma_walk->range;struct dev_pagemap *pgmap =NULL ;
mm/hmm.c static int hmm_vma_handle_pmd(struct mm_walk *walk, · line 494
Quoted 5 lines
mm/hmm.c +0
-2@@ -490,11 +487,14 @@ static int hmm_vma_handle_pmd(struct mm_walk *walk,put_dev_pagemap (pgmap); hmm_vma_walk->last = end;return 0 ; -#else -/* If THP is not enabled then we should never reach this
This old comment says we should never get here
mm/hmm.c static int hmm_vma_handle_pmd(struct mm_walk *walk, · line 495
Quoted 6 lines
mm/hmm.c +5
-0+} +#else /* CONFIG_TRANSPARENT_HUGEPAGE */ +static int hmm_vma_handle_pmd (struct mm_walk *walk,unsigned long addr, +unsigned long end,uint64_t *pfns,pmd_t pmd) +{return -EINVAL ;
So could we just do
#define hmm_vma_handle_pmd NULL
?
At the very least this seems like a WARN_ON too?
Jason
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
Re: [PATCH 11/13] mm: cleanup the hmm_vma_handle_pmd stub
Quoted 14 lines · Jason Gunthorpe
Review
Quoted 1 line
(quoted diff) +0
-1- /* If THP is not enabled then we should never reach thisThis old comment says we should never get here
mm/hmm.c static int hmm_vma_handle_pmd(struct mm_walk *walk, · line 495
Quoted 6 lines
mm/hmm.c +5
-0+} +#else /* CONFIG_TRANSPARENT_HUGEPAGE */ +static int hmm_vma_handle_pmd (struct mm_walk *walk,unsigned long addr, +unsigned long end,uint64_t *pfns,pmd_t pmd) +{return -EINVAL ;So could we just do
#definehmm_vma_handle_pmdNULL
Quoted 3 lines
?
At the very least this seems like a
WARN_ONtoo?
Despite the name of the function hmm_vma_handle_pmd is not a callback for the pagewalk, but actually called from hmm_vma_handle_pmd.
What we could try is just and empty non-inline prototype without an actual implementation, which means if the compiler doesn't optimize the calls away we'll get a link error.
PATCH 12/13 mm: cleanup the hmm_vma_walk_hugetlb_entry stub
In reply to Christoph HellwigStub out the whole function and assign NULL to the .hugetlb_entry method if CONFIG_HUGETLB_PAGE is not set, as the method won't ever be called in that case.
--- mm/hmm.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
mm/hmm.c +4 -4
@@ -769,11 +769,11 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end,#define hmm_vma_walk_pud NULL#endif +#ifdef CONFIG_HUGETLB_PAGE static int hmm_vma_walk_hugetlb_entry (pte_t *pte,unsigned long hmask,unsigned long start,unsigned long end,struct mm_walk *walk) { -#ifdef CONFIG_HUGETLB_PAGE unsigned long addr = start, i, pfn;struct hmm_vma_walk *hmm_vma_walk = walk->private;struct hmm_range *range = hmm_vma_walk->range;
@@ -812,10 +812,10 @@ static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask,return hmm_vma_walk_hole_ (addr, end, fault, write_fault, walk);return ret; -#else /* CONFIG_HUGETLB_PAGE */ -return -EINVAL ; -#endif } +#else +#define hmm_vma_walk_hugetlb_entry NULL +#endif /* CONFIG_HUGETLB_PAGE */ static void hmm_pfns_clear(struct hmm_range *range, uint64_t *pfns,
Signature · 6 lines
2.20.1 _______________________________________________ Nouveau mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/nouveau
Re: [PATCH 12/13] mm: cleanup the hmm_vma_walk_hugetlb_entry stub
Quoted 8 lines · Christoph Hellwig
Stub out the whole function and assign NULL to the .hugetlb_entry method if
CONFIG_HUGETLB_PAGEis not set, as the method won't ever be called in that case.Signed-off-by Christoph Hellwig <[email protected]>--- mm/hmm.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
Jason
PATCH 13/13 mm: allow HMM_MIRROR on all architectures with MMU
In reply to Christoph HellwigThere isn't really any architecture specific code in this page table walk implementation, so drop the dependencies.
--- mm/Kconfig | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
mm/Kconfig +1 -2
@@ -677,8 +677,7 @@ config DEV_PAGEMAP_OPS config HMM_MIRROR bool "HMM mirror CPU page table into a device page table" - depends on (X86_64 || PPC64) - depends on MMU && 64BIT + depends on MMU select MMU_NOTIFIER help Select HMM_MIRROR if you want to mirror range of the CPU page table of a
Signature · 6 lines
2.20.1 _______________________________________________ Nouveau mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/nouveau
Re: [PATCH 13/13] mm: allow HMM_MIRROR on all architectures with MMU
Quoted 6 lines · Christoph Hellwig
There isn't really any architecture specific code in this page table walk implementation, so drop the dependencies.
Signed-off-by Christoph Hellwig <[email protected]>mm/Kconfig | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
Happy to see it, lets try to get this patch + the required parts of this series into linux-next to be really sure
Thanks,
Jason
Re: [PATCH 13/13] mm: allow HMM_MIRROR on all architectures with MMU
Quoted 4 lines · Jason Gunthorpe
Quoted 3 lines · Christoph Hellwig
There isn't really any architecture specific code in this page table walk implementation, so drop the dependencies.
Quoted 8 lines
Quoted 3 lines
Signed-off-by Christoph Hellwig <[email protected]>mm/Kconfig | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)Happy to see it, lets try to get this patch + the required parts of this series into linux-next to be really sure
Reviewed-by Jason Gunthorpe <[email protected]>
Oh, can we make this into a non-user selectable option now?
ie have the drivers that use the API select it?
Thanks,
Jason
Re: [PATCH 13/13] mm: allow HMM_MIRROR on all architectures with MMU
Quoted 3 lines · Jason Gunthorpe
Oh, can we make this into a non-user selectable option now?
ie have the drivers that use the API select it?
Sure, I'll throw in another patch for that.
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau