start sorting out the ZONE_DEVICE refcount mess
Subsystems (24)
Christoph Hellwig
maintainerHi all,
this series removes the offset by one refcount for ZONE_DEVICE pages that are freed back to the driver owning them, which is just device private ones for now, but also the planned device coherent pages and the ehanced p2p ones pending.
It does not address the fsdax pages yet, which will be attacked in a follow on series.
Diffstat:
arch/arm64/mm/mmu.c | 1 arch/powerpc/kvm/book3s_hv_uvmem.c | 1 drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 2 drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 drivers/gpu/drm/drm_cache.c | 2 drivers/gpu/drm/nouveau/nouveau_dmem.c | 3 - drivers/gpu/drm/nouveau/nouveau_svm.c | 1 drivers/infiniband/core/rw.c | 1 drivers/nvdimm/pmem.h | 1 drivers/nvme/host/pci.c | 1 drivers/nvme/target/io-cmd-bdev.c | 1 fs/Kconfig | 2 fs/fuse/virtio_fs.c | 1 include/linux/hmm.h | 9 ---- include/linux/memremap.h | 22 +++++++++- include/linux/mm.h | 59 ++++------------------------- lib/test_hmm.c | 4 + mm/Kconfig | 4 - mm/internal.h | 2 mm/memcontrol.c | 11 +---- mm/memremap.c | 63 ++++++++++++++++--------------- mm/migrate.c | 6 -- mm/swap.c | 49 ++---------------------- 23 files changed, 90 insertions(+), 157 deletions(-)
Christoph Hellwig
maintainerPATCH 1/8 mm: remove a pointless CONFIG_ZONE_DEVICE check in memremap_pages
memremap.c is only built when CONFIG_ZONE_DEVICE is set, so remove the superflous extra check.
--- mm/memremap.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
mm/memremap.c +1 -2
@@ -328,8 +328,7 @@ void *memremap_pages(struct dev_pagemap *pgmap, int nid) }break ;case MEMORY_DEVICE_FS_DAX : -if (!IS_ENABLED (CONFIG_ZONE_DEVICE ) || -IS_ENABLED (CONFIG_FS_DAX_LIMITED )) { +if (IS_ENABLED (CONFIG_FS_DAX_LIMITED )) {WARN (1 ,"File system DAX not supported\n" );return ERR_PTR (-EINVAL ); }
Signature · 1 line
2.30.2
Re: [PATCH 1/8] mm: remove a pointless CONFIG_ZONE_DEVICE check in memremap_pages
Quoted 3 lines · Christoph Hellwig
memremap.cis only built whenCONFIG_ZONE_DEVICEis set, so remove the superflous extra check.
Looks good to me.
Re: [PATCH 1/8] mm: remove a pointless CONFIG_ZONE_DEVICE check in memremap_pages
In reply to Christoph HellwigQuoted 7 lines · Christoph Hellwig
memremap.cis only built whenCONFIG_ZONE_DEVICEis set, so remove the superflous extra check.Signed-off-by Christoph Hellwig <[email protected]>--- mm/memremap.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
Jason
Re: [PATCH 1/8] mm: remove a pointless CONFIG_ZONE_DEVICE check in memremap_pages
In reply to Christoph HellwigQuoted 14 lines · Christoph Hellwig
memremap.cis only built whenCONFIG_ZONE_DEVICEis set, so remove the superflous extra check.Signed-off-by Christoph Hellwig <[email protected]>--- mm/memremap.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)mm/memremap.c +0
-0
mm/memremap.c void *memremap_pages(struct dev_pagemap *pgmap, int nid) · line 334
Quoted 9 lines
mm/memremap.c +1
-2@@ -328,8 +328,7 @@ void *memremap_pages(struct dev_pagemap *pgmap, int nid) }break ;case MEMORY_DEVICE_FS_DAX : -if (!IS_ENABLED (CONFIG_ZONE_DEVICE ) || -IS_ENABLED (CONFIG_FS_DAX_LIMITED )) { +if (IS_ENABLED (CONFIG_FS_DAX_LIMITED )) {WARN (1 ,"File system DAX not supported\n" );return ERR_PTR (-EINVAL ); }
Indeed it does have it in the makefile:-
root@dev mm (for-next) # grep memremap.o Makefile
obj-$(CONFIG_ZONE_DEVICE) += memremap.o
Looks good.
Re: [PATCH 1/8] mm: remove a pointless CONFIG_ZONE_DEVICE check in memremap_pages
In reply to Christoph HellwigQuoted 5 lines · Christoph Hellwig
memremap.cis only built whenCONFIG_ZONE_DEVICEis set, so remove the superflous extra check.Signed-off-by Christoph Hellwig <[email protected]>
Thanks.
Christoph Hellwig
maintainerPATCH 2/8 mm: remove the __KERNEL__ guard from <linux/mm.h>
In reply to Christoph Hellwig__KERNEL__ ifdefs don't make sense outside of include/uapi/.
--- include/linux/mm.h | 4 ---- 1 file changed, 4 deletions(-)
include/linux/mm.h +0 -4
@@ -3,9 +3,6 @@#define _LINUX_MM_H#include <linux/errno.h> - -#ifdef __KERNEL__ -#include <linux/mmdebug.h> #include <linux/gfp.h> #include <linux/bug.h>
@@ -3381,5 +3378,4 @@ madvise_set_anon_name(struct mm_struct *mm, unsigned long start, }#endif -#endif /* __KERNEL__ */ #endif /* _LINUX_MM_H */
Signature · 1 line
2.30.2
Re: [PATCH 2/8] mm: remove the __KERNEL__ guard from <linux/mm.h>
Quoted 2 lines · Christoph Hellwig
__KERNEL__ifdefs don't make sense outside ofinclude/uapi/.
Yes.
Re: [PATCH 2/8] mm: remove the __KERNEL__ guard from <linux/mm.h>
In reply to Christoph HellwigQuoted 6 lines · Christoph Hellwig
__KERNEL__ifdefs don't make sense outside ofinclude/uapi/.Signed-off-by Christoph Hellwig <[email protected]>--- include/linux/mm.h | 4 ---- 1 file changed, 4 deletions(-)
Jason
Re: [PATCH 2/8] mm: remove the __KERNEL__ guard from <linux/mm.h>
In reply to Christoph HellwigQuoted 23 lines · Christoph Hellwig
__KERNEL__ifdefs don't make sense outside ofinclude/uapi/.Signed-off-by Christoph Hellwig <[email protected]>--- include/linux/mm.h | 4 ---- 1 file changed, 4 deletions(-)include/linux/mm.h +0
-3@@ -3,9 +3,6 @@#define _LINUX_MM_H#include <linux/errno.h> - -#ifdef __KERNEL__ -#include <linux/mmdebug.h> #include <linux/gfp.h> #include <linux/bug.h>
include/linux/mm.h madvise_set_anon_name(struct mm_struct *mm, unsigned long start, · line 3381
Quoted 5 lines
include/linux/mm.h +0
-1@@ -3381,5 +3378,4 @@ madvise_set_anon_name(struct mm_struct *mm, unsigned long start, }#endif -#endif /* __KERNEL__ */ #endif /* _LINUX_MM_H */
Looks good.
Re: [PATCH 2/8] mm: remove the __KERNEL__ guard from <linux/mm.h>
In reply to Christoph HellwigQuoted 4 lines · Christoph Hellwig
__KERNEL__ifdefs don't make sense outside ofinclude/uapi/.Signed-off-by Christoph Hellwig <[email protected]>
Thanks.
Christoph Hellwig
maintainerPATCH 3/8 mm: remove pointless includes from <linux/hmm.h>
In reply to Christoph Hellwighmm.h pulls in the world for no good reason at all. Remove the includes and push a few ones into the users instead.
--- drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 1 + drivers/gpu/drm/nouveau/nouveau_dmem.c | 1 + include/linux/hmm.h | 9 ++------- lib/test_hmm.c | 2 ++ 4 files changed, 6 insertions(+), 7 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +1 -0
@@ -24,6 +24,7 @@#include <linux/hmm.h> #include <linux/dma-direction.h> #include <linux/dma-mapping.h> +#include <linux/migrate.h> #include "amdgpu_sync.h" #include "amdgpu_object.h" #include "amdgpu_vm.h"
drivers/gpu/drm/nouveau/nouveau_dmem.c +1 -0
@@ -39,6 +39,7 @@#include <linux/sched/mm.h> #include <linux/hmm.h> +#include <linux/migrate.h> /* * FIXME: this is ugly right now we are using TTM to allocate vram and we pin
include/linux/hmm.h +2 -7
@@ -9,14 +9,9 @@#ifndef LINUX_HMM_H #define LINUX_HMM_H -#include <linux/kconfig.h> -#include <linux/pgtable.h> +#include <linux/mm.h> -#include <linux/device.h> -#include <linux/migrate.h> -#include <linux/memremap.h> -#include <linux/completion.h> -#include <linux/mmu_notifier.h> +struct mmu_interval_notifier ;/* * On output:
lib/test_hmm.c +2 -0
@@ -26,6 +26,8 @@#include <linux/sched/mm.h> #include <linux/platform_device.h> #include <linux/rmap.h> +#include <linux/mmu_notifier.h> +#include <linux/migrate.h> #include "test_hmm_uapi.h"
Signature · 1 line
2.30.2
Re: [PATCH 3/8] mm: remove pointless includes from <linux/hmm.h>
Quoted 10 lines · Christoph Hellwig
hmm.hpulls in the world for no good reason at all. Remove the includes and push a few ones into the users instead.Signed-off-by Christoph Hellwig <[email protected]>--- drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 1 + drivers/gpu/drm/nouveau/nouveau_dmem.c | 1 + include/linux/hmm.h | 9 ++------- lib/test_hmm.c | 2 ++ 4 files changed, 6 insertions(+), 7 deletions(-)
Jason
Re: [PATCH 3/8] mm: remove pointless includes from <linux/hmm.h>
In reply to Christoph HellwigQuoted 4 lines · Christoph Hellwig
hmm.hpulls in the world for no good reason at all. Remove the includes and push a few ones into the users instead.Signed-off-by Christoph Hellwig<[email protected]>
Looks good.
Christoph Hellwig
maintainerPATCH 4/8 mm: move free_devmap_managed_page to memremap.c
In reply to Christoph Hellwigfree_devmap_managed_page has nothing to do with the code in swap.c, move it to live with the rest of the code for devmap handling.
--- include/linux/mm.h | 1 - mm/memremap.c | 21 +++++++++++++++++++++ mm/swap.c | 23 ----------------------- 3 files changed, 21 insertions(+), 24 deletions(-)
include/linux/mm.h +0 -1
@@ -1092,7 +1092,6 @@ static inline bool is_zone_movable_page(const struct page *page) }#ifdef CONFIG_DEV_PAGEMAP_OPS -void free_devmap_managed_page (struct page *page);DECLARE_STATIC_KEY_FALSE (devmap_managed_key);static inline bool page_is_devmap_managed (struct page *page)
mm/memremap.c +21 -0
@@ -501,4 +501,25 @@ void free_devmap_managed_page(struct page *page) page->mapping =NULL ; page->pgmap->ops->page_free(page); } + +void put_devmap_managed_page (struct page *page) +{ +int count; + +if (WARN_ON_ONCE (!page_is_devmap_managed (page))) +return ; + + count =page_ref_dec_return (page); + +/* +* devmap page refcounts are 1-based, rather than 0-based: if +* refcount is 1, then the page is free and the refcount is +* stable because nobody holds a reference on the page. +*/ +if (count ==1 ) +free_devmap_managed_page (page); +else if (!count) +__put_page (page); +} +EXPORT_SYMBOL (put_devmap_managed_page);#endif /* CONFIG_DEV_PAGEMAP_OPS */
mm/swap.c +0 -23
@@ -1153,26 +1153,3 @@ void __init swap_setup(void)* _really_ don't want to cluster much more */ } - -#ifdef CONFIG_DEV_PAGEMAP_OPS -void put_devmap_managed_page (struct page *page) -{ -int count; - -if (WARN_ON_ONCE (!page_is_devmap_managed (page))) -return ; - - count =page_ref_dec_return (page); - -/* -* devmap page refcounts are 1-based, rather than 0-based: if -* refcount is 1, then the page is free and the refcount is -* stable because nobody holds a reference on the page. -*/ -if (count ==1 ) -free_devmap_managed_page (page); -else if (!count) -__put_page (page); -} -EXPORT_SYMBOL (put_devmap_managed_page); -#endif
Signature · 1 line
2.30.2
Re: [PATCH 4/8] mm: move free_devmap_managed_page to memremap.c
Quoted 4 lines · Christoph Hellwig
free_devmap_managed_pagehas nothing to do with the code inswap.c, move it to live with the rest of the code for devmap handling.
Looks good.
Re: [PATCH 4/8] mm: move free_devmap_managed_page to memremap.c
In reply to Christoph HellwigQuoted 9 lines · Christoph Hellwig
free_devmap_managed_pagehas nothing to do with the code inswap.c, move it to live with the rest of the code for devmap handling.Signed-off-by Christoph Hellwig <[email protected]>--- include/linux/mm.h | 1 - mm/memremap.c | 21 +++++++++++++++++++++ mm/swap.c | 23 ----------------------- 3 files changed, 21 insertions(+), 24 deletions(-)
Jason
Re: [PATCH 4/8] mm: move free_devmap_managed_page to memremap.c
In reply to Christoph HellwigQuoted 4 lines · Christoph Hellwig
free_devmap_managed_pagehas nothing to do with the code inswap.c, move it to live with the rest of the code for devmap handling.Signed-off-by Christoph Hellwig<[email protected]>
True, the only devmap code is present in the swap.c is couple of calls in my tree.
Looks good.
Re: [PATCH 4/8] mm: move free_devmap_managed_page to memremap.c
In reply to Christoph HellwigQuoted 5 lines · Christoph Hellwig
free_devmap_managed_pagehas nothing to do with the code inswap.c, move it to live with the rest of the code for devmap handling.Signed-off-by Christoph Hellwig <[email protected]>
Thanks.
Christoph Hellwig
maintainerPATCH 5/8 mm: simplify freeing of devmap managed pages
In reply to Christoph HellwigMake put_devmap_managed_page return if it took charge of the page or not and remove the separate page_is_devmap_managed helper.
--- include/linux/mm.h | 34 ++++++++++------------------------ mm/memremap.c | 20 +++++++++----------- mm/swap.c | 10 +--------- 3 files changed, 20 insertions(+), 44 deletions(-)
include/linux/mm.h +10 -24
@@ -1094,33 +1094,24 @@ static inline bool is_zone_movable_page(const struct page *page)#ifdef CONFIG_DEV_PAGEMAP_OPS DECLARE_STATIC_KEY_FALSE (devmap_managed_key); -static inline bool page_is_devmap_managed (struct page *page) +bool __put_devmap_managed_page (struct page *page); +static inline bool put_devmap_managed_page (struct page *page) {if (!static_branch_unlikely (&devmap_managed_key))return false;if (!is_zone_device_page (page))return false; -switch (page->pgmap->type) { -case MEMORY_DEVICE_PRIVATE : -case MEMORY_DEVICE_FS_DAX : -return true; -default : -break ; - } -return false; +if (page->pgmap->type !=MEMORY_DEVICE_PRIVATE && + page->pgmap->type !=MEMORY_DEVICE_FS_DAX ) +return false; +return __put_devmap_managed_page (page); } -void put_devmap_managed_page (struct page *page); -#else /* CONFIG_DEV_PAGEMAP_OPS */ -static inline bool page_is_devmap_managed (struct page *page) +static inline bool put_devmap_managed_page (struct page *page) {return false; } - -static inline void put_devmap_managed_page (struct page *page) -{ -}#endif /* CONFIG_DEV_PAGEMAP_OPS */ static inline bool is_device_private_page (const struct page *page)
@@ -1220,16 +1211,11 @@ static inline void put_page(struct page *page)struct folio *folio =page_folio (page);/* -* For devmap managed pages we need to catch refcount transition from -* 2 to 1, when refcount reach one it means the page is free and we -* need to inform the device driver through callback. See -* include/linux/memremap.h and HMM for details. +* For some devmap managed pages we need to catch refcount transition +* from 2 to 1: */ -if (page_is_devmap_managed (&folio->page)) { -put_devmap_managed_page (&folio->page); +if (put_devmap_managed_page (&folio->page))return ; - } -folio_put (folio); }
mm/memremap.c +9 -11
@@ -502,24 +502,22 @@ void free_devmap_managed_page(struct page *page) page->pgmap->ops->page_free(page); } -void put_devmap_managed_page (struct page *page) +bool __put_devmap_managed_page (struct page *page) { -int count; - -if (WARN_ON_ONCE (!page_is_devmap_managed (page))) -return ; - - count =page_ref_dec_return (page); -/* * devmap page refcounts are 1-based, rather than 0-based: if * refcount is 1, then the page is free and the refcount is * stable because nobody holds a reference on the page. */ -if (count ==1 ) +switch (page_ref_dec_return (page)) { +case 1 :free_devmap_managed_page (page); -else if (!count) +break ; +case 0 :__put_page (page); +break ; + } +return true; } -EXPORT_SYMBOL (put_devmap_managed_page); +EXPORT_SYMBOL (__put_devmap_managed_page);#endif /* CONFIG_DEV_PAGEMAP_OPS */
mm/swap.c +1 -9
@@ -930,16 +930,8 @@ void release_pages(struct page **pages, int nr)unlock_page_lruvec_irqrestore (lruvec, flags); lruvec =NULL ; } -/* -* ZONE_DEVICE pages that return 'false' from -* page_is_devmap_managed() do not require special -* processing, and instead, expect a call to -* put_page_testzero(). -*/ -if (page_is_devmap_managed (page)) { -put_devmap_managed_page (page); +if (put_devmap_managed_page (page))continue ; - }if (put_page_testzero (page))put_dev_pagemap (page->pgmap);continue ;
Signature · 1 line
2.30.2
Re: [PATCH 5/8] mm: simplify freeing of devmap managed pages
Quoted 9 lines · Christoph Hellwig
Make
put_devmap_managed_pagereturn if it took charge of the page or not and remove the separatepage_is_devmap_managedhelper.Signed-off-by Christoph Hellwig <[email protected]>--- include/linux/mm.h | 34 ++++++++++------------------------ mm/memremap.c | 20 +++++++++----------- mm/swap.c | 10 +--------- 3 files changed, 20 insertions(+), 44 deletions(-)
Jason
Re: [PATCH 5/8] mm: simplify freeing of devmap managed pages
In reply to Christoph HellwigQuoted 3 lines · Christoph Hellwig
Make
put_devmap_managed_pagereturn if it took charge of the page or not and remove the separatepage_is_devmap_managedhelper.
Looks good to me:
Re: [PATCH 5/8] mm: simplify freeing of devmap managed pages
In reply to Christoph HellwigQuoted 5 lines
include/linux/mm.h +2
-1-static inline bool page_is_devmap_managed (struct page *page) +bool __put_devmap_managed_page (struct page *page); +static inline bool put_devmap_managed_page (struct page *page) {if (!static_branch_unlikely (&devmap_managed_key))
include/linux/mm.h static inline bool is_zone_movable_page(const struct page *page) · line 1109
Quoted 10 lines
include/linux/mm.h +0
-7return false;if (!is_zone_device_page (page))return false; -switch (page->pgmap->type) { -case MEMORY_DEVICE_PRIVATE : -case MEMORY_DEVICE_FS_DAX : -return true; -default : -break ; - }
nit:- how some variant of following to makes all cases evident without having to look into memremap.h for other enum values ?
switch (page->pgmap->type) {
case MEMORY_DEVICE_PRIVATE:
case MEMORY_DEVICE_FS_DAX:
return __put_devmap_managed_page(page);
case MEMORY_DEVICE_GENERIC:
case MEMORY_DEVICE_PCI_P2PDMA:
return false;
default:
WARN_ON_ONCE(1);
return false;
}include/linux/mm.h static inline bool is_zone_movable_page(const struct page *page) · line 1107
Quoted 5 lines
include/linux/mm.h +4
-1-return false; +if (page->pgmap->type !=MEMORY_DEVICE_PRIVATE && + page->pgmap->type !=MEMORY_DEVICE_FS_DAX ) +return false; +return __put_devmap_managed_page (page);
nit:- we are only returning true value from __put_devmap_managed_page() in this patch. Perhaps make it __put_dev_map_managed_page() return void and return true from above ?
or maybe someone can send a cleanup once this is merged.
Quoted 2 lines
}
Irrespective of above comment(s), looks good.
Christoph Hellwig
maintainerPATCH 6/8 mm: don't include <linux/memremap.h> in <linux/mm.h>
In reply to Christoph HellwigMove the check for the actual pgmap types that need the free at refcount one behavior into the out of line helper, and thus avoid the need to pull memremap.h into mm.h.
--- arch/arm64/mm/mmu.c | 1 + drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 + drivers/gpu/drm/drm_cache.c | 2 +- drivers/gpu/drm/nouveau/nouveau_dmem.c | 1 + drivers/gpu/drm/nouveau/nouveau_svm.c | 1 + drivers/infiniband/core/rw.c | 1 + drivers/nvdimm/pmem.h | 1 + drivers/nvme/host/pci.c | 1 + drivers/nvme/target/io-cmd-bdev.c | 1 + fs/fuse/virtio_fs.c | 1 + include/linux/memremap.h | 18 ++++++++++++++++++ include/linux/mm.h | 20 -------------------- lib/test_hmm.c | 1 + mm/memremap.c | 6 +++++- 14 files changed, 34 insertions(+), 22 deletions(-)
arch/arm64/mm/mmu.c +1 -0
@@ -17,6 +17,7 @@#include <linux/mman.h> #include <linux/nodemask.h> #include <linux/memblock.h> +#include <linux/memremap.h> #include <linux/memory.h> #include <linux/fs.h> #include <linux/io.h>
drivers/gpu/drm/amd/amdkfd/kfd_priv.h +1 -0
@@ -25,6 +25,7 @@#include <linux/hashtable.h> #include <linux/mmu_notifier.h> +#include <linux/memremap.h> #include <linux/mutex.h> #include <linux/types.h> #include <linux/atomic.h>
drivers/gpu/drm/drm_cache.c +1 -1
@@ -27,11 +27,11 @@/* * Authors: Thomas Hellström <thomas-at-tungstengraphics-dot-com> */ -#include <linux/dma-buf-map.h> #include <linux/export.h> #include <linux/highmem.h> #include <linux/cc_platform.h> +#include <linux/ioport.h> #include <xen/xen.h> #include <drm/drm_cache.h>
drivers/gpu/drm/nouveau/nouveau_dmem.c +1 -0
@@ -39,6 +39,7 @@#include <linux/sched/mm.h> #include <linux/hmm.h> +#include <linux/memremap.h> #include <linux/migrate.h> /*
drivers/gpu/drm/nouveau/nouveau_svm.c +1 -0
@@ -35,6 +35,7 @@#include <linux/sched/mm.h> #include <linux/sort.h> #include <linux/hmm.h> +#include <linux/memremap.h> #include <linux/rmap.h> struct nouveau_svm {
drivers/infiniband/core/rw.c +1 -0
@@ -2,6 +2,7 @@/* * Copyright (c) 2016 HGST, a Western Digital Company. */ +#include <linux/memremap.h> #include <linux/moduleparam.h> #include <linux/slab.h> #include <linux/pci-p2pdma.h>
drivers/nvdimm/pmem.h +1 -0
@@ -3,6 +3,7 @@#define __NVDIMM_PMEM_H__#include <linux/page-flags.h> #include <linux/badblocks.h> +#include <linux/memremap.h> #include <linux/types.h> #include <linux/pfn_t.h> #include <linux/fs.h>
drivers/nvme/host/pci.c +1 -0
@@ -15,6 +15,7 @@#include <linux/init.h> #include <linux/interrupt.h> #include <linux/io.h> +#include <linux/memremap.h> #include <linux/mm.h> #include <linux/module.h> #include <linux/mutex.h>
drivers/nvme/target/io-cmd-bdev.c +1 -0
@@ -6,6 +6,7 @@#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt#include <linux/blkdev.h> #include <linux/blk-integrity.h> +#include <linux/memremap.h> #include <linux/module.h> #include "nvmet.h"
fs/fuse/virtio_fs.c +1 -0
@@ -8,6 +8,7 @@#include <linux/dax.h> #include <linux/pci.h> #include <linux/pfn_t.h> +#include <linux/memremap.h> #include <linux/module.h> #include <linux/virtio.h> #include <linux/virtio_fs.h>
include/linux/memremap.h +18 -0
@@ -1,6 +1,8 @@/* SPDX-License-Identifier: GPL-2.0 */ #ifndef _LINUX_MEMREMAP_H_#define _LINUX_MEMREMAP_H_ + +#include <linux/mm.h> #include <linux/range.h> #include <linux/ioport.h> #include <linux/percpu-refcount.h>
@@ -129,6 +131,22 @@ static inline unsigned long pgmap_vmemmap_nr(struct dev_pagemap *pgmap)return 1 << pgmap->vmemmap_shift; } +static inline bool is_device_private_page (const struct page *page) +{ +return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && +IS_ENABLED (CONFIG_DEVICE_PRIVATE ) && +is_zone_device_page (page) && + page->pgmap->type ==MEMORY_DEVICE_PRIVATE ; +} + +static inline bool is_pci_p2pdma_page (const struct page *page) +{ +return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && +IS_ENABLED (CONFIG_PCI_P2PDMA ) && +is_zone_device_page (page) && + page->pgmap->type ==MEMORY_DEVICE_PCI_P2PDMA ; +} +#ifdef CONFIG_ZONE_DEVICE void *memremap_pages (struct dev_pagemap *pgmap,int nid);void memunmap_pages (struct dev_pagemap *pgmap);
include/linux/mm.h +0 -20
@@ -23,7 +23,6 @@#include <linux/err.h> #include <linux/page-flags.h> #include <linux/page_ref.h> -#include <linux/memremap.h> #include <linux/overflow.h> #include <linux/sizes.h> #include <linux/sched.h>
@@ -1101,9 +1100,6 @@ static inline bool put_devmap_managed_page(struct page *page)return false;if (!is_zone_device_page (page))return false; -if (page->pgmap->type !=MEMORY_DEVICE_PRIVATE && - page->pgmap->type !=MEMORY_DEVICE_FS_DAX ) -return false;return __put_devmap_managed_page (page); }
@@ -1114,22 +1110,6 @@ static inline bool put_devmap_managed_page(struct page *page) }#endif /* CONFIG_DEV_PAGEMAP_OPS */ -static inline bool is_device_private_page (const struct page *page) -{ -return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && -IS_ENABLED (CONFIG_DEVICE_PRIVATE ) && -is_zone_device_page (page) && - page->pgmap->type ==MEMORY_DEVICE_PRIVATE ; -} - -static inline bool is_pci_p2pdma_page (const struct page *page) -{ -return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && -IS_ENABLED (CONFIG_PCI_P2PDMA ) && -is_zone_device_page (page) && - page->pgmap->type ==MEMORY_DEVICE_PCI_P2PDMA ; -} -/* 127: arbitrary random number, small enough to assemble well */ #define folio_ref_zero_or_close_to_overflow (folio) \ ((unsigned int) folio_ref_count(folio) + 127u <= 127u)
lib/test_hmm.c +1 -0
@@ -12,6 +12,7 @@#include <linux/kernel.h> #include <linux/cdev.h> #include <linux/device.h> +#include <linux/memremap.h> #include <linux/mutex.h> #include <linux/rwsem.h> #include <linux/sched.h>
mm/memremap.c +5 -1
@@ -4,7 +4,7 @@#include <linux/io.h> #include <linux/kasan.h> #include <linux/memory_hotplug.h> -#include <linux/mm.h> +#include <linux/memremap.h> #include <linux/pfn_t.h> #include <linux/swap.h> #include <linux/mmzone.h>
@@ -504,6 +504,10 @@ void free_devmap_managed_page(struct page *page)bool __put_devmap_managed_page (struct page *page) { +if (page->pgmap->type !=MEMORY_DEVICE_PRIVATE && + page->pgmap->type !=MEMORY_DEVICE_FS_DAX ) +return false; +/* * devmap page refcounts are 1-based, rather than 0-based: if * refcount is 1, then the page is free and the refcount is
Signature · 1 line
2.30.2
Re: [PATCH 6/8] mm: don't include <linux/memremap.h> in <linux/mm.h>
Quoted 5 lines · Christoph Hellwig
Move the check for the actual pgmap types that need the free at refcount one behavior into the out of line helper, and thus avoid the need to pull
memremap.hintomm.h.Signed-off-by Christoph Hellwig <[email protected]>
I've noticed mm/memcontrol.c uses is_device_private_page() and also needs a memremap.h include added to compile with my configuration.
Logan
Re: [PATCH 6/8] mm: don't include <linux/memremap.h> in <linux/mm.h>
In reply to Christoph HellwigQuoted 5 lines · Christoph Hellwig
Move the check for the actual pgmap types that need the free at refcount one behavior into the out of line helper, and thus avoid the need to pull
memremap.hintomm.h.Signed-off-by Christoph Hellwig <[email protected]>
Quoted 16 lines
--- arch/arm64/mm/mmu.c | 1 + drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 + drivers/gpu/drm/drm_cache.c | 2 +- drivers/gpu/drm/nouveau/nouveau_dmem.c | 1 + drivers/gpu/drm/nouveau/nouveau_svm.c | 1 + drivers/infiniband/core/rw.c | 1 + drivers/nvdimm/pmem.h | 1 + drivers/nvme/host/pci.c | 1 + drivers/nvme/target/io-cmd-bdev.c | 1 + fs/fuse/virtio_fs.c | 1 + include/linux/memremap.h | 18 ++++++++++++++++++ include/linux/mm.h | 20 -------------------- lib/test_hmm.c | 1 + mm/memremap.c | 6 +++++- 14 files changed, 34 insertions(+), 22 deletions(-)
Jason
Re: [PATCH 6/8] mm: don't include <linux/memremap.h> in <linux/mm.h>
In reply to Christoph HellwigAm 2022-02-07 um 01:32 schrieb Christoph Hellwig:
Quoted 5 lines
Move the check for the actual pgmap types that need the free at refcount one behavior into the out of line helper, and thus avoid the need to pull
memremap.hintomm.h.Signed-off-by Christoph Hellwig <[email protected]>
The amdkfd part looks good to me.
It looks like this patch is not based on Alex Sierra's coherent memory series. He added two new helpers is_device_coherent_page and is_dev_private_or_coherent_page that would need to be moved along with is_device_private_page and is_pci_p2pdma_page.
Quoted 259 lines
--- arch/arm64/mm/mmu.c | 1 + drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 + drivers/gpu/drm/drm_cache.c | 2 +- drivers/gpu/drm/nouveau/nouveau_dmem.c | 1 + drivers/gpu/drm/nouveau/nouveau_svm.c | 1 + drivers/infiniband/core/rw.c | 1 + drivers/nvdimm/pmem.h | 1 + drivers/nvme/host/pci.c | 1 + drivers/nvme/target/io-cmd-bdev.c | 1 + fs/fuse/virtio_fs.c | 1 + include/linux/memremap.h | 18 ++++++++++++++++++ include/linux/mm.h | 20 -------------------- lib/test_hmm.c | 1 + mm/memremap.c | 6 +++++- 14 files changed, 34 insertions(+), 22 deletions(-)arch/arm64/mm/mmu.c +1
-0@@ -17,6 +17,7 @@#include <linux/mman.h> #include <linux/nodemask.h> #include <linux/memblock.h> +#include <linux/memremap.h> #include <linux/memory.h> #include <linux/fs.h> #include <linux/io.h> drivers/gpu/drm/amd/amdkfd/kfd_priv.h +1
-0@@ -25,6 +25,7 @@#include <linux/hashtable.h> #include <linux/mmu_notifier.h> +#include <linux/memremap.h> #include <linux/mutex.h> #include <linux/types.h> #include <linux/atomic.h> drivers/gpu/drm/drm_cache.c +1
-1@@ -27,11 +27,11 @@/* * Authors: Thomas Hellström <thomas-at-tungstengraphics-dot-com> */ -#include <linux/dma-buf-map.h> #include <linux/export.h> #include <linux/highmem.h> #include <linux/cc_platform.h> +#include <linux/ioport.h> #include <xen/xen.h> #include <drm/drm_cache.h> drivers/gpu/drm/nouveau/nouveau_dmem.c +1
-0@@ -39,6 +39,7 @@#include <linux/sched/mm.h> #include <linux/hmm.h> +#include <linux/memremap.h> #include <linux/migrate.h> /* drivers/gpu/drm/nouveau/nouveau_svm.c +1
-0@@ -35,6 +35,7 @@#include <linux/sched/mm.h> #include <linux/sort.h> #include <linux/hmm.h> +#include <linux/memremap.h> #include <linux/rmap.h> struct nouveau_svm {drivers/infiniband/core/rw.c +1
-0@@ -2,6 +2,7 @@/* * Copyright (c) 2016 HGST, a Western Digital Company. */ +#include <linux/memremap.h> #include <linux/moduleparam.h> #include <linux/slab.h> #include <linux/pci-p2pdma.h> drivers/nvdimm/pmem.h +1
-0@@ -3,6 +3,7 @@#define __NVDIMM_PMEM_H__#include <linux/page-flags.h> #include <linux/badblocks.h> +#include <linux/memremap.h> #include <linux/types.h> #include <linux/pfn_t.h> #include <linux/fs.h> drivers/nvme/host/pci.c +1
-0@@ -15,6 +15,7 @@#include <linux/init.h> #include <linux/interrupt.h> #include <linux/io.h> +#include <linux/memremap.h> #include <linux/mm.h> #include <linux/module.h> #include <linux/mutex.h> drivers/nvme/target/io-cmd-bdev.c +1
-0@@ -6,6 +6,7 @@#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt#include <linux/blkdev.h> #include <linux/blk-integrity.h> +#include <linux/memremap.h> #include <linux/module.h> #include "nvmet.h" fs/fuse/virtio_fs.c +1
-0@@ -8,6 +8,7 @@#include <linux/dax.h> #include <linux/pci.h> #include <linux/pfn_t.h> +#include <linux/memremap.h> #include <linux/module.h> #include <linux/virtio.h> #include <linux/virtio_fs.h> include/linux/memremap.h +18
-0@@ -1,6 +1,8 @@/* SPDX-License-Identifier: GPL-2.0 */ #ifndef _LINUX_MEMREMAP_H_#define _LINUX_MEMREMAP_H_ + +#include <linux/mm.h> #include <linux/range.h> #include <linux/ioport.h> #include <linux/percpu-refcount.h> @@ -129,6 +131,22 @@ static inline unsigned long pgmap_vmemmap_nr(struct dev_pagemap *pgmap)return 1 << pgmap->vmemmap_shift; } +static inline bool is_device_private_page (const struct page *page) +{ +return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && +IS_ENABLED (CONFIG_DEVICE_PRIVATE ) && +is_zone_device_page (page) && + page->pgmap->type ==MEMORY_DEVICE_PRIVATE ; +} + +static inline bool is_pci_p2pdma_page (const struct page *page) +{ +return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && +IS_ENABLED (CONFIG_PCI_P2PDMA ) && +is_zone_device_page (page) && + page->pgmap->type ==MEMORY_DEVICE_PCI_P2PDMA ; +} +#ifdef CONFIG_ZONE_DEVICE void *memremap_pages (struct dev_pagemap *pgmap,int nid);void memunmap_pages (struct dev_pagemap *pgmap);include/linux/mm.h +0
-20@@ -23,7 +23,6 @@#include <linux/err.h> #include <linux/page-flags.h> #include <linux/page_ref.h> -#include <linux/memremap.h> #include <linux/overflow.h> #include <linux/sizes.h> #include <linux/sched.h> @@ -1101,9 +1100,6 @@ static inline bool put_devmap_managed_page(struct page *page)return false;if (!is_zone_device_page (page))return false; -if (page->pgmap->type !=MEMORY_DEVICE_PRIVATE && - page->pgmap->type !=MEMORY_DEVICE_FS_DAX ) -return false;return __put_devmap_managed_page (page); }@@ -1114,22 +1110,6 @@ static inline bool put_devmap_managed_page(struct page *page) }#endif /* CONFIG_DEV_PAGEMAP_OPS */ -static inline bool is_device_private_page (const struct page *page) -{ -return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && -IS_ENABLED (CONFIG_DEVICE_PRIVATE ) && -is_zone_device_page (page) && - page->pgmap->type ==MEMORY_DEVICE_PRIVATE ; -} - -static inline bool is_pci_p2pdma_page (const struct page *page) -{ -return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && -IS_ENABLED (CONFIG_PCI_P2PDMA ) && -is_zone_device_page (page) && - page->pgmap->type ==MEMORY_DEVICE_PCI_P2PDMA ; -} -/* 127: arbitrary random number, small enough to assemble well */ #define folio_ref_zero_or_close_to_overflow (folio) \ ((unsigned int) folio_ref_count(folio) + 127u <= 127u)lib/test_hmm.c +1
-0@@ -12,6 +12,7 @@#include <linux/kernel.h> #include <linux/cdev.h> #include <linux/device.h> +#include <linux/memremap.h> #include <linux/mutex.h> #include <linux/rwsem.h> #include <linux/sched.h> mm/memremap.c +5
-1@@ -4,7 +4,7 @@#include <linux/io.h> #include <linux/kasan.h> #include <linux/memory_hotplug.h> -#include <linux/mm.h> +#include <linux/memremap.h> #include <linux/pfn_t.h> #include <linux/swap.h> #include <linux/mmzone.h> @@ -504,6 +504,10 @@ void free_devmap_managed_page(struct page *page)bool __put_devmap_managed_page (struct page *page) { +if (page->pgmap->type !=MEMORY_DEVICE_PRIVATE && + page->pgmap->type !=MEMORY_DEVICE_FS_DAX ) +return false; +/* * devmap page refcounts are 1-based, rather than 0-based: if * refcount is 1, then the page is free and the refcount is
Christoph Hellwig
maintainerRe: [PATCH 6/8] mm: don't include <linux/memremap.h> in <linux/mm.h>
Quoted 6 lines · Felix Kuehling
Am 2022-02-07 um 01:32 schrieb Christoph Hellwig:
Quoted 4 lines
Move the check for the actual pgmap types that need the free at refcount one behavior into the out of line helper, and thus avoid the need to pull
memremap.hintomm.h.
Quoted 8 lines
Quoted 1 line
Signed-off-by Christoph Hellwig <[email protected]>The amdkfd part looks good to me.
It looks like this patch is not based on Alex Sierra's coherent memory series. He added two new helpers
is_device_coherent_pageandis_dev_private_or_coherent_pagethat would need to be moved along withis_device_private_pageandis_pci_p2pdma_page.
Yes. I Naked that series because it spreads te mess with the refcount further in this latest version. My intent is that it gets rebased on top of this to avoid that spread. Same for the p2p series form Logan.
Christoph Hellwig
maintainerRe: [PATCH 6/8] mm: don't include <linux/memremap.h> in <linux/mm.h>
In reply to Felix KuehlingQuoted 6 lines · Felix Kuehling
Am 2022-02-07 um 01:32 schrieb Christoph Hellwig:
Quoted 4 lines
Move the check for the actual pgmap types that need the free at refcount one behavior into the out of line helper, and thus avoid the need to pull
memremap.hintomm.h.
Quoted 8 lines
Quoted 1 line
Signed-off-by Christoph Hellwig <[email protected]>The amdkfd part looks good to me.
It looks like this patch is not based on Alex Sierra's coherent memory series. He added two new helpers
is_device_coherent_pageandis_dev_private_or_coherent_pagethat would need to be moved along withis_device_private_pageandis_pci_p2pdma_page.
FYI, here is a branch that contains a rebase of the coherent memory related patches on top of this series:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/pgmap-refcount
I don't have a good way to test this, but I'll at least let the build bot finish before sending it out (probably tomorrow).
Alistair Popple
reviewerRe: [PATCH 6/8] mm: don't include <linux/memremap.h> in <linux/mm.h>
Quoted 16 lines · AEDT Christoph Hellwig
Quoted 6 lines · Felix Kuehling
Am 2022-02-07 um 01:32 schrieb Christoph Hellwig:
Quoted 4 lines
Move the check for the actual pgmap types that need the free at refcount one behavior into the out of line helper, and thus avoid the need to pull
memremap.hintomm.h.Quoted 8 lines
Quoted 1 line
Signed-off-by Christoph Hellwig <[email protected]>The amdkfd part looks good to me.
It looks like this patch is not based on Alex Sierra's coherent memory series. He added two new helpers
is_device_coherent_pageandis_dev_private_or_coherent_pagethat would need to be moved along withis_device_private_pageandis_pci_p2pdma_page.
Quoted 7 lines
FYI, here is a branch that contains a rebase of the coherent memory related patches on top of this series:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/pgmap-refcount
I don't have a good way to test this, but I'll at least let the build bot finish before sending it out (probably tomorrow).
Thanks, I ran up hmm-test which revealed a few minor problems with the rebase. Fixes below.
---
mm/gup.c +0 -1
@@ -2007,7 +2007,6 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages,if (!ret &&list_empty (&movable_page_list) && !isolation_error_count)return nr_pages; - ret =0 ; unpin_pages:for (i =0 ; i < nr_pages; i++)if (!pages[i])
mm/migrate.c +3 -4
@@ -2686,12 +2686,11 @@ static void migrate_vma_insert_page(struct migrate_vma *migrate, swp_entry =make_readable_device_private_entry (page_to_pfn (page)); entry =swp_entry_to_pte (swp_entry); - }else { -if (is_zone_device_page (page) && -is_device_coherent_page (page)) { + }else if (is_zone_device_page (page) && + !is_device_coherent_page (page)) {pr_warn_once ("Unsupported ZONE_DEVICE page type.\n" ); goto abort; - } + }else { entry =mk_pte (page, vma->vm_page_prot);if (vma->vm_flags &VM_WRITE ) entry =pte_mkwrite (pte_mkdirty (entry));
Christoph Hellwig
maintainerRe: [PATCH 6/8] mm: don't include <linux/memremap.h> in <linux/mm.h>
mm/gup.c static long check_and_migrate_movable_pages(unsigned long nr_pages, · line 2010
Quoted 10 lines · Alistair Popple
mm/gup.c +0
-1@@ -2007,7 +2007,6 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages,if (!ret &&list_empty (&movable_page_list) && !isolation_error_count)return nr_pages; - ret =0 ; unpin_pages:
This isn't quite correct as ret is initially set to -EFAULT now. I'll fix it by removing the early ret initialization and always using the goto. I've also added another refactoring patch for this messy function.
I've folded the inversion of the is_device_coherent_page check in migrate.c in as well, thanks!
Felix Kuehling
maintainerRe: [PATCH 6/8] mm: don't include <linux/memremap.h> in <linux/mm.h>
In reply to Christoph HellwigAm 2022-02-09 um 12:48 schrieb Christoph Hellwig:
Quoted 13 lines · Felix Kuehling
Quoted 5 lines
Am 2022-02-07 um 01:32 schrieb Christoph Hellwig:
Quoted 4 lines
Move the check for the actual pgmap types that need the free at refcount one behavior into the out of line helper, and thus avoid the need to pull
memremap.hintomm.h.Quoted 7 lines
Quoted 1 line
Signed-off-by Christoph Hellwig <[email protected]>The amdkfd part looks good to me.
It looks like this patch is not based on Alex Sierra's coherent memory series. He added two new helpers
is_device_coherent_pageandis_dev_private_or_coherent_pagethat would need to be moved along withis_device_private_pageandis_pci_p2pdma_page.
Quoted 7 lines
FYI, here is a branch that contains a rebase of the coherent memory related patches on top of this series:
http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/pgmap-refcount
I don't have a good way to test this, but I'll at least let the build bot finish before sending it out (probably tomorrow).
Thank you for taking care of this rebase! Alex tested it on one of our coherent memory systems and it passed our tests.
I see you also included these rebased patches in your latest 27-patch series. I'll try to review the changes in more detail over the weekend.
Regards,
Felix
Re: [PATCH 6/8] mm: don't include <linux/memremap.h> in <linux/mm.h>
In reply to Christoph HellwigQuoted 4 lines · Christoph Hellwig
Move the check for the actual pgmap types that need the free at refcount one behavior into the out of line helper, and thus avoid the need to pull
memremap.hintomm.h.
Looks good to me assuming the compile bots agree.
Quoted 264 lines
Signed-off-by Christoph Hellwig <[email protected]>--- arch/arm64/mm/mmu.c | 1 + drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 + drivers/gpu/drm/drm_cache.c | 2 +- drivers/gpu/drm/nouveau/nouveau_dmem.c | 1 + drivers/gpu/drm/nouveau/nouveau_svm.c | 1 + drivers/infiniband/core/rw.c | 1 + drivers/nvdimm/pmem.h | 1 + drivers/nvme/host/pci.c | 1 + drivers/nvme/target/io-cmd-bdev.c | 1 + fs/fuse/virtio_fs.c | 1 + include/linux/memremap.h | 18 ++++++++++++++++++ include/linux/mm.h | 20 -------------------- lib/test_hmm.c | 1 + mm/memremap.c | 6 +++++- 14 files changed, 34 insertions(+), 22 deletions(-)arch/arm64/mm/mmu.c +1
-0@@ -17,6 +17,7 @@#include <linux/mman.h> #include <linux/nodemask.h> #include <linux/memblock.h> +#include <linux/memremap.h> #include <linux/memory.h> #include <linux/fs.h> #include <linux/io.h> drivers/gpu/drm/amd/amdkfd/kfd_priv.h +1
-0@@ -25,6 +25,7 @@#include <linux/hashtable.h> #include <linux/mmu_notifier.h> +#include <linux/memremap.h> #include <linux/mutex.h> #include <linux/types.h> #include <linux/atomic.h> drivers/gpu/drm/drm_cache.c +1
-1@@ -27,11 +27,11 @@/* * Authors: Thomas Hellström <thomas-at-tungstengraphics-dot-com> */ -#include <linux/dma-buf-map.h> #include <linux/export.h> #include <linux/highmem.h> #include <linux/cc_platform.h> +#include <linux/ioport.h> #include <xen/xen.h> #include <drm/drm_cache.h> drivers/gpu/drm/nouveau/nouveau_dmem.c +1
-0@@ -39,6 +39,7 @@#include <linux/sched/mm.h> #include <linux/hmm.h> +#include <linux/memremap.h> #include <linux/migrate.h> /* drivers/gpu/drm/nouveau/nouveau_svm.c +1
-0@@ -35,6 +35,7 @@#include <linux/sched/mm.h> #include <linux/sort.h> #include <linux/hmm.h> +#include <linux/memremap.h> #include <linux/rmap.h> struct nouveau_svm {drivers/infiniband/core/rw.c +1
-0@@ -2,6 +2,7 @@/* * Copyright (c) 2016 HGST, a Western Digital Company. */ +#include <linux/memremap.h> #include <linux/moduleparam.h> #include <linux/slab.h> #include <linux/pci-p2pdma.h> drivers/nvdimm/pmem.h +1
-0@@ -3,6 +3,7 @@#define __NVDIMM_PMEM_H__#include <linux/page-flags.h> #include <linux/badblocks.h> +#include <linux/memremap.h> #include <linux/types.h> #include <linux/pfn_t.h> #include <linux/fs.h> drivers/nvme/host/pci.c +1
-0@@ -15,6 +15,7 @@#include <linux/init.h> #include <linux/interrupt.h> #include <linux/io.h> +#include <linux/memremap.h> #include <linux/mm.h> #include <linux/module.h> #include <linux/mutex.h> drivers/nvme/target/io-cmd-bdev.c +1
-0@@ -6,6 +6,7 @@#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt#include <linux/blkdev.h> #include <linux/blk-integrity.h> +#include <linux/memremap.h> #include <linux/module.h> #include "nvmet.h" fs/fuse/virtio_fs.c +1
-0@@ -8,6 +8,7 @@#include <linux/dax.h> #include <linux/pci.h> #include <linux/pfn_t.h> +#include <linux/memremap.h> #include <linux/module.h> #include <linux/virtio.h> #include <linux/virtio_fs.h> include/linux/memremap.h +18
-0@@ -1,6 +1,8 @@/* SPDX-License-Identifier: GPL-2.0 */ #ifndef _LINUX_MEMREMAP_H_#define _LINUX_MEMREMAP_H_ + +#include <linux/mm.h> #include <linux/range.h> #include <linux/ioport.h> #include <linux/percpu-refcount.h> @@ -129,6 +131,22 @@ static inline unsigned long pgmap_vmemmap_nr(struct dev_pagemap *pgmap)return 1 << pgmap->vmemmap_shift; } +static inline bool is_device_private_page (const struct page *page) +{ +return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && +IS_ENABLED (CONFIG_DEVICE_PRIVATE ) && +is_zone_device_page (page) && + page->pgmap->type ==MEMORY_DEVICE_PRIVATE ; +} + +static inline bool is_pci_p2pdma_page (const struct page *page) +{ +return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && +IS_ENABLED (CONFIG_PCI_P2PDMA ) && +is_zone_device_page (page) && + page->pgmap->type ==MEMORY_DEVICE_PCI_P2PDMA ; +} +#ifdef CONFIG_ZONE_DEVICE void *memremap_pages (struct dev_pagemap *pgmap,int nid);void memunmap_pages (struct dev_pagemap *pgmap);include/linux/mm.h +0
-20@@ -23,7 +23,6 @@#include <linux/err.h> #include <linux/page-flags.h> #include <linux/page_ref.h> -#include <linux/memremap.h> #include <linux/overflow.h> #include <linux/sizes.h> #include <linux/sched.h> @@ -1101,9 +1100,6 @@ static inline bool put_devmap_managed_page(struct page *page)return false;if (!is_zone_device_page (page))return false; -if (page->pgmap->type !=MEMORY_DEVICE_PRIVATE && - page->pgmap->type !=MEMORY_DEVICE_FS_DAX ) -return false;return __put_devmap_managed_page (page); }@@ -1114,22 +1110,6 @@ static inline bool put_devmap_managed_page(struct page *page) }#endif /* CONFIG_DEV_PAGEMAP_OPS */ -static inline bool is_device_private_page (const struct page *page) -{ -return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && -IS_ENABLED (CONFIG_DEVICE_PRIVATE ) && -is_zone_device_page (page) && - page->pgmap->type ==MEMORY_DEVICE_PRIVATE ; -} - -static inline bool is_pci_p2pdma_page (const struct page *page) -{ -return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && -IS_ENABLED (CONFIG_PCI_P2PDMA ) && -is_zone_device_page (page) && - page->pgmap->type ==MEMORY_DEVICE_PCI_P2PDMA ; -} -/* 127: arbitrary random number, small enough to assemble well */ #define folio_ref_zero_or_close_to_overflow (folio) \ ((unsigned int) folio_ref_count(folio) + 127u <= 127u)lib/test_hmm.c +1
-0@@ -12,6 +12,7 @@#include <linux/kernel.h> #include <linux/cdev.h> #include <linux/device.h> +#include <linux/memremap.h> #include <linux/mutex.h> #include <linux/rwsem.h> #include <linux/sched.h> mm/memremap.c +5
-2@@ -4,7 +4,7 @@#include <linux/io.h> #include <linux/kasan.h> #include <linux/memory_hotplug.h> -#include <linux/mm.h> +#include <linux/memremap.h> #include <linux/pfn_t.h> #include <linux/swap.h> #include <linux/mmzone.h> @@ -504,6 +504,10 @@ void free_devmap_managed_page(struct page *page)bool __put_devmap_managed_page (struct page *page) { +if (page->pgmap->type !=MEMORY_DEVICE_PRIVATE && + page->pgmap->type !=MEMORY_DEVICE_FS_DAX ) +return false; +/* * devmap page refcounts are 1-based, rather than 0-based: if * refcount is 1, then the page is free and the refcount is -- 2.30.2
Re: [PATCH 6/8] mm: don't include <linux/memremap.h> in <linux/mm.h>
Quoted 10 lines · Dan Williams
Quoted 4 lines · Christoph Hellwig
Move the check for the actual pgmap types that need the free at refcount one behavior into the out of line helper, and thus avoid the need to pull
memremap.hintomm.h.Looks good to me assuming the compile bots agree.
Reviewed-by Dan Williams <[email protected]>
Yeah, same as Logan:
mm/memcontrol.c: In function ‘get_mctgt_type’:mm/memcontrol.c:5724:29: error: implicit declaration of function ‘is_device_private_page’; did you mean ‘is_device_private_entry’? [-Werror=implicit-function-declaration]
5724 | if (is_device_private_page(page))
| ^~~~~~~~~~~~~~~~~~~~~~
| is_device_private_entry...needs:
mm/memcontrol.c +1 -0
@@ -62,6 +62,7 @@#include <linux/tracehook.h> #include <linux/psi.h> #include <linux/seq_buf.h> +#include <linux/memremap.h> #include "internal.h" #include <net/sock.h> #include <net/ip.h>
Christoph Hellwig
maintainerRe: [PATCH 6/8] mm: don't include <linux/memremap.h> in <linux/mm.h>
Quoted 11 lines · Dan Williams
Yeah, same as Logan:
mm/memcontrol.c: In function ‘get_mctgt_type’:mm/memcontrol.c:5724:29: error: implicit declaration of function ‘is_device_private_page’; did you mean ‘is_device_private_entry’? [-Werror=implicit-function-declaration]5724 | if (is_device_private_page(page)) | ^~~~~~~~~~~~~~~~~~~~~~ | is_device_private_entry...needs:
Yeah, the buildbot also complained. I've fixed this up locally now.
Christoph Hellwig
maintainerPATCH 7/8 mm: remove the extra ZONE_DEVICE struct page refcount
In reply to Christoph HellwigZONE_DEVICE struct pages have an extra reference count that complicates the code for put_page() and several places in the kernel that need to check the reference count to see that a page is not being used (gup, compaction, migration, etc.). Clean up the code so the reference count doesn't need to be treated specially for ZONE_DEVICE pages.
Note that this excludes the special idle page wakeup for fsdax pages, which still happens at refcount 1. This is a separate issue and will be sorted out later. Given that only fsdax pages require the notifiacation when the refcount hits 1 now, the PAGEMAP_OPS Kconfig symbol can go away and be replaced with a FS_DAX check for this hook in the put_page fastpath.
Based on an earlier patch from Ralph Campbell <[email protected]>.
--- arch/powerpc/kvm/book3s_hv_uvmem.c | 1 - drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 1 - drivers/gpu/drm/nouveau/nouveau_dmem.c | 1 - fs/Kconfig | 1 - include/linux/memremap.h | 12 +++-- include/linux/mm.h | 6 +-- lib/test_hmm.c | 1 - mm/Kconfig | 4 -- mm/internal.h | 2 + mm/memcontrol.c | 11 ++--- mm/memremap.c | 57 ++++++++---------------- mm/migrate.c | 6 --- mm/swap.c | 16 ++----- 13 files changed, 36 insertions(+), 83 deletions(-)
arch/powerpc/kvm/book3s_hv_uvmem.c +0 -1
@@ -712,7 +712,6 @@ static struct page *kvmppc_uvmem_get_page(unsigned long gpa, struct kvm *kvm) dpage =pfn_to_page (uvmem_pfn); dpage->zone_device_data = pvt; -get_page (dpage);lock_page (dpage);return dpage; out_clear:
drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +0 -1
@@ -225,7 +225,6 @@ svm_migrate_get_vram_page(struct svm_range *prange, unsigned long pfn) page =pfn_to_page (pfn);svm_range_bo_ref (prange->svm_bo); page->zone_device_data = prange->svm_bo; -get_page (page);lock_page (page); }
drivers/gpu/drm/nouveau/nouveau_dmem.c +0 -1
@@ -326,7 +326,6 @@ nouveau_dmem_page_alloc_locked(struct nouveau_drm *drm)return NULL ; } -get_page (page);lock_page (page);return page; }
fs/Kconfig +0 -1
@@ -48,7 +48,6 @@ config FS_DAX bool "File system based Direct Access (DAX) support" depends on MMU depends on !(ARM || MIPS || SPARC) - select DEV_PAGEMAP_OPS if (ZONE_DEVICE && !FS_DAX_LIMITED) select FS_IOMAP select DAX help
include/linux/memremap.h +5 -7
@@ -68,9 +68,9 @@ enum memory_type {struct dev_pagemap_ops {/* -* Called once the page refcount reaches 1. (ZONE_DEVICE pages never -* reach 0 refcount unless there is a refcount bug. This allows the -* device driver to implement its own memory management.) +* Called once the page refcount reaches 0. The reference count will be +* reset to one by the core code after the method is called to prepare +* for handing out the page again. */ void (*page_free)(struct page *page);
@@ -133,16 +133,14 @@ static inline unsigned long pgmap_vmemmap_nr(struct dev_pagemap *pgmap)static inline bool is_device_private_page (const struct page *page) { -return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && -IS_ENABLED (CONFIG_DEVICE_PRIVATE ) && +return IS_ENABLED (CONFIG_DEVICE_PRIVATE ) &&is_zone_device_page (page) && page->pgmap->type ==MEMORY_DEVICE_PRIVATE ; }static inline bool is_pci_p2pdma_page (const struct page *page) { -return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && -IS_ENABLED (CONFIG_PCI_P2PDMA ) && +return IS_ENABLED (CONFIG_PCI_P2PDMA ) &&is_zone_device_page (page) && page->pgmap->type ==MEMORY_DEVICE_PCI_P2PDMA ; }
include/linux/mm.h +3 -3
@@ -1090,7 +1090,7 @@ static inline bool is_zone_movable_page(const struct page *page)return page_zonenum (page) ==ZONE_MOVABLE ; } -#ifdef CONFIG_DEV_PAGEMAP_OPS +#if defined(CONFIG_ZONE_DEVICE ) && defined(CONFIG_FS_DAX )DECLARE_STATIC_KEY_FALSE (devmap_managed_key);bool __put_devmap_managed_page (struct page *page);
@@ -1103,12 +1103,12 @@ static inline bool put_devmap_managed_page(struct page *page) return __put_devmap_managed_page(page); } -#else /* CONFIG_DEV_PAGEMAP_OPS */ +#else /* CONFIG_ZONE_DEVICE && CONFIG_FS_DAX */ static inline bool put_devmap_managed_page (struct page *page) {return false; } -#endif /* CONFIG_DEV_PAGEMAP_OPS */ +#endif /* CONFIG_ZONE_DEVICE && CONFIG_FS_DAX */ /* 127: arbitrary random number, small enough to assemble well */ #define folio_ref_zero_or_close_to_overflow (folio) \
lib/test_hmm.c +0 -1
@@ -566,7 +566,6 @@ static struct page *dmirror_devmem_alloc_page(struct dmirror_device *mdevice) } dpage->zone_device_data = rpage; -get_page (dpage);lock_page (dpage);return dpage;
mm/Kconfig +0 -4
@@ -776,9 +776,6 @@ config ZONE_DEVICE If FS_DAX is enabled, then say Y. -config DEV_PAGEMAP_OPS - bool - # # Helpers to mirror range of the CPU page tables of a process into device page # tables.
@@ -790,7 +787,6 @@ config HMM_MIRROR config DEVICE_PRIVATE bool "Unaddressable device memory (GPU memory, ...)" depends on ZONE_DEVICE - select DEV_PAGEMAP_OPS help Allows creation of struct pages to represent unaddressable device
mm/internal.h +2 -0
@@ -718,4 +718,6 @@ void vunmap_range_noflush(unsigned long start, unsigned long end);int numa_migrate_prep (struct page *page,struct vm_area_struct *vma,unsigned long addr,int page_nid,int *flags); +void free_zone_device_page (struct page *page); +#endif /* __MM_INTERNAL_H */
mm/memcontrol.c +3 -8
@@ -5503,17 +5503,12 @@ static struct page *mc_handle_swap_pte(struct vm_area_struct *vma,return NULL ;/* -* Handle MEMORY_DEVICE_PRIVATE which are ZONE_DEVICE page belonging to -* a device and because they are not accessible by CPU they are store -* as special swap entry in the CPU page table. +* Handle device private pages that are not accessible by the CPU, but +* stored as special swap entries in the page table. */ if (is_device_private_entry (ent)) { page =pfn_swap_entry_to_page (ent); -/* -* MEMORY_DEVICE_PRIVATE means ZONE_DEVICE page and which have -* a refcount of 1 when free (unlike normal page) -*/ -if (!page_ref_add_unless (page,1 ,1 )) +if (!get_page_unless_zero (page))return NULL ;return page; }
mm/memremap.c +19 -38
131 lines not on this page · raw message
mm/migrate.c +0 -6
@@ -341,14 +341,8 @@ static int expected_page_refs(struct address_space *mapping, struct page *page) {int expected_count =1 ; -/* -* Device private pages have an extra refcount as they are -* ZONE_DEVICE pages. -*/ - expected_count +=is_device_private_page (page);if (mapping) expected_count +=compound_nr (page) +page_has_private (page); -return expected_count; }
mm/swap.c +4 -12
@@ -114,17 +114,9 @@ static void __put_compound_page(struct page *page)void __put_page (struct page *page) { -if (is_zone_device_page (page)) { -put_dev_pagemap (page->pgmap); - -/* -* The page belongs to the device that created pgmap. Do -* not return it to page allocator. -*/ -return ; - } - -if (unlikely (PageCompound (page))) +if (unlikely (is_zone_device_page (page))) +free_zone_device_page (page); +else if (unlikely (PageCompound (page)))__put_compound_page (page);else __put_single_page (page);
@@ -933,7 +925,7 @@ void release_pages(struct page **pages, int nr)if (put_devmap_managed_page (page))continue ;if (put_page_testzero (page)) -put_dev_pagemap (page->pgmap); +free_zone_device_page (page);continue ; }
Signature · 1 line
2.30.2
Re: [PATCH 7/8] mm: remove the extra ZONE_DEVICE struct page refcount
Quoted 16 lines · Christoph Hellwig
ZONE_DEVICEstruct pages have an extra reference count that complicates the code forput_page()and several places in the kernel that need to check the reference count to see that a page is not being used (gup, compaction, migration, etc.). Clean up the code so the reference count doesn't need to be treated specially forZONE_DEVICEpages.Note that this excludes the special idle page wakeup for fsdax pages, which still happens at refcount 1. This is a separate issue and will be sorted out later. Given that only fsdax pages require the notifiacation when the refcount hits 1 now, the
PAGEMAP_OPSKconfigsymbol can go away and be replaced with aFS_DAXcheck for this hook in theput_pagefastpath.Based on an earlier patch from Ralph Campbell <[email protected]>.
Signed-off-by Christoph Hellwig <[email protected]>
Quoted 15 lines
--- arch/powerpc/kvm/book3s_hv_uvmem.c | 1 - drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 1 - drivers/gpu/drm/nouveau/nouveau_dmem.c | 1 - fs/Kconfig | 1 - include/linux/memremap.h | 12 +++-- include/linux/mm.h | 6 +-- lib/test_hmm.c | 1 - mm/Kconfig | 4 -- mm/internal.h | 2 + mm/memcontrol.c | 11 ++--- mm/memremap.c | 57 ++++++++---------------- mm/migrate.c | 6 --- mm/swap.c | 16 ++----- 13 files changed, 36 insertions(+), 83 deletions(-)
It looks like a good next step to me
include/linux/memremap.h enum memory_type { · line 73
Quoted 8 lines
struct
dev_pagemap_ops{include/linux/memremap.h +3
-3/* -* Called once the page refcount reaches 1. (ZONE_DEVICE pages never -* reach 0 refcount unless there is a refcount bug. This allows the -* device driver to implement its own memory management.) +* Called once the page refcount reaches 0. The reference count will be +* reset to one by the core code after the method is called to prepare +* for handing out the page again.
I did prefer Ralph's version of this that kept the refcount at 0 while the page was on the free-list. I hope we can get there again after later series :)
Jason
Re: [PATCH 7/8] mm: remove the extra ZONE_DEVICE struct page refcount
In reply to Christoph HellwigQuoted 13 lines · Christoph Hellwig
ZONE_DEVICEstruct pages have an extra reference count that complicates the code forput_page()and several places in the kernel that need to check the reference count to see that a page is not being used (gup, compaction, migration, etc.). Clean up the code so the reference count doesn't need to be treated specially forZONE_DEVICEpages.Note that this excludes the special idle page wakeup for fsdax pages, which still happens at refcount 1. This is a separate issue and will be sorted out later. Given that only fsdax pages require the notifiacation when the refcount hits 1 now, the
PAGEMAP_OPSKconfigsymbol can go away and be replaced with aFS_DAXcheck for this hook in theput_pagefastpath.
Quoted 3 lines
Based on an earlier patch from Ralph Campbell <[email protected]>.
Signed-off-by Christoph Hellwig <[email protected]>
Thanks for working on this, definite step forward.
Re: [PATCH 7/8] mm: remove the extra ZONE_DEVICE struct page refcount
In reply to Christoph HellwigOn Sun, Feb 6, 2022 at 10:33 PM Christoph Hellwig <[email protected]> wrote:
[..]
mm/memremap.c void free_devmap_managed_page(struct page *page) · line 489
Quoted 9 lines
mm/memremap.c +5
-0@@ -500,28 +482,27 @@ void free_devmap_managed_page(struct page *page)*/ page->mapping =NULL ; page->pgmap->ops->page_free (page); + +/* +* Reset the page count to 1 to prepare for handing out the page again. +*/ +set_page_count (page,1 );
Interesting. I had expected that to really fix the refcount problem that fs/dax.c would need to start taking real page references as pages were added to a mapping, just like page cache.
This looks ok to me, and passes my tests. So given I'm still working my way back to fixing the references properly I'm ok for this hack to replace the more broken hack that is there presently.
Christoph Hellwig
maintainerRe: [PATCH 7/8] mm: remove the extra ZONE_DEVICE struct page refcount
Quoted 3 lines · Dan Williams
Interesting. I had expected that to really fix the refcount problem that
fs/dax.cwould need to start taking real page references as pages were added to a mapping, just like page cache.
I think we should do that eventually. But I think this series that just attacks the device private type and extends to the device coherent and p2p enhacements is a good first step to stop the proliferation of the one off refcount and to allow to deal with the fsdax pages in another more focuessed series.
Re: [PATCH 7/8] mm: remove the extra ZONE_DEVICE struct page refcount
Quoted 10 lines · Christoph Hellwig
Quoted 3 lines · Dan Williams
Interesting. I had expected that to really fix the refcount problem that
fs/dax.cwould need to start taking real page references as pages were added to a mapping, just like page cache.I think we should do that eventually. But I think this series that just attacks the device private type and extends to the device coherent and p2p enhacements is a good first step to stop the proliferation of the one off refcount and to allow to deal with the fsdax pages in another more focuessed series.
It is nice, but the other series are still impacted by the fsdax mess
- they still stuff pages into ptes without proper refcounts and have to carry nonsense to dance around this problem.
I certainly would be unhappy if the amd driver, for instance, gained the fsdax problem as well and started pushing 4k pages into PMDs.
Jason
Christoph Hellwig
maintainerRe: [PATCH 7/8] mm: remove the extra ZONE_DEVICE struct page refcount
Quoted 6 lines · Jason Gunthorpe
It is nice, but the other series are still impacted by the fsdax mess
- they still stuff pages into ptes without proper refcounts and have to carry nonsense to dance around this problem.I certainly would be unhappy if the amd driver, for instance, gained the fsdax problem as well and started pushing 4k pages into PMDs.
As said before: I think this all needs to be fixed. But I'd rather fix it gradually and I think this series is a nice step forward. After that we can look at the pte mappings.
Re: [PATCH 7/8] mm: remove the extra ZONE_DEVICE struct page refcount
Quoted 11 lines · Christoph Hellwig
Quoted 6 lines · Jason Gunthorpe
It is nice, but the other series are still impacted by the fsdax mess
- they still stuff pages into ptes without proper refcounts and have to carry nonsense to dance around this problem.I certainly would be unhappy if the amd driver, for instance, gained the fsdax problem as well and started pushing 4k pages into PMDs.
As said before: I think this all needs to be fixed. But I'd rather fix it gradually and I think this series is a nice step forward. After that we can look at the pte mappings.
Right, I agree with this
Jason
Christoph Hellwig
maintainerPATCH 8/8 fsdax: depend on ZONE_DEVICE || FS_DAX_LIMITED
In reply to Christoph HellwigAdd a depends on ZONE_DEVICE support or the s390-specific limited DAX support, as one of the two is required at runtime for fsdax code to actually work.
--- fs/Kconfig | 1 + 1 file changed, 1 insertion(+)
fs/Kconfig +1 -0
@@ -48,6 +48,7 @@ config FS_DAX bool "File system based Direct Access (DAX) support" depends on MMU depends on !(ARM || MIPS || SPARC) + depends on ZONE_DEVICE || FS_DAX_LIMITED select FS_IOMAP select DAX help
Signature · 1 line
2.30.2
Re: [PATCH 8/8] fsdax: depend on ZONE_DEVICE || FS_DAX_LIMITED
Quoted 8 lines · Christoph Hellwig
Add a depends on
ZONE_DEVICEsupport or the s390-specific limited DAX support, as one of the two is required at runtime for fsdax code to actually work.Signed-off-by Christoph Hellwig <[email protected]>--- fs/Kconfig | 1 + 1 file changed, 1 insertion(+)
Makes sense, but leaves me wonder why a kconfig randomizer didn't hit this.. Or maybe it means some of the function stubs on !ZONE_DEVICE are unnecessary now..
Jason
Logan Gunthorpe
Reviewed-byQuoted 27 lines · Christoph Hellwig
Hi all,
this series removes the offset by one refcount for
ZONE_DEVICEpages that are freed back to the driver owning them, which is just device private ones for now, but also the planned device coherent pages and the ehanced p2p ones pending.It does not address the fsdax pages yet, which will be attacked in a follow on series.
Diffstat:
arch/arm64/mm/mmu.c | 1 arch/powerpc/kvm/book3s_hv_uvmem.c | 1 drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 2 drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 drivers/gpu/drm/drm_cache.c | 2 drivers/gpu/drm/nouveau/nouveau_dmem.c | 3 - drivers/gpu/drm/nouveau/nouveau_svm.c | 1 drivers/infiniband/core/rw.c | 1 drivers/nvdimm/pmem.h | 1 drivers/nvme/host/pci.c | 1 drivers/nvme/target/io-cmd-bdev.c | 1 fs/Kconfig | 2 fs/fuse/virtio_fs.c | 1 include/linux/hmm.h | 9 ---- include/linux/memremap.h | 22 +++++++++- include/linux/mm.h | 59 ++++-------------------------
Quoted 8 lines
lib/test_hmm.c | 4 + mm/Kconfig | 4 - mm/internal.h | 2 mm/memcontrol.c | 11 +---- mm/memremap.c | 63 ++++++++++++++++--------------- mm/migrate.c | 6 -- mm/swap.c | 49 ++---------------------- 23 files changed, 90 insertions(+), 157 deletions(-)
Looks good to me. I was wondering about the location of some of this code, so it's nice to see it cleaned up. Except for the one minor issue I noted on patch 6, it all looks good to me. I've reviewed all the patches and tested the series under my p2pdma series.
Logan
Quoted 7 lines · Christoph Hellwig
Hi all,
this series removes the offset by one refcount for
ZONE_DEVICEpages that are freed back to the driver owning them, which is just device private ones for now, but also the planned device coherent pages and the ehanced p2p ones pending.
Many thanks for doing this, I remember the hard time when I read the relevant code. :)
Quoted 31 lines
It does not address the fsdax pages yet, which will be attacked in a follow on series.
Diffstat:
arch/arm64/mm/mmu.c | 1 arch/powerpc/kvm/book3s_hv_uvmem.c | 1 drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 2 drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 1 drivers/gpu/drm/drm_cache.c | 2 drivers/gpu/drm/nouveau/nouveau_dmem.c | 3 - drivers/gpu/drm/nouveau/nouveau_svm.c | 1 drivers/infiniband/core/rw.c | 1 drivers/nvdimm/pmem.h | 1 drivers/nvme/host/pci.c | 1 drivers/nvme/target/io-cmd-bdev.c | 1 fs/Kconfig | 2 fs/fuse/virtio_fs.c | 1 include/linux/hmm.h | 9 ---- include/linux/memremap.h | 22 +++++++++- include/linux/mm.h | 59 ++++------------------------- lib/test_hmm.c | 4 + mm/Kconfig | 4 - mm/internal.h | 2 mm/memcontrol.c | 11 +---- mm/memremap.c | 63 ++++++++++++++++--------------- mm/migrate.c | 6 -- mm/swap.c | 49 ++---------------------- 23 files changed, 90 insertions(+), 157 deletions(-).