add a not device managed memremap_pages v3
Subsystems (4)
Hi Dan and Jason,
Bharata has been working on secure page management for kvmppc guests, and one I thing I noticed is that he had to fake up a struct device just so that it could be passed to the devm_memremap_pages instrastructure for device private memory.
This series adds non-device managed versions of the devm_request_free_mem_region and devm_memremap_pages functions for his use case.
Changes since v2:
- improved changelogs that the the v2 changes into account
Changes since v1:
- don't overload devm_request_free_mem_region
- export the memremap_pages and munmap_pages as kvmppc can be a module
PATCH 1/4 resource: add a not device managed request_free_mem_region variant
Factor out the guts of devm_request_free_mem_region so that we can implement both a device managed and a manually release version as tiny wrappers around it.
--- include/linux/ioport.h | 2 ++ kernel/resource.c | 45 +++++++++++++++++++++++++++++------------- 2 files changed, 33 insertions(+), 14 deletions(-)
include/linux/ioport.h +2 -0
@@ -297,6 +297,8 @@ static inline bool resource_overlaps(struct resource *r1, struct resource *r2)struct resource *devm_request_free_mem_region (struct device *dev,struct resource *base,unsigned long size); +struct resource *request_free_mem_region (struct resource *base, +unsigned long size,const char *name);#endif /* __ASSEMBLY__ */ #endif /* _LINUX_IOPORT_H */
kernel/resource.c +31 -14
64 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH 1/4] resource: add a not device managed request_free_mem_region variant
Quoted 72 lines · Christoph Hellwig
Factor out the guts of
devm_request_free_mem_regionso that we can implement both a device managed and a manually release version as tiny wrappers around it.Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Ira Weiny <[email protected]>--- include/linux/ioport.h | 2 ++ kernel/resource.c | 45 +++++++++++++++++++++++++++++------------- 2 files changed, 33 insertions(+), 14 deletions(-)include/linux/ioport.h +2
-0@@ -297,6 +297,8 @@ static inline bool resource_overlaps(struct resource *r1, struct resource *r2)struct resource *devm_request_free_mem_region (struct device *dev,struct resource *base,unsigned long size); +struct resource *request_free_mem_region (struct resource *base, +unsigned long size,const char *name);#endif /* __ASSEMBLY__ */ #endif /* _LINUX_IOPORT_H */ kernel/resource.c +11
-14@@ -1644,19 +1644,8 @@ void resource_list_free(struct list_head *head)EXPORT_SYMBOL (resource_list_free);#ifdef CONFIG_DEVICE_PRIVATE -/** -* devm_request_free_mem_region - find free region for device private memory -* -* @dev: device struct to bind the resource to -* @size: size in bytes of the device memory to add -* @base: resource tree to look in -* -* This function tries to find an empty range of physical address big enough to -* contain the new resource, so that it can later be hotplugged as ZONE_DEVICE -* memory, which in turn allocates struct pages. -*/ -struct resource *devm_request_free_mem_region (struct device *dev, -struct resource *base,unsigned long size) +static struct resource *__request_free_mem_region (struct device *dev, +struct resource *base,unsigned long size,const char *name) {resource_size_t end, addr;struct resource *res;@@ -1670,7 +1659,10 @@ struct resource *devm_request_free_mem_region(struct device *dev, REGION_DISJOINT)continue ; - res =devm_request_mem_region (dev, addr, size,dev_name (dev)); +if (dev) + res =devm_request_mem_region (dev, addr, size, name); +else + res =request_mem_region (addr, size, name);if (!res)return ERR_PTR (-ENOMEM ); res->desc =IORES_DESC_DEVICE_PRIVATE_MEMORY ;@@ -1679,7 +1671,32 @@ struct resource *devm_request_free_mem_region(struct device *dev,return ERR_PTR (-ERANGE ); } + +/** +* devm_request_free_mem_region - find free region for device private memory +* +* @dev: device struct to bind the resource to
kernel/resource.c struct resource *devm_request_free_mem_region(struct device *dev, · line 1688
Quoted 10 lines
kernel/resource.c +10
-0+* @size: size in bytes of the device memory to add +* @base: resource tree to look in +* +* This function tries to find an empty range of physical address big enough to +* contain the new resource, so that it can later be hotplugged as ZONE_DEVICE +* memory, which in turn allocates struct pages. +*/ +struct resource *devm_request_free_mem_region (struct device *dev, +struct resource *base,unsigned long size) +{
Previously we would loudly crash if someone passed NULL to devm_request_free_mem_region(), but now it will silently work and the result will leak. Perhaps this wants a:
if (!dev)
return NULL;
...to head off those mistakes?
No major heartburn if you keep it as is, you can add:
Re: [PATCH 1/4] resource: add a not device managed request_free_mem_region variant
Quoted 4 lines · Dan Williams
Previously we would loudly crash if someone passed NULL to
devm_request_free_mem_region(), but now it will silently work and the result will leak. Perhaps this wants a:
We'd still instantly crash due to the dev_name dereference, right?
Re: [PATCH 1/4] resource: add a not device managed request_free_mem_region variant
Quoted 8 lines · Christoph Hellwig
Quoted 4 lines · Dan Williams
Previously we would loudly crash if someone passed NULL to
devm_request_free_mem_region(), but now it will silently work and the result will leak. Perhaps this wants a:We'd still instantly crash due to the
dev_namedereference, right?
Whoops, yes.
PATCH 2/4 memremap: remove the dev field in struct dev_pagemap
In reply to Christoph HellwigThe dev field in struct dev_pagemap is only used to print dev_name in two places, which are at best nice to have. Just remove the field and thus the name in those two messages.
--- include/linux/memremap.h | 1 - kernel/memremap.c | 6 +----- mm/page_alloc.c | 2 +- 3 files changed, 2 insertions(+), 7 deletions(-)
include/linux/memremap.h +0 -1
@@ -109,7 +109,6 @@ struct dev_pagemap {struct percpu_ref *ref;struct percpu_ref internal_ref;struct completion done; -struct device *dev;enum memory_type type;unsigned int flags;u64 pci_p2pdma_bus_offset;
kernel/memremap.c +1 -5
@@ -96,7 +96,6 @@ static void dev_pagemap_cleanup(struct dev_pagemap *pgmap)static void devm_memremap_pages_release (void *data) {struct dev_pagemap *pgmap = data; -struct device *dev = pgmap->dev;struct resource *res = &pgmap->res;unsigned long pfn;int nid;
@@ -123,8 +122,7 @@ static void devm_memremap_pages_release(void *data)untrack_pfn (NULL ,PHYS_PFN (res->start),resource_size (res));pgmap_array_delete (res); -dev_WARN_ONCE (dev, pgmap->altmap.alloc, -"%s: failed to free all reserved pages\n" , __func__); +WARN_ONCE (pgmap->altmap.alloc,"failed to free all reserved pages\n" ); }static void dev_pagemap_percpu_release (struct percpu_ref *ref)
@@ -245,8 +243,6 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap) goto err_array; } - pgmap->dev = dev; - error =xa_err (xa_store_range (&pgmap_array,PHYS_PFN (res->start),PHYS_PFN (res->end), pgmap,GFP_KERNEL ));if (error)
mm/page_alloc.c +1 -1
@@ -5982,7 +5982,7 @@ void __ref memmap_init_zone_device(struct zone *zone, } } -pr_info ("%s initialised, %lu pages in %ums\n" ,dev_name (pgmap->dev), +pr_info ("%s initialised %lu pages in %ums\n" , __func__, size,jiffies_to_msecs (jiffies - start)); }
Signature · 1 line
2.20.1
Re: [PATCH 2/4] memremap: remove the dev field in struct dev_pagemap
Quoted 7 lines · Christoph Hellwig
The dev field in struct
dev_pagemapis only used to printdev_namein two places, which are at best nice to have. Just remove the field and thus the name in those two messages.Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Ira Weiny <[email protected]>
Needs the below as well.
/me goes to check if he ever merged the fix to make the unit test stuff get built by default with COMPILE_TEST [1]. Argh! Nope, didn't submit it for 5.3-rc1, sorry for the thrash.
You can otherwise add:
---
(quoted diff) +0 -0
b/tools/testing/nvdimm/test/iomap.c
index cd040b5abffe..3f55f2f99112 100644
tools/testing/nvdimm/test/iomap.c +0 -0
@@ -132,7 +132,6 @@ void *__wrap_devm_memremap_pages(struct device
*dev, struct dev_pagemap *pgmap)
if (!nfit_res)
return devm_memremap_pages(dev, pgmap);- pgmap->dev = dev;
if (!pgmap->ref) {
if (pgmap->ops && (pgmap->ops->kill || pgmap->ops->cleanup))
return ERR_PTR(-EINVAL);Re: [PATCH 2/4] memremap: remove the dev field in struct dev_pagemap
Quoted 15 lines · Dan Williams
Quoted 7 lines · Christoph Hellwig
The dev field in struct
dev_pagemapis only used to printdev_namein two places, which are at best nice to have. Just remove the field and thus the name in those two messages.Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Ira Weiny <[email protected]>Needs the below as well.
/me goes to check if he ever merged the fix to make the unit test stuff get built by default with
COMPILE_TEST[1]. Argh! Nope, didn't submit it for 5.3-rc1, sorry for the thrash.
Quoted 5 lines
You can otherwise add:
Reviewed-by Dan Williams <[email protected]>
Can you get this merged? Do you want it to go with this series?
Jason
Re: [PATCH 2/4] memremap: remove the dev field in struct dev_pagemap
Quoted 17 lines · Jason Gunthorpe
Quoted 15 lines · Dan Williams
Quoted 7 lines · Christoph Hellwig
The dev field in struct
dev_pagemapis only used to printdev_namein two places, which are at best nice to have. Just remove the field and thus the name in those two messages.Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Ira Weiny <[email protected]>Needs the below as well.
/me goes to check if he ever merged the fix to make the unit test stuff get built by default with
COMPILE_TEST[1]. Argh! Nope, didn't submit it for 5.3-rc1, sorry for the thrash.
Quoted 7 lines
Quoted 5 lines
You can otherwise add:
Reviewed-by Dan Williams <[email protected]>Can you get this merged? Do you want it to go with this series?
Yeah, makes some sense to let you merge it so that you can get kbuild-robot reports about any follow-on memremap_pages() work that may trip up the build. Otherwise let me know and I'll get it queued with the other v5.4 libnvdimm pending bits.
Re: [PATCH 2/4] memremap: remove the dev field in struct dev_pagemap
Quoted 22 lines · Dan Williams
Quoted 21 lines · Jason Gunthorpe
Quoted 19 lines · Dan Williams
Quoted 7 lines · Christoph Hellwig
The dev field in struct
dev_pagemapis only used to printdev_namein two places, which are at best nice to have. Just remove the field and thus the name in those two messages.Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Ira Weiny <[email protected]>Needs the below as well.
/me goes to check if he ever merged the fix to make the unit test stuff get built by default with
COMPILE_TEST[1]. Argh! Nope, didn't submit it for 5.3-rc1, sorry for the thrash.You can otherwise add:
Reviewed-by Dan Williams <[email protected]>
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Can you get this merged? Do you want it to go with this series?
Yeah, makes some sense to let you merge it so that you can get kbuild-robot reports about any follow-on
memremap_pages()work that may trip up the build. Otherwise let me know and I'll get it queued with the other v5.4 libnvdimm pending bits.
Done, I used it already to test build the last series from CH..
Jason
Re: [PATCH 2/4] memremap: remove the dev field in struct dev_pagemap
Quoted 25 lines · Jason Gunthorpe
Quoted 24 lines · Dan Williams
Quoted 23 lines · Jason Gunthorpe
Quoted 20 lines · Dan Williams
Quoted 7 lines · Christoph Hellwig
The dev field in struct
dev_pagemapis only used to printdev_namein two places, which are at best nice to have. Just remove the field and thus the name in those two messages.Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Ira Weiny <[email protected]>Needs the below as well.
/me goes to check if he ever merged the fix to make the unit test stuff get built by default with
COMPILE_TEST[1]. Argh! Nope, didn't submit it for 5.3-rc1, sorry for the thrash.You can otherwise add:
Reviewed-by Dan Williams <[email protected]>
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Can you get this merged? Do you want it to go with this series?
Yeah, makes some sense to let you merge it so that you can get kbuild-robot reports about any follow-on
memremap_pages()work that may trip up the build. Otherwise let me know and I'll get it queued with the other v5.4 libnvdimm pending bits.Done, I used it already to test build the last series from CH..
It failed 0-day, I'm guessing some missing kconfig stuff
For now I dropped it, but, if you send a v2 I can forward it toward 0-day again!
Thanks,
Jason
Re: [PATCH 2/4] memremap: remove the dev field in struct dev_pagemap
Quoted 34 lines · Jason Gunthorpe
Quoted 32 lines · Jason Gunthorpe
Quoted 30 lines · Dan Williams
Quoted 17 lines · Jason Gunthorpe
Quoted 15 lines · Dan Williams
Quoted 7 lines · Christoph Hellwig
The dev field in struct
dev_pagemapis only used to printdev_namein two places, which are at best nice to have. Just remove the field and thus the name in those two messages.Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Ira Weiny <[email protected]>Needs the below as well.
/me goes to check if he ever merged the fix to make the unit test stuff get built by default with
COMPILE_TEST[1]. Argh! Nope, didn't submit it for 5.3-rc1, sorry for the thrash.Quoted 7 lines
Quoted 5 lines
You can otherwise add:
Reviewed-by Dan Williams <[email protected]>Can you get this merged? Do you want it to go with this series?
Yeah, makes some sense to let you merge it so that you can get kbuild-robot reports about any follow-on
memremap_pages()work that may trip up the build. Otherwise let me know and I'll get it queued with the other v5.4 libnvdimm pending bits.
Quoted 6 lines
Quoted 1 line
Done, I used it already to test build the last series from CH..
It failed 0-day, I'm guessing some missing kconfig stuff
For now I dropped it, but, if you send a v2 I can forward it toward 0-day again!
The system works!
Sorry for that thrash, I'll track it down.
PATCH 3/4 memremap: don't use a separate devm action for devmap_managed_enable_get
In reply to Christoph HellwigJust clean up for early failures and then piggy back on devm_memremap_pages_release. This helps with a pending not device managed version of devm_memremap_pages.
--- kernel/memremap.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
kernel/memremap.c +10 -5
@@ -21,13 +21,13 @@ DEFINE_STATIC_KEY_FALSE(devmap_managed_key);EXPORT_SYMBOL (devmap_managed_key);static atomic_t devmap_managed_enable; -static void devmap_managed_enable_put (void *data) +static void devmap_managed_enable_put (void ) {if (atomic_dec_and_test (&devmap_managed_enable))static_branch_disable (&devmap_managed_key); } -static int devmap_managed_enable_get (struct device *dev,struct dev_pagemap *pgmap) +static int devmap_managed_enable_get (struct dev_pagemap *pgmap) {if (!pgmap->ops || !pgmap->ops->page_free) {WARN (1 ,"Missing page_free method\n" );
@@ -36,13 +36,16 @@ static int devmap_managed_enable_get(struct device *dev, struct dev_pagemap *pgmif (atomic_inc_return (&devmap_managed_enable) ==1 )static_branch_enable (&devmap_managed_key); - return devm_add_action_or_reset(dev, devmap_managed_enable_put, NULL); + return 0; }#else -static int devmap_managed_enable_get (struct device *dev,struct dev_pagemap *pgmap) +static int devmap_managed_enable_get (struct dev_pagemap *pgmap) {return -EINVAL ; } +static void devmap_managed_enable_put (void ) +{ +}#endif /* CONFIG_DEV_PAGEMAP_OPS */ static void pgmap_array_delete (struct resource *res)
@@ -123,6 +126,7 @@ static void devm_memremap_pages_release(void *data)untrack_pfn (NULL ,PHYS_PFN (res->start),resource_size (res));pgmap_array_delete (res);WARN_ONCE (pgmap->altmap.alloc,"failed to free all reserved pages\n" ); +devmap_managed_enable_put (); }static void dev_pagemap_percpu_release (struct percpu_ref *ref)
@@ -212,7 +216,7 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap) }if (need_devmap_managed) { - error =devmap_managed_enable_get (dev, pgmap); + error =devmap_managed_enable_get (pgmap);if (error)return ERR_PTR (error); }
@@ -321,6 +325,7 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap) err_array:dev_pagemap_kill (pgmap);dev_pagemap_cleanup (pgmap); +devmap_managed_enable_put ();return ERR_PTR (error); }EXPORT_SYMBOL_GPL (devm_memremap_pages);
Signature · 1 line
2.20.1
Re: [PATCH 3/4] memremap: don't use a separate devm action for devmap_managed_enable_get
Quoted 7 lines · Christoph Hellwig
Just clean up for early failures and then piggy back on
devm_memremap_pages_release. This helps with a pending not device managed version ofdevm_memremap_pages.Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Ira Weiny <[email protected]>
Looks good,
PATCH 4/4 memremap: provide a not device managed memremap_pages
In reply to Christoph HellwigThe kvmppc ultravisor code wants a device private memory pool that is system wide and not attached to a device. Instead of faking up one provide a low-level memremap_pages for it.
--- include/linux/memremap.h | 2 + kernel/memremap.c | 84 +++++++++++++++++++++++++--------------- 2 files changed, 54 insertions(+), 32 deletions(-)
include/linux/memremap.h +2 -0
@@ -123,6 +123,8 @@ static inline struct vmem_altmap *pgmap_altmap(struct dev_pagemap *pgmap) }#ifdef CONFIG_ZONE_DEVICE +void *memremap_pages (struct dev_pagemap *pgmap,int nid); +void memunmap_pages (struct dev_pagemap *pgmap);void *devm_memremap_pages (struct device *dev,struct dev_pagemap *pgmap);void devm_memunmap_pages (struct device *dev,struct dev_pagemap *pgmap); struct dev_pagemap *get_dev_pagemap(unsigned long pfn,
kernel/memremap.c +52 -32
140 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH 4/4] memremap: provide a not device managed memremap_pages
Quoted 7 lines · Christoph Hellwig
The kvmppc ultravisor code wants a device private memory pool that is system wide and not attached to a device. Instead of faking up one provide a low-level
memremap_pagesfor it.Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Ira Weiny <[email protected]>
Looks good,
Bharata B Rao
Tested-byQuoted 10 lines · Christoph Hellwig
Hi Dan and Jason,
Bharata has been working on secure page management for kvmppc guests, and one I thing I noticed is that he had to fake up a struct device just so that it could be passed to the
devm_memremap_pagesinstrastructure for device private memory.This series adds non-device managed versions of the
devm_request_free_mem_regionanddevm_memremap_pagesfunctions for his use case.
Tested kvmppc ultravisor patchset with migrate_vma changes and this patchset. (Had to manually patch mm/memremap.c instead of kernel/memremap.c though)
For the series,
Quoted 8 lines · Bharata B Rao
Quoted 7 lines · Christoph Hellwig
Hi Dan and Jason,
Bharata has been working on secure page management for kvmppc guests, and one I thing I noticed is that he had to fake up a struct device just so that it could be passed to the
devm_memremap_pagesinstrastructure for device private memory.
Quoted 7 lines
Quoted 3 lines
This series adds non-device managed versions of the
devm_request_free_mem_regionanddevm_memremap_pagesfunctions for his use case.Tested kvmppc ultravisor patchset with
migrate_vmachanges and this patchset. (Had to manually patchmm/memremap.cinstead ofkernel/memremap.cthough)
Oh. I rebased to the hmm tree, and that didn't have the rename yet. And I didn't even notice that as git handled it transparently.
Jason Gunthorpe
AppliedQuoted 11 lines · Christoph Hellwig
Hi Dan and Jason,
Bharata has been working on secure page management for kvmppc guests, and one I thing I noticed is that he had to fake up a struct device just so that it could be passed to the
devm_memremap_pagesinstrastructure for device private memory.This series adds non-device managed versions of the
devm_request_free_mem_regionanddevm_memremap_pagesfunctions for his use case.
Quoted 6 lines
Changes since v2:
- improved changelogs that the the v2 changes into accountChanges since v1:
- don't overloaddevm_request_free_mem_region
- export thememremap_pagesandmunmap_pagesas kvmppc can be a module
Looks good, I fixed up the patch with Dan's note and reviewed them as well.
Applied to hmm.git as requested
Thanks,
Jason