dma-buf: heaps: Turn heaps into modules
Subsystems (6)
v4 cover dma-buf: heaps: Turn heaps into modules
Hi,
The recent introduction of heaps in the optee driver [1] made possible the creation of heaps as modules.
It's generally a good idea if possible, including for the already existing system and CMA heaps.
The system one is pretty trivial, the CMA one is a bit more involved, especially since we have a call from kernel/dma/contiguous.c to the CMA heap code. This was solved by turning the logic around and making the CMA heap call into the contiguous DMA code.
Let me know what you think,
Maxime
1: https://lore.kernel.org/dri-devel/[email protected]/
---
Changes in v4:
- Fix compilation failure
- Rework to take into account OF_RESERVED_MEM
- Fix regression making the default CMA area disappear if not created
through the DT
- Added some documentation and comments
- Link to v3: https://lore.kernel.org/r/[email protected]
Changes in v3:
- Squashed cma_get_name and cma_alloc/release patches
- Fixed typo in Export dev_get_cma_area commit title
- Fixed compilation failure with DMA_CMA but not OF_RESERVED_MEM
- Link to v2: https://lore.kernel.org/r/[email protected]
Changes in v2:
- Collect tags
- Don't export dma_contiguous_default_area anymore, but export
dev_get_cma_area instead
- Mentioned that heap modules can't be removed
- Link to v1: https://lore.kernel.org/r/[email protected]
---
Maxime Ripard (8):
dma: contiguous: Turn heap registration logic around
dma: contiguous: Make dev_get_cma_area() a proper function
dma: contiguous: Make dma_contiguous_default_area static
dma: contiguous: Export dev_get_cma_area()
mm: cma: Export cma_alloc(), cma_release() and cma_get_name()
dma-buf: heaps: Export mem_accounting parameter
dma-buf: heaps: cma: Turn the heap into a module
dma-buf: heaps: system: Turn the heap into a module
drivers/dma-buf/dma-heap.c | 1 + drivers/dma-buf/heaps/Kconfig | 4 +-- drivers/dma-buf/heaps/cma_heap.c | 22 +++---------- drivers/dma-buf/heaps/system_heap.c | 5 +++ include/linux/dma-buf/heaps/cma.h | 16 --------- include/linux/dma-map-ops.h | 14 ++++---- kernel/dma/contiguous.c | 66 +++++++++++++++++++++++++++++++++---- mm/cma.c | 3 ++ 8 files changed, 82 insertions(+), 49 deletions(-)
---
base-commit: c081b71f11732ad2c443f170ab19c3ebe8a1a422
change-id: 20260225-dma-buf-heaps-as-modules-1034b3ec9f2a
Best regards,
Signature · 1 line
Maxime Ripard <[email protected]>
PATCH v4 1/8 dma: contiguous: Turn heap registration logic around
The CMA heap instantiation was initially developed by having the contiguous DMA code call into the CMA heap to create a new instance every time a reserved memory area is probed.
Turning the CMA heap into a module would create a dependency of the kernel on a module, which doesn't work.
Let's turn the logic around and do the opposite: store all the reserved memory CMA regions into the contiguous DMA code, and provide an iterator for the heap to use when it probes.
--- drivers/dma-buf/heaps/cma_heap.c | 19 ++------------ include/linux/dma-buf/heaps/cma.h | 16 ------------ include/linux/dma-map-ops.h | 5 ++++ kernel/dma/contiguous.c | 55 +++++++++++++++++++++++++++++++++++---- 4 files changed, 57 insertions(+), 38 deletions(-)
drivers/dma-buf/heaps/cma_heap.c +2 -17
@@ -12,11 +12,10 @@#define pr_fmt (fmt) "cma_heap: " fmt#include <linux/cma.h> #include <linux/dma-buf.h> -#include <linux/dma-buf/heaps/cma.h> #include <linux/dma-heap.h> #include <linux/dma-map-ops.h> #include <linux/err.h> #include <linux/highmem.h> #include <linux/io.h>
@@ -28,23 +27,10 @@#include <linux/slab.h> #include <linux/vmalloc.h> #define DEFAULT_CMA_NAME "default_cma_region" -static struct cma *dma_areas[MAX_CMA_AREAS ]__initdata ; -static unsigned int dma_areas_num__initdata ; - -int __init dma_heap_cma_register_heap (struct cma *cma) -{ -if (dma_areas_num >=ARRAY_SIZE (dma_areas)) -return -EINVAL ; - - dma_areas[dma_areas_num++] = cma; - -return 0 ; -} -struct cma_heap {struct dma_heap *heap;struct cma *cma; };
@@ -412,22 +398,21 @@ static int __init __add_cma_heap(struct cma *cma, const char *name) }static int __init add_cma_heaps (void ) {struct cma *default_cma =dev_get_cma_area (NULL ); +struct cma *cma;unsigned int i;int ret;if (default_cma) { ret =__add_cma_heap (default_cma,DEFAULT_CMA_NAME );if (ret)return ret; } -for (i =0 ; i < dma_areas_num; i++) { -struct cma *cma = dma_areas[i]; - +for (i =0 ; (cma =dma_contiguous_get_area_by_idx (i)) !=NULL ; i++) { ret =__add_cma_heap (cma,cma_get_name (cma));if (ret) {pr_warn ("Failed to add CMA heap %s" ,cma_get_name (cma));continue ; }
include/linux/dma-buf/heaps/cma.h deleted +0 -16
@@ -1,16 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef DMA_BUF_HEAP_CMA_H_ -#define DMA_BUF_HEAP_CMA_H_ - -struct cma ; - -#ifdef CONFIG_DMABUF_HEAPS_CMA -int dma_heap_cma_register_heap (struct cma *cma); -#else -static inline int dma_heap_cma_register_heap (struct cma *cma) -{ -return 0 ; -} -#endif // CONFIG_DMABUF_HEAPS_CMA - -#endif // DMA_BUF_HEAP_CMA_H_
include/linux/dma-map-ops.h +5 -0
@@ -97,10 +97,11 @@ static inline struct cma *dev_get_cma_area(struct device *dev) {if (dev && dev->cma_area)return dev->cma_area;return dma_contiguous_default_area; } +struct cma *dma_contiguous_get_area_by_idx (unsigned int idx);void dma_contiguous_reserve (phys_addr_t addr_limit);int __init dma_contiguous_reserve_area (phys_addr_t size,phys_addr_t base,phys_addr_t limit,struct cma **res_cma,bool fixed);
@@ -115,10 +116,14 @@ void dma_contiguous_early_fixup(phys_addr_t base, unsigned long size);#else /* CONFIG_DMA_CMA */ static inline struct cma *dev_get_cma_area (struct device *dev) {return NULL ; } +static inline struct cma *dma_contiguous_get_area_by_idx (unsigned int idx) +{ +return NULL ; +}static inline void dma_contiguous_reserve (phys_addr_t limit) { } static inline int dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base, phys_addr_t limit, struct cma **res_cma,
kernel/dma/contiguous.c +50 -5
97 lines not on this page · raw message
Signature · 1 line
2.53.0
PATCH v4 2/8 dma: contiguous: Make dev_get_cma_area() a proper function
In reply to Maxime RipardAs we try to enable dma-buf heaps, and the CMA one in particular, to compile as modules, we need to export dev_get_cma_area(). It's currently implemented as an inline function that returns either the content of device->cma_area or dma_contiguous_default_area.
Thus, it means we need to export dma_contiguous_default_area, which isn't really something we want any module to have access to.
Instead, let's make dev_get_cma_area() a proper function we will be able to export so we can avoid exporting dma_contiguous_default_area.
--- include/linux/dma-map-ops.h | 7 +------ kernel/dma/contiguous.c | 8 ++++++++ 2 files changed, 9 insertions(+), 6 deletions(-)
include/linux/dma-map-ops.h +1 -6
@@ -91,16 +91,11 @@ static inline void set_dma_ops(struct device *dev,#endif /* CONFIG_ARCH_HAS_DMA_OPS */ #ifdef CONFIG_DMA_CMA extern struct cma *dma_contiguous_default_area; -static inline struct cma *dev_get_cma_area (struct device *dev) -{ -if (dev && dev->cma_area) -return dev->cma_area; -return dma_contiguous_default_area; -} +struct cma *dev_get_cma_area (struct device *dev);struct cma *dma_contiguous_get_area_by_idx (unsigned int idx);void dma_contiguous_reserve (phys_addr_t addr_limit);int __init dma_contiguous_reserve_area (phys_addr_t size,phys_addr_t base,phys_addr_t limit,struct cma **res_cma,bool fixed);
kernel/dma/contiguous.c +8 -0
@@ -129,10 +129,18 @@ early_param("cma", early_cma);bool __init cma_skip_dt_default_reserved_mem (void ) {return size_cmdline !=-1 ; } +struct cma *dev_get_cma_area (struct device *dev) +{ +if (dev && dev->cma_area) +return dev->cma_area; + +return dma_contiguous_default_area; +} +#ifdef CONFIG_DMA_NUMA_CMA static struct cma *dma_contiguous_numa_area[MAX_NUMNODES ];static phys_addr_t numa_cma_size[MAX_NUMNODES ]__initdata ;static struct cma *dma_contiguous_pernuma_area[MAX_NUMNODES ];
Signature · 1 line
2.53.0
PATCH v4 3/8 dma: contiguous: Make dma_contiguous_default_area static
In reply to Maxime RipardNow that dev_get_cma_area() is no longer inline, we don't have any user of dma_contiguous_default_area() outside of contiguous.c so we can make it static.
--- include/linux/dma-map-ops.h | 2 -- kernel/dma/contiguous.c | 2 +- 2 files changed, 1 insertion(+), 3 deletions(-)
include/linux/dma-map-ops.h +0 -2
@@ -89,12 +89,10 @@ static inline void set_dma_ops(struct device *dev, { }#endif /* CONFIG_ARCH_HAS_DMA_OPS */ #ifdef CONFIG_DMA_CMA -extern struct cma *dma_contiguous_default_area; -struct cma *dev_get_cma_area (struct device *dev);struct cma *dma_contiguous_get_area_by_idx (unsigned int idx);void dma_contiguous_reserve (phys_addr_t addr_limit); int __init dma_contiguous_reserve_area(phys_addr_t size, phys_addr_t base,
kernel/dma/contiguous.c +1 -1
@@ -81,11 +81,11 @@ struct cma *dma_contiguous_get_area_by_idx(unsigned int idx)return dma_contiguous_areas[idx]; }EXPORT_SYMBOL_GPL (dma_contiguous_get_area_by_idx); -struct cma *dma_contiguous_default_area; +static struct cma *dma_contiguous_default_area;/* * Default global CMA area size can be defined in kernel's .config. * This is useful mainly for distro maintainers to create a kernel * that works correctly for most supported systems.
Signature · 1 line
2.53.0
PATCH v4 4/8 dma: contiguous: Export dev_get_cma_area()
In reply to Maxime RipardThe CMA dma-buf heap uses the dev_get_cma_area() function to retrieve the default contiguous area.
Now that this function is no longer inlined, and since we want to turn the CMA heap into a module, let's export it.
--- kernel/dma/contiguous.c | 1 + 1 file changed, 1 insertion(+)
kernel/dma/contiguous.c +1 -0
@@ -136,10 +136,11 @@ struct cma *dev_get_cma_area(struct device *dev)if (dev && dev->cma_area)return dev->cma_area;return dma_contiguous_default_area; } +EXPORT_SYMBOL_GPL (dev_get_cma_area);#ifdef CONFIG_DMA_NUMA_CMA static struct cma *dma_contiguous_numa_area[MAX_NUMNODES ];static phys_addr_t numa_cma_size[MAX_NUMNODES ]__initdata ;
Signature · 1 line
2.53.0
PATCH v4 5/8 mm: cma: Export cma_alloc(), cma_release() and cma_get_name()
In reply to Maxime RipardThe CMA dma-buf heap uses cma_alloc() and cma_release() to allocate and free, respectively, its CMA buffers, and cma_get_name() to get the name of the heap instance it's going to create.
However, these functions are not exported. Since we want to turn the CMA heap into a module, let's export them both.
--- mm/cma.c | 3 +++ 1 file changed, 3 insertions(+)
mm/cma.c +3 -0
@@ -50,10 +50,11 @@ unsigned long cma_get_size(const struct cma *cma)const char *cma_get_name (const struct cma *cma) {return cma->name; } +EXPORT_SYMBOL_GPL (cma_get_name);static unsigned long cma_bitmap_aligned_mask (const struct cma *cma,unsigned int align_order) {if (align_order <= cma->order_per_bit)
@@ -949,10 +950,11 @@ struct page *cma_alloc(struct cma *cma, unsigned long count,if (page)set_pages_refcounted (page, count);return page; } +EXPORT_SYMBOL_GPL (cma_alloc);static struct cma_memrange *find_cma_memrange (struct cma *cma,const struct page *pages,unsigned long count) {struct cma_memrange *cmr =NULL ;
@@ -1025,10 +1027,11 @@ bool cma_release(struct cma *cma, const struct page *pages,__cma_release_frozen (cma, cmr, pages, count);return true; } +EXPORT_SYMBOL_GPL (cma_release);bool cma_release_frozen (struct cma *cma,const struct page *pages,unsigned long count) {struct cma_memrange *cmr;
Signature · 1 line
2.53.0
PATCH v4 6/8 dma-buf: heaps: Export mem_accounting parameter
In reply to Maxime RipardThe mem_accounting kernel parameter is used by heaps to know if they should account allocations in their respective cgroup controllers.
Since we're going to allow heaps to compile as modules, we need to export that variable.
--- drivers/dma-buf/dma-heap.c | 1 + 1 file changed, 1 insertion(+)
drivers/dma-buf/dma-heap.c +1 -0
@@ -51,10 +51,11 @@ static DEFINE_XARRAY_ALLOC(dma_heap_minors);bool __read_mostly mem_accounting;module_param (mem_accounting, bool,0444 );MODULE_PARM_DESC (mem_accounting,"Enable cgroup-based memory accounting for dma-buf heap allocations (default=false)." ); +EXPORT_SYMBOL_NS_GPL (mem_accounting,"DMA_BUF_HEAP" );static int dma_heap_buffer_alloc (struct dma_heap *heap,size_t len,u32 fd_flags,u64 heap_flags) {
Signature · 1 line
2.53.0
PATCH v4 7/8 dma-buf: heaps: cma: Turn the heap into a module
In reply to Maxime RipardNow that all the symbols used by the CMA heap are exported, turning the CMA heap into a module becomes pretty easy: we just need to add the usual MODULE_* macros, import the proper namespaces and change the Kconfig symbol to a tristate.
This heap won't be able to unload though, since we're missing a lot of infrastructure to make it safe.
--- drivers/dma-buf/heaps/Kconfig | 2 +- drivers/dma-buf/heaps/cma_heap.c | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-)
drivers/dma-buf/heaps/Kconfig +1 -1
@@ -4,11 +4,11 @@ config DMABUF_HEAPS_SYSTEM help Choose this option to enable the system dmabuf heap. The system heap is backed by pages from the buddy allocator. If in doubt, say Y. config DMABUF_HEAPS_CMA - bool "DMA-BUF CMA Heap" + tristate "DMA-BUF CMA Heap" depends on DMABUF_HEAPS && DMA_CMA help Choose this option to enable dma-buf CMA heap. This heap is backed by the Contiguous Memory Allocator (CMA). If your system has these regions, you should say Y here.
drivers/dma-buf/heaps/cma_heap.c +3 -0
@@ -421,5 +421,8 @@ static int __init add_cma_heaps(void)return 0 ; }module_init (add_cma_heaps);MODULE_DESCRIPTION ("DMA-BUF CMA Heap" ); +MODULE_LICENSE ("GPL" ); +MODULE_IMPORT_NS ("DMA_BUF" ); +MODULE_IMPORT_NS ("DMA_BUF_HEAP" );
Signature · 1 line
2.53.0
PATCH v4 8/8 dma-buf: heaps: system: Turn the heap into a module
In reply to Maxime RipardThe system heap can be easily turned into a module by adding the usual MODULE_* macros, importing the proper namespaces and changing the Kconfig symbol to a tristate.
This heap won't be able to unload though, since we're missing a lot of infrastructure to make it safe.
--- drivers/dma-buf/heaps/Kconfig | 2 +- drivers/dma-buf/heaps/system_heap.c | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-)
drivers/dma-buf/heaps/Kconfig +1 -1
@@ -1,7 +1,7 @@ config DMABUF_HEAPS_SYSTEM - bool "DMA-BUF System Heap" + tristate "DMA-BUF System Heap" depends on DMABUF_HEAPS help Choose this option to enable the system dmabuf heap. The system heap is backed by pages from the buddy allocator. If in doubt, say Y.
drivers/dma-buf/heaps/system_heap.c +5 -0
@@ -442,5 +442,10 @@ static int __init system_heap_create(void)return PTR_ERR (sys_heap);return 0 ; }module_init (system_heap_create); + +MODULE_DESCRIPTION ("DMA-BUF System Heap" ); +MODULE_LICENSE ("GPL" ); +MODULE_IMPORT_NS ("DMA_BUF" ); +MODULE_IMPORT_NS ("DMA_BUF_HEAP" );
Signature · 1 line
2.53.0
Marek Szyprowski
maintainerQuoted 11 lines · Maxime Ripard
The recent introduction of heaps in the optee driver [1] made possible the creation of heaps as modules.
It's generally a good idea if possible, including for the already existing system and CMA heaps.
The system one is pretty trivial, the CMA one is a bit more involved, especially since we have a call from
kernel/dma/contiguous.cto the CMA heap code. This was solved by turning the logic around and making the CMA heap call into the contiguous DMA code.
Quoted 6 lines
Let me know what you think,
Maxime1: https://lore.kernel.org/dri-devel/[email protected]/
Signed-off-by Maxime Ripard <[email protected]>
Applied again patches 1-5 to my dma-mapping-for-next branch. I hope this time it
won't cause new problems in linux-next.
Here is a stable branch to apply remaining dma-buf heaps patches:
Best regards
Signature · 2 lines
Marek Szyprowski, PhD Samsung R&D Institute Poland
Sumit Semwal
maintainerHello Maxime,
Quoted 13 lines · Marek Szyprowski
Quoted 11 lines · Maxime Ripard
The recent introduction of heaps in the optee driver [1] made possible the creation of heaps as modules.
It's generally a good idea if possible, including for the already existing system and CMA heaps.
The system one is pretty trivial, the CMA one is a bit more involved, especially since we have a call from
kernel/dma/contiguous.cto the CMA heap code. This was solved by turning the logic around and making the CMA heap call into the contiguous DMA code.
Quoted 6 lines
Quoted 6 lines
Let me know what you think,
Maxime1: https://lore.kernel.org/dri-devel/[email protected]/
Signed-off-by Maxime Ripard <[email protected]>
Thank you for this patch series; now that it is needed by more folks, I think we can merge this.
Marek, I'll coordinate with you on this - thank you!
Best,
Sumit.
Quoted 9 lines
Applied again patches 1-5 to my dma-mapping-for-next branch. I hope this time it
won't cause new problems in linux-next.
Here is a stable branch to apply remaining dma-buf heaps patches:
Quoted 9 lines
Best regards
--
Marek Szyprowski, PhD
Samsung R&D Institute Poland
--
Thanks and regards,
Sumit Semwal (he / him)
Senior Tech Lead - Platforms and Virtualisation
Linaro.org │ Arm Solutions at Light Speed
Quoted 19 lines · Sumit Semwal
Hello Maxime,
Quoted 16 lines · Marek Szyprowski
Quoted 14 lines · Maxime Ripard
The recent introduction of heaps in the optee driver [1] made possible the creation of heaps as modules.
It's generally a good idea if possible, including for the already existing system and CMA heaps.
The system one is pretty trivial, the CMA one is a bit more involved, especially since we have a call from
kernel/dma/contiguous.cto the CMA heap code. This was solved by turning the logic around and making the CMA heap call into the contiguous DMA code.Let me know what you think,
Maxime
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
1: https://lore.kernel.org/dri-devel/[email protected]/
Signed-off-by Maxime Ripard <[email protected]>Thank you for this patch series; now that it is needed by more folks, I think we can merge this.
Marek, I'll coordinate with you on this - thank you!
If Marek plans on sending it during the next merge window, maybe the best thing to do for us is just to wait for -rc1 and apply the rest of the patches. Otherwise, we can merge the branch in drm-misc.
Maxime
Sumit Semwal
maintainerHi Maxime,
Quoted 25 lines · Maxime Ripard
Quoted 23 lines · Sumit Semwal
Hello Maxime,
Quoted 19 lines · Marek Szyprowski
Quoted 17 lines · Maxime Ripard
The recent introduction of heaps in the optee driver [1] made possible the creation of heaps as modules.
It's generally a good idea if possible, including for the already existing system and CMA heaps.
The system one is pretty trivial, the CMA one is a bit more involved, especially since we have a call from
kernel/dma/contiguous.cto the CMA heap code. This was solved by turning the logic around and making the CMA heap call into the contiguous DMA code.Let me know what you think,
Maxime1: https://lore.kernel.org/dri-devel/[email protected]/
Signed-off-by Maxime Ripard <[email protected]>
Quoted 8 lines
Quoted 4 lines
Thank you for this patch series; now that it is needed by more folks, I think we can merge this.
Marek, I'll coordinate with you on this - thank you!
If Marek plans on sending it during the next merge window, maybe the best thing to do for us is just to wait for -rc1 and apply the rest of the patches. Otherwise, we can merge the branch in drm-misc.
We could do that. Are we ok to wait till -rc1?
Quoted 2 lines
Maxime
Best,
Sumit