Skip to contentSkip to conversation

Device Tree support for CMA (Contiguous Memory Allocator)

PATCH Superseded Marek Szyprowski 5 participants 14 messages Aug 9, 2013 – Aug 19, 2013

Subsystems (6)

v5 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v5 cover Device Tree support for CMA (Contiguous Memory Allocator)

Hello,

This is a fifth version of my proposal for device tree integration for reserved memory and Contiguous Memory Allocator. I've fixed issues pointed by Michal Nazarewicz, Rob Herring and Kumar Gala. For more information, please check the change log at the end of the mail.

In fourth version of this patch set, after the comments from Grant Likely I moved back memory region definitions back to /memory node (as it was in the first version of this proposal). I've also extended the code and made it more generic, added support for so called reserved dma memory (special dma memory regions created by dma_alloc_coherent() function, for exclusive usage for dma allocation for the given device).

Just a few words for those who see this code for the first time:

The proposed bindings allows to define contiguous memory regions of specified base address and size. Then, the defined regions can be assigned to the given device(s) by adding a property with a phanle to the defined contiguous memory region. From the device tree perspective that's all. Once the bindings are added, all the memory allocations from dma-mapping subsystem will be served from the defined contiguous memory regions.

Contiguous Memory Allocator is a framework, which lets to provide a large contiguous memory buffers for (usually a multimedia) devices. The contiguous memory is reserved during early boot and then shared with kernel, which is allowed to allocate it for movable pages. Then, when device driver requests a contigouous buffer, the framework migrates movable pages out of contiguous region and gives it to the driver. When device driver frees the buffer, it is added to kernel memory pool again. For more information, please refer to commit c64be2bb1c6eb43c838b2c6d57 ("drivers: add Contiguous Memory Allocator") and d484864dd96e1830e76895 (CMA merge commit).

Why we need device tree bindings for CMA at all?

Older ARM kernels used so called board-based initialization. Those board files contained a definition of all hardware blocks available on the target system and particular kernel and driver software configuration selected by the board maintainer.

In the new approach the board files will be removed completely and Device Tree approach is used to describe all hardware blocks available on the target system. By definition, the bindings should be software independent, so at least in theory it should be possible to use those bindings with other operating systems than Linux kernel.

Reserved memory configuration belongs to the grey area. It might depend on hardware restriction of the board or modules and low-level configuration done by bootloader. Putting reserved and contiguous memory regions to /memory node and having phandles to those regions in the device nodes however matches well with the device-tree typical style of linking devices with other resources like clocks, interrupts, regulators, power domains, etc. This is the main reason to use such approach instead of putting everything to /chosen node as it has been proposed in v2 and v3.

Best regards
Marek Szyprowski
Samsung R&D Institute Poland

Changelog:

v5:
- renamed "contiguous-memory-region" compatibility string to "linux,contiguous-memory-region" (this one is really specific to Linux kernel implementation)
- renamed "dma-memory-region" property to "memory-region" (suggested by Kumar Gala)
- added support for #address-cells, #size-cells for memory regions (thanks to Rob Herring for suggestion)
- removed generic code to scan specific path in flat device tree (cannot be used to fdt one-pass scan based initialization of memory regions with #address-cells and #size-cells parsing)
- replaced dma_contiguous_set_default_area() and dma_contiguous_add_device() functions with dev_set_cma_area() call

v4: http://thread.gmane.org/gmane.linux.ports.arm.kernel/256491
- corrected Devcie Tree mailing list address (resend)
- moved back contiguous-memory bindings from /chosen/contiguous-memory to /memory nodes as suggested by Grant (see
http://article.gmane.org/gmane.linux.drivers.devicetree/41030
for more details)
- added support for DMA reserved memory with dma_declare_coherent()
- moved code to drivers/of/of_reserved_mem.c
- added generic code to scan specific path in flat device tree

v3: http://thread.gmane.org/gmane.linux.drivers.devicetree/40013/
- fixed issues pointed by Laura and updated documentation

v2: http://thread.gmane.org/gmane.linux.drivers.devicetree/34075
- moved contiguous-memory bindings from /memory to /chosen/contiguous-memory/ node to avoid spreading Linux specific parameters over the whole device tree definitions
- added support for autoconfigured regions (use zero base)
- fixes minor bugs

v1: http://thread.gmane.org/gmane.linux.drivers.devicetree/30111/
- initial proposal

Patch summary:

Marek Szyprowski (3):
drivers: dma-contiguous: clean source code and prepare for device
tree
drivers: of: add initialization code for dma reserved memory
ARM: init: add support for reserved memory defined by device tree

 Documentation/devicetree/bindings/memory.txt |  152 ++++++++++++++++++++
 arch/arm/include/asm/dma-contiguous.h        |    1 -
 arch/arm/mm/init.c                           |    3 +
 arch/x86/include/asm/dma-contiguous.h        |    1 -
 drivers/base/dma-contiguous.c                |  119 ++++++----------
 drivers/of/Kconfig                           |    6 +
 drivers/of/Makefile                          |    1 +
 drivers/of/of_reserved_mem.c                 |  197 ++++++++++++++++++++++++++
 drivers/of/platform.c                        |    5 +
 include/asm-generic/dma-contiguous.h         |   28 ----
 include/linux/dma-contiguous.h               |   55 ++++++-
 include/linux/of_reserved_mem.h              |   14 ++
 12 files changed, 475 insertions(+), 107 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/memory.txt
 create mode 100644 drivers/of/of_reserved_mem.c
 delete mode 100644 include/asm-generic/dma-contiguous.h
 create mode 100644 include/linux/of_reserved_mem.h
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH v5 1/3 drivers: dma-contiguous: clean source code and prepare for device tree

This patch cleans the initialization of dma contiguous framework. The all-in-one dma_declare_contiguous() function is now separated into dma_contiguous_reserve_area() which only steals the the memory from memblock allocator and dma_contiguous_add_device() function, which assigns given device to the specified reserved memory area. This improves the flexibility in defining contiguous memory areas and assigning device to them, because now it is possible to assign more than one device to the given contiguous memory area. Such split in initialization procedure is also required for upcoming device tree support.

---
 arch/arm/include/asm/dma-contiguous.h |    1 -
 arch/x86/include/asm/dma-contiguous.h |    1 -
 drivers/base/dma-contiguous.c         |  119 ++++++++++++---------------------
 include/asm-generic/dma-contiguous.h  |   28 --------
 include/linux/dma-contiguous.h        |   55 ++++++++++++++-
 5 files changed, 97 insertions(+), 107 deletions(-)
 delete mode 100644 include/asm-generic/dma-contiguous.h
arch/arm/include/asm/dma-contiguous.h +0 -1
index 3ed37b4..63277bc 100644
@@ -5,7 +5,6 @@
 #ifdef CONFIG_CMA
 
 #include <linux/types.h>
-#include <asm-generic/dma-contiguous.h>
 
 void dma_contiguous_early_fixup(phys_addr_t base, unsigned long size);
 
arch/x86/include/asm/dma-contiguous.h +0 -1
index c092416..b4b38ba 100644
@@ -4,7 +4,6 @@
 #ifdef __KERNEL__
 
 #include <linux/types.h>
-#include <asm-generic/dma-contiguous.h>
 
 static inline void
 dma_contiguous_early_fixup(phys_addr_t base, unsigned long size) { }
drivers/base/dma-contiguous.c +44 -75

196 lines not on this page · raw message

include/asm-generic/dma-contiguous.h deleted +0 -28
deleted file mode 100644
index 294b1e7..0000000
@@ -1,28 +0,0 @@
-#ifndef ASM_DMA_CONTIGUOUS_H
-#define ASM_DMA_CONTIGUOUS_H
-
-#ifdef __KERNEL__
-#ifdef CONFIG_CMA
-
-#include <linux/device.h>
-#include <linux/dma-contiguous.h>
-
-static inline struct cma *dev_get_cma_area(struct device *dev)
-{
-	if (dev && dev->cma_area)
-		return dev->cma_area;
-	return dma_contiguous_default_area;
-}
-
-static inline void dev_set_cma_area(struct device *dev, struct cma *cma)
-{
-	if (dev)
-		dev->cma_area = cma;
-	if (!dev && !dma_contiguous_default_area)
-		dma_contiguous_default_area = cma;
-}
-
-#endif
-#endif
-
-#endif
include/linux/dma-contiguous.h +53 -2

70 lines not on this page · raw message

Signature · 1 line
1.7.9.5
#2 lore raw

PATCH v5 2/3 drivers: of: add initialization code for dma reserved memory

In reply to Marek Szyprowski

Add device tree support for contiguous and reserved memory regions defined in device tree. Initialization is done in 2 steps. First, the memory is reserved, what happens very early when only flattened device tree is available. Then on device initialization the corresponding cma and reserved regions are assigned to each device structure.

---
 Documentation/devicetree/bindings/memory.txt |  152 ++++++++++++++++++++
 drivers/of/Kconfig                           |    6 +
 drivers/of/Makefile                          |    1 +
 drivers/of/of_reserved_mem.c                 |  197 ++++++++++++++++++++++++++
 drivers/of/platform.c                        |    5 +
 include/linux/of_reserved_mem.h              |   14 ++
 6 files changed, 375 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/memory.txt
 create mode 100644 drivers/of/of_reserved_mem.c
 create mode 100644 include/linux/of_reserved_mem.h
Documentation/devicetree/bindings/memory.txt new file +152 -0
new file mode 100644
index 0000000..167d013
@@ -0,0 +1,152 @@
+*** Memory binding ***
+
+The /memory node provides basic information about the address and size
+of the physical memory. This node is usually filled or updated by the
+bootloader, depending on the actual memory configuration of the given
+hardware.
+
+The memory layout is described by the folllowing node:
+
+memory {
+	device_type = "memory";
+	reg =  <(baseaddr1) (size1)
+		(baseaddr2) (size2)
+		...
+		(baseaddrN) (sizeN)>;
+};
+
+baseaddrX:	the base address of the defined memory bank
+sizeX:		the size of the defined memory bank
+
+More than one memory bank can be defined.
+
+
+*** Reserved memory regions ***
+
+In /memory/reserved-memory node one can create additional nodes
+describing particular reserved (excluded from normal use) memory
+regions. Such memory regions are usually designed for the special usage
+by various device drivers. A good example are contiguous memory
+allocations or memory sharing with other operating system on the same
+hardware board. Those special memory regions might depend on the board
+configuration and devices used on the target system.
+
+Parameters for each memory region can be encoded into the device tree
+wit the following convention:
+
+[(label):] (name)@(address) {
+	compatible = "contiguous-memory-region", "reserved-memory-region";
+	reg = <(address) (size)>;
+	(linux,default-contiguous-region);
+};
+
+label:		label given to the defined region (optional)
+name:		an name given to the defined region
+address:	the base address of the defined region
+size:		the size of the memory region
+
+compatible:	"contiguous-memory-region" - enables binding of this
+		region to Contiguous Memory Allocator (special region for
+		contiguous memory allocations, shared with movable system
+		memory, Linux kernel-specific), alternatively if
+		"reserved-memory-region" - compatibility is defined, given
+		region is assigned for exclusive usage for DMA transfers
+
+linux,default-contiguous-region: property indicating that the region
+		is the default region for all contiguous memory
+		allocations, Linux specific (optional)
+
+Each defined region must use unique name. It is optional to specify the
+base address, so if one wants to use autoconfiguration of the base
+address, he must specify the '0' as base address in the 'reg' property
+and assign ann unique name to such regions.
+
+
+*** Device node's properties ***
+
+Once the regions in the /memory/reserved-memory node are defined, they
+can be assigned to device nodes to enable drivers for their special use.
+The following properties are defined:
+
+memory-region = <&phandle_to_defined_region>;
+
+This property indicates that the device driver should use the
+memory region pointed by the given phandle.
+
+
+*** Example ***
+
+This example defines a memory consisting of 4 memory banks. 3 contiguous
+regions are defined for Linux kernel, one default of all device drivers
+(named contig_mem, placed at 0x72000000, 64MiB), one dedicated to the
+framebuffer device (labelled display_mem, placed at 0x78000000, 8MiB)
+and one for multimedia processing (labelled multimedia_mem, placed at
+0x77000000, 64MiB). 'display_mem' region is then assigned to fb at 12300000
+device for DMA memory allocations (Linux kernel drivers will use CMA is
+available or dma-exclusive usage otherwise). 'multimedia_mem' is
+assigned to scaler at 12500000 and codec at 12600000 devices for contiguous
+memory allocations when CMA driver is enabled.
+
+The reason for creating a separate region for framebuffer device is to
+match the framebuffer base address to the one configured by bootloader,
+so once Linux kernel drivers starts no glitches on the displayed boot
+logo appears. Scaller and codec drivers should share the memory
+allocations.
+
+/ {
+	/* ... */
+	memory {
+		reg =  <0x40000000 0x10000000
+			0x50000000 0x10000000
+			0x60000000 0x10000000
+			0x70000000 0x10000000>;
+
+		reserved-memory {
+			#address-cells = <1>;
+			#size-cells = <1>;
+
+			/*
+			 * global autoconfigured region for contiguous allocations
+			 * (used only with Contiguous Memory Allocator)
+			 */
+			contig_region at 0 {
+				compatible = "contiguous-memory-region";
+				reg = <0x0 0x4000000>;
+				linux,default-contiguous-region;
+			};
+
+			/*
+			 * special region for framebuffer
+			 */
+			display_mem: region at 78000000 {
+				compatible = "contiguous-memory-region", "reserved-memory-region";
+				reg = <0x78000000 0x800000>;
+			};
+
+			/*
+			 * special region for multimedia processing devices
+			 */
+			multimedia_mem: region at 77000000 {
+				compatible = "contiguous-memory-region";
+				reg = <0x77000000 0x4000000>;
+			};
+		};
+	};
+
+	/* ... */
+
+	fb0: fb at 12300000 {
+		status = "okay";
+		memory-region = <&display_mem>;
+	};
+
+	scaler: scaler at 12500000 {
+		status = "okay";
+		memory-region = <&multimedia_mem>;
+	};
+
+	codec: codec at 12600000 {
+		status = "okay";
+		memory-region = <&multimedia_mem>;
+	};
+};
drivers/of/Kconfig +6 -0
index 80e5c13..a83ab43 100644
@@ -80,4 +80,10 @@ config OF_MTD
 	depends on MTD
 	def_bool y
 
+config OF_RESERVED_MEM
+	depends on CMA || (HAVE_GENERIC_DMA_COHERENT && HAVE_MEMBLOCK)
+	def_bool y
+	help
+	  Initialization code for DMA reserved memory
+
 endmenu # OF
drivers/of/Makefile +1 -0
index 1f9c0c4..e7e3322 100644
@@ -10,3 +10,4 @@ obj-$(CONFIG_OF_MDIO)	+= of_mdio.o
 obj-$(CONFIG_OF_PCI)	+= of_pci.o
 obj-$(CONFIG_OF_PCI_IRQ)  += of_pci_irq.o
 obj-$(CONFIG_OF_MTD)	+= of_mtd.o
+obj-$(CONFIG_OF_RESERVED_MEM) += of_reserved_mem.o
drivers/of/of_reserved_mem.c new file +197 -0

197 lines not on this page · raw message

drivers/of/platform.c +5 -0
index e0a6514..1e4e91d 100644
@@ -21,6 +21,7 @@
 #include <linux/of_device.h>
 #include <linux/of_irq.h>
 #include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
 #include <linux/platform_device.h>
 
 const struct of_device_id of_default_bus_match_table[] = {
@@ -196,6 +197,7 @@ EXPORT_SYMBOL(of_device_alloc);
  * Returns pointer to created platform device, or NULL if a device was not
  * registered.  Unavailable devices will not get registered.
  */
+
 struct platform_device *of_platform_device_create_pdata(
 					struct device_node *np,
 					const char *bus_id,
@@ -218,6 +220,8 @@ struct platform_device *of_platform_device_create_pdata(
 	dev->dev.bus = &platform_bus_type;
 	dev->dev.platform_data = platform_data;
 
+	of_reserved_mem_device_init(&dev->dev);
+
 	/* We do not fill the DMA ops for platform devices by default.
 	 * This is currently the responsibility of the platform code
 	 * to do such, possibly using a device notifier
@@ -225,6 +229,7 @@ struct platform_device *of_platform_device_create_pdata(
 
 	if (of_device_add(dev) != 0) {
 		platform_device_put(dev);
+		of_reserved_mem_device_release(&dev->dev);
 		return NULL;
 	}
 
include/linux/of_reserved_mem.h new file +14 -0
new file mode 100644
index 0000000..1274946
@@ -0,0 +1,14 @@
+#ifndef __OF_RESERVED_MEM_H
+#define __OF_RESERVED_MEM_H
+
+#ifdef CONFIG_OF_RESERVED_MEM
+void of_reserved_mem_device_init(struct device *dev);
+void of_reserved_mem_device_release(struct device *dev);
+void __init dma_reserved_mem_of_reserve(void);
+#else
+#define of_reserved_mem_device_init(dev) (void)0
+#define of_reserved_mem_device_release(dev) (void)0
+#define dma_reserved_mem_of_reserve() (void)0
+#endif
+
+#endif /* __OF_RESERVED_MEM_H */
Signature · 1 line
1.7.9.5
#3 lore raw

PATCH v5 2/3 drivers: of: add initialization code for dma reserved memory

Quoted 3 lines · Marek Szyprowski

Add device tree support for contiguous and reserved memory regions defined in device tree. Initialization is done in 2 steps. First, the memory is reserved, what happens very early when only flattened device

s/what/which/

Quoted 2 lines

tree is available. Then on device initialization the corresponding cma and reserved regions are assigned to each device structure.

What this commit message does not tell me is why does the reservation have to happen before the fdt is unflattened? It would greatly simplify the code if it didn't.

Quoted 21 lines
---
 Documentation/devicetree/bindings/memory.txt |  152 ++++++++++++++++++++
 drivers/of/Kconfig                           |    6 +
 drivers/of/Makefile                          |    1 +
 drivers/of/of_reserved_mem.c                 |  197 ++++++++++++++++++++++++++
 drivers/of/platform.c                        |    5 +
 include/linux/of_reserved_mem.h              |   14 ++
 6 files changed, 375 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/memory.txt
 create mode 100644 drivers/of/of_reserved_mem.c
 create mode 100644 include/linux/of_reserved_mem.h
Documentation/devicetree/bindings/memory.txt new file +0 -0
new file mode 100644
index 0000000..167d013

Documentation/devicetree/bindings/memory.txt line 8

Quoted 8 lines
Documentation/devicetree/bindings/memory.txt new file +8 -0
@@ -0,0 +1,152 @@
+*** Memory binding ***
+
+The /memory node provides basic information about the address and size
+of the physical memory. This node is usually filled or updated by the
+bootloader, depending on the actual memory configuration of the given
+hardware.
+
+The memory layout is described by the folllowing node:

typo.

Quoted 203 lines
Documentation/devicetree/bindings/memory.txt new file +144 -0
+
+memory {
+       device_type = "memory";
+       reg =  <(baseaddr1) (size1)
+               (baseaddr2) (size2)
+               ...
+               (baseaddrN) (sizeN)>;
+};
+
+baseaddrX:     the base address of the defined memory bank
+sizeX:         the size of the defined memory bank
+
+More than one memory bank can be defined.
+
+
+*** Reserved memory regions ***
+
+In /memory/reserved-memory node one can create additional nodes
+describing particular reserved (excluded from normal use) memory
+regions. Such memory regions are usually designed for the special usage
+by various device drivers. A good example are contiguous memory
+allocations or memory sharing with other operating system on the same
+hardware board. Those special memory regions might depend on the board
+configuration and devices used on the target system.
+
+Parameters for each memory region can be encoded into the device tree
+wit the following convention:
+
+[(label):] (name)@(address) {
+       compatible = "contiguous-memory-region", "reserved-memory-region";
+       reg = <(address) (size)>;
+       (linux,default-contiguous-region);
+};
+
+label:         label given to the defined region (optional)
+name:          an name given to the defined region
+address:       the base address of the defined region
+size:          the size of the memory region
+
+compatible:    "contiguous-memory-region" - enables binding of this
+               region to Contiguous Memory Allocator (special region for
+               contiguous memory allocations, shared with movable system
+               memory, Linux kernel-specific), alternatively if
+               "reserved-memory-region" - compatibility is defined, given
+               region is assigned for exclusive usage for DMA transfers
+
+linux,default-contiguous-region: property indicating that the region
+               is the default region for all contiguous memory
+               allocations, Linux specific (optional)
+
+Each defined region must use unique name. It is optional to specify the
+base address, so if one wants to use autoconfiguration of the base
+address, he must specify the '0' as base address in the 'reg' property
+and assign ann unique name to such regions.
+
+
+*** Device node's properties ***
+
+Once the regions in the /memory/reserved-memory node are defined, they
+can be assigned to device nodes to enable drivers for their special use.
+The following properties are defined:
+
+memory-region = <&phandle_to_defined_region>;
+
+This property indicates that the device driver should use the
+memory region pointed by the given phandle.
+
+
+*** Example ***
+
+This example defines a memory consisting of 4 memory banks. 3 contiguous
+regions are defined for Linux kernel, one default of all device drivers
+(named contig_mem, placed at 0x72000000, 64MiB), one dedicated to the
+framebuffer device (labelled display_mem, placed at 0x78000000, 8MiB)
+and one for multimedia processing (labelled multimedia_mem, placed at
+0x77000000, 64MiB). 'display_mem' region is then assigned to fb at 12300000
+device for DMA memory allocations (Linux kernel drivers will use CMA is
+available or dma-exclusive usage otherwise). 'multimedia_mem' is
+assigned to scaler at 12500000 and codec at 12600000 devices for contiguous
+memory allocations when CMA driver is enabled.
+
+The reason for creating a separate region for framebuffer device is to
+match the framebuffer base address to the one configured by bootloader,
+so once Linux kernel drivers starts no glitches on the displayed boot
+logo appears. Scaller and codec drivers should share the memory
+allocations.
+
+/ {
+       /* ... */
+       memory {
+               reg =  <0x40000000 0x10000000
+                       0x50000000 0x10000000
+                       0x60000000 0x10000000
+                       0x70000000 0x10000000>;
+
+               reserved-memory {
+                       #address-cells = <1>;
+                       #size-cells = <1>;
+
+                       /*
+                        * global autoconfigured region for contiguous allocations
+                        * (used only with Contiguous Memory Allocator)
+                        */
+                       contig_region at 0 {
+                               compatible = "contiguous-memory-region";
+                               reg = <0x0 0x4000000>;
+                               linux,default-contiguous-region;
+                       };
+
+                       /*
+                        * special region for framebuffer
+                        */
+                       display_mem: region at 78000000 {
+                               compatible = "contiguous-memory-region", "reserved-memory-region";
+                               reg = <0x78000000 0x800000>;
+                       };
+
+                       /*
+                        * special region for multimedia processing devices
+                        */
+                       multimedia_mem: region at 77000000 {
+                               compatible = "contiguous-memory-region";
+                               reg = <0x77000000 0x4000000>;
+                       };
+               };
+       };
+
+       /* ... */
+
+       fb0: fb at 12300000 {
+               status = "okay";
+               memory-region = <&display_mem>;
+       };
+
+       scaler: scaler at 12500000 {
+               status = "okay";
+               memory-region = <&multimedia_mem>;
+       };
+
+       codec: codec at 12600000 {
+               status = "okay";
+               memory-region = <&multimedia_mem>;
+       };
+};
drivers/of/Kconfig +6 -0
index 80e5c13..a83ab43 100644
@@ -80,4 +80,10 @@ config OF_MTD
        depends on MTD
        def_bool y
 
+config OF_RESERVED_MEM
+       depends on CMA || (HAVE_GENERIC_DMA_COHERENT && HAVE_MEMBLOCK)
+       def_bool y
+       help
+         Initialization code for DMA reserved memory
+
 endmenu # OF
drivers/of/Makefile +1 -0
index 1f9c0c4..e7e3322 100644
@@ -10,3 +10,4 @@ obj-$(CONFIG_OF_MDIO) += of_mdio.o
 obj-$(CONFIG_OF_PCI)   += of_pci.o
 obj-$(CONFIG_OF_PCI_IRQ)  += of_pci_irq.o
 obj-$(CONFIG_OF_MTD)   += of_mtd.o
+obj-$(CONFIG_OF_RESERVED_MEM) += of_reserved_mem.o
drivers/of/of_reserved_mem.c new file +29 -0
new file mode 100644
index 0000000..b256b41
@@ -0,0 +1,197 @@
+/*
+ * Device tree based initialization code for reserved memory.
+ *
+ * Copyright (c) 2013 Samsung Electronics Co., Ltd.
+ *             http://www.samsung.com
+ * Author: Marek Szyprowski <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License or (at your optional) any later version of the license.
+ */
+
+#include <asm/dma-contiguous.h>
+
+#include <linux/memblock.h>
+#include <linux/err.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
+#include <linux/of_platform.h>
+#include <linux/mm.h>
+#include <linux/sizes.h>
+#include <linux/mm_types.h>
+#include <linux/dma-contiguous.h>
+#include <linux/dma-mapping.h>
+#include <linux/of_reserved_mem.h>
+
+#define MAX_RESERVED_REGIONS   16
+struct reserved_mem {

drivers/of/of_reserved_mem.c line 39

Quoted 10 lines
drivers/of/of_reserved_mem.c new file +10 -0
+       phys_addr_t             base;
+       unsigned long           size;
+       struct cma              *cma;
+       char                    name[32];
+};
+static struct reserved_mem reserved_mem[MAX_RESERVED_REGIONS];
+static int reserved_mem_count;
+
+static int __init reserved_mem_fdt_scan(unsigned long node, const char *uname,
+                                   int depth, void *data)

early_init_dt_scan_reserved_mem would be a more consistent name.

Quoted 12 lines
drivers/of/of_reserved_mem.c new file +12 -0
+{
+       static int size_cells = OF_ROOT_NODE_SIZE_CELLS_DEFAULT;
+       static int addr_cells = OF_ROOT_NODE_ADDR_CELLS_DEFAULT;
+       static int my_depth;
+       phys_addr_t base, size;
+       int is_cma, is_reserved;
+       unsigned long len;
+       __be32 *prop;
+
+       if (depth == 1 && my_depth == 0 && strcmp(uname, "memory") == 0) {
+               my_depth = depth;
+               /* scan next node */

drivers/of/of_reserved_mem.c line 61

Quoted 10 lines
drivers/of/of_reserved_mem.c new file +10 -0
+               return 0;
+       } else if (depth == 2 && my_depth == 1 &&
+           strcmp(uname, "reserved-memory") == 0) {
+               prop = of_get_flat_dt_prop(node, "#size-cells", NULL);
+               if (prop)
+                       size_cells = be32_to_cpup(prop);
+
+               prop = of_get_flat_dt_prop(node, "#address-cells", NULL);
+               if (prop)
+                       addr_cells = be32_to_cpup(prop);

I think we should just require these be the same size as the memory node which would be dt_root_*_cells.

I'm fine with moving this into drivers/of/fdt.c if that simplifies things.

drivers/of/of_reserved_mem.c line 72

Quoted 11 lines
drivers/of/of_reserved_mem.c new file +11 -0
+
+               my_depth = depth;
+               /* scan next node */
+               return 0;
+       } else if (depth != 3 && my_depth != 2) {
+               /* scan next node */
+               return 0;
+       } else if (depth < my_depth) {
+               /* break scan now */
+               return 1;
+       }

This code bothers me and is hard to follow. I don't think trying to use of_scan_flat_dt is the right approach here. What you really want here is check for reserved-memory node under the memory node and then scan each child node. This could all be done from early_init_dt_scan_memory. u-boot has more rich flat DT functions than the kernel does since the kernel does very little with the flat DT. Look at fdt_next_node and it's callers in u-boot for some examples. But still my first question remains. Can we avoid the flat DT altogether?

Quoted 62 lines
drivers/of/of_reserved_mem.c new file +62 -0
+       /* now we are scanning a /memory/reserved-memory child */
+
+       is_cma = of_flat_dt_is_compatible(node, "linux,contiguous-memory-region");
+       is_reserved = of_flat_dt_is_compatible(node, "reserved-memory-region");
+
+       if (!is_reserved && !(is_cma && IS_ENABLED(CONFIG_CMA))) {
+               /* ignore node and scan next one */
+               return 0;
+       }
+
+       prop = of_get_flat_dt_prop(node, "reg", &len);
+       if (!prop || (len != (size_cells + addr_cells) * sizeof(__be32))) {
+               pr_err("Reserved mem: node %s, incorrect \"reg\" property\n",
+                      uname);
+               /* ignore node and scan next one */
+               return 0;
+       }
+       base = dt_mem_next_cell(addr_cells, &prop);
+       size = dt_mem_next_cell(size_cells, &prop);
+
+       if (!size) {
+               /* ignore node and scan next one */
+               return 0;
+       }
+
+       pr_info("Reserved mem: found %s, memory base %lx, size %ld MiB\n",
+               uname, (unsigned long)base, (unsigned long)size / SZ_1M);
+
+       if (reserved_mem_count == ARRAY_SIZE(reserved_mem))
+               return -ENOSPC;
+
+       reserved_mem[reserved_mem_count].base = base;
+       reserved_mem[reserved_mem_count].size = size;
+       strlcpy(reserved_mem[reserved_mem_count].name, uname,
+               sizeof(reserved_mem[reserved_mem_count].name));
+
+       if (IS_ENABLED(CONFIG_CMA) && is_cma) {
+               struct cma *cma;
+               if (dma_contiguous_reserve_area(size, base, 0, &cma) == 0) {
+                       reserved_mem[reserved_mem_count].cma = cma;
+                       reserved_mem_count++;
+
+                       if (of_get_flat_dt_prop(node,
+                                               "linux,default-contiguous-region",
+                                               NULL))
+                               dma_contiguous_default_area = cma;
+               }
+       } else if (is_reserved) {
+               if (memblock_remove(base, size) == 0)
+                       reserved_mem_count++;
+               else
+                       pr_err("Failed to reserve memory for %s\n", uname);
+       }
+
+       return 0;
+}
+
+static struct reserved_mem *get_dma_memory_region(struct device *dev)
+{
+       struct device_node *node;
+       const char *name;
+       int i;

drivers/of/of_reserved_mem.c line 145

Quoted 10 lines
drivers/of/of_reserved_mem.c new file +10 -0
+
+       node = of_parse_phandle(dev->of_node, "memory-region", 0);
+       if (!node)
+               return NULL;
+
+       name = kbasename(node->full_name);
+       for (i = 0; i < reserved_mem_count; i++)
+               if (strcmp(name, reserved_mem[i].name) == 0)
+                       return &reserved_mem[i];
+       return NULL;

Matching against a struct device_node pointer would be more common way to match. So it would be good to update reserved_mem with a device_node ptr when we unflatten the DT.

Quoted 65 lines
drivers/of/of_reserved_mem.c new file +52 -0
+}
+
+/**
+ * of_reserved_mem_device_init() - assign reserved memory region to given device
+ *
+ * This function assign memory region pointed by "dma-memory-region" device tree
+ * property to the given device.
+ */
+void of_reserved_mem_device_init(struct device *dev)
+{
+       struct reserved_mem *region = get_dma_memory_region(dev);
+       if (!region)
+               return;
+
+       if (region->cma) {
+               dev_set_cma_area(dev, region->cma);
+               pr_info("Assigned CMA %s to %s device\n", region->name,
+                       dev_name(dev));
+       } else {
+               if (dma_declare_coherent_memory(dev, region->base, region->base,
+                   region->size, DMA_MEMORY_MAP | DMA_MEMORY_EXCLUSIVE) != 0)
+                       pr_info("Declared reserved memory %s to %s device\n",
+                               region->name, dev_name(dev));
+       }
+}
+
+/**
+ * of_reserved_mem_device_release() - release reserved memory device structures
+ *
+ * This function releases structures allocated for memory region handling for
+ * the given device.
+ */
+void of_reserved_mem_device_release(struct device *dev)
+{
+       struct reserved_mem *region = get_dma_memory_region(dev);
+       if (!region && !region->cma)
+               dma_release_declared_memory(dev);
+}
+
+/**
+ * dma_reserved_mem_reserve() - grab memory reserved for device exclusive use
+ *
+ * This function grabs memory from early allocator for device exclusive use
+ * defined in device tree structures. It should be called by arch specific code
+ * once the early allocator (memblock) has been activated and all other
+ * subsystems have already allocated/reserved memory.
+ */
+void __init dma_reserved_mem_of_reserve(void)
+{
+       if (initial_boot_params)
+               of_scan_flat_dt(reserved_mem_fdt_scan, NULL);
+}
drivers/of/platform.c +1 -0
index e0a6514..1e4e91d 100644
@@ -21,6 +21,7 @@
 #include <linux/of_device.h>
 #include <linux/of_irq.h>
 #include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
 #include <linux/platform_device.h>
 
 const struct of_device_id of_default_bus_match_table[] = {

drivers/of/platform.c EXPORT_SYMBOL(of_device_alloc); · line 200

Quoted 4 lines
drivers/of/platform.c +1 -0
@@ -196,6 +197,7 @@ EXPORT_SYMBOL(of_device_alloc);
  * Returns pointer to created platform device, or NULL if a device was not
  * registered.  Unavailable devices will not get registered.
  */
+

stray ws change. Please remove.

Quoted 26 lines
drivers/of/platform.c +3 -0
 struct platform_device *of_platform_device_create_pdata(
                                        struct device_node *np,
                                        const char *bus_id,
@@ -218,6 +220,8 @@ struct platform_device *of_platform_device_create_pdata(
        dev->dev.bus = &platform_bus_type;
        dev->dev.platform_data = platform_data;
 
+       of_reserved_mem_device_init(&dev->dev);
+
        /* We do not fill the DMA ops for platform devices by default.
         * This is currently the responsibility of the platform code
         * to do such, possibly using a device notifier
@@ -225,6 +229,7 @@ struct platform_device *of_platform_device_create_pdata(
 
        if (of_device_add(dev) != 0) {
                platform_device_put(dev);
+               of_reserved_mem_device_release(&dev->dev);
                return NULL;
        }
 
include/linux/of_reserved_mem.h new file +0 -0
new file mode 100644
index 0000000..1274946

include/linux/of_reserved_mem.h line 7

Quoted 7 lines
include/linux/of_reserved_mem.h new file +7 -0
@@ -0,0 +1,14 @@
+#ifndef __OF_RESERVED_MEM_H
+#define __OF_RESERVED_MEM_H
+
+#ifdef CONFIG_OF_RESERVED_MEM
+void of_reserved_mem_device_init(struct device *dev);
+void of_reserved_mem_device_release(struct device *dev);
+void __init dma_reserved_mem_of_reserve(void);

Declarations don't need __init annotation.

include/linux/of_reserved_mem.h line 11

Quoted 4 lines
include/linux/of_reserved_mem.h new file +4 -0
+#else
+#define of_reserved_mem_device_init(dev) (void)0
+#define of_reserved_mem_device_release(dev) (void)0
+#define dma_reserved_mem_of_reserve() (void)0

static inline is preferred over defines.

Rob

#4 lore raw

PATCH v5 2/3 drivers: of: add initialization code for dma reserved memory

Hello,

Quoted 6 lines · Rob Herring
Quoted 3 lines · Marek Szyprowski

Add device tree support for contiguous and reserved memory regions defined in device tree. Initialization is done in 2 steps. First, the memory is reserved, what happens very early when only flattened device

Quoted 8 lines

s/what/which/

Quoted 2 lines

tree is available. Then on device initialization the corresponding cma and reserved regions are assigned to each device structure.

What this commit message does not tell me is why does the reservation have to happen before the fdt is unflattened? It would greatly simplify the code if it didn't.

Large memory blocks can be RELIABLY reserved only during early boot. This must happen before the whole memory management subsystem is initialized, because we need to ensure that the given contiguous blocks are not yet allocated by kernel. Also it must happen before kernel mappings for the whole low memory are created, to ensure that there will be no mappings (for reserved blocks) or mapping with special properties can be created (for CMA blocks). This all happens before device tree structures are unflattened, so we need to get reserved memory layout directly from fdt.

Quoted 274 lines
Quoted 21 lines
---
 Documentation/devicetree/bindings/memory.txt |  152 ++++++++++++++++++++
 drivers/of/Kconfig                           |    6 +
 drivers/of/Makefile                          |    1 +
 drivers/of/of_reserved_mem.c                 |  197 ++++++++++++++++++++++++++
 drivers/of/platform.c                        |    5 +
 include/linux/of_reserved_mem.h              |   14 ++
 6 files changed, 375 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/memory.txt
 create mode 100644 drivers/of/of_reserved_mem.c
 create mode 100644 include/linux/of_reserved_mem.h
Documentation/devicetree/bindings/memory.txt new file +0 -0
new file mode 100644
index 0000000..167d013

Documentation/devicetree/bindings/memory.txt line 8

Quoted 8 lines
Documentation/devicetree/bindings/memory.txt new file +8 -0
@@ -0,0 +1,152 @@
+*** Memory binding ***
+
+The /memory node provides basic information about the address and size
+of the physical memory. This node is usually filled or updated by the
+bootloader, depending on the actual memory configuration of the given
+hardware.
+
+The memory layout is described by the folllowing node:

typo.

Quoted 203 lines
Documentation/devicetree/bindings/memory.txt new file +144 -0
+
+memory {
+       device_type = "memory";
+       reg =  <(baseaddr1) (size1)
+               (baseaddr2) (size2)
+               ...
+               (baseaddrN) (sizeN)>;
+};
+
+baseaddrX:     the base address of the defined memory bank
+sizeX:         the size of the defined memory bank
+
+More than one memory bank can be defined.
+
+
+*** Reserved memory regions ***
+
+In /memory/reserved-memory node one can create additional nodes
+describing particular reserved (excluded from normal use) memory
+regions. Such memory regions are usually designed for the special usage
+by various device drivers. A good example are contiguous memory
+allocations or memory sharing with other operating system on the same
+hardware board. Those special memory regions might depend on the board
+configuration and devices used on the target system.
+
+Parameters for each memory region can be encoded into the device tree
+wit the following convention:
+
+[(label):] (name)@(address) {
+       compatible = "contiguous-memory-region", "reserved-memory-region";
+       reg = <(address) (size)>;
+       (linux,default-contiguous-region);
+};
+
+label:         label given to the defined region (optional)
+name:          an name given to the defined region
+address:       the base address of the defined region
+size:          the size of the memory region
+
+compatible:    "contiguous-memory-region" - enables binding of this
+               region to Contiguous Memory Allocator (special region for
+               contiguous memory allocations, shared with movable system
+               memory, Linux kernel-specific), alternatively if
+               "reserved-memory-region" - compatibility is defined, given
+               region is assigned for exclusive usage for DMA transfers
+
+linux,default-contiguous-region: property indicating that the region
+               is the default region for all contiguous memory
+               allocations, Linux specific (optional)
+
+Each defined region must use unique name. It is optional to specify the
+base address, so if one wants to use autoconfiguration of the base
+address, he must specify the '0' as base address in the 'reg' property
+and assign ann unique name to such regions.
+
+
+*** Device node's properties ***
+
+Once the regions in the /memory/reserved-memory node are defined, they
+can be assigned to device nodes to enable drivers for their special use.
+The following properties are defined:
+
+memory-region = <&phandle_to_defined_region>;
+
+This property indicates that the device driver should use the
+memory region pointed by the given phandle.
+
+
+*** Example ***
+
+This example defines a memory consisting of 4 memory banks. 3 contiguous
+regions are defined for Linux kernel, one default of all device drivers
+(named contig_mem, placed at 0x72000000, 64MiB), one dedicated to the
+framebuffer device (labelled display_mem, placed at 0x78000000, 8MiB)
+and one for multimedia processing (labelled multimedia_mem, placed at
+0x77000000, 64MiB). 'display_mem' region is then assigned to fb at 12300000
+device for DMA memory allocations (Linux kernel drivers will use CMA is
+available or dma-exclusive usage otherwise). 'multimedia_mem' is
+assigned to scaler at 12500000 and codec at 12600000 devices for contiguous
+memory allocations when CMA driver is enabled.
+
+The reason for creating a separate region for framebuffer device is to
+match the framebuffer base address to the one configured by bootloader,
+so once Linux kernel drivers starts no glitches on the displayed boot
+logo appears. Scaller and codec drivers should share the memory
+allocations.
+
+/ {
+       /* ... */
+       memory {
+               reg =  <0x40000000 0x10000000
+                       0x50000000 0x10000000
+                       0x60000000 0x10000000
+                       0x70000000 0x10000000>;
+
+               reserved-memory {
+                       #address-cells = <1>;
+                       #size-cells = <1>;
+
+                       /*
+                        * global autoconfigured region for contiguous allocations
+                        * (used only with Contiguous Memory Allocator)
+                        */
+                       contig_region at 0 {
+                               compatible = "contiguous-memory-region";
+                               reg = <0x0 0x4000000>;
+                               linux,default-contiguous-region;
+                       };
+
+                       /*
+                        * special region for framebuffer
+                        */
+                       display_mem: region at 78000000 {
+                               compatible = "contiguous-memory-region", "reserved-memory-region";
+                               reg = <0x78000000 0x800000>;
+                       };
+
+                       /*
+                        * special region for multimedia processing devices
+                        */
+                       multimedia_mem: region at 77000000 {
+                               compatible = "contiguous-memory-region";
+                               reg = <0x77000000 0x4000000>;
+                       };
+               };
+       };
+
+       /* ... */
+
+       fb0: fb at 12300000 {
+               status = "okay";
+               memory-region = <&display_mem>;
+       };
+
+       scaler: scaler at 12500000 {
+               status = "okay";
+               memory-region = <&multimedia_mem>;
+       };
+
+       codec: codec at 12600000 {
+               status = "okay";
+               memory-region = <&multimedia_mem>;
+       };
+};
drivers/of/Kconfig +6 -0
index 80e5c13..a83ab43 100644
@@ -80,4 +80,10 @@ config OF_MTD
        depends on MTD
        def_bool y
 
+config OF_RESERVED_MEM
+       depends on CMA || (HAVE_GENERIC_DMA_COHERENT && HAVE_MEMBLOCK)
+       def_bool y
+       help
+         Initialization code for DMA reserved memory
+
 endmenu # OF
drivers/of/Makefile +1 -0
index 1f9c0c4..e7e3322 100644
@@ -10,3 +10,4 @@ obj-$(CONFIG_OF_MDIO) += of_mdio.o
 obj-$(CONFIG_OF_PCI)   += of_pci.o
 obj-$(CONFIG_OF_PCI_IRQ)  += of_pci_irq.o
 obj-$(CONFIG_OF_MTD)   += of_mtd.o
+obj-$(CONFIG_OF_RESERVED_MEM) += of_reserved_mem.o
drivers/of/of_reserved_mem.c new file +29 -0
new file mode 100644
index 0000000..b256b41
@@ -0,0 +1,197 @@
+/*
+ * Device tree based initialization code for reserved memory.
+ *
+ * Copyright (c) 2013 Samsung Electronics Co., Ltd.
+ *             http://www.samsung.com
+ * Author: Marek Szyprowski <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License or (at your optional) any later version of the license.
+ */
+
+#include <asm/dma-contiguous.h>
+
+#include <linux/memblock.h>
+#include <linux/err.h>
+#include <linux/of.h>
+#include <linux/of_fdt.h>
+#include <linux/of_platform.h>
+#include <linux/mm.h>
+#include <linux/sizes.h>
+#include <linux/mm_types.h>
+#include <linux/dma-contiguous.h>
+#include <linux/dma-mapping.h>
+#include <linux/of_reserved_mem.h>
+
+#define MAX_RESERVED_REGIONS   16
+struct reserved_mem {

drivers/of/of_reserved_mem.c line 39

Quoted 10 lines
drivers/of/of_reserved_mem.c new file +10 -0
+       phys_addr_t             base;
+       unsigned long           size;
+       struct cma              *cma;
+       char                    name[32];
+};
+static struct reserved_mem reserved_mem[MAX_RESERVED_REGIONS];
+static int reserved_mem_count;
+
+static int __init reserved_mem_fdt_scan(unsigned long node, const char *uname,
+                                   int depth, void *data)

early_init_dt_scan_reserved_mem would be a more consistent name.

Quoted 12 lines
drivers/of/of_reserved_mem.c new file +12 -0
+{
+       static int size_cells = OF_ROOT_NODE_SIZE_CELLS_DEFAULT;
+       static int addr_cells = OF_ROOT_NODE_ADDR_CELLS_DEFAULT;
+       static int my_depth;
+       phys_addr_t base, size;
+       int is_cma, is_reserved;
+       unsigned long len;
+       __be32 *prop;
+
+       if (depth == 1 && my_depth == 0 && strcmp(uname, "memory") == 0) {
+               my_depth = depth;
+               /* scan next node */

drivers/of/of_reserved_mem.c line 61

Quoted 10 lines
drivers/of/of_reserved_mem.c new file +10 -0
+               return 0;
+       } else if (depth == 2 && my_depth == 1 &&
+           strcmp(uname, "reserved-memory") == 0) {
+               prop = of_get_flat_dt_prop(node, "#size-cells", NULL);
+               if (prop)
+                       size_cells = be32_to_cpup(prop);
+
+               prop = of_get_flat_dt_prop(node, "#address-cells", NULL);
+               if (prop)
+                       addr_cells = be32_to_cpup(prop);

I think we should just require these be the same size as the memory node which would be dt_root_*_cells.

Quoted 1 line

I'm fine with moving this into drivers/of/fdt.c if that simplifies things.

dt_root_*_cells are global variables, so its not a problem to get access to them. However I wonder how can we ensure that user/device tree creator will set #size-cells/#address-cells to the same values as for root memory node? Would it be enough to state that in binding documentation? If so then the reserved memory code can skip parsing them and use dt_root_*_cells directly, what will simplify the code.

Quoted 12 lines

drivers/of/of_reserved_mem.c line 72

Quoted 11 lines
drivers/of/of_reserved_mem.c new file +11 -0
+
+               my_depth = depth;
+               /* scan next node */
+               return 0;
+       } else if (depth != 3 && my_depth != 2) {
+               /* scan next node */
+               return 0;
+       } else if (depth < my_depth) {
+               /* break scan now */
+               return 1;
+       }
Quoted 5 lines

This code bothers me and is hard to follow. I don't think trying to use of_scan_flat_dt is the right approach here. What you really want here is check for reserved-memory node under the memory node and then scan each child node. This could all be done from early_init_dt_scan_memory.

early_init_dt_scan_memory() is also called from of_scan_flat_dt() and it also implements similar state machine to parse fdt. The only difference is the fact that "memory" is a direct child of root node, so the state machine is much simpler (there is no need to parse /memory/reserved-memory path).

Quoted 5 lines

u-boot has more rich flat DT functions than
the kernel does since the kernel does very little with the flat DT. Look at fdt_next_node and it's callers in u-boot for some examples. But still my first question remains. Can we avoid the flat DT altogether?

I see no such possibility. FDT is unflattened too late to do reliably all operation required for memory reservation.

Quoted 68 lines
Quoted 68 lines
drivers/of/of_reserved_mem.c new file +68 -0
+       /* now we are scanning a /memory/reserved-memory child */
+
+       is_cma = of_flat_dt_is_compatible(node, "linux,contiguous-memory-region");
+       is_reserved = of_flat_dt_is_compatible(node, "reserved-memory-region");
+
+       if (!is_reserved && !(is_cma && IS_ENABLED(CONFIG_CMA))) {
+               /* ignore node and scan next one */
+               return 0;
+       }
+
+       prop = of_get_flat_dt_prop(node, "reg", &len);
+       if (!prop || (len != (size_cells + addr_cells) * sizeof(__be32))) {
+               pr_err("Reserved mem: node %s, incorrect \"reg\" property\n",
+                      uname);
+               /* ignore node and scan next one */
+               return 0;
+       }
+       base = dt_mem_next_cell(addr_cells, &prop);
+       size = dt_mem_next_cell(size_cells, &prop);
+
+       if (!size) {
+               /* ignore node and scan next one */
+               return 0;
+       }
+
+       pr_info("Reserved mem: found %s, memory base %lx, size %ld MiB\n",
+               uname, (unsigned long)base, (unsigned long)size / SZ_1M);
+
+       if (reserved_mem_count == ARRAY_SIZE(reserved_mem))
+               return -ENOSPC;
+
+       reserved_mem[reserved_mem_count].base = base;
+       reserved_mem[reserved_mem_count].size = size;
+       strlcpy(reserved_mem[reserved_mem_count].name, uname,
+               sizeof(reserved_mem[reserved_mem_count].name));
+
+       if (IS_ENABLED(CONFIG_CMA) && is_cma) {
+               struct cma *cma;
+               if (dma_contiguous_reserve_area(size, base, 0, &cma) == 0) {
+                       reserved_mem[reserved_mem_count].cma = cma;
+                       reserved_mem_count++;
+
+                       if (of_get_flat_dt_prop(node,
+                                               "linux,default-contiguous-region",
+                                               NULL))
+                               dma_contiguous_default_area = cma;
+               }
+       } else if (is_reserved) {
+               if (memblock_remove(base, size) == 0)
+                       reserved_mem_count++;
+               else
+                       pr_err("Failed to reserve memory for %s\n", uname);
+       }
+
+       return 0;
+}
+
+static struct reserved_mem *get_dma_memory_region(struct device *dev)
+{
+       struct device_node *node;
+       const char *name;
+       int i;
+
+       node = of_parse_phandle(dev->of_node, "memory-region", 0);
+       if (!node)
+               return NULL;
+
+       name = kbasename(node->full_name);
Quoted 8 lines

drivers/of/of_reserved_mem.c line 145

Quoted 4 lines
drivers/of/of_reserved_mem.c new file +4 -0
+       for (i = 0; i < reserved_mem_count; i++)
+               if (strcmp(name, reserved_mem[i].name) == 0)
+                       return &reserved_mem[i];
+       return NULL;

Matching against a struct device_node pointer would be more common way to match. So it would be good to update reserved_mem with a device_node ptr when we unflatten the DT.

I wonder if it really makes sense. To get device_node ptr I will need to scan /memody/reserved-memory node and match all its children BY NAME with the structures parsed from FDT (stored in reserved_mem array). Then I will need to iterate again for each device node with memory-region property to find the needed entry. Names are unique, IMHO they can serve as a key for matching structures between FDT and regular, unflattened DT.

Quoted 108 lines
Quoted 65 lines
drivers/of/of_reserved_mem.c new file +52 -0
+}
+
+/**
+ * of_reserved_mem_device_init() - assign reserved memory region to given device
+ *
+ * This function assign memory region pointed by "dma-memory-region" device tree
+ * property to the given device.
+ */
+void of_reserved_mem_device_init(struct device *dev)
+{
+       struct reserved_mem *region = get_dma_memory_region(dev);
+       if (!region)
+               return;
+
+       if (region->cma) {
+               dev_set_cma_area(dev, region->cma);
+               pr_info("Assigned CMA %s to %s device\n", region->name,
+                       dev_name(dev));
+       } else {
+               if (dma_declare_coherent_memory(dev, region->base, region->base,
+                   region->size, DMA_MEMORY_MAP | DMA_MEMORY_EXCLUSIVE) != 0)
+                       pr_info("Declared reserved memory %s to %s device\n",
+                               region->name, dev_name(dev));
+       }
+}
+
+/**
+ * of_reserved_mem_device_release() - release reserved memory device structures
+ *
+ * This function releases structures allocated for memory region handling for
+ * the given device.
+ */
+void of_reserved_mem_device_release(struct device *dev)
+{
+       struct reserved_mem *region = get_dma_memory_region(dev);
+       if (!region && !region->cma)
+               dma_release_declared_memory(dev);
+}
+
+/**
+ * dma_reserved_mem_reserve() - grab memory reserved for device exclusive use
+ *
+ * This function grabs memory from early allocator for device exclusive use
+ * defined in device tree structures. It should be called by arch specific code
+ * once the early allocator (memblock) has been activated and all other
+ * subsystems have already allocated/reserved memory.
+ */
+void __init dma_reserved_mem_of_reserve(void)
+{
+       if (initial_boot_params)
+               of_scan_flat_dt(reserved_mem_fdt_scan, NULL);
+}
drivers/of/platform.c +1 -0
index e0a6514..1e4e91d 100644
@@ -21,6 +21,7 @@
 #include <linux/of_device.h>
 #include <linux/of_irq.h>
 #include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
 #include <linux/platform_device.h>
 
 const struct of_device_id of_default_bus_match_table[] = {

drivers/of/platform.c EXPORT_SYMBOL(of_device_alloc); · line 200

Quoted 4 lines
drivers/of/platform.c +1 -0
@@ -196,6 +197,7 @@ EXPORT_SYMBOL(of_device_alloc);
  * Returns pointer to created platform device, or NULL if a device was not
  * registered.  Unavailable devices will not get registered.
  */
+

stray ws change. Please remove.

Quoted 26 lines
drivers/of/platform.c +3 -0
 struct platform_device *of_platform_device_create_pdata(
                                        struct device_node *np,
                                        const char *bus_id,
@@ -218,6 +220,8 @@ struct platform_device *of_platform_device_create_pdata(
        dev->dev.bus = &platform_bus_type;
        dev->dev.platform_data = platform_data;
 
+       of_reserved_mem_device_init(&dev->dev);
+
        /* We do not fill the DMA ops for platform devices by default.
         * This is currently the responsibility of the platform code
         * to do such, possibly using a device notifier
@@ -225,6 +229,7 @@ struct platform_device *of_platform_device_create_pdata(
 
        if (of_device_add(dev) != 0) {
                platform_device_put(dev);
+               of_reserved_mem_device_release(&dev->dev);
                return NULL;
        }
 
include/linux/of_reserved_mem.h new file +0 -0
new file mode 100644
index 0000000..1274946

include/linux/of_reserved_mem.h line 7

Quoted 7 lines
include/linux/of_reserved_mem.h new file +7 -0
@@ -0,0 +1,14 @@
+#ifndef __OF_RESERVED_MEM_H
+#define __OF_RESERVED_MEM_H
+
+#ifdef CONFIG_OF_RESERVED_MEM
+void of_reserved_mem_device_init(struct device *dev);
+void of_reserved_mem_device_release(struct device *dev);
+void __init dma_reserved_mem_of_reserve(void);

Declarations don't need __init annotation.

Quoted 9 lines

include/linux/of_reserved_mem.h line 11

Quoted 4 lines
include/linux/of_reserved_mem.h new file +4 -0
+#else
+#define of_reserved_mem_device_init(dev) (void)0
+#define of_reserved_mem_device_release(dev) (void)0
+#define dma_reserved_mem_of_reserve() (void)0

static inline is preferred over defines.

Rob

Best regards

Signature · 2 lines
Marek Szyprowski
Samsung R&D Institute Poland
#5 lore raw

PATCH v5 2/3 drivers: of: add initialization code for dma reserved memory

Quoted 21 lines · Marek Szyprowski

Hello,

Quoted 7 lines · Rob Herring
Quoted 3 lines · Marek Szyprowski

Add device tree support for contiguous and reserved memory regions defined in device tree. Initialization is done in 2 steps. First, the memory is reserved, what happens very early when only flattened device

Quoted 8 lines

s/what/which/

Quoted 2 lines

tree is available. Then on device initialization the corresponding cma and reserved regions are assigned to each device structure.

What this commit message does not tell me is why does the reservation have to happen before the fdt is unflattened? It would greatly simplify the code if it didn't.

Quoted 9 lines

Large memory blocks can be RELIABLY reserved only during early boot. This must happen before the whole memory management subsystem is initialized, because we need to ensure that the given contiguous blocks are not yet allocated by kernel. Also it must happen before kernel mappings for the whole low memory are created, to ensure that there will be no mappings (for reserved blocks) or mapping with special properties can be created (for CMA blocks). This all happens before device tree structures are unflattened, so we need to get reserved memory layout directly from fdt.

Okay. Just making sure.

Quoted 17 lines
Quoted 11 lines

Review

Quoted 10 lines
drivers/of/of_reserved_mem.c new file +7 -0
+       } else if (depth == 2 && my_depth == 1 &&
+           strcmp(uname, "reserved-memory") == 0) {
+               prop = of_get_flat_dt_prop(node, "#size-cells", NULL);
+               if (prop)
+                       size_cells = be32_to_cpup(prop);
+
+               prop = of_get_flat_dt_prop(node, "#address-cells",

NULL);

(quoted diff) +2 -0
+               if (prop)
+                       addr_cells = be32_to_cpup(prop);
Quoted 4 lines

I think we should just require these be the same size as the memory node which would be dt_root_*_cells.

I'm fine with moving this into drivers/of/fdt.c if that simplifies things.

Quoted 6 lines

dt_root_*_cells are global variables, so its not a problem to get access to them. However I wonder how can we ensure that user/device tree creator will set #size-cells/#address-cells to the same values as for root memory node? Would it be enough to state that in binding documentation? If so then the reserved memory code can skip parsing them and use dt_root_*_cells directly, what will simplify the code.

Yes, just add a note to the binding that the cell sizes are the same as the memory node.

Quoted 19 lines
Quoted 12 lines

drivers/of/of_reserved_mem.c line 72

Quoted 11 lines
drivers/of/of_reserved_mem.c new file +11 -0
+
+               my_depth = depth;
+               /* scan next node */
+               return 0;
+       } else if (depth != 3 && my_depth != 2) {
+               /* scan next node */
+               return 0;
+       } else if (depth < my_depth) {
+               /* break scan now */
+               return 1;
+       }
Quoted 5 lines

This code bothers me and is hard to follow. I don't think trying to use of_scan_flat_dt is the right approach here. What you really want here is check for reserved-memory node under the memory node and then scan each child node. This could all be done from early_init_dt_scan_memory.

Quoted 6 lines

early_init_dt_scan_memory() is also called from of_scan_flat_dt() and it also implements similar state machine to parse fdt. The only difference is the fact that "memory" is a direct child of root node, so the state machine is much simpler (there is no need to parse /memory/reserved-memory path).

If you have already found the memory node, then why find it again? I don't think the existing scan functions handle anything but top-level nodes very well. So doing something like this from early_init_dt_scan_memory() is what I was thinking. It is a very rough sketch:

// find the reserved-memory node

for (ndepth = 0, off = fdt_next_node(blob, parent_offset, &ndepth);
    (off >= 0) && (ndepth > 0);
    off = fdt_next_node(blob, off, &ndepth)) {
    if (ndepth == 1) {
        name = fdt_get_name(blob, off, 0), off);
        if (strcmp(name, "reserved-memory") == 0) {
             parent_offset = off;
             break; // found
    }
}

// find the child nodes

for (ndepth = 0, off = fdt_next_node(blob, parent_offset, &ndepth);
    (off >= 0) && (ndepth == 1);
    off = fdt_next_node(blob, off, &ndepth)) {
    // now handle each child
}

These could probably be further refined with some loop iterator macros.

Quoted 11 lines
Quoted 9 lines

drivers/of/of_reserved_mem.c line 145

Quoted 5 lines
drivers/of/of_reserved_mem.c new file +5 -0
+       name = kbasename(node->full_name);
+       for (i = 0; i < reserved_mem_count; i++)
+               if (strcmp(name, reserved_mem[i].name) == 0)
+                       return &reserved_mem[i];
+       return NULL;

Matching against a struct device_node pointer would be more common way to match. So it would be good to update reserved_mem with a device_node ptr when we unflatten the DT.

Quoted 6 lines

I wonder if it really makes sense. To get device_node ptr I will need to scan /memody/reserved-memory node and match all its children BY NAME with the structures parsed from FDT (stored in reserved_mem array). Then I will need to iterate again for each device node with memory-region property to find the needed entry. Names are unique, IMHO they can serve as a key for matching structures between FDT and regular, unflattened DT.

You are iterating multiple times using a string match versus iterating once more and then doing a pointer match. However, it is not really performance critical and is fine for now.

Rob

#6 lore raw

PATCH v5 2/3 drivers: of: add initialization code for dma reserved memory

Hello,

Quoted 107 lines · Rob Herring
Quoted 21 lines · Marek Szyprowski

Hello,

Quoted 7 lines · Rob Herring
Quoted 3 lines · Marek Szyprowski

Add device tree support for contiguous and reserved memory regions defined in device tree. Initialization is done in 2 steps. First, the memory is reserved, what happens very early when only flattened device

Quoted 8 lines

s/what/which/

Quoted 2 lines

tree is available. Then on device initialization the corresponding cma and reserved regions are assigned to each device structure.

What this commit message does not tell me is why does the reservation have to happen before the fdt is unflattened? It would greatly simplify the code if it didn't.

Quoted 9 lines

Large memory blocks can be RELIABLY reserved only during early boot. This must happen before the whole memory management subsystem is initialized, because we need to ensure that the given contiguous blocks are not yet allocated by kernel. Also it must happen before kernel mappings for the whole low memory are created, to ensure that there will be no mappings (for reserved blocks) or mapping with special properties can be created (for CMA blocks). This all happens before device tree structures are unflattened, so we need to get reserved memory layout directly from fdt.

Okay. Just making sure.

Quoted 17 lines
Quoted 11 lines

Review

Quoted 10 lines
drivers/of/of_reserved_mem.c new file +7 -0
+       } else if (depth == 2 && my_depth == 1 &&
+           strcmp(uname, "reserved-memory") == 0) {
+               prop = of_get_flat_dt_prop(node, "#size-cells", NULL);
+               if (prop)
+                       size_cells = be32_to_cpup(prop);
+
+               prop = of_get_flat_dt_prop(node, "#address-cells",

NULL);

(quoted diff) +2 -0
+               if (prop)
+                       addr_cells = be32_to_cpup(prop);
Quoted 4 lines

I think we should just require these be the same size as the memory node which would be dt_root_*_cells.

I'm fine with moving this into drivers/of/fdt.c if that simplifies things.

Quoted 6 lines

dt_root_*_cells are global variables, so its not a problem to get access to them. However I wonder how can we ensure that user/device tree creator will set #size-cells/#address-cells to the same values as for root memory node? Would it be enough to state that in binding documentation? If so then the reserved memory code can skip parsing them and use dt_root_*_cells directly, what will simplify the code.

Yes, just add a note to the binding that the cell sizes are the same as the memory node.

Quoted 19 lines
Quoted 12 lines

drivers/of/of_reserved_mem.c line 72

Quoted 11 lines
drivers/of/of_reserved_mem.c new file +11 -0
+
+               my_depth = depth;
+               /* scan next node */
+               return 0;
+       } else if (depth != 3 && my_depth != 2) {
+               /* scan next node */
+               return 0;
+       } else if (depth < my_depth) {
+               /* break scan now */
+               return 1;
+       }
Quoted 5 lines

This code bothers me and is hard to follow. I don't think trying to use of_scan_flat_dt is the right approach here. What you really want here is check for reserved-memory node under the memory node and then scan each child node. This could all be done from early_init_dt_scan_memory.

Quoted 6 lines

early_init_dt_scan_memory() is also called from of_scan_flat_dt() and it also implements similar state machine to parse fdt. The only difference is the fact that "memory" is a direct child of root node, so the state machine is much simpler (there is no need to parse /memory/reserved-memory path).

If you have already found the memory node, then why find it again? I don't think the existing scan functions handle anything but top-level nodes very well. So doing something like this from early_init_dt_scan_memory() is what I was thinking. It is a very rough sketch:

// find the reserved-memory node

for (ndepth = 0, off = fdt_next_node(blob, parent_offset, &ndepth);
     (off >= 0) && (ndepth > 0);
     off = fdt_next_node(blob, off, &ndepth)) {
     if (ndepth == 1) {
         name = fdt_get_name(blob, off, 0), off);
         if (strcmp(name, "reserved-memory") == 0) {
              parent_offset = off;
              break; // found
     }
}
Quoted 8 lines

// find the child nodes

for (ndepth = 0, off = fdt_next_node(blob, parent_offset, &ndepth);
     (off >= 0) && (ndepth == 1);
     off = fdt_next_node(blob, off, &ndepth)) {
     // now handle each child
}

These could probably be further refined with some loop iterator macros.

The above code looks pretty nice, but there are some problems with it:

1. Although it would look nice to call it from early_init_dt_scan_memory() it won't be possible, because that time it is too early. memblock structures are initialized after a call to early_init_dt_scan_memory() and until that no changes to memory layout are easily possible.

2. Currently there are no fdt parsing functions in the kernel. I've tried to split of_scan_flat_dt() into fdt_next_node() + fdt_get_name() and use them both in of_scan_flat_dt() and above reserved memory parsing functions. In the end I got quite a lot of code, which is still quite hard to follow.

Because of the above I decided to resurrect of_scan_flat_dt_by_path() function from the previous version and use #size-cells/#address-cells attributes from root node what in the end simplified reserved memory parsing function. I hope that it can be accepted after such changes without introducing a burden caused by the whole infrastructure for manual parsing fdt.

Quoted 18 lines
Quoted 11 lines
Quoted 9 lines

drivers/of/of_reserved_mem.c line 145

Quoted 5 lines
drivers/of/of_reserved_mem.c new file +5 -0
+       name = kbasename(node->full_name);
+       for (i = 0; i < reserved_mem_count; i++)
+               if (strcmp(name, reserved_mem[i].name) == 0)
+                       return &reserved_mem[i];
+       return NULL;

Matching against a struct device_node pointer would be more common way to match. So it would be good to update reserved_mem with a device_node ptr when we unflatten the DT.

Quoted 6 lines

I wonder if it really makes sense. To get device_node ptr I will need to scan /memody/reserved-memory node and match all its children BY NAME with the structures parsed from FDT (stored in reserved_mem array). Then I will need to iterate again for each device node with memory-region property to find the needed entry. Names are unique, IMHO they can serve as a key for matching structures between FDT and regular, unflattened DT.

Quoted 3 lines

You are iterating multiple times using a string match versus iterating once more and then doing a pointer match. However, it is not really performance critical and is fine for now.

Thanks!

Best regards

Signature · 2 lines
Marek Szyprowski
Samsung R&D Institute Poland
#7 lore raw

PATCH v5 2/3 drivers: of: add initialization code for dma reserved memory

Quoted 11 lines · Marek Szyprowski

Hello,

Quoted 7 lines · Rob Herring
Quoted 4 lines · Marek Szyprowski

Hello,

On 8/10/2013 7:33 PM, Rob Herring wrote:

[snip]

Quoted 35 lines
Quoted 25 lines
Quoted 6 lines

early_init_dt_scan_memory() is also called from of_scan_flat_dt() and it also implements similar state machine to parse fdt. The only difference is the fact that "memory" is a direct child of root node, so the state machine is much simpler (there is no need to parse /memory/reserved-memory path).

If you have already found the memory node, then why find it again? I don't think the existing scan functions handle anything but top-level nodes very well. So doing something like this from early_init_dt_scan_memory() is what I was thinking. It is a very rough sketch:

// find the reserved-memory node

for (ndepth = 0, off = fdt_next_node(blob, parent_offset, &ndepth);
     (off >= 0) && (ndepth > 0);
     off = fdt_next_node(blob, off, &ndepth)) {
     if (ndepth == 1) {
         name = fdt_get_name(blob, off, 0), off);
         if (strcmp(name, "reserved-memory") == 0) {
              parent_offset = off;
              break; // found
     }
}
Quoted 8 lines

// find the child nodes

for (ndepth = 0, off = fdt_next_node(blob, parent_offset, &ndepth);
     (off >= 0) && (ndepth == 1);
     off = fdt_next_node(blob, off, &ndepth)) {
     // now handle each child
}

These could probably be further refined with some loop iterator macros.

Quoted 6 lines

The above code looks pretty nice, but there are some problems with it:

1. Although it would look nice to call it from early_init_dt_scan_memory() it won't be possible, because that time it is too early. memblock structures are initialized after a call to early_init_dt_scan_memory() and until that no changes to memory layout are easily possible.

Okay. So the reserved areas have to be setup after memblock_add is called. It seems that what exactly early_init_dt_add_memory_arch does varies by arch. For example, arm64 directly does a memblock_add call and would probably work as I suggested. The arm code just puts the information in an intermediate arch specific struct until later. There is a lot of room for clean-up in the early DT code to reduce the amount of arch code.

Quoted 11 lines

2. Currently there are no fdt parsing functions in the kernel. I've tried to split of_scan_flat_dt() into fdt_next_node() + fdt_get_name() and use them both in of_scan_flat_dt() and above reserved memory parsing functions. In the end I got quite a lot of code, which is still quite hard to follow.

Because of the above I decided to resurrect of_scan_flat_dt_by_path() function from the previous version and use #size-cells/#address-cells attributes from root node what in the end simplified reserved memory parsing function. I hope that it can be accepted after such changes without introducing a burden caused by the whole infrastructure for manual parsing fdt.

This all looks fine at first glance.

Rob

#8 lore raw

PATCH v5 2/3 drivers: of: add initialization code for dma reserved memory

In reply to Marek Szyprowski
Quoted 5 lines · Marek Szyprowski

Add device tree support for contiguous and reserved memory regions defined in device tree. Initialization is done in 2 steps. First, the memory is reserved, what happens very early when only flattened device tree is available. Then on device initialization the corresponding cma and reserved regions are assigned to each device structure.

Hmmm. This seems an awful lot like putting SW configuration/policy information into DT rather than HW description. This feels like a slippery slope... Isn't this kind of thing better handled by a kernel command-line option to set up the CMA size?

Quoted 11 lines
Documentation/devicetree/bindings/memory.txt +9 -0
 
+*** Memory binding ***
+
+The /memory node provides basic information about the address and size
+of the physical memory. This node is usually filled or updated by the
+bootloader, depending on the actual memory configuration of the given
+hardware.
+
+The memory layout is described by the folllowing node:
+

Documentation/devicetree/bindings/memory.txt line 19

Quoted 10 lines
Documentation/devicetree/bindings/memory.txt +10 -0
+memory {
+	device_type = "memory";
+	reg =  <(baseaddr1) (size1)
+		(baseaddr2) (size2)
+		...
+		(baseaddrN) (sizeN)>;
+};
+
+baseaddrX:	the base address of the defined memory bank
+sizeX:		the size of the defined memory bank

You probably want to mention that baseaddrX is #address-cells long and sizeX is #size-cells long. Same for the reserved regions below.

Quoted 1 line

+*** Reserved memory regions ***

...

Documentation/devicetree/bindings/memory.txt line 37

Quoted 4 lines
Documentation/devicetree/bindings/memory.txt +4 -0
+Parameters for each memory region can be encoded into the device tree
+wit the following convention:
+
+[(label):] (name)@(address) {

That line is DT syntax nothing to do with this binding. I would re-write this in the more typical DT binding style where the documentation only specifies the content of the node, not the node itself.

In particular, there's no requirement for a node name to include the unit address (@address) if it's already unique.

Documentation/devicetree/bindings/memory.txt line 40

Quoted 3 lines
Documentation/devicetree/bindings/memory.txt +3 -0
+	compatible = "contiguous-memory-region", "reserved-memory-region";
+	reg = <(address) (size)>;
+	(linux,default-contiguous-region);

(...) isn't a syntax typically used in DT bindings. You'd usually put that in a a list of "Optional Properties:".

Quoted 1 line

+Each defined region must use unique name.

Well, DT nodes are supposed to be names based on the type of object they represent, not by the name/identity of the object they represent.

Documentation/devicetree/bindings/memory.txt line 74

Quoted 10 lines
Documentation/devicetree/bindings/memory.txt +10 -0
+*** Device node's properties ***
+
+Once the regions in the /memory/reserved-memory node are defined, they
+can be assigned to device nodes to enable drivers for their special use.
+The following properties are defined:
+
+memory-region = <&phandle_to_defined_region>;
+
+This property indicates that the device driver should use the
+memory region pointed by the given phandle.

That's quite scary. This is essentially forcing a memory-region property into every single binding that ever exists. I guess that's not too much worse than e.g. interrupts/clocks/..., but I think it's worth somehow requiring bindings to "opt-in" to allowing this property to be part of their binding rather than just definining the property globally.

#9 lore raw

PATCH v5 2/3 drivers: of: add initialization code for dma reserved memory

Quoted 11 lines · Stephen Warren
Quoted 5 lines · Marek Szyprowski

Add device tree support for contiguous and reserved memory regions defined in device tree. Initialization is done in 2 steps. First, the memory is reserved, what happens very early when only flattened device tree is available. Then on device initialization the corresponding cma and reserved regions are assigned to each device structure.

Hmmm. This seems an awful lot like putting SW configuration/policy information into DT rather than HW description. This feels like a slippery slope... Isn't this kind of thing better handled by a kernel command-line option to set up the CMA size?

Sorry, you were not part of the in-person discussion since it happened at Linaro Connect in Dublin. The concern is that we really need a way to describe some of these _system_ properties. They're not necessarily hardware properties, but they are well-known and likely properties of the system that is running.

I.e. say you have a device with a known screen resolution running some reasonably well-known software stacks/applications that you expect will use a certain amount of graphics memory. Or you know how large your camera sensor is, so you know how much carveout the ISP will need for captures. So defining how large the default carveouts should be is very much a property of this system. We would previously have put these settings in a board file and be done, but now we need a way to describe it.

/chosen was proposed, but that's mostly used for things that the sysadmin would adjust (traditionally such as firmware stdin/stdout, etc). /options isn't a good place either.

Having them on the command line doesn't really work well, over time we'll start to accumulate cruft there. It's bad enough for some of the devices out there that do it, I'd say.

So, for the specific case of memory carveouts, this was suggested as the best way forward.

-Olof

#10 lore raw

PATCH v5 2/3 drivers: of: add initialization code for dma reserved memory

Quoted 13 lines · Olof Johansson
Quoted 11 lines · Stephen Warren
Quoted 5 lines · Marek Szyprowski

Add device tree support for contiguous and reserved memory regions defined in device tree. Initialization is done in 2 steps. First, the memory is reserved, what happens very early when only flattened device tree is available. Then on device initialization the corresponding cma and reserved regions are assigned to each device structure.

Hmmm. This seems an awful lot like putting SW configuration/policy information into DT rather than HW description. This feels like a slippery slope... Isn't this kind of thing better handled by a kernel command-line option to set up the CMA size?

Quoted 5 lines

Sorry, you were not part of the in-person discussion since it happened at Linaro Connect in Dublin. The concern is that we really need a way to describe some of these _system_ properties. They're not necessarily hardware properties, but they are well-known and likely properties of the system that is running.

OK, that seems reasonable enough. It's just the first I heard of this.

I rather suspect that similar arguments will be applied to a bunch of other data people want to put into DT. I guess we'll just have to wait and see what gets proposed, and what really is "system data" rather than policy:-)

#11 lore raw

PATCH v5 2/3 drivers: of: add initialization code for dma reserved memory

In reply to Marek Szyprowski

Hi,

Quoted 5 lines · Marek Szyprowski

Add device tree support for contiguous and reserved memory regions defined in device tree. Initialization is done in 2 steps. First, the memory is reserved, what happens very early when only flattened device tree is available. Then on device initialization the corresponding cma and reserved regions are assigned to each device structure.

Bikeshedding on some wording, mostly in case you have to respin for other reasons.

Quoted 22 lines
Documentation/devicetree/bindings/memory.txt new file +16 -0
new file mode 100644
index 0000000..167d013
@@ -0,0 +1,152 @@
+*** Memory binding ***
+
+The /memory node provides basic information about the address and size
+of the physical memory. This node is usually filled or updated by the
+bootloader, depending on the actual memory configuration of the given
+hardware.
+
+The memory layout is described by the folllowing node:
+
+memory {
+	device_type = "memory";
+	reg =  <(baseaddr1) (size1)
+		(baseaddr2) (size2)
+		...
+		(baseaddrN) (sizeN)>;
+};

Documentation/devicetree/bindings/memory.txt line 26

Quoted 10 lines
Documentation/devicetree/bindings/memory.txt new file +10 -0
+
+baseaddrX:	the base address of the defined memory bank
+sizeX:		the size of the defined memory bank
+
+More than one memory bank can be defined.
+
+
+*** Reserved memory regions ***
+
+In /memory/reserved-memory node one can create additional nodes

There's really no requirement on naming here. We should instead search all of /memory for the compatible properties.

Documentation/devicetree/bindings/memory.txt line 37

Quoted 11 lines
Documentation/devicetree/bindings/memory.txt new file +11 -0
+describing particular reserved (excluded from normal use) memory
+regions. Such memory regions are usually designed for the special usage
+by various device drivers. A good example are contiguous memory
+allocations or memory sharing with other operating system on the same
+hardware board. Those special memory regions might depend on the board
+configuration and devices used on the target system.
+
+Parameters for each memory region can be encoded into the device tree
+wit the following convention:
+
+[(label):] (name)@(address) {

@(unit-address)

Documentation/devicetree/bindings/memory.txt line 45

Quoted 8 lines
Documentation/devicetree/bindings/memory.txt new file +8 -0
+	compatible = "contiguous-memory-region", "reserved-memory-region";
+	reg = <(address) (size)>;
+	(linux,default-contiguous-region);
+};
+
+label:		label given to the defined region (optional)
+name:		an name given to the defined region
+address:	the base address of the defined region

unit-address. It's optional unless there are conflicting node names.

Documentation/devicetree/bindings/memory.txt line 53

Quoted 8 lines
Documentation/devicetree/bindings/memory.txt new file +8 -0
+size:		the size of the memory region
+
+compatible:	"contiguous-memory-region" - enables binding of this
+		region to Contiguous Memory Allocator (special region for
+		contiguous memory allocations, shared with movable system
+		memory, Linux kernel-specific), alternatively if
+		"reserved-memory-region" - compatibility is defined, given
+		region is assigned for exclusive usage for DMA transfers

"reserved-memory-region" isn't necessarily just for DMA transfers, it can be used pretty much for anything. pstore comes to mind.

Documentation/devicetree/bindings/memory.txt line 57

Quoted 3 lines
Documentation/devicetree/bindings/memory.txt new file +3 -0
+linux,default-contiguous-region: property indicating that the region
+		is the default region for all contiguous memory
+		allocations, Linux specific (optional)

Hmm. We handle some of these defaults such as device naming (i2c busses, etc) through /alias nodes. Maybe it would be more consistent to do that here too, by providing a phandle there instead.

Documentation/devicetree/bindings/memory.txt line 57

Quoted 1 line
Documentation/devicetree/bindings/memory.txt new file +1 -0
+Each defined region must use unique name.

Hmm, this is quite different from how memory nodes work, and it'd be nice to keep similar semantics. A "name" property would make more sense for areas that need to be named, but even then, why do names here matter? References should be by phandle, not by name. So I think it should likely just be scrapped.

Documentation/devicetree/bindings/memory.txt line 62

Quoted 4 lines

It is optional to specify the

Documentation/devicetree/bindings/memory.txt new file +3 -0
+base address, so if one wants to use autoconfiguration of the base
+address, he must specify the '0' as base address in the 'reg' property
+and assign ann unique name to such regions.

"an unique"

But I don't think this is a good idea, since it violates basic use of the reg properties, where you usually don't have overlaps, and no real requirements on names (as mentioned above).

-Olof

#12 lore raw

PATCH v5 3/3 ARM: init: add support for reserved memory defined by device tree

In reply to Marek Szyprowski

Enable reserved memory initialization from device tree.

---
 arch/arm/mm/init.c |    3 +++
 1 file changed, 3 insertions(+)
arch/arm/mm/init.c +3 -0
index 15225d8..7c1c4b2 100644
@@ -17,6 +17,7 @@
 #include <linux/nodemask.h>
 #include <linux/initrd.h>
 #include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
 #include <linux/highmem.h>
 #include <linux/gfp.h>
 #include <linux/memblock.h>
@@ -377,6 +378,8 @@ void __init arm_memblock_init(struct meminfo *mi, struct machine_desc *mdesc)
 	if (mdesc->reserve)
 		mdesc->reserve();
 
+	dma_reserved_mem_of_reserve();
+
 	/*
 	 * reserve memory for DMA contigouos allocations,
 	 * must come from DMA area inside low memory
Signature · 1 line
1.7.9.5
#13 lore raw

v5 cover Device Tree support for CMA (Contiguous Memory Allocator)

In reply to Marek Szyprowski

Hi Marek,

Quoted 119 lines · On Friday 09 of August 2013 13:51:56 Marek Szyprowski wrote:

Hello,

This is a fifth version of my proposal for device tree integration for reserved memory and Contiguous Memory Allocator. I've fixed issues pointed by Michal Nazarewicz, Rob Herring and Kumar Gala. For more information, please check the change log at the end of the mail.

In fourth version of this patch set, after the comments from Grant Likely I moved back memory region definitions back to /memory node (as it was in the first version of this proposal). I've also extended the code and made it more generic, added support for so called reserved dma memory (special dma memory regions created by dma_alloc_coherent() function, for exclusive usage for dma allocation for the given device).

Just a few words for those who see this code for the first time:

The proposed bindings allows to define contiguous memory regions of specified base address and size. Then, the defined regions can be assigned to the given device(s) by adding a property with a phanle to the defined contiguous memory region. From the device tree perspective that's all. Once the bindings are added, all the memory allocations from dma-mapping subsystem will be served from the defined contiguous memory regions.

Contiguous Memory Allocator is a framework, which lets to provide a large contiguous memory buffers for (usually a multimedia) devices. The contiguous memory is reserved during early boot and then shared with kernel, which is allowed to allocate it for movable pages. Then, when device driver requests a contigouous buffer, the framework migrates movable pages out of contiguous region and gives it to the driver. When device driver frees the buffer, it is added to kernel memory pool again. For more information, please refer to commit c64be2bb1c6eb43c838b2c6d57 ("drivers: add Contiguous Memory Allocator") and d484864dd96e1830e76895 (CMA merge commit).

Why we need device tree bindings for CMA at all?

Older ARM kernels used so called board-based initialization. Those board files contained a definition of all hardware blocks available on the target system and particular kernel and driver software configuration selected by the board maintainer.

In the new approach the board files will be removed completely and Device Tree approach is used to describe all hardware blocks available on the target system. By definition, the bindings should be software independent, so at least in theory it should be possible to use those bindings with other operating systems than Linux kernel.

Reserved memory configuration belongs to the grey area. It might depend on hardware restriction of the board or modules and low-level configuration done by bootloader. Putting reserved and contiguous memory regions to /memory node and having phandles to those regions in the device nodes however matches well with the device-tree typical style of linking devices with other resources like clocks, interrupts, regulators, power domains, etc. This is the main reason to use such approach instead of putting everything to /chosen node as it has been proposed in v2 and v3.

Best regards
Marek Szyprowski
Samsung R&D Institute Poland

Changelog:

v5:
- renamed "contiguous-memory-region" compatibility string to "linux,contiguous-memory-region" (this one is really specific to Linux kernel implementation)
- renamed "dma-memory-region" property to "memory-region" (suggested by Kumar Gala)
- added support for #address-cells, #size-cells for memory regions (thanks to Rob Herring for suggestion)
- removed generic code to scan specific path in flat device tree (cannot be used to fdt one-pass scan based initialization of memory regions
with #address-cells and #size-cells parsing)
- replaced dma_contiguous_set_default_area() and dma_contiguous_add_device() functions with dev_set_cma_area() call

v4: http://thread.gmane.org/gmane.linux.ports.arm.kernel/256491
- corrected Devcie Tree mailing list address (resend)
- moved back contiguous-memory bindings from /chosen/contiguous-memory to /memory nodes as suggested by Grant (see
http://article.gmane.org/gmane.linux.drivers.devicetree/41030
for more details)
- added support for DMA reserved memory with dma_declare_coherent()
- moved code to drivers/of/of_reserved_mem.c
- added generic code to scan specific path in flat device tree

v3: http://thread.gmane.org/gmane.linux.drivers.devicetree/40013/
- fixed issues pointed by Laura and updated documentation

v2: http://thread.gmane.org/gmane.linux.drivers.devicetree/34075
- moved contiguous-memory bindings from /memory to /chosen/contiguous-memory/ node to avoid spreading Linux specific parameters over the whole device tree definitions
- added support for autoconfigured regions (use zero base)
- fixes minor bugs

v1: http://thread.gmane.org/gmane.linux.drivers.devicetree/30111/
- initial proposal

Patch summary:

Marek Szyprowski (3):
drivers: dma-contiguous: clean source code and prepare for device
tree
drivers: of: add initialization code for dma reserved memory
ARM: init: add support for reserved memory defined by device tree

 Documentation/devicetree/bindings/memory.txt |  152 ++++++++++++++++++++
arch/arm/include/asm/dma-contiguous.h        |    1 -
 arch/arm/mm/init.c                           |    3 +
 arch/x86/include/asm/dma-contiguous.h        |    1 -
 drivers/base/dma-contiguous.c                |  119 ++++++----------
 drivers/of/Kconfig                           |    6 +
 drivers/of/Makefile                          |    1 +
 drivers/of/of_reserved_mem.c                 |  197

++++++++++++++++++++++++++ drivers/of/platform.c
| 5 +

Quoted 8 lines
 include/asm-generic/dma-contiguous.h         |   28 ----
 include/linux/dma-contiguous.h               |   55 ++++++-
 include/linux/of_reserved_mem.h              |   14 ++
 12 files changed, 475 insertions(+), 107 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/memory.txt
 create mode 100644 drivers/of/of_reserved_mem.c
 delete mode 100644 include/asm-generic/dma-contiguous.h
 create mode 100644 include/linux/of_reserved_mem.h

We definitely need this and current version looks pretty good to me.

Best regards,
Tomasz