of/fdt: add of_fdt_device_is_available function

For static DT usecases, we don't need the disabled nodes and can skip unflattening. This saves a significant amount of RAM in memory constrained cases. In one example on STM32F469, the RAM usage goes from 118K to 26K.

There are a few cases in the kernel that modify the status property dynamically. These all are changes from enabled to disabled, depend on OF_DYNAMIC or are not FDT based (PDT based).

Cc Nicolas Pitre <[email protected]>

Cc Frank Rowand <[email protected]>

Signed-off-by Rob Herring <[email protected]>

---
For more background, see this presentation from Nico:

https://connect.linaro.org/resource/sfo17/sfo17-100/

 drivers/of/fdt.c | 4 ++++
 1 file changed, 4 insertions(+)

drivers/of/fdt.c +4 -0

index f8c39705418b..efe91c6856a0 100644
@@ -396,6 +396,10 @@ static int unflatten_dt_nodes(const void *blob,
 		if (WARN_ON_ONCE(depth >= FDT_MAX_DEPTH))
 			continue;
 
+		if (!IS_ENABLED(CONFIG_OF_DYNAMIC) &&
+		    !of_fdt_device_is_available(blob, offset))
+			continue;
+
 		if (!populate_node(blob, offset, &mem, nps[depth],
 				   &nps[depth+1], dryrun))
 			return mem - base;

Signature · 1 line

2.11.0

Quoted 11 lines · Rob Herring

For static DT usecases, we don't need the disabled nodes and can skip unflattening. This saves a significant amount of RAM in memory constrained cases. In one example on STM32F469, the RAM usage goes from 118K to 26K.

There are a few cases in the kernel that modify the status property dynamically. These all are changes from enabled to disabled, depend on OF_DYNAMIC or are not FDT based (PDT based).

Cc Nicolas Pitre <[email protected]>

Cc Frank Rowand <[email protected]>

Signed-off-by Rob Herring <[email protected]>

Tested-by Nicolas Pitre <[email protected]>

Quoted 4 lines

---
For more background, see this presentation from Nico:

https://connect.linaro.org/resource/sfo17/sfo17-100/

Oh my ... this is horrible ...

/me hides in shame

Quoted 23 lines

 drivers/of/fdt.c | 4 ++++
 1 file changed, 4 insertions(+)

drivers/of/fdt.c +4 -0

index f8c39705418b..efe91c6856a0 100644
@@ -396,6 +396,10 @@ static int unflatten_dt_nodes(const void *blob,
 		if (WARN_ON_ONCE(depth >= FDT_MAX_DEPTH))
 			continue;
 
+		if (!IS_ENABLED(CONFIG_OF_DYNAMIC) &&
+		    !of_fdt_device_is_available(blob, offset))
+			continue;
+
 		if (!populate_node(blob, offset, &mem, nps[depth],
 				   &nps[depth+1], dryrun))
 			return mem - base;

Signature · 1 line

2.11.0

Quoted 16 lines · Nicolas Pitre

Quoted 11 lines · Rob Herring

For static DT usecases, we don't need the disabled nodes and can skip unflattening. This saves a significant amount of RAM in memory constrained cases. In one example on STM32F469, the RAM usage goes from 118K to 26K.

There are a few cases in the kernel that modify the status property dynamically. These all are changes from enabled to disabled, depend on OF_DYNAMIC or are not FDT based (PDT based).

Cc Nicolas Pitre <[email protected]>

Cc Frank Rowand <[email protected]>

Signed-off-by Rob Herring <[email protected]>

Tested-by Nicolas Pitre <[email protected]>

Quoted 6 lines

Quoted 4 lines

---
For more background, see this presentation from Nico:

https://connect.linaro.org/resource/sfo17/sfo17-100/

Oh my ... this is horrible ...

I found the presentation to be quite good and useful.

The Linaro site does not have the slides, could you please send me a copy?

For static DT usecases, we don't need the disabled nodes and can skip unflattening. This saves a significant amount of RAM in memory constrained cases. In one example on STM32F469, the RAM usage goes from 118K to 26K.

There are a few cases in the kernel that modify the status property dynamically. These all are changes from enabled to disabled, depend on OF_DYNAMIC or are not FDT based (PDT based).

Cc Nicolas Pitre <[email protected]>

Cc Frank Rowand <[email protected]>

Signed-off-by Rob Herring <[email protected]>

---
For more background, see this presentation from Nico:

https://connect.linaro.org/resource/sfo17/sfo17-100/

 drivers/of/fdt.c | 4 ++++
 1 file changed, 4 insertions(+)

drivers/of/fdt.c +0 -0

index f8c39705418b..efe91c6856a0 100644

Quoted 25 lines · Rob Herring

For static DT usecases, we don't need the disabled nodes and can skip unflattening. This saves a significant amount of RAM in memory constrained cases. In one example on STM32F469, the RAM usage goes from 118K to 26K.

There are a few cases in the kernel that modify the status property dynamically. These all are changes from enabled to disabled, depend on OF_DYNAMIC or are not FDT based (PDT based).

Cc Nicolas Pitre <[email protected]>

Cc Frank Rowand <[email protected]>

Signed-off-by Rob Herring <[email protected]>

---
For more background, see this presentation from Nico:

https://connect.linaro.org/resource/sfo17/sfo17-100/

 drivers/of/fdt.c | 4 ++++
 1 file changed, 4 insertions(+)

drivers/of/fdt.c +0 -0

index f8c39705418b..efe91c6856a0 100644

drivers/of/fdt.c static int unflatten_dt_nodes(const void *blob, · line 405

Quoted 10 lines

drivers/of/fdt.c +4 -0

@@ -396,6 +396,10 @@ static int unflatten_dt_nodes(const void *blob,
              if (WARN_ON_ONCE(depth >= FDT_MAX_DEPTH))
                      continue;
 
+             if (!IS_ENABLED(CONFIG_OF_DYNAMIC) &&
+                 !of_fdt_device_is_available(blob, offset))
+                     continue;
+
              if (!populate_node(blob, offset, &mem, nps[depth],
                                 &nps[depth+1], dryrun))
                      return mem - base;

Hi Rob,

I strongly support the idea of this patch, but there may be an issue we have to resolve. I'm pretty sure we had talked about the issue a long time ago, and it has been sitting on my todo list.

We have two sets of node traversal macros and functions. One set honors the status property, and the other ignores it. If I recall our previous discussion properly, we want the normal usage to honor the status property and only a tiny (or maybe non-existent) set of locations to be allowed to ignore the status property.

A rough sense of how often the status property is honored or not is:

   $ git grep for_each_child_of_node | wc -l
   293
   $ git grep of_get_next_child | wc -l
   103

   $ git grep for_each_available_child_of_node | wc -l
   106
   $ git grep of_get_next_available_child | wc -l
   20

Many of the cases where the status flag is ignored will not actually encounter a node that is not available, so many of the cases where status is not checked could currently be checking status.

And just for completeness, there are a number of standalone checks for whether a node is available:

   $ git grep of_device_is_available | wc -l
   128

It will be a pain to manually check all of the sites that ignore the status property, but that task should be done.

In the mean time, maybe we could flush out the few cases that currently depend on ignoring the status property by

- making for_each_child_of_node() and of_get_next_child() actually check for valid status

- provide a temporary (one or two kernel release) CONFIG option to allow the old behavior for for_each_child_of_node() and of_get_next_child() just in case we miss any locations that need to be fixed

- fix up the few places in core device tree code that actually need to ignore status (if such places exist)

In the end, the *_available_*() interfaces should be removed, because the normal behavior of node traversal should be to only traverse nodes that are available.

Quoted 44 lines · Frank Rowand

Quoted 25 lines · Rob Herring

For static DT usecases, we don't need the disabled nodes and can skip unflattening. This saves a significant amount of RAM in memory constrained cases. In one example on STM32F469, the RAM usage goes from 118K to 26K.

There are a few cases in the kernel that modify the status property dynamically. These all are changes from enabled to disabled, depend on OF_DYNAMIC or are not FDT based (PDT based).

Cc Nicolas Pitre <[email protected]>

Cc Frank Rowand <[email protected]>

Signed-off-by Rob Herring <[email protected]>

---
For more background, see this presentation from Nico:

https://connect.linaro.org/resource/sfo17/sfo17-100/

 drivers/of/fdt.c | 4 ++++
 1 file changed, 4 insertions(+)

drivers/of/fdt.c +0 -0

index f8c39705418b..efe91c6856a0 100644

drivers/of/fdt.c static int unflatten_dt_nodes(const void *blob, · line 405

Quoted 10 lines

drivers/of/fdt.c +4 -0

@@ -396,6 +396,10 @@ static int unflatten_dt_nodes(const void *blob,
              if (WARN_ON_ONCE(depth >= FDT_MAX_DEPTH))
                      continue;
 
+             if (!IS_ENABLED(CONFIG_OF_DYNAMIC) &&
+                 !of_fdt_device_is_available(blob, offset))
+                     continue;
+
              if (!populate_node(blob, offset, &mem, nps[depth],
                                 &nps[depth+1], dryrun))
                      return mem - base;

Hi Rob,

I strongly support the idea of this patch, but there may be an issue we have to resolve. I'm pretty sure we had talked about the issue a long time ago, and it has been sitting on my todo list.

Quoted 6 lines

We have two sets of node traversal macros and functions. One set honors the status property, and the other ignores it. If I recall our previous discussion properly, we want the normal usage to honor the status property and only a tiny (or maybe non-existent) set of locations to be allowed to ignore the status property.

Ignoring status is a bug for a static DT. There could be places that expect the node to be present, but disabled. Those may be bugs too.

Quoted 13 lines

A rough sense of how often the status property is honored or not is:

   $ git grep for_each_child_of_node | wc -l
   293
   $ git grep of_get_next_child | wc -l
   103

   $ git grep for_each_available_child_of_node | wc -l
   106
   $ git grep of_get_next_available_child | wc -l
   20

Quoted 4 lines

Many of the cases where the status flag is ignored will not actually encounter a node that is not available, so many of the cases where status is not checked could currently be checking status.

For many nodes, status simply makes no sense or at least is undefined.

Quoted 5 lines

And just for completeness, there are a number of standalone checks for whether a node is available:

   $ git grep of_device_is_available | wc -l
   128

I'm surprised it's that many. It's a low-level detail that the core should handle. We'd also need to make things like of_find_node_by_name honor status.

Quoted 15 lines

It will be a pain to manually check all of the sites that ignore the status property, but that task should be done.

In the mean time, maybe we could flush out the few cases that currently depend on ignoring the status property by

- making for_each_child_of_node() and of_get_next_child() actually check for valid status

- provide a temporary (one or two kernel release) CONFIG option to allow the old behavior for for_each_child_of_node() and of_get_next_child() just in case we miss any locations that need to be fixed

Quoted 6 lines

- fix up the few places in core device tree code that actually need to ignore status (if such places exist)

In the end, the *_available_*() interfaces should be removed, because the normal behavior of node traversal should be to only traverse nodes that are available.

I'm not sure this is really something we want or need to fix.

I could just make this depend on OF_KOBJ instead. Then practically no one would see any change as almost everyone enables sysfs (and in turn /proc/device-tree).

Rob

Add an equivalent function to of_device_is_available for flattened DT, and convert the one existing open coded occurrence.

Cc Nicolas Pitre <[email protected]>

Cc Frank Rowand <[email protected]>

Signed-off-by Rob Herring <[email protected]>

---
 drivers/of/fdt.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

drivers/of/fdt.c +14 -3

index 27c535af0be8..f8c39705418b 100644
@@ -132,6 +132,19 @@ bool of_fdt_is_big_endian(const void *blob, unsigned long node)
 	return false;
 }
 
+static bool of_fdt_device_is_available(const void *blob, unsigned long node)
+{
+	const char *status = fdt_getprop(blob, node, "status", NULL);
+
+	if (!status)
+		return true;
+
+	if (!strcmp(status, "ok") || !strcmp(status, "okay"))
+		return true;
+
+	return false;
+}
+
 /**
  * of_fdt_match - Return true if node matches a list of compatible values
  */

@@ -605,7 +618,6 @@ static int __init __fdt_scan_reserved_mem(unsigned long node, const char *uname,
 					  int depth, void *data)
 {
 	static int found;
-	const char *status;
 	int err;
 
 	if (!found && depth == 1 && strcmp(uname, "reserved-memory") == 0) {

@@ -625,8 +637,7 @@ static int __init __fdt_scan_reserved_mem(unsigned long node, const char *uname,
 		return 1;
 	}
 
-	status = of_get_flat_dt_prop(node, "status", NULL);
-	if (status && strcmp(status, "okay") != 0 && strcmp(status, "ok") != 0)
+	if (!of_fdt_device_is_available(initial_boot_params, node))
 		return 0;
 
 	err = __reserved_mem_reserve_reg(node, uname);

Signature · 1 line

2.11.0