Skip to contentSkip to conversation

remoteproc: core: Honor device tree /alias entries when assigning IDs

PATCH In review Nishanth Menon 4 participants 9 messages Aug 7, 2023 – Aug 23, 2023 Remote processor (remoteproc)

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 remoteproc: core: Honor device tree /alias entries when assigning IDs

On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.

Assign remoteproc device IDs based on device tree /aliases entries if present, falling back to the existing numbering scheme if there is no /aliases entry (which includes when the system isn't booted using DT) or a numbering conflict. If the alias node is not present, the driver behaves as before.

---
Test log: Beaglebone-AI64
https://gist.github.com/nmenon/365cf80d6c0685dd9be7c2cb18d7c937 (along with a test change to force the sequence of initialization)

The report occurred on Beagle discord channel - so I am not sure how to share the logs of the report.

 drivers/remoteproc/remoteproc_core.c | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)
drivers/remoteproc/remoteproc_core.c +21 -1
index 695cce218e8c..a12f3d37b8de 100644
@@ -19,6 +19,7 @@
 #include <linux/delay.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
+#include <linux/of.h>
 #include <linux/device.h>
 #include <linux/panic_notifier.h>
 #include <linux/slab.h>
@@ -2417,6 +2418,25 @@ static int rproc_alloc_ops(struct rproc *rproc, const struct rproc_ops *ops)
 	return 0;
 }
 
+static int rproc_device_get_id(struct device *dev)
+{
+	int of_id = -1, id = -1;
+
+	if (dev->of_node)
+		of_id = of_alias_get_id(dev->of_node, "remoteproc");
+
+	if (of_id >= 0) {
+		id = ida_simple_get(&rproc_dev_index, of_id, of_id + 1, GFP_KERNEL);
+		if (id < 0)
+			dev_warn(dev, "/aliases ID %d not available\n", of_id);
+	}
+
+	if (id < 0)
+		id = ida_alloc(&rproc_dev_index, GFP_KERNEL);
+
+	return id;
+}
+
 /**
  * rproc_alloc() - allocate a remote processor handle
  * @dev: the underlying device
@@ -2476,7 +2496,7 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
 		goto put_device;
 
 	/* Assign a unique device index and name */
-	rproc->index = ida_alloc(&rproc_dev_index, GFP_KERNEL);
+	rproc->index = rproc_device_get_id(dev);
 	if (rproc->index < 0) {
 		dev_err(dev, "ida_alloc failed: %d\n", rproc->index);
 		goto put_device;
Signature · 7 lines
2.40.0


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Reviewed-by on v1 1/1
#1 lore raw
Quoted 10 lines · Nishanth Menon

Assign remoteproc device IDs based on device tree /aliases entries if present, falling back to the existing numbering scheme if there is no /aliases entry (which includes when the system isn't booted using DT) or a numbering conflict. If the alias node is not present, the driver behaves as before.

---

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

maintainer
#2 lore raw
In reply to Nishanth Menon

Hi Nishanth,

Quoted 5 lines · Nishanth Menon

On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote

Quoted 9 lines

proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.

I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?

Thanks,
Mathieu

Quoted 70 lines

Assign remoteproc device IDs based on device tree /aliases entries if present, falling back to the existing numbering scheme if there is no /aliases entry (which includes when the system isn't booted using DT) or a numbering conflict. If the alias node is not present, the driver behaves as before.

---
Test log: Beaglebone-AI64
https://gist.github.com/nmenon/365cf80d6c0685dd9be7c2cb18d7c937 (along with a test change to force the sequence of initialization)

The report occurred on Beagle discord channel - so I am not sure how to share the logs of the report.

 drivers/remoteproc/remoteproc_core.c | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)
drivers/remoteproc/remoteproc_core.c +21 -1
index 695cce218e8c..a12f3d37b8de 100644
@@ -19,6 +19,7 @@
 #include <linux/delay.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
+#include <linux/of.h>
 #include <linux/device.h>
 #include <linux/panic_notifier.h>
 #include <linux/slab.h>
@@ -2417,6 +2418,25 @@ static int rproc_alloc_ops(struct rproc *rproc, const struct rproc_ops *ops)
 	return 0;
 }
 
+static int rproc_device_get_id(struct device *dev)
+{
+	int of_id = -1, id = -1;
+
+	if (dev->of_node)
+		of_id = of_alias_get_id(dev->of_node, "remoteproc");
+
+	if (of_id >= 0) {
+		id = ida_simple_get(&rproc_dev_index, of_id, of_id + 1, GFP_KERNEL);
+		if (id < 0)
+			dev_warn(dev, "/aliases ID %d not available\n", of_id);
+	}
+
+	if (id < 0)
+		id = ida_alloc(&rproc_dev_index, GFP_KERNEL);
+
+	return id;
+}
+
 /**
  * rproc_alloc() - allocate a remote processor handle
  * @dev: the underlying device
@@ -2476,7 +2496,7 @@ struct rproc *rproc_alloc(struct device *dev, const char *name,
 		goto put_device;
 
 	/* Assign a unique device index and name */
-	rproc->index = ida_alloc(&rproc_dev_index, GFP_KERNEL);
+	rproc->index = rproc_device_get_id(dev);
 	if (rproc->index < 0) {
 		dev_err(dev, "ida_alloc failed: %d\n", rproc->index);
 		goto put_device;
Signature · 1 line
2.40.0
#3 lore raw
Quoted 18 lines · Mathieu Poirier

Hi Nishanth,

Quoted 5 lines · Nishanth Menon

On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote

Quoted 9 lines

proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.

Quoted 6 lines

I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?

Correct - *if* remoteproc3 is the constant node reference.

To take a trivial example: We ran into this issue with:
https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28

remoteproc18 apparently changed numbering in a different build.

If remoteproc18 remained the same between different distro builds that would have probably kept the userspace constant. but it does'nt. it dependent purely on probe order, which does'nt let userspace remain consistent.

Same reason and motivation to do the following:
https://git.beagleboard.org/beagleboard/repos-arm64/-/blob/main/bb-customizations/suite/bookworm/debian/86-remoteproc-noroot.rules
in one technique to do it - but that only works if all the distros follow the same udev rules - and there is no reasonable way to enforce that across distributions.

Signature · 3 lines
Regards,
Nishanth Menon
Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3  1A34 DDB5 849D 1736 249D
#4 lore raw
Quoted 38 lines · Nishanth Menon
Quoted 18 lines · Mathieu Poirier

Hi Nishanth,

Quoted 5 lines · Nishanth Menon

On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote

Quoted 9 lines

proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.

Quoted 6 lines

I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?

Correct - *if* remoteproc3 is the constant node reference.

To take a trivial example: We ran into this issue with:
https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28

remoteproc18 apparently changed numbering in a different build.

If remoteproc18 remained the same between different distro builds that would have probably kept the userspace constant. but it does'nt. it dependent purely on probe order, which does'nt let userspace remain consistent.

Quoted 6 lines

Same reason and motivation to do the following:
https://git.beagleboard.org/beagleboard/repos-arm64/-/blob/main/bb-customizations/suite/bookworm/debian/86-remoteproc-noroot.rules
in one technique to do it - but that only works if all the distros follow the same udev rules - and there is no reasonable way to enforce that across distributions.

Enforcing distros to behave the same isn't the job of Device Tree, udev rules seems like a reasonable place. Anyone dealing with Linux should know they should not rely on kernel provided device names/numbers (like with disks, network interfaces, etc.).

If you want to have a path that will always work you could use:

/sys/devices/platform/bus@f4000/bus@f4000\:r5fss@78400000/78400000.r5f/remoteproc/

for the same. I don't like that it makes an ABI out of node names, but better than putting any more Linux configuration in DT IMHO.

Andrew

#5 lore raw
Quoted 51 lines · Andrew Davis
Quoted 38 lines · Nishanth Menon
Quoted 18 lines · Mathieu Poirier

Hi Nishanth,

Quoted 5 lines · Nishanth Menon

On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote

Quoted 9 lines

proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.

Quoted 6 lines

I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?

Correct - *if* remoteproc3 is the constant node reference.

To take a trivial example: We ran into this issue with:
https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28

remoteproc18 apparently changed numbering in a different build.

If remoteproc18 remained the same between different distro builds that would have probably kept the userspace constant. but it does'nt. it dependent purely on probe order, which does'nt let userspace remain consistent.

Quoted 6 lines

Same reason and motivation to do the following:
https://git.beagleboard.org/beagleboard/repos-arm64/-/blob/main/bb-customizations/suite/bookworm/debian/86-remoteproc-noroot.rules
in one technique to do it - but that only works if all the distros follow the same udev rules - and there is no reasonable way to enforce that across distributions.

Enforcing distros to behave the same isn't the job of Device Tree, udev rules seems like a reasonable place. Anyone dealing with Linux should know they should not rely on kernel provided device names/numbers (like with disks, network interfaces, etc.).

Quoted 6 lines

If you want to have a path that will always work you could use:

/sys/devices/platform/bus@f4000/bus@f4000\:r5fss@78400000/78400000.r5f/remoteproc/

for the same. I don't like that it makes an ABI out of node names, but better than putting any more Linux configuration in DT IMHO.

aliases are there for real reasons. So lets not confuse the two. End of the day being able to help userspace is what it does.

Signature · 3 lines
Regards,
Nishanth Menon
Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3  1A34 DDB5 849D 1736 249D
maintainer
#6 lore raw
In reply to Nishanth Menon
Quoted 26 lines · Nishanth Menon
Quoted 18 lines · Mathieu Poirier

Hi Nishanth,

Quoted 5 lines · Nishanth Menon

On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote

Quoted 9 lines

proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.

Quoted 6 lines

I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?

Quoted 7 lines

Correct - *if* remoteproc3 is the constant node reference.

To take a trivial example: We ran into this issue with:
https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28

remoteproc18 apparently changed numbering in a different build.

We are going around in circles. In the above link using an alias will guarantee that "remoteproc18" is available but won't guarantee the functionality enacted by the FW loaded in that remote processor, which is distro dependent.

Quoted 18 lines

If remoteproc18 remained the same between different distro builds that would have probably kept the userspace constant. but it does'nt. it dependent purely on probe order, which does'nt let userspace remain consistent.

Same reason and motivation to do the following:
https://git.beagleboard.org/beagleboard/repos-arm64/-/blob/main/bb-customizations/suite/bookworm/debian/86-remoteproc-noroot.rules
in one technique to do it - but that only works if all the distros follow the same udev rules - and there is no reasonable way to enforce that across distributions.

Signature · 3 lines
Regards,
Nishanth Menon
Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3  1A34 DDB5 849D 1736 249D
#7 lore raw
Quoted 32 lines · Mathieu Poirier
Quoted 31 lines · Nishanth Menon
Quoted 18 lines · Mathieu Poirier

Hi Nishanth,

Quoted 5 lines · Nishanth Menon

On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote

Quoted 9 lines

proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.

Quoted 6 lines

I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?

Correct - *if* remoteproc3 is the constant node reference.

To take a trivial example: We ran into this issue with:
https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28

Quoted 7 lines
Quoted 2 lines

remoteproc18 apparently changed numbering in a different build.

We are going around in circles. In the above link using an alias will guarantee that "remoteproc18" is available but won't guarantee the functionality enacted by the FW loaded in that remote processor, which is distro dependent.

Apologies, but I am trying to comprehend the relationship and probably am failing to see the linkage. Let me try:

If I understand you correctly, you are concerned that distros do not have a mechanism to provide consistent firmware to the correct remote proc for a specific functionality..

if so, distro loads / provides the requisite firmware. How the package distribution scheme works to distribute the firmware and versioning provided varies - One typical pattern has been to use linux-firmware repo[1] (at least in other domains - say GPU, wlink or the likes) and provide package distribution. The other pattern could be build and deploy based on tag (this would be no different from any other package deployment).

On the other hand, If we are looking at the fact that there can be different types of firmware that could be loaded to a remoteproc providing different functionality - that is correct, and at least in case of TI processors very valid - something like openAMP endpoint solutions probably help?

Let me know if I am off-track here..

[1] https://git.ti.com/cgit/processor-firmware/ti-linux-firmware/tree/ti-ipc?h=ti-linux-firmware

Signature · 3 lines
Regards,
Nishanth Menon
Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3  1A34 DDB5 849D 1736 249D
maintainer
#8 lore raw
Quoted 41 lines · Nishanth Menon
Quoted 32 lines · Mathieu Poirier
Quoted 31 lines · Nishanth Menon
Quoted 18 lines · Mathieu Poirier

Hi Nishanth,

Quoted 5 lines · Nishanth Menon

On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote

Quoted 9 lines

proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.

Quoted 6 lines

I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?

Correct - *if* remoteproc3 is the constant node reference.

To take a trivial example: We ran into this issue with:
https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28

Quoted 7 lines
Quoted 2 lines

remoteproc18 apparently changed numbering in a different build.

We are going around in circles. In the above link using an alias will guarantee that "remoteproc18" is available but won't guarantee the functionality enacted by the FW loaded in that remote processor, which is distro dependent.

Quoted 7 lines

Apologies, but I am trying to comprehend the relationship and probably am failing to see the linkage. Let me try:

If I understand you correctly, you are concerned that distros do not have a mechanism to provide consistent firmware to the correct remote proc for a specific functionality..

The point is that aliases will guarantee a naming convention for remote processors but won't guarantee their functionality. Sure, we can add aliases but it won't solve all your problems.

Quoted 8 lines

if so, distro loads / provides the requisite firmware. How the package distribution scheme works to distribute the firmware and versioning provided varies - One typical pattern has been to use linux-firmware repo[1] (at least in other domains - say GPU, wlink or the likes) and provide package distribution. The other pattern could be build and deploy based on tag (this would be no different from any other package deployment).

Quoted 4 lines

On the other hand, If we are looking at the fact that there can be different types of firmware that could be loaded to a remoteproc providing different functionality - that is correct, and at least in case of TI processors very valid

That is exactly what I am referring to.

Quoted 2 lines

- something like openAMP endpoint
solutions probably help?

I am not familiar with openAmP endpoints but certainly willing to consider it as an option.

Quoted 3 lines

Let me know if I am off-track here..

You are on track.

Quoted 5 lines

[1] https://git.ti.com/cgit/processor-firmware/ti-linux-firmware/tree/ti-ipc?h=ti-linux-firmware

Signature · 3 lines
Regards,
Nishanth Menon
Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3  1A34 DDB5 849D 1736 249D