remoteproc: core: Honor device tree /alias entries when assigning IDs
PATCH 1/1 remoteproc: core: Honor device tree /alias entries when assigning IDs
On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.
Assign remoteproc device IDs based on device tree /aliases entries if present, falling back to the existing numbering scheme if there is no /aliases entry (which includes when the system isn't booted using DT) or a numbering conflict. If the alias node is not present, the driver behaves as before.
---
Test log: Beaglebone-AI64
https://gist.github.com/nmenon/365cf80d6c0685dd9be7c2cb18d7c937 (along with a test change to force the sequence of initialization)
The report occurred on Beagle discord channel - so I am not sure how to share the logs of the report.
drivers/remoteproc/remoteproc_core.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-)
drivers/remoteproc/remoteproc_core.c +21 -1
@@ -19,6 +19,7 @@#include <linux/delay.h> #include <linux/kernel.h> #include <linux/module.h> +#include <linux/of.h> #include <linux/device.h> #include <linux/panic_notifier.h> #include <linux/slab.h>
@@ -2417,6 +2418,25 @@ static int rproc_alloc_ops(struct rproc *rproc, const struct rproc_ops *ops)return 0 ; } +static int rproc_device_get_id (struct device *dev) +{ +int of_id =-1 , id =-1 ; + +if (dev->of_node) + of_id =of_alias_get_id (dev->of_node,"remoteproc" ); + +if (of_id >=0 ) { + id =ida_simple_get (&rproc_dev_index, of_id, of_id +1 ,GFP_KERNEL ); +if (id <0 ) +dev_warn (dev,"/aliases ID %d not available\n" , of_id); + } + +if (id <0 ) + id =ida_alloc (&rproc_dev_index,GFP_KERNEL ); + +return id; +} +/** * rproc_alloc() - allocate a remote processor handle * @dev: the underlying device
@@ -2476,7 +2496,7 @@ struct rproc *rproc_alloc(struct device *dev, const char *name, goto put_device;/* Assign a unique device index and name */ - rproc->index =ida_alloc (&rproc_dev_index,GFP_KERNEL ); + rproc->index =rproc_device_get_id (dev);if (rproc->index <0 ) {dev_err (dev,"ida_alloc failed: %d\n" , rproc->index); goto put_device;
Signature · 7 lines
2.40.0 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 10 lines · Nishanth Menon
Assign remoteproc device IDs based on device tree /aliases entries if present, falling back to the existing numbering scheme if there is no /aliases entry (which includes when the system isn't booted using DT) or a numbering conflict. If the alias node is not present, the driver behaves as before.
Cc Robert Nelson<[email protected]>Reported-by Kevin Cahalan<[email protected]>Signed-off-by Nishanth Menon<[email protected]>---
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Mathieu Poirier
maintainerHi Nishanth,
Quoted 5 lines · Nishanth Menon
On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote
Quoted 9 lines
proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.
I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?
Thanks,
Mathieu
Quoted 70 lines
Assign remoteproc device IDs based on device tree /aliases entries if present, falling back to the existing numbering scheme if there is no /aliases entry (which includes when the system isn't booted using DT) or a numbering conflict. If the alias node is not present, the driver behaves as before.
Cc Robert Nelson <[email protected]>Reported-by Kevin Cahalan <[email protected]>Signed-off-by Nishanth Menon <[email protected]>---
Test log: Beaglebone-AI64
https://gist.github.com/nmenon/365cf80d6c0685dd9be7c2cb18d7c937 (along with a test change to force the sequence of initialization)The report occurred on Beagle discord channel - so I am not sure how to share the logs of the report.
drivers/remoteproc/remoteproc_core.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-)drivers/remoteproc/remoteproc_core.c +21
-1@@ -19,6 +19,7 @@#include <linux/delay.h> #include <linux/kernel.h> #include <linux/module.h> +#include <linux/of.h> #include <linux/device.h> #include <linux/panic_notifier.h> #include <linux/slab.h> @@ -2417,6 +2418,25 @@ static int rproc_alloc_ops(struct rproc *rproc, const struct rproc_ops *ops)return 0 ; } +static int rproc_device_get_id (struct device *dev) +{ +int of_id =-1 , id =-1 ; + +if (dev->of_node) + of_id =of_alias_get_id (dev->of_node,"remoteproc" ); + +if (of_id >=0 ) { + id =ida_simple_get (&rproc_dev_index, of_id, of_id +1 ,GFP_KERNEL ); +if (id <0 ) +dev_warn (dev,"/aliases ID %d not available\n" , of_id); + } + +if (id <0 ) + id =ida_alloc (&rproc_dev_index,GFP_KERNEL ); + +return id; +} +/** * rproc_alloc() - allocate a remote processor handle * @dev: the underlying device @@ -2476,7 +2496,7 @@ struct rproc *rproc_alloc(struct device *dev, const char *name, goto put_device;/* Assign a unique device index and name */ - rproc->index =ida_alloc (&rproc_dev_index,GFP_KERNEL ); + rproc->index =rproc_device_get_id (dev);if (rproc->index <0 ) {dev_err (dev,"ida_alloc failed: %d\n" , rproc->index); goto put_device;Signature · 1 line
2.40.0
Quoted 18 lines · Mathieu Poirier
Hi Nishanth,
Quoted 5 lines · Nishanth Menon
On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote
Quoted 9 lines
proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.
Quoted 6 lines
I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?
Correct - *if* remoteproc3 is the constant node reference.
To take a trivial example: We ran into this issue with:
https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28
remoteproc18 apparently changed numbering in a different build.
If remoteproc18 remained the same between different distro builds that would have probably kept the userspace constant. but it does'nt. it dependent purely on probe order, which does'nt let userspace remain consistent.
Same reason and motivation to do the following:
https://git.beagleboard.org/beagleboard/repos-arm64/-/blob/main/bb-customizations/suite/bookworm/debian/86-remoteproc-noroot.rules
in one technique to do it - but that only works if all the distros follow the same udev rules - and there is no reasonable way to enforce that across distributions.
Signature · 3 lines
Regards, Nishanth Menon Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3 1A34 DDB5 849D 1736 249D
Quoted 38 lines · Nishanth Menon
Quoted 18 lines · Mathieu Poirier
Hi Nishanth,
Quoted 5 lines · Nishanth Menon
On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote
Quoted 9 lines
proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.
Quoted 6 lines
I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?
Correct - *if* remoteproc3 is the constant node reference.
To take a trivial example: We ran into this issue with:
https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28remoteproc18 apparently changed numbering in a different build.
If remoteproc18 remained the same between different distro builds that would have probably kept the userspace constant. but it does'nt. it dependent purely on probe order, which does'nt let userspace remain consistent.
Quoted 6 lines
Same reason and motivation to do the following:
https://git.beagleboard.org/beagleboard/repos-arm64/-/blob/main/bb-customizations/suite/bookworm/debian/86-remoteproc-noroot.rules
in one technique to do it - but that only works if all the distros follow the same udev rules - and there is no reasonable way to enforce that across distributions.
Enforcing distros to behave the same isn't the job of Device Tree, udev rules seems like a reasonable place. Anyone dealing with Linux should know they should not rely on kernel provided device names/numbers (like with disks, network interfaces, etc.).
If you want to have a path that will always work you could use:
/sys/devices/platform/bus@f4000/bus@f4000\:r5fss@78400000/78400000.r5f/remoteproc/
for the same. I don't like that it makes an ABI out of node names, but better than putting any more Linux configuration in DT IMHO.
Andrew
Quoted 51 lines · Andrew Davis
Quoted 38 lines · Nishanth Menon
Quoted 18 lines · Mathieu Poirier
Hi Nishanth,
Quoted 5 lines · Nishanth Menon
On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote
Quoted 9 lines
proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.
Quoted 6 lines
I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?
Correct - *if* remoteproc3 is the constant node reference.
To take a trivial example: We ran into this issue with:
https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28remoteproc18 apparently changed numbering in a different build.
If remoteproc18 remained the same between different distro builds that would have probably kept the userspace constant. but it does'nt. it dependent purely on probe order, which does'nt let userspace remain consistent.
Quoted 6 lines
Same reason and motivation to do the following:
https://git.beagleboard.org/beagleboard/repos-arm64/-/blob/main/bb-customizations/suite/bookworm/debian/86-remoteproc-noroot.rules
in one technique to do it - but that only works if all the distros follow the same udev rules - and there is no reasonable way to enforce that across distributions.Enforcing distros to behave the same isn't the job of Device Tree, udev rules seems like a reasonable place. Anyone dealing with Linux should know they should not rely on kernel provided device names/numbers (like with disks, network interfaces, etc.).
Quoted 6 lines
If you want to have a path that will always work you could use:
/sys/devices/platform/bus@f4000/bus@f4000\:r5fss@78400000/78400000.r5f/remoteproc/
for the same. I don't like that it makes an ABI out of node names, but better than putting any more Linux configuration in DT IMHO.
aliases are there for real reasons. So lets not confuse the two. End of the day being able to help userspace is what it does.
Signature · 3 lines
Regards, Nishanth Menon Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3 1A34 DDB5 849D 1736 249D
Mathieu Poirier
maintainerQuoted 26 lines · Nishanth Menon
Quoted 18 lines · Mathieu Poirier
Hi Nishanth,
Quoted 5 lines · Nishanth Menon
On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote
Quoted 9 lines
proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.
Quoted 6 lines
I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?
Quoted 7 lines
Correct - *if* remoteproc3 is the constant node reference.
To take a trivial example: We ran into this issue with:
https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28remoteproc18 apparently changed numbering in a different build.
We are going around in circles. In the above link using an alias will guarantee that "remoteproc18" is available but won't guarantee the functionality enacted by the FW loaded in that remote processor, which is distro dependent.
Quoted 18 lines
If remoteproc18 remained the same between different distro builds that would have probably kept the userspace constant. but it does'nt. it dependent purely on probe order, which does'nt let userspace remain consistent.
Same reason and motivation to do the following:
https://git.beagleboard.org/beagleboard/repos-arm64/-/blob/main/bb-customizations/suite/bookworm/debian/86-remoteproc-noroot.rules
in one technique to do it - but that only works if all the distros follow the same udev rules - and there is no reasonable way to enforce that across distributions.Signature · 3 lines
Regards, Nishanth Menon Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3 1A34 DDB5 849D 1736 249D
Quoted 32 lines · Mathieu Poirier
Quoted 31 lines · Nishanth Menon
Quoted 18 lines · Mathieu Poirier
Hi Nishanth,
Quoted 5 lines · Nishanth Menon
On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote
Quoted 9 lines
proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.
Quoted 6 lines
I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?
Correct - *if* remoteproc3 is the constant node reference.
To take a trivial example: We ran into this issue with:
https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28
Quoted 7 lines
Quoted 2 lines
remoteproc18 apparently changed numbering in a different build.
We are going around in circles. In the above link using an alias will guarantee that "remoteproc18" is available but won't guarantee the functionality enacted by the FW loaded in that remote processor, which is distro dependent.
Apologies, but I am trying to comprehend the relationship and probably am failing to see the linkage. Let me try:
If I understand you correctly, you are concerned that distros do not have a mechanism to provide consistent firmware to the correct remote proc for a specific functionality..
if so, distro loads / provides the requisite firmware. How the package distribution scheme works to distribute the firmware and versioning provided varies - One typical pattern has been to use linux-firmware repo[1] (at least in other domains - say GPU, wlink or the likes) and provide package distribution. The other pattern could be build and deploy based on tag (this would be no different from any other package deployment).
On the other hand, If we are looking at the fact that there can be different types of firmware that could be loaded to a remoteproc providing different functionality - that is correct, and at least in case of TI processors very valid - something like openAMP endpoint solutions probably help?
Let me know if I am off-track here..
[1] https://git.ti.com/cgit/processor-firmware/ti-linux-firmware/tree/ti-ipc?h=ti-linux-firmware
Signature · 3 lines
Regards, Nishanth Menon Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3 1A34 DDB5 849D 1736 249D
Mathieu Poirier
maintainerQuoted 41 lines · Nishanth Menon
Quoted 32 lines · Mathieu Poirier
Quoted 31 lines · Nishanth Menon
Quoted 18 lines · Mathieu Poirier
Hi Nishanth,
Quoted 5 lines · Nishanth Menon
On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote
Quoted 9 lines
proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems.
Quoted 6 lines
I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits?
Correct - *if* remoteproc3 is the constant node reference.
To take a trivial example: We ran into this issue with:
https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28Quoted 7 lines
Quoted 2 lines
remoteproc18 apparently changed numbering in a different build.
We are going around in circles. In the above link using an alias will guarantee that "remoteproc18" is available but won't guarantee the functionality enacted by the FW loaded in that remote processor, which is distro dependent.
Quoted 7 lines
Apologies, but I am trying to comprehend the relationship and probably am failing to see the linkage. Let me try:
If I understand you correctly, you are concerned that distros do not have a mechanism to provide consistent firmware to the correct remote proc for a specific functionality..
The point is that aliases will guarantee a naming convention for remote processors but won't guarantee their functionality. Sure, we can add aliases but it won't solve all your problems.
Quoted 8 lines
if so, distro loads / provides the requisite firmware. How the package distribution scheme works to distribute the firmware and versioning provided varies - One typical pattern has been to use linux-firmware repo[1] (at least in other domains - say GPU, wlink or the likes) and provide package distribution. The other pattern could be build and deploy based on tag (this would be no different from any other package deployment).
Quoted 4 lines
On the other hand, If we are looking at the fact that there can be different types of firmware that could be loaded to a remoteproc providing different functionality - that is correct, and at least in case of TI processors very valid
That is exactly what I am referring to.
Quoted 2 lines
- something like openAMP endpoint
solutions probably help?
I am not familiar with openAmP endpoints but certainly willing to consider it as an option.
Quoted 3 lines
Let me know if I am off-track here..
You are on track.
Quoted 5 lines
[1] https://git.ti.com/cgit/processor-firmware/ti-linux-firmware/tree/ti-ipc?h=ti-linux-firmware
Signature · 3 lines
Regards, Nishanth Menon Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3 1A34 DDB5 849D 1736 249D