imx: bootaux elf firmware support

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux ${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ################################################## 88.3
KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ...
Colibri iMX7 #

Note that the bootaux command retrieved the entry point (PC) from the elf binary.

Also all sections are translated to A7 addresses
in order to properly load the firmware sections into the appropriate locations.

Also cache flushes is taken care of, so that there is no manual dcache flush necessary anymore.

The NXP FreeRTOS BSP already generates elf binaries which can be directly used with this elf binary support.

The last patch adds bootaux support for Vybrid.

Stefan Agner (3):
imx: imx-common: move aux core image parsing to common code
imx: imx-common: add elf firmware support
ARM: vf610: add auxiliary core boot support

 arch/arm/cpu/armv7/mx6/soc.c                |  30 +++++---
 arch/arm/cpu/armv7/mx7/soc.c                |  34 ++++++---
 arch/arm/cpu/armv7/vf610/generic.c          |  42 +++++++++++
 arch/arm/imx-common/Kconfig                 |   2 +-
 arch/arm/imx-common/Makefile                |   4 +-
 arch/arm/imx-common/imx_bootaux.c           | 105
+++++++++++++++++++++++-----
arch/arm/include/asm/arch-vf610/sys_proto.h |   8 +++
arch/arm/include/asm/imx-common/sys_proto.h |   6 ++
configs/colibri_vf_defconfig                |   1 + 9 files changed,

198 insertions(+), 34 deletions(-) create mode 100644 arch/arm/include/asm/arch-vf610/sys_proto.h

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 8 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux ${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ################################################## 88.3
KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ...
Colibri iMX7 #

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Hence, I'm wondering if you could make those patches usable for other platforms as well?

Quoted 4 lines

Note that the bootaux command retrieved the entry point (PC) from the elf binary.

Could you make this code flexible enough to handle not only elf binaries, but also other data (e.g. FPGA bitstreams)?

Maybe it would better to:
-------------------------

Embrace those binaries into FIT file (*.itb)? And allow multiple binaries loading? I'm thinking of work similar to one already did by Andre Przywara for SPL:

"[PATCH v3 00/19] SPL: extend FIT loading support" posted on 31.03.2017?

In that way we would "open" many new use cases, and other platforms (e.g. FPGA's, TI, etc) could benefit from it.
One solid use case is to load different Linux binaries (or/and bare metal programs) to different SoC cores.

The "meta data" (i.e. load address, data type, description), could be extracted from the FIT format (the code is already in u-boot).

IMHO, this is very generic approach.

Quoted 4 lines

Also all sections are translated to A7 addresses
in order to properly load the firmware sections into the appropriate locations.

This would require some tiny arch specific code.

Quoted 24 lines

Also cache flushes is taken care of, so that there is no manual dcache flush necessary anymore.

The NXP FreeRTOS BSP already generates elf binaries which can be directly used with this elf binary support.

The last patch adds bootaux support for Vybrid.

Stefan Agner (3):
imx: imx-common: move aux core image parsing to common code
imx: imx-common: add elf firmware support
ARM: vf610: add auxiliary core boot support

 arch/arm/cpu/armv7/mx6/soc.c                |  30 +++++---
 arch/arm/cpu/armv7/mx7/soc.c                |  34 ++++++---
 arch/arm/cpu/armv7/vf610/generic.c          |  42 +++++++++++
 arch/arm/imx-common/Kconfig                 |   2 +-
 arch/arm/imx-common/Makefile                |   4 +-
 arch/arm/imx-common/imx_bootaux.c           | 105
+++++++++++++++++++++++-----
arch/arm/include/asm/arch-vf610/sys_proto.h |   8 +++
arch/arm/include/asm/imx-common/sys_proto.h |   6 ++
configs/colibri_vf_defconfig                |   1 + 9 files changed,

Quoted 3 lines

198 insertions(+), 34 deletions(-) create mode 100644 arch/arm/include/asm/arch-vf610/sys_proto.h

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de

Hi Lukasz,

Quoted 25 lines · Lukasz Majewski

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 8 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux ${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ################################################## 88.3
KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ...
Colibri iMX7 #

Quoted 8 lines

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Hence, I'm wondering if you could make those patches usable for other platforms as well?

I don't think that this is a good idea. bootaux is meant for auxiliary cores, which often use a different architecture and are not cache coherent (hence the cache flushes).

On SMP systems the main operating system normally starts the secondary core. Otherwise, if you want to run them separately using U-Boot, maybe a new command such as bootsmp would be more suited.

Quoted 41 lines · Stefan Agner

Hi Lukasz,

Quoted 25 lines · Lukasz Majewski

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 8 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux ${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ################################################## 88.3
KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ...
Colibri iMX7 #

Quoted 8 lines

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Hence, I'm wondering if you could make those patches usable for other platforms as well?

I don't think that this is a good idea. bootaux is meant for auxiliary cores, which often use a different architecture and are not cache coherent (hence the cache flushes).

Quoted 4 lines

On SMP systems the main operating system normally starts the secondary core. Otherwise, if you want to run them separately using U-Boot, maybe a new command such as bootsmp would be more suited.

Admitedly, I didn't look at the patch, but if you want to boot ad-hoc cores, you can very well also boot secondary cores on the current CPU complex with the same command. Why not ?

Also, I think this might come useful when booting stuff like "Altera Sparrow" ...

Quoted 42 lines · Marek Vasut

Quoted 41 lines · Stefan Agner

Hi Lukasz,

Quoted 25 lines · Lukasz Majewski

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 8 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux ${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ################################################## 88.3
KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ...
Colibri iMX7 #

Quoted 8 lines

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Hence, I'm wondering if you could make those patches usable for other platforms as well?

I don't think that this is a good idea. bootaux is meant for auxiliary cores, which often use a different architecture and are not cache coherent (hence the cache flushes).

Quoted 7 lines

Quoted 4 lines

On SMP systems the main operating system normally starts the secondary core. Otherwise, if you want to run them separately using U-Boot, maybe a new command such as bootsmp would be more suited.

Admitedly, I didn't look at the patch, but if you want to boot ad-hoc cores, you can very well also boot secondary cores on the current CPU complex with the same command. Why not ?

Sure, it could be done. I just feel it is not the right design.

Auxiliary cores have usually a different view to memory, this is why I had to add the get_host_mapping callback in the elf loader code to let architecture dependent code translate to host addresses. SMP systems don't need that.

Also flush caches is not necessary on some cache coherent CPU's (depending on how your cache coherence between I and D cache looks like).

Creating a new command like bootaux comes with very few overhead.

This are the reasons why I feel creating a new command for a SMP boot case makes more sense. We can still reuse functions which are very similar by moving them into some common location, where it makes sense.

Quoted 3 lines

Also, I think this might come useful when booting stuff like "Altera Sparrow" ...

I am not familiar with that architecture, what kind of core does it provide which needs to be booted by U-Boot?

Quoted 58 lines · Stefan Agner

Quoted 42 lines · Marek Vasut

Quoted 41 lines · Stefan Agner

Hi Lukasz,

Quoted 25 lines · Lukasz Majewski

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 8 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux ${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ################################################## 88.3
KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ...
Colibri iMX7 #

Quoted 8 lines

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Hence, I'm wondering if you could make those patches usable for other platforms as well?

I don't think that this is a good idea. bootaux is meant for auxiliary cores, which often use a different architecture and are not cache coherent (hence the cache flushes).

Quoted 7 lines

Quoted 4 lines

On SMP systems the main operating system normally starts the secondary core. Otherwise, if you want to run them separately using U-Boot, maybe a new command such as bootsmp would be more suited.

Admitedly, I didn't look at the patch, but if you want to boot ad-hoc cores, you can very well also boot secondary cores on the current CPU complex with the same command. Why not ?

Sure, it could be done. I just feel it is not the right design.

Auxiliary cores have usually a different view to memory, this is why I had to add the get_host_mapping callback in the elf loader code to let architecture dependent code translate to host addresses. SMP systems don't need that.

Quoted 3 lines

Also flush caches is not necessary on some cache coherent CPU's (depending on how your cache coherence between I and D cache looks like).

So SMP is just a reduced special-case of this , yes ?

Quoted 2 lines

Creating a new command like bootaux comes with very few overhead.

The overhead is the new command, we already have many ad-hoc commands.

Quoted 5 lines

This are the reasons why I feel creating a new command for a SMP boot case makes more sense. We can still reuse functions which are very similar by moving them into some common location, where it makes sense.

Quoted 1 line

Quoted 5 lines

Quoted 2 lines

Also, I think this might come useful when booting stuff like "Altera Sparrow" ...

I am not familiar with that architecture, what kind of core does it provide which needs to be booted by U-Boot?

The secondary ARM core in the SoCFPGA C-A9 complex or Nios2 core in the FPGA.

Quoted 62 lines · Stefan Agner

Quoted 61 lines · Marek Vasut

Quoted 44 lines · Stefan Agner

Quoted 43 lines

Hi Lukasz,

Quoted 29 lines · Lukasz Majewski

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 9 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux
${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ##################################################
88.3 KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ... Colibri iMX7 #

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

Quoted 6 lines

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Hence, I'm wondering if you could make those patches usable for other platforms as well?

I don't think that this is a good idea. bootaux is meant for auxiliary cores, which often use a different architecture and are not cache coherent (hence the cache flushes).

Quoted 8 lines

Quoted 5 lines

On SMP systems the main operating system normally starts the secondary core. Otherwise, if you want to run them separately using U-Boot, maybe a new command such as bootsmp would be more suited.

Admitedly, I didn't look at the patch, but if you want to boot ad-hoc cores, you can very well also boot secondary cores on the current CPU complex with the same command. Why not ?

Sure, it could be done. I just feel it is not the right design.

Auxiliary cores have usually a different view to memory, this is why I had to add the get_host_mapping callback in the elf loader code to let architecture dependent code translate to host addresses. SMP systems don't need that.

Quoted 5 lines

Quoted 3 lines

Also flush caches is not necessary on some cache coherent CPU's (depending on how your cache coherence between I and D cache looks like).

So SMP is just a reduced special-case of this , yes ?

Yeah, I guess you can get away with dummy callback implementation and a performance hit due to cash flushes.

Quoted 3 lines

Quoted 2 lines

Creating a new command like bootaux comes with very few overhead.

Quoted 3 lines

The overhead is the new command, we already have many ad-hoc commands.

Agreed, and I really wished that this got discussed when that command initially got added. I brought it up back then...
https://lists.denx.de/pipermail/u-boot/2016-January/240323.html

It seemed to be acceptable to just add this ad hoc command, with some "random binary format" support back then... Ok, it is not entirely random, since it is the format of a binary how it ends up in microcontrollers execute in place flash (stack pointer and reset vector are the two first words).... However, making this ad hoc command now generic really feels weird to me, since we would end up supporting that format for A class CPUs etc... bootaux is really suited for auxiliary M-class cores on ARM, as it is right now. Maybe we should have named it bootm ;-)

Quoted 7 lines

Quoted 7 lines

This are the reasons why I feel creating a new command for a SMP boot case makes more sense. We can still reuse functions which are very similar by moving them into some common location, where it makes sense.

Quoted 1 line

Quoted 8 lines

Quoted 5 lines

Quoted 2 lines

Also, I think this might come useful when booting stuff like "Altera Sparrow" ...

I am not familiar with that architecture, what kind of core does it provide which needs to be booted by U-Boot?

The secondary ARM core in the SoCFPGA C-A9 complex or Nios2 core in the FPGA.

In my thinking, the Nios2 core seems like such a remote processor well suited for the bootaux command. For the secondary A9, I would create a new command.

If we want to support the two with the same command, we already have a first problem: How do we address them? Of course, we could add just a index or something, but this would already break backward compatibility of the bootaux command.

--
Stefan

Quoted 11 lines

Quoted 11 lines

Quoted 3 lines

Admitedly, I didn't look at the patch, but if you want to boot ad-hoc cores, you can very well also boot secondary cores on the current CPU complex with the same command. Why not ?

Sure, it could be done. I just feel it is not the right design.

Auxiliary cores have usually a different view to memory, this is why I had to add the get_host_mapping callback in the elf loader code to let architecture dependent code translate to host addresses. SMP systems don't need that.

Quoted 5 lines

Quoted 3 lines

Also flush caches is not necessary on some cache coherent CPU's (depending on how your cache coherence between I and D cache looks like).

So SMP is just a reduced special-case of this , yes ?

Yeah, I guess you can get away with dummy callback implementation and a performance hit due to cash flushes.

Quoted 4 lines

Quoted 1 line

Creating a new command like bootaux comes with very few overhead.

The overhead is the new command, we already have many ad-hoc commands.

Agreed, and I really wished that this got discussed when that command initially got added. I brought it up back then...
https://lists.denx.de/pipermail/u-boot/2016-January/240323.html

It seemed to be acceptable to just add this ad hoc command, with some "random binary format" support back then...

Ok, it is not entirely
random, since it is the format of a binary how it ends up in microcontrollers execute in place flash (stack pointer and reset vector are the two first words)....

However, making this ad hoc command now
generic really feels weird to me, since we would end up supporting that format for A class CPUs etc... bootaux is really suited for auxiliary M-class cores on ARM, as it is right now. Maybe we should have named it bootm ;-)

Quoted 11 lines

Quoted 10 lines

This are the reasons why I feel creating a new command for a SMP boot case makes more sense. We can still reuse functions which are very similar by moving them into some common location, where it makes sense.

Quoted 3 lines

Also, I think this might come useful when booting stuff like "Altera Sparrow" ...

I am not familiar with that architecture, what kind of core does it provide which needs to be booted by U-Boot?

Quoted 2 lines

The secondary ARM core in the SoCFPGA C-A9 complex or Nios2 core in the FPGA.

In my thinking, the Nios2 core seems like such a remote processor well suited for the bootaux command. For the secondary A9, I would create a new command.

If we want to support the two with the same command, we already have a first problem: How do we address them? Of course, we could add just a index or something, but this would already break backward compatibility of the bootaux command.

On 04/04/2017 02:02 AM, Stefan Agner wrote:
[...]

Quoted 15 lines

Quoted 15 lines

Quoted 14 lines

Quoted 3 lines

Admitedly, I didn't look at the patch, but if you want to boot ad-hoc cores, you can very well also boot secondary cores on the current CPU complex with the same command. Why not ?

Sure, it could be done. I just feel it is not the right design.

Auxiliary cores have usually a different view to memory, this is why I had to add the get_host_mapping callback in the elf loader code to let architecture dependent code translate to host addresses. SMP systems don't need that.

Also flush caches is not necessary on some cache coherent CPU's (depending on how your cache coherence between I and D cache looks like).

Quoted 4 lines

Quoted 1 line

So SMP is just a reduced special-case of this , yes ?

Yeah, I guess you can get away with dummy callback implementation and a performance hit due to cash flushes.

Or you can abstract things out ?

Quoted 9 lines

Quoted 4 lines

Quoted 1 line

Creating a new command like bootaux comes with very few overhead.

The overhead is the new command, we already have many ad-hoc commands.

Agreed, and I really wished that this got discussed when that command initially got added. I brought it up back then...
https://lists.denx.de/pipermail/u-boot/2016-January/240323.html

Quoted 2 lines

It seemed to be acceptable to just add this ad hoc command, with some "random binary format" support back then...

Based on that discussion, I only see that noone opposed, but I don't see any agreement.

Quoted 4 lines

Ok, it is not entirely
random, since it is the format of a binary how it ends up in microcontrollers execute in place flash (stack pointer and reset vector are the two first words)....

I thought this command was starting the core by loading data to RAM , not flash ?

Quoted 5 lines

However, making this ad hoc command now
generic really feels weird to me, since we would end up supporting that format for A class CPUs etc... bootaux is really suited for auxiliary M-class cores on ARM, as it is right now. Maybe we should have named it bootm ;-)

We always try to avoid one-off hacks, so it's not weird. I still don't quite understand how it is a problem to abstract things out . I am not asking you to support CA, but to avoid CM specific implementation which cannot be extended to support CA or even MIPS/Nios2/etc .

Quoted 14 lines

Quoted 13 lines

Quoted 10 lines

This are the reasons why I feel creating a new command for a SMP boot case makes more sense. We can still reuse functions which are very similar by moving them into some common location, where it makes sense.

Quoted 3 lines

Also, I think this might come useful when booting stuff like "Altera Sparrow" ...

I am not familiar with that architecture, what kind of core does it provide which needs to be booted by U-Boot?

The secondary ARM core in the SoCFPGA C-A9 complex or Nios2 core in the FPGA.

Quoted 3 lines

In my thinking, the Nios2 core seems like such a remote processor well suited for the bootaux command. For the secondary A9, I would create a new command.

Uh, why, that does not make any sense to me. Both the random core in FPGA and the secondary CA9 core have almost the same bringup sequence. What is so different about this stuff ?

Quoted 4 lines

If we want to support the two with the same command, we already have a first problem: How do we address them? Of course, we could add just a index or something, but this would already break backward compatibility of the bootaux command.

So we already accepted a command which has shit commandline semantics and we're stuck with it , hrm. You can always specify a default core

index for those two iMX boards and it's done. Or just fix the semantics in their default environment, which is not awesome, but I doubt this is wildly used yet.

Quoted 19 lines · Marek Vasut

On 04/04/2017 02:02 AM, Stefan Agner wrote:
[...]

Quoted 17 lines

Quoted 16 lines

Quoted 6 lines

Quoted 3 lines

Admitedly, I didn't look at the patch, but if you want to boot ad-hoc cores, you can very well also boot secondary cores on the current CPU complex with the same command. Why not ?

Sure, it could be done. I just feel it is not the right design.

Quoted 8 lines

Auxiliary cores have usually a different view to memory, this is why I had to add the get_host_mapping callback in the elf loader code to let architecture dependent code translate to host addresses. SMP systems don't need that.

Also flush caches is not necessary on some cache coherent CPU's (depending on how your cache coherence between I and D cache looks like).

So SMP is just a reduced special-case of this , yes ?

Quoted 5 lines

Quoted 2 lines

Yeah, I guess you can get away with dummy callback implementation and a performance hit due to cash flushes.

Or you can abstract things out ?

There is one callback to arch for translation and one for cache flush, what more can I abstract out?

Quoted 12 lines

Quoted 11 lines

Quoted 4 lines

Quoted 1 line

Creating a new command like bootaux comes with very few overhead.

The overhead is the new command, we already have many ad-hoc commands.

Agreed, and I really wished that this got discussed when that command initially got added. I brought it up back then...
https://lists.denx.de/pipermail/u-boot/2016-January/240323.html

It seemed to be acceptable to just add this ad hoc command, with some "random binary format" support back then...

Quoted 3 lines

Based on that discussion, I only see that noone opposed, but I don't see any agreement.

Maybe I need to word my emails a bit stronger, but with that email I actually tried to oppose:
https://lists.denx.de/pipermail/u-boot/2016-January/240989.html

Quoted 8 lines

Quoted 4 lines

Ok, it is not entirely
random, since it is the format of a binary how it ends up in microcontrollers execute in place flash (stack pointer and reset vector are the two first words)....

I thought this command was starting the core by loading data to RAM , not flash ?

Ok, maybe I am a bit unclear here:

bootaux currently supports a Cortex-M specific binary format only. The binary format starts with a Cortex-M class vector table. The vector tables first vector is the "reset vector".

In a regular microcontroller, that binary gets flashed on a NOR flash which is mapped at 0x0 for the CPU. The CPU has no "boot ROM", the CPU starts by calling the reset vector. So when NXP defined how the bootaux command should look like, they just took that binary format which was laying around and implemented a U-Boot command around it.

So this is the history of the binary format. And my point here is, that the binary format supported by bootaux is _very_ Cortex-M class specific.

Quoted 11 lines

Quoted 5 lines

However, making this ad hoc command now
generic really feels weird to me, since we would end up supporting that format for A class CPUs etc... bootaux is really suited for auxiliary M-class cores on ARM, as it is right now. Maybe we should have named it bootm ;-)

We always try to avoid one-off hacks, so it's not weird. I still don't quite understand how it is a problem to abstract things out . I am not asking you to support CA, but to avoid CM specific implementation which cannot be extended to support CA or even MIPS/Nios2/etc .

Again, why would you support a Cortex-M class specific file format for MIPS/Nios2 etc...?

bootaux is M4 specific. We can through it away, and start over, but then we should call the command differently.

Quoted 14 lines

Quoted 14 lines

Quoted 13 lines

Quoted 10 lines

This are the reasons why I feel creating a new command for a SMP boot case makes more sense. We can still reuse functions which are very similar by moving them into some common location, where it makes sense.

Quoted 3 lines

Also, I think this might come useful when booting stuff like "Altera Sparrow" ...

I am not familiar with that architecture, what kind of core does it provide which needs to be booted by U-Boot?

The secondary ARM core in the SoCFPGA C-A9 complex or Nios2 core in the FPGA.

Quoted 8 lines

Quoted 3 lines

In my thinking, the Nios2 core seems like such a remote processor well suited for the bootaux command. For the secondary A9, I would create a new command.

Uh, why, that does not make any sense to me. Both the random core in FPGA and the secondary CA9 core have almost the same bringup sequence. What is so different about this stuff ?

A cache coherent core and a non-cache coherent core remote core somewhere in the SoC is very much different IMHO.

Lets compare how you bring up a A class CPU, or cache coherent SMP style CPU in general:

1. Load code into RAM
2. Some cache flushing might be necessary due to I/D-cache incoherency
3. Write entry point into some architecture specific register
4. kick off the CPU by writing another architecture specific register

M class/remote CPU

1. Load code in RAM, depending on image format translate core specific memory map to host memory map
2. Flush D-cache
3. (Potentially set up IOMMU, not in the NXP case though)
4. Write entry point into some architecture specific register
5. kick off the CPU by writing another architecture specific register

From what I can see, everything is different other than "Load code in RAM" which is image specific. If the image format is complex, we certainly should factor that out to avoid code duplication.

Quoted 7 lines

Quoted 4 lines

If we want to support the two with the same command, we already have a first problem: How do we address them? Of course, we could add just a index or something, but this would already break backward compatibility of the bootaux command.

So we already accepted a command which has shit commandline semantics and we're stuck with it , hrm. You can always specify a default core

Add shitty binary file format to the list.

Quoted 3 lines

index for those two iMX boards and it's done. Or just fix the semantics in their default environment, which is not awesome, but I doubt this is wildly used yet.

I guess my general point is, bootaux is already broken in two ways:
Command line interface does not allow for extensibility, the only format supported right now is a binary format which is not generic.

My patchset was an attempt to improve the situation to give it at least a decent elf format support.

Note that there is also bootelf, where the elf headers have been introduced. Reusing that command seems impractical since it is meant to boot on the primary CPU. My first attempt was reusing at least the load function, but it seemed impractical due to memory map translation and since the load function is rather short. If it helps for the acceptance of this patchset, I still could try to make that happen.

Making the bootaux something completely new, e.g. a generic command under cmd/*, with CPU enumeration capabilities and (pluggable?) image support is something I am not willing to build.

--
Stefan

Quoted 23 lines · Stefan Agner

Quoted 22 lines · Marek Vasut

On 04/04/2017 02:02 AM, Stefan Agner wrote:
[...]

Quoted 19 lines

Quoted 11 lines

Quoted 11 lines

Quoted 3 lines

Admitedly, I didn't look at the patch, but if you want to boot ad-hoc cores, you can very well also boot secondary cores on the current CPU complex with the same command. Why not ?

Sure, it could be done. I just feel it is not the right design.

Auxiliary cores have usually a different view to memory, this is why I had to add the get_host_mapping callback in the elf loader code to let architecture dependent code translate to host addresses. SMP systems don't need that.

Quoted 5 lines

Quoted 3 lines

Also flush caches is not necessary on some cache coherent CPU's (depending on how your cache coherence between I and D cache looks like).

So SMP is just a reduced special-case of this , yes ?

Yeah, I guess you can get away with dummy callback implementation and a performance hit due to cash flushes.

Quoted 5 lines

Quoted 2 lines

Or you can abstract things out ?

There is one callback to arch for translation and one for cache flush, what more can I abstract out?

Well then I don't think I understand your concerns about cache flushing in SMP .

Quoted 16 lines

Quoted 15 lines

Quoted 11 lines

Quoted 4 lines

Quoted 1 line

Creating a new command like bootaux comes with very few overhead.

The overhead is the new command, we already have many ad-hoc commands.

Agreed, and I really wished that this got discussed when that command initially got added. I brought it up back then...
https://lists.denx.de/pipermail/u-boot/2016-January/240323.html

It seemed to be acceptable to just add this ad hoc command, with some "random binary format" support back then...

Based on that discussion, I only see that noone opposed, but I don't see any agreement.

Quoted 3 lines

Maybe I need to word my emails a bit stronger, but with that email I actually tried to oppose:
https://lists.denx.de/pipermail/u-boot/2016-January/240989.html

Well, I do not like adding ad-hoc commands as it's not managable in the long run.

Quoted 21 lines

Quoted 8 lines

Quoted 4 lines

Ok, it is not entirely
random, since it is the format of a binary how it ends up in microcontrollers execute in place flash (stack pointer and reset vector are the two first words)....

I thought this command was starting the core by loading data to RAM , not flash ?

Ok, maybe I am a bit unclear here:

bootaux currently supports a Cortex-M specific binary format only. The binary format starts with a Cortex-M class vector table. The vector tables first vector is the "reset vector".

In a regular microcontroller, that binary gets flashed on a NOR flash which is mapped at 0x0 for the CPU. The CPU has no "boot ROM", the CPU starts by calling the reset vector. So when NXP defined how the bootaux command should look like, they just took that binary format which was laying around and implemented a U-Boot command around it.

Quoted 3 lines

So this is the history of the binary format. And my point here is, that the binary format supported by bootaux is _very_ Cortex-M class specific.

Aha, so I now totally don't understand why this command cannot be fixed/extended to support other/generic cores or SMP systems etc. But looking at the initial proposal, I think maybe the intention of this patchset was not to add that support, but to fix the command to support loading ELF files ? We already have bootelf for that though ...

Quoted 15 lines

Quoted 11 lines

Quoted 5 lines

However, making this ad hoc command now
generic really feels weird to me, since we would end up supporting that format for A class CPUs etc... bootaux is really suited for auxiliary M-class cores on ARM, as it is right now. Maybe we should have named it bootm ;-)

We always try to avoid one-off hacks, so it's not weird. I still don't quite understand how it is a problem to abstract things out . I am not asking you to support CA, but to avoid CM specific implementation which cannot be extended to support CA or even MIPS/Nios2/etc .

Again, why would you support a Cortex-M class specific file format for MIPS/Nios2 etc...?

Quoted 2 lines

bootaux is M4 specific. We can through it away, and start over, but then we should call the command differently.

Unless you look very carefully, it is not clear that it is cortex M4 specific at all based on this discussion. I was under the impression that the goal here was to support all sorts of secondary cores ...

Quoted 43 lines

Quoted 14 lines

Quoted 14 lines

Quoted 13 lines

Quoted 10 lines

This are the reasons why I feel creating a new command for a SMP boot case makes more sense. We can still reuse functions which are very similar by moving them into some common location, where it makes sense.

Quoted 3 lines

Also, I think this might come useful when booting stuff like "Altera Sparrow" ...

I am not familiar with that architecture, what kind of core does it provide which needs to be booted by U-Boot?

The secondary ARM core in the SoCFPGA C-A9 complex or Nios2 core in the FPGA.

Quoted 8 lines

Quoted 3 lines

In my thinking, the Nios2 core seems like such a remote processor well suited for the bootaux command. For the secondary A9, I would create a new command.

Uh, why, that does not make any sense to me. Both the random core in FPGA and the secondary CA9 core have almost the same bringup sequence. What is so different about this stuff ?

A cache coherent core and a non-cache coherent core remote core somewhere in the SoC is very much different IMHO.

Lets compare how you bring up a A class CPU, or cache coherent SMP style CPU in general:

1. Load code into RAM
2. Some cache flushing might be necessary due to I/D-cache incoherency
3. Write entry point into some architecture specific register
4. kick off the CPU by writing another architecture specific register

M class/remote CPU

1. Load code in RAM, depending on image format translate core specific memory map to host memory map
2. Flush D-cache
3. (Potentially set up IOMMU, not in the NXP case though)
4. Write entry point into some architecture specific register
5. kick off the CPU by writing another architecture specific register

Quoted 3 lines

From what I can see, everything is different other than "Load code in RAM" which is image specific. If the image format is complex, we certainly should factor that out to avoid code duplication.

Image format is usually binary blob or elf file, we support both. The rest is "load file to RAM, do magic setup (can be CPU specific), unreset CPU".

Quoted 5 lines

Quoted 5 lines

Quoted 4 lines

If we want to support the two with the same command, we already have a first problem: How do we address them? Of course, we could add just a index or something, but this would already break backward compatibility of the bootaux command.

Quoted 4 lines

Quoted 2 lines

So we already accepted a command which has shit commandline semantics and we're stuck with it , hrm. You can always specify a default core

Add shitty binary file format to the list.

Why do we care about this ? The CM just needs a vectoring table, so set it up (in the elf file or whatever) and point the core at the reset vector.

Quoted 10 lines

Quoted 3 lines

index for those two iMX boards and it's done. Or just fix the semantics in their default environment, which is not awesome, but I doubt this is wildly used yet.

I guess my general point is, bootaux is already broken in two ways:
Command line interface does not allow for extensibility, the only format supported right now is a binary format which is not generic.

My patchset was an attempt to improve the situation to give it at least a decent elf format support.

Maybe this information was lost ... Lukasz ?!

Quoted 6 lines

Note that there is also bootelf, where the elf headers have been introduced. Reusing that command seems impractical since it is meant to boot on the primary CPU. My first attempt was reusing at least the load function, but it seemed impractical due to memory map translation and since the load function is rather short. If it helps for the acceptance of this patchset, I still could try to make that happen.

Extending bootelf might indeed make more sense IMO. But that needs discussion.

Quoted 27 lines · Marek Vasut

Quoted 26 lines · Stefan Agner

Quoted 24 lines · Marek Vasut

On 04/04/2017 02:02 AM, Stefan Agner wrote:
[...]

Quoted 11 lines

Quoted 11 lines

Quoted 11 lines

Quoted 3 lines

Admitedly, I didn't look at the patch, but if you want to boot ad-hoc cores, you can very well also boot secondary cores on the current CPU complex with the same command. Why not ?

Sure, it could be done. I just feel it is not the right design.

Auxiliary cores have usually a different view to memory, this is why I had to add the get_host_mapping callback in the elf loader code to let architecture dependent code translate to host addresses. SMP systems don't need that.

Quoted 8 lines

Quoted 5 lines

Quoted 3 lines

Also flush caches is not necessary on some cache coherent CPU's (depending on how your cache coherence between I and D cache looks like).

So SMP is just a reduced special-case of this , yes ?

Yeah, I guess you can get away with dummy callback implementation and a performance hit due to cash flushes.

Or you can abstract things out ?

Quoted 6 lines

Quoted 2 lines

There is one callback to arch for translation and one for cache flush, what more can I abstract out?

Well then I don't think I understand your concerns about cache flushing in SMP .

It makes things unnecessary slower.

Quoted 20 lines

Quoted 19 lines

Quoted 9 lines

Quoted 9 lines

Quoted 4 lines

Quoted 1 line

Creating a new command like bootaux comes with very few overhead.

The overhead is the new command, we already have many ad-hoc commands.

Agreed, and I really wished that this got discussed when that command initially got added. I brought it up back then...
https://lists.denx.de/pipermail/u-boot/2016-January/240323.html

Quoted 6 lines

Quoted 2 lines

It seemed to be acceptable to just add this ad hoc command, with some "random binary format" support back then...

Based on that discussion, I only see that noone opposed, but I don't see any agreement.

Maybe I need to word my emails a bit stronger, but with that email I actually tried to oppose:
https://lists.denx.de/pipermail/u-boot/2016-January/240989.html

Quoted 3 lines

Well, I do not like adding ad-hoc commands as it's not managable in the long run.

I argue that a remote core loading command is _much_ more manageable than making the bootm command even more complex by support SMP/remoteproc and whatnot usecases... I would even argue that a bunch of those commands are more manageable than a single ifdef/if hell....

That said, I still would push for keeping the image processing code generic, whenever it makes sense.

Quoted 25 lines

Quoted 21 lines

Quoted 8 lines

Quoted 4 lines

Ok, it is not entirely
random, since it is the format of a binary how it ends up in microcontrollers execute in place flash (stack pointer and reset vector are the two first words)....

I thought this command was starting the core by loading data to RAM , not flash ?

Ok, maybe I am a bit unclear here:

bootaux currently supports a Cortex-M specific binary format only. The binary format starts with a Cortex-M class vector table. The vector tables first vector is the "reset vector".

In a regular microcontroller, that binary gets flashed on a NOR flash which is mapped at 0x0 for the CPU. The CPU has no "boot ROM", the CPU starts by calling the reset vector. So when NXP defined how the bootaux command should look like, they just took that binary format which was laying around and implemented a U-Boot command around it.

Quoted 3 lines

So this is the history of the binary format. And my point here is, that the binary format supported by bootaux is _very_ Cortex-M class specific.

Quoted 6 lines

Aha, so I now totally don't understand why this command cannot be fixed/extended to support other/generic cores or SMP systems etc. But looking at the initial proposal, I think maybe the intention of this patchset was not to add that support, but to fix the command to support loading ELF files ? We already have bootelf for that though ...

Yes, that is pretty much it. I would like to teach that command a more generic format, which would be at least a step towards something more generic/standardized.

bootelf is really meant for the primary CPU. That would be an entirely different direction: Make all common boot commands "aux core" capable.

But I would strongly vote against that. First, those commands have already complex arguments and argument handling (e.g. bootm), and their implementation supports use cases which we hardly would ever use on aux cores (initramfs..).

Quoted 15 lines

Quoted 15 lines

Quoted 11 lines

Quoted 5 lines

However, making this ad hoc command now
generic really feels weird to me, since we would end up supporting that format for A class CPUs etc... bootaux is really suited for auxiliary M-class cores on ARM, as it is right now. Maybe we should have named it bootm ;-)

We always try to avoid one-off hacks, so it's not weird. I still don't quite understand how it is a problem to abstract things out . I am not asking you to support CA, but to avoid CM specific implementation which cannot be extended to support CA or even MIPS/Nios2/etc .

Again, why would you support a Cortex-M class specific file format for MIPS/Nios2 etc...?

Quoted 7 lines

Quoted 2 lines

bootaux is M4 specific. We can through it away, and start over, but then we should call the command differently.

Unless you look very carefully, it is not clear that it is cortex M4 specific at all based on this discussion. I was under the impression that the goal here was to support all sorts of secondary cores ...

Yeah it's also only that binary format handling which is M4 specific. And frankly, that is just 2 lines of code, basically dereferencing the first two words:

(quoted diff) +15 -0

+    if (valid_elf_image(addr)) {
+        stack = 0x0;
+        pc = load_elf_image_phdr(addr);
+        if (!pc)
+            return CMD_RET_FAILURE;
+
+    } else {
+        /*
+         * Assume binary file with vector table at the beginning.
+         * Cortex-M4 vector tables start with the stack pointer (SP)
+         * and reset vector (initial PC).
+         */
+        stack = *(u32 *)addr;
+        pc = *(u32 *)(addr + 4);
+    }

Quoted 44 lines

Quoted 44 lines

Quoted 15 lines

Quoted 15 lines

Quoted 14 lines

Quoted 11 lines

This are the reasons why I feel creating a new command for a SMP boot case makes more sense. We can still reuse functions which are very similar by moving them into some common location, where it makes sense.

Quoted 3 lines

Also, I think this might come useful when booting stuff like "Altera Sparrow" ...

I am not familiar with that architecture, what kind of core does it provide which needs to be booted by U-Boot?

The secondary ARM core in the SoCFPGA C-A9 complex or Nios2 core in the FPGA.

Quoted 8 lines

Quoted 3 lines

In my thinking, the Nios2 core seems like such a remote processor well suited for the bootaux command. For the secondary A9, I would create a new command.

Uh, why, that does not make any sense to me. Both the random core in FPGA and the secondary CA9 core have almost the same bringup sequence. What is so different about this stuff ?

A cache coherent core and a non-cache coherent core remote core somewhere in the SoC is very much different IMHO.

Lets compare how you bring up a A class CPU, or cache coherent SMP style CPU in general:

1. Load code into RAM
2. Some cache flushing might be necessary due to I/D-cache incoherency
3. Write entry point into some architecture specific register
4. kick off the CPU by writing another architecture specific register

M class/remote CPU

1. Load code in RAM, depending on image format translate core specific memory map to host memory map
2. Flush D-cache
3. (Potentially set up IOMMU, not in the NXP case though)
4. Write entry point into some architecture specific register
5. kick off the CPU by writing another architecture specific register

Quoted 8 lines

Quoted 3 lines

From what I can see, everything is different other than "Load code in RAM" which is image specific. If the image format is complex, we certainly should factor that out to avoid code duplication.

Image format is usually binary blob or elf file, we support both. The rest is "load file to RAM, do magic setup (can be CPU specific), unreset CPU".

Loading files to RAM is already generic since it is handled by separate commands.

It is really the image handling which has the potential of reusability.

Quoted 8 lines

Quoted 8 lines

Quoted 7 lines

Quoted 4 lines

If we want to support the two with the same command, we already have a first problem: How do we address them? Of course, we could add just a index or something, but this would already break backward compatibility of the bootaux command.

So we already accepted a command which has shit commandline semantics and we're stuck with it , hrm. You can always specify a default core

Quoted 5 lines

Quoted 1 line

Add shitty binary file format to the list.

Why do we care about this ? The CM just needs a vectoring table, so set it up (in the elf file or whatever) and point the core at the reset vector.

Not sure what you want to say here, but I guess we need to keep support for this file format for backward compatibility?

Quoted 20 lines

Quoted 10 lines

Quoted 3 lines

index for those two iMX boards and it's done. Or just fix the semantics in their default environment, which is not awesome, but I doubt this is wildly used yet.

I guess my general point is, bootaux is already broken in two ways:
Command line interface does not allow for extensibility, the only format supported right now is a binary format which is not generic.

My patchset was an attempt to improve the situation to give it at least a decent elf format support.

Maybe this information was lost ... Lukasz ?!

Quoted 6 lines

Note that there is also bootelf, where the elf headers have been introduced. Reusing that command seems impractical since it is meant to boot on the primary CPU. My first attempt was reusing at least the load function, but it seemed impractical due to memory map translation and since the load function is rather short. If it helps for the acceptance of this patchset, I still could try to make that happen.

Quoted 3 lines

Extending bootelf might indeed make more sense IMO. But that needs discussion.

There are two variants of reuse here:
1. Reuse the elf parsing and loading function (function load_elf_image_phdr in this case).
2. Reuse the bootelf command. This would need extending the command with some parameter to tell it to not boot the primary CPU but boot some other auxiliary processor.

Let's get a bit more concrete here, take bootelf as a case study:

1 would introduce the architecture specific translation callback load_elf_image_phdr (see my load_elf_image_phdr implementation in this patchset). For regular/primary boot we can implement a dummy 1:1 translation.

2 would need 1, and changes to do_bootelf. Since the command has already optional arguments adding core selection as an optional argument is not possible. But we could add a parameter, e.g. -c (core):

bootelf -c aux

or

bootelf -c 1

The exact enumeration would have to be discussed. I guess this would also be architecture specific.

Adding the new parameter handling plus callbacks into architecture dependent code probably makes do_bootelf double in length. Also, half of the existing code cannot be reused: The call do_bootelf_exec is primary CPU specific, passing the arguments (this is likely not supported by remote cores) and the return code handling (since a remote core would not return), autostart handling, not sure?

2 would also need to either disable the section header functionality for the remote core, or we add another if in do_bootelf and avoid parsing section header even if requested...

Again, I strongly disfavor solution 2. That makes boot commands with image types we want to support even more complex. bootelf already starts to look ugly when I start implementing that, and bootelf is really one of the cleanest and simplest boot commands I have seen... I probably would give it a shot if this is the only way to bring elf format support for aux cores upstream... Good luck for the guy implementing FIT image support (bootm) for auxiliary cores :-D

1 we could do, and I would be willing to do it. I personally feel that the small code duplication is cleaner than reusing that function from cmd/elf.c and implementing the dummy 1:1 translation, but both works for me....

IMHO, commands are really lightweight and nice in U-Boot, I think we should make use of them, even when it means some namespace pollution.

--
Stefan

Quoted 31 lines · Stefan Agner

Quoted 30 lines · Marek Vasut

Quoted 28 lines · Stefan Agner

Quoted 17 lines · Marek Vasut

On 04/04/2017 02:02 AM, Stefan Agner wrote:
[...]

Quoted 15 lines

Quoted 15 lines

Quoted 14 lines

Quoted 3 lines

Admitedly, I didn't look at the patch, but if you want to boot ad-hoc cores, you can very well also boot secondary cores on the current CPU complex with the same command. Why not ?

Sure, it could be done. I just feel it is not the right design.

Auxiliary cores have usually a different view to memory, this is why I had to add the get_host_mapping callback in the elf loader code to let architecture dependent code translate to host addresses. SMP systems don't need that.

Also flush caches is not necessary on some cache coherent CPU's (depending on how your cache coherence between I and D cache looks like).

Quoted 7 lines

Quoted 4 lines

Quoted 1 line

So SMP is just a reduced special-case of this , yes ?

Yeah, I guess you can get away with dummy callback implementation and a performance hit due to cash flushes.

Or you can abstract things out ?

There is one callback to arch for translation and one for cache flush, what more can I abstract out?

Quoted 5 lines

Quoted 3 lines

Well then I don't think I understand your concerns about cache flushing in SMP .

It makes things unnecessary slower.

You always have to flush cache if you expect the peripheral to read cached memory , so I don't think I understand your remark ...

Quoted 20 lines

Quoted 20 lines

Quoted 19 lines

Quoted 9 lines

Quoted 9 lines

Quoted 4 lines

Quoted 1 line

Creating a new command like bootaux comes with very few overhead.

The overhead is the new command, we already have many ad-hoc commands.

Agreed, and I really wished that this got discussed when that command initially got added. I brought it up back then...
https://lists.denx.de/pipermail/u-boot/2016-January/240323.html

Quoted 6 lines

Quoted 2 lines

It seemed to be acceptable to just add this ad hoc command, with some "random binary format" support back then...

Based on that discussion, I only see that noone opposed, but I don't see any agreement.

Maybe I need to word my emails a bit stronger, but with that email I actually tried to oppose:
https://lists.denx.de/pipermail/u-boot/2016-January/240989.html

Quoted 8 lines

Quoted 3 lines

Well, I do not like adding ad-hoc commands as it's not managable in the long run.

I argue that a remote core loading command is _much_ more manageable than making the bootm command even more complex by support SMP/remoteproc and whatnot usecases... I would even argue that a bunch of those commands are more manageable than a single ifdef/if hell....

I guess a boot* option to start remote core would do ?

Quoted 2 lines

That said, I still would push for keeping the image processing code generic, whenever it makes sense.

Agreed

Quoted 39 lines

Quoted 25 lines

Quoted 21 lines

Quoted 8 lines

Quoted 4 lines

Ok, it is not entirely
random, since it is the format of a binary how it ends up in microcontrollers execute in place flash (stack pointer and reset vector are the two first words)....

I thought this command was starting the core by loading data to RAM , not flash ?

Ok, maybe I am a bit unclear here:

bootaux currently supports a Cortex-M specific binary format only. The binary format starts with a Cortex-M class vector table. The vector tables first vector is the "reset vector".

In a regular microcontroller, that binary gets flashed on a NOR flash which is mapped at 0x0 for the CPU. The CPU has no "boot ROM", the CPU starts by calling the reset vector. So when NXP defined how the bootaux command should look like, they just took that binary format which was laying around and implemented a U-Boot command around it.

Quoted 3 lines

So this is the history of the binary format. And my point here is, that the binary format supported by bootaux is _very_ Cortex-M class specific.

Quoted 6 lines

Aha, so I now totally don't understand why this command cannot be fixed/extended to support other/generic cores or SMP systems etc. But looking at the initial proposal, I think maybe the intention of this patchset was not to add that support, but to fix the command to support loading ELF files ? We already have bootelf for that though ...

Yes, that is pretty much it. I would like to teach that command a more generic format, which would be at least a step towards something more generic/standardized.

bootelf is really meant for the primary CPU. That would be an entirely different direction: Make all common boot commands "aux core" capable.

Quoted 4 lines

But I would strongly vote against that. First, those commands have already complex arguments and argument handling (e.g. bootm), and their implementation supports use cases which we hardly would ever use on aux cores (initramfs..).

That might need some further thinking/consolidation . IMO out of scope of this discussion ...

Quoted 95 lines

Quoted 15 lines

Quoted 15 lines

Quoted 11 lines

Quoted 5 lines

However, making this ad hoc command now
generic really feels weird to me, since we would end up supporting that format for A class CPUs etc... bootaux is really suited for auxiliary M-class cores on ARM, as it is right now. Maybe we should have named it bootm ;-)

We always try to avoid one-off hacks, so it's not weird. I still don't quite understand how it is a problem to abstract things out . I am not asking you to support CA, but to avoid CM specific implementation which cannot be extended to support CA or even MIPS/Nios2/etc .

Again, why would you support a Cortex-M class specific file format for MIPS/Nios2 etc...?

Quoted 7 lines

Quoted 2 lines

bootaux is M4 specific. We can through it away, and start over, but then we should call the command differently.

Unless you look very carefully, it is not clear that it is cortex M4 specific at all based on this discussion. I was under the impression that the goal here was to support all sorts of secondary cores ...

Yeah it's also only that binary format handling which is M4 specific. And frankly, that is just 2 lines of code, basically dereferencing the first two words:

(quoted diff) +15 -0

+    if (valid_elf_image(addr)) {
+        stack = 0x0;
+        pc = load_elf_image_phdr(addr);
+        if (!pc)
+            return CMD_RET_FAILURE;
+
+    } else {
+        /*
+         * Assume binary file with vector table at the beginning.
+         * Cortex-M4 vector tables start with the stack pointer (SP)
+         * and reset vector (initial PC).
+         */
+        stack = *(u32 *)addr;
+        pc = *(u32 *)(addr + 4);
+    }

Quoted 44 lines

Quoted 44 lines

Quoted 15 lines

Quoted 15 lines

Quoted 14 lines

Quoted 11 lines

This are the reasons why I feel creating a new command for a SMP boot case makes more sense. We can still reuse functions which are very similar by moving them into some common location, where it makes sense.

Quoted 3 lines

Also, I think this might come useful when booting stuff like "Altera Sparrow" ...

I am not familiar with that architecture, what kind of core does it provide which needs to be booted by U-Boot?

The secondary ARM core in the SoCFPGA C-A9 complex or Nios2 core in the FPGA.

Quoted 8 lines

Quoted 3 lines

In my thinking, the Nios2 core seems like such a remote processor well suited for the bootaux command. For the secondary A9, I would create a new command.

Uh, why, that does not make any sense to me. Both the random core in FPGA and the secondary CA9 core have almost the same bringup sequence. What is so different about this stuff ?

A cache coherent core and a non-cache coherent core remote core somewhere in the SoC is very much different IMHO.

Lets compare how you bring up a A class CPU, or cache coherent SMP style CPU in general:

1. Load code into RAM
2. Some cache flushing might be necessary due to I/D-cache incoherency
3. Write entry point into some architecture specific register
4. kick off the CPU by writing another architecture specific register

M class/remote CPU

1. Load code in RAM, depending on image format translate core specific memory map to host memory map
2. Flush D-cache
3. (Potentially set up IOMMU, not in the NXP case though)
4. Write entry point into some architecture specific register
5. kick off the CPU by writing another architecture specific register

Quoted 8 lines

Quoted 3 lines

From what I can see, everything is different other than "Load code in RAM" which is image specific. If the image format is complex, we certainly should factor that out to avoid code duplication.

Image format is usually binary blob or elf file, we support both. The rest is "load file to RAM, do magic setup (can be CPU specific), unreset CPU".

Quoted 4 lines

Loading files to RAM is already generic since it is handled by separate commands.

It is really the image handling which has the potential of reusability.

OK

Quoted 10 lines

Quoted 10 lines

Quoted 9 lines

Quoted 7 lines

Quoted 4 lines

If we want to support the two with the same command, we already have a first problem: How do we address them? Of course, we could add just a index or something, but this would already break backward compatibility of the bootaux command.

So we already accepted a command which has shit commandline semantics and we're stuck with it , hrm. You can always specify a default core

Add shitty binary file format to the list.

Quoted 6 lines

Quoted 3 lines

Why do we care about this ? The CM just needs a vectoring table, so set it up (in the elf file or whatever) and point the core at the reset vector.

Not sure what you want to say here, but I guess we need to keep support for this file format for backward compatibility?

I guess so, I didn't look into the requirements for the CM4 core on MX6.

Quoted 45 lines

Quoted 20 lines

Quoted 10 lines

Quoted 3 lines

index for those two iMX boards and it's done. Or just fix the semantics in their default environment, which is not awesome, but I doubt this is wildly used yet.

I guess my general point is, bootaux is already broken in two ways:
Command line interface does not allow for extensibility, the only format supported right now is a binary format which is not generic.

My patchset was an attempt to improve the situation to give it at least a decent elf format support.

Maybe this information was lost ... Lukasz ?!

Quoted 6 lines

Note that there is also bootelf, where the elf headers have been introduced. Reusing that command seems impractical since it is meant to boot on the primary CPU. My first attempt was reusing at least the load function, but it seemed impractical due to memory map translation and since the load function is rather short. If it helps for the acceptance of this patchset, I still could try to make that happen.

Quoted 3 lines

Extending bootelf might indeed make more sense IMO. But that needs discussion.

There are two variants of reuse here:
1. Reuse the elf parsing and loading function (function load_elf_image_phdr in this case).
2. Reuse the bootelf command. This would need extending the command with some parameter to tell it to not boot the primary CPU but boot some other auxiliary processor.

Let's get a bit more concrete here, take bootelf as a case study:

1 would introduce the architecture specific translation callback load_elf_image_phdr (see my load_elf_image_phdr implementation in this patchset). For regular/primary boot we can implement a dummy 1:1 translation.

2 would need 1, and changes to do_bootelf. Since the command has already optional arguments adding core selection as an optional argument is not possible. But we could add a parameter, e.g. -c (core):

Quoted 5 lines

bootelf -c aux

or

bootelf -c 1

Works for me ...

Quoted 9 lines

The exact enumeration would have to be discussed. I guess this would also be architecture specific.

Adding the new parameter handling plus callbacks into architecture dependent code probably makes do_bootelf double in length. Also, half of the existing code cannot be reused: The call do_bootelf_exec is primary CPU specific, passing the arguments (this is likely not supported by remote cores) and the return code handling (since a remote core would not return), autostart handling, not sure?

Quoted 6 lines

2 would also need to either disable the section header functionality for the remote core, or we add another if in do_bootelf and avoid parsing section header even if requested...

Quoted 7 lines

Again, I strongly disfavor solution 2. That makes boot commands with image types we want to support even more complex. bootelf already starts to look ugly when I start implementing that, and bootelf is really one of the cleanest and simplest boot commands I have seen... I probably would give it a shot if this is the only way to bring elf format support for aux cores upstream... Good luck for the guy implementing FIT image support (bootm) for auxiliary cores :-D

So what is the problem with this ? Load the payload somewhere and ungate the secondary core , why is that so hard ?

Quoted 12 lines

1 we could do, and I would be willing to do it. I personally feel that the small code duplication is cleaner than reusing that function from cmd/elf.c and implementing the dummy 1:1 translation, but both works for me....

IMHO, commands are really lightweight and nice in U-Boot, I think we should make use of them, even when it means some namespace pollution.

--
Stefan

Quoted 37 lines

Hi Lukasz,

Quoted 25 lines · Lukasz Majewski

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 8 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux
${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ##################################################
88.3 KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ... Colibri iMX7 #

Quoted 8 lines

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Hence, I'm wondering if you could make those patches usable for other platforms as well?

Quoted 8 lines

I don't think that this is a good idea. bootaux is meant for auxiliary cores, which often use a different architecture and are not cache coherent (hence the cache flushes).

On SMP systems the main operating system normally starts the secondary core. Otherwise, if you want to run them separately using U-Boot, maybe a new command such as bootsmp would be more suited.

Admitedly, I didn't look at the patch, but if you want to boot ad-hoc cores, you can very well also boot secondary cores on the current CPU complex with the same command. Why not ?

Also, I think this might come useful when booting stuff like "Altera Sparrow" ...

Hi Lukasz,

Quoted 31 lines · Lukasz Majewski

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 8 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux
${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ################################################## 88.3
KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ...
Colibri iMX7 #

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Quoted 2 lines

Hence, I'm wondering if you could make those patches usable for other platforms as well?

I don't think that this is a good idea. bootaux is meant for auxiliary cores, which often use a different architecture and are not cache coherent (hence the cache flushes).

On SMP systems the main operating system normally starts the secondary core.

Otherwise, if you want to run them separately using U-Boot, maybe a new command such as bootsmp would be more suited.

Quoted 8 lines

Quoted 5 lines

Note that the bootaux command retrieved the entry point (PC) from the elf binary.

Could you make this code flexible enough to handle not only elf binaries, but also other data (e.g. FPGA bitstreams)?

The code of bootaux is rather small, I don't see much value into stuff boot code for other peripherals into it.

I don't know how FPGA
bistream loading typically works, but I guess it is quite different from loading a firmware into RAM/SRAM and flush caches...

I am not against reuse and unification, I just feel that this is not really a case where we gain much.

Quoted 17 lines

Maybe it would better to:
-------------------------

Embrace those binaries into FIT file (*.itb)? And allow multiple binaries loading? I'm thinking of work similar to one already did by Andre Przywara for SPL:

"[PATCH v3 00/19] SPL: extend FIT loading support" posted on 31.03.2017?

In that way we would "open" many new use cases, and other platforms (e.g. FPGA's, TI, etc) could benefit from it.
One solid use case is to load different Linux binaries (or/and bare metal programs) to different SoC cores.

Quoted 4 lines

The "meta data" (i.e. load address, data type, description), could be extracted from the FIT format (the code is already in u-boot).

IMHO, this is very generic approach.

I did some experiments with using FIT images for auxiliary core firmware, however, it did not add a lot of advantage over using elf:
http://git.toradex.com/cgit/u-boot-toradex.git/commit/?h=2015.04-toradex-next&id=d1d416f272e840e8139aec911f89a70fe5523eb2

Firmwares are already built and available in the elf file format. The Linux remoteproc framework, which is meant to handle this kind of cores too, supports elf firmware loading too, so supporting elf in U-Boot too is a nice alignment. Also using FIT adds an additional step when building firm wares...

Quoted 7 lines

Quoted 6 lines

Also all sections are translated to A7 addresses in order to properly load the firmware sections into the appropriate locations.

Quoted 2 lines

This would require some tiny arch specific code.

Yes, but in my rather small patchset (164 additional lines) this is actually almost the bulk of changes :-)

--
Stefan

Quoted 41 lines

Quoted 24 lines

Also cache flushes is taken care of, so that there is no manual dcache flush necessary anymore.

The NXP FreeRTOS BSP already generates elf binaries which can be directly used with this elf binary support.

The last patch adds bootaux support for Vybrid.

Stefan Agner (3):
imx: imx-common: move aux core image parsing to common code
imx: imx-common: add elf firmware support
ARM: vf610: add auxiliary core boot support

 arch/arm/cpu/armv7/mx6/soc.c                |  30 +++++---
 arch/arm/cpu/armv7/mx7/soc.c                |  34 ++++++---
 arch/arm/cpu/armv7/vf610/generic.c          |  42 +++++++++++
 arch/arm/imx-common/Kconfig                 |   2 +-
 arch/arm/imx-common/Makefile                |   4 +-
 arch/arm/imx-common/imx_bootaux.c           | 105
+++++++++++++++++++++++-----
arch/arm/include/asm/arch-vf610/sys_proto.h |   8 +++
arch/arm/include/asm/imx-common/sys_proto.h |   6 ++
configs/colibri_vf_defconfig                |   1 + 9 files

Quoted 3 lines

changed, 198 insertions(+), 34 deletions(-) create mode 100644 arch/arm/include/asm/arch-vf610/sys_proto.h

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang
Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email:
wd at denx.de

Hi Stefan,

Quoted 34 lines

Hi Lukasz,

Quoted 31 lines · Lukasz Majewski

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 8 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux
${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ################################################## 88.3
KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ...
Colibri iMX7 #

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Quoted 6 lines

Quoted 2 lines

Hence, I'm wondering if you could make those patches usable for other platforms as well?

I don't think that this is a good idea. bootaux is meant for auxiliary cores, which often use a different architecture and are not cache coherent (hence the cache flushes).

I do remember, that I saw similar "tiny" patch to add "just one" ad-hoc command to do the same (conceptually) for TI SoC floating on the u-boot mailing list.

Please correct me if I'm wrong, but bootaux is IMX specific and does work, which very likely, will be also required by other SoC vendors (TI's Sitara is also equipped with Cortex-M4 and PRUSS).

Unification of such effort can save us all a lot of trouble in a very near future.

Quoted 4 lines

On SMP systems the main operating system normally starts the secondary core.

I think that there are some conceptual similarities between loading code to SMP core and Cortex M3. Of course some "tweaks" would be needed.

Quoted 2 lines

Otherwise, if you want to run them separately using U-Boot, maybe a new command such as bootsmp would be more suited.

Hmm - I do think that it would be too much:

- bootm for generic single core
- bootaux for IMX
- bootsmp for SMP (on IMX/TI/RK?)
- boot?? for ??

I would like to avoid adding new commands for doing conceptually the same work.

Even better, we could extend bootm to support the "multi binary" case too, but IMHO it would be also correct to have "bootaux" to handle generic binaries loading.

Quoted 9 lines

Quoted 8 lines

Quoted 5 lines

Note that the bootaux command retrieved the entry point (PC) from the elf binary.

Could you make this code flexible enough to handle not only elf binaries, but also other data (e.g. FPGA bitstreams)?

Quoted 2 lines

The code of bootaux is rather small, I don't see much value into stuff boot code for other peripherals into it.

But I'm not asking to write support for other vendor's SoC/use cases.

I'm just wondering if you could write generic command (framework) to support this use case and as an example add support for your IMX's Cortex-M3/4.

We would kill two birds with one stone - IMX is supported from the very beginning and we do have generic code to extend it by other vendors.

Quoted 3 lines

I don't know how FPGA
bistream loading typically works, but I guess it is quite different from loading a firmware into RAM/SRAM and flush caches...

FPGA quirks would be handled in arch/SoC specific code.

Quoted 1 line

Quoted 2 lines

I am not against reuse and unification, I just feel that this is not really a case where we gain much.

With generic "bootaux/bootm" command we can point other developers who would like to add such booting code to the already available framework.

Also, we would prevent other "ad-hoc" commands adding to u-boot.

Quoted 17 lines

Quoted 17 lines

Maybe it would better to:
-------------------------

Embrace those binaries into FIT file (*.itb)? And allow multiple binaries loading? I'm thinking of work similar to one already did by Andre Przywara for SPL:

"[PATCH v3 00/19] SPL: extend FIT loading support" posted on 31.03.2017?

In that way we would "open" many new use cases, and other platforms (e.g. FPGA's, TI, etc) could benefit from it.
One solid use case is to load different Linux binaries (or/and bare metal programs) to different SoC cores.

Quoted 8 lines

Quoted 4 lines

The "meta data" (i.e. load address, data type, description), could be extracted from the FIT format (the code is already in u-boot).

IMHO, this is very generic approach.

I did some experiments with using FIT images for auxiliary core firmware, however, it did not add a lot of advantage over using elf:
http://git.toradex.com/cgit/u-boot-toradex.git/commit/?h=2015.04-toradex-next&id=d1d416f272e840e8139aec911f89a70fe5523eb2

Unfortunately, not all use cases allow using ELF format. FIT gives more flexibility:

Quoted 6 lines

Firmwares are already built and available in the elf file format. The Linux remoteproc framework, which is meant to handle this kind of cores too, supports elf firmware loading too, so supporting elf in U-Boot too is a nice alignment. Also using FIT adds an additional step when building firm wares...

But this is all OK.

The ELF binary would be wrapped in FIT (with e.g. "elf" property, even 1 to 1 mapping). Then the 'bootaux/bootm' could parse ELF (which is also generic). And then some "IMX specific" (arch/SoC) code would be executed.

Quoted 8 lines

Quoted 8 lines

Quoted 7 lines

Also all sections are translated to A7 addresses in order to properly load the firmware sections into the appropriate locations.

Quoted 5 lines

Quoted 2 lines

This would require some tiny arch specific code.

Yes, but in my rather small patchset (164 additional lines) this is actually almost the bulk of changes :-)

Yes, I fully understand you :-) - I do remember when I wanted to add 3 lines (polarity inversion) to IMX PWM subsystem some time ago ... :-)

Best regards,
�ukasz Majewski

Quoted 45 lines

--
Stefan

Quoted 41 lines

Quoted 24 lines

Also cache flushes is taken care of, so that there is no manual dcache flush necessary anymore.

The NXP FreeRTOS BSP already generates elf binaries which can be directly used with this elf binary support.

The last patch adds bootaux support for Vybrid.

Stefan Agner (3):
imx: imx-common: move aux core image parsing to common code
imx: imx-common: add elf firmware support
ARM: vf610: add auxiliary core boot support

 arch/arm/cpu/armv7/mx6/soc.c                |  30 +++++---
 arch/arm/cpu/armv7/mx7/soc.c                |  34 ++++++---
 arch/arm/cpu/armv7/vf610/generic.c          |  42 +++++++++++
 arch/arm/imx-common/Kconfig                 |   2 +-
 arch/arm/imx-common/Makefile                |   4 +-
 arch/arm/imx-common/imx_bootaux.c           | 105
+++++++++++++++++++++++-----
arch/arm/include/asm/arch-vf610/sys_proto.h |   8 +++
arch/arm/include/asm/imx-common/sys_proto.h |   6 ++
configs/colibri_vf_defconfig                |   1 + 9 files

Quoted 3 lines

changed, 198 insertions(+), 34 deletions(-) create mode 100644 arch/arm/include/asm/arch-vf610/sys_proto.h

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang
Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email:
wd at denx.de

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de

Quoted 43 lines

Hi Stefan,

Quoted 34 lines

Hi Lukasz,

Quoted 31 lines · Lukasz Majewski

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 8 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux
${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ##################################################
88.3 KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ... Colibri iMX7 #

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Quoted 6 lines

Quoted 2 lines

Hence, I'm wondering if you could make those patches usable for other platforms as well?

I don't think that this is a good idea. bootaux is meant for auxiliary cores, which often use a different architecture and are not cache coherent (hence the cache flushes).

Quoted 7 lines

I do remember, that I saw similar "tiny" patch to add "just one" ad-hoc command to do the same (conceptually) for TI SoC floating on the u-boot mailing list.

Please correct me if I'm wrong, but bootaux is IMX specific and does work, which very likely, will be also required by other SoC vendors (TI's Sitara is also equipped with Cortex-M4 and PRUSS).

bootaux is currently IMX specific, and its currently supported binary format is M-class specific.

Quoted 1 line

Quoted 2 lines

Unification of such effort can save us all a lot of trouble in a very near future.

In OSS, you do not develop for the future. It gets developed when its here.

Quoted 10 lines

Quoted 5 lines

On SMP systems the main operating system normally starts the secondary core.

I think that there are some conceptual similarities between loading code to SMP core and Cortex M3. Of course some "tweaks" would be needed.

There are conceptual similarities between a car and a truck, still, they are likely manufactured in a different assembly line, probably by a different producer.

I guess in the end it really depends on where you draw the line: There are differences between loading code which will get executed by the primary code, loading code to execute explicitly on a SMP core, and loading code to execute on a remote processor.

The first case is already well supported, and we need to keep support backward compatibility.

The second case, IMHO, is a somewhat silly case: A SMP system usually gets driven by a single OS image, and that OS makes sure to initialize the cores (maybe with the help of firmware, such as the PSCI interface on ARM). There is no need for a boot loader command to execute a image on a secondary core explicitly...

The third case is probably a case where we could start think about unification efforts.

Quoted 3 lines

Quoted 2 lines

Otherwise, if you want to run them separately using U-Boot, maybe a new command such as bootsmp would be more suited.

Quoted 6 lines

Hmm - I do think that it would be too much:

- bootm for generic single core
- bootaux for IMX
- bootsmp for SMP (on IMX/TI/RK?)
- boot?? for ??

There is at least also bootz and bootelf.

Quoted 21 lines

I would like to avoid adding new commands for doing conceptually the same work.

Even better, we could extend bootm to support the "multi binary" case too, but IMHO it would be also correct to have "bootaux" to handle generic binaries loading.

Quoted 11 lines

Quoted 8 lines

Quoted 5 lines

Note that the bootaux command retrieved the entry point (PC) from the elf binary.

Could you make this code flexible enough to handle not only elf binaries, but also other data (e.g. FPGA bitstreams)?

The code of bootaux is rather small, I don't see much value into stuff boot code for other peripherals into it.

Quoted 7 lines

But I'm not asking to write support for other vendor's SoC/use cases.

I'm just wondering if you could write generic command (framework) to support this use case and as an example add support for your IMX's Cortex-M3/4.

Sure, mv arch/arm/imx-common/imx_bootaux.c cmd/, there is the framework :-)

But this promotes the M class specific binary format to a generic format supported for all cores.

Quoted 11 lines

We would kill two birds with one stone - IMX is supported from the very beginning and we do have generic code to extend it by other vendors.

Quoted 3 lines

I don't know how FPGA
bistream loading typically works, but I guess it is quite different from loading a firmware into RAM/SRAM and flush caches...

FPGA quirks would be handled in arch/SoC specific code.

Quoted 1 line

Quoted 9 lines

Quoted 2 lines

I am not against reuse and unification, I just feel that this is not really a case where we gain much.

With generic "bootaux/bootm" command we can point other developers who would like to add such booting code to the already available framework.

Also, we would prevent other "ad-hoc" commands adding to u-boot.

Extending bootm does not seem like a good idea. bootm is already rather complex, making it even more complex by supporting the auxiliary core case seems really not work well.

Also, bootm supports lots of features which you probably never would use or test on a remote core (e.g. initramfs etc...)

Quoted 32 lines

Quoted 21 lines

Quoted 21 lines

Maybe it would better to:
-------------------------

Embrace those binaries into FIT file (*.itb)? And allow multiple binaries loading? I'm thinking of work similar to one already did by Andre Przywara for SPL:

"[PATCH v3 00/19] SPL: extend FIT loading support" posted on 31.03.2017?

In that way we would "open" many new use cases, and other platforms (e.g. FPGA's, TI, etc) could benefit from it. One solid use case is to load different Linux binaries (or/and bare metal programs) to different SoC cores.

The "meta data" (i.e. load address, data type, description), could be extracted from the FIT format (the code is already in u-boot).

Quoted 6 lines

Quoted 1 line

IMHO, this is very generic approach.

I did some experiments with using FIT images for auxiliary core firmware, however, it did not add a lot of advantage over using elf:
http://git.toradex.com/cgit/u-boot-toradex.git/commit/?h=2015.04-toradex-next&id=d1d416f272e840e8139aec911f89a70fe5523eb2

Unfortunately, not all use cases allow using ELF format. FIT gives more flexibility:

Review

Quoted 4 lines

(quoted diff) +0 -2

-  ./doc/README.dfutftp -> different images loading
 
-  Andre's patch to load multiple binaries in SPL - [PATCH v3 00/19]
  SPL: extend FIT loading support"

Are flexible, but very much U-Boot specific. And much more complex to support.

Quoted 9 lines

Quoted 6 lines

Firmwares are already built and available in the elf file format. The Linux remoteproc framework, which is meant to handle this kind of cores too, supports elf firmware loading too, so supporting elf in U-Boot too is a nice alignment. Also using FIT adds an additional step when building firm wares...

But this is all OK.

Quoted 5 lines

The ELF binary would be wrapped in FIT (with e.g. "elf" property, even 1 to 1 mapping). Then the 'bootaux/bootm' could parse ELF (which is also generic). And then some "IMX specific" (arch/SoC) code would be executed.

So we go from a nacked binary loaded directly to the place it has been linked to, to a double wrapped firmware file...?

Not sure if user will appriciate that extra work and boot time. And developers the extra code.

Maybe in a perfect world that just works, because you know, FIT is generic, and elf is generic... But that is just not how it works in practice. The existing FIT code is rather complex, supports lots of corner cases. The elf code supports different header types... Loading the elf sections (which use M4 specific addressing) needs address translation to get to suitable host addresses...

Quoted 12 lines

Quoted 12 lines

Quoted 11 lines

Quoted 8 lines

Also all sections are translated to A7 addresses in order to properly load the firmware sections into the appropriate locations.

This would require some tiny arch specific code.

Quoted 7 lines

Quoted 2 lines

Yes, but in my rather small patchset (164 additional lines) this is actually almost the bulk of changes :-)

Yes, I fully understand you :-) - I do remember when I wanted to add 3 lines (polarity inversion) to IMX PWM subsystem some time ago ... :-)

Hehe, yeah that was a rather long episode.

It was a bit different, though, the core framework was already there, it was mostly about adopting a particular driver to that framework. Building a new framework is a bit more involved...

--
Stefan

Quoted 62 lines

Best regards,
�ukasz Majewski

Quoted 45 lines

--
Stefan

Quoted 41 lines

Quoted 24 lines

Also cache flushes is taken care of, so that there is no manual dcache flush necessary anymore.

The NXP FreeRTOS BSP already generates elf binaries which can be directly used with this elf binary support.

The last patch adds bootaux support for Vybrid.

Stefan Agner (3):
imx: imx-common: move aux core image parsing to common code
imx: imx-common: add elf firmware support
ARM: vf610: add auxiliary core boot support

 arch/arm/cpu/armv7/mx6/soc.c                |  30 +++++---
 arch/arm/cpu/armv7/mx7/soc.c                |  34 ++++++---
 arch/arm/cpu/armv7/vf610/generic.c          |  42 +++++++++++
 arch/arm/imx-common/Kconfig                 |   2 +-
 arch/arm/imx-common/Makefile                |   4 +-
 arch/arm/imx-common/imx_bootaux.c           | 105
+++++++++++++++++++++++-----
arch/arm/include/asm/arch-vf610/sys_proto.h |   8 +++
arch/arm/include/asm/imx-common/sys_proto.h |   6 ++
configs/colibri_vf_defconfig                |   1 + 9 files

Quoted 3 lines

changed, 198 insertions(+), 34 deletions(-) create mode 100644 arch/arm/include/asm/arch-vf610/sys_proto.h

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang
Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax:
(+49)-8142-66989-80 Email: wd at denx.de

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang
Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email:
wd at denx.de

Hi Stefan,

Quoted 56 lines · Lukasz Majewski

Quoted 43 lines

Hi Stefan,

Quoted 34 lines

Hi Lukasz,

Quoted 31 lines · Lukasz Majewski

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 8 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux
${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ##################################################
88.3 KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ... Colibri iMX7 #

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Quoted 6 lines

Quoted 2 lines

Hence, I'm wondering if you could make those patches usable for other platforms as well?

I don't think that this is a good idea. bootaux is meant for auxiliary cores, which often use a different architecture and are not cache coherent (hence the cache flushes).

Quoted 7 lines

I do remember, that I saw similar "tiny" patch to add "just one" ad-hoc command to do the same (conceptually) for TI SoC floating on the u-boot mailing list.

Please correct me if I'm wrong, but bootaux is IMX specific and does work, which very likely, will be also required by other SoC vendors (TI's Sitara is also equipped with Cortex-M4 and PRUSS).

bootaux is currently IMX specific, and its currently supported binary format is M-class specific.

Quoted 1 line

Quoted 5 lines

Quoted 2 lines

Unification of such effort can save us all a lot of trouble in a very near future.

In OSS, you do not develop for the future. It gets developed when its here.

I cannot agree here. When you perceive threat then you prepare for it.

Quoted 20 lines

Quoted 10 lines

Quoted 5 lines

On SMP systems the main operating system normally starts the secondary core.

I think that there are some conceptual similarities between loading code to SMP core and Cortex M3. Of course some "tweaks" would be needed.

There are conceptual similarities between a car and a truck, still, they are likely manufactured in a different assembly line, probably by a different producer.

I guess in the end it really depends on where you draw the line: There are differences between loading code which will get executed by the primary code, loading code to execute explicitly on a SMP core, and loading code to execute on a remote processor.

Quoted 8 lines

The first case is already well supported, and we need to keep support backward compatibility.

The second case, IMHO, is a somewhat silly case: A SMP system usually gets driven by a single OS image, and that OS makes sure to initialize the cores (maybe with the help of firmware, such as the PSCI interface on ARM). There is no need for a boot loader command to execute a image on a secondary core explicitly...

I do understand (and agree) with your point with SMP.

But, I do know at least one use case when somebody would like to start two binaries (those are bare metal programs, taking care of SoC setup, IPC, etc).

And maybe Linux with some FPGA based IP block already configured in u-boot.

Quoted 8 lines

The third case is probably a case where we could start think about unification efforts.

Quoted 3 lines

Quoted 2 lines

Otherwise, if you want to run them separately using U-Boot, maybe a new command such as bootsmp would be more suited.

Quoted 8 lines

Quoted 6 lines

Hmm - I do think that it would be too much:

- bootm for generic single core
- bootaux for IMX
- bootsmp for SMP (on IMX/TI/RK?)
- boot?? for ??

There is at least also bootz and bootelf.

I will reply to the other thread regarding bootelf.

Quoted 112 lines

Quoted 21 lines

I would like to avoid adding new commands for doing conceptually the same work.

Even better, we could extend bootm to support the "multi binary" case too, but IMHO it would be also correct to have "bootaux" to handle generic binaries loading.

Quoted 11 lines

Quoted 8 lines

Quoted 5 lines

Note that the bootaux command retrieved the entry point (PC) from the elf binary.

Could you make this code flexible enough to handle not only elf binaries, but also other data (e.g. FPGA bitstreams)?

The code of bootaux is rather small, I don't see much value into stuff boot code for other peripherals into it.

Quoted 7 lines

But I'm not asking to write support for other vendor's SoC/use cases.

I'm just wondering if you could write generic command (framework) to support this use case and as an example add support for your IMX's Cortex-M3/4.

Sure, mv arch/arm/imx-common/imx_bootaux.c cmd/, there is the framework :-)

But this promotes the M class specific binary format to a generic format supported for all cores.

Quoted 11 lines

We would kill two birds with one stone - IMX is supported from the very beginning and we do have generic code to extend it by other vendors.

Quoted 3 lines

I don't know how FPGA
bistream loading typically works, but I guess it is quite different from loading a firmware into RAM/SRAM and flush caches...

FPGA quirks would be handled in arch/SoC specific code.

Quoted 1 line

Quoted 9 lines

Quoted 2 lines

I am not against reuse and unification, I just feel that this is not really a case where we gain much.

With generic "bootaux/bootm" command we can point other developers who would like to add such booting code to the already available framework.

Also, we would prevent other "ad-hoc" commands adding to u-boot.

Extending bootm does not seem like a good idea. bootm is already rather complex, making it even more complex by supporting the auxiliary core case seems really not work well.

Also, bootm supports lots of features which you probably never would use or test on a remote core (e.g. initramfs etc...)

Quoted 32 lines

Quoted 21 lines

Quoted 21 lines

Maybe it would better to:
-------------------------

Embrace those binaries into FIT file (*.itb)? And allow multiple binaries loading? I'm thinking of work similar to one already did by Andre Przywara for SPL:

"[PATCH v3 00/19] SPL: extend FIT loading support" posted on 31.03.2017?

In that way we would "open" many new use cases, and other platforms (e.g. FPGA's, TI, etc) could benefit from it. One solid use case is to load different Linux binaries (or/and bare metal programs) to different SoC cores.

The "meta data" (i.e. load address, data type, description), could be extracted from the FIT format (the code is already in u-boot).

Quoted 6 lines

Quoted 1 line

IMHO, this is very generic approach.

I did some experiments with using FIT images for auxiliary core firmware, however, it did not add a lot of advantage over using elf:
http://git.toradex.com/cgit/u-boot-toradex.git/commit/?h=2015.04-toradex-next&id=d1d416f272e840e8139aec911f89a70fe5523eb2

Unfortunately, not all use cases allow using ELF format. FIT gives more flexibility:

Review

Quoted 4 lines

(quoted diff) +0 -2

-  ./doc/README.dfutftp -> different images loading
 
-  Andre's patch to load multiple binaries in SPL - [PATCH v3 00/19]
  SPL: extend FIT loading support"

Are flexible, but very much U-Boot specific. And much more complex to support.

Quoted 9 lines

Quoted 6 lines

Firmwares are already built and available in the elf file format. The Linux remoteproc framework, which is meant to handle this kind of cores too, supports elf firmware loading too, so supporting elf in U-Boot too is a nice alignment. Also using FIT adds an additional step when building firm wares...

But this is all OK.

Quoted 8 lines

Quoted 5 lines

The ELF binary would be wrapped in FIT (with e.g. "elf" property, even 1 to 1 mapping). Then the 'bootaux/bootm' could parse ELF (which is also generic). And then some "IMX specific" (arch/SoC) code would be executed.

So we go from a nacked binary loaded directly to the place it has been linked to, to a double wrapped firmware file...?

We would have elf binary file embedded into FIT file, these would bring flexibility.

FIT support is in place (u-boot/spl).

In such a way you can use any binary in any format.

But I must admit that we are going off-topic here.....

Quoted 10 lines

Not sure if user will appriciate that extra work and boot time. And developers the extra code.

Maybe in a perfect world that just works, because you know, FIT is generic, and elf is generic... But that is just not how it works in practice. The existing FIT code is rather complex, supports lots of corner cases. The elf code supports different header types... Loading the elf sections (which use M4 specific addressing) needs address translation to get to suitable host addresses...

Maybe I'm an idealist :-)

One analogy comes to my mind between linux and u-boot.

The proliferation of u-boot commands and linux board files. Why Linux spend tremendous resources to remove board files and switch to device tree?

Quoted 20 lines

Quoted 12 lines

Quoted 12 lines

Quoted 11 lines

Quoted 8 lines

Also all sections are translated to A7 addresses in order to properly load the firmware sections into the appropriate locations.

This would require some tiny arch specific code.

Quoted 7 lines

Quoted 2 lines

Yes, but in my rather small patchset (164 additional lines) this is actually almost the bulk of changes :-)

Yes, I fully understand you :-) - I do remember when I wanted to add 3 lines (polarity inversion) to IMX PWM subsystem some time ago ... :-)

Quoted 71 lines

Hehe, yeah that was a rather long episode.

It was a bit different, though, the core framework was already there, it was mostly about adopting a particular driver to that framework. Building a new framework is a bit more involved...

--
Stefan

Quoted 62 lines

Best regards,
�ukasz Majewski

Quoted 45 lines

--
Stefan

Quoted 41 lines

Quoted 24 lines

Also cache flushes is taken care of, so that there is no manual dcache flush necessary anymore.

The NXP FreeRTOS BSP already generates elf binaries which can be directly used with this elf binary support.

The last patch adds bootaux support for Vybrid.

Stefan Agner (3):
imx: imx-common: move aux core image parsing to common code
imx: imx-common: add elf firmware support
ARM: vf610: add auxiliary core boot support

 arch/arm/cpu/armv7/mx6/soc.c                |  30 +++++---
 arch/arm/cpu/armv7/mx7/soc.c                |  34 ++++++---
 arch/arm/cpu/armv7/vf610/generic.c          |  42 +++++++++++
 arch/arm/imx-common/Kconfig                 |   2 +-
 arch/arm/imx-common/Makefile                |   4 +-
 arch/arm/imx-common/imx_bootaux.c           | 105
+++++++++++++++++++++++-----
arch/arm/include/asm/arch-vf610/sys_proto.h |   8 +++
arch/arm/include/asm/imx-common/sys_proto.h |   6 ++
configs/colibri_vf_defconfig                |   1 + 9 files

Quoted 3 lines

changed, 198 insertions(+), 34 deletions(-) create mode 100644 arch/arm/include/asm/arch-vf610/sys_proto.h

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang
Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax:
(+49)-8142-66989-80 Email: wd at denx.de

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang
Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email:
wd at denx.de

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de

Quoted 264 lines · Lukasz Majewski

Hi Stefan,

Quoted 56 lines · Lukasz Majewski

Quoted 43 lines

Hi Stefan,

Quoted 34 lines

Hi Lukasz,

Quoted 31 lines · Lukasz Majewski

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 8 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux
${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ##################################################
88.3 KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ... Colibri iMX7 #

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Quoted 6 lines

Quoted 2 lines

Hence, I'm wondering if you could make those patches usable for other platforms as well?

I don't think that this is a good idea. bootaux is meant for auxiliary cores, which often use a different architecture and are not cache coherent (hence the cache flushes).

Quoted 7 lines

I do remember, that I saw similar "tiny" patch to add "just one" ad-hoc command to do the same (conceptually) for TI SoC floating on the u-boot mailing list.

Please correct me if I'm wrong, but bootaux is IMX specific and does work, which very likely, will be also required by other SoC vendors (TI's Sitara is also equipped with Cortex-M4 and PRUSS).

bootaux is currently IMX specific, and its currently supported binary format is M-class specific.

Quoted 1 line

Quoted 5 lines

Quoted 2 lines

Unification of such effort can save us all a lot of trouble in a very near future.

In OSS, you do not develop for the future. It gets developed when its here.

I cannot agree here. When you perceive threat then you prepare for it.

Quoted 20 lines

Quoted 10 lines

Quoted 5 lines

On SMP systems the main operating system normally starts the secondary core.

I think that there are some conceptual similarities between loading code to SMP core and Cortex M3. Of course some "tweaks" would be needed.

There are conceptual similarities between a car and a truck, still, they are likely manufactured in a different assembly line, probably by a different producer.

I guess in the end it really depends on where you draw the line: There are differences between loading code which will get executed by the primary code, loading code to execute explicitly on a SMP core, and loading code to execute on a remote processor.

Quoted 8 lines

The first case is already well supported, and we need to keep support backward compatibility.

The second case, IMHO, is a somewhat silly case: A SMP system usually gets driven by a single OS image, and that OS makes sure to initialize the cores (maybe with the help of firmware, such as the PSCI interface on ARM). There is no need for a boot loader command to execute a image on a secondary core explicitly...

I do understand (and agree) with your point with SMP.

But, I do know at least one use case when somebody would like to start two binaries (those are bare metal programs, taking care of SoC setup, IPC, etc).

And maybe Linux with some FPGA based IP block already configured in u-boot.

Quoted 8 lines

The third case is probably a case where we could start think about unification efforts.

Quoted 3 lines

Quoted 2 lines

Otherwise, if you want to run them separately using U-Boot, maybe a new command such as bootsmp would be more suited.

Quoted 8 lines

Quoted 6 lines

Hmm - I do think that it would be too much:

- bootm for generic single core
- bootaux for IMX
- bootsmp for SMP (on IMX/TI/RK?)
- boot?? for ??

There is at least also bootz and bootelf.

I will reply to the other thread regarding bootelf.

Quoted 112 lines

Quoted 21 lines

I would like to avoid adding new commands for doing conceptually the same work.

Even better, we could extend bootm to support the "multi binary" case too, but IMHO it would be also correct to have "bootaux" to handle generic binaries loading.

Quoted 11 lines

Quoted 8 lines

Quoted 5 lines

Note that the bootaux command retrieved the entry point (PC) from the elf binary.

Could you make this code flexible enough to handle not only elf binaries, but also other data (e.g. FPGA bitstreams)?

The code of bootaux is rather small, I don't see much value into stuff boot code for other peripherals into it.

Quoted 7 lines

But I'm not asking to write support for other vendor's SoC/use cases.

I'm just wondering if you could write generic command (framework) to support this use case and as an example add support for your IMX's Cortex-M3/4.

Sure, mv arch/arm/imx-common/imx_bootaux.c cmd/, there is the framework :-)

But this promotes the M class specific binary format to a generic format supported for all cores.

Quoted 11 lines

We would kill two birds with one stone - IMX is supported from the very beginning and we do have generic code to extend it by other vendors.

Quoted 3 lines

I don't know how FPGA
bistream loading typically works, but I guess it is quite different from loading a firmware into RAM/SRAM and flush caches...

FPGA quirks would be handled in arch/SoC specific code.

Quoted 1 line

Quoted 9 lines

Quoted 2 lines

I am not against reuse and unification, I just feel that this is not really a case where we gain much.

With generic "bootaux/bootm" command we can point other developers who would like to add such booting code to the already available framework.

Also, we would prevent other "ad-hoc" commands adding to u-boot.

Extending bootm does not seem like a good idea. bootm is already rather complex, making it even more complex by supporting the auxiliary core case seems really not work well.

Also, bootm supports lots of features which you probably never would use or test on a remote core (e.g. initramfs etc...)

Quoted 32 lines

Quoted 21 lines

Quoted 21 lines

Maybe it would better to:
-------------------------

Embrace those binaries into FIT file (*.itb)? And allow multiple binaries loading? I'm thinking of work similar to one already did by Andre Przywara for SPL:

"[PATCH v3 00/19] SPL: extend FIT loading support" posted on 31.03.2017?

In that way we would "open" many new use cases, and other platforms (e.g. FPGA's, TI, etc) could benefit from it. One solid use case is to load different Linux binaries (or/and bare metal programs) to different SoC cores.

The "meta data" (i.e. load address, data type, description), could be extracted from the FIT format (the code is already in u-boot).

Quoted 6 lines

Quoted 1 line

IMHO, this is very generic approach.

I did some experiments with using FIT images for auxiliary core firmware, however, it did not add a lot of advantage over using elf:
http://git.toradex.com/cgit/u-boot-toradex.git/commit/?h=2015.04-toradex-next&id=d1d416f272e840e8139aec911f89a70fe5523eb2

Unfortunately, not all use cases allow using ELF format. FIT gives more flexibility:

Review

Quoted 4 lines

(quoted diff) +0 -2

-  ./doc/README.dfutftp -> different images loading
 
-  Andre's patch to load multiple binaries in SPL - [PATCH v3 00/19]
  SPL: extend FIT loading support"

Are flexible, but very much U-Boot specific. And much more complex to support.

Quoted 9 lines

Quoted 6 lines

Firmwares are already built and available in the elf file format. The Linux remoteproc framework, which is meant to handle this kind of cores too, supports elf firmware loading too, so supporting elf in U-Boot too is a nice alignment. Also using FIT adds an additional step when building firm wares...

But this is all OK.

Quoted 8 lines

Quoted 5 lines

The ELF binary would be wrapped in FIT (with e.g. "elf" property, even 1 to 1 mapping). Then the 'bootaux/bootm' could parse ELF (which is also generic). And then some "IMX specific" (arch/SoC) code would be executed.

So we go from a nacked binary loaded directly to the place it has been linked to, to a double wrapped firmware file...?

We would have elf binary file embedded into FIT file, these would bring flexibility.

FIT support is in place (u-boot/spl).

In such a way you can use any binary in any format.

But I must admit that we are going off-topic here.....

Quoted 10 lines

Not sure if user will appriciate that extra work and boot time. And developers the extra code.

Maybe in a perfect world that just works, because you know, FIT is generic, and elf is generic... But that is just not how it works in practice. The existing FIT code is rather complex, supports lots of corner cases. The elf code supports different header types... Loading the elf sections (which use M4 specific addressing) needs address translation to get to suitable host addresses...

Quoted 8 lines

Maybe I'm an idealist :-)

One analogy comes to my mind between linux and u-boot.

The proliferation of u-boot commands and linux board files. Why Linux spend tremendous resources to remove board files and switch to device tree?

I argue that is the right way of doing it: Do ad-hoc solutions, whatever makes sense, and once you understand the full breath of design space it is much easier to build a suitable framework. At that point, do the refactoring and build that suitable framework.

If you design a framework without the understanding of the whole design space, you will end up with something which does not work well and needs lots of work-arounds etc...

Of course, it is a bit different since both examples have outside facing impact (change to device tree as well as changes to U-Boot commands).

Quoted 292 lines · Stefan Agner

Quoted 273 lines · Lukasz Majewski

Hi Stefan,

Quoted 59 lines · Lukasz Majewski

Quoted 45 lines

Hi Stefan,

Quoted 36 lines

Hi Lukasz,

Quoted 33 lines · Lukasz Majewski

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 9 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux
${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is
192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading:
################################################## 88.3 KiB 5.4 MiB/s done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ... Colibri iMX7 #

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Quoted 6 lines

Quoted 2 lines

Hence, I'm wondering if you could make those patches usable for other platforms as well?

I don't think that this is a good idea. bootaux is meant for auxiliary cores, which often use a different architecture and are not cache coherent (hence the cache flushes).

Quoted 8 lines

I do remember, that I saw similar "tiny" patch to add "just one" ad-hoc command to do the same (conceptually) for TI SoC floating on the u-boot mailing list.

Please correct me if I'm wrong, but bootaux is IMX specific and does work, which very likely, will be also required by other SoC vendors (TI's Sitara is also equipped with Cortex-M4 and PRUSS).

bootaux is currently IMX specific, and its currently supported binary format is M-class specific.

Quoted 1 line

Quoted 5 lines

Quoted 2 lines

Unification of such effort can save us all a lot of trouble in a very near future.

In OSS, you do not develop for the future. It gets developed when its here.

I cannot agree here. When you perceive threat then you prepare for it.

Quoted 23 lines

Quoted 10 lines

Quoted 5 lines

On SMP systems the main operating system normally starts the secondary core.

I think that there are some conceptual similarities between loading code to SMP core and Cortex M3. Of course some "tweaks" would be needed.

There are conceptual similarities between a car and a truck, still, they are likely manufactured in a different assembly line, probably by a different producer.

I guess in the end it really depends on where you draw the line:
There are differences between loading code which will get executed by the primary code, loading code to execute explicitly on a SMP core, and loading code to execute on a remote processor.

The first case is already well supported, and we need to keep support backward compatibility.

Quoted 6 lines

The second case, IMHO, is a somewhat silly case: A SMP system usually gets driven by a single OS image, and that OS makes sure to initialize the cores (maybe with the help of firmware, such as the PSCI interface on ARM). There is no need for a boot loader command to execute a image on a secondary core explicitly...

I do understand (and agree) with your point with SMP.

But, I do know at least one use case when somebody would like to start two binaries (those are bare metal programs, taking care of SoC setup, IPC, etc).

And maybe Linux with some FPGA based IP block already configured in u-boot.

Quoted 8 lines

The third case is probably a case where we could start think about unification efforts.

Quoted 3 lines

Quoted 2 lines

Otherwise, if you want to run them separately using U-Boot, maybe a new command such as bootsmp would be more suited.

Quoted 8 lines

Quoted 6 lines

Hmm - I do think that it would be too much:

- bootm for generic single core
- bootaux for IMX
- bootsmp for SMP (on IMX/TI/RK?)
- boot?? for ??

There is at least also bootz and bootelf.

I will reply to the other thread regarding bootelf.

Quoted 115 lines

Quoted 21 lines

I would like to avoid adding new commands for doing conceptually the same work.

Even better, we could extend bootm to support the "multi binary" case too, but IMHO it would be also correct to have "bootaux" to handle generic binaries loading.

Quoted 11 lines

Quoted 8 lines

Quoted 5 lines

Note that the bootaux command retrieved the entry point (PC) from the elf binary.

Could you make this code flexible enough to handle not only elf binaries, but also other data (e.g. FPGA bitstreams)?

The code of bootaux is rather small, I don't see much value into stuff boot code for other peripherals into it.

Quoted 7 lines

But I'm not asking to write support for other vendor's SoC/use cases.

I'm just wondering if you could write generic command (framework) to support this use case and as an example add support for your IMX's Cortex-M3/4.

Sure, mv arch/arm/imx-common/imx_bootaux.c cmd/, there is the framework :-)

But this promotes the M class specific binary format to a generic format supported for all cores.

Quoted 15 lines

We would kill two birds with one stone - IMX is supported from the very beginning and we do have generic code to extend it by other vendors.

Quoted 4 lines

I don't know how FPGA
bistream loading typically works, but I guess it is quite different from loading a firmware into RAM/SRAM and flush caches...

FPGA quirks would be handled in arch/SoC specific code.

Quoted 3 lines

I am not against reuse and unification, I just feel that this is not really a case where we gain much.

Quoted 7 lines

With generic "bootaux/bootm" command we can point other developers who would like to add such booting code to the already available framework.

Also, we would prevent other "ad-hoc" commands adding to u-boot.

Extending bootm does not seem like a good idea. bootm is already rather complex, making it even more complex by supporting the auxiliary core case seems really not work well.

Also, bootm supports lots of features which you probably never would use or test on a remote core (e.g. initramfs etc...)

Quoted 34 lines

Quoted 21 lines

Quoted 21 lines

Maybe it would better to:
-------------------------

Embrace those binaries into FIT file (*.itb)? And allow multiple binaries loading? I'm thinking of work similar to one already did by Andre Przywara for SPL:

"[PATCH v3 00/19] SPL: extend FIT loading support" posted on 31.03.2017?

In that way we would "open" many new use cases, and other platforms (e.g. FPGA's, TI, etc) could benefit from it. One solid use case is to load different Linux binaries (or/and bare metal programs) to different SoC cores.

The "meta data" (i.e. load address, data type, description), could be extracted from the FIT format (the code is already in u-boot).

Quoted 6 lines

Quoted 1 line

IMHO, this is very generic approach.

I did some experiments with using FIT images for auxiliary core firmware, however, it did not add a lot of advantage over using elf:
http://git.toradex.com/cgit/u-boot-toradex.git/commit/?h=2015.04-toradex-next&id=d1d416f272e840e8139aec911f89a70fe5523eb2

Unfortunately, not all use cases allow using ELF format. FIT gives more flexibility:

(quoted diff) +0 -2

-  ./doc/README.dfutftp -> different images loading
 
-  Andre's patch to load multiple binaries in SPL - [PATCH v3

Quoted 2 lines

00/19] SPL: extend FIT loading support"

Are flexible, but very much U-Boot specific. And much more complex to support.

Quoted 10 lines

Quoted 7 lines

Firmwares are already built and available in the elf file format. The Linux remoteproc framework, which is meant to handle this kind of cores too, supports elf firmware loading too, so supporting elf in U-Boot too is a nice alignment. Also using FIT adds an additional step when building firm wares...

But this is all OK.

Quoted 8 lines

Quoted 5 lines

The ELF binary would be wrapped in FIT (with e.g. "elf" property, even 1 to 1 mapping). Then the 'bootaux/bootm' could parse ELF (which is also generic). And then some "IMX specific" (arch/SoC) code would be executed.

So we go from a nacked binary loaded directly to the place it has been linked to, to a double wrapped firmware file...?

We would have elf binary file embedded into FIT file, these would bring flexibility.

FIT support is in place (u-boot/spl).

In such a way you can use any binary in any format.

But I must admit that we are going off-topic here.....

Quoted 11 lines

Not sure if user will appriciate that extra work and boot time. And developers the extra code.

Maybe in a perfect world that just works, because you know, FIT is generic, and elf is generic... But that is just not how it works in practice. The existing FIT code is rather complex, supports lots of corner cases. The elf code supports different header types... Loading the elf sections (which use M4 specific addressing) needs address translation to get to suitable host addresses...

Quoted 8 lines

Maybe I'm an idealist :-)

One analogy comes to my mind between linux and u-boot.

The proliferation of u-boot commands and linux board files. Why Linux spend tremendous resources to remove board files and switch to device tree?

I argue that is the right way of doing it: Do ad-hoc solutions, whatever makes sense, and once you understand the full breath of design space it is much easier to build a suitable framework. At that point, do the refactoring and build that suitable framework.

If you design a framework without the understanding of the whole design space, you will end up with something which does not work well and needs lots of work-arounds etc...

Quoted 3 lines

Of course, it is a bit different since both examples have outside facing impact (change to device tree as well as changes to U-Boot commands).

To ask the silly question, isn't cmd/remoteproc.c part of the proper framework for a solution here?

Quoted 283 lines · Stefan Agner

Quoted 264 lines · Lukasz Majewski

Hi Stefan,

Quoted 56 lines · Lukasz Majewski

Quoted 43 lines

Hi Stefan,

Quoted 34 lines

Hi Lukasz,

Quoted 31 lines · Lukasz Majewski

Hi Stefan,

Thanks for your patch. Please allow me to share some ideas for improvements.

Quoted 8 lines

From: Stefan Agner <[email protected]>

This patchset enables to boot elf binaries on secondary Cortex-M class cores available on i.MX 6SoloX/7Solo/7Dual. This makes handling and loading firmwares much more convinient since all information where the firmware has to be loaded to is contained in the elf headers. A typical usage looks like this:

Quoted 11 lines

Colibri iMX7 # tftp ${loadaddr} firmware.elf && bootaux
${loadaddr} Using FEC0 device
TFTP from server 192.168.10.1; our IP address is 192.168.10.2 Filename 'firmware.elf'.
Load address: 0x80800000
Loading: ##################################################
88.3 KiB 5.4 MiB/s
done
Bytes transferred = 90424 (16138 hex)
## Starting auxiliary core at 0x1FFF8311 ... Colibri iMX7 #

I can find some other platforms (not only IMX), which would benefit from this code - the generic 'bootaux' command.

One good example would to allow multiple binaries for different SoC Cores (e.g. 2x Cortex-A8) to be loaded and started by u-boot.

Quoted 6 lines

Quoted 2 lines

Hence, I'm wondering if you could make those patches usable for other platforms as well?

I don't think that this is a good idea. bootaux is meant for auxiliary cores, which often use a different architecture and are not cache coherent (hence the cache flushes).

Quoted 7 lines

I do remember, that I saw similar "tiny" patch to add "just one" ad-hoc command to do the same (conceptually) for TI SoC floating on the u-boot mailing list.

Please correct me if I'm wrong, but bootaux is IMX specific and does work, which very likely, will be also required by other SoC vendors (TI's Sitara is also equipped with Cortex-M4 and PRUSS).

bootaux is currently IMX specific, and its currently supported binary format is M-class specific.

Quoted 1 line

Quoted 5 lines

Quoted 2 lines

Unification of such effort can save us all a lot of trouble in a very near future.

In OSS, you do not develop for the future. It gets developed when its here.

I cannot agree here. When you perceive threat then you prepare for it.

Quoted 20 lines

Quoted 10 lines

Quoted 5 lines

On SMP systems the main operating system normally starts the secondary core.

I think that there are some conceptual similarities between loading code to SMP core and Cortex M3. Of course some "tweaks" would be needed.

There are conceptual similarities between a car and a truck, still, they are likely manufactured in a different assembly line, probably by a different producer.

I guess in the end it really depends on where you draw the line: There are differences between loading code which will get executed by the primary code, loading code to execute explicitly on a SMP core, and loading code to execute on a remote processor.

Quoted 8 lines

The first case is already well supported, and we need to keep support backward compatibility.

The second case, IMHO, is a somewhat silly case: A SMP system usually gets driven by a single OS image, and that OS makes sure to initialize the cores (maybe with the help of firmware, such as the PSCI interface on ARM). There is no need for a boot loader command to execute a image on a secondary core explicitly...

I do understand (and agree) with your point with SMP.

But, I do know at least one use case when somebody would like to start two binaries (those are bare metal programs, taking care of SoC setup, IPC, etc).

And maybe Linux with some FPGA based IP block already configured in u-boot.

Quoted 8 lines

The third case is probably a case where we could start think about unification efforts.

Quoted 3 lines

Quoted 2 lines

Otherwise, if you want to run them separately using U-Boot, maybe a new command such as bootsmp would be more suited.

Quoted 8 lines

Quoted 6 lines

Hmm - I do think that it would be too much:

- bootm for generic single core
- bootaux for IMX
- bootsmp for SMP (on IMX/TI/RK?)
- boot?? for ??

There is at least also bootz and bootelf.

I will reply to the other thread regarding bootelf.

Quoted 112 lines

Quoted 21 lines

I would like to avoid adding new commands for doing conceptually the same work.

Even better, we could extend bootm to support the "multi binary" case too, but IMHO it would be also correct to have "bootaux" to handle generic binaries loading.

Quoted 11 lines

Quoted 8 lines

Quoted 5 lines

Note that the bootaux command retrieved the entry point (PC) from the elf binary.

Could you make this code flexible enough to handle not only elf binaries, but also other data (e.g. FPGA bitstreams)?

The code of bootaux is rather small, I don't see much value into stuff boot code for other peripherals into it.

Quoted 7 lines

But I'm not asking to write support for other vendor's SoC/use cases.

I'm just wondering if you could write generic command (framework) to support this use case and as an example add support for your IMX's Cortex-M3/4.

Sure, mv arch/arm/imx-common/imx_bootaux.c cmd/, there is the framework :-)

But this promotes the M class specific binary format to a generic format supported for all cores.

Quoted 11 lines

We would kill two birds with one stone - IMX is supported from the very beginning and we do have generic code to extend it by other vendors.

Quoted 3 lines

I don't know how FPGA
bistream loading typically works, but I guess it is quite different from loading a firmware into RAM/SRAM and flush caches...

FPGA quirks would be handled in arch/SoC specific code.

Quoted 1 line

Quoted 9 lines

Quoted 2 lines

I am not against reuse and unification, I just feel that this is not really a case where we gain much.

With generic "bootaux/bootm" command we can point other developers who would like to add such booting code to the already available framework.

Also, we would prevent other "ad-hoc" commands adding to u-boot.

Extending bootm does not seem like a good idea. bootm is already rather complex, making it even more complex by supporting the auxiliary core case seems really not work well.

Also, bootm supports lots of features which you probably never would use or test on a remote core (e.g. initramfs etc...)

Quoted 32 lines

Quoted 21 lines

Quoted 21 lines

Maybe it would better to:
-------------------------

Embrace those binaries into FIT file (*.itb)? And allow multiple binaries loading? I'm thinking of work similar to one already did by Andre Przywara for SPL:

"[PATCH v3 00/19] SPL: extend FIT loading support" posted on 31.03.2017?

In that way we would "open" many new use cases, and other platforms (e.g. FPGA's, TI, etc) could benefit from it. One solid use case is to load different Linux binaries (or/and bare metal programs) to different SoC cores.

The "meta data" (i.e. load address, data type, description), could be extracted from the FIT format (the code is already in u-boot).

Quoted 6 lines

Quoted 1 line

IMHO, this is very generic approach.

I did some experiments with using FIT images for auxiliary core firmware, however, it did not add a lot of advantage over using elf:
http://git.toradex.com/cgit/u-boot-toradex.git/commit/?h=2015.04-toradex-next&id=d1d416f272e840e8139aec911f89a70fe5523eb2

Unfortunately, not all use cases allow using ELF format. FIT gives more flexibility:

Review

Quoted 4 lines

(quoted diff) +0 -2

-  ./doc/README.dfutftp -> different images loading
 
-  Andre's patch to load multiple binaries in SPL - [PATCH v3 00/19]
  SPL: extend FIT loading support"

Are flexible, but very much U-Boot specific. And much more complex to support.

Quoted 9 lines

Quoted 6 lines

Firmwares are already built and available in the elf file format. The Linux remoteproc framework, which is meant to handle this kind of cores too, supports elf firmware loading too, so supporting elf in U-Boot too is a nice alignment. Also using FIT adds an additional step when building firm wares...

But this is all OK.

Quoted 8 lines

Quoted 5 lines

The ELF binary would be wrapped in FIT (with e.g. "elf" property, even 1 to 1 mapping). Then the 'bootaux/bootm' could parse ELF (which is also generic). And then some "IMX specific" (arch/SoC) code would be executed.

So we go from a nacked binary loaded directly to the place it has been linked to, to a double wrapped firmware file...?

We would have elf binary file embedded into FIT file, these would bring flexibility.

FIT support is in place (u-boot/spl).

In such a way you can use any binary in any format.

But I must admit that we are going off-topic here.....

Quoted 10 lines

Not sure if user will appriciate that extra work and boot time. And developers the extra code.

Maybe in a perfect world that just works, because you know, FIT is generic, and elf is generic... But that is just not how it works in practice. The existing FIT code is rather complex, supports lots of corner cases. The elf code supports different header types... Loading the elf sections (which use M4 specific addressing) needs address translation to get to suitable host addresses...

Quoted 8 lines

Maybe I'm an idealist :-)

One analogy comes to my mind between linux and u-boot.

The proliferation of u-boot commands and linux board files. Why Linux spend tremendous resources to remove board files and switch to device tree?

I argue that is the right way of doing it: Do ad-hoc solutions, whatever makes sense, and once you understand the full breath of design space it is much easier to build a suitable framework. At that point, do the refactoring and build that suitable framework.

If you design a framework without the understanding of the whole design space, you will end up with something which does not work well and needs lots of work-arounds etc...

Quoted 2 lines

Of course, it is a bit different since both examples have outside facing impact (change to device tree as well as changes to U-Boot commands).

To ask the silly question, isn't cmd/remoteproc.c part of the proper framework for a solution here?