sunxi-babelfish (FEX-to-DT translator)
Hi everyone,
One of the great strength of Allwinner's kernel is that it behaves pretty much like DT does: their kernel image is mostly generic, and all the device specific part is stored in a binary file (compiled from a FEX file), that gets loaded together with the kernel by the bootloader.
The kernel will then parse that file to get all the informations it needs (which devices are enabled, which pins are muxed to what function, etc.). Sounds familiar?
FEX and DT however are not compatible at all, so one board out there might not have any support in mainline, while it already passes the hardware description to the kernel.
Hence why I started to work on sunxi-babelfish. It aims at being a very thin wrapper around the kernel image, that would generate an uImage, with the babelfish code itself, plus the compiled DTSI we have for the targetted SoCs, and finally the kernel image. During bootup, babelfish will take one of these DTBs, and complete it using the informations fetched from the compiled FEX file the board is feeding it, before finally passing the DTB to Linux that will continue its usual boot.
I've been working on this for a week, but we already have all the pieces of infrastructure needed, plus some basic support for the devices themselves. Currently, babelfish will only care about the ATAGS and the UARTs, but I expect the list of devices it's aware of to grow quite quickly.
The code is stored here:
https://github.com/mripard/sunxi-babelfish/
There's a lot of room for improvements, so patches are welcome :)
Once downloaded, you can compile it using:
CROSS_COMPILE=arm-linux- ZIMAGE=kernel/arch/arm/boot/zImage make uImage
It will obviously generate a uImage, that you will be able to boot from u-boot.
This has been tested on the A20-olinuxino, so there might be some glitches on the other SoCs, but I don't really expect any at the moment, since stuff are pretty generic for now.
Another nice thing about it is that a FEX-to-DT compiler comes at no cost afterwards. Since dtc is able to generate a DTS from /proc/device-tree, you can easily generate a DTS for your previously non-supported board using
dtc -I fs -O dts -o my-board.dts /proc/device-tree/
Anyway, enough talking. Any feedback is, as usual, appreciated.
Maxime
Signature · 10 lines
Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20140107/46139cde/attachment.sig>
Quoted 1 line · Maxime Ripard
Anyway, enough talking. Any feedback is, as usual, appreciated.
I think it's a pretty cool idea. Have you considered integrating this with the impedence-matcher project at
https://github.com/zonque/pxa-impedance-matcher ?
That one basically does the same thing for atags-based boot loaders and I would guess that parts of the implementation can be shared as well, plus the combination could support a wider range of devices.
Arnd
Hi Arnd,
Quoted 4 lines · Arnd Bergmann
Quoted 1 line · Maxime Ripard
Anyway, enough talking. Any feedback is, as usual, appreciated.
Quoted 8 lines
I think it's a pretty cool idea. Have you considered integrating this with the impedence-matcher project at
https://github.com/zonque/pxa-impedance-matcher ?
That one basically does the same thing for atags-based boot loaders and I would guess that parts of the implementation can be shared as well, plus the combination could support a wider range of devices.
Yep, I've considered it at first. Actually, I even took some bits out of it to use them.
But I thought this would be too far off the pxa-impedance-matcher to bother trying to fit it in. IIRC, pxa-impedance-matcher is taking the mach-id (mostly) to take an already existing DT and give it to the kernel, while we have pretty much the opposite where we have a generic mach-id, and build up the DTB from another source of information.
I also integrated some fexc bits to be able to parse the script.bin file, which is highly sunxi specific.
But if Daniel's interested in merging all of this, I can totally consider it more seriously.
Maxime
Signature · 10 lines
Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20140108/bd63bdf9/attachment.sig>
Quoted 9 lines · On Wednesday 08 January 2014 19:58:37 Maxime Ripard wrote:
Quoted 8 lines
I think it's a pretty cool idea. Have you considered integrating this with the impedence-matcher project at
https://github.com/zonque/pxa-impedance-matcher ?
That one basically does the same thing for atags-based boot loaders and I would guess that parts of the implementation can be shared as well, plus the combination could support a wider range of devices.
Quoted 8 lines
Yep, I've considered it at first. Actually, I even took some bits out of it to use them.
But I thought this would be too far off the pxa-impedance-matcher to bother trying to fit it in. IIRC, pxa-impedance-matcher is taking the mach-id (mostly) to take an already existing DT and give it to the kernel, while we have pretty much the opposite where we have a generic mach-id, and build up the DTB from another source of information.
Yes, I understand the difference.
Quoted 2 lines
I also integrated some fexc bits to be able to parse the script.bin file, which is highly sunxi specific.
I believe that the pxa-impedance-matcher now also contains some other parts that go beyond just picking the right dtb file and interpreting the atags, but I don't think any of them is as complex as the fex parsing.
Quoted 2 lines
But if Daniel's interested in merging all of this, I can totally consider it more seriously.
Let's see what he thinks then.
Arnd
Quoted 8 lines · Maxime Ripard
Hence why I started to work on sunxi-babelfish. It aims at being a very thin wrapper around the kernel image, that would generate an uImage, with the babelfish code itself, plus the compiled DTSI we have for the targetted SoCs, and finally the kernel image. During bootup, babelfish will take one of these DTBs, and complete it using the informations fetched from the compiled FEX file the board is feeding it, before finally passing the DTB to Linux that will continue its usual boot.
Very cool!
Quoted 7 lines
[...]
Another nice thing about it is that a FEX-to-DT compiler comes at no cost afterwards. Since dtc is able to generate a DTS from/proc/device-tree, you can easily generate a DTS for your previously non-supported board usingdtc -I fs -O dts -o
my-board.dts/proc/device-tree/
I wonder if it might also be useful to compile this as a normal user space application which takes the fex and produces the dtb (or dts)?
I suppose producing something upstreamable would be different/harder though, since it would need to not inline/merge the dtsi file but just leave the include stmt. That's probably a different tool.
Ian.
Quoted 10 lines · Ian Campbell
Quoted 6 lines · Maxime Ripard
Another nice thing about it is that a FEX-to-DT compiler comes at no cost afterwards. Since dtc is able to generate a DTS from
/proc/device-tree, you can easily generate a DTS for your previously non-supported board usingdtc -I fs -O dts -o
my-board.dts/proc/device-tree/I wonder if it might also be useful to compile this as a normal user space application which takes the fex and produces the dtb (or dts)?
At the moment, no, libfdt only operates on DTB, so we would have to chain this somehow with dtc to at least do the DTB to DTS conversion I guess.
It's not something that I'm pretty fond of either. Or, I don't really see the benefit of it.
Let's say that you have such tool, I'm guessing one would want it to mass-convert fex files to DT. But then, you'd still have to boot that DT to be sure that everything is working, so you're pretty much losing the mass-convertion thing, since you would have boot this with a baremetal babelfish anyway.
Quoted 3 lines
I suppose producing something upstreamable would be different/harder though, since it would need to not inline/merge the dtsi file but just leave the include stmt. That's probably a different tool.
Yeah, dtc outputs a "flat" DTS, without any include, so you still need to edit it by hand to remove the common part, remove the duplicated pinctrl nodes, fix the phandle, etc.
But most of that can be scripted I guess.
Signature · 10 lines
Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20140108/9d418e5e/attachment.sig>
Quoted 16 lines · Maxime Ripard
Quoted 10 lines · Ian Campbell
Quoted 6 lines · Maxime Ripard
Another nice thing about it is that a FEX-to-DT compiler comes at no cost afterwards. Since dtc is able to generate a DTS from
/proc/device-tree, you can easily generate a DTS for your previously non-supported board usingdtc -I fs -O dts -o
my-board.dts/proc/device-tree/I wonder if it might also be useful to compile this as a normal user space application which takes the fex and produces the dtb (or dts)?
At the moment, no, libfdt only operates on DTB, so we would have to chain this somehow with dtc to at least do the DTB to DTS conversion I guess.
Quoted 8 lines
It's not something that I'm pretty fond of either. Or, I don't really see the benefit of it.
Let's say that you have such tool, I'm guessing one would want it to mass-convert fex files to DT. But then, you'd still have to boot that DT to be sure that everything is working, so you're pretty much losing the mass-convertion thing, since you would have boot this with a baremetal babelfish anyway.
I wasn't really thinking of mass conversion. just an easy way for people to produce a starting point for their platform to send upstream.
Quoted 9 lines
Quoted 3 lines
I suppose producing something upstreamable would be different/harder though, since it would need to not inline/merge the dtsi file but just leave the include stmt. That's probably a different tool.
Yeah, dtc outputs a "flat" DTS, without any include, so you still need to edit it by hand to remove the common part, remove the duplicated pinctrl nodes, fix the phandle, etc.
But most of that can be scripted I guess.
I think this could be done as a separate tool which read a FEX and outputted the dts directly (i.e. with printf). The output would have the #include for the right dtsi in place and the appropriate overrides derived from the FEX. That's not really anything like your sunxi-babelfish though.
Ian.
Hi Ian,
Quoted 37 lines · Ian Campbell
Quoted 16 lines · Maxime Ripard
Quoted 10 lines · Ian Campbell
Quoted 6 lines · Maxime Ripard
Another nice thing about it is that a FEX-to-DT compiler comes at no cost afterwards. Since dtc is able to generate a DTS from
/proc/device-tree, you can easily generate a DTS for your previously non-supported board usingdtc -I fs -O dts -o
my-board.dts/proc/device-tree/I wonder if it might also be useful to compile this as a normal user space application which takes the fex and produces the dtb (or dts)?
At the moment, no, libfdt only operates on DTB, so we would have to chain this somehow with dtc to at least do the DTB to DTS conversion I guess.
Quoted 8 lines
It's not something that I'm pretty fond of either. Or, I don't really see the benefit of it.
Let's say that you have such tool, I'm guessing one would want it to mass-convert fex files to DT. But then, you'd still have to boot that DT to be sure that everything is working, so you're pretty much losing the mass-convertion thing, since you would have boot this with a baremetal babelfish anyway.
I wasn't really thinking of mass conversion. just an easy way for people to produce a starting point for their platform to send upstream.
Quoted 8 lines
Quoted 3 lines
I suppose producing something upstreamable would be different/harder though, since it would need to not inline/merge the dtsi file but just leave the include stmt. That's probably a different tool.
Yeah, dtc outputs a "flat" DTS, without any include, so you still need to edit it by hand to remove the common part, remove the duplicated pinctrl nodes, fix the phandle, etc.
Quoted 7 lines
Quoted 1 line
But most of that can be scripted I guess.
I think this could be done as a separate tool which read a FEX and outputted the dts directly (i.e. with printf). The output would have the #include for the right dtsi in place and the appropriate overrides derived from the FEX. That's not really anything like your sunxi-babelfish though.
I was thinking about exactly this in the context of my (stalled) online fex / board submission helper tool.
I.e. $RANDOM_USER submits their script.bin / script.fex file then it produces (amongst other things) a dts based on it's contents using sensible conversions based on what other knowledgeable people have done in the past.
I was also thinking about adding a mass-converting interface to this tool so we could regenerate the u-boot memory info and kernel dts files from the contents of sunxi-boards.
Thanks,
Signature · 5 lines
Julian Calaby Email: julian.calaby at gmail.com Profile: http://www.google.com/profiles/julian.calaby/ .Plan: http://sites.google.com/site/juliancalaby/