Moving ARM dts files
Olof, Arnd,
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERS will also need updating.
A few questions:
Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can.
Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example.
What to do with armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things.
Sub-arch maintainers,
'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues.
Rob
8<----------------------------------------------------------- #!/usr/bin/env python3
import os
import re
from git import Git
import glob
vendor_map = {
'imx': 'fsl',
'ls': 'fsl',
'vf': 'fsl',
'qcom': 'qcom',
'am3' : 'ti',
'am4' : 'ti',
'am5' : 'ti',
'da' : 'ti',
'dm' : 'ti',
'dra' : 'ti',
'keystone' : 'ti',
'omap' : 'ti',
'nspire' : 'ti',
'armada' : 'marvell',
'berlin' : 'marvell',
'dove' : 'marvell',
'kirkwood' : 'marvell',
'orion' : 'marvell',
'pxa' : 'marvell',
'mvebu' : 'marvell',
'mmp2' : 'marvell',
'arm-' : 'arm',
'integ' : 'arm',
've' : 'arm',
'aspeed' : 'aspeed',
'at91' : 'atmel',
'sama' : 'atmel',
'bcm' : 'brcm',
'exynos' : 'samsung',
's3c' : 'samsung',
's5p' : 'samsung',
'gemini' : 'cortina',
'hi3' : 'hisilicon',
'hip' : 'hisilicon',
'hisi' : 'hisilicon',
'mt' : 'mediatek',
'meson' : 'amlogic',
'owl' : 'actions',
'r7' : 'renesas',
'r8' : 'renesas',
'r9' : 'renesas',
'rk' : 'rockchip',
'socfpga' : 'altera',
'st' : 'st',
'spear' : 'st',
'sun' : 'allwinner',
'tegra' : 'nvidia',
'zynq' : 'xilinx',
'wm' : 'wm',
'uniph' : 'socionext',
'zx' : 'zte',
}if __name__ == "__main__":
g = Git('.')
g.checkout("HEAD", "arch/arm/boot/dts/Makefile")
dts_make = open("arch/arm/boot/dts/Makefile", "r").read()
# make entries 1 line
make2 = re.sub(r'\\\n', '', dts_make)
for k,v in vendor_map.items():
for f in glob.iglob("arch/arm/boot/dts/" + k + "*.*"):
new_dir = "arch/arm/boot/dts/" + v + "/"
base = os.path.splitext(os.path.basename(f))[0]
os.makedirs(new_dir, exist_ok=True)
g.mv(f, new_dir)
# Remove the file from the makefile
dts_make = re.sub('.*' + base + r'\.dtb.*\\\n', '', dts_make)
dts_make = re.sub('.*' + base + r'\.dtb', '', dts_make)
# extract the matching makefile entry
reg = re.search(r'.*' + base + r'.*', make2)
if not reg:
continue
entry = reg.group(0)
make2 = re.sub(r'.*' + base + r'.*', '', make2)
if entry:
makefile = open(new_dir + 'Makefile', 'a+')
print(entry, file=makefile)
makefile.close()
for d in sorted(glob.iglob("arch/arm/boot/dts/*/")):
dts_make += 'subdir-y += ' + d.split(os.path.sep)[-2] + '\n'
# Add license and sort entries of sub-dir makefile
vendor_make = '# SPDX-License-Identifier: GPL-2.0\n'
make_lines = open(d + 'Makefile', 'r').readlines()
for l in sorted(make_lines):
vendor_make += l
vendor_make = re.sub(r'\t', r'\\\n\t', vendor_make)
f = open(d + 'Makefile', 'w')
f.write(vendor_make)
f.close()
g.add(d + 'Makefile')
# Remove entries with no dtbs left
dts_make = re.sub(r'.*\+= \\\n\n', '', dts_make)
open("arch/arm/boot/dts/Makefile", "w").write(dts_make)
g.add("arch/arm/boot/dts/Makefile")Hello Rob,
Quoted 56 lines · Rob Herring
Olof, Arnd,
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can.
Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example.
What to do with
armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things.Sub-arch maintainers,
'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues.Rob
8<----------------------------------------------------------- #!/usr/bin/env python3
import os
import re
from git import Git
import globvendor_map = { 'imx': 'fsl', 'ls': 'fsl', 'vf': 'fsl', 'qcom': 'qcom', 'am3' : 'ti', 'am4' : 'ti', 'am5' : 'ti', 'da' : 'ti', 'dm' : 'ti', 'dra' : 'ti', 'keystone' : 'ti', 'omap' : 'ti', 'nspire' : 'ti', 'armada' : 'marvell', 'berlin' : 'marvell', 'dove' : 'marvell', 'kirkwood' : 'marvell', 'orion' : 'marvell', 'pxa' : 'marvell',
Quoted 8 lines
'mvebu' : 'marvell',
'mmp2' : 'marvell',
'arm-' : 'arm',
'integ' : 'arm',
've' : 'arm',
'aspeed' : 'aspeed',
'at91' : 'atmel',
'sama' : 'atmel',Unfortunately, we have many boards without a proper prefix. Wouldn't it be better to use arch/arm/boot/dts/Makefile and map dtb-$(CONFIG_*) to a vendor directory?
This way you would be sure to not miss any. This would also ease the creation of subdirs if we decide to do that.
Signature · 3 lines
Alexandre Belloni, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Quoted 69 lines · Alexandre Belloni
Hello Rob,
Quoted 56 lines · Rob Herring
Olof, Arnd,
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can.
Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example.
What to do with
armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things.Sub-arch maintainers,
'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues.Rob
8<----------------------------------------------------------- #!/usr/bin/env python3
import os
import re
from git import Git
import globvendor_map = { 'imx': 'fsl', 'ls': 'fsl', 'vf': 'fsl', 'qcom': 'qcom', 'am3' : 'ti', 'am4' : 'ti', 'am5' : 'ti', 'da' : 'ti', 'dm' : 'ti', 'dra' : 'ti', 'keystone' : 'ti', 'omap' : 'ti', 'nspire' : 'ti', 'armada' : 'marvell', 'berlin' : 'marvell', 'dove' : 'marvell', 'kirkwood' : 'marvell', 'orion' : 'marvell', 'pxa' : 'marvell',Quoted 8 lines
'mvebu' : 'marvell', 'mmp2' : 'marvell', 'arm-' : 'arm', 'integ' : 'arm', 've' : 'arm', 'aspeed' : 'aspeed', 'at91' : 'atmel', 'sama' : 'atmel',
Quoted 3 lines
Unfortunately, we have many boards without a proper prefix. Wouldn't it be better to use
arch/arm/boot/dts/Makefileand map dtb-$(CONFIG_*) to a vendor directory?
Only 13 boards by my count Atmel. And 2 boards each for omap3 and Renesas. That's manageable to do by hand or just add to the map.
Though in hindsight I probably would have used the config.
Quoted 2 lines
This way you would be sure to not miss any. This would also ease the creation of subdirs if we decide to do that.
Quoted 70 lines · Rob Herring
Olof, Arnd,
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can.
Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example.
What to do with
armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things.Sub-arch maintainers,
'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues.Rob
8<----------------------------------------------------------- #!/usr/bin/env python3
import os
import re
from git import Git
import globvendor_map = { 'imx': 'fsl', 'ls': 'fsl', 'vf': 'fsl', 'qcom': 'qcom', 'am3' : 'ti', 'am4' : 'ti', 'am5' : 'ti', 'da' : 'ti', 'dm' : 'ti', 'dra' : 'ti', 'keystone' : 'ti', 'omap' : 'ti', 'nspire' : 'ti', 'armada' : 'marvell', 'berlin' : 'marvell', 'dove' : 'marvell', 'kirkwood' : 'marvell', 'orion' : 'marvell', 'pxa' : 'marvell', 'mvebu' : 'marvell', 'mmp2' : 'marvell', 'arm-' : 'arm', 'integ' : 'arm', 've' : 'arm', 'aspeed' : 'aspeed', 'at91' : 'atmel', 'sama' : 'atmel', 'bcm' : 'brcm', 'exynos' : 'samsung', 's3c' : 'samsung', 's5p' : 'samsung', 'gemini' : 'cortina', 'hi3' : 'hisilicon',
Quoted 8 lines
'hip' : 'hisilicon',
'hisi' : 'hisilicon',
'mt' : 'mediatek',
'meson' : 'amlogic',
'owl' : 'actions',
'r7' : 'renesas',
'r8' : 'renesas',
'r9' : 'renesas',The above 'r' prefixes could be a bit longer if desired.
e.g.:
'r7s'
'r8a'
'r9a'Quoted 10 lines
'rk' : 'rockchip',
'socfpga' : 'altera',
'st' : 'st',
'spear' : 'st',
'sun' : 'allwinner',
'tegra' : 'nvidia',
'zynq' : 'xilinx',
'wm' : 'wm',
'uniph' : 'socionext',
'zx' : 'zte',For renesas I believe you also want:
'emev2' 'gr-peach' 'iwg20d-q7' 'sh'
Quoted 58 lines
}
if
__name__== "__main__":g = Git('.') g.checkout("HEAD", "arch/arm/boot/dts/Makefile") dts_make = open("arch/arm/boot/dts/Makefile", "r").read() # make entries 1 line make2 = re.sub(r'\\\n', '', dts_make) for k,v in vendor_map.items(): for f in glob.iglob("arch/arm/boot/dts/" + k + "*.*"): new_dir = "arch/arm/boot/dts/" + v + "/" base = os.path.splitext(os.path.basename(f))[0] os.makedirs(new_dir, exist_ok=True) g.mv(f, new_dir) # Remove the file from the makefile dts_make = re.sub('.*' + base + r'\.dtb.*\\\n', '', dts_make) dts_make = re.sub('.*' + base + r'\.dtb', '', dts_make) # extract the matching makefile entry reg = re.search(r'.*' + base + r'.*', make2) if not reg: continue entry = reg.group(0) make2 = re.sub(r'.*' + base + r'.*', '', make2) if entry: makefile = open(new_dir + 'Makefile', 'a+') print(entry, file=makefile) makefile.close() for d in sorted(glob.iglob("arch/arm/boot/dts/*/")): dts_make += 'subdir-y += ' + d.split(os.path.sep)[-2] + '\n' # Add license and sort entries of sub-dir makefile vendor_make = '# SPDX-License-Identifier: GPL-2.0\n' make_lines = open(d + 'Makefile', 'r').readlines() for l in sorted(make_lines): vendor_make += l vendor_make = re.sub(r'\t', r'\\\n\t', vendor_make) f = open(d + 'Makefile', 'w') f.write(vendor_make) f.close() g.add(d + 'Makefile') # Remove entries with no dtbs left dts_make = re.sub(r'.*\+= \\\n\n', '', dts_make) open("arch/arm/boot/dts/Makefile", "w").write(dts_make) g.add("arch/arm/boot/dts/Makefile")
Rob,
Am 04.12.18 um 19:36 schrieb Rob Herring:
Quoted 10 lines
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs?
This refactoring is a terrible idea!
While it would've been nice to have more structure from the start, bootloaders like U-Boot expect a flat structure for arm .dtb files now. If you start installing them into subdirs instead, they won't find the files anymore under the hardcoded name.
Doing this only for new platforms would be much less invasive and allow to prepare bootloaders accordingly. Alternatively, white-list which ones are safe to move around. But don't just script a refactoring because it looks nicer in the source tree, without testing what side effects this can have for board/distro users of the compiled files in practice. We already had that discussion for arm64 because Debian chose to ignore the kernel-installed subdirectories and installed .dtb files into a flat directory, which collided with openSUSE sticking to the kernel choice.
This topic becomes even more important with EBBR: There is neither a mechanism in place to sync .dts files into U-Boot or EDK2 source trees, nor are capsule updates implemented in U-Boot for easily deploying such bootloaders with new .dts sources or paths yet. And I can assure you that just getting users to dd the right bootloader can be difficult... Since DT forward and backward compatibility is often being neglected, for example with optional properties or renamed compatibles that break booting with previous drivers, new kernel versions often need updated Device Trees to make use of new/enhanced drivers. Therefore it is unfortunately often enough a necessity to load newer kernel-based .dtb files matching the kernel (as opposed to the dream of kernel-independent hardware descriptions) when working with the latest -rc or -next kernels at least. For examples of DTs needing updates, look no further than Linaro's 96boards - in case of hikey960/EDK2 GRUB is another layer where .dtb paths may be hardcoded, ditto for arm; and Armada was an example where the upstream bindings for the network IP changed incompatibly.
DT overlays are another topic that is not making any progress upstream according to the ELCE BoF, so beyond the Raspberry Pi the only known working way to apply them is to write a U-Boot boot.scr script, which can either reuse $fdtcontroladdr DT or use the filename $fdtfile or hardcode one, the latter two of which would break with your renaming.
So expect people to be using .dtb files, expect them to be affected by file movements to subdirectories here, and don't expect each user to understand or be able to fix things themselves if they fall apart as result of your changes and they suddenly no longer have Ethernet/Wifi.
Regards,
Andreas
Signature · 7 lines
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) _______________________________________________ boot-architecture mailing list [email protected] https://lists.linaro.org/mailman/listinfo/boot-architecture
Quoted 9 lines · Andreas Färber
Rob,
Am 04.12.18 um 19:36 schrieb Rob Herring:
Quoted 5 lines
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
Quoted 7 lines
Quoted 5 lines
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs?
This refactoring is a terrible idea!
How do you really feel?
Quoted 7 lines
While it would've been nice to have more structure from the start, bootloaders like U-Boot expect a flat structure for arm .dtb files now. If you start installing them into subdirs instead, they won't find the files anymore under the hardcoded name.
Doing this only for new platforms would be much less invasive and allow to prepare bootloaders accordingly.
That was my suggestion where this started for the new RDA platform. Olof preferred to move everything and that's my desire too.
Quoted 2 lines
Alternatively, white-list which ones
are safe to move around.
I'd prefer to know which ones the distros don't want moved. That should be easier to figure out. We also need that anyways in context of what platforms we care about compatibility.
Another option is dtbs_install target could flatten the installed dtbs. That is the only part the distros should depend on.
Quoted 6 lines
But don't just script a refactoring because it
looks nicer in the source tree, without testing what side effects this can have for board/distro users of the compiled files in practice. We already had that discussion for arm64 because Debian chose to ignore the kernel-installed subdirectories and installed .dtb files into a flat directory, which collided with openSUSE sticking to the kernel choice.
So everyone already deals with subdirs or not with arm and arm64 already, seems like they can deal with this. I will raise the topic on the cross-distro list though.
Quoted 4 lines
This topic becomes even more important with EBBR: There is neither a mechanism in place to sync .dts files into U-Boot or EDK2 source trees, nor are capsule updates implemented in U-Boot for easily deploying such bootloaders with new .dts sources or paths yet.
EBBR actually says firmware (including dtbs) goes in directories named by vendor.
Quoted 5 lines
And I can assure you
that just getting users to dd the right bootloader can be difficult... Since DT forward and backward compatibility is often being neglected, for example with optional properties or renamed compatibles that break booting with previous drivers, new kernel versions often need updated
Quoted 8 lines
Device Trees to make use of new/enhanced drivers. Therefore it is unfortunately often enough a necessity to load newer kernel-based .dtb files matching the kernel (as opposed to the dream of kernel-independent hardware descriptions) when working with the latest -rc or -next kernels at least. For examples of DTs needing updates, look no further than Linaro's 96boards - in case of hikey960/EDK2 GRUB is another layer where .dtb paths may be hardcoded, ditto for arm; and Armada was an example where the upstream bindings for the network IP changed incompatibly.
Compatibility is an issue, yes, but that really has nothing to do with this.
Quoted 5 lines
DT overlays are another topic that is not making any progress upstream according to the ELCE BoF, so beyond the Raspberry Pi the only known working way to apply them is to write a U-Boot boot.scr script, which can either reuse $fdtcontroladdr DT or use the filename $fdtfile or hardcode one, the latter two of which would break with your renaming.
DT overlays also have nothing to do with this as there aren't any in the kernel. I'm not inclined to take any either with a flat tree. We're already at 1800+ files.
Rob
_______________________________________________
boot-architecture mailing list
[email protected]
https://lists.linaro.org/mailman/listinfo/boot-architecture
Quoted 25 lines · Rob Herring
Quoted 9 lines · Andreas Färber
Rob,
Am 04.12.18 um 19:36 schrieb Rob Herring:
Quoted 5 lines
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
Quoted 7 lines
Quoted 5 lines
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs?
This refactoring is a terrible idea!
How do you really feel?
Quoted 5 lines
While it would've been nice to have more structure from the start, bootloaders like U-Boot expect a flat structure for arm .dtb files now. If you start installing them into subdirs instead, they won't find the files anymore under the hardcoded name.
Quoted 5 lines
Quoted 2 lines
Doing this only for new platforms would be much less invasive and allow to prepare bootloaders accordingly.
That was my suggestion where this started for the new RDA platform. Olof preferred to move everything and that's my desire too.
I think we might really be stuck with "new stuff only in subdirs". There's a whole huge world out there that depends on these layouts and it's almost but not quite like the API vs ABI thing. You can change the kernel API at whim but this is something with a lot of external users, and those folks aren't supposed to get broken.
Quoted 9 lines
Quoted 2 lines
Alternatively, white-list which ones
are safe to move around.I'd prefer to know which ones the distros don't want moved. That should be easier to figure out. We also need that anyways in context of what platforms we care about compatibility.
Another option is
dtbs_installtarget could flatten the installed dtbs. That is the only part the distros should depend on.
But it's not just "the distros" that'll get bit by this too. Or rather, it's the "big distros" and all the OpenEmbedded/buildroot/etc ones too that follow off the built-in logic to find the DTB that'll get bit too with a rename.
[snip]
Quoted 9 lines
Quoted 5 lines
DT overlays are another topic that is not making any progress upstream according to the ELCE BoF, so beyond the Raspberry Pi the only known working way to apply them is to write a U-Boot boot.scr script, which can either reuse $fdtcontroladdr DT or use the filename $fdtfile or hardcode one, the latter two of which would break with your renaming.
DT overlays also have nothing to do with this as there aren't any in the kernel. I'm not inclined to take any either with a flat tree. We're already at 1800+ files.
I'm going to do my best to not hijack the thread to be about overlays but that's a whole chicken and egg type problem too. But yes, it's not exactly related to moving or not existing dts/dtb files but something about where / how those are to be stored and maintained really does need to be done.
Signature · 1 line
Tom
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Am 05.12.18 um 05:17 schrieb Rob Herring:
Quoted 34 lines · Andreas Färber
Quoted 9 lines
Rob,
Am 04.12.18 um 19:36 schrieb Rob Herring:
Quoted 5 lines
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
Quoted 7 lines
Quoted 5 lines
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs?
This refactoring is a terrible idea!
How do you really feel?
Quoted 7 lines
While it would've been nice to have more structure from the start, bootloaders like U-Boot expect a flat structure for arm .dtb files now. If you start installing them into subdirs instead, they won't find the files anymore under the hardcoded name.
Doing this only for new platforms would be much less invasive and allow to prepare bootloaders accordingly.
That was my suggestion where this started for the new RDA platform. Olof preferred to move everything and that's my desire too.
Quoted 2 lines
Alternatively, white-list which ones
are safe to move around.
Quoted 6 lines
I'd prefer to know which ones the distros don't want moved. That should be easier to figure out. We also need that anyways in context of what platforms we care about compatibility.
Another option is
dtbs_installtarget could flatten the installed dtbs. That is the only part the distros should depend on.
I'd be okay with distinguishing source vs. install location. Due to the issue I mention below (and more) we can't use install_dtbs for openSUSE and had to reimplement it, which we'd need to (and can) adjust.
Quoted 10 lines
Quoted 6 lines
But don't just script a refactoring because it
looks nicer in the source tree, without testing what side effects this can have for board/distro users of the compiled files in practice. We already had that discussion for arm64 because Debian chose to ignore the kernel-installed subdirectories and installed .dtb files into a flat directory, which collided with openSUSE sticking to the kernel choice.So everyone already deals with subdirs or not with arm and arm64 already, seems like they can deal with this. I will raise the topic on the cross-distro list though.
Sounds like you're twisting words... The keyword was "hardcoded" paths - one way or another (not "and") depending on the kernel choices being flat for arm, vendor subdir for arm64.
Quoted 7 lines
Quoted 4 lines
This topic becomes even more important with EBBR: There is neither a mechanism in place to sync .dts files into U-Boot or EDK2 source trees, nor are capsule updates implemented in U-Boot for easily deploying such bootloaders with new .dts sources or paths yet.
EBBR actually says firmware (including dtbs) goes in directories named by vendor.
Fine, but unrelated.
Quoted 22 lines
Quoted 5 lines
And I can assure you
that just getting users to dd the right bootloader can be difficult... Since DT forward and backward compatibility is often being neglected, for example with optional properties or renamed compatibles that break booting with previous drivers, new kernel versions often need updatedQuoted 8 lines
Device Trees to make use of new/enhanced drivers. Therefore it is unfortunately often enough a necessity to load newer kernel-based .dtb files matching the kernel (as opposed to the dream of kernel-independent hardware descriptions) when working with the latest -rc or -next kernels at least. For examples of DTs needing updates, look no further than Linaro's 96boards - in case of hikey960/EDK2 GRUB is another layer where .dtb paths may be hardcoded, ditto for arm; and Armada was an example where the upstream bindings for the network IP changed incompatibly.
Compatibility is an issue, yes, but that really has nothing to do with this.
Quoted 5 lines
DT overlays are another topic that is not making any progress upstream according to the ELCE BoF, so beyond the Raspberry Pi the only known working way to apply them is to write a U-Boot boot.scr script, which can either reuse $fdtcontroladdr DT or use the filename $fdtfile or hardcode one, the latter two of which would break with your renaming.
Quoted 3 lines
DT overlays also have nothing to do with this as there aren't any in the kernel. I'm not inclined to take any either with a flat tree. We're already at 1800+ files.
Read again: a) Breaking DT changes and b) the desire to use Overlays instead of replacing the bootloaders for each change are _reasons_ why people depend on .dtb filenames from the kernel source tree for their boot flow today. Nothing to do with downstream .dtbo files.
For example, remember when I reported that the kernel didn't compile DTs with -@? No reaction except for Frank asking to be CC'ed - was it ever fixed??? Do EDK2's or U-Boot's built-in DTs compile with -@ today? Raspberry Pi overlays in U-Boot work because it switched to passing the DT through from the proprietary firmware.
Point being that while it would be nice to get a current, compatible DT via UEFI tables and ignore .dtb filenames outside of a few bootloaders, in reality we're not quite there yet for all platforms.
I see no problem (except for naming choices) moving new targets like RDA to subfolders because we can then hardcode it the new way; I also assume deeply embedded targets like stm32f4 or targets with no mainline bootloaders yet like owl-s500 could be refactored.
I do see problems refactoring widely used SBC targets like sunXi though.
Regards,
Andreas
Signature · 8 lines
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg) _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 40 lines · Andreas Färber
Am 05.12.18 um 05:17 schrieb Rob Herring:
Quoted 38 lines · Andreas Färber
Quoted 9 lines
Rob,
Am 04.12.18 um 19:36 schrieb Rob Herring:
Quoted 5 lines
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
Quoted 7 lines
Quoted 5 lines
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs?
This refactoring is a terrible idea!
How do you really feel?
Quoted 7 lines
While it would've been nice to have more structure from the start, bootloaders like U-Boot expect a flat structure for arm .dtb files now. If you start installing them into subdirs instead, they won't find the files anymore under the hardcoded name.
Doing this only for new platforms would be much less invasive and allow to prepare bootloaders accordingly.
That was my suggestion where this started for the new RDA platform. Olof preferred to move everything and that's my desire too.
Quoted 2 lines
Alternatively, white-list which ones
are safe to move around.I'd prefer to know which ones the distros don't want moved. That should be easier to figure out. We also need that anyways in context of what platforms we care about compatibility.
Quoted 6 lines
Quoted 2 lines
Another option is
dtbs_installtarget could flatten the installed dtbs. That is the only part the distros should depend on.I'd be okay with distinguishing source vs. install location. Due to the issue I mention below (and more) we can't use
install_dtbsfor openSUSE and had to reimplement it, which we'd need to (and can) adjust.
What would be needed for dtbs_install to work? arm64 needs to support a flat install? If it doesn't work for Debian or openSUSE, I'm not sure why we have it. So I'd like to make it work.
Quoted 20 lines
Quoted 10 lines
Quoted 6 lines
But don't just script a refactoring because it
looks nicer in the source tree, without testing what side effects this can have for board/distro users of the compiled files in practice. We already had that discussion for arm64 because Debian chose to ignore the kernel-installed subdirectories and installed .dtb files into a flat directory, which collided with openSUSE sticking to the kernel choice.So everyone already deals with subdirs or not with arm and arm64 already, seems like they can deal with this. I will raise the topic on the cross-distro list though.
Sounds like you're twisting words... The keyword was "hardcoded" paths - one way or another (not "and") depending on the kernel choices being flat for arm, vendor subdir for arm64.
Quoted 5 lines
Quoted 4 lines
This topic becomes even more important with EBBR: There is neither a mechanism in place to sync .dts files into U-Boot or EDK2 source trees, nor are capsule updates implemented in U-Boot for easily deploying such bootloaders with new .dts sources or paths yet.
Quoted 4 lines
Quoted 2 lines
EBBR actually says firmware (including dtbs) goes in directories named by vendor.
Fine, but unrelated.
If the distros want dtbs in a flat dir and EBBR says otherwise, then it is related.
Quoted 26 lines
Quoted 22 lines
Quoted 5 lines
And I can assure you
that just getting users to dd the right bootloader can be difficult... Since DT forward and backward compatibility is often being neglected, for example with optional properties or renamed compatibles that break booting with previous drivers, new kernel versions often need updatedQuoted 8 lines
Device Trees to make use of new/enhanced drivers. Therefore it is unfortunately often enough a necessity to load newer kernel-based .dtb files matching the kernel (as opposed to the dream of kernel-independent hardware descriptions) when working with the latest -rc or -next kernels at least. For examples of DTs needing updates, look no further than Linaro's 96boards - in case of hikey960/EDK2 GRUB is another layer where .dtb paths may be hardcoded, ditto for arm; and Armada was an example where the upstream bindings for the network IP changed incompatibly.
Compatibility is an issue, yes, but that really has nothing to do with this.
Quoted 5 lines
DT overlays are another topic that is not making any progress upstream according to the ELCE BoF, so beyond the Raspberry Pi the only known working way to apply them is to write a U-Boot boot.scr script, which can either reuse $fdtcontroladdr DT or use the filename $fdtfile or hardcode one, the latter two of which would break with your renaming.
Quoted 3 lines
DT overlays also have nothing to do with this as there aren't any in the kernel. I'm not inclined to take any either with a flat tree. We're already at 1800+ files.
Quoted 8 lines
Read again: a) Breaking DT changes and b) the desire to use Overlays instead of replacing the bootloaders for each change are _reasons_ why people depend on .dtb filenames from the kernel source tree for their boot flow today. Nothing to do with downstream .dtbo files.
For example, remember when I reported that the kernel didn't compile DTs with -@? No reaction except for Frank asking to be CC'ed - was it ever fixed??? Do EDK2's or U-Boot's built-in DTs compile with -@ today?
IIRC, Frank objected to changing this globally because it will bloat all dtbs. And then no one did the work to make it a per dtb option. Maybe that was the same issue in another thread.
Quoted 12 lines
Raspberry Pi overlays in U-Boot work because it switched to passing the DT through from the proprietary firmware.
Point being that while it would be nice to get a current, compatible DT via UEFI tables and ignore .dtb filenames outside of a few bootloaders, in reality we're not quite there yet for all platforms.I see no problem (except for naming choices) moving new targets like RDA to subfolders because we can then hardcode it the new way; I also assume deeply embedded targets like stm32f4 or targets with no mainline bootloaders yet like owl-s500 could be refactored.
I do see problems refactoring widely used SBC targets like sunXi though.
Quoted 7 lines
Regards,
Andreas--
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg)
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 9 lines · Rob Herring
Quoted 3 lines · Andreas Färber
I'd be okay with distinguishing source vs. install location. Due to the issue I mention below (and more) we can't use
install_dtbsfor openSUSE and had to reimplement it, which we'd need to (and can) adjust.What would be needed for
dtbs_installto work? arm64 needs to support a flat install? If it doesn't work for Debian or openSUSE, I'm not sure why we have it. So I'd like to make it work.
Correct me if I'm wrong but as far as the flat vs directory thing goes isn't the issue that this winds up being a rename for an existing 32 bit system? If you just install the dtbs in the default location then a bootloader or whatever that is hard coded to look for foo-bar.dtb won't see the new foo/foo-bar.dtb (or whatever) and will continue to use the old binary. It's not the fact that that it's in a directory, it's the fact that the bootloader sees the name it needs to look for change (if it's looking on a filesystem at all). This isn't a problem for arm64 as the location isn't changing, it's used directories from day one.
The issues with the existing install_dtbs sounded unrelated to this.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 12 lines · Mark Brown
Quoted 9 lines · Rob Herring
Quoted 3 lines · Andreas Färber
I'd be okay with distinguishing source vs. install location. Due to the issue I mention below (and more) we can't use
install_dtbsfor openSUSE and had to reimplement it, which we'd need to (and can) adjust.What would be needed for
dtbs_installto work? arm64 needs to support a flat install? If it doesn't work for Debian or openSUSE, I'm not sure why we have it. So I'd like to make it work.
Quoted 9 lines
Correct me if I'm wrong but as far as the flat vs directory thing goes isn't the issue that this winds up being a rename for an existing 32 bit system? If you just install the dtbs in the default location then a bootloader or whatever that is hard coded to look for foo-bar.dtb won't see the new foo/foo-bar.dtb (or whatever) and will continue to use the old binary. It's not the fact that that it's in a directory, it's the fact that the bootloader sees the name it needs to look for change (if it's looking on a filesystem at all). This isn't a problem for arm64 as the location isn't changing, it's used directories from day one.
Yeah, install needs to remain flat even if the dts files move into subdirectories. It will be painful for everybody if the install location moves.
Quoted 1 line
The issues with the existing
install_dtbssounded unrelated to this.
Agreed.
-Olof
_______________________________________________
boot-architecture mailing list
[email protected]
https://lists.linaro.org/mailman/listinfo/boot-architecture
Quoted 12 lines · Mark Brown
Quoted 9 lines · Rob Herring
Quoted 3 lines · Andreas Färber
I'd be okay with distinguishing source vs. install location. Due to the issue I mention below (and more) we can't use
install_dtbsfor openSUSE and had to reimplement it, which we'd need to (and can) adjust.What would be needed for
dtbs_installto work? arm64 needs to support a flat install? If it doesn't work for Debian or openSUSE, I'm not sure why we have it. So I'd like to make it work.
Quoted 8 lines
Correct me if I'm wrong but as far as the flat vs directory thing goes isn't the issue that this winds up being a rename for an existing 32 bit system? If you just install the dtbs in the default location then a bootloader or whatever that is hard coded to look for foo-bar.dtb won't see the new foo/foo-bar.dtb (or whatever) and will continue to use the old binary. It's not the fact that that it's in a directory, it's the fact that the bootloader sees the name it needs to look for change (if it's looking on a filesystem at all).
Correct.
Quoted 2 lines
This isn't a problem for arm64 as
the location isn't changing, it's used directories from day one.
The kernel may have used directories, but that's not what the distros did according to Andreas:
Quoted 3 lines
We already had that discussion for arm64 because Debian chose to ignore the kernel-installed subdirectories and installed .dtb files into a flat directory, which collided with openSUSE sticking to the kernel choice.
So are the distros different or who changed to align? That's not clear from this thread.
Quoted 1 line
The issues with the existing
install_dtbssounded unrelated to this.
Maybe, what are the issues? We can't change the source layout transparently if dtbs_install is not being used.
My question here is whether a flat install is useful on arm64. We can either have a kconfig variable that arm32 sets to do flat installs or it could be some command line make variable and then any user can pick what they want for any arch.
Rob
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 6 lines · Rob Herring
Quoted 1 line · Mark Brown
The issues with the existing
install_dtbssounded unrelated to this.Maybe, what are the issues? We can't change the source layout transparently if
dtbs_installis not being used.
I thought that was the thing with adding -@ so overlays could be used?
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 10 lines · Mark Brown
Quoted 6 lines · Rob Herring
Quoted 1 line · Mark Brown
The issues with the existing
install_dtbssounded unrelated to this.Maybe, what are the issues? We can't change the source layout transparently if
dtbs_installis not being used.I thought that was the thing with adding -@ so overlays could be used?
I don't think so as that is during building, not install. Any user can set '-@' with 'make DTC_FLAGS="-@" ...' already. The issue with that was changing the default globally and no way to set per platform. Now that I think about, moving the sources to subdirs may allow setting DTC_FLAGS per subdir which may be good enough.
Rob
_______________________________________________
boot-architecture mailing list
[email protected]
https://lists.linaro.org/mailman/listinfo/boot-architecture
Quoted 111 lines · Rob Herring
Quoted 40 lines · Andreas Färber
Am 05.12.18 um 05:17 schrieb Rob Herring:
Quoted 38 lines · Andreas Färber
Quoted 9 lines
Rob,
Am 04.12.18 um 19:36 schrieb Rob Herring:
Quoted 5 lines
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
Quoted 7 lines
Quoted 5 lines
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs?
This refactoring is a terrible idea!
How do you really feel?
Quoted 7 lines
While it would've been nice to have more structure from the start, bootloaders like U-Boot expect a flat structure for arm .dtb files now. If you start installing them into subdirs instead, they won't find the files anymore under the hardcoded name.
Doing this only for new platforms would be much less invasive and allow to prepare bootloaders accordingly.
That was my suggestion where this started for the new RDA platform. Olof preferred to move everything and that's my desire too.
Quoted 2 lines
Alternatively, white-list which ones
are safe to move around.I'd prefer to know which ones the distros don't want moved. That should be easier to figure out. We also need that anyways in context of what platforms we care about compatibility.
Quoted 6 lines
Quoted 2 lines
Another option is
dtbs_installtarget could flatten the installed dtbs. That is the only part the distros should depend on.I'd be okay with distinguishing source vs. install location. Due to the issue I mention below (and more) we can't use
install_dtbsfor openSUSE and had to reimplement it, which we'd need to (and can) adjust.What would be needed for
dtbs_installto work? arm64 needs to support a flat install? If it doesn't work for Debian or openSUSE, I'm not sure why we have it. So I'd like to make it work.Quoted 20 lines
Quoted 10 lines
Quoted 6 lines
But don't just script a refactoring because it
looks nicer in the source tree, without testing what side effects this can have for board/distro users of the compiled files in practice. We already had that discussion for arm64 because Debian chose to ignore the kernel-installed subdirectories and installed .dtb files into a flat directory, which collided with openSUSE sticking to the kernel choice.So everyone already deals with subdirs or not with arm and arm64 already, seems like they can deal with this. I will raise the topic on the cross-distro list though.
Sounds like you're twisting words... The keyword was "hardcoded" paths - one way or another (not "and") depending on the kernel choices being flat for arm, vendor subdir for arm64.
Quoted 5 lines
Quoted 4 lines
This topic becomes even more important with EBBR: There is neither a mechanism in place to sync .dts files into U-Boot or EDK2 source trees, nor are capsule updates implemented in U-Boot for easily deploying such bootloaders with new .dts sources or paths yet.
Quoted 4 lines
Quoted 2 lines
EBBR actually says firmware (including dtbs) goes in directories named by vendor.
Fine, but unrelated.
If the distros want dtbs in a flat dir and EBBR says otherwise, then it is related.
Quoted 31 lines
Quoted 22 lines
Quoted 5 lines
And I can assure you
that just getting users to dd the right bootloader can be difficult... Since DT forward and backward compatibility is often being neglected, for example with optional properties or renamed compatibles that break booting with previous drivers, new kernel versions often need updatedQuoted 8 lines
Device Trees to make use of new/enhanced drivers. Therefore it is unfortunately often enough a necessity to load newer kernel-based .dtb files matching the kernel (as opposed to the dream of kernel-independent hardware descriptions) when working with the latest -rc or -next kernels at least. For examples of DTs needing updates, look no further than Linaro's 96boards - in case of hikey960/EDK2 GRUB is another layer where .dtb paths may be hardcoded, ditto for arm; and Armada was an example where the upstream bindings for the network IP changed incompatibly.
Compatibility is an issue, yes, but that really has nothing to do with this.
Quoted 5 lines
DT overlays are another topic that is not making any progress upstream according to the ELCE BoF, so beyond the Raspberry Pi the only known working way to apply them is to write a U-Boot boot.scr script, which can either reuse $fdtcontroladdr DT or use the filename $fdtfile or hardcode one, the latter two of which would break with your renaming.
Quoted 3 lines
DT overlays also have nothing to do with this as there aren't any in the kernel. I'm not inclined to take any either with a flat tree. We're already at 1800+ files.
Read again: a) Breaking DT changes and b) the desire to use Overlays instead of replacing the bootloaders for each change are _reasons_ why people depend on .dtb filenames from the kernel source tree for their boot flow today. Nothing to do with downstream .dtbo files.
Quoted 7 lines
Quoted 3 lines
For example, remember when I reported that the kernel didn't compile DTs with -@? No reaction except for Frank asking to be CC'ed - was it ever fixed??? Do EDK2's or U-Boot's built-in DTs compile with -@ today?
IIRC, Frank objected to changing this globally because it will bloat all dtbs. And then no one did the work to make it a per dtb option. Maybe that was the same issue in another thread.
Being the author of one of the patches to pass in -@ so we could have overlays work out of the box, no, there was some other problem with making it only happen for some sub-set of DTBs. AFAIK the current answer is the one of a few years ago that no, if you want symbols for overlays to work you pass in ..whatever it is.. so that -@ is passed in. In the end it felt like there was more concern over the core concept than anything else and I moved along due to other pressing concerns.
Signature · 1 line
Tom
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 80 lines · Rob Herring
Olof, Arnd,
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can.
Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example.
What to do with
armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things.Sub-arch maintainers,
'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues.Rob
8<----------------------------------------------------------- #!/usr/bin/env python3
import os
import re
from git import Git
import globvendor_map = { 'imx': 'fsl', 'ls': 'fsl', 'vf': 'fsl', 'qcom': 'qcom', 'am3' : 'ti', 'am4' : 'ti', 'am5' : 'ti', 'da' : 'ti', 'dm' : 'ti', 'dra' : 'ti', 'keystone' : 'ti', 'omap' : 'ti', 'nspire' : 'ti', 'armada' : 'marvell', 'berlin' : 'marvell', 'dove' : 'marvell', 'kirkwood' : 'marvell', 'orion' : 'marvell', 'pxa' : 'marvell', 'mvebu' : 'marvell', 'mmp2' : 'marvell', 'arm-' : 'arm', 'integ' : 'arm', 've' : 'arm', 'aspeed' : 'aspeed', 'at91' : 'atmel', 'sama' : 'atmel', 'bcm' : 'brcm', 'exynos' : 'samsung', 's3c' : 'samsung', 's5p' : 'samsung', 'gemini' : 'cortina', 'hi3' : 'hisilicon', 'hip' : 'hisilicon', 'hisi' : 'hisilicon', 'mt' : 'mediatek', 'meson' : 'amlogic', 'owl' : 'actions', 'r7' : 'renesas', 'r8' : 'renesas', 'r9' : 'renesas', 'rk' : 'rockchip',
Quoted 8 lines
'socfpga' : 'altera',
'st' : 'st',
'spear' : 'st',
'sun' : 'allwinner',
'tegra' : 'nvidia',
'zynq' : 'xilinx',
'wm' : 'wm',
'uniph' : 'socionext',I tested this script, and
I confirmed all of my DT files were moved to the socionext subdirectory.
However, this would break arm64 build
because some of my arm64 DT files include arm32 ones.
I think it is common to share base platform boards between arm and arm64.
Of course, It will be pretty easy to fix it up by hand later, though.
Quoted 58 lines
'zx' : 'zte', }if
__name__== "__main__":g = Git('.') g.checkout("HEAD", "arch/arm/boot/dts/Makefile") dts_make = open("arch/arm/boot/dts/Makefile", "r").read() # make entries 1 line make2 = re.sub(r'\\\n', '', dts_make) for k,v in vendor_map.items(): for f in glob.iglob("arch/arm/boot/dts/" + k + "*.*"): new_dir = "arch/arm/boot/dts/" + v + "/" base = os.path.splitext(os.path.basename(f))[0] os.makedirs(new_dir, exist_ok=True) g.mv(f, new_dir) # Remove the file from the makefile dts_make = re.sub('.*' + base + r'\.dtb.*\\\n', '', dts_make) dts_make = re.sub('.*' + base + r'\.dtb', '', dts_make) # extract the matching makefile entry reg = re.search(r'.*' + base + r'.*', make2) if not reg: continue entry = reg.group(0) make2 = re.sub(r'.*' + base + r'.*', '', make2) if entry: makefile = open(new_dir + 'Makefile', 'a+') print(entry, file=makefile) makefile.close() for d in sorted(glob.iglob("arch/arm/boot/dts/*/")): dts_make += 'subdir-y += ' + d.split(os.path.sep)[-2] + '\n' # Add license and sort entries of sub-dir makefile vendor_make = '# SPDX-License-Identifier: GPL-2.0\n' make_lines = open(d + 'Makefile', 'r').readlines() for l in sorted(make_lines): vendor_make += l vendor_make = re.sub(r'\t', r'\\\n\t', vendor_make) f = open(d + 'Makefile', 'w') f.write(vendor_make) f.close() g.add(d + 'Makefile') # Remove entries with no dtbs left dts_make = re.sub(r'.*\+= \\\n\n', '', dts_make) open("arch/arm/boot/dts/Makefile", "w").write(dts_make) g.add("arch/arm/boot/dts/Makefile")
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 91 lines · Masahiro Yamada
Quoted 80 lines · Rob Herring
Olof, Arnd,
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can.
Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example.
What to do with
armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things.Sub-arch maintainers,
'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues.Rob
8<----------------------------------------------------------- #!/usr/bin/env python3
import os
import re
from git import Git
import globvendor_map = { 'imx': 'fsl', 'ls': 'fsl', 'vf': 'fsl', 'qcom': 'qcom', 'am3' : 'ti', 'am4' : 'ti', 'am5' : 'ti', 'da' : 'ti', 'dm' : 'ti', 'dra' : 'ti', 'keystone' : 'ti', 'omap' : 'ti', 'nspire' : 'ti', 'armada' : 'marvell', 'berlin' : 'marvell', 'dove' : 'marvell', 'kirkwood' : 'marvell', 'orion' : 'marvell', 'pxa' : 'marvell', 'mvebu' : 'marvell', 'mmp2' : 'marvell', 'arm-' : 'arm', 'integ' : 'arm', 've' : 'arm', 'aspeed' : 'aspeed', 'at91' : 'atmel', 'sama' : 'atmel', 'bcm' : 'brcm', 'exynos' : 'samsung', 's3c' : 'samsung', 's5p' : 'samsung', 'gemini' : 'cortina', 'hi3' : 'hisilicon', 'hip' : 'hisilicon', 'hisi' : 'hisilicon', 'mt' : 'mediatek', 'meson' : 'amlogic', 'owl' : 'actions', 'r7' : 'renesas', 'r8' : 'renesas', 'r9' : 'renesas', 'rk' : 'rockchip',Quoted 8 lines
'socfpga' : 'altera', 'st' : 'st', 'spear' : 'st', 'sun' : 'allwinner', 'tegra' : 'nvidia', 'zynq' : 'xilinx', 'wm' : 'wm', 'uniph' : 'socionext',
Quoted 7 lines
I tested this script, and
I confirmed all of my DT files were moved to the socionext subdirectory.However, this would break arm64 build
because some of my arm64 DT files include arm32 ones.I think it is common to share base platform boards between arm and arm64.
+1
Just a note here that for Zynq and ZynqMP there should be doable to run Microblaze in programmable logic with reusing fixed IPs. There could be also other soft core CPUs which could do this on Xilinx devices.
It means maybe moving this out of arm/arm64 to more generic location would be one more step to consider as the part of this move.
Thanks,
Michal
Hi Rob,
Quoted 36 lines · On Tue, 4 Dec 2018 12:36:49 -0600 Rob Herring <[email protected]> wrote:
Olof, Arnd,
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can.
Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example.
What to do with
armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things.Sub-arch maintainers,
'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues.Rob
8<----------------------------------------------------------- #!/usr/bin/env python3
import os
import re
from git import Git
import glob
Quoted 16 lines
vendor_map = {
'imx': 'fsl',
'ls': 'fsl',
'vf': 'fsl',
'qcom': 'qcom',
'am3' : 'ti',
'am4' : 'ti',
'am5' : 'ti',
'da' : 'ti',
'dm' : 'ti',
'dra' : 'ti',
'keystone' : 'ti',
'omap' : 'ti',
'nspire' : 'ti',
'armada' : 'marvell',
'berlin' : 'marvell',Now, berlin SoC is synaptics' SoC ;)
Thanks
Quoted 6 lines · Jisheng Zhang
Quoted 2 lines · Rob Herring
'armada' : 'marvell', 'berlin' : 'marvell',Now, berlin SoC is synaptics' SoC ;)
This illustrates perfectly the artificial nature of using vendor names as prefixes with DT properties, prefix names, directories etc.
Companies start out purporting to be some eternal entity and the next day they buy each other left and right and license their hardware IP to whoever wants it.
It actually makes much more sense to organize these files by the SoC family name, because that doesn't change when the SoC is sold to another company.
omap/* containing all OMAP platforms, msm/* for all Qualcomm SoCs etc. SoC names/codenames are at least eternal once they have been manufactured and we can keep them together no matter what vendor currently controls it.
However I think there was a fork in the road ages ago when someone or something decided to use vendor prefixes for DT properties leading to this situation that we can no longer back out of.
It has the side effect of splitting DTS files with the same SoC in two different folders marvell/* and synaptics/* it's a bit meh.
Yours,
Linus Walleij
Quoted 11 lines · On Wed, 5 Dec 2018 09:19:49 +0100 Linus Walleij wrote:
Quoted 6 lines · Jisheng Zhang
Quoted 2 lines · Rob Herring
'armada' : 'marvell', 'berlin' : 'marvell',Now, berlin SoC is synaptics' SoC ;)
This illustrates perfectly the artificial nature of using vendor names as prefixes with DT properties, prefix names, directories etc.
Quoted 7 lines
Companies start out purporting to be some eternal entity and the next day they buy each other left and right and license their hardware IP to whoever wants it.
It actually makes much more sense to organize these files by the SoC family name, because that doesn't change when the SoC is sold to another company.
If the SoC is sold to another company, then
case1: The original SoC family is renamed to another family.
case2: Based on the original SoC, a newer SoC family comes out.
I'm not sure it's still fine to put the new or renamed SoCs' files into the original SoC directory.
Another issue is: who will be the maintainer of new or renamed SoC family?
Thanks,
Jisheng
Quoted 17 lines
omap/* containing all OMAP platforms, msm/* for all Qualcomm SoCs etc. SoC names/codenames are at least eternal once they have been manufactured and we can keep them together no matter what vendor currently controls it.
However I think there was a fork in the road ages ago when someone or something decided to use vendor prefixes for DT properties leading to this situation that we can no longer back out of.
It has the side effect of splitting DTS files with the same SoC in two different folders marvell/* and synaptics/* it's a bit meh.
Yours,
Linus Walleij
Quoted 7 lines · Jisheng Zhang
[Me]
Quoted 3 lines
It actually makes much more sense to organize these files by the SoC family name, because that doesn't change when the SoC is sold to another company.
If the SoC is sold to another company, then
Quoted 8 lines
case1: The original SoC family is renamed to another family.
case2: Based on the original SoC, a newer SoC family comes out.
I'm not sure it's still fine to put the new or renamed SoCs' files into the original SoC directory.
Another issue is: who will be the maintainer of new or renamed SoC family?
Of course marketing prefer to associate the acquired company's SoC with itself and rename it.
But that doesn't account for the case where the same SoC is produced by the new owner under the old name.
Example:
The Gemini SoC arch/arm/boot/dts/gemini-*
https://dflund.se/~triad/krad/gemini/
This SoC was produced by StorLink Semiconductor in Taiwan early 2000s.
At some point between 2000 and 2008 the company changed its name and/or got restructured and was renamed Storm Semiconductor. The masks for the packaging was retained so the chips still said "StorLink".
Then in 2008 Cortina Systems acquired Storm Semiconductor eventually changed the masks and and renamed the identical chips from "SLnnnn" to "CSnnnn" but the chip inside the package was still the same.
The SoC is still codenamed "Gemini", that is the only constant.
And now we will put these devicetrees into folder...?
storlink/
storm/
cortina/
If we look at the chip packageing, it should be different folders depending on what supplier was used when the device was manufactured.
This is why the Vendor scheme is not really working.
Another issue is illustrated again with the Synaptics device:
an SoC produced exclusively for Synaptics (IIUC) but with a intimate relationship to a very easily identifed SoC.
This naming nomenclature is a half-measure and a can of worms, that is my only point.
Yours,
Linus Walleij
Quoted 9 lines · Linus Walleij
Quoted 6 lines · Jisheng Zhang
Quoted 2 lines · Rob Herring
'armada' : 'marvell', 'berlin' : 'marvell',Now, berlin SoC is synaptics' SoC ;)
But we're not changing the names of existing SoCs and at least so far there aren't new 32-bit berlin SoCs.
Quoted 10 lines
This illustrates perfectly the artificial nature of using vendor names as prefixes with DT properties, prefix names, directories etc.
Companies start out purporting to be some eternal entity and the next day they buy each other left and right and license their hardware IP to whoever wants it.
It actually makes much more sense to organize these files by the SoC family name, because that doesn't change when the SoC is sold to another company.
Uh, no. i.MX23 is a Sigmatel chip STMP?? which shared absolutely nothing with other i.MX chips except for the derivatives that followed. IIRC, STMP chips were even partially upstream.
Marketing names change at the whim of marketing and don't even require a sale of a company.
Quoted 4 lines
omap/* containing all OMAP platforms, msm/* for all Qualcomm SoCs etc. SoC names/codenames are at least eternal once they have been manufactured and we can keep them together no matter what vendor currently controls it.
Where do TI amXXXX chips go? Not all QC chips are 'msm' and I think that name is abandoned now. No solution is going to be perfect.
We are already use vendors for arm64 (except for the oddball exynos), so if we change arm, we shouldn't do something different. EBBR is also going with vendor for firmware directories in the EFI system partition. Speak up if you want to change that before 1.0.
Quoted 4 lines
However I think there was a fork in the road ages ago when someone or something decided to use vendor prefixes for DT properties leading to this situation that we can no longer back out of.
It was simpler times...
Quoted 3 lines
It has the side effect of splitting DTS files with the same SoC in two different folders marvell/* and synaptics/* it's a bit meh.
All I really care about here is things are organized by maintainers. Someone care to write a script to ensure all 1800 files have a maintainer attached to them (that is not me)?
Rob
Quoted 4 lines · Rob Herring
We are already use vendors for arm64 (except for the oddball exynos), so if we change arm, we shouldn't do something different. EBBR is also going with vendor for firmware directories in the EFI system partition. Speak up if you want to change that before 1.0.
I don't think there is a silver bullet for this, except "use vendor or whatever makes most sense", like if in doubt use vendor but no need to overspecify. Rough consensus, not perfect consensus.
Quoted 3 lines
All I really care about here is things are organized by maintainers. Someone care to write a script to ensure all 1800 files have a maintainer attached to them (that is not me)?
OK why not let the maintainers choose the name of the subdir rather than insist to set it to $vendor? Just consistency?
For the stuff I maintain please use:
'arm-' : 'arm',
'integ' : 'arm',
've' : 'arm',These are fine.
Drop this
- 'gemini' : 'cortina',
Just use the "gemini" prefix.
Because this was three different vendors so let's stick with the SoC codename.
And for 'ste' I would use 'st-ericsson' or 'stericsson' (also a valid vendor) so it is readable.
It's no super-big deal so if you think my view is just annoying I am not going to persist, I will just get over it.
Thanks,
Linus Walleij
Hello,
Quoted 1 line · Rob Herring
i.MX23 is a Sigmatel chip STMP??
I think the Freescale i.MX23 didn't exist at Sigmatel back then. AFAIK this is a new design using IP from Sigmatel after the aquisition.
Best regards
Uwe
Signature · 2 lines
Pengutronix e.K. | Uwe Kleine-König | Industrial Linux Solutions | http://www.pengutronix.de/ |
Quoted 8 lines · Uwe Kleine-König
Hello,
Quoted 1 line · Rob Herring
i.MX23 is a Sigmatel chip STMP??
I think the Freescale i.MX23 didn't exist at Sigmatel back then. AFAIK this is a new design using IP from Sigmatel after the aquisition.
It is not. I was in the i.MX group which Sigmatel was merged into at the time. Purely marketing rebranding.
Rob
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 5 lines · Rob Herring
Quoted 3 lines · Uwe Kleine-König
Hello,
Quoted 9 lines
Quoted 5 lines · Rob Herring
Quoted 1 line
i.MX23 is a Sigmatel chip STMP??
I think the Freescale i.MX23 didn't exist at Sigmatel back then. AFAIK this is a new design using IP from Sigmatel after the aquisition.
It is not. I was in the i.MX group which Sigmatel was merged into at the time. Purely marketing rebranding.
Wouldn't it be easier to name the directory to the corresponding mach-* entry?
So, imx23 and imx28 would go to mxs/, other imx in imx/. And this also solves the Marvell mess with the Synaptics Socs going in berlin/ and the other ones in mvebu/.
Signature · 3 lines
Alexandre Belloni, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Quoted 16 lines · Alexandre Belloni
Quoted 5 lines · Rob Herring
Quoted 3 lines · Uwe Kleine-König
Hello,
Quoted 9 lines
Quoted 5 lines · Rob Herring
Quoted 1 line
i.MX23 is a Sigmatel chip STMP??
I think the Freescale i.MX23 didn't exist at Sigmatel back then. AFAIK this is a new design using IP from Sigmatel after the aquisition.
It is not. I was in the i.MX group which Sigmatel was merged into at the time. Purely marketing rebranding.
Quoted 6 lines
Wouldn't it be easier to name the directory to the corresponding mach-* entry?
So, imx23 and imx28 would go to mxs/, other imx in imx/. And this also solves the Marvell mess with the Synaptics Socs going in berlin/ and the other ones in mvebu/.
I like this idea.
We discussed merging all ARM reference design mach-* to one dir if I just name that mach-arm then we get a convergence to the vendor name in some organic way.
Yours,
Linus Walleij
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 19 lines · Linus Walleij
Quoted 16 lines · Alexandre Belloni
Quoted 14 lines · Rob Herring
Quoted 8 lines · Uwe Kleine-König
Hello,
Quoted 1 line · Rob Herring
i.MX23 is a Sigmatel chip STMP??
I think the Freescale i.MX23 didn't exist at Sigmatel back then. AFAIK this is a new design using IP from Sigmatel after the aquisition.
It is not. I was in the i.MX group which Sigmatel was merged into at the time. Purely marketing rebranding.
Quoted 8 lines
Quoted 6 lines
Wouldn't it be easier to name the directory to the corresponding mach-* entry?
So, imx23 and imx28 would go to mxs/, other imx in imx/. And this also solves the Marvell mess with the Synaptics Socs going in berlin/ and the other ones in mvebu/.
I like this idea.
Fine by me though I think marvell gets more complicated than that. I'll try a pass at that at least for the cases with a mixture of families.
Quoted 3 lines
We discussed merging all ARM reference design mach-* to one dir if I just name that mach-arm then we get a convergence to the vendor name in some organic way.
TBC, you want .../boot/dts/arm/* for all the ARM, Ltd boards? Just making sure as you were arguing against vendor names. :)
Rob
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 8 lines · Rob Herring
Quoted 3 lines · Linus Walleij
We discussed merging all ARM reference design mach-* to one dir if I just name that mach-arm then we get a convergence to the vendor name in some organic way.
TBC, you want .../boot/dts/arm/* for all the ARM, Ltd boards? Just making sure as you were arguing against vendor names. :)
I am not against vendor names, but I am also for SoC names because I think whatever makes most sense should be the rule, so both/and not either/or. One does not exclude the other. It's just a name.
In this case what
I meant was that while we (me and Arnd) originally discussed merging it all into mach-versatile (how would you know) if I instead merge it all into mach-arm, we get a 1:1 correspondence between mach-dir and vendor name and DTS dir so everyone is happy.
Yours,
Linus Walleij
Quoted 15 lines · Linus Walleij
Quoted 8 lines · Rob Herring
Quoted 3 lines · Linus Walleij
We discussed merging all ARM reference design mach-* to one dir if I just name that mach-arm then we get a convergence to the vendor name in some organic way.
TBC, you want .../boot/dts/arm/* for all the ARM, Ltd boards? Just making sure as you were arguing against vendor names. :)
I am not against vendor names, but I am also for SoC names because I think whatever makes most sense should be the rule, so both/and not either/or. One does not exclude the other. It's just a name.
Quoted 6 lines
In this case what
I meant was that while we (me and Arnd) originally discussed merging it all into mach-versatile (how would you know) if I instead merge it all into mach-arm, we get a 1:1 correspondence between mach-dir and vendor name and DTS dir so everyone is happy.
With the number of trade names we've already seen with the TI SoCs, I'd probably prefer arch/arm/boot/dts/ti. What used to be omap is now dra7 and am437x and so on.
And for timing, doing this just before -rc1 gets tagged seems like a good time to do it. At least I have still pending large dts changes waiting that I'd rather not send a pull request out for until next week :)
Regards,
Tony
Quoted 18 lines · Rob Herring
Olof, Arnd,
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can.
Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example.
Quoted 7 lines
What to do with
armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things.Sub-arch maintainers,
'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues.
It'll certainly break some of our scripts (CI, building scripts, overlay makefiles, etc.). We already experienced it when adding the "dts" directory. But if it's needed...
Quoted 9 lines
8<----------------------------------------------------------- #!/usr/bin/env python3
import os
import re
from git import Git
import glob
vendor_map= {
[..]
Quoted 4 lines
'integ' : 'arm',
've' : 'arm',
'aspeed' : 'aspeed',
'at91' : 'atmel',As it's a new directory, what about creating a "microchip" one as it's our vendor's name now.
Quoted 1 line
'sama' : 'atmel',
If desired, you can extend to sama5d for the current files.
Adding the entry 'sam9' : "microchip", can be future proof...
Best regards,
Signature · 1 line
Nicolas Ferre
Quoted 80 lines · Rob Herring
Olof, Arnd,
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can.
Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example.
What to do with
armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things.Sub-arch maintainers,
'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues.Rob
8<----------------------------------------------------------- #!/usr/bin/env python3
import os
import re
from git import Git
import globvendor_map = { 'imx': 'fsl', 'ls': 'fsl', 'vf': 'fsl', 'qcom': 'qcom', 'am3' : 'ti', 'am4' : 'ti', 'am5' : 'ti', 'da' : 'ti', 'dm' : 'ti', 'dra' : 'ti', 'keystone' : 'ti', 'omap' : 'ti', 'nspire' : 'ti', 'armada' : 'marvell', 'berlin' : 'marvell', 'dove' : 'marvell', 'kirkwood' : 'marvell', 'orion' : 'marvell', 'pxa' : 'marvell', 'mvebu' : 'marvell', 'mmp2' : 'marvell', 'arm-' : 'arm', 'integ' : 'arm', 've' : 'arm', 'aspeed' : 'aspeed', 'at91' : 'atmel', 'sama' : 'atmel', 'bcm' : 'brcm', 'exynos' : 'samsung', 's3c' : 'samsung', 's5p' : 'samsung', 'gemini' : 'cortina', 'hi3' : 'hisilicon', 'hip' : 'hisilicon', 'hisi' : 'hisilicon', 'mt' : 'mediatek', 'meson' : 'amlogic', 'owl' : 'actions', 'r7' : 'renesas', 'r8' : 'renesas', 'r9' : 'renesas', 'rk' : 'rockchip', 'socfpga' : 'altera',
Quoted 8 lines
'st' : 'st',
'spear' : 'st',
'sun' : 'allwinner',
'tegra' : 'nvidia',
'zynq' : 'xilinx',
'wm' : 'wm',
'uniph' : 'socionext',
'zx' : 'zte','ox8' : 'oxsemi',
I volunteer as a tribute !
This is a low-priority platform, you can move them in a separate directory if you want.
Neil
Quoted 63 lines
}
if
__name__== "__main__":g = Git('.') g.checkout("HEAD", "arch/arm/boot/dts/Makefile") dts_make = open("arch/arm/boot/dts/Makefile", "r").read() # make entries 1 line make2 = re.sub(r'\\\n', '', dts_make) for k,v in vendor_map.items(): for f in glob.iglob("arch/arm/boot/dts/" + k + "*.*"): new_dir = "arch/arm/boot/dts/" + v + "/" base = os.path.splitext(os.path.basename(f))[0] os.makedirs(new_dir, exist_ok=True) g.mv(f, new_dir) # Remove the file from the makefile dts_make = re.sub('.*' + base + r'\.dtb.*\\\n', '', dts_make) dts_make = re.sub('.*' + base + r'\.dtb', '', dts_make) # extract the matching makefile entry reg = re.search(r'.*' + base + r'.*', make2) if not reg: continue entry = reg.group(0) make2 = re.sub(r'.*' + base + r'.*', '', make2) if entry: makefile = open(new_dir + 'Makefile', 'a+') print(entry, file=makefile) makefile.close() for d in sorted(glob.iglob("arch/arm/boot/dts/*/")): dts_make += 'subdir-y += ' + d.split(os.path.sep)[-2] + '\n' # Add license and sort entries of sub-dir makefile vendor_make = '# SPDX-License-Identifier: GPL-2.0\n' make_lines = open(d + 'Makefile', 'r').readlines() for l in sorted(make_lines): vendor_make += l vendor_make = re.sub(r'\t', r'\\\n\t', vendor_make) f = open(d + 'Makefile', 'w') f.write(vendor_make) f.close() g.add(d + 'Makefile') # Remove entries with no dtbs left dts_make = re.sub(r'.*\+= \\\n\n', '', dts_make) open("arch/arm/boot/dts/Makefile", "w").write(dts_make) g.add("arch/arm/boot/dts/Makefile")_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 37 lines · Rob Herring
Olof, Arnd,
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can.
Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example.
What to do with
armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things.Sub-arch maintainers,
'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues.Rob
8<----------------------------------------------------------- #!/usr/bin/env python3
import os
import re
from git import Git
import glob
Quoted 4 lines
vendor_map = {
'imx': 'fsl',
'ls': 'fsl',
'vf': 'fsl',Probably we can update "fsl" to "nxp" now since you are at it. :) We didn't try to update this previously because for the past two years we were thinking that it will need to be updated to qcom again soon.
Quoted 105 lines
'qcom': 'qcom', 'am3' : 'ti', 'am4' : 'ti', 'am5' : 'ti', 'da' : 'ti', 'dm' : 'ti', 'dra' : 'ti', 'keystone' : 'ti', 'omap' : 'ti', 'nspire' : 'ti', 'armada' : 'marvell', 'berlin' : 'marvell', 'dove' : 'marvell', 'kirkwood' : 'marvell', 'orion' : 'marvell', 'pxa' : 'marvell', 'mvebu' : 'marvell', 'mmp2' : 'marvell', 'arm-' : 'arm', 'integ' : 'arm', 've' : 'arm', 'aspeed' : 'aspeed', 'at91' : 'atmel', 'sama' : 'atmel', 'bcm' : 'brcm', 'exynos' : 'samsung', 's3c' : 'samsung', 's5p' : 'samsung', 'gemini' : 'cortina', 'hi3' : 'hisilicon', 'hip' : 'hisilicon', 'hisi' : 'hisilicon', 'mt' : 'mediatek', 'meson' : 'amlogic', 'owl' : 'actions', 'r7' : 'renesas', 'r8' : 'renesas', 'r9' : 'renesas', 'rk' : 'rockchip', 'socfpga' : 'altera', 'st' : 'st', 'spear' : 'st', 'sun' : 'allwinner', 'tegra' : 'nvidia', 'zynq' : 'xilinx', 'wm' : 'wm', 'uniph' : 'socionext', 'zx' : 'zte', }if
__name__== "__main__":g = Git('.') g.checkout("HEAD", "arch/arm/boot/dts/Makefile") dts_make = open("arch/arm/boot/dts/Makefile", "r").read() # make entries 1 line make2 = re.sub(r'\\\n', '', dts_make) for k,v in vendor_map.items(): for f in glob.iglob("arch/arm/boot/dts/" + k + "*.*"): new_dir = "arch/arm/boot/dts/" + v + "/" base = os.path.splitext(os.path.basename(f))[0] os.makedirs(new_dir, exist_ok=True) g.mv(f, new_dir) # Remove the file from the makefile dts_make = re.sub('.*' + base + r'\.dtb.*\\\n', '', dts_make) dts_make = re.sub('.*' + base + r'\.dtb', '', dts_make) # extract the matching makefile entry reg = re.search(r'.*' + base + r'.*', make2) if not reg: continue entry = reg.group(0) make2 = re.sub(r'.*' + base + r'.*', '', make2) if entry: makefile = open(new_dir + 'Makefile', 'a+') print(entry, file=makefile) makefile.close() for d in sorted(glob.iglob("arch/arm/boot/dts/*/")): dts_make += 'subdir-y += ' + d.split(os.path.sep)[-2] + '\n' # Add license and sort entries of sub-dir makefile vendor_make = '# SPDX-License-Identifier: GPL-2.0\n' make_lines = open(d + 'Makefile', 'r').readlines() for l in sorted(make_lines): vendor_make += l vendor_make = re.sub(r'\t', r'\\\n\t', vendor_make) f = open(d + 'Makefile', 'w') f.write(vendor_make) f.close() g.add(d + 'Makefile') # Remove entries with no dtbs left dts_make = re.sub(r'.*\+= \\\n\n', '', dts_make) open("arch/arm/boot/dts/Makefile", "w").write(dts_make) g.add("arch/arm/boot/dts/Makefile")
Quoted 19 lines · Rob Herring
Olof, Arnd,
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can.
Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example.
Quoted 7 lines
What to do with
armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things.Sub-arch maintainers,
'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues.
Here's an updated mapping filled out with the rest of the platforms and using SoC family names in some cases as discussed. The move is completely scripted now including include fixups (though any new includes could break things). So mainly just need to bikeshed the directory mapping. Not sure if marvell should be split up more or not. I split out pxa2xx and pxa3xx, but then there's other pxa chips I think aren't really related. TI is still all one directory except nspire. I was going to split out davinci too, but it's only a couple of files. Sub-arch maintainers need to chime in with what they want.
A branch is here including a fix to dtbs_install:git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git arm-dts-move
vendor_map = {
'alphascale' : 'alphascale',
'alpine' : 'alpine',
'artpec' : 'axis',
'atlas' : 'sirf',
'prima' : 'sirf',
'axm' : 'lsi',
'cx9' : 'cnxt',
'ecx' : 'calxeda',
'highbank' : 'calxeda',
'efm' : 'efm32',
'ep7' : 'cirrus',
'mxs': 'mxs',
'imx23': 'mxs',
'imx28': 'mxs',
'imx': 'imx',
'ls': 'fsl',
'vf': 'fsl',
'qcom': 'qcom',
'am3' : 'ti',
'am4' : 'ti',
'am5' : 'ti',
'dra' : 'ti',
'keystone' : 'ti',
'omap' : 'ti',
'compulab' : 'ti',
'logicpd' : 'ti',
'elpida' : 'ti',
'motorola-cpcap' : 'ti',
'twl' : 'ti',
'da' : 'ti',
'dm' : 'ti',
'nspire' : 'nspire',
'armada' : 'marvell',
'dove' : 'marvell',
'kirkwood' : 'marvell',
'orion' : 'marvell',
'mvebu' : 'marvell',
'mmp2' : 'marvell',
'berlin' : 'berlin',
'pxa2' : 'pxa',
'pxa3' : 'pxa',
'pxa' : 'marvell',
'arm-' : 'arm',
'integ' : 'arm',
'mps' : 'arm',
've' : 'arm',
'aspeed' : 'aspeed',
'at91' : 'at91',
'sama' : 'at91',
'usb_' : 'at91',
'tny_' : 'at91',
'mpa1600' : 'at91',
'animeo_ip' : 'at91',
'aks-cdu' : 'at91',
'ethernut5' : 'at91',
'evk-pro3' : 'at91',
'pm9g45' : 'at91',
'ge86' : 'at91',
'bcm' : 'brcm',
'exynos' : 'exynos',
's3c' : 'samsung',
's5p' : 'samsung',
'gemini' : 'gemini',
'hi3' : 'hisilicon',
'hip' : 'hisilicon',
'hisi' : 'hisilicon',
'mt' : 'mediatek',
'meson' : 'meson',
'moxa' : 'moxa',
'nuvo' : 'nuvoton',
'lpc' : 'lpc',
'owl' : 'actions',
'ox8' : 'oxsemi',
'picox' : 'picoxcell',
'r7' : 'renesas',
'r8' : 'renesas',
'r9' : 'renesas',
'emev2' : 'renesas',
'sh73a' : 'renesas',
'gr-' : 'renesas',
'iwg' : 'renesas',
'rk' : 'rockchip',
'rv11' : 'rockchip',
'socfpga' : 'socfpga',
'stm' : 'stm32',
'sti' : 'sti',
'st-pin' : 'sti',
'ste' : 'st-ericsson',
'spear' : 'spear',
'sun' : 'allwinner',
'axp' : 'allwinner',
'tango' : 'sigma',
'tegra' : 'nvidia',
'uniph' : 'socionext',
'vt8500' : 'vt8500',
'wm8' : 'vt8500',
'xen' : 'xen',
'zx' : 'zte',
'zynq' : 'xilinx',
}Quoted 97 lines · Rob Herring
Quoted 19 lines · Rob Herring
Olof, Arnd,
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can.
Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example.
Quoted 7 lines
What to do with
armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things.Sub-arch maintainers,
'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues.Here's an updated mapping filled out with the rest of the platforms and using SoC family names in some cases as discussed. The move is completely scripted now including include fixups (though any new includes could break things). So mainly just need to bikeshed the directory mapping. Not sure if marvell should be split up more or not. I split out pxa2xx and pxa3xx, but then there's other pxa chips I think aren't really related. TI is still all one directory except nspire. I was going to split out davinci too, but it's only a couple of files. Sub-arch maintainers need to chime in with what they want.
A branch is here including a fix to
dtbs_install:git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.gitarm-dts-movevendor_map = { 'alphascale' : 'alphascale', 'alpine' : 'alpine', 'artpec' : 'axis', 'atlas' : 'sirf', 'prima' : 'sirf', 'axm' : 'lsi', 'cx9' : 'cnxt', 'ecx' : 'calxeda', 'highbank' : 'calxeda', 'efm' : 'efm32', 'ep7' : 'cirrus', 'mxs': 'mxs', 'imx23': 'mxs', 'imx28': 'mxs', 'imx': 'imx', 'ls': 'fsl', 'vf': 'fsl', 'qcom': 'qcom', 'am3' : 'ti', 'am4' : 'ti', 'am5' : 'ti', 'dra' : 'ti', 'keystone' : 'ti', 'omap' : 'ti', 'compulab' : 'ti', 'logicpd' : 'ti', 'elpida' : 'ti', 'motorola-cpcap' : 'ti', 'twl' : 'ti', 'da' : 'ti', 'dm' : 'ti', 'nspire' : 'nspire', 'armada' : 'marvell', 'dove' : 'marvell', 'kirkwood' : 'marvell', 'orion' : 'marvell', 'mvebu' : 'marvell', 'mmp2' : 'marvell', 'berlin' : 'berlin', 'pxa2' : 'pxa', 'pxa3' : 'pxa', 'pxa' : 'marvell', 'arm-' : 'arm', 'integ' : 'arm', 'mps' : 'arm', 've' : 'arm', 'aspeed' : 'aspeed', 'at91' : 'at91', 'sama' : 'at91', 'usb_' : 'at91', 'tny_' : 'at91', 'mpa1600' : 'at91', 'animeo_ip' : 'at91', 'aks-cdu' : 'at91',
Quoted 8 lines
'ethernut5' : 'at91',
'evk-pro3' : 'at91',
'pm9g45' : 'at91',
'ge86' : 'at91',
'bcm' : 'brcm',
'exynos' : 'exynos',
's3c' : 'samsung',
's5p' : 'samsung',Since Exynos is Samsung, I would prefer consistency here so let's put everything under vendor - samsung. I understand that It will be different than arm64 but that's the problem of choosing non-vendor name for arm64 at first place. So let's go with:
'exynos' : 'samsung',
's3c' : 'samsung',
's5p' : 'samsung',Best regards,
Krzysztof
Quoted 38 lines
'gemini' : 'gemini',
'hi3' : 'hisilicon',
'hip' : 'hisilicon',
'hisi' : 'hisilicon',
'mt' : 'mediatek',
'meson' : 'meson',
'moxa' : 'moxa',
'nuvo' : 'nuvoton',
'lpc' : 'lpc',
'owl' : 'actions',
'ox8' : 'oxsemi',
'picox' : 'picoxcell',
'r7' : 'renesas',
'r8' : 'renesas',
'r9' : 'renesas',
'emev2' : 'renesas',
'sh73a' : 'renesas',
'gr-' : 'renesas',
'iwg' : 'renesas',
'rk' : 'rockchip',
'rv11' : 'rockchip',
'socfpga' : 'socfpga',
'stm' : 'stm32',
'sti' : 'sti',
'st-pin' : 'sti',
'ste' : 'st-ericsson',
'spear' : 'spear',
'sun' : 'allwinner',
'axp' : 'allwinner',
'tango' : 'sigma',
'tegra' : 'nvidia',
'uniph' : 'socionext',
'vt8500' : 'vt8500',
'wm8' : 'vt8500',
'xen' : 'xen',
'zx' : 'zte',
'zynq' : 'xilinx',
}Linus Walleij
Acked-byQuoted 2 lines · Rob Herring
Here's an updated mapping filled out with the rest of the platforms and using SoC family names in some cases as discussed.
This looks really good to me!
Quoted 4 lines
The move is
completely scripted now including include fixups (though any new includes could break things). So mainly just need to bikeshed the directory mapping.
I did have a serious minute to think over whether this was a bikeshedding type discussion and the outcome is as unimportant as the color of some bikeshed. Went as far as to re-read Parsons (hilarious!) original essay and the first chapter of Wittgensteins Tractatus. I concluded it was more profound than that for several reasons.
Yours,
Linus Walleij
Quoted 28 lines · Rob Herring
Quoted 19 lines · Rob Herring
Olof, Arnd,
I've put together a script to move the dts files and update the makefiles. It doesn't handle files not following a common prefix which isn't many and some includes within the dts files will need some fixups by hand.
MAINTAINERSwill also need updating.A few questions:
Do we want to move absolutely everything to subdirs? There's quite a few platforms with only 1-2 platforms. I haven't added these to the list yet, but can.
Do any vendors need another level of directories? davinci, omap, nspire, etc. for TI for example.
Quoted 7 lines
What to do with
armv7m.dtsi? I guess it should remain and we just fixup the include. There may be a few other cross vendor things.Sub-arch maintainers,
'vendor_map' below is the mapping of file prefix to new subdirectory (the SoC vendor prefix). Please comment if there are any issues.
Quoted 9 lines
Here's an updated mapping filled out with the rest of the platforms and using SoC family names in some cases as discussed. The move is completely scripted now including include fixups (though any new includes could break things). So mainly just need to bikeshed the directory mapping. Not sure if marvell should be split up more or not. I split out pxa2xx and pxa3xx, but then there's other pxa chips I think aren't really related. TI is still all one directory except nspire. I was going to split out davinci too, but it's only a couple of files. Sub-arch maintainers need to chime in with what they want.
I think we can debate this forever if we want to, and of course when product families change owners (or when companies merge or change names) it will be tempting to rename or move files. But I think we'll treat this the same way as we do with bindings and other aspects of the kernel: some platforms will have legacy names that might not describe their current state but documents the lineage of the product. For that, it might be slightly easier to use product family names instead of company ditto, but I really don't care enough to request changes.
Also, since the subdirectory doesn't influence install path on 32-bit, renames shouldn't be the end of the world (but I'm likely to turn down those requests beyond the possible initial "we miscategorized this platform" kind).
-Olof
Quoted 2 lines · Rob Herring
I've put together a script to move the dts files and update the makefiles.
Anything here which the split-devicetree tree[0] needs to care about?
I think this is all just about file movement contained entirely underneath arch/arm/boot/dts, in which case the answer *should* be "no".
Ian.
[0] https://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git/