dtc: add symlink (-L) output to dtbs

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

Quoted 4 lines · Jason Cooper

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

My take is that the DTB filenames are part of the ABI, and therefore the DTS filenames are also part of the ABI. Why would we want to rename them?

Quoted 4 lines · Jason Cooper

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

My take is that the DTB filenames are part of the ABI, and therefore the DTS filenames are also part of the ABI. Why would we want to rename them?

Quoted 6 lines · Stephen Warren

Quoted 4 lines · Jason Cooper

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

Quoted 2 lines

My take is that the DTB filenames are part of the ABI, and therefore the DTS filenames are also part of the ABI. Why would we want to rename them?

I agree with the ABI part, but for long term I think compatible strings are a better choice for the ABI than filenames. A link provides for a way to transition.

Quoted 10 lines · Rob Herring

Quoted 8 lines · Stephen Warren

Quoted 4 lines · Jason Cooper

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

My take is that the DTB filenames are part of the ABI, and therefore the DTS filenames are also part of the ABI. Why would we want to rename them?

Quoted 3 lines

I agree with the ABI part, but for long term I think compatible strings are a better choice for the ABI than filenames. A link provides for a way to transition.

But this change isn't making the compatible value be the ABI, it's still keeping the filename as the ABI, but creating a different way of choosing the filename.

In other words, if the compatible value is the ABI, then the algorithm is:

* Search for all available DTBs
* Parse each DTB to find one with a matching compatible value

Whereas if the filename is the ABI, the algorithm is:

* Calculate the filename
* Load that one specific file, without looking at its content

Some additional thoughts:

* The filename is only relevant to the bootloader (or whatever locates/selects a DTB). The kernel/OS just gets a single DTB passed to it and uses it, without any naming metadata[1].

* Perhaps different bootloaders work different ways. For example, I've set up U-Boot on all Tegra devices so that if U-Boot expands the string "${soc}-${board}.dtb" using its environment, it'll generate the correct DTB filename.

* Other boot-loaders might like "${compatible_parsed_from_the_dtb}" to be the filename; that's Jason's proposal.

* Other SW (which I mention in [1]) might want to locate DTBs based on ARM machine ID rather than compatible value.

* As such, we're really talking about an indexing process when installing/packaging/searching DTBs. Shouldn't this file renaming happen as part of that process, not the build process?

* When we create symlinks or any kind of index, I think we want to create links for all entries in the compatible property (or other index source).

Consider if we have two boards:

Cardhu revision A02
Cardhu revision A04

Initially we think they're the same because we didn't read the schematics well enough, but then later we split them. That means we'll start out with one file:

nvidia,tegra30-cardhu.dtb

then later have instead:

nvidia,tegra30-cardhu-a02.dtb
nvidia,tegra30-cardhu-a04.dtb

Similarly, perhaps one DTB is (initially thought to be?) suitable for multiple ARM machine IDs, etc.

So, bootloaders would have to search for $vendor,$soc-$board-$rev.dtb then fall back to $vendor,$soc-$board.dtb (both those values are in the compatible properties in those DTBs).

But then "nvidia,tegra30" is in the compatible property for many boards. I wonder how we resolve that?

2) right now: distributions are creating utilities, such a flash-kernel in Debian, that pick from a slew of dtbs, append it to the zImage, and create the uImage. The bootloaders in these installs don't have support for fdt.

Our problem (as I see it) is that #2 is creating an ABI out of something we never realized would become an ABI, the filenames created by 'make dtbs'.

Quoted 8 lines

2) right now: distributions are creating utilities, such a flash-kernel in Debian, that pick from a slew of dtbs, append it to the zImage, and create the uImage. The bootloaders in these installs don't have support for fdt.

Our problem (as I see it) is that #2 is creating an ABI out of something we never realized would become an ABI, the filenames created by 'make dtbs'.

Well, flash-kernel could be taught how to parse .dtb files to get the compatibility strings out of them, rather than use filenames.

It might still be early enough to give distributions some guidelines about what DT folks consider ABI and what is not. How they should find a fitting .dtb file and how they should not.

Stephen,

Quoted 15 lines · Stephen Warren

Quoted 13 lines · Rob Herring

Quoted 8 lines · Stephen Warren

Quoted 4 lines · Jason Cooper

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

My take is that the DTB filenames are part of the ABI, and therefore the DTS filenames are also part of the ABI. Why would we want to rename them?

I agree with the ABI part, but for long term I think compatible strings are a better choice for the ABI than filenames. A link provides for a way to transition.

Quoted 3 lines

But this change isn't making the compatible value be the ABI, it's still keeping the filename as the ABI, but creating a different way of choosing the filename.

Right, which provides a path towards a slightly more sane ABI. If we choose to implement this, or another variant, we get to shape a migration path towards an ABI we designed. As opposed to locked in to one we didn't even see coming.

Quoted 4 lines

* Perhaps different bootloaders work different ways. For example, I've set up U-Boot on all Tegra devices so that if U-Boot expands the string "${soc}-${board}.dtb" using its environment, it'll generate the correct DTB filename.

Why does the bootloader have to choose? Is this a development scenario or a deployed scenario?

Quoted 3 lines

* As such, we're really talking about an indexing process when installing/packaging/searching DTBs. Shouldn't this file renaming happen as part of that process, not the build process?

I agree, but it's not. Hence the patch.

Quoted 3 lines

So, bootloaders would have to search for $vendor,$soc-$board-$rev.dtb then fall back to $vendor,$soc-$board.dtb (both those values are in the compatible properties in those DTBs).

I disagree here. I don't think this decision/discovery process should be done by the bootloader. It should be done by the distribution installer/firmware upgrade utility. The risk of breaking boot is too high.

Quoted 20 lines · Jason Cooper

Quoted 18 lines · Stephen Warren

Quoted 7 lines · Rob Herring

Quoted 6 lines · Stephen Warren

Quoted 4 lines · Jason Cooper

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

Quoted 6 lines

Quoted 2 lines

My take is that the DTB filenames are part of the ABI, and therefore the DTS filenames are also part of the ABI. Why would we want to rename them?

I agree with the ABI part, but for long term I think compatible strings are a better choice for the ABI than filenames. A link provides for a way to transition.

But this change isn't making the compatible value be the ABI, it's still keeping the filename as the ABI, but creating a different way of choosing the filename.

Quoted 4 lines

Right, which provides a path towards a slightly more sane ABI. If we choose to implement this, or another variant, we get to shape a migration path towards an ABI we designed. As opposed to locked in to one we didn't even see coming.

I don't really agree here; I specifically named all the Tegra DTB/.dts files in a sane fashion precisely so that U-Boot could easily determine which one to load. It's hard to see how this wasn't a predictable issue.

Quoted 21 lines · Stephen Warren

Quoted 20 lines · Jason Cooper

Quoted 18 lines · Stephen Warren

Quoted 7 lines · Rob Herring

Quoted 6 lines · Stephen Warren

Quoted 4 lines · Jason Cooper

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

Quoted 6 lines

Quoted 2 lines

My take is that the DTB filenames are part of the ABI, and therefore the DTS filenames are also part of the ABI. Why would we want to rename them?

I agree with the ABI part, but for long term I think compatible strings are a better choice for the ABI than filenames. A link provides for a way to transition.

But this change isn't making the compatible value be the ABI, it's still keeping the filename as the ABI, but creating a different way of choosing the filename.

Quoted 8 lines

Quoted 4 lines

Right, which provides a path towards a slightly more sane ABI. If we choose to implement this, or another variant, we get to shape a migration path towards an ABI we designed. As opposed to locked in to one we didn't even see coming.

I don't really agree here; I specifically named all the Tegra DTB/.dts files in a sane fashion precisely so that U-Boot could easily determine which one to load. It's hard to see how this wasn't a predictable issue.

This is the difference between a commercial effort and a hobbyist effort. I think I have a plan for how we can both get what we want. I could change the argument to 'dtc ... -L arch/arm/boot ...'. Thus:

$ cd arch/arm/boot
$ ls -l *.dtb        # minus a few columns
...
globalscale,mirabox.dtb -> dts/armada-370-mirabox.dtb
...

This way, aftermarket users (debian installers, etc) could go to the same place they go for zImages, and find a standard named dtb. They wouldn't need to parse the dtbs, just look at the current compatible string for the board they are installing on and find the matching file. For commercial efforts, nothing changes, the well-thought out *.dtb filenames are still in the same place they always were. A self-imposed vendor ABI, if you will.

Then, in situations where dts filenames need to change, that can happen (typically per vendor) because the ABI, from the kernel maintainers pov, is the symlinks in arch/arm/boot/.

As for the migration, there wouldn't need to be one beyond this first step of providing symlinks. That way, vendor-preferred filenames stay as they are.

Would that work better for you?

I don't think a distro should need to know the DTB filename at all. Rather, the/a bootloader should know which platform it's running on, and provide a variable/... to the boot script/... that defines the DTB filename. That would completely remove all the knowledge of DTB filenames from distros.

Quoted 5 lines · Stephen Warren

I don't think a distro should need to know the DTB filename at all. Rather, the/a bootloader should know which platform it's running on, and provide a variable/... to the boot script/... that defines the DTB filename. That would completely remove all the knowledge of DTB filenames from distros.

At some point the distro has to install a dtb into some cannonical place so the bootloader can find it..

Are you thinking that distros will have to ship a /boot/dtbs/*.dtb with every dtb from the kernel build?

Quoted 5 lines · Stephen Warren

I don't think a distro should need to know the DTB filename at all. Rather, the/a bootloader should know which platform it's running on, and provide a variable/... to the boot script/... that defines the DTB filename. That would completely remove all the knowledge of DTB filenames from distros.

At some point the distro has to install a dtb into some cannonical place so the bootloader can find it..

Are you thinking that distros will have to ship a /boot/dtbs/*.dtb with every dtb from the kernel build?

What happens when you install two different versions of the kernel?

What about boot schemes that can load a kernel version dependent dtb?

Quoted 10 lines · Jason Gunthorpe

Quoted 5 lines · Stephen Warren

I don't think a distro should need to know the DTB filename at all. Rather, the/a bootloader should know which platform it's running on, and provide a variable/... to the boot script/... that defines the DTB filename. That would completely remove all the knowledge of DTB filenames from distros.

At some point the distro has to install a dtb into some cannonical place so the bootloader can find it..

Quoted 2 lines

Are you thinking that distros will have to ship a /boot/dtbs/*.dtb with every dtb from the kernel build?

Yes. How else would the distro support booting the install on arbitrary boards?

Quoted 8 lines · Jason Gunthorpe

Quoted 5 lines · Stephen Warren

I don't think a distro should need to know the DTB filename at all. Rather, the/a bootloader should know which platform it's running on, and provide a variable/... to the boot script/... that defines the DTB filename. That would completely remove all the knowledge of DTB filenames from distros.

Quoted 5 lines

At some point the distro has to install a dtb into some cannonical place so the bootloader can find it..

Are you thinking that distros will have to ship a /boot/dtbs/*.dtb with every dtb from the kernel build?

Yes. How else would the distro support booting the install on arbitrary boards?

Quoted 1 line

What happens when you install two different versions of the kernel?

/boot/zImage-$version
/boot/dtbs-$version/*.dtb

Quoted 1 line

What about boot schemes that can load a kernel version dependent dtb?

I think that's solved by installing the DTB files in a kernel-version-specific directory.

This is the difference between a commercial effort and a hobbyist effort. I think I have a plan for how we can both get what we want. I could change the argument to 'dtc ... -L arch/arm/boot ...'. Thus:

$ cd arch/arm/boot
$ ls -l *.dtb        # minus a few columns
...
globalscale,mirabox.dtb -> dts/armada-370-mirabox.dtb
...

This way, aftermarket users (debian installers, etc) could go to the same place they go for zImages, and find a standard named dtb. They wouldn't need to parse the dtbs, just look at the current compatible string for the board they are installing on and find the matching file.

We have ways and means to deal with the installation of kernel images and modules. There's well defined makefile targets for this, and make variables which control where and how these get put. For the kernel image, you can specify the script to be run (which is called with the location of the *Image file) for copying. Pulling it manually out of arch/arm/boot is really a failure of the distro packaging people to get their process right.

The same goes for the DT files: if we don't provide a method for the kernel build system to install these in a sane way, then that's our failing. However, distros should be asking for this, and not picking the files out of arch/arm/boot/dts.

Quoted 8 lines · Russell King - ARM Linux

We have ways and means to deal with the installation of kernel images and modules. There's well defined makefile targets for this, and make variables which control where and how these get put. For the kernel image, you can specify the script to be run (which is called with the location of the *Image file) for copying. Pulling it manually out of arch/arm/boot is really a failure of the distro packaging people to get their process right.

Quoted 4 lines

The same goes for the DT files: if we don't provide a method for the kernel build system to install these in a sane way, then that's our failing. However, distros should be asking for this, and not picking the files out of arch/arm/boot/dts.

Exactly. V2 hopefully this weekend implementing 'make dtbs-install'

thx,

Jason.

Hi Jason,

Quoted 14 lines · Jason Cooper

Quoted 12 lines · Russell King - ARM Linux

We have ways and means to deal with the installation of kernel images and modules. There's well defined makefile targets for this, and make variables which control where and how these get put. For the kernel image, you can specify the script to be run (which is called with the location of the *Image file) for copying. Pulling it manually out of arch/arm/boot is really a failure of the distro packaging people to get their process right.

The same goes for the DT files: if we don't provide a method for the kernel build system to install these in a sane way, then that's our failing. However, distros should be asking for this, and not picking the files out of arch/arm/boot/dts.

Quoted 1 line

Exactly. V2 hopefully this weekend implementing 'make dtbs-install'

Just a small comment, I think the make target should be called 'dtbs_install and not of 'dtbs-install' to be consistent with make {modules,firmware,headers}_install

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

Let's provide a consistent naming structure for consumers to script against. Or at least, as consistent as the dts properties themselves.

With this patch, adding the '-L' option to the dtc commandline will cause dtc to create a symlink to the generated dtb, using the board compatible string as the filename, eg:

globalscale,mirabox.dtb -> armada-370-mirabox.dtb

Quoted 12 lines · Jason Cooper

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

Let's provide a consistent naming structure for consumers to script against. Or at least, as consistent as the dts properties themselves.

With this patch, adding the '-L' option to the dtc commandline will cause dtc to create a symlink to the generated dtb, using the board compatible string as the filename, eg:

Quoted 1 line

globalscale,mirabox.dtb -> armada-370-mirabox.dtb

This whole thread is just crazy. The filename is not an ABI. Anything digging into the kernel build tree is a) wrong, and b) userspace tooling consuming the kernel build products which is /not/ ABI and can be changed

If we want to have a tool or script for creating a well formed directory full of .dtb files because it would be helpful to U-Boot or something else, then that is fine. Make it part of the "make install" path.

Quoted 13 lines · Grant Likely

Quoted 12 lines · Jason Cooper

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

Let's provide a consistent naming structure for consumers to script against. Or at least, as consistent as the dts properties themselves.

With this patch, adding the '-L' option to the dtc commandline will cause dtc to create a symlink to the generated dtb, using the board compatible string as the filename, eg:

Quoted 5 lines

Quoted 1 line

globalscale,mirabox.dtb -> armada-370-mirabox.dtb

This whole thread is just crazy. The filename is not an ABI. Anything digging into the kernel build tree is a) wrong, and b) userspace tooling consuming the kernel build products which is /not/ ABI and can be changed

whew. I still think there is a need for an install target. The fact that our changing filenames might break things is an indicator of that.

Quoted 3 lines

If we want to have a tool or script for creating a well formed directory full of .dtb files because it would be helpful to U-Boot or something else, then that is fine. Make it part of the "make install" path.

I'm currently reworking the patch(es) to implement 'make dtbs-install'. I'll hopefully have them ready this weekend or sooner. I'm taking the same tack as install.sh (/sbin/installdtbs takes precedence).

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

Let's provide a consistent naming structure for consumers to script against. Or at least, as consistent as the dts properties themselves.

With this patch, adding the '-O compat' option to the dtc commandline will cause dtc to parse the provided file, and print out the board compatible string to stdout.

This will facilitate an 'installdtbs.sh' script in the kernel for naming dtb files by their compatible string, eg:

$ dtc -I dtb -O compat arch/arm/boot/dts/armada-370-mirabox.dtb
globalscale,mirabox

This change will also simplify distribution install scripts that need to search through many dtbs to find the right one for a target board.

Signed-off-by Jason Cooper <[email protected]>

---
changes since v1:
- made patch against in-tree dtc code to facilitate testing

Quoted 21 lines · Jason Cooper

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

Let's provide a consistent naming structure for consumers to script against. Or at least, as consistent as the dts properties themselves.

With this patch, adding the '-O compat' option to the dtc commandline will cause dtc to parse the provided file, and print out the board compatible string to stdout.

This will facilitate an 'installdtbs.sh' script in the kernel for naming dtb files by their compatible string, eg:

$ dtc -I dtb -O compat arch/arm/boot/dts/armada-370-mirabox.dtb
globalscale,mirabox

This change will also simplify distribution install scripts that need to search through many dtbs to find the right one for a target board.

Quoted 4 lines

Signed-off-by Jason Cooper <[email protected]>

---
changes since v1:
- made patch against in-tree dtc code to facilitate testing

I assume this patch would first get applied to the upstream dtc, then back-ported into the kernel though?

I wonder if dtc is the correct place to put this feature at all though. It seems like a tiny standalone utility using libfdt would be better. Actually, perhaps the existing fdtget utility (which seems to be scripts/dtc/fdtget.c in the kernel although I don't know if it's built by the kernel yet) can be used rather than creating a new one? Using fdtget seems like it'd allow more flexibility later, if the naming rules change, via scripting rather than having to edit the dtc source code.

Quoted 18 lines · Jason Cooper

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

Let's provide a consistent naming structure for consumers to script against. Or at least, as consistent as the dts properties themselves.

With this patch, adding the '-O compat' option to the dtc commandline will cause dtc to parse the provided file, and print out the board compatible string to stdout.

This will facilitate an 'installdtbs.sh' script in the kernel for naming dtb files by their compatible string, eg:

$ dtc -I dtb -O compat arch/arm/boot/dts/armada-370-mirabox.dtb
globalscale,mirabox

Quoted 7 lines

This change will also simplify distribution install scripts that need to search through many dtbs to find the right one for a target board.

Signed-off-by Jason Cooper <[email protected]>

---
changes since v1:
- made patch against in-tree dtc code to facilitate testing

I assume this patch would first get applied to the upstream dtc, then back-ported into the kernel though?

I wonder if dtc is the correct place to put this feature at all though. It seems like a tiny standalone utility using libfdt would be better. Actually, perhaps the existing fdtget utility (which seems to be scripts/dtc/fdtget.c in the kernel although I don't know if it's built by the kernel yet) can be used rather than creating a new one? Using fdtget seems like it'd allow more flexibility later, if the naming rules change, via scripting rather than having to edit the dtc source code.

Unlike other build products in the Linux kernel, the devicetree blobs are simply the name of their source file, s/dts/dtb/. There is also no 'make *install' mechanism to put them in a standard place with a standard naming structure.

Unfortunately, users have begun scripting pulling the needed dtbs from within the kernel tree, thus hardcoding the dtbs names. In turn, this means any changes to the dts filenames breaks these scripts.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a vendor or distribution supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

  /lib/modules/${kernel_version}/devicetree/${board_compat}.dtb

arch/arm/boot/installdtbs.sh +2 -0

 
+for dtb in `find "$3" -name "*.dtb"`; do
+	# we use dtc to parse the dtb, get the board compatible string,

Quoted 9 lines · Jason Cooper

Unlike other build products in the Linux kernel, the devicetree blobs are simply the name of their source file, s/dts/dtb/. There is also no 'make *install' mechanism to put them in a standard place with a standard naming structure.

Unfortunately, users have begun scripting pulling the needed dtbs from within the kernel tree, thus hardcoding the dtbs names. In turn, this means any changes to the dts filenames breaks these scripts.

Quoted 7 lines

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a vendor or distribution supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

  /lib/modules/${kernel_version}/devicetree/${board_compat}.dtb

Is co-mingling the DTs in the same (top-level) directory as modules a good idea. I guess there is an explicit devicetree/ sub-directory so they're easy to pull out of the source tree, but even so, they're certainly not modules. I would assume that distros would put them into e.g. /boot/dtbs/${kernelversion} or something more like that. Would it make sense for the install target to use a path more like that, or perhaps at least /dtbs/${kernelversion} to keep it separate from modules?

arch/arm/Makefile PHONY += dtbs · line 317

Quoted 3 lines

arch/arm/Makefile +1 -0

 
+dtbs_install: dtbs

arch/arm/Makefile PHONY += dtbs · line 319

Quoted 2 lines

arch/arm/Makefile +2 -0

+	$(CONFIG_SHELL) $(srctree)/$(boot)/installdtbs.sh $(KERNELRELEASE) \
+	"$(MODLIB)/devicetree" "$(srctree)/$(boot)/dts"

Architectures besides ARM use device trees. Shouldn't "make dtbs_install" work for them too?

Quoted 4 lines

arch/arm/boot/installdtbs.sh +2 -0

 
+for dtb in `find "$3" -name "*.dtb"`; do
+	# we use dtc to parse the dtb, get the board compatible string,

arch/arm/boot/installdtbs.sh line 44

Quoted 10 lines

arch/arm/boot/installdtbs.sh +10 -0

+	# and then copy the dtb to $2/${board_compatible}.dtb
+	compat="`$DTC -I dtb -O compat "$dtb"`"
+
+	if [ -e "$2/${compat}.dtb" ]; then
+		echo "Install $dtb: $2/${compat}.dtb already exists!" 1>&2
+		exit 1
+	else
+		cp "$dtb" "$2/${compat}.dtb"
+	fi
+done

This only appears to create ${compat}.dtb, and not ${dtb} too.

So, it doesn't seem to address part of the cover letter, "In addition, some vendors have done a diligent job naming their devicetree source files appropriately and we don't want to break their setups." Was that deliberate? If so, I guess I need to send some patches to U-Boot.

Quoted 15 lines · Stephen Warren

Quoted 14 lines · Jason Cooper

Unlike other build products in the Linux kernel, the devicetree blobs are simply the name of their source file, s/dts/dtb/. There is also no 'make *install' mechanism to put them in a standard place with a standard naming structure.

Unfortunately, users have begun scripting pulling the needed dtbs from within the kernel tree, thus hardcoding the dtbs names. In turn, this means any changes to the dts filenames breaks these scripts.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a vendor or distribution supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

Quoted 1 line

This only appears to create ${compat}.dtb, and not ${dtb} too.

See my comment, above.

Quoted 4 lines

So, it doesn't seem to address part of the cover letter, "In addition, some vendors have done a diligent job naming their devicetree source files appropriately and we don't want to break their setups." Was that deliberate? If so, I guess I need to send some patches to U-Boot.

I assume you are still pulling from arch/arm/boot/dts/*.dtb, right? Those build products don't go away with this patch, so you should be fine. Unless I'm mis-understanding your workflow...

Quoted 16 lines · Jason Cooper

Quoted 15 lines · Stephen Warren

Quoted 14 lines · Jason Cooper

Unlike other build products in the Linux kernel, the devicetree blobs are simply the name of their source file, s/dts/dtb/. There is also no 'make *install' mechanism to put them in a standard place with a standard naming structure.

Unfortunately, users have begun scripting pulling the needed dtbs from within the kernel tree, thus hardcoding the dtbs names. In turn, this means any changes to the dts filenames breaks these scripts.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a vendor or distribution supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

...

Quoted 9 lines

Quoted 1 line

This only appears to create ${compat}.dtb, and not ${dtb} too.

See my comment, above.

Quoted 4 lines

So, it doesn't seem to address part of the cover letter, "In addition, some vendors have done a diligent job naming their devicetree source files appropriately and we don't want to break their setups." Was that deliberate? If so, I guess I need to send some patches to U-Boot.

Quoted 3 lines

I assume you are still pulling from arch/arm/boot/dts/*.dtb, right? Those build products don't go away with this patch, so you should be fine. Unless I'm mis-understanding your workflow...

Yes, but I thought the whole point of this was that everyone would/could switch to running "make dtbs_install", then pulling out the files from the install tree, instead of any use-case requiring obtaining files directly from the source/build tree?

Quoted 31 lines · Stephen Warren

Quoted 16 lines · Jason Cooper

Quoted 15 lines · Stephen Warren

Quoted 14 lines · Jason Cooper

Unlike other build products in the Linux kernel, the devicetree blobs are simply the name of their source file, s/dts/dtb/. There is also no 'make *install' mechanism to put them in a standard place with a standard naming structure.

Unfortunately, users have begun scripting pulling the needed dtbs from within the kernel tree, thus hardcoding the dtbs names. In turn, this means any changes to the dts filenames breaks these scripts.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a vendor or distribution supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

...

Quoted 12 lines

Quoted 1 line

This only appears to create ${compat}.dtb, and not ${dtb} too.

See my comment, above.

Quoted 4 lines

So, it doesn't seem to address part of the cover letter, "In addition, some vendors have done a diligent job naming their devicetree source files appropriately and we don't want to break their setups." Was that deliberate? If so, I guess I need to send some patches to U-Boot.

I assume you are still pulling from arch/arm/boot/dts/*.dtb, right? Those build products don't go away with this patch, so you should be fine. Unless I'm mis-understanding your workflow...

Quoted 4 lines

Yes, but I thought the whole point of this was that everyone would/could switch to running "make dtbs_install", then pulling out the files from the install tree, instead of any use-case requiring obtaining files directly from the source/build tree?

Yes, in the script, it checks for the existence of /sbin/installdtbs. If it's there, it will override the default behavior and execute that script.

Quoted 37 lines · Jason Cooper

Quoted 35 lines · Stephen Warren

Quoted 16 lines · Jason Cooper

Quoted 15 lines · Stephen Warren

Quoted 14 lines · Jason Cooper

Unlike other build products in the Linux kernel, the devicetree blobs are simply the name of their source file, s/dts/dtb/. There is also no 'make *install' mechanism to put them in a standard place with a standard naming structure.

Unfortunately, users have begun scripting pulling the needed dtbs from within the kernel tree, thus hardcoding the dtbs names. In turn, this means any changes to the dts filenames breaks these scripts.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a vendor or distribution supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

...

Quoted 4 lines

Quoted 1 line

This only appears to create ${compat}.dtb, and not ${dtb} too.

See my comment, above.

Quoted 8 lines

Quoted 4 lines

So, it doesn't seem to address part of the cover letter, "In addition, some vendors have done a diligent job naming their devicetree source files appropriately and we don't want to break their setups." Was that deliberate? If so, I guess I need to send some patches to U-Boot.

I assume you are still pulling from arch/arm/boot/dts/*.dtb, right? Those build products don't go away with this patch, so you should be fine. Unless I'm mis-understanding your workflow...

Yes, but I thought the whole point of this was that everyone would/could switch to running "make dtbs_install", then pulling out the files from the install tree, instead of any use-case requiring obtaining files directly from the source/build tree?

Quoted 3 lines

Yes, in the script, it checks for the existence of /sbin/installdtbs. If it's there, it will override the default behavior and execute that script.

But that means everyone (a lot of people, and all distros) has to write, or at least download and install, such a script.

Equally, that script would incorrectly affect the naming of non-Tegra DTBs that follow the new convention. The naming of the Tegra DTBs is based more on the fact that they're the Tegra DTBs than the fact that I'm using them.

Quoted 38 lines · Stephen Warren

Quoted 37 lines · Jason Cooper

Quoted 35 lines · Stephen Warren

Quoted 16 lines · Jason Cooper

Quoted 15 lines · Stephen Warren

Quoted 14 lines · Jason Cooper

Unlike other build products in the Linux kernel, the devicetree blobs are simply the name of their source file, s/dts/dtb/. There is also no 'make *install' mechanism to put them in a standard place with a standard naming structure.

Unfortunately, users have begun scripting pulling the needed dtbs from within the kernel tree, thus hardcoding the dtbs names. In turn, this means any changes to the dts filenames breaks these scripts.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a vendor or distribution supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

...

Quoted 4 lines

Quoted 1 line

This only appears to create ${compat}.dtb, and not ${dtb} too.

See my comment, above.

Quoted 8 lines

Quoted 4 lines

So, it doesn't seem to address part of the cover letter, "In addition, some vendors have done a diligent job naming their devicetree source files appropriately and we don't want to break their setups." Was that deliberate? If so, I guess I need to send some patches to U-Boot.

I assume you are still pulling from arch/arm/boot/dts/*.dtb, right? Those build products don't go away with this patch, so you should be fine. Unless I'm mis-understanding your workflow...

Yes, but I thought the whole point of this was that everyone would/could switch to running "make dtbs_install", then pulling out the files from the install tree, instead of any use-case requiring obtaining files directly from the source/build tree?

Quoted 6 lines

Quoted 3 lines

Yes, in the script, it checks for the existence of /sbin/installdtbs. If it's there, it will override the default behavior and execute that script.

But that means everyone (a lot of people, and all distros) has to write, or at least download and install, such a script.

They would already be modifying their build environment to make use of 'make dtbs_install', and few extra lines of shell script to integrate with existing systems certainly isn't unreasonable?

Quoted 4 lines

Equally, that script would incorrectly affect the naming of non-Tegra DTBs that follow the new convention. The naming of the Tegra DTBs is based more on the fact that they're the Tegra DTBs than the fact that I'm using them.

Umm, I wrote that script in 2 minutes. I'm sure you could come up with something Tegra-specific (perhaps using fdtget?) that places only the Tegra dtbs in the location that works with Tegra's u-boot.

Again, the point is not that it's hard to write the script. As you demonstrated, it's easy. The problem is that then, everybody has to do something different for Tegra, forever. No matter how small the actual cost of doing it is, it's still non-scalable to require that everyone know about this special case. I'm not convinced the issue would be isolated to Tegra either.

As for renaming the files from what we have in the kernel, I'd say... why bother.

What if we have two DT blobs with the same compatible string (which is not unlikely when you have two configurations for a platform)?

I'd recommend keeping the names in the kernel tree when installing. If a distro needs to find the compatible string, then that's a bit of parsing they should do. Otherwise, we lose the ability to tell users "you need this DT blob file called X" because X will depend whether it's been installed or not.

And in some cases where getting the wrong DT blob could end up quite literally destroying your hardware (eg, by setting an LDO regulator to bypass mode), it's extremely important that anything here is _simple_ and doesn't involve lots of indirection.

All I'm suggesting doing is extending the same standard we have for compatible strings at the node level to the board level. If we don't change the filename, then we *have* to extend the consistent naming to the dts files as well.

Quoted 3 lines · Jason Cooper

All I'm suggesting doing is extending the same standard we have for compatible strings at the node level to the board level. If we don't change the filename,

Quoted 2 lines

then we *have* to extend the consistent naming to the dts files as well.

That, in itself, is not a bad thing to do, and totally solves the problem.

By "That" are you referring to extending the standard for compatible strings to the board level, or forcing consistent naming on the dts files?

Quoted 3 lines · Jason Cooper

By "That" are you referring to extending the standard for compatible strings to the board level, or forcing consistent naming on the dts files?

A consistent file naming on both the dtb and dts files.

Quoted 6 lines · Russell King - ARM Linux

Quoted 3 lines · Jason Cooper

By "That" are you referring to extending the standard for compatible strings to the board level, or forcing consistent naming on the dts files?

A consistent file naming on both the dtb and dts files.

It's time to knock the idea of naming the DT blobs after the compatible string on its head as well. That's really not going to work. Just leave the filenames as is, and move the problem into userspace to solve.

See - by doing this, you are building policy into the kernel build system and potentially taking away choice from userspace.

Quoted 6 lines · Russell King - ARM Linux

Quoted 3 lines · Jason Cooper

By "That" are you referring to extending the standard for compatible strings to the board level, or forcing consistent naming on the dts files?

A consistent file naming on both the dtb and dts files.

It's time to knock the idea of naming the DT blobs after the compatible string on its head as well. That's really not going to work. Just leave the filenames as is, and move the problem into userspace to solve.

Just use the filenames we already have in the kernel tree. Let userspace parse the DT blobs for the compatible strings - it can then provide the user with the choice of all and _also_ show that certain of DT blobs may be compatible with a range of boards as well.

Quoted 12 lines · Russell King - ARM Linux

Quoted 6 lines · Russell King - ARM Linux

Quoted 3 lines · Jason Cooper

By "That" are you referring to extending the standard for compatible strings to the board level, or forcing consistent naming on the dts files?

A consistent file naming on both the dtb and dts files.

It's time to knock the idea of naming the DT blobs after the compatible string on its head as well. That's really not going to work. Just leave the filenames as is, and move the problem into userspace to solve.

[...]

Quoted 1 line

Quoted 4 lines

Just use the filenames we already have in the kernel tree. Let userspace parse the DT blobs for the compatible strings - it can then provide the user with the choice of all and _also_ show that certain of DT blobs may be compatible with a range of boards as well.

Yes, the consistent filename format is a non-problem. The files that exist already have names. Nothing else is going to use them. Done.

Quoted 15 lines · Grant Likely

Quoted 12 lines · Russell King - ARM Linux

Quoted 6 lines · Russell King - ARM Linux

Quoted 3 lines · Jason Cooper

By "That" are you referring to extending the standard for compatible strings to the board level, or forcing consistent naming on the dts files?

A consistent file naming on both the dtb and dts files.

It's time to knock the idea of naming the DT blobs after the compatible string on its head as well. That's really not going to work. Just leave the filenames as is, and move the problem into userspace to solve.

[...]

Quoted 1 line

Quoted 7 lines

Quoted 4 lines

Just use the filenames we already have in the kernel tree. Let userspace parse the DT blobs for the compatible strings - it can then provide the user with the choice of all and _also_ show that certain of DT blobs may be compatible with a range of boards as well.

Yes, the consistent filename format is a non-problem. The files that exist already have names. Nothing else is going to use them. Done.

Grant, you have the wrong end of the stick. This discussion is about what happens when files from

	$(objtree)/arch/arm/boot/dts

get installed into

	/lib/modules/$(kernelversion)/...

and their renaming from the filenames we already have in arch/arm/boot/dts to a filename generated from the compatible string.

Sure it may look untidy, particularly to those of us with obsessive-compulsive tendencies, but it isn't actually a problem.

That's why I prefer to rename the dtbs, it removes the need to parse the dtb (fdtget *isn't* currently built/installed by default in dtc)

On 11/19/2013 07:23 AM, Jason Cooper wrote:
...

Quoted 2 lines

That's why I prefer to rename the dtbs, it removes the need to parse the dtb (fdtget *isn't* currently built/installed by default in dtc)

fdtget is in fact both built and installed by default. At the very least in the latest dtc source, and it's certainly part of at least the Ubuntu packages for older versions too, so I assume it's been the default for a while.

Quoted 6 lines · Stephen Warren

Again, the point is not that it's hard to write the script. As you demonstrated, it's easy. The problem is that then, everybody has to do something different for Tegra, forever. No matter how small the actual cost of doing it is, it's still non-scalable to require that everyone know about this special case. I'm not convinced the issue would be isolated to Tegra either.

That's why there's the facility to allow an override to the script, just like there's the facility to override the default script when running "make install".

Quoted 8 lines · Russell King - ARM Linux

Quoted 6 lines · Stephen Warren

Again, the point is not that it's hard to write the script. As you demonstrated, it's easy. The problem is that then, everybody has to do something different for Tegra, forever. No matter how small the actual cost of doing it is, it's still non-scalable to require that everyone know about this special case. I'm not convinced the issue would be isolated to Tegra either.

Quoted 3 lines

That's why there's the facility to allow an override to the script, just like there's the facility to override the default script when running "make install".

Again, you are completely missing the point about that not scaling at all.

Quoted 9 lines · Stephen Warren

Quoted 8 lines · Russell King - ARM Linux

Quoted 6 lines · Stephen Warren

Again, the point is not that it's hard to write the script. As you demonstrated, it's easy. The problem is that then, everybody has to do something different for Tegra, forever. No matter how small the actual cost of doing it is, it's still non-scalable to require that everyone know about this special case. I'm not convinced the issue would be isolated to Tegra either.

Quoted 5 lines

Quoted 3 lines

That's why there's the facility to allow an override to the script, just like there's the facility to override the default script when running "make install".

Again, you are completely missing the point about that not scaling at all.

No I'm not. What you're looking for is perfection. Perfection is something we can't have here - we can have "good enough".

Look, the problem is very simple: we have a bunch of DT descriptions in the kernel, which we build to a blob. We don't want distributions poking around in the kernel source tree to retrieve these blobs, so we provide a way to install them somewhere.

The installation should just copy the blobs to a path in the filesystem _or_ allow a distro to hook into the installation process, just like we provide for the installation of the zImage. That is our job done. We should do *nothing* more.

If we start doing more than that, we're getting into the realms of distribution policy, and that's something we should not be involved in.

This is precisely the reason why we don't get involved with working out what steps are needed to install a kernel zImage onto a target: that is _not_ the kernel's job to work out that you may need to package it up and maybe copy it over to a TFTP server, or maybe copy it onto a SD card, or maybe place it in /boot and do a certain number of other steps.

All that the kernel build's job here is to provide the results of the kernel build. It is not about working out which DT blob is right for any particular platform.

Quoted 4 lines · Russell King - ARM Linux

The installation should just copy the blobs to a path in the filesystem _or_ allow a distro to hook into the installation process, just like we provide for the installation of the zImage. That is our job done. We should do *nothing* more.

OK, if you're arguing that there should be absolutely zero renaming at all under any circumstances, I'm fine with that.

Quoted 11 lines · Russell King - ARM Linux

Quoted 6 lines · Stephen Warren

Again, the point is not that it's hard to write the script. As you demonstrated, it's easy. The problem is that then, everybody has to do something different for Tegra, forever. No matter how small the actual cost of doing it is, it's still non-scalable to require that everyone know about this special case. I'm not convinced the issue would be isolated to Tegra either.

That's why there's the facility to allow an override to the script, just like there's the facility to override the default script when running "make install".

Again, you are completely missing the point about that not scaling at all.

But I will go and investigate what it takes to support renaming the DTBs. Everyone (using Tegra) will have to update their bootloader, but perhaps that can be dealt with.

Quoted 15 lines · Stephen Warren

Quoted 11 lines · Russell King - ARM Linux

Quoted 6 lines · Stephen Warren

Again, the point is not that it's hard to write the script. As you demonstrated, it's easy. The problem is that then, everybody has to do something different for Tegra, forever. No matter how small the actual cost of doing it is, it's still non-scalable to require that everyone know about this special case. I'm not convinced the issue would be isolated to Tegra either.

That's why there's the facility to allow an override to the script, just like there's the facility to override the default script when running "make install".

Again, you are completely missing the point about that not scaling at all.

Quoted 3 lines

But I will go and investigate what it takes to support renaming the DTBs. Everyone (using Tegra) will have to update their bootloader, but perhaps that can be dealt with.

Setting aside the idea of hard-coding filenames into any bootloader binary, did you catch that the proposed solution would allow you to continue using the dts filenames as they currently are?

The find command in my example /sbin/installdtbs script spits out the in-tree dtb filename as it currently stands, and then you do what you want with it in your script. Including, copying it, filename intact, into the location of your choice. How could that possibly require you to upgrade your bootloader?

I must admit I'm a bit mystified as to what Tegra is doing that this breaks it so horribly...

Quoted 20 lines · Jason Cooper

Quoted 18 lines · Stephen Warren

Quoted 11 lines · Russell King - ARM Linux

Quoted 6 lines · Stephen Warren

Again, the point is not that it's hard to write the script. As you demonstrated, it's easy. The problem is that then, everybody has to do something different for Tegra, forever. No matter how small the actual cost of doing it is, it's still non-scalable to require that everyone know about this special case. I'm not convinced the issue would be isolated to Tegra either.

That's why there's the facility to allow an override to the script, just like there's the facility to override the default script when running "make install".

Again, you are completely missing the point about that not scaling at all.

But I will go and investigate what it takes to support renaming the DTBs. Everyone (using Tegra) will have to update their bootloader, but perhaps that can be dealt with.

Quoted 3 lines

Setting aside the idea of hard-coding filenames into any bootloader binary, did you catch that the proposed solution would allow you to continue using the dts filenames as they currently are?

I don't consider requiring people to install a custom /sbin/installdtbs in order to override renaming done by the default in-kernel scripts/installdtbs a solution.

Quoted 2 lines

I must admit I'm a bit mystified as to what Tegra is doing that this breaks it so horribly...

The idea is that a disto boot process runs as follow:

* U-Boot runs and initializes.

* U-Boot searches the boot disk (or any disk that could be a boot disk) for e.g. /boot/boot.scr, and executes it.

* boot.scr loads the kernel, initrd, DTB, sets up the command-line, and boots the kernel.

A few points to make here:

a) There must be a boot.scr or similar on disk, to provide a way for distros to parameterize the boot process. At the very least they need to configure the kernel command-line, specify whether an initrd is used at all, etc.

b) boot.scr is responsible for loading the zImage etc. from disk, rather than the default U-Boot environment doing this. This allows distros complete flexibility re: where to put the zImage etc. on disk. On distro might store a single copy in /boot/zImage, another might use /boot/zImage-${kernelversion}, another might use /boot/${kernelversion}/zImage, etc. By putting the onus on boot.scr to do the loading, the distro has complete flexibility in file naming here.

c) There are a number of system-specific parameters required to implement zImage/DTB loading. For example, the SDRAM address that they should be loaded to. Distros shouldn't have to detect which board they're running on and calculate the values of all those parameters. Instead, U-Boot should provide the values to boot.scr, by setting certain standard variables in the environment.

For example, the zImage gets loaded to $kernel_addr_r, the DTB to $fdt_addr_r, etc.

d) Finally, boot.scr needs to load the DTB. This requires knowing which DTB to load. Again, the distro shouldn't have to detect which board they're running on, and either install the correct DTB to a static filename, or make a decision on the DTB filename to load. Instead, distros should simply install all DTBs generated by the kernel build, and use some run-time information to calculate the DTB filename using a completely HW-agnostic and generic algorithm.

To support this, U-Boot can be configured to add certain standard environment variables to the default environment. These define which SoC and board the code is running on. These are ${soc} and ${board}. If you then calculate the DTB filename as ${soc}-${board}.dtb, that should work anywhere. This is why keeping the current in-kernel DTB filenames is important, so they match the assumption that this algorithm works.

Note: if the firmware contains an embedded DTB, boot.scr could detect this e.g. by the standard variable $fdt_addr being set, and hence skiip loading a DTB. IIRC, this is how Calxeda boards are set up.

So finally, if the kernel "make dtbs_install" starts renaming DTB files, then I need to adjust the boot.scr we're using to use some other algorithm to calculate the DTB filename. Given the filename isn't then purely derived from ${soc} and ${board}, I probably need to set up some additional (or replacement) variables in the standard U-Boot environment that contain the extra fields required to calculate the DTB filename. The existing ${vendor} might work. However, it'd be simplest if I just had U-Boot export something like ${dtb_filename}, which contained the (or a list of) filenames to try.

You can take a look at my boot.scr generation tool at:
https://github.com/NVIDIA/tegra-uboot-scripts

That probably works on the Raspberry Pi too (since I set up the Pi's U-Boot port to work the same way) and likely would require minimal changes to work on at least some TI and Calxeda boards.

The idea is that eventually, and board that wants to support a generic distro booting on it would export the standard variables I mentioned above, have a default U-Boot environment that simply searched for and executed boot.scr, and distros would use a generic boot.scr as generated by the tool I linked to above.

On Tue, Nov 19, 2013 at 12:53:07PM -0700, Stephen Warren wrote:

Quoted 15 lines

d) Finally, boot.scr needs to load the DTB. This requires knowing which DTB to load. Again, the distro shouldn't have to detect which board they're running on, and either install the correct DTB to a static filename, or make a decision on the DTB filename to load. Instead, distros should simply install all DTBs generated by the kernel build, and use some run-time information to calculate the DTB filename using a completely HW-agnostic and generic algorithm.

To support this, U-Boot can be configured to add certain standard environment variables to the default environment. These define which SoC and board the code is running on. These are ${soc} and ${board}. If you then calculate the DTB filename as ${soc}-${board}.dtb, that should work anywhere. This is why keeping the current in-kernel DTB filenames is important, so they match the assumption that this algorithm works.

Quoted 3 lines

Note: if the firmware contains an embedded DTB, boot.scr could detect this e.g. by the standard variable $fdt_addr being set, and hence skiip loading a DTB. IIRC, this is how Calxeda boards are set up.

Ok, so the ${soc}-${board} is a work-around until all bootloaders are providing a dtb, or until all bootloaders are providing a stable dtb?

Quoted 1 line · Jason Cooper

On Tue, Nov 19, 2013 at 12:53:07PM -0700, Stephen Warren wrote:

...

Quoted 15 lines

Quoted 15 lines

d) Finally, boot.scr needs to load the DTB. This requires knowing which DTB to load. Again, the distro shouldn't have to detect which board they're running on, and either install the correct DTB to a static filename, or make a decision on the DTB filename to load. Instead, distros should simply install all DTBs generated by the kernel build, and use some run-time information to calculate the DTB filename using a completely HW-agnostic and generic algorithm.

To support this, U-Boot can be configured to add certain standard environment variables to the default environment. These define which SoC and board the code is running on. These are ${soc} and ${board}. If you then calculate the DTB filename as ${soc}-${board}.dtb, that should work anywhere. This is why keeping the current in-kernel DTB filenames is important, so they match the assumption that this algorithm works.

Quoted 6 lines

Quoted 3 lines

Note: if the firmware contains an embedded DTB, boot.scr could detect this e.g. by the standard variable $fdt_addr being set, and hence skiip loading a DTB. IIRC, this is how Calxeda boards are set up.

Ok, so the ${soc}-${board} is a work-around until all bootloaders are providing a dtb, or until all bootloaders are providing a stable dtb?

I suppose so yes.

However, I'm not really convinced we'll see that many products with a DT embedded into them that's stable, complete[1], and based on upstream bindings, in the particularly near future. I think distros will likely need to deal with DTs-in-filesystems for a while yet. But anyway, either way should work out just fine.

[1] For example, we aren't that far off some reasonably stable basic bindings for Tegra now . However, there are still devices we haven't added to DT at all, so while the DT ABI stability issue is fading, the need-to-upgrade-your-DTB case still exists to get new features, and upgrading the DTB via the filesystem is a lot easier and more convenient than doing so via firmware updates or firmware flashing commands.

Quoted 5 lines · Jason Cooper

In order for 'make dtbs_install' to be maintainable, we need a reliable way to determine the board compatible string for a given dtb. fdtget does that easily as written, but currently isn't built by the kernel.

scripts/dtc/Makefile +0 -0

scripts/dtc/Makefile line 12

Quoted 3 lines

scripts/dtc/Makefile +3 -0

+include $(src)/libfdt/Makefile.libfdt
+
+fdtget-objs	:= fdtget.o util.o $(patsubst %, libfdt/%, $(LIBFDT_OBJS))

I observe that there's a scripts/dtc/Makefile.dtc that /isn't/ used by scripts/dtc/Makefile. Hence, having scripts/dtc/Makefile use scripts/dtc/libfdt/Makefile.libfdt seems slightly inconsistent. That said, it's probably a good idea to use Makefile.libfdt, so perhaps the answer here is to update Makefile to use Makefile.dtc sometime, rather than not use Makefile.libfdt in this patch?

Consumers of the Linux kernel's build products are beginning to hardcode the filenames of the dtbs generated. Since the dtb filenames are currently the dts filename s/dts/dtb/, this prevents the kernel community from renaming dts files as needed.

Let's provide a consistent naming structure for consumers to script against. Or at least, as consistent as the dts properties themselves.

With this patch, adding the '-O compat' option to the dtc commandline will cause dtc to parse the provided file, and print out the board compatible string to stdout.

This will facilitate an 'installdtbs.sh' script in the kernel for naming dtb files by their compatible string, eg:

$ dtc -I dtb -O compat arch/arm/boot/dts/armada-370-mirabox.dtb
globalscale,mirabox

This change will also simplify distribution install scripts that need to search through many dtbs to find the right one for a target board.

Signed-off-by Jason Cooper <[email protected]>

---
changes since v1:
- made patch against in-tree dtc code to facilitate testing
- dtc prints compatible string to stdout now, instead of creating symlink
- should be more flexible for end-users

 scripts/dtc/dtc.c      | 3 +++
 scripts/dtc/dtc.h      | 1 +
 scripts/dtc/flattree.c | 9 +++++++++
 3 files changed, 13 insertions(+)

scripts/dtc/dtc.c +3 -0

index a375683c1534..89264bb0a3dd 100644
@@ -68,6 +68,7 @@ static void  __attribute__ ((noreturn)) usage(void)
 	fprintf(stderr, "\t\tOutput formats are:\n");
 	fprintf(stderr, "\t\t\tdts - device tree source text\n");
 	fprintf(stderr, "\t\t\tdtb - device tree blob\n");
+	fprintf(stderr, "\t\t\tcompat - print board compatible string\n");
 	fprintf(stderr, "\t\t\tasm - assembler source\n");
 	fprintf(stderr, "\t-V <output version>\n");
 	fprintf(stderr, "\t\tBlob version to produce, defaults to %d (relevant for dtb\n\t\tand asm output only)\n", DEFAULT_FDT_VERSION);

@@ -250,6 +251,8 @@ int main(int argc, char *argv[])
 		dt_to_blob(outf, bi, outversion);
 	} else if (streq(outform, "asm")) {
 		dt_to_asm(outf, bi, outversion);
+	} else if (streq(outform, "compat")) {
+		dt_to_compat(bi);
 	} else if (streq(outform, "null")) {
 		/* do nothing */
 	} else {

scripts/dtc/dtc.h +1 -0

index 3e42a071070e..d4e47c697c2f 100644
@@ -255,6 +255,7 @@ void process_checks(int force, struct boot_info *bi);
 
 void dt_to_blob(FILE *f, struct boot_info *bi, int version);
 void dt_to_asm(FILE *f, struct boot_info *bi, int version);
+void dt_to_compat(struct boot_info *bi);
 
 struct boot_info *dt_from_blob(const char *fname);
 

scripts/dtc/flattree.c +9 -0

index 665dad7bb465..bdbd3d7e8964 100644
@@ -577,6 +577,15 @@ void dt_to_asm(FILE *f, struct boot_info *bi, int version)
 	data_free(strbuf);
 }
 
+void dt_to_compat(struct boot_info *bi)
+{
+	struct property *prop;
+
+	prop = get_property(bi->dt, "compatible");
+
+	printf("%s\n", prop->val.val);
+}
+
 struct inbuf {
 	char *base, *limit, *ptr;
 };

Signature · 7 lines

1.8.4.3


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Unlike other build products in the Linux kernel, the devicetree blobs are simply the name of their source file, s/dts/dtb/. There is also no 'make *install' mechanism to put them in a standard place with a standard naming structure.

Unfortunately, users have begun scripting pulling the needed dtbs from within the kernel tree, thus hardcoding the dtbs names. In turn, this means any changes to the dts filenames breaks these scripts.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a vendor or distribution supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

  /lib/modules/${kernel_version}/devicetree/${board_compat}.dtb

This solves several problems for us. The board compatible string should be unique, this enforces/highlights that. While devicetrees are stablilizing, the fact is, devicetrees aren't (yet) independent of kernel version. This install target facilitates keeping track of that.

Once the devicetree files are moved to their own repository, this install target can be removed as users will be modifying their scripts anyway.

Signed-off-by Jason Cooper <[email protected]>

---
changes since v2:
- use fdtget instead of a modified dtc to get the board compat string

changes since v1:
- added this patch

 Makefile                     |  3 ++-
 arch/arm/Makefile            |  4 ++++
 arch/arm/boot/installdtbs.sh | 44 ++++++++++++++++++++++++++++++++++++++++++++

 3 files changed, 50 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm/boot/installdtbs.sh

Makefile +2 -1

index 920ad07180c9..29d609e972d6 100644
@@ -339,6 +339,7 @@ OBJDUMP		= $(CROSS_COMPILE)objdump
 AWK		= awk
 GENKSYMS	= scripts/genksyms/genksyms
 INSTALLKERNEL  := installkernel
+INSTALLDTBS    := installdtbs
 DEPMOD		= /sbin/depmod
 PERL		= perl
 CHECK		= sparse

@@ -391,7 +392,7 @@ KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(S
 export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION
 export ARCH SRCARCH CONFIG_SHELL HOSTCC HOSTCFLAGS CROSS_COMPILE AS LD CC
 export CPP AR NM STRIP OBJCOPY OBJDUMP
-export MAKE AWK GENKSYMS INSTALLKERNEL PERL UTS_MACHINE
+export MAKE AWK GENKSYMS INSTALLKERNEL INSTALLDTBS PERL UTS_MACHINE
 export HOSTCXX HOSTCXXFLAGS LDFLAGS_MODULE CHECK CHECKFLAGS
 
 export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS LDFLAGS

arch/arm/Makefile +4 -0

index c99b1086d83d..c16ebb1e74b0 100644
@@ -314,6 +314,10 @@ PHONY += dtbs
 dtbs: scripts
 	$(Q)$(MAKE) $(build)=$(boot)/dts MACHINE=$(MACHINE) dtbs
 
+dtbs_install: dtbs
+	$(CONFIG_SHELL) $(srctree)/$(boot)/installdtbs.sh $(KERNELRELEASE) \
+	"$(MODLIB)/devicetree" "$(srctree)/$(boot)/dts"
+
 # We use MRPROPER_FILES and CLEAN_FILES now
 archclean:
 	$(Q)$(MAKE) $(clean)=$(boot)

arch/arm/boot/installdtbs.sh new file +33 -0

new file mode 100644
index 000000000000..3adef376188f
@@ -0,0 +1,44 @@
+#!/bin/sh
+#
+# This file is subject to the terms and conditions of the GNU General Public
+# License.  See the file "COPYING" in the main directory of this archive
+# for more details.
+#
+# Copyright (C) 1995 by Linus Torvalds
+#
+# Adapted from code in arch/i386/boot/Makefile by H. Peter Anvin
+#
+# Further adapted from arch/x86/boot/install.sh by Jason Cooper
+#
+# "make dtbs_install" script
+#
+# Arguments:
+#   $1 - kernel version
+#   $2 - default install path (blank if root directory)
+#   $3 - directory containing dtbs
+#
+
+# User may have a custom install script
+
+if [ -x ~/bin/${INSTALLDTBS} ]; then exec ~/bin/${INSTALLDTBS} "$@"; fi
+if [ -x /sbin/${INSTALLDTBS} ]; then exec /sbin/${INSTALLDTBS} "$@"; fi
+
+FDTGET=./scripts/dtc/fdtget
+
+# Default install
+[ -d "$2" ] && rm -rf "$2"
+
+mkdir -p "$2"
+
+for dtb in `find "$3" -name "*.dtb"`; do

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

  /lib/modules/${kernel_version}/devicetree/${dts_filename}.dtb

This is done to keep dtbs from different kernel versions separate until things have settled down. Once the dtbs are stable, and not so strongly linked to the kernel version, the devicetree files will most likely move to their own repo. Users will need to upgrade install scripts at that time.

Signed-off-by Jason Cooper <[email protected]>

---
Question: should I make a note about the filenames not being considered an ABI, or just let it be?

Quoted 9 lines · Jason Cooper

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

Quoted 1 line

  /lib/modules/${kernel_version}/devicetree/${dts_filename}.dtb

I still don't see why you wouldn't install the files in /lib/devicetrees, but I suppose that location is fine.

Quoted 10 lines · Stephen Warren

Quoted 9 lines · Jason Cooper

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

Quoted 4 lines

Quoted 1 line

  /lib/modules/${kernel_version}/devicetree/${dts_filename}.dtb

I still don't see why you wouldn't install the files in /lib/devicetrees, but I suppose that location is fine.

/boot/devicetrees/${kernel_version} would be my choice. /boot is more likely to be available to firmware than /lib would be.

Quoted 9 lines · Jason Cooper

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

Quoted 1 line

  /lib/modules/${kernel_version}/devicetree/${dts_filename}.dtb

I still don't see why you wouldn't install the files in /lib/devicetrees, but I suppose that location is fine.

Quoted 10 lines

This is done to keep dtbs from different kernel versions separate until things have settled down. Once the dtbs are stable, and not so strongly linked to the kernel version, the devicetree files will most likely move to their own repo. Users will need to upgrade install scripts at that time.

Signed-off-by Jason Cooper <[email protected]>

---
Question: should I make a note about the filenames not being considered an ABI, or just let it be?

I still believe they're an ABI.

I guess Grant meant that they aren't an ABI to the kernel at run-time, unlike the content which is an ABI. That's fine.

However, I believe the files certainly are an ABI to any script that takes them from the "make dtbs_install" output directory.

Perhaps the best resolution is to just say nothing and let it be:-)

arch/arm/Makefile PHONY += dtbs · line 319

Quoted 5 lines

arch/arm/Makefile +3 -0

 
+dtbs_install: dtbs
+	$(CONFIG_SHELL) $(srctree)/scripts/installdtbs.sh $(KERNELRELEASE) \
+	"$(MODLIB)/devicetree" "$(srctree)/$(boot)/dts"

I still think this rule should be in Makefile not arch/arm/Makefile.

Aside from those couple of issues, this version looks OK to me.

Quoted 10 lines · Stephen Warren

Quoted 9 lines · Jason Cooper

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

Quoted 4 lines

Quoted 1 line

  /lib/modules/${kernel_version}/devicetree/${dts_filename}.dtb

I still don't see why you wouldn't install the files in /lib/devicetrees, but I suppose that location is fine.

Hmm, after sleeping on it a night, I've come around. If we install to /lib/devicetree/${kernel_version}/, then when the devicetree moves to it's own repo, it should be able to install to /lib/devicetree.

So this:
1) hints at the still strong connection between a dtb and the kernel version
2) doesn't sound odd like installing dtbs into a modules directory
3) allows a future independent repository to install non-kernel-version-dependent dtbs in a sane location.

I'm still not in favor of installing to /boot/anything since /boot is often a separate, small partition. I even set mine not to automount. The number and size of dtbs is only going to grow. Better to install them in a location with lots of space, and /lib is used to having the kernel modules installed in it.

I'm still not in favor of installing to /boot/anything since /boot is often a separate, small partition. I even set mine not to automount. The number and size of dtbs is only going to grow. Better to install them in a location with lots of space, and /lib is used to having the kernel modules installed in it.

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

  /boot/devicetrees/${kernel_version}/${dts_filename}.dtb

This is done to keep dtbs from different kernel versions separate until things have settled down. Once the dtbs are stable, and not so strongly linked to the kernel version, the devicetree files will most likely move to their own repo. Users will need to upgrade install scripts at that time.

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

/boot/devicetrees/${kernel_version}/${dts_filename}.dtb

This is done to keep dtbs from different kernel versions separate until things have settled down. Once the dtbs are stable, and not so strongly linked to the kernel version, the devicetree files will most likely move to their own repo. Users will need to upgrade install scripts at that time.

Signed-off-by Jason Cooper <[email protected]>

---
changes since v4:
- move make target from arch/arm/Makefile to Makefile (swarren)
- change default install location to /boot/devicetrees/$KERNVER (swarren/gcl)
- add INSTALL_DTBS_PATH for changing build root (jac)

changes since v3:
- drop renaming files to ${compat}.dtb (rmk/swarren)
- move installdtbs.sh to ./scripts/ (gcl)

changes since v2:
- use fdtget instead of a modified dtc to get the board compat string

changes since v1:
- added this patch

Makefile | 20 +++++++++++++++++++-
scripts/installdtbs.sh | 33 +++++++++++++++++++++++++++++++++

2 files changed, 52 insertions(+), 1 deletion(-)

create mode 100644 scripts/installdtbs.sh

Makefile +0 -0

index 920ad07180c9..bd28cb211411 100644
@@ -339,6 +339,7 @@ OBJDUMP		= $(CROSS_COMPILE)objdump

AWK = awk
GENKSYMS = scripts/genksyms/genksyms
INSTALLKERNEL := installkernel
+INSTALLDTBS := installdtbs
DEPMOD = /sbin/depmod
PERL = perl
CHECK = sparse

Makefile +0 -0

@@ -391,7 +392,7 @@ KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(S

export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION export ARCH SRCARCH CONFIG_SHELL HOSTCC HOSTCFLAGS CROSS_COMPILE AS LD CC export CPP AR NM STRIP OBJCOPY OBJDUMP

Makefile +1 -1

-export MAKE AWK GENKSYMS INSTALLKERNEL PERL UTS_MACHINE
+export MAKE AWK GENKSYMS INSTALLKERNEL INSTALLDTBS PERL UTS_MACHINE

export HOSTCXX HOSTCXXFLAGS LDFLAGS_MODULE CHECK CHECKFLAGS

export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS LDFLAGS

Makefile +0 -0

@@ -704,6 +705,15 @@ export KBUILD_IMAGE ?= vmlinux

export INSTALL_PATH ?= /boot

#

Makefile +5 -0

+
+DTBBOOT = $(INSTALL_DTBS_PATH)/boot/devicetrees/$(KERNELRELEASE)
+export DTBBOOT
+
+#

# INSTALL_MOD_PATH specifies a prefix to MODLIB for module directory

# relocations required by build roots.  This is not defined in the
# makefile but the argument can be passed to make if needed.

Makefile +8 -0

@@ -914,6 +924,14 @@ firmware_install: FORCE
 	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.fwinst obj=firmware __fw_install
 
 # ---------------------------------------------------------------------------
+# devicetree install
+PHONY += dtbs_install
+
+dtbs_install:
+	$(CONFIG_SHELL) $(srctree)/scripts/installdtbs.sh $(KERNELRELEASE) \
+	"$(DTBBOOT)" "$(srctree)"
+
+# ---------------------------------------------------------------------------
 # Kernel headers
 
 #Default location for installed headers

scripts/installdtbs.sh new file +33 -0

new file mode 100644
index 000000000000..11027f00c3a4
@@ -0,0 +1,33 @@
+#!/bin/sh
+#
+# This file is subject to the terms and conditions of the GNU General Public
+# License.  See the file "COPYING" in the main directory of this archive
+# for more details.
+#
+# Copyright (C) 1995 by Linus Torvalds
+#
+# Adapted from code in arch/i386/boot/Makefile by H. Peter Anvin
+#
+# Further adapted from arch/x86/boot/install.sh by Jason Cooper
+#
+# "make dtbs_install" script
+#
+# Arguments:
+#   $1 - kernel version
+#   $2 - default install path (blank if root directory)
+#   $3 - directory containing dtbs
+#
+
+# User may have a custom install script
+
+if [ -x ~/bin/${INSTALLDTBS} ]; then exec ~/bin/${INSTALLDTBS} "$@"; fi
+if [ -x /sbin/${INSTALLDTBS} ]; then exec /sbin/${INSTALLDTBS} "$@"; fi
+
+# Default install
+[ -d "$2" ] && rm -rf "$2"
+
+mkdir -p "$2"
+
+for dtb in `find "$3" -name "*.dtb"`; do
+	cp "$dtb" "$2/${dtb##*/}"
+done

Signature · 6 lines

1.8.4.4

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Quoted 3 lines · Jason Cooper

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

Quoted 7 lines

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

/boot/devicetrees/${kernel_version}/${dts_filename}.dtb

Quoted 6 lines · Kumar Gala

Quoted 3 lines · Jason Cooper

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

Quoted 7 lines

Quoted 7 lines

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

/boot/devicetrees/${kernel_version}/${dts_filename}.dtb

Random bikeshed of the day. Maybe this was already covered in the long thread and just tell me to shut up and go away if it was and I'm not presenting new arguments:

I like this whole approach, but I'm all for shorter pathnames as long as they are unique. Also, keeping it below 8 can be useful for FAT /boot filesystems. I propose /boot/dtb/ instead.

Quoted 19 lines · Olof Johansson

Quoted 6 lines · Kumar Gala

Quoted 3 lines · Jason Cooper

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

Quoted 7 lines

Quoted 7 lines

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

/boot/devicetrees/${kernel_version}/${dts_filename}.dtb

Random bikeshed of the day. Maybe this was already covered in the long thread and just tell me to shut up and go away if it was and I'm not presenting new arguments:

Quoted 3 lines

I like this whole approach, but I'm all for shorter pathnames as long as they are unique. Also, keeping it below 8 can be useful for FAT /boot filesystems. I propose /boot/dtb/ instead.

Makes sense to me.

Quoted 24 lines · Jason Cooper

Makefile +0 -0

index 920ad07180c9..bd28cb211411 100644
@@ -339,6 +339,7 @@ OBJDUMP		= $(CROSS_COMPILE)objdump

AWK = awk
GENKSYMS = scripts/genksyms/genksyms
INSTALLKERNEL := installkernel
+INSTALLDTBS := installdtbs
DEPMOD = /sbin/depmod
PERL = perl
CHECK = sparse

Makefile +0 -0

@@ -391,7 +392,7 @@ KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(S

export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION export ARCH SRCARCH CONFIG_SHELL HOSTCC HOSTCFLAGS CROSS_COMPILE AS LD CC export CPP AR NM STRIP OBJCOPY OBJDUMP

Makefile +1 -1

-export MAKE AWK GENKSYMS INSTALLKERNEL PERL UTS_MACHINE
+export MAKE AWK GENKSYMS INSTALLKERNEL INSTALLDTBS PERL UTS_MACHINE

export HOSTCXX HOSTCXXFLAGS LDFLAGS_MODULE CHECK CHECKFLAGS

export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS LDFLAGS

Makefile +0 -0

@@ -704,6 +705,15 @@ export KBUILD_IMAGE ?= vmlinux

export INSTALL_PATH ?= /boot

Makefile export KBUILD_IMAGE ?= vmlinux · line 711

Quoted 5 lines

#

Makefile +4 -0

+# INSTALL_DTBS_PATH specifies a prefix for relocations required by build roots.
+# Like INSTALL_MOD_PATH, it isn't defined in the Makefile, but can be passed as
+# an argument if needed.
+#

Nit, do we want to match INSTALL_MOD_PATH and drop the ?S? for INSTALL_DTB_PATH?

Quoted 23 lines

Makefile +5 -0

+
+DTBBOOT = $(INSTALL_DTBS_PATH)/boot/devicetrees/$(KERNELRELEASE)
+export DTBBOOT
+
+#

# INSTALL_MOD_PATH specifies a prefix to MODLIB for module directory

# relocations required by build roots.  This is not defined in the
# makefile but the argument can be passed to make if needed.

Makefile +8 -0

@@ -914,6 +924,14 @@ firmware_install: FORCE
 	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.fwinst obj=firmware __fw_install
 
 # ---------------------------------------------------------------------------
+# devicetree install
+PHONY += dtbs_install
+
+dtbs_install:
+	$(CONFIG_SHELL) $(srctree)/scripts/installdtbs.sh $(KERNELRELEASE) \
+	"$(DTBBOOT)" "$(srctree)"
+
+# ---------------------------------------------------------------------------
 # Kernel headers
 
 #Default location for installed headers

On Nov 21, 2013, at 1:35 PM, Jason Cooper <[email protected]> wrote:

Makefile export KBUILD_IMAGE ?= vmlinux · line 711

Quoted 5 lines

#

Makefile +4 -0

+# INSTALL_DTBS_PATH specifies a prefix for relocations required by build roots.
+# Like INSTALL_MOD_PATH, it isn't defined in the Makefile, but can be passed as
+# an argument if needed.
+#

Nit, do we want to match INSTALL_MOD_PATH and drop the ?S? for INSTALL_DTB_PATH?

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

  /boot/devicetrees/${kernel_version}/${dts_filename}.dtb

This is done to keep dtbs from different kernel versions separate until things have settled down. Once the dtbs are stable, and not so strongly linked to the kernel version, the devicetree files will most likely move to their own repo. Users will need to upgrade install scripts at that time.

Signed-off-by Jason Cooper <[email protected]>

---
changes since v4:
- move make target from arch/arm/Makefile to Makefile (swarren)
- change default install location to /boot/devicetrees/$KERNVER (swarren/gcl)
- add INSTALL_DTBS_PATH for changing build root (jac)

changes since v3:
- drop renaming files to ${compat}.dtb (rmk/swarren)
- move installdtbs.sh to ./scripts/ (gcl)

changes since v2:
- use fdtget instead of a modified dtc to get the board compat string

changes since v1:
- added this patch

 Makefile               | 20 +++++++++++++++++++-

 scripts/installdtbs.sh | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 52 insertions(+), 1 deletion(-)
 create mode 100644 scripts/installdtbs.sh

Makefile +19 -1

index 920ad07180c9..bd28cb211411 100644
@@ -339,6 +339,7 @@ OBJDUMP		= $(CROSS_COMPILE)objdump
 AWK		= awk
 GENKSYMS	= scripts/genksyms/genksyms
 INSTALLKERNEL  := installkernel
+INSTALLDTBS    := installdtbs
 DEPMOD		= /sbin/depmod
 PERL		= perl
 CHECK		= sparse

@@ -391,7 +392,7 @@ KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(S
 export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION
 export ARCH SRCARCH CONFIG_SHELL HOSTCC HOSTCFLAGS CROSS_COMPILE AS LD CC
 export CPP AR NM STRIP OBJCOPY OBJDUMP
-export MAKE AWK GENKSYMS INSTALLKERNEL PERL UTS_MACHINE
+export MAKE AWK GENKSYMS INSTALLKERNEL INSTALLDTBS PERL UTS_MACHINE
 export HOSTCXX HOSTCXXFLAGS LDFLAGS_MODULE CHECK CHECKFLAGS
 
 export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS LDFLAGS

@@ -704,6 +705,15 @@ export KBUILD_IMAGE ?= vmlinux
 export	INSTALL_PATH ?= /boot
 
 #
+# INSTALL_DTBS_PATH specifies a prefix for relocations required by build roots.
+# Like INSTALL_MOD_PATH, it isn't defined in the Makefile, but can be passed as
+# an argument if needed.
+#
+
+DTBBOOT = $(INSTALL_DTBS_PATH)/boot/devicetrees/$(KERNELRELEASE)
+export DTBBOOT
+
+#
 # INSTALL_MOD_PATH specifies a prefix to MODLIB for module directory
 # relocations required by build roots.  This is not defined in the
 # makefile but the argument can be passed to make if needed.

@@ -914,6 +924,14 @@ firmware_install: FORCE
 	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.fwinst obj=firmware __fw_install
 
 # ---------------------------------------------------------------------------
+# devicetree install
+PHONY += dtbs_install
+
+dtbs_install:
+	$(CONFIG_SHELL) $(srctree)/scripts/installdtbs.sh $(KERNELRELEASE) \
+	"$(DTBBOOT)" "$(srctree)"
+
+# ---------------------------------------------------------------------------
 # Kernel headers
 
 #Default location for installed headers

scripts/installdtbs.sh new file +33 -0

new file mode 100644
index 000000000000..11027f00c3a4
@@ -0,0 +1,33 @@
+#!/bin/sh
+#
+# This file is subject to the terms and conditions of the GNU General Public
+# License.  See the file "COPYING" in the main directory of this archive
+# for more details.
+#
+# Copyright (C) 1995 by Linus Torvalds
+#
+# Adapted from code in arch/i386/boot/Makefile by H. Peter Anvin
+#
+# Further adapted from arch/x86/boot/install.sh by Jason Cooper
+#
+# "make dtbs_install" script
+#
+# Arguments:
+#   $1 - kernel version
+#   $2 - default install path (blank if root directory)
+#   $3 - directory containing dtbs
+#
+
+# User may have a custom install script
+
+if [ -x ~/bin/${INSTALLDTBS} ]; then exec ~/bin/${INSTALLDTBS} "$@"; fi
+if [ -x /sbin/${INSTALLDTBS} ]; then exec /sbin/${INSTALLDTBS} "$@"; fi
+
+# Default install
+[ -d "$2" ] && rm -rf "$2"
+
+mkdir -p "$2"
+
+for dtb in `find "$3" -name "*.dtb"`; do
+	cp "$dtb" "$2/${dtb##*/}"
+done

Signature · 7 lines

1.8.4.4


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Quoted 34 lines · Jason Cooper

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install a given dtb into

  /boot/devicetrees/${kernel_version}/${dts_filename}.dtb

This is done to keep dtbs from different kernel versions separate until things have settled down. Once the dtbs are stable, and not so strongly linked to the kernel version, the devicetree files will most likely move to their own repo. Users will need to upgrade install scripts at that time.

Signed-off-by Jason Cooper <[email protected]>

---
changes since v4:
- move make target from arch/arm/Makefile to Makefile (swarren)
- change default install location to /boot/devicetrees/$KERNVER (swarren/gcl)
- add INSTALL_DTBS_PATH for changing build root (jac)

changes since v3:
- drop renaming files to ${compat}.dtb (rmk/swarren)
- move installdtbs.sh to ./scripts/ (gcl)

changes since v2:
- use fdtget instead of a modified dtc to get the board compat string

changes since v1:
- added this patch

Quoted 1 line

 Makefile               | 20 +++++++++++++++++++-

Adding it to Makefile enables it unconditionally for all architectures, even the ones with no DT support. I suspect (but have no evidence for) that this will be considered bad form. I have no problem with each architecture adding the build rule explicitly. The script is in a common place and that is the important bit.

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install the built dtbs into

  /lib/devicetrees/${kernel_version}/${dts_filename}.dtb

This is done to keep dtbs from different kernel versions separate until things have settled down. Once the dtbs are stable, and not so strongly linked to the kernel version, the devicetree files will most likely move to their own repo. Users will need to upgrade install scripts at that time.

/lib has been selected over /boot since /boot is often a FAT filesystem and a majority of the dts filenames are longer than 8+3.

Signed-off-by Jason Cooper <[email protected]>

---
Note: Stephen, I haven't added your Ack because I wanted to make sure you were Ok with the change to /lib in light of /boot partitions formatted to FAT

Quoted 17 lines · Jason Cooper

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install the built dtbs into

  /lib/devicetrees/${kernel_version}/${dts_filename}.dtb

This is done to keep dtbs from different kernel versions separate until things have settled down. Once the dtbs are stable, and not so strongly linked to the kernel version, the devicetree files will most likely move to their own repo. Users will need to upgrade install scripts at that time.

Quoted 7 lines

/lib has been selected over /boot since /boot is often a FAT filesystem and a majority of the dts filenames are longer than 8+3.

Signed-off-by Jason Cooper <[email protected]>

---
Note: Stephen, I haven't added your Ack because I wanted to make sure you were Ok with the change to /lib in light of /boot partitions formatted to FAT

This script shouldn't be installing directly to the filesystem, but rather some staging area that then gets copied to the desired location by distro-/use-specific scripts. As such, the filesystem in /boot isn't relevant.

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install the built dtbs into

  /lib/devicetrees/${kernel_version}/${dts_filename}.dtb

This is done to keep dtbs from different kernel versions separate until things have settled down. Once the dtbs are stable, and not so strongly linked to the kernel version, the devicetree files will most likely move to their own repo. Users will need to upgrade install scripts at that time.

/lib has been selected over /boot since /boot is often a FAT filesystem and a majority of the dts filenames are longer than 8+3.

Signed-off-by Jason Cooper <[email protected]>

---

Note: Stephen, I haven't added your Ack because I wanted to make sure you were Ok with the change to /lib in light of /boot partitions formatted to FAT

changes since v5:
- move make target back to arch/arm/Makefile (gcl)
- change default install location to /lib from /boot (FAT) (ojn,jac)

changes since v4:
- move make target from arch/arm/Makefile to Makefile (swarren)
- change default install location to /boot/devicetrees/$KERNVER (swarren/gcl)
- add INSTALL_DTBS_PATH for changing build root (jac)

changes since v3:
- drop renaming files to ${compat}.dtb (rmk/swarren)
- move installdtbs.sh to ./scripts/ (gcl)

changes since v2:
- use fdtget instead of a modified dtc to get the board compat string

changes since v1:
- added this patch

 Makefile               | 12 +++++++++++-
 arch/arm/Makefile      | 10 ++++++++++
 scripts/installdtbs.sh | 33 +++++++++++++++++++++++++++++++++
 3 files changed, 54 insertions(+), 1 deletion(-)
 create mode 100644 scripts/installdtbs.sh

Makefile +11 -1

index c0c2d58e3998..240e0d52b126 100644
@@ -339,6 +339,7 @@ OBJDUMP		= $(CROSS_COMPILE)objdump
 AWK		= awk
 GENKSYMS	= scripts/genksyms/genksyms
 INSTALLKERNEL  := installkernel
+INSTALLDTBS    := installdtbs
 DEPMOD		= /sbin/depmod
 PERL		= perl
 CHECK		= sparse

@@ -391,7 +392,7 @@ KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(S
 export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION
 export ARCH SRCARCH CONFIG_SHELL HOSTCC HOSTCFLAGS CROSS_COMPILE AS LD CC
 export CPP AR NM STRIP OBJCOPY OBJDUMP
-export MAKE AWK GENKSYMS INSTALLKERNEL PERL UTS_MACHINE
+export MAKE AWK GENKSYMS INSTALLKERNEL INSTALLDTBS PERL UTS_MACHINE
 export HOSTCXX HOSTCXXFLAGS LDFLAGS_MODULE CHECK CHECKFLAGS
 
 export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS LDFLAGS

@@ -704,6 +705,15 @@ export KBUILD_IMAGE ?= vmlinux
 export	INSTALL_PATH ?= /boot
 
 #
+# INSTALL_DTBS_PATH specifies a prefix for relocations required by build roots.
+# Like INSTALL_MOD_PATH, it isn't defined in the Makefile, but can be passed as
+# an argument if needed.
+#
+
+DTBBOOT = $(INSTALL_DTBS_PATH)/lib/devicetrees/$(KERNELRELEASE)
+export DTBBOOT
+
+#
 # INSTALL_MOD_PATH specifies a prefix to MODLIB for module directory
 # relocations required by build roots.  This is not defined in the
 # makefile but the argument can be passed to make if needed.

arch/arm/Makefile +10 -0

index c99b1086d83d..400a3a7ee899 100644
@@ -314,6 +314,12 @@ PHONY += dtbs
 dtbs: scripts
 	$(Q)$(MAKE) $(build)=$(boot)/dts MACHINE=$(MACHINE) dtbs
 
+PHONY += dtbs_install
+
+dtbs_install: dtbs
+	$(CONFIG_SHELL) $(srctree)/scripts/installdtbs.sh $(KERNELRELEASE) \
+	"$(DTBBOOT)" "$(srctree)"
+
 # We use MRPROPER_FILES and CLEAN_FILES now
 archclean:
 	$(Q)$(MAKE) $(clean)=$(boot)

@@ -331,6 +337,10 @@ define archhelp
   echo  '  bootpImage    - Combined zImage and initial RAM disk'
   echo  '                  (supply initrd image via make variable INITRD=<path>)'
   echo  '* dtbs          - Build device tree blobs for enabled boards'
+  echo  '  dtbs_install  - Install dtbs'
+  echo  '                  Install using (your) ~/bin/$(INSTALLDTBS) or'
+  echo  '                  (distribution) /sbin/$(INSTALLDTBS) or'
+  echo  '                  install to $(DTBBOOT)'
   echo  '  install       - Install uncompressed kernel'
   echo  '  zinstall      - Install compressed kernel'
   echo  '  uinstall      - Install U-Boot wrapped compressed kernel'

scripts/installdtbs.sh new file +33 -0

new file mode 100644
index 000000000000..11027f00c3a4
@@ -0,0 +1,33 @@
+#!/bin/sh
+#
+# This file is subject to the terms and conditions of the GNU General Public
+# License.  See the file "COPYING" in the main directory of this archive
+# for more details.
+#
+# Copyright (C) 1995 by Linus Torvalds
+#
+# Adapted from code in arch/i386/boot/Makefile by H. Peter Anvin
+#
+# Further adapted from arch/x86/boot/install.sh by Jason Cooper
+#
+# "make dtbs_install" script
+#
+# Arguments:
+#   $1 - kernel version
+#   $2 - default install path (blank if root directory)
+#   $3 - directory containing dtbs
+#
+
+# User may have a custom install script
+
+if [ -x ~/bin/${INSTALLDTBS} ]; then exec ~/bin/${INSTALLDTBS} "$@"; fi
+if [ -x /sbin/${INSTALLDTBS} ]; then exec /sbin/${INSTALLDTBS} "$@"; fi
+
+# Default install
+[ -d "$2" ] && rm -rf "$2"
+
+mkdir -p "$2"
+
+for dtb in `find "$3" -name "*.dtb"`; do
+	cp "$dtb" "$2/${dtb##*/}"
+done

Signature · 6 lines

1.8.4.4

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Grant,

Quoted 17 lines · Jason Cooper

Unlike other build products in the Linux kernel, there is no 'make *install' mechanism to put devicetree blobs in a standard place.

This patch is an attempt to fix this problem. Akin to 'make install', this creates a new make target, dtbs_install. The script that gets called defers to a distribution or user supplied installdtbs binary, if found in the system. Otherwise, the default action is to install the built dtbs into

  /lib/devicetrees/${kernel_version}/${dts_filename}.dtb

This is done to keep dtbs from different kernel versions separate until things have settled down. Once the dtbs are stable, and not so strongly linked to the kernel version, the devicetree files will most likely move to their own repo. Users will need to upgrade install scripts at that time.

Quoted 5 lines

/lib has been selected over /boot since /boot is often a FAT filesystem and a majority of the dts filenames are longer than 8+3.

Signed-off-by Jason Cooper <[email protected]>

---

Is this good to go?

No other kernel installation target moves the target directory out of the way, even deleting an old version of it. These are destructive operations, ones which the kernel build system should not be making.

Remove this behaviour.

If this behaviour is required at installation time, this should be done by the installation external to the kernel makefiles, just like it would be done for 'make modules_install'.

Signed-off-by Russell King <[email protected]>

---
 scripts/Makefile.dtbinst | 2 --
 1 file changed, 2 deletions(-)

Okay, I'm digging up an old version of this patch - v7 was merged but I find *nowhere* where that version was posted to people involved in this discussion.

The reason is that I would've commented on v7, because of this stupid thing (which is now in scripts/Makefile.dtbsinst):

(quoted diff) +3 -0

+       $(Q)if [ -d $(INSTALL_DTBS_PATH).old ]; then rm -rf $(INSTALL_DTBS_PATH).old; fi
+       $(Q)if [ -d $(INSTALL_DTBS_PATH) ]; then mv $(INSTALL_DTBS_PATH) $(INSTALL_DTBS_PATH).old; fi
+       $(Q)mkdir -p $(INSTALL_DTBS_PATH)

What gives any kernel installation target the right to move a directory out of the way?

Let's say that you do this:

make install INSTALL_PATH=$sys_root/boot make dtbs_install INSTALL_DTBS_PATH=$sys_root/boot

The result is that dtbs_install thinks it has the right to rename that boot directory to boot.old, create a new one, and place the DTBs in there, thereby leaving you with no kernel to boot - and if you run it again, it _deletes_ the original directory.

No *other* kernel install target has this behaviour, not even 'make modules_installl'. This is stupid and dangerous behaviour. At the very least, this behaviour should be *well* documented.

Russell,

Quoted 3 lines · Russell King - ARM Linux

Okay, I'm digging up an old version of this patch - v7 was merged but I find *nowhere* where that version was posted to people involved in this discussion.

fwiw, I just went through my archive of patch submissions:

$ ls ~/projects/linux/dtbs_install/
v2  v3  v4  v5  v6
$

I never wrote a v7. :-( Regardless,

Quoted 13 lines · Jason Cooper

Russell,

Quoted 3 lines · Russell King - ARM Linux

Okay, I'm digging up an old version of this patch - v7 was merged but I find *nowhere* where that version was posted to people involved in this discussion.

fwiw, I just went through my archive of patch submissions:

$ ls ~/projects/linux/dtbs_install/
v2  v3  v4  v5  v6
$

Quoted 1 line

I never wrote a v7. :-( Regardless,

It was posted by Grant, and posted to LKML, devicetree@ and linux-kbuild. The patch was directly cc:d to Jason, Michal Marek, Russell and Rob Herring.

Quoted 13 lines · Jason Cooper

Russell,

Quoted 3 lines · Russell King - ARM Linux

Okay, I'm digging up an old version of this patch - v7 was merged but I find *nowhere* where that version was posted to people involved in this discussion.

fwiw, I just went through my archive of patch submissions:

$ ls ~/projects/linux/dtbs_install/
v2  v3  v4  v5  v6
$

Quoted 1 line

I never wrote a v7. :-( Regardless,

It was posted by Grant, and posted to LKML, devicetree@ and linux-kbuild. The patch was directly cc:d to Jason, Michal Marek, Russell and Rob Herring.