Skip to contentSkip to conversation

dt-bindings: DTB based validation

PATCH Superseded Rob Herring 3 participants 16 messages Mar 3, 2022 – Mar 10, 2022

Subsystems (3)
Device tree bindings Kbuild Device tree

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Rob Herring

author maintainer
#0 lore raw

cover dt-bindings: DTB based validation

This series switches the kernel's DT schema validation from YAML encoded DT files to using DTB files directly. See patch 2 for the full reasoning of why. The diffstat also shows this is a nice simplification (at least from the kernel side). Patch 1 is further reworking of how DT_SCHEMA_FILES works and builds on [1].

Overall, the build time is about the same (still slow) though we do save a dtc call for dtbs_check. Extracting the type information is not cached in any way, so there's some opportunity for a slight optimization there.

Switching to DTB validation found various issues in the bindings and examples. Patches for those issues have already been posted and applied over the last month or so.

Rob

[1] https://lore.kernel.org/all/[email protected]/

Rob Herring (2):
dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate
dt-bindings: kbuild: Use DTB files for validation

 Documentation/devicetree/bindings/Makefile    | 35 ++++---------------
 .../devicetree/bindings/writing-schema.rst    | 12 -------
 scripts/Makefile.lib                          | 23 +++++-------
 scripts/dtc/Makefile                          | 13 -------
 scripts/dtc/update-dtc-source.sh              |  2 +-
 5 files changed, 15 insertions(+), 70 deletions(-)
Signature · 1 line
2.32.0
author maintainer
#1 lore raw

PATCH 1/2 dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate

In preparation for supporting validation of DTB files, the full processed schema will always be needed in order to extract type information from it. Therefore, the processed schema containing only what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has gained an option, -l or --limit, to specify which schema(s) to use for validation.

As the command line option is new, we the minimum dtschema version must be updated.

Cc Masahiro Yamada <[email protected]>
Signed-off-by Rob Herring <[email protected]>
---
 Documentation/devicetree/bindings/Makefile | 28 +++-------------------
 scripts/Makefile.lib                       |  3 +--
 2 files changed, 4 insertions(+), 27 deletions(-)
Documentation/devicetree/bindings/Makefile +3 -25
index 61ec18ecc931..246ba0ecab64 100644
@@ -6,7 +6,7 @@ DT_MK_SCHEMA ?= dt-mk-schema
 DT_SCHEMA_LINT := $(shell which yamllint || \
   echo "warning: yamllint not installed, skipping. To install, run 'pip install yamllint'" >&2)
 
-DT_SCHEMA_MIN_VERSION = 2021.2.1
+DT_SCHEMA_MIN_VERSION = 2022.3
 
 PHONY += check_dtschema_version
 check_dtschema_version:

@@ -25,9 +25,6 @@ quiet_cmd_extract_ex = DTEX    $@
 $(obj)/%.example.dts: $(src)/%.yaml check_dtschema_version FORCE
 	$(call if_changed,extract_ex)
 
-# Use full schemas when checking %.example.dts
-DT_TMP_SCHEMA := $(obj)/processed-schema-examples.json
-
 find_all_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \
 		-name 'processed-schema*' ! \
 		-name '*.example.dt.yaml' \)

@@ -70,29 +67,10 @@ override DTC_FLAGS := \
 # Disable undocumented compatible checks until warning free
 override DT_CHECKER_FLAGS ?=
 
-$(obj)/processed-schema-examples.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version FORCE
+$(obj)/processed-schema.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version FORCE
 	$(call if_changed_rule,chkdt)
 
-ifeq ($(DT_SCHEMA_FILES),)
-
-# Unless DT_SCHEMA_FILES is specified, use the full schema for dtbs_check too.
-# Just copy processed-schema-examples.json
-
-$(obj)/processed-schema.json: $(obj)/processed-schema-examples.json FORCE
-	$(call if_changed,copy)
-
-else
-
-# If DT_SCHEMA_FILES is specified, use it for processed-schema.json
-
-$(obj)/processed-schema.json: DT_MK_SCHEMA_FLAGS := -u
-$(obj)/processed-schema.json: $(CHK_DT_DOCS) check_dtschema_version FORCE
-	$(call if_changed,mk_schema)
-
-endif
-
-always-$(CHECK_DT_BINDING) += processed-schema-examples.json
-always-$(CHECK_DTBS)       += processed-schema.json
+always-y += processed-schema.json
 always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dts, $(CHK_DT_DOCS))
 always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dt.yaml, $(CHK_DT_DOCS))
scripts/Makefile.lib +1 -2
index 79be57fdd32a..9f1e8442564e 100644
@@ -361,9 +361,8 @@ $(multi-dtb-y): FORCE
 $(call multi_depend, $(multi-dtb-y), .dtb, -dtbs)
 
 DT_CHECKER ?= dt-validate
-DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),,-m)
+DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),-l $(DT_SCHEMA_FILES),-m)
 DT_BINDING_DIR := Documentation/devicetree/bindings
-# DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile
 DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.json
 
 quiet_cmd_dtb_check =	CHECK   $@
Signature · 1 line
2.32.0
#2 lore raw

Re: [PATCH 1/2] dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate

Hi Rob,

Quoted 7 lines · Rob Herring

In preparation for supporting validation of DTB files, the full processed schema will always be needed in order to extract type information from it. Therefore, the processed schema containing only what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has gained an option, -l or --limit, to specify which schema(s) to use for validation.

Quoted 5 lines

As the command line option is new, we the minimum dtschema version must be updated.

Cc Masahiro Yamada <[email protected]>
Signed-off-by Rob Herring <[email protected]>

Thanks for your patch!

Documentation/devicetree/bindings/Makefile DT_MK_SCHEMA ?= dt-mk-schema · line 9

Quoted 8 lines
Documentation/devicetree/bindings/Makefile +1 -1
@@ -6,7 +6,7 @@ DT_MK_SCHEMA ?= dt-mk-schema
 DT_SCHEMA_LINT := $(shell which yamllint || \
   echo "warning: yamllint not installed, skipping. To install, run 'pip install yamllint'" >&2)
 
-DT_SCHEMA_MIN_VERSION = 2021.2.1
+DT_SCHEMA_MIN_VERSION = 2022.3

This doesn't work as-is, as that version hasn't been tagged yet ;-)

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

author maintainer
#3 lore raw

Re: [PATCH 1/2] dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate

Quoted 24 lines · Geert Uytterhoeven

Hi Rob,

Quoted 7 lines · Rob Herring

In preparation for supporting validation of DTB files, the full processed schema will always be needed in order to extract type information from it. Therefore, the processed schema containing only what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has gained an option, -l or --limit, to specify which schema(s) to use for validation.

Quoted 5 lines

As the command line option is new, we the minimum dtschema version must be updated.

Cc Masahiro Yamada <[email protected]>
Signed-off-by Rob Herring <[email protected]>

Thanks for your patch!

Quoted 6 lines
Documentation/devicetree/bindings/Makefile +0 -0
@@ -6,7 +6,7 @@ DT_MK_SCHEMA ?= dt-mk-schema
 DT_SCHEMA_LINT := $(shell which yamllint || \
   echo "warning: yamllint not installed, skipping. To install, run 'pip install yamllint'" >&2)
Quoted 4 lines

Documentation/devicetree/bindings/Makefile DT_MK_SCHEMA ?= dt-mk-schema · line 9

Quoted 2 lines
Documentation/devicetree/bindings/Makefile +1 -1
-DT_SCHEMA_MIN_VERSION = 2021.2.1
+DT_SCHEMA_MIN_VERSION = 2022.3

This doesn't work as-is, as that version hasn't been tagged yet ;-)

I had to make sure people are paying attention. You win the prize. :)

It's there now.

Rob

#4 lore raw

Re: [PATCH 1/2] dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate

Hi Rob,

Quoted 23 lines · Rob Herring
Quoted 22 lines · Geert Uytterhoeven
Quoted 7 lines · Rob Herring

In preparation for supporting validation of DTB files, the full processed schema will always be needed in order to extract type information from it. Therefore, the processed schema containing only what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has gained an option, -l or --limit, to specify which schema(s) to use for validation.

Quoted 5 lines

As the command line option is new, we the minimum dtschema version must be updated.

Cc Masahiro Yamada <[email protected]>
Signed-off-by Rob Herring <[email protected]>

Thanks for your patch!

Quoted 6 lines
Documentation/devicetree/bindings/Makefile +0 -0
@@ -6,7 +6,7 @@ DT_MK_SCHEMA ?= dt-mk-schema
 DT_SCHEMA_LINT := $(shell which yamllint || \
   echo "warning: yamllint not installed, skipping. To install, run 'pip install yamllint'" >&2)
Quoted 8 lines
Quoted 4 lines

Documentation/devicetree/bindings/Makefile DT_MK_SCHEMA ?= dt-mk-schema · line 9

Quoted 2 lines
Documentation/devicetree/bindings/Makefile +1 -1
-DT_SCHEMA_MIN_VERSION = 2021.2.1
+DT_SCHEMA_MIN_VERSION = 2022.3

This doesn't work as-is, as that version hasn't been tagged yet ;-)

I had to make sure people are paying attention. You win the prize. :)

It's there now.

Thanks, confirmed.

With this series applied, the various salvator-xs DTS files are now throwing up:

sata: size (19) error for type phandle
backlight: size (11) error for type phandle

I tried to find out what caused that, but couldn't find it. Do you have a clue?
Thanks!

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

author maintainer
#5 lore raw

Re: [PATCH 1/2] dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate

Quoted 35 lines · Geert Uytterhoeven

Hi Rob,

Quoted 23 lines · Rob Herring
Quoted 22 lines · Geert Uytterhoeven
Quoted 7 lines · Rob Herring

In preparation for supporting validation of DTB files, the full processed schema will always be needed in order to extract type information from it. Therefore, the processed schema containing only what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has gained an option, -l or --limit, to specify which schema(s) to use for validation.

Quoted 5 lines

As the command line option is new, we the minimum dtschema version must be updated.

Cc Masahiro Yamada <[email protected]>
Signed-off-by Rob Herring <[email protected]>

Thanks for your patch!

Quoted 6 lines
Documentation/devicetree/bindings/Makefile +0 -0
@@ -6,7 +6,7 @@ DT_MK_SCHEMA ?= dt-mk-schema
 DT_SCHEMA_LINT := $(shell which yamllint || \
   echo "warning: yamllint not installed, skipping. To install, run 'pip install yamllint'" >&2)
Quoted 8 lines
Quoted 4 lines

Documentation/devicetree/bindings/Makefile DT_MK_SCHEMA ?= dt-mk-schema · line 9

Quoted 2 lines
Documentation/devicetree/bindings/Makefile +1 -1
-DT_SCHEMA_MIN_VERSION = 2021.2.1
+DT_SCHEMA_MIN_VERSION = 2022.3

This doesn't work as-is, as that version hasn't been tagged yet ;-)

I had to make sure people are paying attention. You win the prize. :)

It's there now.

Quoted 7 lines

Thanks, confirmed.

With this series applied, the various salvator-xs DTS files are now throwing up:

sata: size (19) error for type phandle
backlight: size (11) error for type phandle

Those come from the code decoding the properties[1]. Unfortunately, I haven't come up with a prettier way to report those with the filename. I may just remove it because if decoding the property fails, we'll get schema errors later on anyways.

But I don't see any 'sata' properties in the DTS files and 'backlight' is a node. Are you building with '-@'? I probably need to skip __symbols__ nodes. The overlay side is handled because examples are built as overlays (to allow unresolved phandles).

Rob

[1] https://github.com/devicetree-org/dt-schema/blob/main/dtschema/dtb.py#L149

#6 lore raw

Re: [PATCH 1/2] dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate

Hi Rob,

Quoted 28 lines · Rob Herring
Quoted 13 lines · Geert Uytterhoeven
Quoted 12 lines · Rob Herring
Quoted 11 lines · Geert Uytterhoeven
Quoted 10 lines · Rob Herring

In preparation for supporting validation of DTB files, the full processed schema will always be needed in order to extract type information from it. Therefore, the processed schema containing only what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has gained an option, -l or --limit, to specify which schema(s) to use for validation.

As the command line option is new, we the minimum dtschema version must be updated.

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Cc Masahiro Yamada <[email protected]>
Signed-off-by Rob Herring <[email protected]>

With this series applied, the various salvator-xs DTS files are now throwing up:

sata: size (19) error for type phandle
backlight: size (11) error for type phandle

Those come from the code decoding the properties[1]. Unfortunately, I haven't come up with a prettier way to report those with the filename. I may just remove it because if decoding the property fails, we'll get schema errors later on anyways.

Quoted 4 lines

But I don't see any 'sata' properties in the DTS files and 'backlight' is a node. Are you building with '-@'? I probably need to skip __symbols__ nodes. The overlay side is handled because examples are built as overlays (to allow unresolved phandles).

Yes I am. Dropping the "-@" fixes the issue.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

author maintainer
#7 lore raw

Re: [PATCH 1/2] dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate

Quoted 31 lines · Geert Uytterhoeven

Hi Rob,

Quoted 28 lines · Rob Herring
Quoted 13 lines · Geert Uytterhoeven
Quoted 12 lines · Rob Herring
Quoted 11 lines · Geert Uytterhoeven
Quoted 10 lines · Rob Herring

In preparation for supporting validation of DTB files, the full processed schema will always be needed in order to extract type information from it. Therefore, the processed schema containing only what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has gained an option, -l or --limit, to specify which schema(s) to use for validation.

As the command line option is new, we the minimum dtschema version must be updated.

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Cc Masahiro Yamada <[email protected]>
Signed-off-by Rob Herring <[email protected]>

With this series applied, the various salvator-xs DTS files are now throwing up:

sata: size (19) error for type phandle
backlight: size (11) error for type phandle

Those come from the code decoding the properties[1]. Unfortunately, I haven't come up with a prettier way to report those with the filename. I may just remove it because if decoding the property fails, we'll get schema errors later on anyways.

Quoted 6 lines
Quoted 4 lines

But I don't see any 'sata' properties in the DTS files and 'backlight' is a node. Are you building with '-@'? I probably need to skip __symbols__ nodes. The overlay side is handled because examples are built as overlays (to allow unresolved phandles).

Yes I am. Dropping the "-@" fixes the issue.

FYI, now fixed in v2022.03.1.

Rob

#8 lore raw

Re: [PATCH 1/2] dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate

In reply to Rob Herring
Quoted 77 lines · Rob Herring

In preparation for supporting validation of DTB files, the full processed schema will always be needed in order to extract type information from it. Therefore, the processed schema containing only what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has gained an option, -l or --limit, to specify which schema(s) to use for validation.

As the command line option is new, we the minimum dtschema version must be updated.

Cc Masahiro Yamada <[email protected]>
Signed-off-by Rob Herring <[email protected]>
---
 Documentation/devicetree/bindings/Makefile | 28 +++-------------------
 scripts/Makefile.lib                       |  3 +--
 2 files changed, 4 insertions(+), 27 deletions(-)
Documentation/devicetree/bindings/Makefile +3 -25
index 61ec18ecc931..246ba0ecab64 100644
@@ -6,7 +6,7 @@ DT_MK_SCHEMA ?= dt-mk-schema
 DT_SCHEMA_LINT := $(shell which yamllint || \
   echo "warning: yamllint not installed, skipping. To install, run 'pip install yamllint'" >&2)
 
-DT_SCHEMA_MIN_VERSION = 2021.2.1
+DT_SCHEMA_MIN_VERSION = 2022.3
 
 PHONY += check_dtschema_version
 check_dtschema_version:

@@ -25,9 +25,6 @@ quiet_cmd_extract_ex = DTEX    $@
 $(obj)/%.example.dts: $(src)/%.yaml check_dtschema_version FORCE
 	$(call if_changed,extract_ex)
 
-# Use full schemas when checking %.example.dts
-DT_TMP_SCHEMA := $(obj)/processed-schema-examples.json
-
 find_all_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \
 		-name 'processed-schema*' ! \
 		-name '*.example.dt.yaml' \)

@@ -70,29 +67,10 @@ override DTC_FLAGS := \
 # Disable undocumented compatible checks until warning free
 override DT_CHECKER_FLAGS ?=
 
-$(obj)/processed-schema-examples.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version FORCE
+$(obj)/processed-schema.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version FORCE
 	$(call if_changed_rule,chkdt)
 
-ifeq ($(DT_SCHEMA_FILES),)
-
-# Unless DT_SCHEMA_FILES is specified, use the full schema for dtbs_check too.
-# Just copy processed-schema-examples.json
-
-$(obj)/processed-schema.json: $(obj)/processed-schema-examples.json FORCE
-	$(call if_changed,copy)
-
-else
-
-# If DT_SCHEMA_FILES is specified, use it for processed-schema.json
-
-$(obj)/processed-schema.json: DT_MK_SCHEMA_FLAGS := -u
-$(obj)/processed-schema.json: $(CHK_DT_DOCS) check_dtschema_version FORCE
-	$(call if_changed,mk_schema)
-
-endif
-
-always-$(CHECK_DT_BINDING) += processed-schema-examples.json
-always-$(CHECK_DTBS)       += processed-schema.json
+always-y += processed-schema.json
 always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dts, $(CHK_DT_DOCS))
 always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dt.yaml, $(CHK_DT_DOCS))
scripts/Makefile.lib +0 -0
index 79be57fdd32a..9f1e8442564e 100644

scripts/Makefile.lib $(multi-dtb-y): FORCE · line 366

Quoted 8 lines
scripts/Makefile.lib +1 -2
@@ -361,9 +361,8 @@ $(multi-dtb-y): FORCE
 $(call multi_depend, $(multi-dtb-y), .dtb, -dtbs)
 
 DT_CHECKER ?= dt-validate
-DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),,-m)
+DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),-l $(DT_SCHEMA_FILES),-m)
 DT_BINDING_DIR := Documentation/devicetree/bindings
-# DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile
 DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.json

This could now use := instead of ?=

Apart from the fact that 2022.3 hasn't been tagged yet as pointed out by Geert, this looks fine to me (but I'm no expert in this area).

Quoted 2 lines
scripts/Makefile.lib +0 -0
 quiet_cmd_dtb_check =	CHECK   $@
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#9 lore raw

Re: [PATCH 1/2] dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate

Quoted 80 lines · Laurent Pinchart
Quoted 79 lines · Rob Herring

In preparation for supporting validation of DTB files, the full processed schema will always be needed in order to extract type information from it. Therefore, the processed schema containing only what DT_SCHEMA_FILES specifies won't work. Instead, dt-validate has gained an option, -l or --limit, to specify which schema(s) to use for validation.

As the command line option is new, we the minimum dtschema version must be updated.

Cc Masahiro Yamada <[email protected]>
Signed-off-by Rob Herring <[email protected]>
---
 Documentation/devicetree/bindings/Makefile | 28 +++-------------------
 scripts/Makefile.lib                       |  3 +--
 2 files changed, 4 insertions(+), 27 deletions(-)
Documentation/devicetree/bindings/Makefile +3 -25
index 61ec18ecc931..246ba0ecab64 100644
@@ -6,7 +6,7 @@ DT_MK_SCHEMA ?= dt-mk-schema
 DT_SCHEMA_LINT := $(shell which yamllint || \
   echo "warning: yamllint not installed, skipping. To install, run 'pip install yamllint'" >&2)
 
-DT_SCHEMA_MIN_VERSION = 2021.2.1
+DT_SCHEMA_MIN_VERSION = 2022.3
 
 PHONY += check_dtschema_version
 check_dtschema_version:

@@ -25,9 +25,6 @@ quiet_cmd_extract_ex = DTEX    $@
 $(obj)/%.example.dts: $(src)/%.yaml check_dtschema_version FORCE
 	$(call if_changed,extract_ex)
 
-# Use full schemas when checking %.example.dts
-DT_TMP_SCHEMA := $(obj)/processed-schema-examples.json
-
 find_all_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \
 		-name 'processed-schema*' ! \
 		-name '*.example.dt.yaml' \)

@@ -70,29 +67,10 @@ override DTC_FLAGS := \
 # Disable undocumented compatible checks until warning free
 override DT_CHECKER_FLAGS ?=
 
-$(obj)/processed-schema-examples.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version FORCE
+$(obj)/processed-schema.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version FORCE
 	$(call if_changed_rule,chkdt)
 
-ifeq ($(DT_SCHEMA_FILES),)
-
-# Unless DT_SCHEMA_FILES is specified, use the full schema for dtbs_check too.
-# Just copy processed-schema-examples.json
-
-$(obj)/processed-schema.json: $(obj)/processed-schema-examples.json FORCE
-	$(call if_changed,copy)
-
-else
-
-# If DT_SCHEMA_FILES is specified, use it for processed-schema.json
-
-$(obj)/processed-schema.json: DT_MK_SCHEMA_FLAGS := -u
-$(obj)/processed-schema.json: $(CHK_DT_DOCS) check_dtschema_version FORCE
-	$(call if_changed,mk_schema)
-
-endif
-
-always-$(CHECK_DT_BINDING) += processed-schema-examples.json
-always-$(CHECK_DTBS)       += processed-schema.json
+always-y += processed-schema.json
 always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dts, $(CHK_DT_DOCS))
 always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dt.yaml, $(CHK_DT_DOCS))
scripts/Makefile.lib +0 -0
index 79be57fdd32a..9f1e8442564e 100644
@@ -361,9 +361,8 @@ $(multi-dtb-y): FORCE
 $(call multi_depend, $(multi-dtb-y), .dtb, -dtbs)
Quoted 8 lines

scripts/Makefile.lib $(multi-dtb-y): FORCE · line 366

Quoted 6 lines
scripts/Makefile.lib +1 -2
 DT_CHECKER ?= dt-validate
-DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),,-m)
+DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),-l $(DT_SCHEMA_FILES),-m)
 DT_BINDING_DIR := Documentation/devicetree/bindings
-# DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile
 DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.json

This could now use := instead of ?=

Yes, though I think it is possible we may still want to override it. Other than debugging perhaps, I don't have an immediate reason.

Quoted 2 lines

Apart from the fact that 2022.3 hasn't been tagged yet as pointed out by Geert, this looks fine to me (but I'm no expert in this area).

It's now there.

Rob

author maintainer
#10 lore raw

PATCH 2/2 dt-bindings: kbuild: Use DTB files for validation

In reply to Rob Herring

Switch the DT validation to use DTB files directly instead of a DTS to YAML conversion.

The original motivation for supporting validation on DTB files was to enable running validation on a running system (e.g. 'dt-validate /sys/firmware/fdt') or other cases where the original source DTS is not available.

The YAML format was not without issues. Using DTBs with the schema type information solves some of those problems. The YAML format relies on the DTS source level information including bracketing of properties, size directives, and phandle tags all of which are lost in a DTB file. While standardizing the bracketing is a good thing, it does cause a lot of extra warnings and churn to fix them.

Another issue has been signed types are not validated correctly as sign information is not propagated to YAML. Using the schema type information allows for proper handling of signed types. YAML also can't represent the full range of 64-bit integers as numbers are stored as floats by most/all parsers.

The DTB validation works by decoding property values using the type information in the schemas themselves. The main corner case this does not work for is matrix types where neither dimension is fixed. For now, checking the dimensions in these cases are skipped.

Signed-off-by Rob Herring <[email protected]>
---
 Documentation/devicetree/bindings/Makefile    |  7 +++----
 .../devicetree/bindings/writing-schema.rst    | 12 -----------
 scripts/Makefile.lib                          | 20 +++++++------------
 scripts/dtc/Makefile                          | 13 ------------
 scripts/dtc/update-dtc-source.sh              |  2 +-
 5 files changed, 11 insertions(+), 43 deletions(-)
Documentation/devicetree/bindings/Makefile +3 -4
index 246ba0ecab64..b8bd6a8ec43a 100644
@@ -26,8 +26,7 @@ $(obj)/%.example.dts: $(src)/%.yaml check_dtschema_version FORCE
 	$(call if_changed,extract_ex)
 
 find_all_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \
-		-name 'processed-schema*' ! \
-		-name '*.example.dt.yaml' \)
+		-name 'processed-schema*' \)
 
 find_cmd = $(find_all_cmd) | grep -F "$(DT_SCHEMA_FILES)"
 CHK_DT_DOCS := $(shell $(find_cmd))

@@ -72,9 +71,9 @@ $(obj)/processed-schema.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version
 
 always-y += processed-schema.json
 always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dts, $(CHK_DT_DOCS))
-always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dt.yaml, $(CHK_DT_DOCS))
+always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dtb, $(CHK_DT_DOCS))
 
 # Hack: avoid 'Argument list too long' error for 'make clean'. Remove most of
 # build artifacts here before they are processed by scripts/Makefile.clean
 clean-files = $(shell find $(obj) \( -name '*.example.dts' -o \
-			-name '*.example.dt.yaml' \) -delete 2>/dev/null)
+			-name '*.example.dtb' \) -delete 2>/dev/null)
Documentation/devicetree/bindings/writing-schema.rst +0 -12
index 3b00fe981494..b4258bf81be5 100644
@@ -123,18 +123,6 @@ project can be installed with pip::
 Several executables (dt-doc-validate, dt-mk-schema, dt-validate) will be
 installed. Ensure they are in your PATH (~/.local/bin by default).
 
-dtc must also be built with YAML output support enabled. This requires that
-libyaml and its headers be installed on the host system. For some distributions
-that involves installing the development package, such as:
-
-Debian::
-
-  apt-get install libyaml-dev
-
-Fedora::
-
-  dnf -y install libyaml-devel
-
 Running checks
 ~~~~~~~~~~~~~~
scripts/Makefile.lib +7 -13
index 9f1e8442564e..4629af60160b 100644
@@ -87,11 +87,6 @@ base-dtb-y := $(foreach m, $(multi-dtb-y), $(firstword $(call suffix-search, $m,
 
 always-y			+= $(dtb-y)
 
-ifneq ($(CHECK_DTBS),)
-always-y += $(patsubst %.dtb,%.dt.yaml, $(real-dtb-y))
-always-y += $(patsubst %.dtbo,%.dt.yaml, $(real-dtb-y))
-endif
-
 # Add subdir path
 
 extra-y		:= $(addprefix $(obj)/,$(extra-y))

@@ -347,12 +342,6 @@ cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ;
 		-d $(depfile).dtc.tmp $(dtc-tmp) ; \
 	cat $(depfile).pre.tmp $(depfile).dtc.tmp > $(depfile)
 
-$(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
-	$(call if_changed_dep,dtc)
-
-$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
-	$(call if_changed_dep,dtc)
-
 quiet_cmd_fdtoverlay = DTOVL   $@
       cmd_fdtoverlay = $(objtree)/scripts/dtc/fdtoverlay -o $@ -i $(real-prereqs)
 

@@ -365,17 +354,22 @@ DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),-l $(DT_SCHEMA_FILES),-m)
 DT_BINDING_DIR := Documentation/devicetree/bindings
 DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.json
 
+ifneq ($(CHECK_DTBS)$(CHECK_DT_BINDING),)
 quiet_cmd_dtb_check =	CHECK   $@
-      cmd_dtb_check =	$(DT_CHECKER) $(DT_CHECKER_FLAGS) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@
+      cmd_dtb_check =	$(DT_CHECKER) $(DT_CHECKER_FLAGS) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@ || true
+endif
 
 define rule_dtc
 	$(call cmd_and_fixdep,dtc)
 	$(call cmd,dtb_check)
 endef
 
-$(obj)/%.dt.yaml: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
+$(obj)/%.dtb: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
 	$(call if_changed_rule,dtc)
 
+$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
+	$(call if_changed_dep,dtc)
+
 dtc-tmp = $(subst $(comma),_,$(dot-target).dts.tmp)
 
 # Bzip2
scripts/dtc/Makefile +0 -13
index 1cba78e1dce6..4d32b9497da9 100644
@@ -17,20 +17,7 @@ fdtoverlay-objs	:= $(libfdt) fdtoverlay.o util.o
 
 # Source files need to get at the userspace version of libfdt_env.h to compile
 HOST_EXTRACFLAGS += -I $(srctree)/$(src)/libfdt
-
-ifeq ($(shell pkg-config --exists yaml-0.1 2>/dev/null && echo yes),)
-ifneq ($(CHECK_DT_BINDING)$(CHECK_DTBS),)
-$(error dtc needs libyaml for DT schema validation support. \
-	Install the necessary libyaml development package.)
-endif
 HOST_EXTRACFLAGS += -DNO_YAML
-else
-dtc-objs	+= yamltree.o
-# To include <yaml.h> installed in a non-default path
-HOSTCFLAGS_yamltree.o := $(shell pkg-config --cflags yaml-0.1)
-# To link libyaml installed in a non-default path
-HOSTLDLIBS_dtc	:= $(shell pkg-config --libs yaml-0.1)
-endif
 
 # Generated files need one more search path to include headers in source tree
 HOSTCFLAGS_dtc-lexer.lex.o := -I $(srctree)/$(src)
scripts/dtc/update-dtc-source.sh +1 -1
index 32ff17ffd089..94627541533e 100755
@@ -32,7 +32,7 @@ DTC_UPSTREAM_PATH=`pwd`/../dtc
 DTC_LINUX_PATH=`pwd`/scripts/dtc
 
 DTC_SOURCE="checks.c data.c dtc.c dtc.h flattree.c fstree.c livetree.c srcpos.c \
-		srcpos.h treesource.c util.c util.h version_gen.h yamltree.c \
+		srcpos.h treesource.c util.c util.h version_gen.h \
 		dtc-lexer.l dtc-parser.y"
 LIBFDT_SOURCE="fdt.c fdt.h fdt_addresses.c fdt_empty_tree.c \
 		fdt_overlay.c fdt_ro.c fdt_rw.c fdt_strerror.c fdt_sw.c \
Signature · 1 line
2.32.0
#11 lore raw

Re: [PATCH 2/2] dt-bindings: kbuild: Use DTB files for validation

Hi Rob,

Thank you for the patch.

Quoted 8 lines · Rob Herring

Switch the DT validation to use DTB files directly instead of a DTS to YAML conversion.

The original motivation for supporting validation on DTB files was to enable running validation on a running system (e.g. 'dt-validate /sys/firmware/fdt') or other cases where the original source DTS is not available.

Quoted 6 lines

The YAML format was not without issues. Using DTBs with the schema type information solves some of those problems. The YAML format relies on the DTS source level information including bracketing of properties, size directives, and phandle tags all of which are lost in a DTB file. While standardizing the bracketing is a good thing, it does cause a lot of extra warnings and churn to fix them.

It does indeed, but it's a bit sad to let that feature go :-S

It's nice to drop the dependency on libyaml though, and the ability to validate a DTB is nice too, so I think it's worth it.

Quoted 161 lines

Another issue has been signed types are not validated correctly as sign information is not propagated to YAML. Using the schema type information allows for proper handling of signed types. YAML also can't represent the full range of 64-bit integers as numbers are stored as floats by most/all parsers.

The DTB validation works by decoding property values using the type information in the schemas themselves. The main corner case this does not work for is matrix types where neither dimension is fixed. For now, checking the dimensions in these cases are skipped.

Signed-off-by Rob Herring <[email protected]>
---
 Documentation/devicetree/bindings/Makefile    |  7 +++----
 .../devicetree/bindings/writing-schema.rst    | 12 -----------
 scripts/Makefile.lib                          | 20 +++++++------------
 scripts/dtc/Makefile                          | 13 ------------
 scripts/dtc/update-dtc-source.sh              |  2 +-
 5 files changed, 11 insertions(+), 43 deletions(-)
Documentation/devicetree/bindings/Makefile +3 -4
index 246ba0ecab64..b8bd6a8ec43a 100644
@@ -26,8 +26,7 @@ $(obj)/%.example.dts: $(src)/%.yaml check_dtschema_version FORCE
 	$(call if_changed,extract_ex)
 
 find_all_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \
-		-name 'processed-schema*' ! \
-		-name '*.example.dt.yaml' \)
+		-name 'processed-schema*' \)
 
 find_cmd = $(find_all_cmd) | grep -F "$(DT_SCHEMA_FILES)"
 CHK_DT_DOCS := $(shell $(find_cmd))

@@ -72,9 +71,9 @@ $(obj)/processed-schema.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version
 
 always-y += processed-schema.json
 always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dts, $(CHK_DT_DOCS))
-always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dt.yaml, $(CHK_DT_DOCS))
+always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dtb, $(CHK_DT_DOCS))
 
 # Hack: avoid 'Argument list too long' error for 'make clean'. Remove most of
 # build artifacts here before they are processed by scripts/Makefile.clean
 clean-files = $(shell find $(obj) \( -name '*.example.dts' -o \
-			-name '*.example.dt.yaml' \) -delete 2>/dev/null)
+			-name '*.example.dtb' \) -delete 2>/dev/null)
Documentation/devicetree/bindings/writing-schema.rst +0 -12
index 3b00fe981494..b4258bf81be5 100644
@@ -123,18 +123,6 @@ project can be installed with pip::
 Several executables (dt-doc-validate, dt-mk-schema, dt-validate) will be
 installed. Ensure they are in your PATH (~/.local/bin by default).
 
-dtc must also be built with YAML output support enabled. This requires that
-libyaml and its headers be installed on the host system. For some distributions
-that involves installing the development package, such as:
-
-Debian::
-
-  apt-get install libyaml-dev
-
-Fedora::
-
-  dnf -y install libyaml-devel
-
 Running checks
 ~~~~~~~~~~~~~~
scripts/Makefile.lib +7 -13
index 9f1e8442564e..4629af60160b 100644
@@ -87,11 +87,6 @@ base-dtb-y := $(foreach m, $(multi-dtb-y), $(firstword $(call suffix-search, $m,
 
 always-y			+= $(dtb-y)
 
-ifneq ($(CHECK_DTBS),)
-always-y += $(patsubst %.dtb,%.dt.yaml, $(real-dtb-y))
-always-y += $(patsubst %.dtbo,%.dt.yaml, $(real-dtb-y))
-endif
-
 # Add subdir path
 
 extra-y		:= $(addprefix $(obj)/,$(extra-y))

@@ -347,12 +342,6 @@ cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ;
 		-d $(depfile).dtc.tmp $(dtc-tmp) ; \
 	cat $(depfile).pre.tmp $(depfile).dtc.tmp > $(depfile)
 
-$(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
-	$(call if_changed_dep,dtc)
-
-$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
-	$(call if_changed_dep,dtc)
-
 quiet_cmd_fdtoverlay = DTOVL   $@
       cmd_fdtoverlay = $(objtree)/scripts/dtc/fdtoverlay -o $@ -i $(real-prereqs)
 

@@ -365,17 +354,22 @@ DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),-l $(DT_SCHEMA_FILES),-m)
 DT_BINDING_DIR := Documentation/devicetree/bindings
 DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.json
 
+ifneq ($(CHECK_DTBS)$(CHECK_DT_BINDING),)
 quiet_cmd_dtb_check =	CHECK   $@
-      cmd_dtb_check =	$(DT_CHECKER) $(DT_CHECKER_FLAGS) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@
+      cmd_dtb_check =	$(DT_CHECKER) $(DT_CHECKER_FLAGS) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@ || true
+endif
 
 define rule_dtc
 	$(call cmd_and_fixdep,dtc)
 	$(call cmd,dtb_check)
 endef
 
-$(obj)/%.dt.yaml: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
+$(obj)/%.dtb: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
 	$(call if_changed_rule,dtc)
 
+$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
+	$(call if_changed_dep,dtc)
+
 dtc-tmp = $(subst $(comma),_,$(dot-target).dts.tmp)
 
 # Bzip2
scripts/dtc/Makefile +0 -13
index 1cba78e1dce6..4d32b9497da9 100644
@@ -17,20 +17,7 @@ fdtoverlay-objs	:= $(libfdt) fdtoverlay.o util.o
 
 # Source files need to get at the userspace version of libfdt_env.h to compile
 HOST_EXTRACFLAGS += -I $(srctree)/$(src)/libfdt
-
-ifeq ($(shell pkg-config --exists yaml-0.1 2>/dev/null && echo yes),)
-ifneq ($(CHECK_DT_BINDING)$(CHECK_DTBS),)
-$(error dtc needs libyaml for DT schema validation support. \
-	Install the necessary libyaml development package.)
-endif
 HOST_EXTRACFLAGS += -DNO_YAML
-else
-dtc-objs	+= yamltree.o
-# To include <yaml.h> installed in a non-default path
-HOSTCFLAGS_yamltree.o := $(shell pkg-config --cflags yaml-0.1)
-# To link libyaml installed in a non-default path
-HOSTLDLIBS_dtc	:= $(shell pkg-config --libs yaml-0.1)
-endif
 
 # Generated files need one more search path to include headers in source tree
 HOSTCFLAGS_dtc-lexer.lex.o := -I $(srctree)/$(src)
scripts/dtc/update-dtc-source.sh +1 -1
index 32ff17ffd089..94627541533e 100755
@@ -32,7 +32,7 @@ DTC_UPSTREAM_PATH=`pwd`/../dtc
 DTC_LINUX_PATH=`pwd`/scripts/dtc
 
 DTC_SOURCE="checks.c data.c dtc.c dtc.h flattree.c fstree.c livetree.c srcpos.c \
-		srcpos.h treesource.c util.c util.h version_gen.h yamltree.c \
+		srcpos.h treesource.c util.c util.h version_gen.h \
 		dtc-lexer.l dtc-parser.y"
 LIBFDT_SOURCE="fdt.c fdt.h fdt_addresses.c fdt_empty_tree.c \
 		fdt_overlay.c fdt_ro.c fdt_rw.c fdt_strerror.c fdt_sw.c \
Signature · 3 lines
Regards,

Laurent Pinchart
author maintainer
#12 lore raw

Re: [PATCH 2/2] dt-bindings: kbuild: Use DTB files for validation

Quoted 13 lines · Laurent Pinchart

Hi Rob,

Thank you for the patch.

Quoted 8 lines · Rob Herring

Switch the DT validation to use DTB files directly instead of a DTS to YAML conversion.

The original motivation for supporting validation on DTB files was to enable running validation on a running system (e.g. 'dt-validate /sys/firmware/fdt') or other cases where the original source DTS is not available.

Quoted 8 lines
Quoted 6 lines

The YAML format was not without issues. Using DTBs with the schema type information solves some of those problems. The YAML format relies on the DTS source level information including bracketing of properties, size directives, and phandle tags all of which are lost in a DTB file. While standardizing the bracketing is a good thing, it does cause a lot of extra warnings and churn to fix them.

It does indeed, but it's a bit sad to let that feature go :-S

I agree, but I think that checking is better served with a DTS linter.

Rob

#13 lore raw

Re: [PATCH 2/2] dt-bindings: kbuild: Use DTB files for validation

In reply to Rob Herring

Hi Rob,

Quoted 21 lines · Rob Herring

Switch the DT validation to use DTB files directly instead of a DTS to YAML conversion.

The original motivation for supporting validation on DTB files was to enable running validation on a running system (e.g. 'dt-validate /sys/firmware/fdt') or other cases where the original source DTS is not available.

The YAML format was not without issues. Using DTBs with the schema type information solves some of those problems. The YAML format relies on the DTS source level information including bracketing of properties, size directives, and phandle tags all of which are lost in a DTB file. While standardizing the bracketing is a good thing, it does cause a lot of extra warnings and churn to fix them.

Another issue has been signed types are not validated correctly as sign information is not propagated to YAML. Using the schema type information allows for proper handling of signed types. YAML also can't represent the full range of 64-bit integers as numbers are stored as floats by most/all parsers.

Quoted 6 lines

The DTB validation works by decoding property values using the type information in the schemas themselves. The main corner case this does not work for is matrix types where neither dimension is fixed. For now, checking the dimensions in these cases are skipped.

Signed-off-by Rob Herring <[email protected]>

Thanks for your patch!

While investigating why a newly added device node to DTS was not instantiated as a platform device, I discovered an issue with this
patch: "make dtbs" no longer rebuilds DTB files that need a rebuild.

How to reproduce:

    $ git checkout next-20220307
    # apply this series and its dependency:
    # dt-bindings: kbuild: Support partial matches with DT_SCHEMA_FILES
    # dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate
    # dt-bindings: kbuild: Use DTB files for validation
    $ make ARCH=arm shmobile_defconfig
    $ make ARCH=arm dtbs
    $ touch arch/arm/boot/dts/r8a7791.dtsi
    $ make ARCH=arm dtbs
    # The above command does NOT cause:
    # DTC     arch/arm/boot/dts/r8a7791-koelsch.dtb
    # DTC     arch/arm/boot/dts/r8a7791-porter.dtb

I don't see anything wrong with this patch at first sight, though.

Quoted 140 lines
Documentation/devicetree/bindings/Makefile +3 -4
@@ -26,8 +26,7 @@ $(obj)/%.example.dts: $(src)/%.yaml check_dtschema_version FORCE
        $(call if_changed,extract_ex)
 
 find_all_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \
-               -name 'processed-schema*' ! \
-               -name '*.example.dt.yaml' \)
+               -name 'processed-schema*' \)
 
 find_cmd = $(find_all_cmd) | grep -F "$(DT_SCHEMA_FILES)"
 CHK_DT_DOCS := $(shell $(find_cmd))

@@ -72,9 +71,9 @@ $(obj)/processed-schema.json: $(DT_DOCS) $(src)/.yamllint check_dtschema_version
 
 always-y += processed-schema.json
 always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dts, $(CHK_DT_DOCS))
-always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dt.yaml, $(CHK_DT_DOCS))
+always-$(CHECK_DT_BINDING) += $(patsubst $(srctree)/$(src)/%.yaml,%.example.dtb, $(CHK_DT_DOCS))
 
 # Hack: avoid 'Argument list too long' error for 'make clean'. Remove most of
 # build artifacts here before they are processed by scripts/Makefile.clean
 clean-files = $(shell find $(obj) \( -name '*.example.dts' -o \
-                       -name '*.example.dt.yaml' \) -delete 2>/dev/null)
+                       -name '*.example.dtb' \) -delete 2>/dev/null)
Documentation/devicetree/bindings/writing-schema.rst +0 -12
index 3b00fe981494..b4258bf81be5 100644
@@ -123,18 +123,6 @@ project can be installed with pip::
 Several executables (dt-doc-validate, dt-mk-schema, dt-validate) will be
 installed. Ensure they are in your PATH (~/.local/bin by default).
 
-dtc must also be built with YAML output support enabled. This requires that
-libyaml and its headers be installed on the host system. For some distributions
-that involves installing the development package, such as:
-
-Debian::
-
-  apt-get install libyaml-dev
-
-Fedora::
-
-  dnf -y install libyaml-devel
-
 Running checks
 ~~~~~~~~~~~~~~
scripts/Makefile.lib +7 -13
index 9f1e8442564e..4629af60160b 100644
@@ -87,11 +87,6 @@ base-dtb-y := $(foreach m, $(multi-dtb-y), $(firstword $(call suffix-search, $m,
 
 always-y                       += $(dtb-y)
 
-ifneq ($(CHECK_DTBS),)
-always-y += $(patsubst %.dtb,%.dt.yaml, $(real-dtb-y))
-always-y += $(patsubst %.dtbo,%.dt.yaml, $(real-dtb-y))
-endif
-
 # Add subdir path
 
 extra-y                := $(addprefix $(obj)/,$(extra-y))

@@ -347,12 +342,6 @@ cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ;
                -d $(depfile).dtc.tmp $(dtc-tmp) ; \
        cat $(depfile).pre.tmp $(depfile).dtc.tmp > $(depfile)
 
-$(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
-       $(call if_changed_dep,dtc)
-
-$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
-       $(call if_changed_dep,dtc)
-
 quiet_cmd_fdtoverlay = DTOVL   $@
       cmd_fdtoverlay = $(objtree)/scripts/dtc/fdtoverlay -o $@ -i $(real-prereqs)
 

@@ -365,17 +354,22 @@ DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),-l $(DT_SCHEMA_FILES),-m)
 DT_BINDING_DIR := Documentation/devicetree/bindings
 DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.json
 
+ifneq ($(CHECK_DTBS)$(CHECK_DT_BINDING),)
 quiet_cmd_dtb_check =  CHECK   $@
-      cmd_dtb_check =  $(DT_CHECKER) $(DT_CHECKER_FLAGS) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@
+      cmd_dtb_check =  $(DT_CHECKER) $(DT_CHECKER_FLAGS) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@ || true
+endif
 
 define rule_dtc
        $(call cmd_and_fixdep,dtc)
        $(call cmd,dtb_check)
 endef
 
-$(obj)/%.dt.yaml: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
+$(obj)/%.dtb: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
        $(call if_changed_rule,dtc)
 
+$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
+       $(call if_changed_dep,dtc)
+
 dtc-tmp = $(subst $(comma),_,$(dot-target).dts.tmp)
 
 # Bzip2
scripts/dtc/Makefile +0 -13
index 1cba78e1dce6..4d32b9497da9 100644
@@ -17,20 +17,7 @@ fdtoverlay-objs      := $(libfdt) fdtoverlay.o util.o
 
 # Source files need to get at the userspace version of libfdt_env.h to compile
 HOST_EXTRACFLAGS += -I $(srctree)/$(src)/libfdt
-
-ifeq ($(shell pkg-config --exists yaml-0.1 2>/dev/null && echo yes),)
-ifneq ($(CHECK_DT_BINDING)$(CHECK_DTBS),)
-$(error dtc needs libyaml for DT schema validation support. \
-       Install the necessary libyaml development package.)
-endif
 HOST_EXTRACFLAGS += -DNO_YAML
-else
-dtc-objs       += yamltree.o
-# To include <yaml.h> installed in a non-default path
-HOSTCFLAGS_yamltree.o := $(shell pkg-config --cflags yaml-0.1)
-# To link libyaml installed in a non-default path
-HOSTLDLIBS_dtc := $(shell pkg-config --libs yaml-0.1)
-endif
 
 # Generated files need one more search path to include headers in source tree
 HOSTCFLAGS_dtc-lexer.lex.o := -I $(srctree)/$(src)
scripts/dtc/update-dtc-source.sh +1 -2
index 32ff17ffd089..94627541533e 100755
@@ -32,7 +32,7 @@ DTC_UPSTREAM_PATH=`pwd`/../dtc
 DTC_LINUX_PATH=`pwd`/scripts/dtc
 
 DTC_SOURCE="checks.c data.c dtc.c dtc.h flattree.c fstree.c livetree.c srcpos.c \
-               srcpos.h treesource.c util.c util.h version_gen.h yamltree.c \
+               srcpos.h treesource.c util.c util.h version_gen.h \
                dtc-lexer.l dtc-parser.y"
 LIBFDT_SOURCE="fdt.c fdt.h fdt_addresses.c fdt_empty_tree.c \
                fdt_overlay.c fdt_ro.c fdt_rw.c fdt_strerror.c fdt_sw.c \
--
Quoted 1 line

2.32.0

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

author maintainer
#14 lore raw

Re: [PATCH 2/2] dt-bindings: kbuild: Use DTB files for validation

Quoted 52 lines · Geert Uytterhoeven

Hi Rob,

Quoted 21 lines · Rob Herring

Switch the DT validation to use DTB files directly instead of a DTS to YAML conversion.

The original motivation for supporting validation on DTB files was to enable running validation on a running system (e.g. 'dt-validate /sys/firmware/fdt') or other cases where the original source DTS is not available.

The YAML format was not without issues. Using DTBs with the schema type information solves some of those problems. The YAML format relies on the DTS source level information including bracketing of properties, size directives, and phandle tags all of which are lost in a DTB file. While standardizing the bracketing is a good thing, it does cause a lot of extra warnings and churn to fix them.

Another issue has been signed types are not validated correctly as sign information is not propagated to YAML. Using the schema type information allows for proper handling of signed types. YAML also can't represent the full range of 64-bit integers as numbers are stored as floats by most/all parsers.

Quoted 6 lines

The DTB validation works by decoding property values using the type information in the schemas themselves. The main corner case this does not work for is matrix types where neither dimension is fixed. For now, checking the dimensions in these cases are skipped.

Signed-off-by Rob Herring <[email protected]>

Thanks for your patch!

While investigating why a newly added device node to DTS was not instantiated as a platform device, I discovered an issue with this
patch: "make dtbs" no longer rebuilds DTB files that need a rebuild.

How to reproduce:

    $ git checkout next-20220307
    # apply this series and its dependency:
    # dt-bindings: kbuild: Support partial matches with DT_SCHEMA_FILES
    # dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate
    # dt-bindings: kbuild: Use DTB files for validation
    $ make ARCH=arm shmobile_defconfig
    $ make ARCH=arm dtbs
    $ touch arch/arm/boot/dts/r8a7791.dtsi
    $ make ARCH=arm dtbs
    # The above command does NOT cause:
    # DTC     arch/arm/boot/dts/r8a7791-koelsch.dtb
    # DTC     arch/arm/boot/dts/r8a7791-porter.dtb
Quoted 1 line

I don't see anything wrong with this patch at first sight, though.

Was this a clean tree?

I think I reproduced it, but then couldn't... But then after a 'make clean', 'make dtbs' would error out. I think the issue in both cases was processed-schema.json always a dependency when it should be conditional on 'dtbs_check'. The patch below fixes that. Can you give it a try too.

scripts/Makefile.lib +2 -2
index 4629af60160b..9d5320a47ef8 100644
@@ -349,12 +349,12 @@ $(multi-dtb-y): FORCE
        $(call if_changed,fdtoverlay)
 $(call multi_depend, $(multi-dtb-y), .dtb, -dtbs)
 
+ifneq ($(CHECK_DTBS)$(CHECK_DT_BINDING),)
 DT_CHECKER ?= dt-validate
 DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),-l $(DT_SCHEMA_FILES),-m)
 DT_BINDING_DIR := Documentation/devicetree/bindings
-DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.json
+DT_TMP_SCHEMA := $(objtree)/$(DT_BINDING_DIR)/processed-schema.json
 
-ifneq ($(CHECK_DTBS)$(CHECK_DT_BINDING),)
 quiet_cmd_dtb_check =  CHECK   $@
       cmd_dtb_check =  $(DT_CHECKER) $(DT_CHECKER_FLAGS) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@ || true
 endif
Tested-by
#15 lore raw

Re: [PATCH 2/2] dt-bindings: kbuild: Use DTB files for validation

Hi Rob,

Quoted 51 lines · Rob Herring
Quoted 50 lines · Geert Uytterhoeven
Quoted 21 lines · Rob Herring

Switch the DT validation to use DTB files directly instead of a DTS to YAML conversion.

The original motivation for supporting validation on DTB files was to enable running validation on a running system (e.g. 'dt-validate /sys/firmware/fdt') or other cases where the original source DTS is not available.

The YAML format was not without issues. Using DTBs with the schema type information solves some of those problems. The YAML format relies on the DTS source level information including bracketing of properties, size directives, and phandle tags all of which are lost in a DTB file. While standardizing the bracketing is a good thing, it does cause a lot of extra warnings and churn to fix them.

Another issue has been signed types are not validated correctly as sign information is not propagated to YAML. Using the schema type information allows for proper handling of signed types. YAML also can't represent the full range of 64-bit integers as numbers are stored as floats by most/all parsers.

Quoted 6 lines

The DTB validation works by decoding property values using the type information in the schemas themselves. The main corner case this does not work for is matrix types where neither dimension is fixed. For now, checking the dimensions in these cases are skipped.

Signed-off-by Rob Herring <[email protected]>

Thanks for your patch!

While investigating why a newly added device node to DTS was not instantiated as a platform device, I discovered an issue with this
patch: "make dtbs" no longer rebuilds DTB files that need a rebuild.

How to reproduce:

    $ git checkout next-20220307
    # apply this series and its dependency:
    # dt-bindings: kbuild: Support partial matches with DT_SCHEMA_FILES
    # dt-bindings: kbuild: Pass DT_SCHEMA_FILES to dt-validate
    # dt-bindings: kbuild: Use DTB files for validation
    $ make ARCH=arm shmobile_defconfig
    $ make ARCH=arm dtbs
    $ touch arch/arm/boot/dts/r8a7791.dtsi
    $ make ARCH=arm dtbs
    # The above command does NOT cause:
    # DTC     arch/arm/boot/dts/r8a7791-koelsch.dtb
    # DTC     arch/arm/boot/dts/r8a7791-porter.dtb
Quoted 3 lines
Quoted 1 line

I don't see anything wrong with this patch at first sight, though.

Was this a clean tree?

The above was a clean tree.

Quoted 4 lines

I think I reproduced it, but then couldn't... But then after a 'make clean', 'make dtbs' would error out. I think the issue in both cases was processed-schema.json always a dependency when it should be conditional on 'dtbs_check'. The patch below fixes that. Can you give it a try too.

I first saw the issue in my normal work tree. However, I cannot reproduce it in that tree anymore :-(

I can reproduce it in the clean tree after following the instructions above, and your patch fixes that, so

Tested-by Geert Uytterhoeven <[email protected]>
Quoted 7 lines
scripts/Makefile.lib +1 -0
@@ -349,12 +349,12 @@ $(multi-dtb-y): FORCE
        $(call if_changed,fdtoverlay)
 $(call multi_depend, $(multi-dtb-y), .dtb, -dtbs)
 
+ifneq ($(CHECK_DTBS)$(CHECK_DT_BINDING),)

scripts/Makefile.lib $(multi-dtb-y): FORCE · line 360

Quoted 10 lines
scripts/Makefile.lib +1 -2
 DT_CHECKER ?= dt-validate
 DT_CHECKER_FLAGS ?= $(if $(DT_SCHEMA_FILES),-l $(DT_SCHEMA_FILES),-m)
 DT_BINDING_DIR := Documentation/devicetree/bindings
-DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.json
+DT_TMP_SCHEMA := $(objtree)/$(DT_BINDING_DIR)/processed-schema.json
 
-ifneq ($(CHECK_DTBS)$(CHECK_DT_BINDING),)
 quiet_cmd_dtb_check =  CHECK   $@
       cmd_dtb_check =  $(DT_CHECKER) $(DT_CHECKER_FLAGS) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@ || true
 endif

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds