kbuild: Always validate DT binding examples

PATCH In review Rob Herring 3 participants 18 messages Feb 29, 2020 – Apr 21, 2020

Subsystems (2)

Device tree bindings Kbuild

lore

Share & tools

lore

Rob Herring

author maintainer

2020-02-29 00:37 UTC #0 lore raw

PATCH v2 1/2 kbuild: Always validate DT binding examples

Most folks only run dt_binding_check on the single schema they care about by setting DT_SCHEMA_FILES. That means example is only checked against that one schema which is not always sufficient.

Let's address this by splitting processed-schema.yaml into 2 files: one that's always all schemas for the examples and one that's just the schema in DT_SCHEMA_FILES for dtbs.

Cc Michal Marek <[email protected]>

Cc [email protected]

Co-developed-by Masahiro Yamada <[email protected]>

Signed-off-by Rob Herring <[email protected]>

---
Masahiro, given you pretty much re-wrote this, I added you as Co-developed-by.

Based on next/master

 Documentation/devicetree/bindings/.gitignore |  2 +-
 Documentation/devicetree/bindings/Makefile   | 22 +++++++++++++-------
 scripts/Makefile.lib                         |  3 ++-
 3 files changed, 17 insertions(+), 10 deletions(-)

Documentation/devicetree/bindings/.gitignore +1 -1

index ef82fcfcccab..57afa1533a5f 100644
@@ -1,2 +1,2 @@
 *.example.dts
-processed-schema.yaml
+processed-schema*.yaml

Documentation/devicetree/bindings/Makefile +14 -8

index 646cb3525373..7c40d5ba1b51 100644
@@ -2,7 +2,6 @@
 DT_DOC_CHECKER ?= dt-doc-validate
 DT_EXTRACT_EX ?= dt-extract-example
 DT_MK_SCHEMA ?= dt-mk-schema
-DT_MK_SCHEMA_FLAGS := $(if $(DT_SCHEMA_FILES), -u)
 
 quiet_cmd_chk_binding = CHKDT   $(patsubst $(srctree)/%,%,$<)
       cmd_chk_binding = $(DT_DOC_CHECKER) -u $(srctree)/$(src) $< ; \

@@ -11,26 +10,33 @@ quiet_cmd_chk_binding = CHKDT   $(patsubst $(srctree)/%,%,$<)
 $(obj)/%.example.dts: $(src)/%.yaml FORCE
 	$(call if_changed,chk_binding)
 
-DT_TMP_SCHEMA := processed-schema.yaml
+# Use full schemas when checking %.example.dts
+DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml
 
 quiet_cmd_mk_schema = SCHEMA  $@
       cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs)
 
-DT_DOCS = $(shell \
+DT_DOCS = $(addprefix $(src)/, \
+	$(shell \
 	cd $(srctree)/$(src) && \
 	find * \( -name '*.yaml' ! \
-		-name $(DT_TMP_SCHEMA) ! \
+		-name 'processed-schema*' ! \
 		-name '*.example.dt.yaml' \) \
-	)
+	))
 
-DT_SCHEMA_FILES ?= $(addprefix $(src)/,$(DT_DOCS))
+DT_SCHEMA_FILES ?= $(DT_DOCS)
 
 ifeq ($(CHECK_DTBS),)
 extra-y += $(patsubst $(src)/%.yaml,%.example.dts, $(DT_SCHEMA_FILES))
 extra-y += $(patsubst $(src)/%.yaml,%.example.dt.yaml, $(DT_SCHEMA_FILES))
+extra-y += processed-schema-examples.yaml
+
+$(obj)/processed-schema-examples.yaml: $(DT_DOCS) FORCE
+	$(call if_changed,mk_schema)
 endif
 
-$(obj)/$(DT_TMP_SCHEMA): $(DT_SCHEMA_FILES) FORCE
+$(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := -u
+$(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) FORCE
 	$(call if_changed,mk_schema)
 
-extra-y += $(DT_TMP_SCHEMA)
+extra-y += processed-schema.yaml

scripts/Makefile.lib +2 -1

index f5ff506e4a24..b12dd5ba4896 100644
@@ -307,7 +307,8 @@ $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
 
 DT_CHECKER ?= dt-validate
 DT_BINDING_DIR := Documentation/devicetree/bindings
-DT_TMP_SCHEMA := $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml
+# DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile
+DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml
 
 quiet_cmd_dtb_check =	CHECK   $@
       cmd_dtb_check =	$(DT_CHECKER) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@

Signature · 1 line

2.20.1

Rob Herring

author maintainer

2020-02-29 00:37 UTC #1 lore raw

PATCH v2 2/2 kbuild: Build DT binding examples with dtc warnings enabled

Now that we have a separate rule for DT binding examples, we can customize the dtc options. Let's adjust the dtc warnings to me more strict by default so the examples get cleaned up as they get converted to schema.

Leaving 'avoid_unnecessary_addr_size' and 'graph_child_address' warnings disabled as examples tend to be incomplete and they generates a lot of warnings.

Cc Michal Marek <[email protected]>

Cc [email protected]

Co-developed-by Masahiro Yamada <[email protected]>

Signed-off-by Rob Herring <[email protected]>

---
 Documentation/devicetree/bindings/Makefile | 4 ++++
 1 file changed, 4 insertions(+)

Documentation/devicetree/bindings/Makefile +4 -0

index 7c40d5ba1b51..b62c0470f122 100644
@@ -31,6 +31,10 @@ extra-y += $(patsubst $(src)/%.yaml,%.example.dts, $(DT_SCHEMA_FILES))
 extra-y += $(patsubst $(src)/%.yaml,%.example.dt.yaml, $(DT_SCHEMA_FILES))
 extra-y += processed-schema-examples.yaml
 
+override DTC_FLAGS := \
+	-Wno-avoid_unnecessary_addr_size \
+	-Wno-graph_child_address
+
 $(obj)/processed-schema-examples.yaml: $(DT_DOCS) FORCE
 	$(call if_changed,mk_schema)
 endif

Signature · 1 line

2.20.1

Masahiro Yamada

Acked-by on v2 1/2

2020-02-29 04:26 UTC #2 lore raw

In reply to Rob Herring

Hi Rob,

Quoted 9 lines · Rob Herring

Most folks only run dt_binding_check on the single schema they care about by setting DT_SCHEMA_FILES. That means example is only checked against that one schema which is not always sufficient.
Let's address this by splitting processed-schema.yaml into 2 files: one that's always all schemas for the examples and one that's just the schema in DT_SCHEMA_FILES for dtbs.

Quoted 7 lines

Cc Michal Marek <[email protected]>
Cc [email protected]
Co-developed-by Masahiro Yamada <[email protected]>
Signed-off-by Rob Herring <[email protected]>
---
Masahiro, given you pretty much re-wrote this, I added you as Co-developed-by.

I can apply both to my kbuild tree.

Or, if you want to apply them to your tree,
please add my

Acked-by Masahiro Yamada <[email protected]>

Either way is fine.

Thanks.

Quoted 86 lines

Based on next/master

 Documentation/devicetree/bindings/.gitignore |  2 +-
 Documentation/devicetree/bindings/Makefile   | 22 +++++++++++++-------
 scripts/Makefile.lib                         |  3 ++-
 3 files changed, 17 insertions(+), 10 deletions(-)

Documentation/devicetree/bindings/.gitignore +1 -1

index ef82fcfcccab..57afa1533a5f 100644
@@ -1,2 +1,2 @@
 *.example.dts
-processed-schema.yaml
+processed-schema*.yaml

Documentation/devicetree/bindings/Makefile +14 -8

index 646cb3525373..7c40d5ba1b51 100644
@@ -2,7 +2,6 @@
 DT_DOC_CHECKER ?= dt-doc-validate
 DT_EXTRACT_EX ?= dt-extract-example
 DT_MK_SCHEMA ?= dt-mk-schema
-DT_MK_SCHEMA_FLAGS := $(if $(DT_SCHEMA_FILES), -u)
 
 quiet_cmd_chk_binding = CHKDT   $(patsubst $(srctree)/%,%,$<)
       cmd_chk_binding = $(DT_DOC_CHECKER) -u $(srctree)/$(src) $< ; \

@@ -11,26 +10,33 @@ quiet_cmd_chk_binding = CHKDT   $(patsubst $(srctree)/%,%,$<)
 $(obj)/%.example.dts: $(src)/%.yaml FORCE
        $(call if_changed,chk_binding)
 
-DT_TMP_SCHEMA := processed-schema.yaml
+# Use full schemas when checking %.example.dts
+DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml
 
 quiet_cmd_mk_schema = SCHEMA  $@
       cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs)
 
-DT_DOCS = $(shell \
+DT_DOCS = $(addprefix $(src)/, \
+       $(shell \
        cd $(srctree)/$(src) && \
        find * \( -name '*.yaml' ! \
-               -name $(DT_TMP_SCHEMA) ! \
+               -name 'processed-schema*' ! \
                -name '*.example.dt.yaml' \) \
-       )
+       ))
 
-DT_SCHEMA_FILES ?= $(addprefix $(src)/,$(DT_DOCS))
+DT_SCHEMA_FILES ?= $(DT_DOCS)
 
 ifeq ($(CHECK_DTBS),)
 extra-y += $(patsubst $(src)/%.yaml,%.example.dts, $(DT_SCHEMA_FILES))
 extra-y += $(patsubst $(src)/%.yaml,%.example.dt.yaml, $(DT_SCHEMA_FILES))
+extra-y += processed-schema-examples.yaml
+
+$(obj)/processed-schema-examples.yaml: $(DT_DOCS) FORCE
+       $(call if_changed,mk_schema)
 endif
 
-$(obj)/$(DT_TMP_SCHEMA): $(DT_SCHEMA_FILES) FORCE
+$(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := -u
+$(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) FORCE
        $(call if_changed,mk_schema)
 
-extra-y += $(DT_TMP_SCHEMA)
+extra-y += processed-schema.yaml

scripts/Makefile.lib +2 -2

index f5ff506e4a24..b12dd5ba4896 100644
@@ -307,7 +307,8 @@ $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
 
 DT_CHECKER ?= dt-validate
 DT_BINDING_DIR := Documentation/devicetree/bindings
-DT_TMP_SCHEMA := $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml
+# DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile
+DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml
 
 quiet_cmd_dtb_check =  CHECK   $@
       cmd_dtb_check =  $(DT_CHECKER) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@
--

2.20.1

Signature · 2 lines

Best Regards
Masahiro Yamada

Rob Herring

author maintainer

2020-03-02 16:06 UTC #3 lore raw

Quoted 23 lines · Masahiro Yamada

Hi Rob,
Quoted 9 lines · Rob Herring
Most folks only run dt_binding_check on the single schema they care about by setting DT_SCHEMA_FILES. That means example is only checked against that one schema which is not always sufficient.
Let's address this by splitting processed-schema.yaml into 2 files: one that's always all schemas for the examples and one that's just the schema in DT_SCHEMA_FILES for dtbs.
Quoted 7 lines
Cc Michal Marek <[email protected]>
Cc [email protected]
Co-developed-by Masahiro Yamada <[email protected]>
Signed-off-by Rob Herring <[email protected]>
---
Masahiro, given you pretty much re-wrote this, I added you as Co-developed-by.

Quoted 1 line

I can apply both to my kbuild tree.

Please apply to your tree.

Rob

Quoted 103 lines

Or, if you want to apply them to your tree,
please add my

Acked-by Masahiro Yamada <[email protected]>

Either way is fine.

Thanks.

Quoted 84 lines

Based on next/master

 Documentation/devicetree/bindings/.gitignore |  2 +-
 Documentation/devicetree/bindings/Makefile   | 22 +++++++++++++-------
 scripts/Makefile.lib                         |  3 ++-
 3 files changed, 17 insertions(+), 10 deletions(-)

Documentation/devicetree/bindings/.gitignore +1 -1

index ef82fcfcccab..57afa1533a5f 100644
@@ -1,2 +1,2 @@
 *.example.dts
-processed-schema.yaml
+processed-schema*.yaml

Documentation/devicetree/bindings/Makefile +14 -8

index 646cb3525373..7c40d5ba1b51 100644
@@ -2,7 +2,6 @@
 DT_DOC_CHECKER ?= dt-doc-validate
 DT_EXTRACT_EX ?= dt-extract-example
 DT_MK_SCHEMA ?= dt-mk-schema
-DT_MK_SCHEMA_FLAGS := $(if $(DT_SCHEMA_FILES), -u)
 
 quiet_cmd_chk_binding = CHKDT   $(patsubst $(srctree)/%,%,$<)
       cmd_chk_binding = $(DT_DOC_CHECKER) -u $(srctree)/$(src) $< ; \

@@ -11,26 +10,33 @@ quiet_cmd_chk_binding = CHKDT   $(patsubst $(srctree)/%,%,$<)
 $(obj)/%.example.dts: $(src)/%.yaml FORCE
        $(call if_changed,chk_binding)
 
-DT_TMP_SCHEMA := processed-schema.yaml
+# Use full schemas when checking %.example.dts
+DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml
 
 quiet_cmd_mk_schema = SCHEMA  $@
       cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs)
 
-DT_DOCS = $(shell \
+DT_DOCS = $(addprefix $(src)/, \
+       $(shell \
        cd $(srctree)/$(src) && \
        find * \( -name '*.yaml' ! \
-               -name $(DT_TMP_SCHEMA) ! \
+               -name 'processed-schema*' ! \
                -name '*.example.dt.yaml' \) \
-       )
+       ))
 
-DT_SCHEMA_FILES ?= $(addprefix $(src)/,$(DT_DOCS))
+DT_SCHEMA_FILES ?= $(DT_DOCS)
 
 ifeq ($(CHECK_DTBS),)
 extra-y += $(patsubst $(src)/%.yaml,%.example.dts, $(DT_SCHEMA_FILES))
 extra-y += $(patsubst $(src)/%.yaml,%.example.dt.yaml, $(DT_SCHEMA_FILES))
+extra-y += processed-schema-examples.yaml
+
+$(obj)/processed-schema-examples.yaml: $(DT_DOCS) FORCE
+       $(call if_changed,mk_schema)
 endif
 
-$(obj)/$(DT_TMP_SCHEMA): $(DT_SCHEMA_FILES) FORCE
+$(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := -u
+$(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) FORCE
        $(call if_changed,mk_schema)
 
-extra-y += $(DT_TMP_SCHEMA)
+extra-y += processed-schema.yaml

scripts/Makefile.lib +2 -2

index f5ff506e4a24..b12dd5ba4896 100644
@@ -307,7 +307,8 @@ $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
 
 DT_CHECKER ?= dt-validate
 DT_BINDING_DIR := Documentation/devicetree/bindings
-DT_TMP_SCHEMA := $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml
+# DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile
+DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml
 
 quiet_cmd_dtb_check =  CHECK   $@
       cmd_dtb_check =  $(DT_CHECKER) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@
--

Quoted 2 lines

2.20.1

--
Best Regards
Masahiro Yamada

Masahiro Yamada

2020-03-03 13:46 UTC #4 lore raw

Quoted 25 lines · Rob Herring

Quoted 23 lines · Masahiro Yamada
Hi Rob,
Quoted 16 lines · Rob Herring
Most folks only run dt_binding_check on the single schema they care about by setting DT_SCHEMA_FILES. That means example is only checked against that one schema which is not always sufficient.
Let's address this by splitting processed-schema.yaml into 2 files: one that's always all schemas for the examples and one that's just the schema in DT_SCHEMA_FILES for dtbs.
Cc Michal Marek <[email protected]>
Cc [email protected]
Co-developed-by Masahiro Yamada <[email protected]>
Signed-off-by Rob Herring <[email protected]>
---
Masahiro, given you pretty much re-wrote this, I added you as Co-developed-by.

Quoted 3 lines

Quoted 1 line
I can apply both to my kbuild tree.
Please apply to your tree.

Done.
Thanks.

Signature · 2 lines

Best Regards
Masahiro Yamada

Laurent Pinchart

2020-04-21 10:08 UTC #5 lore raw

In reply to Rob Herring

Hi Rob,

Quoted 7 lines · Rob Herring

Most folks only run dt_binding_check on the single schema they care about by setting DT_SCHEMA_FILES. That means example is only checked against that one schema which is not always sufficient.
Let's address this by splitting processed-schema.yaml into 2 files: one that's always all schemas for the examples and one that's just the schema in DT_SCHEMA_FILES for dtbs.

This broke

make DT_SCHEMA_FILES=Documentation/devicetree/.. dt_binding_check

for me :-(

make[1]: Entering directory '/home/laurent/src/linux/output/arm32'
make[2]: execvp: /bin/sh: Argument list too long
make[2]: *** [/home/laurent/src/linux/Documentation/devicetree/bindings/Makefile:35: Documentation/devicetree/bindings/processed-schema-examples.yaml] Error 127 make[1]: *** [/home/laurent/src/linux/Makefile:1296: dt_binding_check] Error 2
make[1]: Leaving directory '/home/laurent/src/linux/output/arm32'
make: *** [Makefile:180: sub-make] Error 2

Before the patch, dt-mk-schema was called with DT_SCHEMA_FILES only, and it is now called with DT_DOCS unconditionally.

Quoted 90 lines

Cc Michal Marek <[email protected]>

Cc [email protected]

Co-developed-by Masahiro Yamada <[email protected]>

Signed-off-by Rob Herring <[email protected]>

---
Masahiro, given you pretty much re-wrote this, I added you as Co-developed-by.

Based on next/master

 Documentation/devicetree/bindings/.gitignore |  2 +-
 Documentation/devicetree/bindings/Makefile   | 22 +++++++++++++-------
 scripts/Makefile.lib                         |  3 ++-
 3 files changed, 17 insertions(+), 10 deletions(-)

Documentation/devicetree/bindings/.gitignore +1 -1

index ef82fcfcccab..57afa1533a5f 100644
@@ -1,2 +1,2 @@
 *.example.dts
-processed-schema.yaml
+processed-schema*.yaml

Documentation/devicetree/bindings/Makefile +14 -8

index 646cb3525373..7c40d5ba1b51 100644
@@ -2,7 +2,6 @@
 DT_DOC_CHECKER ?= dt-doc-validate
 DT_EXTRACT_EX ?= dt-extract-example
 DT_MK_SCHEMA ?= dt-mk-schema
-DT_MK_SCHEMA_FLAGS := $(if $(DT_SCHEMA_FILES), -u)
 
 quiet_cmd_chk_binding = CHKDT   $(patsubst $(srctree)/%,%,$<)
       cmd_chk_binding = $(DT_DOC_CHECKER) -u $(srctree)/$(src) $< ; \

@@ -11,26 +10,33 @@ quiet_cmd_chk_binding = CHKDT   $(patsubst $(srctree)/%,%,$<)
 $(obj)/%.example.dts: $(src)/%.yaml FORCE
 	$(call if_changed,chk_binding)
 
-DT_TMP_SCHEMA := processed-schema.yaml
+# Use full schemas when checking %.example.dts
+DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml
 
 quiet_cmd_mk_schema = SCHEMA  $@
       cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs)
 
-DT_DOCS = $(shell \
+DT_DOCS = $(addprefix $(src)/, \
+	$(shell \
 	cd $(srctree)/$(src) && \
 	find * \( -name '*.yaml' ! \
-		-name $(DT_TMP_SCHEMA) ! \
+		-name 'processed-schema*' ! \
 		-name '*.example.dt.yaml' \) \
-	)
+	))
 
-DT_SCHEMA_FILES ?= $(addprefix $(src)/,$(DT_DOCS))
+DT_SCHEMA_FILES ?= $(DT_DOCS)
 
 ifeq ($(CHECK_DTBS),)
 extra-y += $(patsubst $(src)/%.yaml,%.example.dts, $(DT_SCHEMA_FILES))
 extra-y += $(patsubst $(src)/%.yaml,%.example.dt.yaml, $(DT_SCHEMA_FILES))
+extra-y += processed-schema-examples.yaml
+
+$(obj)/processed-schema-examples.yaml: $(DT_DOCS) FORCE
+	$(call if_changed,mk_schema)
 endif
 
-$(obj)/$(DT_TMP_SCHEMA): $(DT_SCHEMA_FILES) FORCE
+$(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := -u
+$(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) FORCE
 	$(call if_changed,mk_schema)
 
-extra-y += $(DT_TMP_SCHEMA)
+extra-y += processed-schema.yaml

scripts/Makefile.lib +2 -1

index f5ff506e4a24..b12dd5ba4896 100644
@@ -307,7 +307,8 @@ $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
 
 DT_CHECKER ?= dt-validate
 DT_BINDING_DIR := Documentation/devicetree/bindings
-DT_TMP_SCHEMA := $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml
+# DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile
+DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml
 
 quiet_cmd_dtb_check =	CHECK   $@
       cmd_dtb_check =	$(DT_CHECKER) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@

Signature · 3 lines

Regards,

Laurent Pinchart

Masahiro Yamada

2020-04-21 10:46 UTC #6 lore raw

Quoted 8 lines · Laurent Pinchart

Hi Rob,
Quoted 4 lines · Rob Herring
Most folks only run dt_binding_check on the single schema they care about by setting DT_SCHEMA_FILES. That means example is only checked against that one schema which is not always sufficient.

Quoted 7 lines

Quoted 3 lines
Let's address this by splitting processed-schema.yaml into 2 files: one that's always all schemas for the examples and one that's just the schema in DT_SCHEMA_FILES for dtbs.
This broke
make DT_SCHEMA_FILES=Documentation/devicetree/.. dt_binding_check

What is intended by
"DT_SCHEMA_FILES=Documentation/devicetree/.." ?

Quoted 108 lines

for me :-(

Before the patch, dt-mk-schema was called with DT_SCHEMA_FILES only, and it is now called with DT_DOCS unconditionally.

Quoted 81 lines

Cc Michal Marek <[email protected]>

Cc [email protected]

Co-developed-by Masahiro Yamada <[email protected]>

Signed-off-by Rob Herring <[email protected]>

---
Masahiro, given you pretty much re-wrote this, I added you as Co-developed-by.

Based on next/master

 Documentation/devicetree/bindings/.gitignore |  2 +-
 Documentation/devicetree/bindings/Makefile   | 22 +++++++++++++-------
 scripts/Makefile.lib                         |  3 ++-
 3 files changed, 17 insertions(+), 10 deletions(-)

Documentation/devicetree/bindings/.gitignore +1 -1

index ef82fcfcccab..57afa1533a5f 100644
@@ -1,2 +1,2 @@
 *.example.dts
-processed-schema.yaml
+processed-schema*.yaml

Documentation/devicetree/bindings/Makefile +14 -8

index 646cb3525373..7c40d5ba1b51 100644
@@ -2,7 +2,6 @@
 DT_DOC_CHECKER ?= dt-doc-validate
 DT_EXTRACT_EX ?= dt-extract-example
 DT_MK_SCHEMA ?= dt-mk-schema
-DT_MK_SCHEMA_FLAGS := $(if $(DT_SCHEMA_FILES), -u)
 
 quiet_cmd_chk_binding = CHKDT   $(patsubst $(srctree)/%,%,$<)
       cmd_chk_binding = $(DT_DOC_CHECKER) -u $(srctree)/$(src) $< ; \

@@ -11,26 +10,33 @@ quiet_cmd_chk_binding = CHKDT   $(patsubst $(srctree)/%,%,$<)
 $(obj)/%.example.dts: $(src)/%.yaml FORCE
      $(call if_changed,chk_binding)
 
-DT_TMP_SCHEMA := processed-schema.yaml
+# Use full schemas when checking %.example.dts
+DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml
 
 quiet_cmd_mk_schema = SCHEMA  $@
       cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs)
 
-DT_DOCS = $(shell \
+DT_DOCS = $(addprefix $(src)/, \
+     $(shell \
      cd $(srctree)/$(src) && \
      find * \( -name '*.yaml' ! \
-             -name $(DT_TMP_SCHEMA) ! \
+             -name 'processed-schema*' ! \
              -name '*.example.dt.yaml' \) \
-     )
+     ))
 
-DT_SCHEMA_FILES ?= $(addprefix $(src)/,$(DT_DOCS))
+DT_SCHEMA_FILES ?= $(DT_DOCS)
 
 ifeq ($(CHECK_DTBS),)
 extra-y += $(patsubst $(src)/%.yaml,%.example.dts, $(DT_SCHEMA_FILES))
 extra-y += $(patsubst $(src)/%.yaml,%.example.dt.yaml, $(DT_SCHEMA_FILES))
+extra-y += processed-schema-examples.yaml
+
+$(obj)/processed-schema-examples.yaml: $(DT_DOCS) FORCE
+     $(call if_changed,mk_schema)
 endif
 
-$(obj)/$(DT_TMP_SCHEMA): $(DT_SCHEMA_FILES) FORCE
+$(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := -u
+$(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) FORCE
      $(call if_changed,mk_schema)
 
-extra-y += $(DT_TMP_SCHEMA)
+extra-y += processed-schema.yaml

scripts/Makefile.lib +0 -0

index f5ff506e4a24..b12dd5ba4896 100644

scripts/Makefile.lib $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE · line 314

Quoted 9 lines

scripts/Makefile.lib +2 -1

@@ -307,7 +307,8 @@ $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
 
 DT_CHECKER ?= dt-validate
 DT_BINDING_DIR := Documentation/devicetree/bindings
-DT_TMP_SCHEMA := $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml
+# DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile
+DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml
 
 quiet_cmd_dtb_check =        CHECK   $@
       cmd_dtb_check =        $(DT_CHECKER) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@

--
Regards,

Laurent Pinchart

Signature · 2 lines

Best Regards
Masahiro Yamada

Laurent Pinchart

2020-04-21 11:05 UTC #7 lore raw

Hi Yamada-san,

Quoted 10 lines · Masahiro Yamada

Quoted 9 lines · Laurent Pinchart
Quoted 7 lines · Rob Herring
Most folks only run dt_binding_check on the single schema they care about by setting DT_SCHEMA_FILES. That means example is only checked against that one schema which is not always sufficient.
Let's address this by splitting processed-schema.yaml into 2 files: one that's always all schemas for the examples and one that's just the schema in DT_SCHEMA_FILES for dtbs.

Quoted 6 lines

Quoted 3 lines
This broke
make DT_SCHEMA_FILES=Documentation/devicetree/.. dt_binding_check
What is intended by
"DT_SCHEMA_FILES=Documentation/devicetree/.." ?

My bad, I forgot to write that ... is the continuation of the string. It's any yaml schema file that has an example.

Quoted 102 lines

for me :-(

Before the patch, dt-mk-schema was called with DT_SCHEMA_FILES only, and it is now called with DT_DOCS unconditionally.

Quoted 90 lines

Cc Michal Marek <[email protected]>

Cc [email protected]

Co-developed-by Masahiro Yamada <[email protected]>

Signed-off-by Rob Herring <[email protected]>

---
Masahiro, given you pretty much re-wrote this, I added you as Co-developed-by.

Based on next/master

 Documentation/devicetree/bindings/.gitignore |  2 +-
 Documentation/devicetree/bindings/Makefile   | 22 +++++++++++++-------
 scripts/Makefile.lib                         |  3 ++-
 3 files changed, 17 insertions(+), 10 deletions(-)

Documentation/devicetree/bindings/.gitignore +1 -1

index ef82fcfcccab..57afa1533a5f 100644
@@ -1,2 +1,2 @@
 *.example.dts
-processed-schema.yaml
+processed-schema*.yaml

Documentation/devicetree/bindings/Makefile +14 -8

index 646cb3525373..7c40d5ba1b51 100644
@@ -2,7 +2,6 @@
 DT_DOC_CHECKER ?= dt-doc-validate
 DT_EXTRACT_EX ?= dt-extract-example
 DT_MK_SCHEMA ?= dt-mk-schema
-DT_MK_SCHEMA_FLAGS := $(if $(DT_SCHEMA_FILES), -u)
 
 quiet_cmd_chk_binding = CHKDT   $(patsubst $(srctree)/%,%,$<)
       cmd_chk_binding = $(DT_DOC_CHECKER) -u $(srctree)/$(src) $< ; \

@@ -11,26 +10,33 @@ quiet_cmd_chk_binding = CHKDT   $(patsubst $(srctree)/%,%,$<)
 $(obj)/%.example.dts: $(src)/%.yaml FORCE
      $(call if_changed,chk_binding)
 
-DT_TMP_SCHEMA := processed-schema.yaml
+# Use full schemas when checking %.example.dts
+DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml
 
 quiet_cmd_mk_schema = SCHEMA  $@
       cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs)
 
-DT_DOCS = $(shell \
+DT_DOCS = $(addprefix $(src)/, \
+     $(shell \
      cd $(srctree)/$(src) && \
      find * \( -name '*.yaml' ! \
-             -name $(DT_TMP_SCHEMA) ! \
+             -name 'processed-schema*' ! \
              -name '*.example.dt.yaml' \) \
-     )
+     ))
 
-DT_SCHEMA_FILES ?= $(addprefix $(src)/,$(DT_DOCS))
+DT_SCHEMA_FILES ?= $(DT_DOCS)
 
 ifeq ($(CHECK_DTBS),)
 extra-y += $(patsubst $(src)/%.yaml,%.example.dts, $(DT_SCHEMA_FILES))
 extra-y += $(patsubst $(src)/%.yaml,%.example.dt.yaml, $(DT_SCHEMA_FILES))
+extra-y += processed-schema-examples.yaml
+
+$(obj)/processed-schema-examples.yaml: $(DT_DOCS) FORCE
+     $(call if_changed,mk_schema)
 endif
 
-$(obj)/$(DT_TMP_SCHEMA): $(DT_SCHEMA_FILES) FORCE
+$(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := -u
+$(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) FORCE
      $(call if_changed,mk_schema)
 
-extra-y += $(DT_TMP_SCHEMA)
+extra-y += processed-schema.yaml

scripts/Makefile.lib +2 -1

index f5ff506e4a24..b12dd5ba4896 100644
@@ -307,7 +307,8 @@ $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
 
 DT_CHECKER ?= dt-validate
 DT_BINDING_DIR := Documentation/devicetree/bindings
-DT_TMP_SCHEMA := $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml
+# DT_TMP_SCHEMA may be overridden from Documentation/devicetree/bindings/Makefile
+DT_TMP_SCHEMA ?= $(objtree)/$(DT_BINDING_DIR)/processed-schema.yaml
 
 quiet_cmd_dtb_check =        CHECK   $@
       cmd_dtb_check =        $(DT_CHECKER) -u $(srctree)/$(DT_BINDING_DIR) -p $(DT_TMP_SCHEMA) $@

Signature · 3 lines

Regards,

Laurent Pinchart

Masahiro Yamada

2020-04-21 13:16 UTC #8 lore raw

Hi Laurent,

Quoted 16 lines · Laurent Pinchart

Hi Yamada-san,
Quoted 12 lines · Masahiro Yamada
Quoted 11 lines · Laurent Pinchart
Quoted 7 lines · Rob Herring
Most folks only run dt_binding_check on the single schema they care about by setting DT_SCHEMA_FILES. That means example is only checked against that one schema which is not always sufficient.
Let's address this by splitting processed-schema.yaml into 2 files: one that's always all schemas for the examples and one that's just the schema in DT_SCHEMA_FILES for dtbs.
This broke

Quoted 7 lines

Quoted 4 lines
Quoted 1 line
make DT_SCHEMA_FILES=Documentation/devicetree/.. dt_binding_check
What is intended by
"DT_SCHEMA_FILES=Documentation/devicetree/.." ?
My bad, I forgot to write that ... is the continuation of the string. It's any yaml schema file that has an example.

Ah, OK. I just input verbatim.

Is it broken?

You can specify any individual file(s) under Documentation/devicetree/bindings/.

For example, the following worked for me.

$ make DT_SCHEMA_FILES=Documentation/devicetree/bindings/arm/psci.yaml
 dt_binding_check
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.[ch]
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTCC  scripts/dtc/yamltree.o
  HOSTLD  scripts/dtc/dtc
  CHKDT   Documentation/devicetree/bindings/arm/psci.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema-examples.yaml
  DTC     Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema.yaml

Signature · 2 lines

Best Regards
Masahiro Yamada

Laurent Pinchart

2020-04-21 13:47 UTC #9 lore raw

Hi Yamada-san,

Quoted 45 lines · Masahiro Yamada

Quoted 13 lines · Laurent Pinchart
Quoted 12 lines · Masahiro Yamada
Quoted 11 lines · Laurent Pinchart
Quoted 7 lines · Rob Herring
Most folks only run dt_binding_check on the single schema they care about by setting DT_SCHEMA_FILES. That means example is only checked against that one schema which is not always sufficient.
Let's address this by splitting processed-schema.yaml into 2 files: one that's always all schemas for the examples and one that's just the schema in DT_SCHEMA_FILES for dtbs.
This broke
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
make DT_SCHEMA_FILES=Documentation/devicetree/.. dt_binding_check
What is intended by
"DT_SCHEMA_FILES=Documentation/devicetree/.." ?
My bad, I forgot to write that ... is the continuation of the string. It's any yaml schema file that has an example.
Ah, OK. I just input verbatim.
Is it broken?
You can specify any individual file(s) under Documentation/devicetree/bindings/.
For example, the following worked for me.
$ make DT_SCHEMA_FILES=Documentation/devicetree/bindings/arm/psci.yaml
 dt_binding_check
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.[ch]
  HOSTCC  scripts/dtc/dtc-lexer.lex.o

Quoted 8 lines

  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTCC  scripts/dtc/yamltree.o
  HOSTLD  scripts/dtc/dtc
  CHKDT   Documentation/devicetree/bindings/arm/psci.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema-examples.yaml
  DTC     Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema.yaml

This is getting interesting.

~/src/kernel/linux $ make O=output/x86 DT_SCHEMA_FILES=Documentation/devicetree/bindings/arm/psci.yaml dt_binding_check make[1]: Entering directory '/home/laurent/src/kernel/linux/output/x86'

  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.[ch]
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTCC  scripts/dtc/yamltree.o
  HOSTLD  scripts/dtc/dtc
  CHKDT   Documentation/devicetree/bindings/arm/psci.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema-examples.yaml
  DTC     Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema.yaml

make[1]: Leaving directory '/home/laurent/src/kernel/linux/output/x86'

~/src/kernel/this_is_a_long_directory_name/linux $ make O=output/x86 DT_SCHEMA_FILES=Documentation/devicetree/bindings/arm/psci.yaml dt_binding_check make[1]: Entering directory '/home/laurent/src/kernel/this_is_a_long_directory_name/linux/output/x86'

  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.[ch]
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTCC  scripts/dtc/yamltree.o
  HOSTLD  scripts/dtc/dtc
  CHKDT   Documentation/devicetree/bindings/arm/psci.yaml

make[2]: execvp: /bin/sh: Argument list too long
make[2]: *** [/home/laurent/src/kernel/this_is_a_long_directory_name/linux/Documentation/devicetree/bindings/Makefile:38: Documentation/devicetree/bindings/processed-schema-examples.yaml] Error 127 make[1]: *** [/home/laurent/src/kernel/this_is_a_long_directory_name/linux/Makefile:1300: dt_binding_check] Error 2
make[1]: Leaving directory '/home/laurent/src/kernel/this_is_a_long_directory_name/linux/output/x86'
make: *** [Makefile:180: sub-make] Error 2

It seems to only fail with out of tree builds (O=...). I expect that failures will become more common the more YAML bindings we have, even without long directory names.

Signature · 3 lines

Regards,

Laurent Pinchart

Rob Herring

author maintainer

2020-04-21 14:01 UTC #10 lore raw

Quoted 110 lines · Laurent Pinchart

Hi Yamada-san,
Quoted 45 lines · Masahiro Yamada
Quoted 13 lines · Laurent Pinchart
Quoted 12 lines · Masahiro Yamada
Quoted 11 lines · Laurent Pinchart
Quoted 7 lines · Rob Herring
Most folks only run dt_binding_check on the single schema they care about by setting DT_SCHEMA_FILES. That means example is only checked against that one schema which is not always sufficient.
Let's address this by splitting processed-schema.yaml into 2 files: one that's always all schemas for the examples and one that's just the schema in DT_SCHEMA_FILES for dtbs.
This broke
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
make DT_SCHEMA_FILES=Documentation/devicetree/.. dt_binding_check
What is intended by
"DT_SCHEMA_FILES=Documentation/devicetree/.." ?
My bad, I forgot to write that ... is the continuation of the string. It's any yaml schema file that has an example.
Ah, OK. I just input verbatim.
Is it broken?
You can specify any individual file(s) under Documentation/devicetree/bindings/.
For example, the following worked for me.
$ make DT_SCHEMA_FILES=Documentation/devicetree/bindings/arm/psci.yaml
 dt_binding_check
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.[ch]
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
Quoted 8 lines
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTCC  scripts/dtc/yamltree.o
  HOSTLD  scripts/dtc/dtc
  CHKDT   Documentation/devicetree/bindings/arm/psci.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema-examples.yaml
  DTC     Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema.yaml
This is getting interesting.
~/src/kernel/linux $ make O=output/x86 DT_SCHEMA_FILES=Documentation/devicetree/bindings/arm/psci.yaml dt_binding_check make[1]: Entering directory '/home/laurent/src/kernel/linux/output/x86'
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.[ch]
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTCC  scripts/dtc/yamltree.o
  HOSTLD  scripts/dtc/dtc
  CHKDT   Documentation/devicetree/bindings/arm/psci.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema-examples.yaml
  DTC     Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema.yaml
make[1]: Leaving directory '/home/laurent/src/kernel/linux/output/x86'
~/src/kernel/this_is_a_long_directory_name/linux $ make O=output/x86 DT_SCHEMA_FILES=Documentation/devicetree/bindings/arm/psci.yaml dt_binding_check make[1]: Entering directory '/home/laurent/src/kernel/this_is_a_long_directory_name/linux/output/x86'
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.[ch]
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTCC  scripts/dtc/yamltree.o
  HOSTLD  scripts/dtc/dtc
  CHKDT   Documentation/devicetree/bindings/arm/psci.yaml
make[2]: execvp: /bin/sh: Argument list too long
make[2]: *** [/home/laurent/src/kernel/this_is_a_long_directory_name/linux/Documentation/devicetree/bindings/Makefile:38: Documentation/devicetree/bindings/processed-schema-examples.yaml] Error 127 make[1]: *** [/home/laurent/src/kernel/this_is_a_long_directory_name/linux/Makefile:1300: dt_binding_check] Error 2
make[1]: Leaving directory '/home/laurent/src/kernel/this_is_a_long_directory_name/linux/output/x86'
make: *** [Makefile:180: sub-make] Error 2

Quoted 3 lines

It seems to only fail with out of tree builds (O=...). I expect that failures will become more common the more YAML bindings we have, even without long directory names.

dt-mk-schema can take and recurse a single directory already, so does this fix it for you:

@@ -22,7 +22,7 @@ $(obj)/%.example.dts: $(src)/%.yaml check_dtschema_version FORCE
DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml

quiet_cmd_mk_schema = SCHEMA $@
- cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@
$(real-prereqs)
+ cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@
$(srctree)/$(src)

 DT_DOCS = $(addprefix $(src)/, \
        $(shell \

Masahiro Yamada

2020-04-21 16:57 UTC #11 lore raw

Hi Rob,

Quoted 12 lines · Rob Herring

Quoted 4 lines
It seems to only fail with out of tree builds (O=...). I expect that failures will become more common the more YAML bindings we have, even without long directory names.
dt-mk-schema can take and recurse a single directory already, so does this fix it for you:
(quoted diff) +0 -0
@@ -22,7 +22,7 @@ $(obj)/%.example.dts: $(src)/%.yaml
check_dtschema_version FORCE
DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml

Quoted 5 lines

quiet_cmd_mk_schema = SCHEMA $@
- cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@
$(real-prereqs)
+ cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@
$(srctree)/$(src)

I am just curious.

How come the tool excludes 'processed-schema*' and '*.example.dt.yaml' from $(srctree)/$(src) ?

Signature · 2 lines

Best Regards
Masahiro Yamada

Rob Herring

author maintainer

2020-04-21 19:31 UTC #12 lore raw

Quoted 26 lines · Masahiro Yamada

Hi Rob,
Quoted 12 lines · Rob Herring
Quoted 4 lines
It seems to only fail with out of tree builds (O=...). I expect that failures will become more common the more YAML bindings we have, even without long directory names.
dt-mk-schema can take and recurse a single directory already, so does this fix it for you:
(quoted diff) +0 -0
@@ -22,7 +22,7 @@ $(obj)/%.example.dts: $(src)/%.yaml
check_dtschema_version FORCE
DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml
Quoted 5 lines
quiet_cmd_mk_schema = SCHEMA $@
- cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@
$(real-prereqs)
+ cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@
$(srctree)/$(src)
I am just curious.

Quoted 2 lines

How come the tool excludes 'processed-schema*' and '*.example.dt.yaml' from $(srctree)/$(src) ?

Uggg, it wouldn't. Can't everyone build out of tree. ;) I guess the options are:

- Call dt-mk-schema on each file individually appending the result
- Make dt-mk-schema take stdin or a file with a list of files as Masahiro suggested

I'd like to avoid a dt-mk-schema change so I'll try to make the former work. It's only been a day since adding a minimum version. :(

Rob

Rob Herring

author maintainer

2020-04-21 20:40 UTC #13 lore raw

Quoted 31 lines · Rob Herring

Quoted 26 lines · Masahiro Yamada
Hi Rob,
Quoted 12 lines · Rob Herring
Quoted 4 lines
It seems to only fail with out of tree builds (O=...). I expect that failures will become more common the more YAML bindings we have, even without long directory names.
dt-mk-schema can take and recurse a single directory already, so does this fix it for you:
(quoted diff) +0 -0
@@ -22,7 +22,7 @@ $(obj)/%.example.dts: $(src)/%.yaml
check_dtschema_version FORCE
DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml
Quoted 5 lines
quiet_cmd_mk_schema = SCHEMA $@
- cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@
$(real-prereqs)
+ cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@
$(srctree)/$(src)
I am just curious.
Quoted 2 lines
How come the tool excludes 'processed-schema*' and '*.example.dt.yaml' from $(srctree)/$(src) ?

Quoted 8 lines

Uggg, it wouldn't. Can't everyone build out of tree. ;) I guess the options are:
- Call dt-mk-schema on each file individually appending the result
- Make dt-mk-schema take stdin or a file with a list of files as Masahiro suggested
I'd like to avoid a dt-mk-schema change so I'll try to make the former work. It's only been a day since adding a minimum version. :(

Laurent, does this work for you? I've reproduced the problem, but just double checking.

DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml

quiet_cmd_mk_schema = SCHEMA $@
- cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@
$(real-prereqs)
+ cmd_mk_schema = $(file >[email protected], $(real-prereqs)) \
+ cat [email protected] | xargs $(DT_MK_SCHEMA)
$(DT_MK_SCHEMA_FLAGS) -o $@

 DT_DOCS = $(addprefix $(src)/, \
        $(shell \

Laurent Pinchart

2020-04-21 18:29 UTC #14 lore raw

In reply to Rob Herring

Hi Rob,

Quoted 108 lines · Rob Herring

Quoted 107 lines · Laurent Pinchart
Quoted 45 lines · Masahiro Yamada
Quoted 13 lines · Laurent Pinchart
Quoted 12 lines · Masahiro Yamada
Quoted 11 lines · Laurent Pinchart
Quoted 7 lines · Rob Herring
Most folks only run dt_binding_check on the single schema they care about by setting DT_SCHEMA_FILES. That means example is only checked against that one schema which is not always sufficient.
Let's address this by splitting processed-schema.yaml into 2 files: one that's always all schemas for the examples and one that's just the schema in DT_SCHEMA_FILES for dtbs.
This broke
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
make DT_SCHEMA_FILES=Documentation/devicetree/.. dt_binding_check
What is intended by
"DT_SCHEMA_FILES=Documentation/devicetree/.." ?
My bad, I forgot to write that ... is the continuation of the string. It's any yaml schema file that has an example.
Ah, OK. I just input verbatim.
Is it broken?
You can specify any individual file(s) under Documentation/devicetree/bindings/.
For example, the following worked for me.
$ make DT_SCHEMA_FILES=Documentation/devicetree/bindings/arm/psci.yaml
 dt_binding_check
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.[ch]
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
Quoted 8 lines
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTCC  scripts/dtc/yamltree.o
  HOSTLD  scripts/dtc/dtc
  CHKDT   Documentation/devicetree/bindings/arm/psci.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema-examples.yaml
  DTC     Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema.yaml
This is getting interesting.
~/src/kernel/linux $ make O=output/x86 DT_SCHEMA_FILES=Documentation/devicetree/bindings/arm/psci.yaml dt_binding_check make[1]: Entering directory '/home/laurent/src/kernel/linux/output/x86'
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.[ch]
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTCC  scripts/dtc/yamltree.o
  HOSTLD  scripts/dtc/dtc
  CHKDT   Documentation/devicetree/bindings/arm/psci.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema-examples.yaml
  DTC     Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/psci.example.dt.yaml
  SCHEMA  Documentation/devicetree/bindings/processed-schema.yaml
make[1]: Leaving directory '/home/laurent/src/kernel/linux/output/x86'
~/src/kernel/this_is_a_long_directory_name/linux $ make O=output/x86 DT_SCHEMA_FILES=Documentation/devicetree/bindings/arm/psci.yaml dt_binding_check make[1]: Entering directory '/home/laurent/src/kernel/this_is_a_long_directory_name/linux/output/x86'
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  LEX     scripts/dtc/dtc-lexer.lex.c
  YACC    scripts/dtc/dtc-parser.tab.[ch]
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTCC  scripts/dtc/yamltree.o
  HOSTLD  scripts/dtc/dtc
  CHKDT   Documentation/devicetree/bindings/arm/psci.yaml
make[2]: execvp: /bin/sh: Argument list too long
make[2]: *** [/home/laurent/src/kernel/this_is_a_long_directory_name/linux/Documentation/devicetree/bindings/Makefile:38: Documentation/devicetree/bindings/processed-schema-examples.yaml] Error 127 make[1]: *** [/home/laurent/src/kernel/this_is_a_long_directory_name/linux/Makefile:1300: dt_binding_check] Error 2
make[1]: Leaving directory '/home/laurent/src/kernel/this_is_a_long_directory_name/linux/output/x86'
make: *** [Makefile:180: sub-make] Error 2

Quoted 6 lines

Quoted 3 lines
It seems to only fail with out of tree builds (O=...). I expect that failures will become more common the more YAML bindings we have, even without long directory names.
dt-mk-schema can take and recurse a single directory already, so does this fix it for you:

Yes, the change below fixes my problem, thank you.

Quoted 12 lines

(quoted diff) +0 -0
@@ -22,7 +22,7 @@ $(obj)/%.example.dts: $(src)/%.yaml
check_dtschema_version FORCE
DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml
quiet_cmd_mk_schema = SCHEMA $@
- cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@
$(real-prereqs)
+ cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@
$(srctree)/$(src)
 DT_DOCS = $(addprefix $(src)/, \
        $(shell \

Signature · 3 lines

Regards,

Laurent Pinchart

Masahiro Yamada

2020-04-21 17:37 UTC #15 lore raw

In reply to Laurent Pinchart

Hi Laurent,

Quoted 11 lines · Laurent Pinchart

HOSTLD scripts/dtc/dtc
CHKDT Documentation/devicetree/bindings/arm/psci.yaml
make[2]: execvp: /bin/sh: Argument list too long
make[2]: *** [/home/laurent/src/kernel/this_is_a_long_directory_name/linux/Documentation/devicetree/bindings/Makefile:38: Documentation/devicetree/bindings/processed-schema-examples.yaml] Error 127 make[1]: *** [/home/laurent/src/kernel/this_is_a_long_directory_name/linux/Makefile:1300: dt_binding_check] Error 2
make[1]: Leaving directory '/home/laurent/src/kernel/this_is_a_long_directory_name/linux/output/x86'
make: *** [Makefile:180: sub-make] Error 2
It seems to only fail with out of tree builds (O=...). I expect that failures will become more common the more YAML bindings we have, even without long directory names.

Indeed. This will be a common problem sooner or later.

Rob already suggested a solution.

Another way is to use a pipe.

See the following code in scripts/Makefile.modpost

# Read out modules.order instead of expanding $(modules) to pass in modpost. # Otherwise, allmodconfig would fail with "Argument list too long". quiet_cmd_modpost = MODPOST $(words $(modules)) modules

      cmd_modpost = sed 's/ko$$/o/' $(MODORDER) | $(MODPOST)

The same issue happens for modpost
because allmodconfig floods the command line.

Signature · 2 lines

Best Regards
Masahiro Yamada

Rob Herring

author maintainer

2020-04-21 13:47 UTC #16 lore raw

In reply to Masahiro Yamada

Quoted 29 lines · Masahiro Yamada

Hi Laurent,
Quoted 16 lines · Laurent Pinchart
Hi Yamada-san,
Quoted 12 lines · Masahiro Yamada
Quoted 11 lines · Laurent Pinchart
Quoted 7 lines · Rob Herring
Most folks only run dt_binding_check on the single schema they care about by setting DT_SCHEMA_FILES. That means example is only checked against that one schema which is not always sufficient.
Let's address this by splitting processed-schema.yaml into 2 files: one that's always all schemas for the examples and one that's just the schema in DT_SCHEMA_FILES for dtbs.
This broke
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
make DT_SCHEMA_FILES=Documentation/devicetree/.. dt_binding_check
What is intended by
"DT_SCHEMA_FILES=Documentation/devicetree/.." ?
My bad, I forgot to write that ... is the continuation of the string. It's any yaml schema file that has an example.

Quoted 7 lines

Ah, OK. I just input verbatim.
Is it broken?
You can specify any individual file(s) under Documentation/devicetree/bindings/.
For example, the following worked for me.

Me too.

There is however a bug I just noticed. We're now always setting '-u' for dtbs_check which means the core schema are never used.

Masahiro, Got a better suggestion than adding the ifeq:

ifeq ("$(origin DT_SCHEMA_FILES)", "command line") $(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := -u endif

Rob

Masahiro Yamada

2020-04-21 17:32 UTC #17 lore raw

Hi Rob,

Quoted 41 lines · Rob Herring

Quoted 29 lines · Masahiro Yamada
Hi Laurent,
Quoted 16 lines · Laurent Pinchart
Hi Yamada-san,
Quoted 12 lines · Masahiro Yamada
Quoted 11 lines · Laurent Pinchart
Quoted 7 lines · Rob Herring
Most folks only run dt_binding_check on the single schema they care about by setting DT_SCHEMA_FILES. That means example is only checked against that one schema which is not always sufficient.
Let's address this by splitting processed-schema.yaml into 2 files: one that's always all schemas for the examples and one that's just the schema in DT_SCHEMA_FILES for dtbs.
This broke
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
make DT_SCHEMA_FILES=Documentation/devicetree/.. dt_binding_check
What is intended by
"DT_SCHEMA_FILES=Documentation/devicetree/.." ?
My bad, I forgot to write that ... is the continuation of the string. It's any yaml schema file that has an example.
Quoted 7 lines
Ah, OK. I just input verbatim.
Is it broken?
You can specify any individual file(s) under Documentation/devicetree/bindings/.
For example, the following worked for me.
Me too.

Quoted 8 lines

There is however a bug I just noticed. We're now always setting '-u' for dtbs_check which means the core schema are never used.
Masahiro, Got a better suggestion than adding the ifeq:
ifeq ("$(origin DT_SCHEMA_FILES)", "command line") $(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := -u endif

Hmm, this does not work when DT_SCHEMA_FILES is
passed as the environment variable.

ifneq ("$(origin DT_SCHEMA_FILES)", "file") $(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := -u endif

... should work, I think.

Maybe, the following also works. Please choose whatever you like.

(quoted diff) +0 -0

b/Documentation/devicetree/bindings/Makefile
index 1df680d07461..a939bb18f66b 100644

Documentation/devicetree/bindings/Makefile +2 -1

@@ -2,6 +2,7 @@
 DT_DOC_CHECKER ?= dt-doc-validate
 DT_EXTRACT_EX ?= dt-extract-example
 DT_MK_SCHEMA ?= dt-mk-schema
+DT_MK_SCHEMA_USERONLY_FLAG := $(if $(DT_SCHEMA_FILES), -u)
 
 quiet_cmd_chk_binding = CHKDT   $(patsubst $(srctree)/%,%,$<)
       cmd_chk_binding = $(DT_DOC_CHECKER) -u $(srctree)/$(src) $< ; \

@@ -37,7 +38,7 @@ override DTC_FLAGS := \
 $(obj)/processed-schema-examples.yaml: $(DT_DOCS) FORCE
        $(call if_changed,mk_schema)
 
-$(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS := -u
+$(obj)/processed-schema.yaml: DT_MK_SCHEMA_FLAGS :=

$(DT_MK_SCHEMA_USERONLY_FLAG)
$(obj)/processed-schema.yaml: $(DT_SCHEMA_FILES) FORCE

        $(call if_changed,mk_schema)

Signature · 2 lines

Best Regards
Masahiro Yamada