dt-bindings: Fix command line length limit calling dt-mk-schema
Rob Herring
maintainerPATCH v2 1/1 dt-bindings: Fix command line length limit calling dt-mk-schema
As the number of schemas has increased, we're starting to hit the error "execvp: /bin/sh: Argument list too long". This is due to passing all the schema files on the command line to dt-mk-schema. It currently is only with out of tree builds and is intermittent depending on the file path lengths.
Commit 2ba06cd8565b ("kbuild: Always validate DT binding examples") made hitting this proplem more likely since the example validation now always gets the full list of schemas.
Fix this by passing the schema file list in a pipe and using xargs. We end up doing the find twice, but the time is insignificant compared to the dt-mk-schema time.
--- Documentation/devicetree/bindings/Makefile | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
Documentation/devicetree/bindings/Makefile +10 -8
@@ -14,16 +14,18 @@ $(obj)/%.example.dts: $(src)/%.yaml FORCE # Use full schemas when checking %.example.dts DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml +find_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \ + -name 'processed-schema*' ! \ + -name '*.example.dt.yaml' \) + quiet_cmd_mk_schema = SCHEMA $@ - cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs) + cmd_mk_schema = rm -f $@ ; \ + $(if $(DT_MK_SCHEMA_FLAGS), \ + echo $(real-prereqs), \ + $(find_cmd)) | \ + xargs $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) >> $@ -DT_DOCS = $(addprefix $(src)/, \ - $(shell \ - cd $(srctree)/$(src) && \ - find * \( -name '*.yaml' ! \ - -name 'processed-schema*' ! \ - -name '*.example.dt.yaml' \) \ - )) +DT_DOCS = $(shell $(find_cmd) | sed -e 's|^$(srctree)/||') DT_SCHEMA_FILES ?= $(DT_DOCS)
Signature · 1 line
2.20.1
Quoted 11 lines · Rob Herring
As the number of schemas has increased, we're starting to hit the error "execvp:
/bin/sh: Argument list too long". This is due to passing all the schema files on the command line to dt-mk-schema. It currently is only with out of tree builds and is intermittent depending on the file path lengths.Commit
2ba06cd8565b("kbuild: Always validate DT binding examples") made hitting this proplem more likely since the example validation now always gets the full list of schemas.
Quoted 7 lines
Fix this by passing the schema file list in a pipe and using xargs. We end up doing the find twice, but the time is insignificant compared to the dt-mk-schema time.
Reported-by Laurent Pinchart <[email protected]>Cc Masahiro Yamada <[email protected]>Signed-off-by Rob Herring <[email protected]>
Quoted 38 lines
--- Documentation/devicetree/bindings/Makefile | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)Documentation/devicetree/bindings/Makefile +10
-9@@ -14,16 +14,18 @@ $(obj)/%.example.dts: $(src)/%.yaml FORCE # Use full schemas when checking %.example.dts DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml +find_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \ + -name 'processed-schema*' ! \ + -name '*.example.dt.yaml' \) + quiet_cmd_mk_schema = SCHEMA $@ - cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs) + cmd_mk_schema = rm -f $@ ; \ + $(if $(DT_MK_SCHEMA_FLAGS), \ + echo $(real-prereqs), \ + $(find_cmd)) | \ + xargs $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) >> $@ -DT_DOCS = $(addprefix $(src)/, \ - $(shell \ - cd $(srctree)/$(src) && \ - find * \( -name '*.yaml' ! \ - -name 'processed-schema*' ! \ - -name '*.example.dt.yaml' \) \ - )) +DT_DOCS = $(shell $(find_cmd) | sed -e 's|^$(srctree)/||') DT_SCHEMA_FILES ?= $(DT_DOCS) --2.20.1
Signature · 2 lines
Best Regards Masahiro Yamada
Hi Rob,
Thank you for the patch.
Quoted 10 lines · Rob Herring
As the number of schemas has increased, we're starting to hit the error "execvp:
/bin/sh: Argument list too long". This is due to passing all the schema files on the command line to dt-mk-schema. It currently is only with out of tree builds and is intermittent depending on the file path lengths.Commit
2ba06cd8565b("kbuild: Always validate DT binding examples") made hitting this proplem more likely since the example validation now always gets the full list of schemas.
Quoted 7 lines
Fix this by passing the schema file list in a pipe and using xargs. We end up doing the find twice, but the time is insignificant compared to the dt-mk-schema time.
Reported-by Laurent Pinchart <[email protected]>Cc Masahiro Yamada <[email protected]>Signed-off-by Rob Herring <[email protected]>
Quoted 35 lines
--- Documentation/devicetree/bindings/Makefile | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)Documentation/devicetree/bindings/Makefile +10
-8@@ -14,16 +14,18 @@ $(obj)/%.example.dts: $(src)/%.yaml FORCE # Use full schemas when checking %.example.dts DT_TMP_SCHEMA := $(obj)/processed-schema-examples.yaml +find_cmd = find $(srctree)/$(src) \( -name '*.yaml' ! \ + -name 'processed-schema*' ! \ + -name '*.example.dt.yaml' \) + quiet_cmd_mk_schema = SCHEMA $@ - cmd_mk_schema = $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) -o $@ $(real-prereqs) + cmd_mk_schema = rm -f $@ ; \ + $(if $(DT_MK_SCHEMA_FLAGS), \ + echo $(real-prereqs), \ + $(find_cmd)) | \ + xargs $(DT_MK_SCHEMA) $(DT_MK_SCHEMA_FLAGS) >> $@ -DT_DOCS = $(addprefix $(src)/, \ - $(shell \ - cd $(srctree)/$(src) && \ - find * \( -name '*.yaml' ! \ - -name 'processed-schema*' ! \ - -name '*.example.dt.yaml' \) \ - )) +DT_DOCS = $(shell $(find_cmd) | sed -e 's|^$(srctree)/||') DT_SCHEMA_FILES ?= $(DT_DOCS)
Signature · 3 lines
Regards, Laurent Pinchart