kbuild: make clean rule robust against too long argument error
PATCH 1/1 kbuild: make clean rule robust against too long argument error
Commit cd968b97c492 ("kbuild: make built-in.a rule robust against too long argument error") made a build rule robust against "Argument list too long" error.
Eugeniu Rosca reported the same error occurred when cleaning an external module.
The $(obj)/ prefix can be a very long path for external modules.
Apply a similar solution to 'make clean'.
---
scripts/Makefile.clean | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
scripts/Makefile.clean +2 -1
@@ -37,8 +37,9 @@ __clean-files := $(wildcard $(addprefix $(obj)/, $(__clean-files))) # ========================================================================== +# Use xargs to make this robust against "Argument list too long" error quiet_cmd_clean = CLEAN $(obj) - cmd_clean = rm -rf $(__clean-files) + cmd_clean = printf '$(obj)/%s ' $(patsubst $(obj)/%,%,$(__clean-files)) | xargs rm -rf __clean: $(subdir-ymn) ifneq ($(strip $(__clean-files)),)
Signature · 1 line
2.39.2
Hello Yamada-san,
Many thanks for your feedback and for your patch. It indeed addresses the issue I have reported in [1].
Quoted 8 lines · Masahiro Yamada
Commit
cd968b97c492("kbuild: make built-in.a rule robust against too long argument error") made a build rule robust against "Argument list too long" error.Eugeniu Rosca reported the same error occurred when cleaning an external module.
The $(obj)/ prefix can be a very long path for external modules.
Confirmed. I am seeing an instance of $(obj) being 150 characters long, due to an out-of-tree module deeply buried in a specific Yocto build.
In the current vanilla version of 'make clean' (w/o this patch), the $(obj) prefix is applied to each and every file being removed, dramatically increasing the strlen of arguments passed to 'rm -rf'.
Quoted 15 lines
Apply a similar solution to 'make clean'.
Reported-by Eugeniu Rosca <[email protected]>Signed-off-by Masahiro Yamada <[email protected]>---
scripts/Makefile.clean | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)scripts/Makefile.clean +0
-0
scripts/Makefile.clean __clean-files := $(wildcard $(addprefix $(obj)/, $(__clean-files))) · line 40
Quoted 4 lines
scripts/Makefile.clean +1
-0@@ -37,8 +37,9 @@ __clean-files := $(wildcard $(addprefix $(obj)/, $(__clean-files))) # ========================================================================== +# Use xargs to make this robust against "Argument list too long" error
Please, correct me if I am wrong, but it looks like the magic/brilliance is in the 'patsubst' function, since below version also fails for me:
NOK: cmd_clean = printf '%s ' $(__clean-files) | xargs rm -rf
When running 'make clean' for my particular out-of-tree kernel module, 'patsubst' is able to decrease the total number of characters passed to the shell's argument list from ~204k to ~47k, which elegantly prevents the "Argument list too long" error.
Quoted 9 lines
scripts/Makefile.clean +1
-1quiet_cmd_clean = CLEAN $(obj) - cmd_clean = rm -rf $(__clean-files) + cmd_clean = printf '$(obj)/%s ' $(patsubst $(obj)/%,%,$(__clean-files)) | xargs rm -rf __clean: $(subdir-ymn) ifneq ($(strip $(__clean-files)),)Signature · 1 line
2.39.2
Since it fixes the problem reported in [1] (much appreciated):
[1] https://lore.kernel.org/linux-kbuild/20230616194505.GA27753@lxhi-065/
Signature · 2 lines
Best regards, Eugeniu Rosca
Quoted 39 lines · Eugeniu Rosca
Hello Yamada-san,
Many thanks for your feedback and for your patch. It indeed addresses the issue I have reported in [1].
Quoted 8 lines · Masahiro Yamada
Commit
cd968b97c492("kbuild: make built-in.a rule robust against too long argument error") made a build rule robust against "Argument list too long" error.Eugeniu Rosca reported the same error occurred when cleaning an external module.
The $(obj)/ prefix can be a very long path for external modules.
Confirmed. I am seeing an instance of $(obj) being 150 characters long, due to an out-of-tree module deeply buried in a specific Yocto build.
In the current vanilla version of 'make clean' (w/o this patch), the $(obj) prefix is applied to each and every file being removed, dramatically increasing the strlen of arguments passed to 'rm -rf'.
Quoted 16 lines
Apply a similar solution to 'make clean'.
Reported-by Eugeniu Rosca <[email protected]>Signed-off-by Masahiro Yamada <[email protected]>---
scripts/Makefile.clean | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)scripts/Makefile.clean +0
-0@@ -37,8 +37,9 @@ __clean-files := $(wildcard $(addprefix $(obj)/, $(__clean-files)))
Quoted 8 lines
scripts/Makefile.clean __clean-files := $(wildcard $(addprefix $(obj)/, $(__clean-files))) · line 40
Quoted 3 lines
scripts/Makefile.clean +1
-0# ========================================================================== +# Use xargs to make this robust against "Argument list too long" errorPlease, correct me if I am wrong, but it looks like the magic/brilliance is in the 'patsubst' function, since below version also fails for me:
NOK:
cmd_clean= printf '%s ' $(__clean-files) | xargs rm -rf
Right.
Now, 'printf' (instead of 'rm') failed with the too long argument list.
GNU Make does not have the length limit, but shell does. So, the full-path list must be passed via stdout instead of the command line.
The comment might be confusing.
I will repeat the same comment written in scripts/Makefile.build:
# To make this rule robust against "Argument list too long" error,
# remove $(obj)/ prefix, and restore it by a shell command.
Signature · 2 lines
Best Regards Masahiro Yamada
Hello Yamada-san,
Quoted 1 line · Masahiro Yamada
On Sun, Jun 18, 2023 at 5:07 AM Eugeniu Rosca <[email protected]> wrote:
[snip]
Quoted 13 lines
Quoted 4 lines
Please, correct me if I am wrong, but it looks like the magic/brilliance is in the 'patsubst' function, since below version also fails for me:
NOK:
cmd_clean= printf '%s ' $(__clean-files) | xargs rm -rfRight.
Now, 'printf' (instead of 'rm') failed with the too long argument list.GNU Make does not have the length limit, but shell does. So, the full-path list must be passed via stdout instead of the command line.
Quoted 6 lines
The comment might be confusing.
I will repeat the same comment written in
scripts/Makefile.build:# To make this rule robust against "Argument list too long" error,
# remove $(obj)/ prefix, and restore it by a shell command.
Sounds good to me. No open questions from my end :)
Signature · 2 lines
Best regards, Eugeniu Rosca