Skip to contentSkip to conversation

perf build: fix "argument list too long" in second location

PATCH Applied Markus Mayer 3 participants 10 messages Apr 9 – Apr 22 perf events

lore
Share & tools
lore
#0 lore raw

cover perf build: fix "argument list too long" in second location

Hi all,

My apologies for not catching this one sooner. Using xargs in the actual build recipe worked, but it turns out echoing $^ in quiet_cmd_rm also offends the shell. Somehow, it didn't show up during my tests before submitting the previous fix[1] for this issue or I would have addressed both together.

But then it did start to show up. Like this:

GEN linux/tools/perf/pmu-events/arch/arm64/arm/neoverse-v3/extra-metricgroups.json
TEST linux/tools/perf/pmu-events/metric_test.log
make[5]: /bin/sh: Argument list too long
make[5]: *** [pmu-events/Build:220: prune_orphans] Error 127

This was a bit of a head-scratcher before I realized that this came from
quiet_cmd_rm = RM $^
and that no attempt of putting quotes around $^ would help.

With change proposed here, it'll look like this instead:

  GEN     linux/tools/perf/pmu-events/arch/arm64/arm/neoverse-v3/extra-metricgroups.json
  TEST    linux/tools/perf/pmu-events/metric_test.log
  RM      ...634 orphan file(s)...
  LD      linux/tools/perf/util/perf-util-in.o

This is probably a good thing, regardless of the shell complaining, since listing 634 files in a so-called "quiet" message is probably not very fitting.

I don't know why my build has 634 orphans every single time it runs (straight-up cross-build on x86_64 for ARM64) or if that is a lot. I do get the impression that it is probably rather unusual. Not really doing anything special, though.

Also, the build machines are running Ubuntu 24.04. As such, /bin/sh is provided by /bin/dash. This may also factor into why I might be seeing this issue while others may not.

Again, sorry for the churn. Hopefully this is it now on the orphan front.

Regards,
-Markus

[1] https://lore.kernel.org/linux-perf-users/[email protected]/

Markus Mayer (1):
perf build: fix "argument list too long" in second location

 tools/perf/pmu-events/Build | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Signature · 1 line
2.53.0
#1 lore raw

PATCH 1/1 perf build: fix "argument list too long" in second location

Turns out that displaying "RM $^" via quiet_cmd_rm can also upset the shell and cause it to display "argument list too long".

Trying to quote $^ doesn't help.

In the end, *not* displaying the (potentially long) list of files is probably the right thing to do for a "quiet" message, anyway. Instead, let's display a count of how many files were removed. There is always V=1 if more detail is required.

  TEST    linux/tools/perf/pmu-events/metric_test.log
  RM      ...634 orphan file(s)...
  LD      linux/tools/perf/util/perf-util-in.o

Also move the comment regarding xargs before the rule, so it doesn't show up in the build output.

---
 tools/perf/pmu-events/Build | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/pmu-events/Build +2 -2
index dc5f94862a3b..dc1df2d57ddc 100644
@@ -211,10 +211,10 @@ ifneq ($(strip $(ORPHAN_FILES)),)
 
 # Message for $(call echo-cmd,rm). Generally cleaning files isn't part
 # of a build step.
-quiet_cmd_rm  = RM      $^
+quiet_cmd_rm = RM      ...$(words $^) orphan file(s)...
 
+# The list of files can be long. Use xargs to prevent issues.
 prune_orphans: $(ORPHAN_FILES)
-	# The list of files can be long. Use xargs to prevent issues.
 	$(Q)$(call echo-cmd,rm)echo "$^" | xargs rm -f
 
 JEVENTS_DEPS += prune_orphans
Signature · 1 line
2.53.0
reviewer Reviewed-by on v1 1/1
#2 lore raw
Quoted 10 lines · Markus Mayer

Turns out that displaying "RM $^" via quiet_cmd_rm can also upset the shell and cause it to display "argument list too long".

Trying to quote $^ doesn't help.

In the end, *not* displaying the (potentially long) list of files is probably the right thing to do for a "quiet" message, anyway. Instead, let's display a count of how many files were removed. There is always V=1 if more detail is required.

Quoted 7 lines
   TEST    linux/tools/perf/pmu-events/metric_test.log
   RM      ...634 orphan file(s)...
   LD      linux/tools/perf/util/perf-util-in.o

Also move the comment regarding xargs before the rule, so it doesn't show up in the build output.

I'm assuming this is just a cosmetic issue so it doesn't need a fixes tag? And "upsetting the shell" doesn't cause a build failure?

Otherwise looks good to me.

Quoted 22 lines
---
  tools/perf/pmu-events/Build | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/pmu-events/Build +2 -2
index dc5f94862a3b..dc1df2d57ddc 100644
@@ -211,10 +211,10 @@ ifneq ($(strip $(ORPHAN_FILES)),)
 
  # Message for $(call echo-cmd,rm). Generally cleaning files isn't part
  # of a build step.
-quiet_cmd_rm  = RM      $^
+quiet_cmd_rm = RM      ...$(words $^) orphan file(s)...
 
+# The list of files can be long. Use xargs to prevent issues.
  prune_orphans: $(ORPHAN_FILES)
-	# The list of files can be long. Use xargs to prevent issues.
  	$(Q)$(call echo-cmd,rm)echo "$^" | xargs rm -f
 
  JEVENTS_DEPS += prune_orphans
#3 lore raw
Quoted 16 lines · James Clark
Quoted 14 lines · Markus Mayer

Turns out that displaying "RM $^" via quiet_cmd_rm can also upset the shell and cause it to display "argument list too long".

Trying to quote $^ doesn't help.

In the end, *not* displaying the (potentially long) list of files is probably the right thing to do for a "quiet" message, anyway. Instead, let's display a count of how many files were removed. There is always V=1 if more detail is required.

   TEST    linux/tools/perf/pmu-events/metric_test.log
   RM      ...634 orphan file(s)...
   LD      linux/tools/perf/util/perf-util-in.o
Quoted 6 lines
Quoted 3 lines

Also move the comment regarding xargs before the rule, so it doesn't show up in the build output.

I'm assuming this is just a cosmetic issue so it doesn't need a fixes tag? And "upsetting the shell" doesn't cause a build failure?

No. It's not just cosmetic. It absolutely DOES cause a build failure.

From the cover letter in this thread:

GEN linux/tools/perf/pmu-events/arch/arm64/arm/neoverse-v3/extra-metricgroups.json
TEST linux/tools/perf/pmu-events/metric_test.log
make[5]: /bin/sh: Argument list too long
make[5]: *** [pmu-events/Build:220: prune_orphans] Error 127

Quoted 3 lines

Otherwise looks good to me.

Thanks!

Regards,
-Markus

Quoted 23 lines
Quoted 22 lines
---
  tools/perf/pmu-events/Build | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/pmu-events/Build +2 -2
index dc5f94862a3b..dc1df2d57ddc 100644
@@ -211,10 +211,10 @@ ifneq ($(strip $(ORPHAN_FILES)),)
 
  # Message for $(call echo-cmd,rm). Generally cleaning files isn't part
  # of a build step.
-quiet_cmd_rm  = RM      $^
+quiet_cmd_rm = RM      ...$(words $^) orphan file(s)...
 
+# The list of files can be long. Use xargs to prevent issues.
  prune_orphans: $(ORPHAN_FILES)
-     # The list of files can be long. Use xargs to prevent issues.
      $(Q)$(call echo-cmd,rm)echo "$^" | xargs rm -f
 
  JEVENTS_DEPS += prune_orphans
reviewer
#4 lore raw
Quoted 24 lines · Markus Mayer
Quoted 16 lines · James Clark
Quoted 14 lines · Markus Mayer

Turns out that displaying "RM $^" via quiet_cmd_rm can also upset the shell and cause it to display "argument list too long".

Trying to quote $^ doesn't help.

In the end, *not* displaying the (potentially long) list of files is probably the right thing to do for a "quiet" message, anyway. Instead, let's display a count of how many files were removed. There is always V=1 if more detail is required.

    TEST    linux/tools/perf/pmu-events/metric_test.log
    RM      ...634 orphan file(s)...
    LD      linux/tools/perf/util/perf-util-in.o
Quoted 6 lines
Quoted 3 lines

Also move the comment regarding xargs before the rule, so it doesn't show up in the build output.

I'm assuming this is just a cosmetic issue so it doesn't need a fixes tag? And "upsetting the shell" doesn't cause a build failure?

Quoted 9 lines

No. It's not just cosmetic. It absolutely DOES cause a build failure.

From the cover letter in this thread:

GEN linux/tools/perf/pmu-events/arch/arm64/arm/neoverse-v3/extra-metricgroups.json
TEST linux/tools/perf/pmu-events/metric_test.log
make[5]: /bin/sh: Argument list too long
make[5]: *** [pmu-events/Build:220: prune_orphans] Error 127

Ah in that case it should have a fixes: tag then. And it's worth putting the actual build failure in the commit message IMO.

Quoted 32 lines
Quoted 3 lines

Otherwise looks good to me.

Thanks!

Regards,
-Markus

Quoted 23 lines
Quoted 22 lines
---
   tools/perf/pmu-events/Build | 4 ++--
   1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/pmu-events/Build +2 -2
index dc5f94862a3b..dc1df2d57ddc 100644
@@ -211,10 +211,10 @@ ifneq ($(strip $(ORPHAN_FILES)),)
 
   # Message for $(call echo-cmd,rm). Generally cleaning files isn't part
   # of a build step.
-quiet_cmd_rm  = RM      $^
+quiet_cmd_rm = RM      ...$(words $^) orphan file(s)...
 
+# The list of files can be long. Use xargs to prevent issues.
   prune_orphans: $(ORPHAN_FILES)
-     # The list of files can be long. Use xargs to prevent issues.
       $(Q)$(call echo-cmd,rm)echo "$^" | xargs rm -f
 
   JEVENTS_DEPS += prune_orphans
#5 lore raw
Quoted 30 lines · James Clark
Quoted 28 lines · Markus Mayer
Quoted 16 lines · James Clark
Quoted 14 lines · Markus Mayer

Turns out that displaying "RM $^" via quiet_cmd_rm can also upset the shell and cause it to display "argument list too long".

Trying to quote $^ doesn't help.

In the end, *not* displaying the (potentially long) list of files is probably the right thing to do for a "quiet" message, anyway. Instead, let's display a count of how many files were removed. There is always V=1 if more detail is required.

    TEST    linux/tools/perf/pmu-events/metric_test.log
    RM      ...634 orphan file(s)...
    LD      linux/tools/perf/util/perf-util-in.o
Quoted 6 lines
Quoted 3 lines

Also move the comment regarding xargs before the rule, so it doesn't show up in the build output.

I'm assuming this is just a cosmetic issue so it doesn't need a fixes tag? And "upsetting the shell" doesn't cause a build failure?

No. It's not just cosmetic. It absolutely DOES cause a build failure.

From the cover letter in this thread:

Quoted 8 lines
Quoted 5 lines

GEN linux/tools/perf/pmu-events/arch/arm64/arm/neoverse-v3/extra-metricgroups.json
TEST linux/tools/perf/pmu-events/metric_test.log
make[5]: /bin/sh: Argument list too long
make[5]: *** [pmu-events/Build:220: prune_orphans] Error 127

Ah in that case it should have a fixes: tag then. And it's worth putting the actual build failure in the commit message IMO.

Unfortunately, it looks like the patch was already applied.[1] Is there anything I (or someone else) can do to fix up that commit message in the way you are suggesting?

Thanks,
-Markus

[1] https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/commit/?h=perf-tools-next&id=97ab89686a9e

maintainer
#6 lore raw

Hello,

Quoted 37 lines · Markus Mayer
Quoted 36 lines · James Clark
Quoted 33 lines · Markus Mayer
Quoted 16 lines · James Clark
Quoted 14 lines · Markus Mayer

Turns out that displaying "RM $^" via quiet_cmd_rm can also upset the shell and cause it to display "argument list too long".

Trying to quote $^ doesn't help.

In the end, *not* displaying the (potentially long) list of files is probably the right thing to do for a "quiet" message, anyway. Instead, let's display a count of how many files were removed. There is always V=1 if more detail is required.

    TEST    linux/tools/perf/pmu-events/metric_test.log
    RM      ...634 orphan file(s)...
    LD      linux/tools/perf/util/perf-util-in.o
Quoted 6 lines
Quoted 3 lines

Also move the comment regarding xargs before the rule, so it doesn't show up in the build output.

I'm assuming this is just a cosmetic issue so it doesn't need a fixes tag? And "upsetting the shell" doesn't cause a build failure?

No. It's not just cosmetic. It absolutely DOES cause a build failure.

From the cover letter in this thread:

GEN linux/tools/perf/pmu-events/arch/arm64/arm/neoverse-v3/extra-metricgroups.json
TEST linux/tools/perf/pmu-events/metric_test.log
make[5]: /bin/sh: Argument list too long
make[5]: *** [pmu-events/Build:220: prune_orphans] Error 127

Quoted 6 lines
Quoted 2 lines

Ah in that case it should have a fixes: tag then. And it's worth putting the actual build failure in the commit message IMO.

Unfortunately, it looks like the patch was already applied.[1] Is there anything I (or someone else) can do to fix up that commit message in the way you are suggesting?

It's already merged to the Linus tree. I don't think we can change that anymore.

Thanks,
Namhyung

#7 lore raw
Quoted 44 lines · Namhyung Kim

Hello,

Quoted 40 lines · Markus Mayer
Quoted 38 lines · James Clark
Quoted 24 lines · Markus Mayer
Quoted 16 lines · James Clark
Quoted 14 lines · Markus Mayer

Turns out that displaying "RM $^" via quiet_cmd_rm can also upset the shell and cause it to display "argument list too long".

Trying to quote $^ doesn't help.

In the end, *not* displaying the (potentially long) list of files is probably the right thing to do for a "quiet" message, anyway. Instead, let's display a count of how many files were removed. There is always V=1 if more detail is required.

    TEST    linux/tools/perf/pmu-events/metric_test.log
    RM      ...634 orphan file(s)...
    LD      linux/tools/perf/util/perf-util-in.o
Quoted 6 lines
Quoted 3 lines

Also move the comment regarding xargs before the rule, so it doesn't show up in the build output.

I'm assuming this is just a cosmetic issue so it doesn't need a fixes tag? And "upsetting the shell" doesn't cause a build failure?

Quoted 9 lines

No. It's not just cosmetic. It absolutely DOES cause a build failure.

From the cover letter in this thread:

GEN linux/tools/perf/pmu-events/arch/arm64/arm/neoverse-v3/extra-metricgroups.json
TEST linux/tools/perf/pmu-events/metric_test.log
make[5]: /bin/sh: Argument list too long
make[5]: *** [pmu-events/Build:220: prune_orphans] Error 127

Ah in that case it should have a fixes: tag then. And it's worth putting the actual build failure in the commit message IMO.

Quoted 6 lines
Quoted 3 lines

Unfortunately, it looks like the patch was already applied.[1] Is there anything I (or someone else) can do to fix up that commit message in the way you are suggesting?

It's already merged to the Linus tree. I don't think we can change that anymore.

Agreed. If Linus already merged it, it is too late. Sorry guys.

Regards,
-Markus

reviewer
#8 lore raw
Quoted 49 lines · Markus Mayer
Quoted 48 lines · Namhyung Kim

Hello,

Quoted 43 lines · Markus Mayer
Quoted 30 lines · James Clark
Quoted 28 lines · Markus Mayer
Quoted 16 lines · James Clark
Quoted 14 lines · Markus Mayer

Turns out that displaying "RM $^" via quiet_cmd_rm can also upset the shell and cause it to display "argument list too long".

Trying to quote $^ doesn't help.

In the end, *not* displaying the (potentially long) list of files is probably the right thing to do for a "quiet" message, anyway. Instead, let's display a count of how many files were removed. There is always V=1 if more detail is required.

     TEST    linux/tools/perf/pmu-events/metric_test.log
     RM      ...634 orphan file(s)...
     LD      linux/tools/perf/util/perf-util-in.o
Quoted 6 lines
Quoted 3 lines

Also move the comment regarding xargs before the rule, so it doesn't show up in the build output.

I'm assuming this is just a cosmetic issue so it doesn't need a fixes tag? And "upsetting the shell" doesn't cause a build failure?

No. It's not just cosmetic. It absolutely DOES cause a build failure.

From the cover letter in this thread:

Quoted 8 lines
Quoted 5 lines

GEN linux/tools/perf/pmu-events/arch/arm64/arm/neoverse-v3/extra-metricgroups.json
TEST linux/tools/perf/pmu-events/metric_test.log
make[5]: /bin/sh: Argument list too long
make[5]: *** [pmu-events/Build:220: prune_orphans] Error 127

Ah in that case it should have a fixes: tag then. And it's worth putting the actual build failure in the commit message IMO.

Unfortunately, it looks like the patch was already applied.[1] Is there anything I (or someone else) can do to fix up that commit message in the way you are suggesting?

Quoted 7 lines
Quoted 2 lines

It's already merged to the Linus tree. I don't think we can change that anymore.

Agreed. If Linus already merged it, it is too late. Sorry guys.

Regards,
-Markus

No worries, it's not that big of a deal

maintainer Applied
#9 lore raw
In reply to Markus Mayer
Quoted 11 lines · Markus Mayer

Turns out that displaying "RM $^" via quiet_cmd_rm can also upset the shell and cause it to display "argument list too long".

Trying to quote $^ doesn't help.

In the end, *not* displaying the (potentially long) list of files is probably the right thing to do for a "quiet" message, anyway. Instead, let's display a count of how many files were removed. There is always V=1 if more detail is required.

[...]

Applied to perf-tools-next, thanks!

Best regards,
Namhyung