package/linux-tools: Fix parallel compilation
PATCH v2 1/1 package/linux-tools: Fix parallel compilation
linux-tools opportunistically set linux as only a patch dependency. This unfortunately introduces a race condition in the shared linux scripts when using PER_PACKAGE_DIRECTORIES and using top level parallelism. The race manifests as the error "/bin/sh: 1: scripts/basic/fixdep:
Permission denied". This happens when the linux package and the linux-tools package are being compiled in parallel.
The linux-tools currently using fixdep are perf and rtla. When the timing is correct, perf, rtla, or the kernel Makefile will try to use fixdep while one of the others is compiling fixed, resulting in fixdep being briefly not available.
To fix this, set linux as a build dependency instead of a patch dependency.
---
While the timing of this seems like it should very rarely be triggered,
on my build server I hit this issue nearly 100% of the time.
---
Changes in v2:
- Added more details to how/why the race occurs
- Link to v1: https://lore.kernel.org/r/[email protected]
--- package/linux-tools/linux-tools.mk | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
package/linux-tools/linux-tools.mk +6 -2
@@ -19,8 +19,12 @@ include $(sort $(wildcard package/linux-tools/*.mk.in)) -# We only need the kernel to be extracted, not actually built -LINUX_TOOLS_PATCH_DEPENDENCIES = linux +# Prevent the kernel from being compiled in parallel with linux-tools. The +# kernel and some linux-tools (such as perf) use fixdep to manage kconfig +# dependencies. There is a race condition where the kernel and linux-tools try +# to build fixdep while the other tries to use fixdep, resulting in the use of +# fixdep failing. +LINUX_TOOLS_DEPENDENCIES = linux # Install Linux kernel tools in the staging directory since some tools # may install shared libraries and headers (e.g. cpupower).
---
base-commit: 7336c5fa93a896130511d661f0d42359a046ec9a
change-id: 20250409-fix_linux_tools_parallel-2323829caf82
Signature · 6 lines
- Charlie _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 24 lines · On Mon, 21 Apr 2025 23:36:59 -0700 Charlie Jenkins <[email protected]> wrote:
linux-tools opportunistically set linux as only a patch dependency. This unfortunately introduces a race condition in the shared linux scripts when using
PER_PACKAGE_DIRECTORIESand using top level parallelism. The race manifests as the error "/bin/sh: 1:scripts/basic/fixdep:
Permission denied". This happens when the linux package and the linux-tools package are being compiled in parallel.The linux-tools currently using fixdep are perf and rtla. When the timing is correct, perf, rtla, or the kernel
Makefilewill try to use fixdep while one of the others is compiling fixed, resulting in fixdep being briefly not available.To fix this, set linux as a build dependency instead of a patch dependency.
Signed-off-by Charlie Jenkins <[email protected]>Reviewed-by Yann E. MORIN <[email protected]>---
While the timing of this seems like it should very rarely be triggered,
on my build server I hit this issue nearly 100% of the time.
---
Changes in v2:
- Added more details to how/why the race occurs
- Link to v1: https://lore.kernel.org/r/[email protected]
Quoted 3 lines
--- package/linux-tools/linux-tools.mk | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
Nice commit message, thanks a lot! Applied.
Thomas
Signature · 7 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 12 lines · Charlie Jenkins
linux-tools opportunistically set linux as only a patch dependency. This unfortunately introduces a race condition in the shared linux scripts when using
PER_PACKAGE_DIRECTORIESand using top level parallelism. The race manifests as the error "/bin/sh: 1:scripts/basic/fixdep:
Permission denied". This happens when the linux package and the linux-tools package are being compiled in parallel.The linux-tools currently using fixdep are perf and rtla. When the timing is correct, perf, rtla, or the kernel
Makefilewill try to use fixdep while one of the others is compiling fixed, resulting in fixdep being briefly not available.
Quoted 5 lines
To fix this, set linux as a build dependency instead of a patch dependency.
Signed-off-by Charlie Jenkins <[email protected]>Reviewed-by Yann E. MORIN <[email protected]>
Applied to 2025.02.x, thanks.
Regards,
Arnout
Quoted 32 lines
---
While the timing of this seems like it should very rarely be triggered,
on my build server I hit this issue nearly 100% of the time.
---
Changes in v2:
- Added more details to how/why the race occurs
- Link to v1: https://lore.kernel.org/r/[email protected]--- package/linux-tools/linux-tools.mk | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)package/linux-tools/linux-tools.mk +6
-3@@ -19,8 +19,12 @@ include $(sort $(wildcard package/linux-tools/*.mk.in)) -# We only need the kernel to be extracted, not actually built -LINUX_TOOLS_PATCH_DEPENDENCIES = linux +# Prevent the kernel from being compiled in parallel with linux-tools. The +# kernel and some linux-tools (such as perf) use fixdep to manage kconfig +# dependencies. There is a race condition where the kernel and linux-tools try +# to build fixdep while the other tries to use fixdep, resulting in the use of +# fixdep failing. +LINUX_TOOLS_DEPENDENCIES = linux # Install Linux kernel tools in the staging directory since some tools # may install shared libraries and headers (e.g. cpupower). ---
Quoted 2 lines
base-commit:
7336c5fa93a896130511d661f0d42359a046ec9a
change-id: 20250409-fix_linux_tools_parallel-2323829caf82
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot