tools build: propagate build failures from tools/build/Makefile.build
Subsystems (3)
PATCH 1/1 tools build: propagate build failures from tools/build/Makefile.build
The '&&' command seems to have a bad effect when $(cmd_$(1)) exits with non-zero effect: the command failure is masked (despite set -e) and all but the first command of $(dep-cmd) is executed (successfully, as they are mostly printfs), thus overall returning 0 in the end.
This means in practice that despite compilation errors, tools's build Makefile will return success. We see this very reliably with libbpf's Makefile, which doesn't get compilation error propagated properly. This in turns causes issues with selftests build, as well as bpftool and other projects that rely on building libbpf.
The fix is simple: don't use &&. Given set -e, we don't need to chain commands with &&. The shell will exit on first failure, giving desired behavior and propagating error properly.
---
I'm sending this against bpf-next tree, given libbpf is affected enough for me to debug this fun problem that no one seemed to notice (or care, at least) in almost 5 years. If there is a better kernel tree, please let me know.
tools/build/Build.include | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tools/build/Build.include +2 -1
@@ -74,7 +74,8 @@ dep-cmd = $(if $(wildcard $(fixdep)), # dependencies in the cmd file if_changed_dep = $(if $(strip $(any-prereq) $(arg-check)), \ @set -e; \ - $(echo-cmd) $(cmd_$(1)) && $(dep-cmd)) + $(echo-cmd) $(cmd_$(1)); \ + $(dep-cmd)) # if_changed - execute command if any prerequisite is newer than # target, or command line has changed
Signature · 1 line
2.24.1
Quoted 12 lines · Andrii Nakryiko
The '&&' command seems to have a bad effect when $(cmd_$(1)) exits with non-zero effect: the command failure is masked (despite
set -e) and all but the first command of $(dep-cmd) is executed (successfully, as they are mostly printfs), thus overall returning 0 in the end.This means in practice that despite compilation errors, tools's build
Makefilewill return success. We see this very reliably with libbpf'sMakefile, which doesn't get compilation error propagated properly. This in turns causes issues with selftests build, as well as bpftool and other projects that rely on building libbpf.
Quoted 8 lines
The fix is simple: don't use &&. Given
set -e, we don't need to chain commands with &&. The shell will exit on first failure, giving desired behavior and propagating error properly.Cc Jiri Olsa <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Fixes 275e2d95591e ("tools build: Move dependency copy into function")Signed-off-by Andrii Nakryiko <[email protected]>
Jiri, Arnaldo,
could you please review and ack?
Quoted 4 lines · Andrii Nakryiko
The '&&' command seems to have a bad effect when $(cmd_$(1)) exits with non-zero effect: the command failure is masked (despite
set -e) and all but the first command of $(dep-cmd) is executed (successfully, as they are mostly printfs), thus overall returning 0 in the end.
nice, thanks for digging into this,
any idea why is the failure masked?
jirka
Quoted 41 lines
This means in practice that despite compilation errors, tools's build
Makefilewill return success. We see this very reliably with libbpf'sMakefile, which doesn't get compilation error propagated properly. This in turns causes issues with selftests build, as well as bpftool and other projects that rely on building libbpf.The fix is simple: don't use &&. Given
set -e, we don't need to chain commands with &&. The shell will exit on first failure, giving desired behavior and propagating error properly.Cc Jiri Olsa <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Fixes 275e2d95591e ("tools build: Move dependency copy into function")Signed-off-by Andrii Nakryiko <[email protected]>---
I'm sending this against bpf-next tree, given libbpf is affected enough for me to debug this fun problem that no one seemed to notice (or care, at least) in almost 5 years. If there is a better kernel tree, please let me know.
tools/build/Build.include | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)tools/build/Build.include +2
-1@@ -74,7 +74,8 @@ dep-cmd = $(if $(wildcard $(fixdep)), # dependencies in the cmd file if_changed_dep = $(if $(strip $(any-prereq) $(arg-check)), \ @set -e; \ - $(echo-cmd) $(cmd_$(1)) && $(dep-cmd)) + $(echo-cmd) $(cmd_$(1)); \ + $(dep-cmd)) # if_changed - execute command if any prerequisite is newer than # target, or command line has changedSignature · 1 line
2.24.1
Quoted 9 lines · Jiri Olsa
Quoted 4 lines · Andrii Nakryiko
The '&&' command seems to have a bad effect when $(cmd_$(1)) exits with non-zero effect: the command failure is masked (despite
set -e) and all but the first command of $(dep-cmd) is executed (successfully, as they are mostly printfs), thus overall returning 0 in the end.nice, thanks for digging into this,
any idea why is the failure masked?
Two things.
1. In make, assume you have command f = a in one function and g = b; c in another. If you write f && g, you end up with (a && b); c, right?
2. Try this shell script:
set -ex
false && true
true
It will return success. It won't execute the first true command, as expected, but won't terminate the shell as you'd expect from set -e.
So basically, having a "logical operator" in a sequence of commands negates the effect of set -e. Intuitively I'd expect that from ||, but seems like && does that as well. if [] has similar effect -- any failing command in an if check doesn't trigger an early termination of a script.
Quoted 47 lines
Acked-by Jiri Olsa <[email protected]>jirka
Quoted 41 lines
This means in practice that despite compilation errors, tools's build
Makefilewill return success. We see this very reliably with libbpf'sMakefile, which doesn't get compilation error propagated properly. This in turns causes issues with selftests build, as well as bpftool and other projects that rely on building libbpf.The fix is simple: don't use &&. Given
set -e, we don't need to chain commands with &&. The shell will exit on first failure, giving desired behavior and propagating error properly.Cc Jiri Olsa <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Fixes 275e2d95591e ("tools build: Move dependency copy into function")Signed-off-by Andrii Nakryiko <[email protected]>---
I'm sending this against bpf-next tree, given libbpf is affected enough for me to debug this fun problem that no one seemed to notice (or care, at least) in almost 5 years. If there is a better kernel tree, please let me know.
tools/build/Build.include | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)tools/build/Build.include +2
-2@@ -74,7 +74,8 @@ dep-cmd = $(if $(wildcard $(fixdep)), # dependencies in the cmd file if_changed_dep = $(if $(strip $(any-prereq) $(arg-check)), \ @set -e; \ - $(echo-cmd) $(cmd_$(1)) && $(dep-cmd)) + $(echo-cmd) $(cmd_$(1)); \ + $(dep-cmd)) # if_changed - execute command if any prerequisite is newer than # target, or command line has changed --2.24.1
Quoted 22 lines · Andrii Nakryiko
Quoted 9 lines · Jiri Olsa
Quoted 4 lines · Andrii Nakryiko
The '&&' command seems to have a bad effect when $(cmd_$(1)) exits with non-zero effect: the command failure is masked (despite
set -e) and all but the first command of $(dep-cmd) is executed (successfully, as they are mostly printfs), thus overall returning 0 in the end.nice, thanks for digging into this,
any idea why is the failure masked?Two things.
1. In make, assume you have command f = a in one function and g = b; c in another. If you write f && g, you end up with (a && b); c, right?
2. Try this shell script:
set -ex
false && true
true
Quoted 8 lines
It will return success. It won't execute the first true command, as expected, but won't terminate the shell as you'd expect from set -e.
So basically, having a "logical operator" in a sequence of commands negates the effect of
set -e. Intuitively I'd expect that from ||, but seems like && does that as well. if [] has similar effect -- any failing command in an if check doesn't trigger an early termination of a script.
nice, thanks for explanation
jirka
Quoted 49 lines
Quoted 48 lines
Acked-by Jiri Olsa <[email protected]>jirka
Quoted 41 lines
This means in practice that despite compilation errors, tools's build
Makefilewill return success. We see this very reliably with libbpf'sMakefile, which doesn't get compilation error propagated properly. This in turns causes issues with selftests build, as well as bpftool and other projects that rely on building libbpf.The fix is simple: don't use &&. Given
set -e, we don't need to chain commands with &&. The shell will exit on first failure, giving desired behavior and propagating error properly.Cc Jiri Olsa <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Fixes 275e2d95591e ("tools build: Move dependency copy into function")Signed-off-by Andrii Nakryiko <[email protected]>---
I'm sending this against bpf-next tree, given libbpf is affected enough for me to debug this fun problem that no one seemed to notice (or care, at least) in almost 5 years. If there is a better kernel tree, please let me know.
tools/build/Build.include | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)tools/build/Build.include +2
-2@@ -74,7 +74,8 @@ dep-cmd = $(if $(wildcard $(fixdep)), # dependencies in the cmd file if_changed_dep = $(if $(strip $(any-prereq) $(arg-check)), \ @set -e; \ - $(echo-cmd) $(cmd_$(1)) && $(dep-cmd)) + $(echo-cmd) $(cmd_$(1)); \ + $(dep-cmd)) # if_changed - execute command if any prerequisite is newer than # target, or command line has changed --2.24.1
Quoted 11 lines · Andrii Nakryiko
The '&&' command seems to have a bad effect when $(cmd_$(1)) exits with non-zero effect: the command failure is masked (despite
set -e) and all but the first command of $(dep-cmd) is executed (successfully, as they are mostly printfs), thus overall returning 0 in the end.This means in practice that despite compilation errors, tools's build
Makefilewill return success. We see this very reliably with libbpf'sMakefile, which doesn't get compilation error propagated properly. This in turns causes issues with selftests build, as well as bpftool and other projects that rely on building libbpf.
Quoted 8 lines
The fix is simple: don't use &&. Given
set -e, we don't need to chain commands with &&. The shell will exit on first failure, giving desired behavior and propagating error properly.Cc Jiri Olsa <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Fixes 275e2d95591e ("tools build: Move dependency copy into function")Signed-off-by Andrii Nakryiko <[email protected]>
Applied, thanks!