perf python: Fix splitting CC into compiler and options
Arnaldo Carvalho de Melo
maintainerPATCH 1/1 perf python: Fix splitting CC into compiler and options
Just fyi, I'm carrying this in the perf tools tree.
- Arnaldo
----
Noticed this build failure on archlinux:base when building with clang:
clang-14: error: optimization flag '-ffat-lto-objects' is not supported [-Werror,-Wignored-optimization-argument]
In tools/perf/util/setup.py we check if clang supports that option, but since commit 3cad53a6f9cdbafa ("perf python: Account for multiple words in CC") this got broken as in the common case where CC="clang":
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
cc="clang"
print(cc.split()[0])
clang
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
option="-ffat-lto-objects"
print(str(cc.split()[1:]) + option)
[]-ffat-lto-objects
Quoted 1 line
Quoted 1 line
Quoted 1 line
And then the Popen will call clang with that bogus option name that in turn will not produce the b"unknown argument" or b"is not supported" that this function uses to detect if the option is not available and thus later on clang will be called with an unknown/unsupported option.
Fix it by looking if really there are options in the provided CC variable, and if so override 'cc' with the first token and append the options to the 'option' variable.
--- tools/perf/util/setup.py | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
tools/perf/util/setup.py +11 -2
@@ -3,11 +3,20 @@ from subprocess import Popen, PIPE from re import sub cc = getenv("CC") -cc_is_clang = b"clang version" in Popen([cc.split()[0], "-v"], stderr=PIPE).stderr.readline() + +# Check if CC has options, as is the case in yocto, where it uses CC="cc --sysroot..." +cc_tokens = cc.split() +if len(cc_tokens) > 1: + cc = cc_tokens[0] + cc_options = " ".join([str(e) for e in cc_tokens[1:]]) + " " +else: + cc_options = "" + +cc_is_clang = b"clang version" in Popen([cc, "-v"], stderr=PIPE).stderr.readline() src_feature_tests = getenv('srctree') + '/tools/build/feature' def clang_has_option(option): - cc_output = Popen([cc.split()[0], str(cc.split()[1:]) + option, path.join(src_feature_tests, "test-hello.c") ], stderr=PIPE).stderr.readlines() + cc_output = Popen([cc, cc_options + option, path.join(src_feature_tests, "test-hello.c") ], stderr=PIPE).stderr.readlines() return [o for o in cc_output if ((b"unknown argument" in o) or (b"is not supported" in o))] == [ ] if cc_is_clang:
Signature · 1 line
2.38.1
Quoted 56 lines · Arnaldo Carvalho de Melo
Just fyi, I'm carrying this in the perf tools tree.
- Arnaldo
----
Noticed this build failure on archlinux:base when building with clang:
clang-14: error: optimization flag '-ffat-lto-objects' is not supported [-Werror,-Wignored-optimization-argument]
In
tools/perf/util/setup.pywe check if clang supports that option, but since commit3cad53a6f9cdbafa("perf python: Account for multiple words in CC") this got broken as in the common case where CC="clang":Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
cc="clang"
print(cc.split()[0])clang
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
option="-ffat-lto-objects"
print(str(cc.split()[1:]) + option)[]-ffat-lto-objects
Quoted 1 line
Quoted 1 line
Quoted 1 line
And then the Popen will call clang with that bogus option name that in turn will not produce the b"unknown argument" or b"is not supported" that this function uses to detect if the option is not available and thus later on clang will be called with an unknown/unsupported option.
Fix it by looking if really there are options in the provided CC variable, and if so override 'cc' with the first token and append the options to the 'option' variable.
Fixes 3cad53a6f9cdbafa ("perf python: Account for multiple words in CC")Cc Adrian Hunter <[email protected]>Cc Fangrui Song <[email protected]>Cc Florian Fainelli <[email protected]>Cc Ian Rogers <[email protected]>Cc Jiri Olsa <[email protected]>Cc John Keeping <[email protected]>Cc Khem Raj <[email protected]>Cc Leo Yan <[email protected]>Cc Michael Petlan <[email protected]>Cc Namhyung Kim <[email protected]>Cc Nathan Chancellor <[email protected]>Cc Nick Desaulniers <[email protected]>Cc Sedat Dilek <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>--- tools/perf/util/setup.py | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)tools/perf/util/setup.py +0
-0
tools/perf/util/setup.py from subprocess import Popen, PIPE · line 10
Quoted 9 lines
tools/perf/util/setup.py +5
-1@@ -3,11 +3,20 @@ from subprocess import Popen, PIPE from re import sub cc = getenv("CC") -cc_is_clang = b"clang version" in Popen([cc.split()[0], "-v"], stderr=PIPE).stderr.readline() + +# Check if CC has options, as is the case in yocto, where it uses CC="cc --sysroot..." +cc_tokens = cc.split() +if len(cc_tokens) > 1: + cc = cc_tokens[0]
What if someone is using CC="cache clang"? Then cc is set to cache and the cc_is_clang check below will fail.
Quoted 16 lines
tools/perf/util/setup.py +6
-2+ cc_options = " ".join([str(e) for e in cc_tokens[1:]]) + " " +else: + cc_options = "" + +cc_is_clang = b"clang version" in Popen([cc, "-v"], stderr=PIPE).stderr.readline() src_feature_tests = getenv('srctree') + '/tools/build/feature' def clang_has_option(option): - cc_output = Popen([cc.split()[0], str(cc.split()[1:]) + option, path.join(src_feature_tests, "test-hello.c") ], stderr=PIPE).stderr.readlines() + cc_output = Popen([cc, cc_options + option, path.join(src_feature_tests, "test-hello.c") ], stderr=PIPE).stderr.readlines() return [o for o in cc_output if ((b"unknown argument" in o) or (b"is not supported" in o))] == [ ] if cc_is_clang: --2.38.1
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 61 lines · Nick Desaulniers
Quoted 58 lines · Arnaldo Carvalho de Melo
Just fyi, I'm carrying this in the perf tools tree.
- Arnaldo
----
Noticed this build failure on archlinux:base when building with clang:
clang-14: error: optimization flag '-ffat-lto-objects' is not supported [-Werror,-Wignored-optimization-argument]
In
tools/perf/util/setup.pywe check if clang supports that option, but since commit3cad53a6f9cdbafa("perf python: Account for multiple words in CC") this got broken as in the common case where CC="clang":Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
cc="clang"
print(cc.split()[0])clang
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
option="-ffat-lto-objects"
print(str(cc.split()[1:]) + option)[]-ffat-lto-objects
Quoted 1 line
Quoted 1 line
Quoted 1 line
And then the Popen will call clang with that bogus option name that in turn will not produce the b"unknown argument" or b"is not supported" that this function uses to detect if the option is not available and thus later on clang will be called with an unknown/unsupported option.
Fix it by looking if really there are options in the provided CC variable, and if so override 'cc' with the first token and append the options to the 'option' variable.
Fixes 3cad53a6f9cdbafa ("perf python: Account for multiple words in CC")Cc Adrian Hunter <[email protected]>Cc Fangrui Song <[email protected]>Cc Florian Fainelli <[email protected]>Cc Ian Rogers <[email protected]>Cc Jiri Olsa <[email protected]>Cc John Keeping <[email protected]>Cc Khem Raj <[email protected]>Cc Leo Yan <[email protected]>Cc Michael Petlan <[email protected]>Cc Namhyung Kim <[email protected]>Cc Nathan Chancellor <[email protected]>Cc Nick Desaulniers <[email protected]>Cc Sedat Dilek <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>--- tools/perf/util/setup.py | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)tools/perf/util/setup.py +0
-0@@ -3,11 +3,20 @@ from subprocess import Popen, PIPE from re import sub
Quoted 10 lines
tools/perf/util/setup.py from subprocess import Popen, PIPE · line 10
Quoted 7 lines
tools/perf/util/setup.py +5
-1cc = getenv("CC") -cc_is_clang = b"clang version" in Popen([cc.split()[0], "-v"], stderr=PIPE).stderr.readline() + +# Check if CC has options, as is the case in yocto, where it uses CC="cc --sysroot..." +cc_tokens = cc.split() +if len(cc_tokens) > 1: + cc = cc_tokens[0]What if someone is using
CC="cache clang"? Then cc is set tocacheand thecc_is_clangcheck below will fail.
s/cache/ccache/g
Quoted 22 lines
Quoted 15 lines
tools/perf/util/setup.py +6
-2+ cc_options = " ".join([str(e) for e in cc_tokens[1:]]) + " " +else: + cc_options = "" + +cc_is_clang = b"clang version" in Popen([cc, "-v"], stderr=PIPE).stderr.readline() src_feature_tests = getenv('srctree') + '/tools/build/feature' def clang_has_option(option): - cc_output = Popen([cc.split()[0], str(cc.split()[1:]) + option, path.join(src_feature_tests, "test-hello.c") ], stderr=PIPE).stderr.readlines() + cc_output = Popen([cc, cc_options + option, path.join(src_feature_tests, "test-hello.c") ], stderr=PIPE).stderr.readlines() return [o for o in cc_output if ((b"unknown argument" in o) or (b"is not supported" in o))] == [ ] if cc_is_clang: --Quoted 2 lines
2.38.1
--
Thanks,
~Nick Desaulniers
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 60 lines · Nick Desaulniers
Quoted 58 lines · Arnaldo Carvalho de Melo
Just fyi, I'm carrying this in the perf tools tree.
- Arnaldo
----
Noticed this build failure on archlinux:base when building with clang:
clang-14: error: optimization flag '-ffat-lto-objects' is not supported [-Werror,-Wignored-optimization-argument]
In
tools/perf/util/setup.pywe check if clang supports that option, but since commit3cad53a6f9cdbafa("perf python: Account for multiple words in CC") this got broken as in the common case where CC="clang":Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
cc="clang"
print(cc.split()[0])clang
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
option="-ffat-lto-objects"
print(str(cc.split()[1:]) + option)[]-ffat-lto-objects
Quoted 1 line
Quoted 1 line
Quoted 1 line
And then the Popen will call clang with that bogus option name that in turn will not produce the b"unknown argument" or b"is not supported" that this function uses to detect if the option is not available and thus later on clang will be called with an unknown/unsupported option.
Fix it by looking if really there are options in the provided CC variable, and if so override 'cc' with the first token and append the options to the 'option' variable.
Fixes 3cad53a6f9cdbafa ("perf python: Account for multiple words in CC")Cc Adrian Hunter <[email protected]>Cc Fangrui Song <[email protected]>Cc Florian Fainelli <[email protected]>Cc Ian Rogers <[email protected]>Cc Jiri Olsa <[email protected]>Cc John Keeping <[email protected]>Cc Khem Raj <[email protected]>Cc Leo Yan <[email protected]>Cc Michael Petlan <[email protected]>Cc Namhyung Kim <[email protected]>Cc Nathan Chancellor <[email protected]>Cc Nick Desaulniers <[email protected]>Cc Sedat Dilek <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>--- tools/perf/util/setup.py | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)tools/perf/util/setup.py +0
-0@@ -3,11 +3,20 @@ from subprocess import Popen, PIPE from re import sub
Quoted 10 lines
tools/perf/util/setup.py from subprocess import Popen, PIPE · line 10
Quoted 7 lines
tools/perf/util/setup.py +5
-1cc = getenv("CC") -cc_is_clang = b"clang version" in Popen([cc.split()[0], "-v"], stderr=PIPE).stderr.readline() + +# Check if CC has options, as is the case in yocto, where it uses CC="cc --sysroot..." +cc_tokens = cc.split() +if len(cc_tokens) > 1: + cc = cc_tokens[0]What if someone is using
CC="cache clang"? Then cc is set tocacheand thecc_is_clangcheck below will fail.
Agreed, but this is a preexisting bug, let's fix this with a follow-up patch,
- Arnaldo
Quoted 19 lines
Quoted 17 lines
tools/perf/util/setup.py +5
-2+ cc_options = " ".join([str(e) for e in cc_tokens[1:]]) + " " +else: + cc_options = "" +cc_is_clang = b"clang version" in Popen([cc, "-v"], stderr=PIPE).stderr.readline() src_feature_tests = getenv('srctree') + '/tools/build/feature' def clang_has_option(option): - cc_output = Popen([cc.split()[0], str(cc.split()[1:]) + option, path.join(src_feature_tests, "test-hello.c") ], stderr=PIPE).stderr.readlines() + cc_output = Popen([cc, cc_options + option, path.join(src_feature_tests, "test-hello.c") ], stderr=PIPE).stderr.readlines() return [o for o in cc_output if ((b"unknown argument" in o) or (b"is not supported" in o))] == [ ] if cc_is_clang: --2.38.1
Quoted 75 lines · Arnaldo Carvalho de Melo
Quoted 60 lines · Nick Desaulniers
Quoted 58 lines · Arnaldo Carvalho de Melo
Just fyi, I'm carrying this in the perf tools tree.
- Arnaldo
----
Noticed this build failure on archlinux:base when building with clang:
clang-14: error: optimization flag '-ffat-lto-objects' is not supported [-Werror,-Wignored-optimization-argument]
In
tools/perf/util/setup.pywe check if clang supports that option, but since commit3cad53a6f9cdbafa("perf python: Account for multiple words in CC") this got broken as in the common case where CC="clang":Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
cc="clang"
print(cc.split()[0])clang
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
option="-ffat-lto-objects"
print(str(cc.split()[1:]) + option)[]-ffat-lto-objects
Quoted 1 line
Quoted 1 line
Quoted 1 line
And then the Popen will call clang with that bogus option name that in turn will not produce the b"unknown argument" or b"is not supported" that this function uses to detect if the option is not available and thus later on clang will be called with an unknown/unsupported option.
Fix it by looking if really there are options in the provided CC variable, and if so override 'cc' with the first token and append the options to the 'option' variable.
Fixes 3cad53a6f9cdbafa ("perf python: Account for multiple words in CC")Cc Adrian Hunter <[email protected]>Cc Fangrui Song <[email protected]>Cc Florian Fainelli <[email protected]>Cc Ian Rogers <[email protected]>Cc Jiri Olsa <[email protected]>Cc John Keeping <[email protected]>Cc Khem Raj <[email protected]>Cc Leo Yan <[email protected]>Cc Michael Petlan <[email protected]>Cc Namhyung Kim <[email protected]>Cc Nathan Chancellor <[email protected]>Cc Nick Desaulniers <[email protected]>Cc Sedat Dilek <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>--- tools/perf/util/setup.py | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)tools/perf/util/setup.py +0
-0@@ -3,11 +3,20 @@ from subprocess import Popen, PIPE from re import subQuoted 10 lines
tools/perf/util/setup.py from subprocess import Popen, PIPE · line 10
Quoted 7 lines
tools/perf/util/setup.py +5
-1cc = getenv("CC") -cc_is_clang = b"clang version" in Popen([cc.split()[0], "-v"], stderr=PIPE).stderr.readline() + +# Check if CC has options, as is the case in yocto, where it uses CC="cc --sysroot..." +cc_tokens = cc.split() +if len(cc_tokens) > 1: + cc = cc_tokens[0]What if someone is using
CC="cache clang"? Then cc is set tocacheand thecc_is_clangcheck below will fail.
Quoted 1 line
Agreed, but this is a preexisting bug, let's fix this with a follow-up patch,
Ack; it is.
Quoted 23 lines
- Arnaldo
Quoted 19 lines
Quoted 17 lines
tools/perf/util/setup.py +5
-2+ cc_options = " ".join([str(e) for e in cc_tokens[1:]]) + " " +else: + cc_options = "" +cc_is_clang = b"clang version" in Popen([cc, "-v"], stderr=PIPE).stderr.readline() src_feature_tests = getenv('srctree') + '/tools/build/feature' def clang_has_option(option): - cc_output = Popen([cc.split()[0], str(cc.split()[1:]) + option, path.join(src_feature_tests, "test-hello.c") ], stderr=PIPE).stderr.readlines() + cc_output = Popen([cc, cc_options + option, path.join(src_feature_tests, "test-hello.c") ], stderr=PIPE).stderr.readlines() return [o for o in cc_output if ((b"unknown argument" in o) or (b"is not supported" in o))] == [ ] if cc_is_clang: --2.38.1
Signature · 2 lines
Thanks, ~Nick Desaulniers