-fcanon-file-prefix and cc-rs compiler flag mixing
cover -fcanon-file-prefix and cc-rs compiler flag mixing
These two commits are very losely related to each other.
Patch 1 adds -fcanon-prefix-map to the compiler flags, when it is applicable (when gcc >=13 is used). This helps with finding the correct debug sources
for the -src package. Without this flag some recipes miss some source
files from this package, and sometimes they miss all files. (E.g. sqlite, audiofile, librsvg)
Patch 2 tries to work around a bug (or at least unexpected behavior) in the cc-rs crate.
This patch tries to recognize if there are any flags passed to the compiler that shouldn't be there.
How these patches are related: normally the *-prefix-map flags are only passed to cross-compiled targets. However the cc crate passes this flag to the native builds also (due to flag merging), and in case the host compiler is too old to recognize this flag, than the build fails.
Though I was able to build rust with both of these patches for both native and class targets, I wasn't able to configure qemu to run in my docker env, and due to this I was unable to verify if the selftests pass in an OS where gcc is older than v13.
While not mentioned explicitly, but especially with the 2nd patch I'm testing the waters, it is an RFC also.
---
Gyorgy Sarvari (2):
bitbake.conf, gcc toolchain: add -fcanon-prefix-map to map-prefixes rust-common.bbclass: filter out incorrect compiler flags in wrapper
meta/classes-recipe/rust-common.bbclass | 24 +++++++++++++++++++++++ meta/classes/toolchain/gcc-native.bbclass | 3 +++ meta/classes/toolchain/gcc.bbclass | 2 ++ meta/conf/bitbake.conf | 2 +- 4 files changed, 30 insertions(+), 1 deletion(-)
PATCH 1/2 bitbake.conf, gcc toolchain: add -fcanon-prefix-map to map-prefixes
This patch adds -fcanon-prefix-map to the list of compile file-prefixes list in case at least gcc 13 is used (it's a gcc-only flag).
This flag used to be part of this list in the past, but was removed with [1].
Since then the source file paths are not canonicalized, which makes the system to miss some (and sometimes all) source files to include in the corresponding -src packages. As an example sqlite3 and audiofile (from meta-oe) produces empty src packages, mostly due to the pathes:
they frequently use relative paths that don't match up with the absolute path specified in the file-preifx map:pwd/../../foo.cpp and ${S}/foo.cpp might refer to the same file, but the first one won't match the prefix-map, so it is omitted.
This patch adds this option again with gcc: for class-target it is uses always, and for class-native it is used when the host gcc is at least version 13.
--- meta/classes/toolchain/gcc-native.bbclass | 3 +++ meta/classes/toolchain/gcc.bbclass | 2 ++ meta/conf/bitbake.conf | 2 +- 3 files changed, 6 insertions(+), 1 deletion(-)
meta/classes/toolchain/gcc-native.bbclass +3 -0
@@ -13,3 +13,6 @@ BUILD_OBJDUMP = "${BUILD_PREFIX}objdump" BUILD_NM = "${BUILD_PREFIX}nm" BUILD_READELF = "${BUILD_PREFIX}readelf" +# gcc started to support -fcanon-prefix-map from version 13 +DEBUG_PREFIX_MAP_EXTRA:class-native = \ + "${@'-fcanon-prefix-map' if bb.utils.vercmp_string_op(oe.utils.get_host_gcc_version(d), '13.0.0', '>=') else ''}"
meta/classes/toolchain/gcc.bbclass +2 -0
@@ -30,4 +30,6 @@ PREFERRED_PROVIDER_virtual/nativesdk-cross-cc:class-cross-canadian = "gcc-crosss PREFERRED_PROVIDER_virtual/nativesdk-cross-c++:class-cross-canadian = "gcc-crosssdk-${SDK_SYS}" PREFERRED_PROVIDER_virtual/nativesdk-compilerlibs:class-cross-canadian = "nativesdk-gcc-runtime" +DEBUG_PREFIX_MAP_EXTRA = "-fcanon-prefix-map" + TCOVERRIDE = "toolchain-gcc"
meta/conf/bitbake.conf +1 -1
@@ -650,7 +650,7 @@ EXTRA_OEMAKE:prepend:task-install = "${PARALLEL_MAKEINST} " ################################################################## TARGET_DBGSRC_DIR ?= "/usr/src/debug/${PN}/${PV}" # Beware: applied last to first -DEBUG_PREFIX_MAP ?= "\ +DEBUG_PREFIX_MAP ?= "${DEBUG_PREFIX_MAP_EXTRA} \ -ffile-prefix-map=${S}=${TARGET_DBGSRC_DIR} \ -ffile-prefix-map=${B}=${TARGET_DBGSRC_DIR} \ -ffile-prefix-map=${STAGING_DIR_HOST}= \
Re: [OE-core] [PATCH 1/2] bitbake.conf, gcc toolchain: add -fcanon-prefix-map to map-prefixes
Looks good to me. Covers all cases that I brought up.
Quoted 82 lines · Gyorgy Sarvari via lists.openembedded.org
This patch adds -fcanon-prefix-map to the list of compile file-prefixes list in case at least gcc 13 is used (it's a gcc-only flag).
This flag used to be part of this list in the past, but was removed with [1].
Since then the source file paths are not canonicalized, which makes the system to miss some (and sometimes all) source files to include in the corresponding -src packages. As an example sqlite3 and audiofile (from meta-oe) produces empty src packages, mostly due to the pathes:
they frequently use relative paths that don't match up with the absolute path specified in the file-preifx map:pwd/../../foo.cpp and ${S}/foo.cpp might refer to the same file, but the first one won't match the prefix-map, so it is omitted.This patch adds this option again with gcc: for class-target it is uses always, and for class-native it is used when the host gcc is at least version 13.
Signed-off-by Gyorgy Sarvari <[email protected]>--- meta/classes/toolchain/gcc-native.bbclass | 3 +++ meta/classes/toolchain/gcc.bbclass | 2 ++ meta/conf/bitbake.conf | 2 +- 3 files changed, 6 insertions(+), 1 deletion(-)meta/classes/toolchain/gcc-native.bbclass +3
-0@@ -13,3 +13,6 @@ BUILD_OBJDUMP = "${BUILD_PREFIX}objdump" BUILD_NM = "${BUILD_PREFIX}nm" BUILD_READELF = "${BUILD_PREFIX}readelf" +# gcc started to support -fcanon-prefix-map from version 13 +DEBUG_PREFIX_MAP_EXTRA:class-native = \ + "${@'-fcanon-prefix-map' if
bb.utils.vercmp_string_op(oe.utils.get_host_gcc_version(d), '13.0.0', '>=')else ''}"meta/classes/toolchain/gcc.bbclass +0
-0@@ -30,4 +30,6 @@PREFERRED_PROVIDER_virtual/nativesdk-cross-cc:class-cross-canadian = "gcc-crosss
PREFERRED_PROVIDER_virtual/nativesdk-cross-c++:class-cross-canadian =
"gcc-crosssdk-${SDK_SYS}"
PREFERRED_PROVIDER_virtual/nativesdk-compilerlibs:class-cross-canadian =
"nativesdk-gcc-runtime"meta/classes/toolchain/gcc.bbclass +2
-0+DEBUG_PREFIX_MAP_EXTRA = "-fcanon-prefix-map" + TCOVERRIDE = "toolchain-gcc"meta/conf/bitbake.conf +0
-0@@ -650,7 +650,7 @@ EXTRA_OEMAKE:prepend:task-install ="${PARALLEL_MAKEINST} "
##################################################################
TARGET_DBGSRC_DIR?= "/usr/src/debug/${PN}/${PV}" # Beware: applied last to firstmeta/conf/bitbake.conf +1
-2-DEBUG_PREFIX_MAP ?= "\ +DEBUG_PREFIX_MAP ?= "${DEBUG_PREFIX_MAP_EXTRA} \ -ffile-prefix-map=${S}=${TARGET_DBGSRC_DIR} \ -ffile-prefix-map=${B}=${TARGET_DBGSRC_DIR} \ -ffile-prefix-map=${STAGING_DIR_HOST}= \ -=-=-=-=-=-=-=-=-=-=-=-Links: You receive all messages sent to this group. View/Reply Online (#226618):
https://lists.openembedded.org/g/openembedded-core/message/226618
Mute This Topic: https://lists.openembedded.org/mt/116389619/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/2] bitbake.conf, gcc toolchain: add -fcanon-prefix-map to map-prefixes
In reply to Gyorgy SarvariQuoted 57 lines · Gyorgy Sarvari via lists.openembedded.org
This patch adds -fcanon-prefix-map to the list of compile file-prefixes list in case at least gcc 13 is used (it's a gcc-only flag).
This flag used to be part of this list in the past, but was removed with [1].
Since then the source file paths are not canonicalized, which makes the system to miss some (and sometimes all) source files to include in the corresponding -src packages. As an example sqlite3 and audiofile (from meta-oe) produces empty src packages, mostly due to the pathes:
they frequently use relative paths that don't match up with the absolute path specified in the file-preifx map:pwd/../../foo.cpp and ${S}/foo.cpp might refer to the same file, but the first one won't match the prefix-map, so it is omitted.This patch adds this option again with gcc: for class-target it is uses always, and for class-native it is used when the host gcc is at least version 13.
Signed-off-by Gyorgy Sarvari <[email protected]>---
meta/classes/toolchain/gcc-native.bbclass | 3 +++ meta/classes/toolchain/gcc.bbclass | 2 ++ meta/conf/bitbake.conf | 2 +-3 files changed, 6 insertions(+), 1 deletion(-)meta/classes/toolchain/gcc-native.bbclass +0
-0@@ -13,3 +13,6 @@ BUILD_OBJDUMP = "${BUILD_PREFIX}objdump"BUILD_NM = "${BUILD_PREFIX}nm"
BUILD_READELF = "${BUILD_PREFIX}readelf"meta/classes/toolchain/gcc-native.bbclass +3
-0+# gcc started to support -fcanon-prefix-map from version 13 +DEBUG_PREFIX_MAP_EXTRA:class-native = \ + "${@'-fcanon-prefix-map' if bb.utils.vercmp_string_op(oe.utils.get_host_gcc_version(d), '13.0.0', '>=') else ''}"meta/classes/toolchain/gcc.bbclass +0
-0@@ -30,4 +30,6 @@ PREFERRED_PROVIDER_virtual/nativesdk-cross-cc:class-cross-canadian = "gcc-crosssPREFERRED_PROVIDER_virtual/nativesdk-cross-c++:class-cross-canadian = "gcc-crosssdk-${SDK_SYS}" PREFERRED_PROVIDER_virtual/nativesdk-compilerlibs:class-cross-canadian = "nativesdk-gcc-runtime"
meta/classes/toolchain/gcc.bbclass +2
-0+DEBUG_PREFIX_MAP_EXTRA = "-fcanon-prefix-map" +TCOVERRIDE = "toolchain-gcc"
meta/conf/bitbake.conf +0
-0@@ -650,7 +650,7 @@ EXTRA_OEMAKE:prepend:task-install = "${PARALLEL_MAKEINST} " ##################################################################TARGET_DBGSRC_DIR ?= "
/usr/src/debug/${PN}/${PV}" # Beware: applied last to firstmeta/conf/bitbake.conf +1
-1-DEBUG_PREFIX_MAP ?= "\ +DEBUG_PREFIX_MAP ?= "${DEBUG_PREFIX_MAP_EXTRA} \
Quoted 4 lines
-ffile-prefix-map=${S}=${TARGET_DBGSRC_DIR} \
-ffile-prefix-map=${B}=${TARGET_DBGSRC_DIR} \
-ffile-prefix-map=${STAGING_DIR_HOST}= \
I think we should do something like this patch but just with only the target pieces, not the native ones.
The gcc version check is a slow function call since it execs a command and we really don't want a call like that dropped inside a variable.
native components don't have -src packages so I'm not sure we need that element of this. The simplest thing would be to drop that piece.
This issue was identified as a release blocker so I'll probably hack the patch and put something into master-next for testing overnight.
Cheers,
Richard
Re: [OE-core] [PATCH 1/2] bitbake.conf, gcc toolchain: add -fcanon-prefix-map to map-prefixes
Quoted 64 lines · Richard Purdie via lists.openembedded.org
Quoted 4 lines · Gyorgy Sarvari via lists.openembedded.org
This patch adds -fcanon-prefix-map to the list of compile file-prefixes list in case at least gcc 13 is used (it's a gcc-only flag).
This flag used to be part of this list in the past, but was removed with
[1].
Quoted 14 lines
Since then the source file paths are not canonicalized, which makes the system to miss some (and sometimes all) source files to include in the corresponding -src packages. As an example sqlite3 and audiofile (from meta-oe) produces empty src packages, mostly due to the pathes:
they frequently use relative paths that don't match up with the absolute path specified in the file-preifx map:pwd/../../foo.cpp and ${S}/foo.cpp might refer to the same file, but the first one won't match the prefix-map, so it is omitted.This patch adds this option again with gcc: for class-target it is uses always, and for class-native it is used when the host gcc is at least version 13.
Review
Quoted 8 lines
Signed-off-by Gyorgy Sarvari <[email protected]>--- meta/classes/toolchain/gcc-native.bbclass | 3 +++ meta/classes/toolchain/gcc.bbclass | 2 ++ meta/conf/bitbake.conf | 2 +- 3 files changed, 6 insertions(+), 1 deletion(-)(quoted diff) +0
-0b/meta/classes/toolchain/gcc-native.bbclass
meta/classes/toolchain/gcc-native.bbclass BUILD_OBJDUMP = "${BUILD_PREFIX}objdump" · line 18
Quoted 10 lines
index
a708bd0389..8708ad0211100644meta/classes/toolchain/gcc-native.bbclass +3
-0@@ -13,3 +13,6 @@ BUILD_OBJDUMP = "${BUILD_PREFIX}objdump" BUILD_NM = "${BUILD_PREFIX}nm" BUILD_READELF = "${BUILD_PREFIX}readelf" +# gcc started to support -fcanon-prefix-map from version 13 +DEBUG_PREFIX_MAP_EXTRA:class-native = \ + "${@'-fcanon-prefix-map' if
bb.utils.vercmp_string_op(oe.utils.get_host_gcc_version(d), '13.0.0', '>=')else ''}"Review
Quoted 1 line
(quoted diff) +0
-0b/meta/classes/toolchain/gcc.bbclass
meta/classes/toolchain/gcc.bbclass
Quoted 4 lines
index
a5adb5ca37..0ed49ba892100644meta/classes/toolchain/gcc.bbclass +0
-0@@ -30,4 +30,6 @@PREFERRED_PROVIDER_virtual/nativesdk-cross-cc:class-cross-canadian = "gcc-crosss
meta/classes/toolchain/gcc.bbclass line 30
Quoted 1 line
meta/classes/toolchain/gcc.bbclass +0
-0PREFERRED_PROVIDER_virtual/nativesdk-cross-c++:class-cross-canadian ="gcc-crosssdk-${SDK_SYS}"
meta/classes/toolchain/gcc.bbclass line 31
Quoted 1 line
meta/classes/toolchain/gcc.bbclass +0
-0PREFERRED_PROVIDER_virtual/nativesdk-compilerlibs:class-cross-canadian= "nativesdk-gcc-runtime"
meta/conf/bitbake.conf EXTRA_OEMAKE:prepend:task-install =
Quoted 9 lines
meta/classes/toolchain/gcc.bbclass +2
-0+DEBUG_PREFIX_MAP_EXTRA = "-fcanon-prefix-map" + TCOVERRIDE = "toolchain-gcc"meta/conf/bitbake.conf +0
-0@@ -650,7 +650,7 @@ EXTRA_OEMAKE:prepend:task-install ="${PARALLEL_MAKEINST} "
Quoted 13 lines
meta/conf/bitbake.conf EXTRA_OEMAKE:prepend:task-install = "${PARALLEL_MAKEINST} " · line 656
Quoted 9 lines
meta/conf/bitbake.conf +1
-1################################################################## TARGET_DBGSRC_DIR ?= "/usr/src/debug/${PN}/${PV}" # Beware: applied last to first -DEBUG_PREFIX_MAP ?= "\ +DEBUG_PREFIX_MAP ?= "${DEBUG_PREFIX_MAP_EXTRA} \ -ffile-prefix-map=${S}=${TARGET_DBGSRC_DIR} \ -ffile-prefix-map=${B}=${TARGET_DBGSRC_DIR} \ -ffile-prefix-map=${STAGING_DIR_HOST}= \I think we should do something like this patch but just with only the target pieces, not the native ones.
cc-rs is perhaps the problematic bit, if thats sorted with the second patch then I agree, we can keep it from target alone.
Quoted 25 lines
The gcc version check is a slow function call since it execs a command and we really don't want a call like that dropped inside a variable.
native components don't have -src packages so I'm not sure we need that element of this. The simplest thing would be to drop that piece.
This issue was identified as a release blocker so I'll probably hack the patch and put something into master-next for testing overnight.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#226641):
https://lists.openembedded.org/g/openembedded-core/message/226641
Mute This Topic: https://lists.openembedded.org/mt/116389619/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/2] bitbake.conf, gcc toolchain: add -fcanon-prefix-map to map-prefixes
In reply to Richard PurdiePlease try the v2 of the cc-rs patch - I see that v1 got into master-next, but it will fail selftests.
Quoted 77 lines · Richard Purdie
Quoted 57 lines · Gyorgy Sarvari via lists.openembedded.org
This patch adds -fcanon-prefix-map to the list of compile file-prefixes list in case at least gcc 13 is used (it's a gcc-only flag).
This flag used to be part of this list in the past, but was removed with [1].
Since then the source file paths are not canonicalized, which makes the system to miss some (and sometimes all) source files to include in the corresponding -src packages. As an example sqlite3 and audiofile (from meta-oe) produces empty src packages, mostly due to the pathes:
they frequently use relative paths that don't match up with the absolute path specified in the file-preifx map:pwd/../../foo.cpp and ${S}/foo.cpp might refer to the same file, but the first one won't match the prefix-map, so it is omitted.This patch adds this option again with gcc: for class-target it is uses always, and for class-native it is used when the host gcc is at least version 13.
Signed-off-by Gyorgy Sarvari <[email protected]>---
meta/classes/toolchain/gcc-native.bbclass | 3 +++ meta/classes/toolchain/gcc.bbclass | 2 ++ meta/conf/bitbake.conf | 2 +-3 files changed, 6 insertions(+), 1 deletion(-)meta/classes/toolchain/gcc-native.bbclass +0
-0@@ -13,3 +13,6 @@ BUILD_OBJDUMP = "${BUILD_PREFIX}objdump"BUILD_NM = "${BUILD_PREFIX}nm"
BUILD_READELF = "${BUILD_PREFIX}readelf"meta/classes/toolchain/gcc-native.bbclass +3
-0+# gcc started to support -fcanon-prefix-map from version 13 +DEBUG_PREFIX_MAP_EXTRA:class-native = \ + "${@'-fcanon-prefix-map' if bb.utils.vercmp_string_op(oe.utils.get_host_gcc_version(d), '13.0.0', '>=') else ''}"meta/classes/toolchain/gcc.bbclass +0
-0@@ -30,4 +30,6 @@ PREFERRED_PROVIDER_virtual/nativesdk-cross-cc:class-cross-canadian = "gcc-crosssPREFERRED_PROVIDER_virtual/nativesdk-cross-c++:class-cross-canadian = "gcc-crosssdk-${SDK_SYS}" PREFERRED_PROVIDER_virtual/nativesdk-compilerlibs:class-cross-canadian = "nativesdk-gcc-runtime"
meta/classes/toolchain/gcc.bbclass +2
-0+DEBUG_PREFIX_MAP_EXTRA = "-fcanon-prefix-map" +TCOVERRIDE = "toolchain-gcc"
meta/conf/bitbake.conf +0
-0@@ -650,7 +650,7 @@ EXTRA_OEMAKE:prepend:task-install = "${PARALLEL_MAKEINST} " ##################################################################TARGET_DBGSRC_DIR ?= "
/usr/src/debug/${PN}/${PV}" # Beware: applied last to firstmeta/conf/bitbake.conf +1
-1-DEBUG_PREFIX_MAP ?= "\ +DEBUG_PREFIX_MAP ?= "${DEBUG_PREFIX_MAP_EXTRA} \Quoted 4 lines
-ffile-prefix-map=${S}=${TARGET_DBGSRC_DIR} \
-ffile-prefix-map=${B}=${TARGET_DBGSRC_DIR} \
-ffile-prefix-map=${STAGING_DIR_HOST}= \I think we should do something like this patch but just with only the target pieces, not the native ones.
The gcc version check is a slow function call since it execs a command and we really don't want a call like that dropped inside a variable.
native components don't have -src packages so I'm not sure we need that element of this. The simplest thing would be to drop that piece.
This issue was identified as a release blocker so I'll probably hack the patch and put something into master-next for testing overnight.
Cheers,
Richard
Re: [OE-core] [PATCH 1/2] bitbake.conf, gcc toolchain: add -fcanon-prefix-map to map-prefixes
I think we also need weak assignment DEBUG_PREFIX_MAP_EXTRA to empty in bitbake.conf, because the unexpanded variable with clang instead of gcc (with v2 from ML as well as the version currently in master-next) causes:
meta-oe/meta-oe/recipes-extended/ostree/ostree_2024.10.bb: Error during parse shell code, the last 5 lines are:
fi
else
bbfatal "no configure script found at $cfgscript"
fi
WARNING: meta-oe/meta-oe/recipes-extended/ostree/ostree_2024.10.bb:
Exception during build_dependencies for oe_runconf
WARNING: meta-oe/meta-oe/recipes-extended/ostree/ostree_2024.10.bb:
Error during finalise of meta-oe/meta-oe/recipes-extended/ostree/ostree_2024.10.bb
ERROR: Unable to parse meta-oe/meta-oe/recipes-extended/ostree/ostree_2024.10.bb Traceback (most recent call last):
File "bitbake/lib/bb/cooker.py", line 2312, in parse_next
raise result
bb.pysh.pyshlex.NeedMore
ERROR: Parsing halted due to errors, see error messages above
Quoted 93 lines · Gyorgy Sarvari via lists.openembedded.org
Please try the v2 of the cc-rs patch - I see that v1 got into master-next, but it will fail selftests.
Quoted 71 lines · Richard Purdie
Quoted 53 lines · Gyorgy Sarvari via lists.openembedded.org
This patch adds -fcanon-prefix-map to the list of compile file-prefixes list in case at least gcc 13 is used (it's a gcc-only flag).
This flag used to be part of this list in the past, but was removed with [1].
Since then the source file paths are not canonicalized, which makes the system to miss some (and sometimes all) source files to include in the corresponding -src packages. As an example sqlite3 and audiofile (from meta-oe) produces empty src packages, mostly due to the pathes:
they frequently use relative paths that don't match up with the absolute path specified in the file-preifx map:pwd/../../foo.cpp and ${S}/foo.cpp might refer to the same file, but the first one won't match the prefix-map, so it is omitted.This patch adds this option again with gcc: for class-target it is uses always, and for class-native it is used when the host gcc is at least version 13.
Signed-off-by Gyorgy Sarvari <[email protected]>--- meta/classes/toolchain/gcc-native.bbclass | 3 +++ meta/classes/toolchain/gcc.bbclass | 2 ++ meta/conf/bitbake.conf | 2 +- 3 files changed, 6 insertions(+), 1 deletion(-)meta/classes/toolchain/gcc-native.bbclass +3
-0@@ -13,3 +13,6 @@ BUILD_OBJDUMP = "${BUILD_PREFIX}objdump" BUILD_NM = "${BUILD_PREFIX}nm" BUILD_READELF = "${BUILD_PREFIX}readelf" +# gcc started to support -fcanon-prefix-map from version 13 +DEBUG_PREFIX_MAP_EXTRA:class-native = \ + "${@'-fcanon-prefix-map' if bb.utils.vercmp_string_op(oe.utils.get_host_gcc_version(d), '13.0.0', '>=') else ''}"meta/classes/toolchain/gcc.bbclass +2
-0@@ -30,4 +30,6 @@ PREFERRED_PROVIDER_virtual/nativesdk-cross-cc:class-cross-canadian = "gcc-crosss PREFERRED_PROVIDER_virtual/nativesdk-cross-c++:class-cross-canadian = "gcc-crosssdk-${SDK_SYS}" PREFERRED_PROVIDER_virtual/nativesdk-compilerlibs:class-cross-canadian = "nativesdk-gcc-runtime" +DEBUG_PREFIX_MAP_EXTRA = "-fcanon-prefix-map" + TCOVERRIDE = "toolchain-gcc"meta/conf/bitbake.conf +0
-0meta/conf/bitbake.conf EXTRA_OEMAKE:prepend:task-install = "${PARALLEL_MAKEINST} " · line 656
Quoted 8 lines
meta/conf/bitbake.conf +1
-1@@ -650,7 +650,7 @@ EXTRA_OEMAKE:prepend:task-install = "${PARALLEL_MAKEINST} " ################################################################## TARGET_DBGSRC_DIR ?= "/usr/src/debug/${PN}/${PV}" # Beware: applied last to first -DEBUG_PREFIX_MAP ?= "\ +DEBUG_PREFIX_MAP ?= "${DEBUG_PREFIX_MAP_EXTRA} \ -ffile-prefix-map=${S}=${TARGET_DBGSRC_DIR} \ -ffile-prefix-map=${B}=${TARGET_DBGSRC_DIR} \ -ffile-prefix-map=${STAGING_DIR_HOST}= \I think we should do something like this patch but just with only the target pieces, not the native ones.
The gcc version check is a slow function call since it execs a command and we really don't want a call like that dropped inside a variable.
native components don't have -src packages so I'm not sure we need that element of this. The simplest thing would be to drop that piece.
Quoted 6 lines
This issue was identified as a release blocker so I'll probably hack the patch and put something into master-next for testing overnight.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#226652): https://lists.openembedded.org/g/openembedded-core/message/226652
Mute This Topic: https://lists.openembedded.org/mt/116389619/3617156
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
PATCH 2/2 rust-common.bbclass: filter out incorrect compiler flags in wrapper
In reply to Gyorgy SarvariThis patch is a workaround for https://bugzilla.yoctoproject.org/show_bug.cgi?id=15976
cc-rs crate is used by a number of projects, including rust's bootstrap also to invoke the systems c/c++ compiler.
A few updates ago it has changed the way CFLAGS/CXXFLAGS are handled: it now merges them with HOST_C*FLAGS and TARGET_C*FLAGS.
This is a problem when a recipe is cross compiled, but it has a build dependency which uses this crate to be built. In this case the C*FLAGS variable contains target flags, while the HOST_C*FLAGS contains host-specific flags, and when the two are mixed, the output is not what one expects.
This change tries to filter out the incorrect flags:
- If the wrapper is invoked as a c or c++ compiler wrapper,
- then determines if it compiles for host or for target
- Depending on the on above, it considers the TARGET_*FLAGS or HOST*FLAGS correct, and the C*FLAGS variable content incorrect.
- It subtracts the correct set from the incorrect set, and drops the remainder from the compiler flags. (Which might be often an empty list, so the flags are frequently unchanged)
--- meta/classes-recipe/rust-common.bbclass | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
meta/classes-recipe/rust-common.bbclass +23 -0
@@ -148,6 +148,29 @@ create_wrapper_rust () { binary = orig_binary.split()[0] args = orig_binary.split() + sys.argv[1:] + + # The following is trying to be a workaround for + # https://bugzilla.yoctoproject.org/show_bug.cgi?id=15976 + # cc-rs crate passes both host AND target flags at the same time + # to this script, in case a recipe is cross-compiled, but a crate + # is a build dependency, and is compiled for the host. + # This tries to filter out the inappropriate flags. + if sys.argv[0].endswith("-cc"): + script_type = "C" + elif sys.argv[0].endswith("-cxx"): + script_type = "CXX" + else: + script_type = "other" + + flags_to_remove = [] + if script_type != "other": + host_or_target = "HOST" if "build-rust-" in sys.argv[0] else "TARGET" + incorrect_flags = os.getenv("%sFLAGS" % script_type).split() + correct_flags = os.getenv("%s_%sFLAGS" % (host_or_target, script_type)).split() + flags_to_remove = set(incorrect_flags) - set(correct_flags) + + args = list(filter(lambda flag: flag not in flags_to_remove, args)) + if extras: args.append(extras) os.execvp(binary, args)
Re: [OE-core] [PATCH 2/2] rust-common.bbclass: filter out incorrect compiler flags in wrapper
Quoted 22 lines · Gyorgy Sarvari via lists.openembedded.org
This patch is a workaround for https://bugzilla.yoctoproject.org/show_bug.cgi?id=15976
cc-rs crate is used by a number of projects, including rust's bootstrap also to invoke the systems c/c++ compiler.
A few updates ago it has changed the way CFLAGS/CXXFLAGS are handled: it now merges them with HOST_C*FLAGS and TARGET_C*FLAGS.
This is a problem when a recipe is cross compiled, but it has a build dependency which uses this crate to be built. In this case the C*FLAGS variable contains target flags, while the HOST_C*FLAGS contains host-specific flags, and when the two are mixed, the output is not what one expects.
This change tries to filter out the incorrect flags:
- If the wrapper is invoked as a c or c++ compiler wrapper,
- then determines if it compiles for host or for target
- Depending on the on above, it considers the TARGET_*FLAGS or HOST*FLAGS correct, and the C*FLAGS variable content incorrect.
- It subtracts the correct set from the incorrect set, and drops the remainder from the compiler flags. (Which might be often an empty list, so the flags are frequently unchanged)
Quoted 2 lines
Signed-off-by Gyorgy Sarvari <[email protected]>---
Hi Gyorgy,
Thanks for your patch.
I suspect one of the commit of this series is responsible of this rust test failure:
2025-11-20 21:59:48,098 - oe-selftest - INFO - FAIL: rust.RustSelfTestSystemEmulated.test_rust (subunit.RemotedTestCase)
...
File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/rust.py", line 131, in test_rust
retval = runCmd(cmd)
File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'export TARGET_VENDOR="-poky"; export PATH=/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/python3-native:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/hosttools:$PATH; export RUST_TARGET_PATH=/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/rust-targets; export RUSTFLAGS='-C strip=debuginfo'; export TEST_DEVICE_ADDR="192.168.7.6:12345"; cd /srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/sources/rustc-1.90.0-src; python3 src/bootstrap/bootstrap.py test --exclude src/bootstrap --exclude src/doc/rustc --exclude src/doc/rustdoc --exclude src/doc/unstable-book --exclude src/etc/test-float-parse --exclude src/librustdoc --exclude src/rustdoc-json-types --exclude src/tools/coverage-dump --exclude src/tools/jsondoclint --exclude src/tools/lint-docs --exclude src/tools/replace-version-placeholder --exclude src/tools/rust-analyzer --exclude src/tools/rustdoc-themes --exclude src/tools/rust-installer --exclude src/tools/test-float-parse --exclude src/tools/suggest-tests --exclude src/tools/tidy --exclude tests/assembly-llvm/asm/aarch64-outline-atomics.rs --exclude tests/codegen-llvm/issues/issue-122805.rs --exclude tests/codegen-llvm/thread-local.rs --exclude tests/mir-opt/ --exclude tests/run-make --exclude tests/run-make-fulldeps --exclude tests/rustdoc --exclude tests/rustdoc-json --exclude tests/rustdoc-js-std --exclude tests/ui/abi/stack-probes-lto.rs --exclude tests/ui/abi/stack-probes.rs --exclude tests/ui/codegen/mismatched-data-layouts.rs --exclude tests/codegen-llvm/rust-abi-arch-specific-adjustment.rs --exclude tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs --exclude tests/ui/feature-gates/version_check.rs --exclude tests/ui-fulldeps/ --exclude tests/ui/process/nofile-limit.rs --exclude tidyselftest --no-doc --no-fail-fast --bless --target x86_64-poky-linux-gnu' returned non-zero exit status 1:
...
https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/2691
Can you have a look at this error?
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Re: [OE-core] [PATCH 2/2] rust-common.bbclass: filter out incorrect compiler flags in wrapper
Quoted 43 lines · Mathieu Dubois-Briand
Quoted 22 lines · Gyorgy Sarvari via lists.openembedded.org
This patch is a workaround for https://bugzilla.yoctoproject.org/show_bug.cgi?id=15976
cc-rs crate is used by a number of projects, including rust's bootstrap also to invoke the systems c/c++ compiler.
A few updates ago it has changed the way CFLAGS/CXXFLAGS are handled: it now merges them with HOST_C*FLAGS and TARGET_C*FLAGS.
This is a problem when a recipe is cross compiled, but it has a build dependency which uses this crate to be built. In this case the C*FLAGS variable contains target flags, while the HOST_C*FLAGS contains host-specific flags, and when the two are mixed, the output is not what one expects.
This change tries to filter out the incorrect flags:
- If the wrapper is invoked as a c or c++ compiler wrapper,
- then determines if it compiles for host or for target
- Depending on the on above, it considers the TARGET_*FLAGS or HOST*FLAGS correct, and the C*FLAGS variable content incorrect.
- It subtracts the correct set from the incorrect set, and drops the remainder from the compiler flags. (Which might be often an empty list, so the flags are frequently unchanged)Quoted 2 lines
Signed-off-by Gyorgy Sarvari <[email protected]>---
Hi Gyorgy,
Thanks for your patch.
I suspect one of the commit of this series is responsible of this rust test failure:
2025-11-20 21:59:48,098 - oe-selftest - INFO - FAIL: rust.RustSelfTestSystemEmulated.test_rust (subunit.RemotedTestCase)...
File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, inwrapped_f
returnfunc(*args, **kwargs)
File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/rust.py", line 131, intest_rust
retval = runCmd(cmd)
File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
raiseAssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'export TARGET_VENDOR="-poky"; export PATH=/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/python3-native:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/hosttools:$PATH; export RUST_TARGET_PATH=/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/rust-targets; export RUSTFLAGS='-C strip=debuginfo'; export TEST_DEVICE_ADDR="192.168.7.6:12345"; cd/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/sources/rustc-1.90.0-src; python3src/bootstrap/bootstrap.pytest --exclude src/bootstrap --exclude src/doc/rustc --exclude src/doc/rustdoc --exclude src/doc/unstable-book --exclude src/etc/test-float-parse --exclude src/librustdoc --exclude src/rustdoc-json-types --exclude src/tools/coverage-dump --exclude src/tools/jsondoclint --exclude src/tools/lint-docs --exclude src/tools/replace-version-placeholder --exclude src/tools/rust-analyzer --exclude src/tools/rustdoc-themes --exclude src/tools/rust-installer --exclude src/tools/test-float-parse --exclude src/tools/suggest-tests --exclude src/tools/tidy --excludetests/assembly-llvm/asm/aarch64-outline-atomics.rs--excludetests/codegen-llvm/issues/issue-122805.rs--excludetests/codegen-llvm/thread-local.rs--exclude tests/mir-opt/ --exclude tests/run-make --exclude tests/run-make-fulldeps --exclude tests/rustdoc --exclude tests/rustdoc-json --exclude tests/rustdoc-js-std --excludetests/ui/abi/stack-probes-lto.rs--excludetests/ui/abi/stack-probes.rs--excludetests/ui/codegen/mismatched-data-layouts.rs--excludetests/codegen-llvm/rust-abi-arch-specific-adjustment.rs--excludetests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs--excludetests/ui/feature-gates/version_check.rs--exclude tests/ui-fulldeps/ --excludetests/ui/process/nofile-limit.rs--exclude tidyselftest --no-doc --no-fail-fast --bless --target x86_64-poky-linux-gnu' returned non-zero exit status 1:
...
Quoted 3 lines
https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/2691
Can you have a look at this error?
Yes, I did already:
https://lists.openembedded.org/g/openembedded-core/message/226640 - I believe this should take care of this issue
(but I have to admit that currently I struggle executing the selftest on my machine - both inside and outside of docker I get mysterious qemu startup errors).
Quoted 3 lines
Thanks,
Mathieu
Re: [OE-core] [PATCH 2/2] rust-common.bbclass: filter out incorrect compiler flags in wrapper
Quoted 46 lines · Gyorgy Sarvari via lists.openembedded.org
Quoted 45 lines · Mathieu Dubois-Briand
Quoted 22 lines · Gyorgy Sarvari via lists.openembedded.org
This patch is a workaround for https://bugzilla.yoctoproject.org/show_bug.cgi?id=15976
cc-rs crate is used by a number of projects, including rust's bootstrap also to invoke the systems c/c++ compiler.
A few updates ago it has changed the way CFLAGS/CXXFLAGS are handled: it now merges them with HOST_C*FLAGS and TARGET_C*FLAGS.
This is a problem when a recipe is cross compiled, but it has a build dependency which uses this crate to be built. In this case the C*FLAGS variable contains target flags, while the HOST_C*FLAGS contains host-specific flags, and when the two are mixed, the output is not what one expects.
This change tries to filter out the incorrect flags:
- If the wrapper is invoked as a c or c++ compiler wrapper,
- then determines if it compiles for host or for target
- Depending on the on above, it considers the TARGET_*FLAGS or HOST*FLAGS correct, and the C*FLAGS variable content incorrect.
- It subtracts the correct set from the incorrect set, and drops the remainder from the compiler flags. (Which might be often an empty list, so the flags are frequently unchanged)Quoted 2 lines
Signed-off-by Gyorgy Sarvari <[email protected]>---
Hi Gyorgy,
Thanks for your patch.
I suspect one of the commit of this series is responsible of this rust test failure:
2025-11-20 21:59:48,098 - oe-selftest - INFO - FAIL: rust.RustSelfTestSystemEmulated.test_rust (subunit.RemotedTestCase)...
File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, inwrapped_f
returnfunc(*args, **kwargs)
File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/rust.py", line 131, intest_rust
retval = runCmd(cmd)
File "/srv/pokybuild/yocto-worker/qemux86-64-tc/build/layers/openembedded-core/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
raiseAssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'export TARGET_VENDOR="-poky"; export PATH=/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/python3-native:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/hosttools:$PATH; export RUST_TARGET_PATH=/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/rust-targets; export RUSTFLAGS='-C strip=debuginfo'; export TEST_DEVICE_ADDR="192.168.7.6:12345"; cd/srv/pokybuild/yocto-worker/qemux86-64-tc/build/build-st-1700604/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/sources/rustc-1.90.0-src; python3src/bootstrap/bootstrap.pytest --exclude src/bootstrap --exclude src/doc/rustc --exclude src/doc/rustdoc --exclude src/doc/unstable-book --exclude src/etc/test-float-parse --exclude src/librustdoc --exclude src/rustdoc-json-types --exclude src/tools/coverage-dump --exclude src/tools/jsondoclint --exclude src/tools/lint-docs --exclude src/tools/replace-version-placeholder --exclude src/tools/rust-analyzer --exclude src/tools/rustdoc-themes --exclude src/tools/rust-installer --exclude src/tools/test-float-parse --exclude src/tools/suggest-tests --exclude src/tools/tidy --exclude tests/assembly-llvm/asm/aarch64-outline-atomics.rs --exclude tests/codegen-llvm/issues/issue-122805.rs --exclude tests/codegen-llvm/thread-local.rs --exclude tests/mir-opt/ --exclude tests/run-make --exclude tests/run-make-fulldeps --exclude tests/rustdoc --exclude tests/rustdoc-json --exclude tests/rustdoc-js-std --exclude tests/ui/abi/stack-probes-lto.rs --exclude tests/ui/abi/stack-probes.rs --exclude tests/ui/codegen/mismatched-data-layouts.rs --exclude tests/codegen-llvm/rust-abi-arch-specific-adjustment.rs --exclude tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs --exclude tests/ui/feature-gates/version_check.rs --exclude tests/ui-fulldeps/ --exclude tests/ui/process/nofile-limit.rs --exclude tidyselftest --no-doc --no-fail-fast --bless --target x86_64-poky-linux-gnu' returned non-zero exit status 1:
...https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/2691
Quoted 8 lines
Quoted 1 line
Can you have a look at this error?
Yes, I did already:
https://lists.openembedded.org/g/openembedded-core/message/226640 - I believe this should take care of this issue
(but I have to admit that currently I struggle executing the selftest on my machine - both inside and outside of docker I get mysterious qemu startup errors).
I think I saw that issue in selftest with your patch applied.
I'm trying a simpler patch on master-next right now as a workaround.
Cheers,
Richard