Skip to contentSkip to conversation

xen, xen-tools: pass prefix maps to fix buildpaths reproducible build issues

PATCH In review Christopher Clark 3 participants 3 messages Jul 10, 2022 – Jul 12, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xen, xen-tools: pass prefix maps to fix buildpaths reproducible build issues

To fix warnings when the buildpaths QA test is enabled, pass the file and debug prefix map options that were previously supplied only for reproducible builds, adding them to the DEBUG_PREFIX_MAP variable and supplying that to CC.

Testing indicates that passing the prefix maps needs to be via CC rather than the EXTRA_CFLAGS_XEN_CORE/TOOLS variables.

---
 recipes-extended/xen/xen.inc | 21 ++++++++++-----------
 1 file changed, 10 insertions(+), 11 deletions(-)
recipes-extended/xen/xen.inc +10 -11
index ed6e6a7..2bbf1e3 100644
@@ -88,12 +88,13 @@ export CROSS_COMPILE="${TARGET_PREFIX}"
 # overide LDFLAGS to allow xen to build without: "x86_64-oe-linux-ld: unrecognized option '-Wl,-O1'"
 export LDFLAGS=""
 
-# Pass through the Yocto distro compiler flags via the Xen-provided variables.
-# Special handling:
-# - The Yocto distro compiler flags are typically set to be appropriate for
-#   user-space software rather than for generation of a hypervisor binary, so
-#   only pass the debug prefix map to the hypervisor build:
-EXTRA_CFLAGS_XEN_CORE="${DEBUG_PREFIX_MAP}"
+# No additional C flags for the main hypervisor build
+EXTRA_CFLAGS_XEN_CORE ?= ""
+# Add prefix maps to support buildpaths QA test and reproducibility
+DEBUG_PREFIX_MAP:append = " \
+    -ffile-prefix-map=${S}=${PN}-source \
+    -fdebug-prefix-map=${WORKDIR}=${PN} \
+    "
 
 # - The Xen tools build for x86 systems with HVM-mode enabled includes hvmloader
 #   which fails to build when "-m64" is included in flags set via the
@@ -109,11 +110,11 @@ TUNE_CCARGS:x86-64=""
 #   It must not be compiled with SSE compiler options enabled and the Xen build
 #   explicitly clears CFLAGS to ensure that, so such options must not be passed
 #   in via the tool variable. hvmloader is required to run HVM-mode guest VMs.
-CC="${CCACHE}${HOST_PREFIX}gcc ${TOOLCHAIN_OPTIONS} ${CC_REPRODUCIBLE_OPTIONS}"
+CC="${CCACHE}${HOST_PREFIX}gcc ${TOOLCHAIN_OPTIONS} ${DEBUG_PREFIX_MAP} ${CC_REPRODUCIBLE_OPTIONS}"
 EXTRA_CFLAGS_XEN_TOOLS="${HOST_CC_ARCH} ${CFLAGS}"
 # 32-bit ARM needs the TUNE_CCARGS component of HOST_CC_ARCH to be passed
 # in CC to ensure that configure can compile binaries for the right arch.
-CC:arm="${CCACHE}${HOST_PREFIX}gcc ${TUNE_CCARGS} ${TOOLCHAIN_OPTIONS} ${CC_REPRODUCIBLE_OPTIONS}"
+CC:arm="${CCACHE}${HOST_PREFIX}gcc ${TUNE_CCARGS} ${TOOLCHAIN_OPTIONS} ${DEBUG_PREFIX_MAP} ${CC_REPRODUCIBLE_OPTIONS}"
 
 # There are no Xen-provided variables for C++, so append to the tool variables:
 CPP:append = " ${CPPFLAGS}"
@@ -158,9 +159,7 @@ EXTRA_OEMAKE += "${@['', 'XEN_WHOAMI=${PF} XEN_DOMAIN=${DISTRO} XEN_BUILD_HOST=$
                     [d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1']}${@get_build_time_vars(d)}"
 
 # Improve build reproducibility: compiler flags to remove filesystem differences.
-CC_REPRODUCIBLE_OPTIONS = "${@['', '-gno-record-gcc-switches ' + \
-                                   '-ffile-prefix-map=${S}=${PN}-source ' + \
-                                   '-fdebug-prefix-map=${WORKDIR}=${PN}'] \
+CC_REPRODUCIBLE_OPTIONS = "${@['', '-gno-record-gcc-switches'] \
                              [d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1']}"
 
 # check for XSM in package config to allow XSM_ENABLE to be set
Signature · 1 line
2.25.1
#1 lore raw

Looks good to me, and better than what we had before.

I've fast tracked it into master, so we can see what the next set of autobuilder runs uncover.

Many thanks for looking at this.

Bruce

In message: [meta-virtualization][PATCH] xen, xen-tools: pass prefix maps to fix buildpaths reproducible build issues

Quoted 65 lines · on 09/07/2022 Christopher Clark wrote:

To fix warnings when the buildpaths QA test is enabled, pass the file and debug prefix map options that were previously supplied only for reproducible builds, adding them to the DEBUG_PREFIX_MAP variable and supplying that to CC.

Testing indicates that passing the prefix maps needs to be via CC rather than the EXTRA_CFLAGS_XEN_CORE/TOOLS variables.

---
 recipes-extended/xen/xen.inc | 21 ++++++++++-----------
 1 file changed, 10 insertions(+), 11 deletions(-)
recipes-extended/xen/xen.inc +10 -11
index ed6e6a7..2bbf1e3 100644
@@ -88,12 +88,13 @@ export CROSS_COMPILE="${TARGET_PREFIX}"
 # overide LDFLAGS to allow xen to build without: "x86_64-oe-linux-ld: unrecognized option '-Wl,-O1'"
 export LDFLAGS=""
 
-# Pass through the Yocto distro compiler flags via the Xen-provided variables.
-# Special handling:
-# - The Yocto distro compiler flags are typically set to be appropriate for
-#   user-space software rather than for generation of a hypervisor binary, so
-#   only pass the debug prefix map to the hypervisor build:
-EXTRA_CFLAGS_XEN_CORE="${DEBUG_PREFIX_MAP}"
+# No additional C flags for the main hypervisor build
+EXTRA_CFLAGS_XEN_CORE ?= ""
+# Add prefix maps to support buildpaths QA test and reproducibility
+DEBUG_PREFIX_MAP:append = " \
+    -ffile-prefix-map=${S}=${PN}-source \
+    -fdebug-prefix-map=${WORKDIR}=${PN} \
+    "
 
 # - The Xen tools build for x86 systems with HVM-mode enabled includes hvmloader
 #   which fails to build when "-m64" is included in flags set via the
@@ -109,11 +110,11 @@ TUNE_CCARGS:x86-64=""
 #   It must not be compiled with SSE compiler options enabled and the Xen build
 #   explicitly clears CFLAGS to ensure that, so such options must not be passed
 #   in via the tool variable. hvmloader is required to run HVM-mode guest VMs.
-CC="${CCACHE}${HOST_PREFIX}gcc ${TOOLCHAIN_OPTIONS} ${CC_REPRODUCIBLE_OPTIONS}"
+CC="${CCACHE}${HOST_PREFIX}gcc ${TOOLCHAIN_OPTIONS} ${DEBUG_PREFIX_MAP} ${CC_REPRODUCIBLE_OPTIONS}"
 EXTRA_CFLAGS_XEN_TOOLS="${HOST_CC_ARCH} ${CFLAGS}"
 # 32-bit ARM needs the TUNE_CCARGS component of HOST_CC_ARCH to be passed
 # in CC to ensure that configure can compile binaries for the right arch.
-CC:arm="${CCACHE}${HOST_PREFIX}gcc ${TUNE_CCARGS} ${TOOLCHAIN_OPTIONS} ${CC_REPRODUCIBLE_OPTIONS}"
+CC:arm="${CCACHE}${HOST_PREFIX}gcc ${TUNE_CCARGS} ${TOOLCHAIN_OPTIONS} ${DEBUG_PREFIX_MAP} ${CC_REPRODUCIBLE_OPTIONS}"
 
 # There are no Xen-provided variables for C++, so append to the tool variables:
 CPP:append = " ${CPPFLAGS}"
@@ -158,9 +159,7 @@ EXTRA_OEMAKE += "${@['', 'XEN_WHOAMI=${PF} XEN_DOMAIN=${DISTRO} XEN_BUILD_HOST=$
                     [d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1']}${@get_build_time_vars(d)}"
 
 # Improve build reproducibility: compiler flags to remove filesystem differences.
-CC_REPRODUCIBLE_OPTIONS = "${@['', '-gno-record-gcc-switches ' + \
-                                   '-ffile-prefix-map=${S}=${PN}-source ' + \
-                                   '-fdebug-prefix-map=${WORKDIR}=${PN}'] \
+CC_REPRODUCIBLE_OPTIONS = "${@['', '-gno-record-gcc-switches'] \
                              [d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1']}"
 
 # check for XSM in package config to allow XSM_ENABLE to be set
Signature · 1 line
2.25.1
Reviewed-by Tested-by on v1 1/1
#2 lore raw
In reply to Christopher Clark

Hi Christopher,

Quoted 11 lines · Christopher Clark

To fix warnings when the buildpaths QA test is enabled, pass the file and debug prefix map options that were previously supplied only for reproducible builds, adding them to the DEBUG_PREFIX_MAP variable and supplying that to CC.

Testing indicates that passing the prefix maps needs to be via CC rather than the EXTRA_CFLAGS_XEN_CORE/TOOLS variables.

Also I applied this in kirkstone and did a build and run on arm64, arm32 and x86 with success so:

Cheers
Bertrand

Quoted 56 lines

---
recipes-extended/xen/xen.inc | 21 ++++++++++-----------

1 file changed, 10 insertions(+), 11 deletions(-)
recipes-extended/xen/xen.inc +0 -0
index ed6e6a7..2bbf1e3 100644
@@ -88,12 +88,13 @@ export CROSS_COMPILE="${TARGET_PREFIX}"
 # overide LDFLAGS to allow xen to build without: "x86_64-oe-linux-ld: unrecognized option '-Wl,-O1'"

export LDFLAGS=""

recipes-extended/xen/xen.inc +7 -6
-# Pass through the Yocto distro compiler flags via the Xen-provided variables.
-# Special handling:
-# - The Yocto distro compiler flags are typically set to be appropriate for
-#   user-space software rather than for generation of a hypervisor binary, so
-#   only pass the debug prefix map to the hypervisor build:
-EXTRA_CFLAGS_XEN_CORE="${DEBUG_PREFIX_MAP}"
+# No additional C flags for the main hypervisor build
+EXTRA_CFLAGS_XEN_CORE ?= ""
+# Add prefix maps to support buildpaths QA test and reproducibility
+DEBUG_PREFIX_MAP:append = " \
+    -ffile-prefix-map=${S}=${PN}-source \
+    -fdebug-prefix-map=${WORKDIR}=${PN} \
+    "

# - The Xen tools build for x86 systems with HVM-mode enabled includes hvmloader
# which fails to build when "-m64" is included in flags set via the

recipes-extended/xen/xen.inc +1 -1
@@ -109,11 +110,11 @@ TUNE_CCARGS:x86-64=""
 #   It must not be compiled with SSE compiler options enabled and the Xen build
 #   explicitly clears CFLAGS to ensure that, so such options must not be passed
 #   in via the tool variable. hvmloader is required to run HVM-mode guest VMs.
-CC="${CCACHE}${HOST_PREFIX}gcc ${TOOLCHAIN_OPTIONS} ${CC_REPRODUCIBLE_OPTIONS}"
+CC="${CCACHE}${HOST_PREFIX}gcc ${TOOLCHAIN_OPTIONS} ${DEBUG_PREFIX_MAP} ${CC_REPRODUCIBLE_OPTIONS}"

EXTRA_CFLAGS_XEN_TOOLS="${HOST_CC_ARCH} ${CFLAGS}"
# 32-bit ARM needs the TUNE_CCARGS component of HOST_CC_ARCH to be passed

# in CC to ensure that configure can compile binaries for the right arch.
recipes-extended/xen/xen.inc +1 -1
-CC:arm="${CCACHE}${HOST_PREFIX}gcc ${TUNE_CCARGS} ${TOOLCHAIN_OPTIONS} ${CC_REPRODUCIBLE_OPTIONS}"
+CC:arm="${CCACHE}${HOST_PREFIX}gcc ${TUNE_CCARGS} ${TOOLCHAIN_OPTIONS} ${DEBUG_PREFIX_MAP} ${CC_REPRODUCIBLE_OPTIONS}"

# There are no Xen-provided variables for C++, so append to the tool variables:
CPP:append = " ${CPPFLAGS}"

recipes-extended/xen/xen.inc +1 -3
@@ -158,9 +159,7 @@ EXTRA_OEMAKE += "${@['', 'XEN_WHOAMI=${PF} XEN_DOMAIN=${DISTRO} XEN_BUILD_HOST=$
                    [d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1']}${@get_build_time_vars(d)}"
 
 # Improve build reproducibility: compiler flags to remove filesystem differences.
-CC_REPRODUCIBLE_OPTIONS = "${@['', '-gno-record-gcc-switches ' + \
-                                   '-ffile-prefix-map=${S}=${PN}-source ' + \
-                                   '-fdebug-prefix-map=${WORKDIR}=${PN}'] \
+CC_REPRODUCIBLE_OPTIONS = "${@['', '-gno-record-gcc-switches'] \
                             [d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1']}"
 
 # check for XSM in package config to allow XSM_ENABLE to be set
Signature · 1 line
2.25.1