Skip to contentSkip to conversation

staging.bbclass: process installed dependencies in deterministic order as well

PATCH In review Martin Jansa 2 participants 3 messages Nov 13, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 staging.bbclass: process installed dependencies in deterministic order as well

From: Martin Jansa <[email protected]>

* similarly as direct dependencies before
* this doesn't fix any issue (at least AFAIK),
just keeps the log files more deterministic to avoid unnecessary churn like in:

perl-native.log.do_configure:
-NOTE: Installed into sysroot: ['zlib-native', 'make-native', 'perlcross-native', 'gdbm-native', 'xz-native', 'gettext-minimal-native', 'texinfo-dummy-native', 'libtool-native'] +NOTE: Installed into sysroot: ['zlib-native', 'make-native', 'perlcross-native', 'gdbm-native', 'libtool-native', 'gettext-minimal-native', 'texinfo-dummy-native', 'xz-native']

and similarly in "Note: Skipping as already exists in sysroot:

---
 meta/classes-global/staging.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-global/staging.bbclass +1 -1
index d229f40107..cf1e4600fd 100644
@@ -521,7 +521,7 @@ python extend_recipe_sysroot() {
 
     binfiles = {}
     # Now handle installs
-    for dep in configuredeps:
+    for dep in sorted(configuredeps):
         c = setscenedeps[dep][0]
         if c not in installed:
             continue
Signature · 1 line
2.42.1
#1 lore raw

PATCH 2/2 bitbake.conf, ovmf, go-cross-canadian, cargo, libjpeg-turbo, ffmpeg, acpica, perf: drop ${PE} and ${PR} from -f{file,macro,debug}-prefix-map

* ${EXTENDPE} and ${PR} was already dropped from ${WORKDIR} in:
https://lists.openembedded.org/g/openembedded-core/message/185818
adjust -fmacro-prefix-map and -fdebug-prefix-map similarly. As we don't depend on PR in WORKDIR maybe we don't need it in /usr/src/debug paths as well anymore.

* The interesting side-effect of this is that for DISTROs which still use PR (e.g. webOS OSE), the difference in -fdebug-prefix-map causes all binaries to be slightly different when rebuilt with different PR (due to this path, even when otherwise they are binary reproducible) and when the number of digits in PR changes (e.g. from r9 to r10) it also results in slightly bigger binaries (and more unnecessary changes in buildhistory diffs).

For example this foo binary, the difference between "new" and "old" is extra patch applied in SRC_URI (which doesn't affect the foo binary, but was the reason for PR bump).

And the binary is bigger with r10, identical with r6 and slightly different due to debug-prefix-map between r6 and r7:

  $ ls -la 1.0.0-175-*/image/usr/sbin/foo
  -rwxr-xr-x 1 bitbake bitbake 30182460 Nov  8 08:29 1.0.0-175-new-r10/image/usr/sbin/foo
  -rwxr-xr-x 1 bitbake bitbake 30182224 Nov  8 08:02 1.0.0-175-new-r6/image/usr/sbin/foo
  -rwxr-xr-x 1 bitbake bitbake 30182224 Nov  8 07:49 1.0.0-175-new-r7/image/usr/sbin/foo
  -rwxr-xr-x 1 bitbake bitbake 30182224 Nov  8 07:39 1.0.0-175-old-r6/image/usr/sbin/foo
  $ md5sum 1.0.0-175-*/image/usr/sbin/foo
  8e3ab67d596f8cc2f9c6320dab10af01  1.0.0-175-new-r10/image/usr/sbin/foo
  f6ff1e64fe6affb9ba0d8f278cf7ed79  1.0.0-175-new-r6/image/usr/sbin/foo
  6469a0b10aac233911e63c5ea97b03c0  1.0.0-175-new-r7/image/usr/sbin/foo
  f6ff1e64fe6affb9ba0d8f278cf7ed79  1.0.0-175-old-r6/image/usr/sbin/foo
---
 meta/conf/bitbake.conf                            | 8 ++++----
 meta/recipes-core/ovmf/ovmf_git.bb                | 2 +-
 meta/recipes-devtools/go/go-cross-canadian.inc    | 2 +-
 meta/recipes-devtools/rust/cargo_1.70.0.bb        | 2 +-
 meta/recipes-extended/acpica/acpica_20230628.bb   | 2 +-
 meta/recipes-graphics/jpeg/libjpeg-turbo_3.0.1.bb | 2 +-
 meta/recipes-kernel/perf/perf.bb                  | 2 +-
 meta/recipes-multimedia/ffmpeg/ffmpeg_6.0.bb      | 2 +-
 8 files changed, 11 insertions(+), 11 deletions(-)
meta/conf/bitbake.conf +4 -4
index 67ff769046..3836a389b8 100644
@@ -648,10 +648,10 @@ EXTRA_OEMAKE:prepend:task-install = "${PARALLEL_MAKEINST} "
 ##################################################################
 # Beware: applied last to first
 DEBUG_PREFIX_MAP ?= "-fcanon-prefix-map \
- -fmacro-prefix-map=${S}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR} \
- -fdebug-prefix-map=${S}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR} \
- -fmacro-prefix-map=${B}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR} \
- -fdebug-prefix-map=${B}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR} \
+ -fmacro-prefix-map=${S}=/usr/src/debug/${PN}/${PV} \
+ -fdebug-prefix-map=${S}=/usr/src/debug/${PN}/${PV} \
+ -fmacro-prefix-map=${B}=/usr/src/debug/${PN}/${PV} \
+ -fdebug-prefix-map=${B}=/usr/src/debug/${PN}/${PV} \
  -fdebug-prefix-map=${STAGING_DIR_HOST}= \
  -fmacro-prefix-map=${STAGING_DIR_HOST}= \
  -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
meta/recipes-core/ovmf/ovmf_git.bb +1 -1
index 6bbe21ac5c..bf67a2a4a8 100644
@@ -122,7 +122,7 @@ fix_toolchain:append:class-native() {
 # --debug-prefix-map to nasm (we carry a patch to nasm for this). The
 # tools definitions are built by ovmf-native so we need to pass this in
 # at target build time when we know the right values.
-export NASM_PREFIX_MAP = "--debug-prefix-map=${WORKDIR}=/usr/src/debug/ovmf/${EXTENDPE}${PV}-${PR}"
+export NASM_PREFIX_MAP = "--debug-prefix-map=${WORKDIR}=/usr/src/debug/ovmf/${PV}"
 export GCC_PREFIX_MAP = "${DEBUG_PREFIX_MAP} -Wno-stringop-overflow -Wno-maybe-uninitialized"
 
 GCC_VER="$(${CC} -v 2>&1 | tail -n1 | awk '{print $3}')"
meta/recipes-devtools/go/go-cross-canadian.inc +1 -1
index 45758f3361..0a72e89bbd 100644
@@ -8,7 +8,7 @@ PN = "go-cross-canadian-${TRANSLATED_TARGET_ARCH}"
 # it uses gcc on build machine during go-cross-canadian bootstrap, but
 # the gcc version may be old and not support option '-fmacro-prefix-map'
 # which is one of default values of DEBUG_PREFIX_MAP
-DEBUG_PREFIX_MAP = "-fdebug-prefix-map=${WORKDIR}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR} \
+DEBUG_PREFIX_MAP = "-fdebug-prefix-map=${WORKDIR}=/usr/src/debug/${PN}/${PV} \
                     -fdebug-prefix-map=${STAGING_DIR_HOST}= \
                     -fdebug-prefix-map=${STAGING_DIR_NATIVE}= \
                     "
meta/recipes-devtools/rust/cargo_1.70.0.bb +1 -1
index a5c1064106..9a23ba9d5f 100644
@@ -19,7 +19,7 @@ CARGO_VENDORING_DIRECTORY = "${RUSTSRC}/vendor"
 
 inherit cargo pkgconfig
 
-DEBUG_PREFIX_MAP += "-fdebug-prefix-map=${RUSTSRC}/vendor=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}"
+DEBUG_PREFIX_MAP += "-fdebug-prefix-map=${RUSTSRC}/vendor=/usr/src/debug/${PN}/${PV}"
 
 do_cargo_setup_snapshot () {
 	${WORKDIR}/rust-snapshot-components/${CARGO_SNAPSHOT}/install.sh --prefix="${WORKDIR}/${CARGO_SNAPSHOT}" --disable-ldconfig
meta/recipes-extended/acpica/acpica_20230628.bb +1 -1
index b7921516a2..8ca1f0892f 100644
@@ -36,7 +36,7 @@ EXTRA_OEMAKE = "CC='${CC}' \
                 INSTALLDIR=${bindir} \
                 INSTALLFLAGS= \
                 YACC=bison \
-                YFLAGS='-y --file-prefix-map=${WORKDIR}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}' \
+                YFLAGS='-y --file-prefix-map=${WORKDIR}=/usr/src/debug/${PN}/${PV}' \
                 "
 
 do_install() {
meta/recipes-graphics/jpeg/libjpeg-turbo_3.0.1.bb +1 -1
index 99ed82dac4..ca1910966e 100644
@@ -24,7 +24,7 @@ RCONFLICTS:${PN} += "jpeg"
 
 inherit cmake pkgconfig
 
-export NASMENV = "--reproducible --debug-prefix-map=${WORKDIR}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}"
+export NASMENV = "--reproducible --debug-prefix-map=${WORKDIR}=/usr/src/debug/${PN}/${PV}"
 
 # Add nasm-native dependency consistently for all build arches is hard
 EXTRA_OECMAKE:append:class-native = " -DWITH_SIMD=False"
meta/recipes-kernel/perf/perf.bb +1 -1
index a90acd55ef..dee8b50b31 100644
@@ -98,7 +98,7 @@ EXTRA_OEMAKE = '\
     AR="${AR}" \
     LD="${LD}" \
     EXTRA_CFLAGS="-ldw -I${S}" \
-    YFLAGS='-y --file-prefix-map=${WORKDIR}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}' \
+    YFLAGS='-y --file-prefix-map=${WORKDIR}=/usr/src/debug/${PN}/${PV}' \
     EXTRA_LDFLAGS="${PERF_EXTRA_LDFLAGS}" \
     perfexecdir=${libexecdir} \
     NO_GTK2=1 \
meta/recipes-multimedia/ffmpeg/ffmpeg_6.0.bb +1 -1
index e65f80d64b..0ad2f80217 100644
@@ -151,7 +151,7 @@ do_configure() {
     export TMPDIR="${B}/tmp"
     mkdir -p ${B}/tmp
     ${S}/configure ${EXTRA_OECONF}
-    sed -i -e "s,^X86ASMFLAGS=.*,& --debug-prefix-map=${S}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR} --debug-prefix-map=${B}=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR},g" ${B}/ffbuild/config.mak
+    sed -i -e "s,^X86ASMFLAGS=.*,& --debug-prefix-map=${S}=/usr/src/debug/${PN}/${PV} --debug-prefix-map=${B}=/usr/src/debug/${PN}/${PV},g" ${B}/ffbuild/config.mak
 }
 
 # patch out build host paths for reproducibility
Signature · 1 line
2.42.1
#2 lore raw

Patchtest results for [PATCH 2/2] bitbake.conf, ovmf, go-cross-canadian, cargo, libjpeg-turbo, ffmpeg, acpica, perf: drop ${PE} and ${PR} from -f{file,macro,debug}-prefix-map

Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information:

---
Testing patch /home/patchtest/share/mboxes/2-2-bitbake.conf-ovmf-go-cross-canadian-cargo-libjpeg-turbo-ffmpeg-acpica-perf-drop-PE-and-PR-from--f-file-macro-debug--prefix-map.patch

FAIL: test shortlog length: Edit shortlog so that it is 90 characters or less (currently 140 characters) (test_mbox.TestMbox.test_shortlog_length)

PASS: pretest src uri left files (test_metadata.TestMetadata.pretest_src_uri_left_files)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test lic files chksum modified not mentioned (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test src uri left files (test_metadata.TestMetadata.test_src_uri_left_files)

SKIP: pretest pylint: No python related patches, skipping test (test_python_pylint.PyLint.pretest_pylint)
SKIP: test CVE tag format: No new CVE patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new CVE patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new CVE patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: No python related patches, skipping test (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)
SKIP: test target mailing list: Series merged, no reason to check other mailing lists (test_mbox.TestMbox.test_target_mailing_list)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!