gcc: Ensure that the shared source directory shared the same sstate hashes

PATCH In review Richard Purdie 3 participants 5 messages Jan 19, 2012 – Jan 20, 2012

Share & tools

Richard Purdie

author

2012-01-19 19:46 UTC #0 lore raw

PATCH 1/1 gcc: Ensure that the shared source directory shared the same sstate hashes

The fetch/unpack/patch/headerfix tasks are shared and hence their sstate hashes should also match. Sadly this is not the case since:

a) gcc-runtime applies an additional patch
b) The do_headerfix task was missing from libgcc
c) The do_headerfix task is a shell task and hence depends on all exported variables which can vary between cross and target recipes.

To fix this, the patch moves the patch to the common code, adds the headerfix task to a common include file and disabled shell dependencies on the do_headerfix task since its clear in this case we don't need thsoe dependencies since we just call sed.

With this patch applied, all these recipes now share common sstate checksums.

Signed-off-by Richard Purdie <[email protected]>

---

meta/recipes-devtools/gcc/gcc-4.6.inc +1 -0

index 4bbb2d2..a76fa0b 100644
@@ -73,6 +73,7 @@ SRC_URI = "svn://gcc.gnu.org/svn/gcc/branches;module=${BRANCH};proto=http \
 	   file://pr47551.patch \
 	   file://gcc-arm-set-cost.patch \
 	   file://GPLUSPLUS_INCLUDE_DIR_with_sysroot.patch \
+	   file://fortran-cross-compile-hack.patch \
 	  "
 
 SRC_URI_append_sh3  = " file://sh3-installfix-fixheaders.patch "

meta/recipes-devtools/gcc/gcc-common.inc +17 -0

index fe112d9..ed41b0f 100644
@@ -88,3 +88,20 @@ python workshared_clean () {
 	bb.note("Removing " + dir)
 	oe.path.remove(dir)
 }
+
+do_headerfix () {
+	# Change the default dynamic linker path, in case $base_liddir is non-standard
+	# (e.g. in multilib or sdk cases)
+	#
+	# We want something like the following:
+	# 	#define GLIBC_DYNAMIC_LINKER64 "/lib64/ld-linux-x86-64.so.2"
+	# becomes
+	#	#define GLIBC_DYNAMIC_LINKER64 SYSTEMLIBS_DIR "ld-linux-x86-64.so.2"
+	#
+	sed -i ${S}/gcc/config/*/linux*.h -e \
+		's#\(GLIBC_DYNAMIC_LINKER[^ ]*\) \( *"/lib.*\)/\(.*\)#\1 SYSTEMLIBS_DIR "\3#'
+}
+
+addtask headerfix after do_unpack before do_patch
+
+do_headerfix[vardepvalue] = "PATH"

meta/recipes-devtools/gcc/gcc-configure-common.inc +0 ~~-15~~

index d014980..c4b6ac1 100644
@@ -62,21 +62,6 @@ SYSTEMHEADERS = "${target_includedir}"
 SYSTEMLIBS = "${target_base_libdir}/"
 SYSTEMLIBS1 = "${target_libdir}/"
 
-do_headerfix () {
-	# Change the default dynamic linker path, in case $base_liddir is non-standard
-	# (e.g. in multilib or sdk cases)
-	#
-	# We want something like the following:
-	# 	#define GLIBC_DYNAMIC_LINKER64 "/lib64/ld-linux-x86-64.so.2"
-	# becomes
-	#	#define GLIBC_DYNAMIC_LINKER64 SYSTEMLIBS_DIR "ld-linux-x86-64.so.2"
-	#
-	sed -i ${S}/gcc/config/*/linux*.h -e \
-		's#\(GLIBC_DYNAMIC_LINKER[^ ]*\) \( *"/lib.*\)/\(.*\)#\1 SYSTEMLIBS_DIR "\3#'
-}
-
-addtask headerfix after do_unpack before do_patch
-
 do_configure_prepend () {
 	# teach gcc to find correct target includedir when checking libc ssp support
 	mkdir -p ${B}/gcc

meta/recipes-devtools/gcc/gcc-runtime_4.6.bb +0 -2

index 568b5af..97468db 100644
@@ -2,8 +2,6 @@ require gcc-${PV}.inc
 require gcc-configure-runtime.inc
 require gcc-package-runtime.inc
 
-SRC_URI_append = "file://fortran-cross-compile-hack.patch"
-
 ARCH_FLAGS_FOR_TARGET += "-isystem${STAGING_INCDIR}"
 
 EXTRA_OECONF += "--disable-libunwind-exceptions"

Saul Wold

2012-01-20 08:26 UTC #1 lore raw

Quoted 9 lines · Richard Purdie

The fetch/unpack/patch/headerfix tasks are shared and hence their sstate hashes should also match. Sadly this is not the case since:
a) gcc-runtime applies an additional patch
b) The do_headerfix task was missing from libgcc
c) The do_headerfix task is a shell task and hence depends on all exported variables which can vary between cross and target recipes.

Quoted 7 lines

To fix this, the patch moves the patch to the common code, adds the headerfix task to a common include file and disabled shell dependencies on the do_headerfix task since its clear in this case we don't need thsoe dependencies since we just call sed.
With this patch applied, all these recipes now share common sstate checksums.

Richard,

I tried both a sstate build with and existing tmp and a clean tmp, the existing tmp seemed to work ok, but with a clean tmp (and sstate), I got the following patch issue still.

This was with BB_SIGNATURE_HANDLER ?= 'basichash' set.

ERROR: Command Error: exit status: 1 Output:
Could not link file .pc/gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch/configure' to configure': No such file or directory Applying patch gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch
patching file configure.ac
Hunk #1 FAILED at 3073.
1 out of 1 hunk FAILED -- rejects in file configure.ac
patching file configure
Hunk #1 FAILED at 7594.
1 out of 1 hunk FAILED -- rejects in file configure
Patch gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch does not apply (enforce with -f)
ERROR: Function failed: patch_do_patch
ERROR: Logfile of failure stored in: /intel/poky2/builds/binutils/tmp/work-shared/gcc-4.6.2+svnr181430-r20/temp/log.do_patch.15457
NOTE: package libgcc-4.6.2+svnr181430-r20: task do_patch: Failed
ERROR: Task 882 (/intel/poky2/distro/meta/recipes-devtools/gcc/libgcc_4.6.bb, do_patch) failed with exit code '1'

I have not tried a clean sstate / clean tmp.

Sau!

Quoted 86 lines

Signed-off-by Richard Purdie<[email protected]>

---

meta/recipes-devtools/gcc/gcc-4.6.inc +1 -0

index 4bbb2d2..a76fa0b 100644
@@ -73,6 +73,7 @@ SRC_URI = "svn://gcc.gnu.org/svn/gcc/branches;module=${BRANCH};proto=http \
  	   file://pr47551.patch \
  	   file://gcc-arm-set-cost.patch \
  	   file://GPLUSPLUS_INCLUDE_DIR_with_sysroot.patch \
+	   file://fortran-cross-compile-hack.patch \
  	  "
 
  SRC_URI_append_sh3  = " file://sh3-installfix-fixheaders.patch "

meta/recipes-devtools/gcc/gcc-common.inc +17 -0

index fe112d9..ed41b0f 100644
@@ -88,3 +88,20 @@ python workshared_clean () {
  	bb.note("Removing " + dir)
  	oe.path.remove(dir)
  }
+
+do_headerfix () {
+	# Change the default dynamic linker path, in case $base_liddir is non-standard
+	# (e.g. in multilib or sdk cases)
+	#
+	# We want something like the following:
+	# 	#define GLIBC_DYNAMIC_LINKER64 "/lib64/ld-linux-x86-64.so.2"
+	# becomes
+	#	#define GLIBC_DYNAMIC_LINKER64 SYSTEMLIBS_DIR "ld-linux-x86-64.so.2"
+	#
+	sed -i ${S}/gcc/config/*/linux*.h -e \
+		's#\(GLIBC_DYNAMIC_LINKER[^ ]*\) \( *"/lib.*\)/\(.*\)#\1 SYSTEMLIBS_DIR "\3#'
+}
+
+addtask headerfix after do_unpack before do_patch
+
+do_headerfix[vardepvalue] = "PATH"

meta/recipes-devtools/gcc/gcc-configure-common.inc +0 ~~-15~~

index d014980..c4b6ac1 100644
@@ -62,21 +62,6 @@ SYSTEMHEADERS = "${target_includedir}"
  SYSTEMLIBS = "${target_base_libdir}/"
  SYSTEMLIBS1 = "${target_libdir}/"
 
-do_headerfix () {
-	# Change the default dynamic linker path, in case $base_liddir is non-standard
-	# (e.g. in multilib or sdk cases)
-	#
-	# We want something like the following:
-	# 	#define GLIBC_DYNAMIC_LINKER64 "/lib64/ld-linux-x86-64.so.2"
-	# becomes
-	#	#define GLIBC_DYNAMIC_LINKER64 SYSTEMLIBS_DIR "ld-linux-x86-64.so.2"
-	#
-	sed -i ${S}/gcc/config/*/linux*.h -e \
-		's#\(GLIBC_DYNAMIC_LINKER[^ ]*\) \( *"/lib.*\)/\(.*\)#\1 SYSTEMLIBS_DIR "\3#'
-}
-
-addtask headerfix after do_unpack before do_patch
-
  do_configure_prepend () {
  	# teach gcc to find correct target includedir when checking libc ssp support
  	mkdir -p ${B}/gcc

meta/recipes-devtools/gcc/gcc-runtime_4.6.bb +0 -2

index 568b5af..97468db 100644
@@ -2,8 +2,6 @@ require gcc-${PV}.inc
  require gcc-configure-runtime.inc
  require gcc-package-runtime.inc
 
-SRC_URI_append = "file://fortran-cross-compile-hack.patch"
-
  ARCH_FLAGS_FOR_TARGET += "-isystem${STAGING_INCDIR}"
 
  EXTRA_OECONF += "--disable-libunwind-exceptions"

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Lu, Lianhao

2012-01-20 09:00 UTC #2 lore raw

Quoted 52 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Saul Wold
Sent: Friday, January 20, 2012 4:19 PM
To: Patches and discussions about the oe-core layer
Subject: Re: [OE-core] [PATCH] gcc: Ensure that the shared source directory shared the same sstate hashes
Quoted 9 lines · Richard Purdie
The fetch/unpack/patch/headerfix tasks are shared and hence their sstate hashes should also match. Sadly this is not the case since:
a) gcc-runtime applies an additional patch
b) The do_headerfix task was missing from libgcc
c) The do_headerfix task is a shell task and hence depends on all exported variables which can vary between cross and target recipes.
Quoted 7 lines
To fix this, the patch moves the patch to the common code, adds the headerfix task to a common include file and disabled shell dependencies on the do_headerfix task since its clear in this case we don't need thsoe dependencies since we just call sed.
With this patch applied, all these recipes now share common sstate checksums.
Richard,
I tried both a sstate build with and existing tmp and a clean tmp, the existing tmp seemed to work ok, but with a clean tmp (and sstate), I got the following patch issue still.
This was with BB_SIGNATURE_HANDLER ?= 'basichash' set.
ERROR: Command Error: exit status: 1 Output:
Could not link file
.pc/gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch/configure' to configure': No
such file or directory
Applying patch gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch
patching file configure.ac
Hunk #1 FAILED at 3073.
1 out of 1 hunk FAILED -- rejects in file configure.ac
patching file configure
Hunk #1 FAILED at 7594.
1 out of 1 hunk FAILED -- rejects in file configure
Patch gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch does not apply (enforce with -f)
ERROR: Function failed: patch_do_patch
ERROR: Logfile of failure stored in:
/intel/poky2/builds/binutils/tmp/work-shared/gcc-4.6.2+svnr181430-r20/temp/log.do_patch.15457
NOTE: package libgcc-4.6.2+svnr181430-r20: task do_patch: Failed
ERROR: Task 882
(/intel/poky2/distro/meta/recipes-devtools/gcc/libgcc_4.6.bb, do_patch)
failed with exit code '1'

Quoted 2 lines

I have not tried a clean sstate / clean tmp.

Well, using the poky-conrib/rpurdie/t1 branch, it seems that the libgcc still has different signature with gcc-cross-initial for task patch. But bitbake-diffsigs reports nothing. gcc-runtime seems to be ok now.

Quoted 92 lines

Quoted 82 lines

Signed-off-by Richard Purdie<[email protected]>

---

meta/recipes-devtools/gcc/gcc-4.6.inc +1 -0

index 4bbb2d2..a76fa0b 100644
@@ -73,6 +73,7 @@ SRC_URI = "svn://gcc.gnu.org/svn/gcc/branches;module=${BRANCH};proto=http \
  	   file://pr47551.patch \
  	   file://gcc-arm-set-cost.patch \
  	   file://GPLUSPLUS_INCLUDE_DIR_with_sysroot.patch \
+	   file://fortran-cross-compile-hack.patch \
  	  "
 
  SRC_URI_append_sh3  = " file://sh3-installfix-fixheaders.patch "

meta/recipes-devtools/gcc/gcc-common.inc +17 -0

index fe112d9..ed41b0f 100644
@@ -88,3 +88,20 @@ python workshared_clean () {
  	bb.note("Removing " + dir)
  	oe.path.remove(dir)
  }
+
+do_headerfix () {
+	# Change the default dynamic linker path, in case $base_liddir is non-standard
+	# (e.g. in multilib or sdk cases)
+	#
+	# We want something like the following:
+	# 	#define GLIBC_DYNAMIC_LINKER64 "/lib64/ld-linux-x86-64.so.2"
+	# becomes
+	#	#define GLIBC_DYNAMIC_LINKER64 SYSTEMLIBS_DIR "ld-linux-x86-64.so.2"
+	#
+	sed -i ${S}/gcc/config/*/linux*.h -e \
+		's#\(GLIBC_DYNAMIC_LINKER[^ ]*\) \( *"/lib.*\)/\(.*\)#\1 SYSTEMLIBS_DIR "\3#'
+}
+
+addtask headerfix after do_unpack before do_patch
+
+do_headerfix[vardepvalue] = "PATH"

meta/recipes-devtools/gcc/gcc-configure-common.inc +0 ~~-15~~

index d014980..c4b6ac1 100644
@@ -62,21 +62,6 @@ SYSTEMHEADERS = "${target_includedir}"
  SYSTEMLIBS = "${target_base_libdir}/"
  SYSTEMLIBS1 = "${target_libdir}/"
 
-do_headerfix () {
-	# Change the default dynamic linker path, in case $base_liddir is non-standard
-	# (e.g. in multilib or sdk cases)
-	#
-	# We want something like the following:
-	# 	#define GLIBC_DYNAMIC_LINKER64 "/lib64/ld-linux-x86-64.so.2"
-	# becomes
-	#	#define GLIBC_DYNAMIC_LINKER64 SYSTEMLIBS_DIR "ld-linux-x86-64.so.2"
-	#
-	sed -i ${S}/gcc/config/*/linux*.h -e \
-		's#\(GLIBC_DYNAMIC_LINKER[^ ]*\) \( *"/lib.*\)/\(.*\)#\1 SYSTEMLIBS_DIR "\3#'
-}
-
-addtask headerfix after do_unpack before do_patch
-
  do_configure_prepend () {
  	# teach gcc to find correct target includedir when checking libc ssp support
  	mkdir -p ${B}/gcc

meta/recipes-devtools/gcc/gcc-runtime_4.6.bb +0 -2

index 568b5af..97468db 100644
@@ -2,8 +2,6 @@ require gcc-${PV}.inc
  require gcc-configure-runtime.inc
  require gcc-package-runtime.inc
 
-SRC_URI_append = "file://fortran-cross-compile-hack.patch"
-
  ARCH_FLAGS_FOR_TARGET += "-isystem${STAGING_INCDIR}"
 
  EXTRA_OECONF += "--disable-libunwind-exceptions"

Quoted 5 lines

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Richard Purdie

author

2012-01-20 13:46 UTC #3 lore raw

Quoted 52 lines · Lianhao

Quoted 52 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Saul Wold
Sent: Friday, January 20, 2012 4:19 PM
To: Patches and discussions about the oe-core layer
Subject: Re: [OE-core] [PATCH] gcc: Ensure that the shared source directory shared the same sstate hashes
Quoted 9 lines · Richard Purdie
The fetch/unpack/patch/headerfix tasks are shared and hence their sstate hashes should also match. Sadly this is not the case since:
a) gcc-runtime applies an additional patch
b) The do_headerfix task was missing from libgcc
c) The do_headerfix task is a shell task and hence depends on all exported variables which can vary between cross and target recipes.
Quoted 7 lines
To fix this, the patch moves the patch to the common code, adds the headerfix task to a common include file and disabled shell dependencies on the do_headerfix task since its clear in this case we don't need thsoe dependencies since we just call sed.
With this patch applied, all these recipes now share common sstate checksums.
Richard,
I tried both a sstate build with and existing tmp and a clean tmp, the existing tmp seemed to work ok, but with a clean tmp (and sstate), I got the following patch issue still.
This was with BB_SIGNATURE_HANDLER ?= 'basichash' set.
ERROR: Command Error: exit status: 1 Output:
Could not link file
.pc/gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch/configure' to configure': No
such file or directory
Applying patch gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch
patching file configure.ac
Hunk #1 FAILED at 3073.
1 out of 1 hunk FAILED -- rejects in file configure.ac
patching file configure
Hunk #1 FAILED at 7594.
1 out of 1 hunk FAILED -- rejects in file configure
Patch gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch does not apply (enforce with -f)
ERROR: Function failed: patch_do_patch
ERROR: Logfile of failure stored in:
/intel/poky2/builds/binutils/tmp/work-shared/gcc-4.6.2+svnr181430-r20/temp/log.do_patch.15457
NOTE: package libgcc-4.6.2+svnr181430-r20: task do_patch: Failed
ERROR: Task 882
(/intel/poky2/distro/meta/recipes-devtools/gcc/libgcc_4.6.bb, do_patch)
failed with exit code '1'

Quoted 4 lines

Quoted 2 lines
I have not tried a clean sstate / clean tmp.
Well, using the poky-conrib/rpurdie/t1 branch, it seems that the libgcc still has different signature with gcc-cross-initial for task patch. But bitbake-diffsigs reports nothing. gcc-runtime seems to be ok now.

I looked into this and the problem is that the quilt-native dependency is removed in the target cases (libgcc/gcc-runtime) but not in the cross cases thanks to BB_HASHTASK_WHITELIST. I'm trying to figure out how to solve that.

Cheers,

Richard

Richard Purdie

author

2012-01-20 17:12 UTC #4 lore raw

In reply to Saul Wold

Quoted 26 lines · Saul Wold

I tried both a sstate build with and existing tmp and a clean tmp, the existing tmp seemed to work ok, but with a clean tmp (and sstate), I got the following patch issue still.
This was with BB_SIGNATURE_HANDLER ?= 'basichash' set.
ERROR: Command Error: exit status: 1 Output:
Could not link file
.pc/gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch/configure' to configure': No
such file or directory
Applying patch gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch
patching file configure.ac
Hunk #1 FAILED at 3073.
1 out of 1 hunk FAILED -- rejects in file configure.ac
patching file configure
Hunk #1 FAILED at 7594.
1 out of 1 hunk FAILED -- rejects in file configure
Patch gcc-4.3.1-ARCH_FLAGS_FOR_TARGET.patch does not apply (enforce with -f)
ERROR: Function failed: patch_do_patch
ERROR: Logfile of failure stored in:
/intel/poky2/builds/binutils/tmp/work-shared/gcc-4.6.2+svnr181430-r20/temp/log.do_patch.15457
NOTE: package libgcc-4.6.2+svnr181430-r20: task do_patch: Failed
ERROR: Task 882
(/intel/poky2/distro/meta/recipes-devtools/gcc/libgcc_4.6.bb, do_patch)
failed with exit code '1'

Quoted 1 line

I have not tried a clean sstate / clean tmp.

This only happens with basichash and is due to the stamp mismatch Lianhao mentions. I happens if the stamps are inconsistent and race against each other with the same WORKDIR. My patches should address this problem (although the posted patch has a small bug).

Cheers,

Richard