Skip to contentSkip to conversation

kernel-yocto.bbclass: set KERNEL_LD also for merge_config.sh

PATCH In review Martin Jansa 3 participants 14 messages Feb 5, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/4 kernel-yocto.bbclass: set KERNEL_LD also for merge_config.sh

* when ld-is-gold is used, merge_config.sh silently fails and doesn't generate anything useful in .config (and also include directory isn't created):

tmp-glibc-bfd-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build:
total 164K

  drwxr-xr-x 4 martin martin 4.0K Feb  5 02:27 .
  drwxrwxr-x 7 martin martin 4.0K Feb  5 02:27 ..
  -rw-rw-r-- 1 martin martin 133K Feb  5 02:28 .config
  -rw-rw-r-- 1 martin martin   39 Feb  5 02:27 .gitignore
  drwxr-xr-x 4 martin martin 4.0K Feb  5 02:27 include
  -rw-rw-r-- 1 martin martin  201 Feb  5 02:27 Makefile
  drwxrwxr-x 4 martin martin 4.0K Feb  5 02:27 scripts
  lrwxrwxrwx 1 martin martin   61 Feb  5 02:27 source -> /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source

tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build:
total 28K

  drwxr-xr-x 3 martin martin 4.0K Feb  5 02:26 .
  drwxrwxr-x 7 martin martin 4.0K Feb  5 02:26 ..
  -rw-rw-r-- 1 martin martin   74 Feb  5 02:26 .config
  -rw-rw-r-- 1 martin martin   39 Feb  5 02:26 .gitignore
  -rw-rw-r-- 1 martin martin  201 Feb  5 02:26 Makefile
  drwxrwxr-x 4 martin martin 4.0K Feb  5 02:26 scripts
  lrwxrwxrwx 1 martin martin   61 Feb  5 02:26 source -> /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source
  $ cat tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config
  CONFIG_LOCALVERSION="-yocto-standard"

* and because the failure is silent, it will happily start building kernel with default defconfig, which even builds OK for qemux86-64 but on qemux86 I've noticed this issue, because incorrectly configured kernel build fails with:

  $ tail -n 20 tmp-glibc-gold-qemux86/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/log.do_compile
    LD       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/libsubcmd-in.o
    AR       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/libsubcmd.a
    HOSTCC  scripts/mod/mk_elfconfig
    CC      scripts/mod/devicetable-offsets.s
    CC      scripts/mod/empty.o
  cc1: error: code model 'kernel' not supported in the 32 bit mode
  cc1: sorry, unimplemented: 64-bit mode not compiled in
  make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/Makefile.build:99: scripts/mod/devicetable-offsets.s] Error 1
  make[2]: *** Waiting for unfinished jobs....
  cc1: error: code model 'kernel' not supported in the 32 bit mode
  cc1: sorry, unimplemented: 64-bit mode not compiled in
  make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/Makefile.build:266: scripts/mod/empty.o] Error 1
  make[1]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:1111: prepare0] Error 2
  make[1]: *** Waiting for unfinished jobs....
    LD       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/objtool-in.o
  /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/tools/objtool
    LINK     /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/objtool
  make: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:179: sub-make] Error 2
  WARNING: exit code 1 from a shell command.

* the issue happens in log.do_kernel_configme, but the log is completely useless:

cat tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/log.do_kernel_configme
DEBUG: Executing python function extend_recipe_sysroot
NOTE: Direct dependencies are ['virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-devtools/bison/bison_3.5.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot', 'virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-extended/bc/bc_1.07.1.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/binutils/binutils-cross_2.33.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/quilt/quilt-native_0.66.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/gcc/gcc-cross_9.2.bb:do_populate_sysroot']
NOTE: Installed into sysroot: ['bison-native', 'bc-native', 'binutils-cross-i686', 'gcc-cross-i686', 'autoconf-native', 'xz-native', 'flex-native', 'gnu-config-native', 'automake-native', 'libtool-native', 'texinfo-dummy-native', 'gettext-minimal-native', 'readline-native', 'zlib-native', 'gmp-native', 'libmpc-native', 'mpfr-native', 'linux-libc-headers', 'm4-native', 'ncurses-native', 'pkgconfig-native']
NOTE: Skipping as already exists in sysroot: ['kern-tools-native', 'quilt-native']
DEBUG: sed -e 's:^[^/]*/:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot-native/:g' /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/bison-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gcc-cross-i686/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/autoconf-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gnu-config-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/automake-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/libtool-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gmp-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/ncurses-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/pkgconfig-native/fixmepath | xargs sed -i -e 's:FIXMESTAGINGDIRTARGET:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot:g; s:FIXMESTAGINGDIRHOST:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot-native:g' -e 's:FIXME_PSEUDO_SYSROOT:/OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/pseudo-native:g' -e 's:FIXME_HOSTTOOLS_DIR:/OE/build/oe-core/tmp-glibc/hosttools:g' -e 's:FIXME_PKGDATA_DIR:/OE/build/oe-core/tmp-glibc/pkgdata/qemux86:g' -e 's:FIXME_PSEUDO_LOCALSTATEDIR:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/pseudo/:g' -e 's:FIXME_LOGFIFO:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/fifo.10476:g'
DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing shell function do_kernel_configme
DEBUG: Shell function do_kernel_configme finished

]because merge_config.sh is redirected to work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log and bbfatal_log called in do_kernel_configme only when merge_config.sh fails which it should in this case but doesn't.

The merge_config_build.log shows 5.4 specific error:
scripts/Kconfig.include:39: gold linker 'i686-oe-linux-ld' not supported
but then it happily continues reporting what isn't applied in not created .config file

--- tmp-glibc-bfd-qemux86-do_kernel_configme/work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log 2020-02-05 02:28:01.455520207 +0100 +++ tmp-glibc-gold-qemux86-do_kernel_configme/work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log 2020-02-05 02:26:30.656470909 +0100 @@ -171,586 +171,5113 @@
HOSTCC scripts/kconfig/symbol.o
HOSTLD scripts/kconfig/conf
scripts/kconfig/conf --alldefconfig Kconfig

  -./.tmp.config.jDjIEYi9Yq:2208:warning: unexpected data:  #
  -./.tmp.config.jDjIEYi9Yq:2209:warning: unexpected data:  # Generic Driver Options
  -./.tmp.config.jDjIEYi9Yq:2210:warning: unexpected data:  #

-./.tmp.config.jDjIEYi9Yq:3102:warning: symbol value 'm' invalid for SAMPLE_SECCOMP -./.tmp.config.jDjIEYi9Yq:3119:warning: symbol value 'm' invalid for NF_CT_PROTO_GRE -./.tmp.config.jDjIEYi9Yq:3120:warning: symbol value 'm' invalid for NF_CT_PROTO_SCTP -./.tmp.config.jDjIEYi9Yq:3121:warning: symbol value 'm' invalid for NF_CT_PROTO_UDPLITE
-#
-# configuration written to .config
-#
+scripts/Kconfig.include:39: gold linker 'i686-oe-linux-ld' not supported
+make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/kconfig/Makefile:73: alldefconfig] Error 1 +make[1]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:567: alldefconfig] Error 2
make[1]: Leaving directory '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build'
+make: *** [Makefile:179: sub-make] Error 2
+grep: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config: No such file or directory +Value requested for CONFIG_LOCALVERSION not in final .config
+Requested value: CONFIG_LOCALVERSION=""
+Actual value:
+
+grep: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config: No such file or directory +Value requested for CONFIG_LOCALVERSION_AUTO not in final .config
+Requested value: # CONFIG_LOCALVERSION_AUTO is not set
+Actual value:

* I don't know why merge_config.sh uses separate more difficult to find merge_config_build.log instead of leaving the output to end in log.do_kernel_configme, I'll send it as separate commit so that it can be discussed there.

---
 meta/classes/kernel-yocto.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/kernel-yocto.bbclass +1 -1
index 141eeafeb9..d2796b01bf 100644
@@ -345,7 +345,7 @@ do_kernel_configme() {
 		bbfatal_log "Could not find configuration queue (${meta_dir}/config.queue)"
 	fi
 
-	CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}"	HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}" CC="${KERNEL_CC}" ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} > ${meta_dir}/cfg/merge_config_build.log 2>&1
+	CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}"	HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}" CC="${KERNEL_CC}" LD="${KERNEL_LD}" ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} > ${meta_dir}/cfg/merge_config_build.log 2>&1
 	if [ $? -ne 0 ]; then
 		bbfatal_log "Could not configure ${KMACHINE}-${LINUX_KERNEL_TYPE}"
 	fi
Signature · 1 line
2.20.1
#1 lore raw

PATCH 2/4 kernel-yocto.bbclass: do_kernel_configme: don't use +errexit, merge_config_build.log and fail when /.config wasn't created

* for whatever reason, instead of silently continuing to build default kernel config

---
 meta/classes/kernel-yocto.bbclass | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
meta/classes/kernel-yocto.bbclass +3 -4
index d2796b01bf..8e642cd999 100644
@@ -318,8 +318,6 @@ do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot"
 do_kernel_configme[depends] += "bc-native:do_populate_sysroot bison-native:do_populate_sysroot"
 do_kernel_configme[dirs] += "${S} ${B}"
 do_kernel_configme() {
-	set +e
-
 	# translate the kconfig_mode into something that merge_config.sh
 	# understands
 	case ${KCONFIG_MODE} in
@@ -345,8 +343,9 @@ do_kernel_configme() {
 		bbfatal_log "Could not find configuration queue (${meta_dir}/config.queue)"
 	fi
 
-	CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}"	HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}" CC="${KERNEL_CC}" LD="${KERNEL_LD}" ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} > ${meta_dir}/cfg/merge_config_build.log 2>&1
-	if [ $? -ne 0 ]; then
+	CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}" CC="${KERNEL_CC}" LD="${KERNEL_LD}" ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs}
+
+	if [ $? -ne 0 -o ! -f ${B}/.config ]; then
 		bbfatal_log "Could not configure ${KMACHINE}-${LINUX_KERNEL_TYPE}"
 	fi
 
Signature · 1 line
2.20.1
#2 lore raw

Re: [PATCH 2/4] kernel-yocto.bbclass: do_kernel_configme: don't use +errexit, merge_config_build.log and fail when /.config wasn't created

Quoted 22 lines · Martin Jansa

* for whatever reason, instead of silently continuing to build default kernel config

---
 meta/classes/kernel-yocto.bbclass | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
meta/classes/kernel-yocto.bbclass +0 -2
index d2796b01bf..8e642cd999 100644
@@ -318,8 +318,6 @@ do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot"
 do_kernel_configme[depends] += "bc-native:do_populate_sysroot bison-native:do_populate_sysroot"
 do_kernel_configme[dirs] += "${S} ${B}"
 do_kernel_configme() {
-       set +e
-
        # translate the kconfig_mode into something that merge_config.sh
        # understands
        case ${KCONFIG_MODE} in

meta/classes/kernel-yocto.bbclass do_kernel_configme() { · line 347

Quoted 7 lines
meta/classes/kernel-yocto.bbclass +2 -2
@@ -345,8 +343,9 @@ do_kernel_configme() {
                bbfatal_log "Could not find configuration queue (${meta_dir}/config.queue)"
        fi
 
-       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}" CC="${KERNEL_CC}" LD="${KERNEL_LD}" ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} > ${meta_dir}/cfg/merge_config_build.log 2>&1
-       if [ $? -ne 0 ]; then
+       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}" CC="${KERNEL_CC}" LD="${KERNEL_LD}" ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs}
+

At one point there was a modified merge_config that I used while the kernel caught up to the functionality that we needed, it didn't return zero in this scenario. Hence why just checking the return code was enough.

This makes sense to me, and explains the difference between our runs, and why my clean oe-core + bitbake test worked as well.

Bruce

Quoted 7 lines
meta/classes/kernel-yocto.bbclass +1 -1
+       if [ $? -ne 0 -o ! -f ${B}/.config ]; then
                bbfatal_log "Could not configure ${KMACHINE}-${LINUX_KERNEL_TYPE}"
        fi
 
--

2.20.1

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
#3 lore raw

Re: [PATCH 2/4] kernel-yocto.bbclass: do_kernel_configme: don't use +errexit, merge_config_build.log and fail when /.config wasn't created

In reply to Martin Jansa

Richard,

I've grabbed all 4 patches in this series and am running some additional tests, and will do a quick AB run to see if they interact with any of the other layers, etc. I'll send a consolidated pull request on Wednesday.

Cheers,

Bruce

Quoted 36 lines · Martin Jansa

* for whatever reason, instead of silently continuing to build default kernel config

---
 meta/classes/kernel-yocto.bbclass | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
meta/classes/kernel-yocto.bbclass +3 -5
index d2796b01bf..8e642cd999 100644
@@ -318,8 +318,6 @@ do_kernel_configme[depends] += "virtual/${TARGET_PREFIX}gcc:do_populate_sysroot"
 do_kernel_configme[depends] += "bc-native:do_populate_sysroot bison-native:do_populate_sysroot"
 do_kernel_configme[dirs] += "${S} ${B}"
 do_kernel_configme() {
-       set +e
-
        # translate the kconfig_mode into something that merge_config.sh
        # understands
        case ${KCONFIG_MODE} in
@@ -345,8 +343,9 @@ do_kernel_configme() {
                bbfatal_log "Could not find configuration queue (${meta_dir}/config.queue)"
        fi
 
-       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}" CC="${KERNEL_CC}" LD="${KERNEL_LD}" ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} > ${meta_dir}/cfg/merge_config_build.log 2>&1
-       if [ $? -ne 0 ]; then
+       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}" CC="${KERNEL_CC}" LD="${KERNEL_LD}" ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs}
+
+       if [ $? -ne 0 -o ! -f ${B}/.config ]; then
                bbfatal_log "Could not configure ${KMACHINE}-${LINUX_KERNEL_TYPE}"
        fi
 
--

2.20.1

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
#4 lore raw

PATCH 3/4 kern-tools-native: pass LD variable from shell environment to the make call

In reply to Martin Jansa

* since 5.4 kernel Kconfig will fail immediately when it detects that LD points to gold linker:

scripts/Kconfig.include:39: gold linker 'i686-oe-linux-ld' not supported

* in OE we already pass bfd linker in KERNEL_LD variable to merge_config.sh but we need to pass it also into the make call here

---
 .../kern-tools/kern-tools-native_git.bb       |  2 ++
 ...pass-LD-variable-from-shell-environm.patch | 32 +++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100644 meta/recipes-kernel/kern-tools/kern-tools/0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +2 -0
index 57ec1abc6d..8053483355 100644
@@ -22,3 +22,5 @@ do_install() {
 	cd ${S}/git
 	make DESTDIR=${D}${bindir} install
 }
+
+SRC_URI += "file://0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch;patchdir=git"
meta/recipes-kernel/kern-tools/kern-tools/0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch new file +32 -0
new file mode 100644
index 0000000000..d7d3d9dd2b
@@ -0,0 +1,32 @@
+From 091dbfede29c128a1a467d7f3c0aedc33ba3d2cb Mon Sep 17 00:00:00 2001
+From: Martin Jansa <[email protected]>
+Date: Wed, 5 Feb 2020 03:26:57 +0100
+Subject: [PATCH] merge_config.sh: pass LD variable from shell environment to
+ make
+
+* since 5.4 kernel Kconfig will fail immediately when it detects
+  that LD points to gold linker:
+
+  scripts/Kconfig.include:39:  gold linker 'i686-oe-linux-ld' not supported
+
+* in OE we already pass bfd linker in KERNEL_LD variable to merge_config.sh
+  but we need to pass it also into the make call here
+
+Signed-off-by: Martin Jansa <[email protected]>
+---
+ tools/merge_config.sh | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/tools/merge_config.sh b/tools/merge_config.sh
+index 296e759..11f3085 100755
+--- a/tools/merge_config.sh
++++ b/tools/merge_config.sh
+@@ -153,7 +153,7 @@ fi
+ # Use the merged file as the starting point for:
+ # alldefconfig: Fills in any missing symbols with Kconfig default
+ # allnoconfig: Fills in any missing symbols with # CONFIG_* is not set
+-make KCONFIG_ALLCONFIG=$TMP_FILE $OUTPUT_ARG $ALLTARGET
++make LD=$LD KCONFIG_ALLCONFIG=$TMP_FILE $OUTPUT_ARG $ALLTARGET
+ 
+ 
+ # Check all specified config values took (might have missed-dependency issues)
Signature · 1 line
2.20.1
#5 lore raw

Re: [PATCH 3/4] kern-tools-native: pass LD variable from shell environment to the make call

I'll take this into the repo directly and send a SRCREV bump after I've run it through a some of my kernel release steps.

Cheers,

Bruce

Quoted 67 lines · Martin Jansa

* since 5.4 kernel Kconfig will fail immediately when it detects that LD points to gold linker:

scripts/Kconfig.include:39: gold linker 'i686-oe-linux-ld' not supported

* in OE we already pass bfd linker in KERNEL_LD variable to merge_config.sh but we need to pass it also into the make call here

---
 .../kern-tools/kern-tools-native_git.bb       |  2 ++
 ...pass-LD-variable-from-shell-environm.patch | 32 +++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100644 meta/recipes-kernel/kern-tools/kern-tools/0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +2 -0
index 57ec1abc6d..8053483355 100644
@@ -22,3 +22,5 @@ do_install() {
        cd ${S}/git
        make DESTDIR=${D}${bindir} install
 }
+
+SRC_URI += "file://0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch;patchdir=git"
meta/recipes-kernel/kern-tools/kern-tools/0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch new file +32 -1
new file mode 100644
index 0000000000..d7d3d9dd2b
@@ -0,0 +1,32 @@
+From 091dbfede29c128a1a467d7f3c0aedc33ba3d2cb Mon Sep 17 00:00:00 2001
+From: Martin Jansa <[email protected]>
+Date: Wed, 5 Feb 2020 03:26:57 +0100
+Subject: [PATCH] merge_config.sh: pass LD variable from shell environment to
+ make
+
+* since 5.4 kernel Kconfig will fail immediately when it detects
+  that LD points to gold linker:
+
+  scripts/Kconfig.include:39:  gold linker 'i686-oe-linux-ld' not supported
+
+* in OE we already pass bfd linker in KERNEL_LD variable to merge_config.sh
+  but we need to pass it also into the make call here
+
+Signed-off-by: Martin Jansa <[email protected]>
+---
+ tools/merge_config.sh | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/tools/merge_config.sh b/tools/merge_config.sh
+index 296e759..11f3085 100755
+--- a/tools/merge_config.sh
++++ b/tools/merge_config.sh
+@@ -153,7 +153,7 @@ fi
+ # Use the merged file as the starting point for:
+ # alldefconfig: Fills in any missing symbols with Kconfig default
+ # allnoconfig: Fills in any missing symbols with # CONFIG_* is not set
+-make KCONFIG_ALLCONFIG=$TMP_FILE $OUTPUT_ARG $ALLTARGET
++make LD=$LD KCONFIG_ALLCONFIG=$TMP_FILE $OUTPUT_ARG $ALLTARGET
+
+
+ # Check all specified config values took (might have missed-dependency issues)
--

2.20.1

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
#6 lore raw

PATCH 4/4 kern-tools-native: use more common S value and oe_runmake

In reply to Martin Jansa
---
 .../kern-tools/kern-tools-native_git.bb               | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +7 -4
index 8053483355..28080e510f 100644
@@ -11,16 +11,19 @@ PV = "0.2+git${SRCPV}"
 inherit native
 
 SRC_URI = "git://git.yoctoproject.org/yocto-kernel-tools.git"
-S = "${WORKDIR}"
+S = "${WORKDIR}/git"
 UPSTREAM_CHECK_COMMITS = "1"
 
+do_configure() {
+	:
+}
+
 do_compile() { 
 	:
 }
 
 do_install() {
-	cd ${S}/git
-	make DESTDIR=${D}${bindir} install
+	oe_runmake DESTDIR=${D}${bindir} install
 }
 
-SRC_URI += "file://0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch;patchdir=git"
+SRC_URI += "file://0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch"
Signature · 1 line
2.20.1
#7 lore raw

Re: [PATCH 4/4] kern-tools-native: use more common S value and oe_runmake

Quoted 11 lines · Martin Jansa
---
 .../kern-tools/kern-tools-native_git.bb               | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +0 -0
index 8053483355..28080e510f 100644

meta/recipes-kernel/kern-tools/kern-tools-native_git.bb PV = "0.2+git${SRCPV}" · line 14

Quoted 5 lines
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +1 -1
@@ -11,16 +11,19 @@ PV = "0.2+git${SRCPV}"
 inherit native
 
 SRC_URI = "git://git.yoctoproject.org/yocto-kernel-tools.git"
-S = "${WORKDIR}"
+S = "${WORKDIR}/git"

While not common, this was on purpose. This isn't something that is typically patched, it is something I queue and bump for release.

So I'm ok with the cleanup, but I'll drop the patch tweaking part when I queue for test, since I'll have the patch already in the repo.

Bruce

Quoted 21 lines
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +6 -4
 UPSTREAM_CHECK_COMMITS = "1"
 
+do_configure() {
+       :
+}
+
 do_compile() {
        :
 }
 
 do_install() {
-       cd ${S}/git
-       make DESTDIR=${D}${bindir} install
+       oe_runmake DESTDIR=${D}${bindir} install
 }
 
-SRC_URI += "file://0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch;patchdir=git"
+SRC_URI += "file://0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch"
--

2.20.1

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
#8 lore raw

Re: [PATCH 4/4] kern-tools-native: use more common S value and oe_runmake

OK, the patchdir cleanup won't be needed as you've already applied that .patch file in the repo.

But I have forgot to update LIC_FILES_CHKSUM, which shouldn't have the "git/" anymore as it's relative to ${S}.

LIC_FILES_CHKSUM = "file://tools/kgit;beginline=5;endline=9;md5=9c30e971d435e249624278c3e343e501"

Please adjust in your consolidated pull requests.

Quoted 58 lines · Bruce Ashfield

Review

Quoted 7 lines · Martin Jansa
---
 .../kern-tools/kern-tools-native_git.bb               | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
(quoted diff) +0 -0

b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb

meta/recipes-kernel/kern-tools/kern-tools-native_git.bb PV = "0.2+git${SRCPV}" · line 14

Quoted 9 lines

index 8053483355..28080e510f 100644

meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +1 -1
@@ -11,16 +11,19 @@ PV = "0.2+git${SRCPV}"
 inherit native
 
 SRC_URI = "git://git.yoctoproject.org/yocto-kernel-tools.git"
-S = "${WORKDIR}"
+S = "${WORKDIR}/git"

While not common, this was on purpose. This isn't something that is typically patched, it is something I queue and bump for release.

So I'm ok with the cleanup, but I'll drop the patch tweaking part when I queue for test, since I'll have the patch already in the repo.

Bruce

Quoted 7 lines
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +4 -0
 UPSTREAM_CHECK_COMMITS = "1"
 
+do_configure() {
+       :
+}
+
 do_compile() {

meta/recipes-kernel/kern-tools/kern-tools-native_git.bb PV = "0.2+git${SRCPV}"

Quoted 10 lines
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +1 -3
        :
 }
 
 do_install() {
-       cd ${S}/git
-       make DESTDIR=${D}${bindir} install
+       oe_runmake DESTDIR=${D}${bindir} install
 }
 
-SRC_URI +=

"file://0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch;patchdir=git"

meta/recipes-kernel/kern-tools/kern-tools-native_git.bb PV = "0.2+git${SRCPV}" · line 29

Quoted 1 line
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +1 -0
+SRC_URI +=

"file://0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch"

Quoted 3 lines

--
2.20.1

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

#9 lore raw

Re: [PATCH 4/4] kern-tools-native: use more common S value and oe_runmake

Quoted 8 lines · Martin Jansa

OK, the patchdir cleanup won't be needed as you've already applied that .patch file in the repo.

But I have forgot to update LIC_FILES_CHKSUM, which shouldn't have the "git/" anymore as it's relative to ${S}.

LIC_FILES_CHKSUM = "file://tools/kgit;beginline=5;endline=9;md5=9c30e971d435e249624278c3e343e501"

Please adjust in your consolidated pull requests.

Absolutely, I tweaked and squashed it here.

But I've definitely taken the cleanups, since there's no valid reason to make it so hard to patch (even if I grab and merge things).

Cheers,

Bruce

Quoted 56 lines · Bruce Ashfield
Quoted 54 lines · Martin Jansa
Quoted 11 lines
---
 .../kern-tools/kern-tools-native_git.bb               | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +0 -0
index 8053483355..28080e510f 100644

meta/recipes-kernel/kern-tools/kern-tools-native_git.bb PV = "0.2+git${SRCPV}" · line 14

Quoted 5 lines
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +1 -1
@@ -11,16 +11,19 @@ PV = "0.2+git${SRCPV}"
 inherit native
 
 SRC_URI = "git://git.yoctoproject.org/yocto-kernel-tools.git"
-S = "${WORKDIR}"
+S = "${WORKDIR}/git"

While not common, this was on purpose. This isn't something that is typically patched, it is something I queue and bump for release.

So I'm ok with the cleanup, but I'll drop the patch tweaking part when I queue for test, since I'll have the patch already in the repo.

Bruce

Quoted 19 lines
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +6 -4
 UPSTREAM_CHECK_COMMITS = "1"
 
+do_configure() {
+       :
+}
+
 do_compile() {
        :
 }
 
 do_install() {
-       cd ${S}/git
-       make DESTDIR=${D}${bindir} install
+       oe_runmake DESTDIR=${D}${bindir} install
 }
 
-SRC_URI += "file://0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch;patchdir=git"
+SRC_URI += "file://0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch"
--
Quoted 2 lines

2.20.1

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
#10 lore raw

Re: [PATCH 4/4] kern-tools-native: use more common S value and oe_runmake

Quoted 6 lines · Bruce Ashfield
Quoted 5 lines · Martin Jansa

OK, the patchdir cleanup won't be needed as you've already applied that .patch file in the repo.

But I have forgot to update LIC_FILES_CHKSUM, which shouldn't have the "git/" anymore as it's relative to ${S}.

Quoted 8 lines
Quoted 3 lines

LIC_FILES_CHKSUM = "file://tools/kgit;beginline=5;endline=9;md5=9c30e971d435e249624278c3e343e501"

Please adjust in your consolidated pull requests.

Absolutely, I tweaked and squashed it here.

But I've definitely taken the cleanups, since there's no valid reason to make it so hard to patch (even if I grab and merge things).

BTW there is still another issue with gold enabled builds in runtime

When running "bitbake -k -c testimage core-image-sato-sdk-ptest" to test my artifacts changes I've noticed:

Stopped HTTPService on 0.0.0.0:0
Traceback (most recent call last):
File "/OE/build/oe-core/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
return func(*args, **kwargs)
File "/OE/build/oe-core/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
return func(*args, **kwargs)
File "/OE/build/oe-core/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
return func(*args, **kwargs)
[Previous line repeated 2 more times]
File "/OE/build/oe-core/openembedded-core/meta/lib/oeqa/runtime/cases/kernelmodule.py", line 46, in test_kernel_module

    self.assertEqual(status, 0, msg='\n'.join([cmd, output]))
AssertionError: 2 != 0 : cd /usr/src/kernel && make scripts prepare
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  YACC    scripts/kconfig/parser.tab.[ch]
  HOSTCC  scripts/kconfig/lexer.lex.o
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTLD  scripts/kconfig/conf

scripts/kconfig/conf --syncconfig Kconfig
scripts/Kconfig.include:39: gold linker 'ld' not supported
make[2]: *** [scripts/kconfig/Makefile:73: syncconfig] Error 1
make[1]: *** [Makefile:567: syncconfig] Error 2
make: *** [Makefile:678: include/config/auto.conf.cmd] Error 2
make: *** [include/config/auto.conf.cmd] Deleting file 'include/config/tristate.conf'

Looks like kernelmodule.py also needs to be adjusted to pass KERNEL_LD as LD.

Regards,

Quoted 62 lines
Quoted 55 lines · Bruce Ashfield
Quoted 54 lines · Martin Jansa
Quoted 11 lines
---
 .../kern-tools/kern-tools-native_git.bb               | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +0 -0
index 8053483355..28080e510f 100644

meta/recipes-kernel/kern-tools/kern-tools-native_git.bb PV = "0.2+git${SRCPV}" · line 14

Quoted 5 lines
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +1 -1
@@ -11,16 +11,19 @@ PV = "0.2+git${SRCPV}"
 inherit native
 
 SRC_URI = "git://git.yoctoproject.org/yocto-kernel-tools.git"
-S = "${WORKDIR}"
+S = "${WORKDIR}/git"

While not common, this was on purpose. This isn't something that is typically patched, it is something I queue and bump for release.

So I'm ok with the cleanup, but I'll drop the patch tweaking part when I queue for test, since I'll have the patch already in the repo.

Bruce

Quoted 19 lines
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +6 -4
 UPSTREAM_CHECK_COMMITS = "1"
 
+do_configure() {
+       :
+}
+
 do_compile() {
        :
 }
 
 do_install() {
-       cd ${S}/git
-       make DESTDIR=${D}${bindir} install
+       oe_runmake DESTDIR=${D}${bindir} install
 }
 
-SRC_URI += "file://0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch;patchdir=git"
+SRC_URI += "file://0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch"
--
Quoted 2 lines

2.20.1

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
#11 lore raw

✗ patchtest: failure for "kernel-yocto.bbclass: set KERN..." and 3 more

In reply to Martin Jansa

== Series Details ==

Series: "kernel-yocto.bbclass: set KERN..." and 3 more
Revision: 1
URL : https://patchwork.openembedded.org/series/22426/
State : failure

== Summary ==

Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:

* Patch            [2/4] kernel-yocto.bbclass: do_kernel_configme: don't use +errexit, merge_config_build.log and fail when /.config wasn't created
 Issue             Commit shortlog is too long [test_shortlog_length] 
  Suggested fix    Edit shortlog so that it is 90 characters or less (currently 122 characters)

* Issue Added patch file is missing Upstream-Status in the header [test_upstream_status_presence_format]

  Suggested fix    Add Upstream-Status: <Valid status> to the header of meta/recipes-kernel/kern-tools/kern-tools/0001-merge_config.sh-pass-LD-variable-from-shell-environm.patch
  Standard format  Upstream-Status: <Valid status>
  Valid status     Pending, Accepted, Backport, Denied, Inappropriate [reason], Submitted [where]

If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).

---

Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
#12 lore raw
In reply to Martin Jansa
Quoted 106 lines · Martin Jansa

* when ld-is-gold is used, merge_config.sh silently fails and doesn't generate anything useful in .config (and also include directory isn't created):

tmp-glibc-bfd-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build:
total 164K

  drwxr-xr-x 4 martin martin 4.0K Feb  5 02:27 .
  drwxrwxr-x 7 martin martin 4.0K Feb  5 02:27 ..
  -rw-rw-r-- 1 martin martin 133K Feb  5 02:28 .config
  -rw-rw-r-- 1 martin martin   39 Feb  5 02:27 .gitignore
  drwxr-xr-x 4 martin martin 4.0K Feb  5 02:27 include
  -rw-rw-r-- 1 martin martin  201 Feb  5 02:27 Makefile
  drwxrwxr-x 4 martin martin 4.0K Feb  5 02:27 scripts
  lrwxrwxrwx 1 martin martin   61 Feb  5 02:27 source -> /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source

tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build:
total 28K

  drwxr-xr-x 3 martin martin 4.0K Feb  5 02:26 .
  drwxrwxr-x 7 martin martin 4.0K Feb  5 02:26 ..
  -rw-rw-r-- 1 martin martin   74 Feb  5 02:26 .config
  -rw-rw-r-- 1 martin martin   39 Feb  5 02:26 .gitignore
  -rw-rw-r-- 1 martin martin  201 Feb  5 02:26 Makefile
  drwxrwxr-x 4 martin martin 4.0K Feb  5 02:26 scripts
  lrwxrwxrwx 1 martin martin   61 Feb  5 02:26 source -> /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source
  $ cat tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config
  CONFIG_LOCALVERSION="-yocto-standard"

* and because the failure is silent, it will happily start building kernel with default defconfig, which even builds OK for qemux86-64 but on qemux86 I've noticed this issue, because incorrectly configured kernel build fails with:

  $ tail -n 20 tmp-glibc-gold-qemux86/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/log.do_compile
    LD       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/libsubcmd-in.o
    AR       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/libsubcmd.a
    HOSTCC  scripts/mod/mk_elfconfig
    CC      scripts/mod/devicetable-offsets.s
    CC      scripts/mod/empty.o
  cc1: error: code model 'kernel' not supported in the 32 bit mode
  cc1: sorry, unimplemented: 64-bit mode not compiled in
  make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/Makefile.build:99: scripts/mod/devicetable-offsets.s] Error 1
  make[2]: *** Waiting for unfinished jobs....
  cc1: error: code model 'kernel' not supported in the 32 bit mode
  cc1: sorry, unimplemented: 64-bit mode not compiled in
  make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/Makefile.build:266: scripts/mod/empty.o] Error 1
  make[1]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:1111: prepare0] Error 2
  make[1]: *** Waiting for unfinished jobs....
    LD       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/objtool-in.o
  /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/tools/objtool
    LINK     /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/objtool
  make: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:179: sub-make] Error 2
  WARNING: exit code 1 from a shell command.

* the issue happens in log.do_kernel_configme, but the log is completely useless:

cat tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/log.do_kernel_configme
DEBUG: Executing python function extend_recipe_sysroot
NOTE: Direct dependencies are ['virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-devtools/bison/bison_3.5.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot', 'virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-extended/bc/bc_1.07.1.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/binutils/binutils-cross_2.33.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/quilt/quilt-native_0.66.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/gcc/gcc-cross_9.2.bb:do_populate_sysroot']
NOTE: Installed into sysroot: ['bison-native', 'bc-native', 'binutils-cross-i686', 'gcc-cross-i686', 'autoconf-native', 'xz-native', 'flex-native', 'gnu-config-native', 'automake-native', 'libtool-native', 'texinfo-dummy-native', 'gettext-minimal-native', 'readline-native', 'zlib-native', 'gmp-native', 'libmpc-native', 'mpfr-native', 'linux-libc-headers', 'm4-native', 'ncurses-native', 'pkgconfig-native']
NOTE: Skipping as already exists in sysroot: ['kern-tools-native', 'quilt-native']
DEBUG: sed -e 's:^[^/]*/:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot-native/:g' /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/bison-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gcc-cross-i686/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/autoconf-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gnu-config-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/automake-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/libtool-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gmp-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/ncurses-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/pkgconfig-native/fixmepath | xargs sed -i -e 's:FIXMESTAGINGDIRTARGET:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot:g; s:FIXMESTAGINGDIRHOST:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot-native:g' -e 's:FIXME_PSEUDO_SYSROOT:/OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/pseudo-native:g' -e 's:FIXME_HOSTTOOLS_DIR:/OE/build/oe-core/tmp-glibc/hosttools:g' -e 's:FIXME_PKGDATA_DIR:/OE/build/oe-core/tmp-glibc/pkgdata/qemux86:g' -e 's:FIXME_PSEUDO_LOCALSTATEDIR:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/pseudo/:g' -e 's:FIXME_LOGFIFO:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/fifo.10476:g'
DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing shell function do_kernel_configme
DEBUG: Shell function do_kernel_configme finished

]because merge_config.sh is redirected to work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log and bbfatal_log called in do_kernel_configme only when merge_config.sh fails which it should in this case but doesn't.

The merge_config_build.log shows 5.4 specific error:
scripts/Kconfig.include:39: gold linker 'i686-oe-linux-ld' not supported
but then it happily continues reporting what isn't applied in not created .config file

--- tmp-glibc-bfd-qemux86-do_kernel_configme/work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log 2020-02-05 02:28:01.455520207 +0100 +++ tmp-glibc-gold-qemux86-do_kernel_configme/work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log 2020-02-05 02:26:30.656470909 +0100 @@ -171,586 +171,5113 @@
HOSTCC scripts/kconfig/symbol.o
HOSTLD scripts/kconfig/conf
scripts/kconfig/conf --alldefconfig Kconfig

  -./.tmp.config.jDjIEYi9Yq:2208:warning: unexpected data:  #
  -./.tmp.config.jDjIEYi9Yq:2209:warning: unexpected data:  # Generic Driver Options
  -./.tmp.config.jDjIEYi9Yq:2210:warning: unexpected data:  #

-./.tmp.config.jDjIEYi9Yq:3102:warning: symbol value 'm' invalid for SAMPLE_SECCOMP -./.tmp.config.jDjIEYi9Yq:3119:warning: symbol value 'm' invalid for NF_CT_PROTO_GRE -./.tmp.config.jDjIEYi9Yq:3120:warning: symbol value 'm' invalid for NF_CT_PROTO_SCTP -./.tmp.config.jDjIEYi9Yq:3121:warning: symbol value 'm' invalid for NF_CT_PROTO_UDPLITE
-#
-# configuration written to .config
-#
+scripts/Kconfig.include:39: gold linker 'i686-oe-linux-ld' not supported
+make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/kconfig/Makefile:73: alldefconfig] Error 1 +make[1]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:567: alldefconfig] Error 2
make[1]: Leaving directory '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build'
+make: *** [Makefile:179: sub-make] Error 2
+grep: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config: No such file or directory +Value requested for CONFIG_LOCALVERSION not in final .config
+Requested value: CONFIG_LOCALVERSION=""
+Actual value:
+
+grep: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config: No such file or directory +Value requested for CONFIG_LOCALVERSION_AUTO not in final .config
+Requested value: # CONFIG_LOCALVERSION_AUTO is not set
+Actual value:

Quoted 3 lines

* I don't know why merge_config.sh uses separate more difficult to find merge_config_build.log instead of leaving the output to end in log.do_kernel_configme, I'll send it as separate commit so that it can be discussed there.

That's because there's a LOT of noise in the merge_config log, the separate task for doing the audit has heuristics to highlight important errors, and allows configuration fragments to override each other, have optional fragments, etc.

I already have the various audit level variables in use for linux-yocto style kernels, at the highest level it could be dumped, but absolutely not by default.

Cheers,

Bruce

Quoted 22 lines
---
 meta/classes/kernel-yocto.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/kernel-yocto.bbclass +1 -2
index 141eeafeb9..d2796b01bf 100644
@@ -345,7 +345,7 @@ do_kernel_configme() {
                bbfatal_log "Could not find configuration queue (${meta_dir}/config.queue)"
        fi
 
-       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}" CC="${KERNEL_CC}" ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} > ${meta_dir}/cfg/merge_config_build.log 2>&1
+       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}" CC="${KERNEL_CC}" LD="${KERNEL_LD}" ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} > ${meta_dir}/cfg/merge_config_build.log 2>&1
        if [ $? -ne 0 ]; then
                bbfatal_log "Could not configure ${KMACHINE}-${LINUX_KERNEL_TYPE}"
        fi
--

2.20.1

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
#13 lore raw
Quoted 108 lines · Bruce Ashfield
Quoted 106 lines · Martin Jansa

* when ld-is-gold is used, merge_config.sh silently fails and doesn't generate anything useful in .config (and also include directory isn't created):

tmp-glibc-bfd-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build:
total 164K

  drwxr-xr-x 4 martin martin 4.0K Feb  5 02:27 .
  drwxrwxr-x 7 martin martin 4.0K Feb  5 02:27 ..
  -rw-rw-r-- 1 martin martin 133K Feb  5 02:28 .config
  -rw-rw-r-- 1 martin martin   39 Feb  5 02:27 .gitignore
  drwxr-xr-x 4 martin martin 4.0K Feb  5 02:27 include
  -rw-rw-r-- 1 martin martin  201 Feb  5 02:27 Makefile
  drwxrwxr-x 4 martin martin 4.0K Feb  5 02:27 scripts
  lrwxrwxrwx 1 martin martin   61 Feb  5 02:27 source -> /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source

tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build:
total 28K

  drwxr-xr-x 3 martin martin 4.0K Feb  5 02:26 .
  drwxrwxr-x 7 martin martin 4.0K Feb  5 02:26 ..
  -rw-rw-r-- 1 martin martin   74 Feb  5 02:26 .config
  -rw-rw-r-- 1 martin martin   39 Feb  5 02:26 .gitignore
  -rw-rw-r-- 1 martin martin  201 Feb  5 02:26 Makefile
  drwxrwxr-x 4 martin martin 4.0K Feb  5 02:26 scripts
  lrwxrwxrwx 1 martin martin   61 Feb  5 02:26 source -> /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source
  $ cat tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config
  CONFIG_LOCALVERSION="-yocto-standard"

* and because the failure is silent, it will happily start building kernel with default defconfig, which even builds OK for qemux86-64 but on qemux86 I've noticed this issue, because incorrectly configured kernel build fails with:

  $ tail -n 20 tmp-glibc-gold-qemux86/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/log.do_compile
    LD       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/libsubcmd-in.o
    AR       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/libsubcmd.a
    HOSTCC  scripts/mod/mk_elfconfig
    CC      scripts/mod/devicetable-offsets.s
    CC      scripts/mod/empty.o
  cc1: error: code model 'kernel' not supported in the 32 bit mode
  cc1: sorry, unimplemented: 64-bit mode not compiled in
  make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/Makefile.build:99: scripts/mod/devicetable-offsets.s] Error 1
  make[2]: *** Waiting for unfinished jobs....
  cc1: error: code model 'kernel' not supported in the 32 bit mode
  cc1: sorry, unimplemented: 64-bit mode not compiled in
  make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/Makefile.build:266: scripts/mod/empty.o] Error 1
  make[1]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:1111: prepare0] Error 2
  make[1]: *** Waiting for unfinished jobs....
    LD       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/objtool-in.o
  /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/tools/objtool
    LINK     /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/objtool
  make: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:179: sub-make] Error 2
  WARNING: exit code 1 from a shell command.

* the issue happens in log.do_kernel_configme, but the log is completely useless:

cat tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/log.do_kernel_configme
DEBUG: Executing python function extend_recipe_sysroot
NOTE: Direct dependencies are ['virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-devtools/bison/bison_3.5.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot', 'virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-extended/bc/bc_1.07.1.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/binutils/binutils-cross_2.33.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/quilt/quilt-native_0.66.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/gcc/gcc-cross_9.2.bb:do_populate_sysroot']
NOTE: Installed into sysroot: ['bison-native', 'bc-native', 'binutils-cross-i686', 'gcc-cross-i686', 'autoconf-native', 'xz-native', 'flex-native', 'gnu-config-native', 'automake-native', 'libtool-native', 'texinfo-dummy-native', 'gettext-minimal-native', 'readline-native', 'zlib-native', 'gmp-native', 'libmpc-native', 'mpfr-native', 'linux-libc-headers', 'm4-native', 'ncurses-native', 'pkgconfig-native']
NOTE: Skipping as already exists in sysroot: ['kern-tools-native', 'quilt-native']
DEBUG: sed -e 's:^[^/]*/:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot-native/:g' /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/bison-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gcc-cross-i686/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/autoconf-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gnu-config-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/automake-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/libtool-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gmp-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/ncurses-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/pkgconfig-native/fixmepath | xargs sed -i -e 's:FIXMESTAGINGDIRTARGET:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot:g; s:FIXMESTAGINGDIRHOST:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot-native:g' -e 's:FIXME_PSEUDO_SYSROOT:/OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/pseudo-native:g' -e 's:FIXME_HOSTTOOLS_DIR:/OE/build/oe-core/tmp-glibc/hosttools:g' -e 's:FIXME_PKGDATA_DIR:/OE/build/oe-core/tmp-glibc/pkgdata/qemux86:g' -e 's:FIXME_PSEUDO_LOCALSTATEDIR:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/pseudo/:g' -e 's:FIXME_LOGFIFO:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/fifo.10476:g'
DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing shell function do_kernel_configme
DEBUG: Shell function do_kernel_configme finished

]because merge_config.sh is redirected to work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log and bbfatal_log called in do_kernel_configme only when merge_config.sh fails which it should in this case but doesn't.

The merge_config_build.log shows 5.4 specific error:
scripts/Kconfig.include:39: gold linker 'i686-oe-linux-ld' not supported
but then it happily continues reporting what isn't applied in not created .config file

--- tmp-glibc-bfd-qemux86-do_kernel_configme/work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log 2020-02-05 02:28:01.455520207 +0100 +++ tmp-glibc-gold-qemux86-do_kernel_configme/work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log 2020-02-05 02:26:30.656470909 +0100 @@ -171,586 +171,5113 @@
HOSTCC scripts/kconfig/symbol.o
HOSTLD scripts/kconfig/conf
scripts/kconfig/conf --alldefconfig Kconfig

  -./.tmp.config.jDjIEYi9Yq:2208:warning: unexpected data:  #
  -./.tmp.config.jDjIEYi9Yq:2209:warning: unexpected data:  # Generic Driver Options
  -./.tmp.config.jDjIEYi9Yq:2210:warning: unexpected data:  #

-./.tmp.config.jDjIEYi9Yq:3102:warning: symbol value 'm' invalid for SAMPLE_SECCOMP -./.tmp.config.jDjIEYi9Yq:3119:warning: symbol value 'm' invalid for NF_CT_PROTO_GRE -./.tmp.config.jDjIEYi9Yq:3120:warning: symbol value 'm' invalid for NF_CT_PROTO_SCTP -./.tmp.config.jDjIEYi9Yq:3121:warning: symbol value 'm' invalid for NF_CT_PROTO_UDPLITE
-#
-# configuration written to .config
-#
+scripts/Kconfig.include:39: gold linker 'i686-oe-linux-ld' not supported
+make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/kconfig/Makefile:73: alldefconfig] Error 1 +make[1]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:567: alldefconfig] Error 2
make[1]: Leaving directory '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build'
+make: *** [Makefile:179: sub-make] Error 2
+grep: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config: No such file or directory +Value requested for CONFIG_LOCALVERSION not in final .config
+Requested value: CONFIG_LOCALVERSION=""
+Actual value:
+
+grep: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config: No such file or directory +Value requested for CONFIG_LOCALVERSION_AUTO not in final .config
+Requested value: # CONFIG_LOCALVERSION_AUTO is not set
+Actual value:

Quoted 8 lines
Quoted 3 lines

* I don't know why merge_config.sh uses separate more difficult to find merge_config_build.log instead of leaving the output to end in log.do_kernel_configme, I'll send it as separate commit so that it can be discussed there.

That's because there's a LOT of noise in the merge_config log, the separate task for doing the audit has heuristics to highlight important errors, and allows configuration fragments to override each other, have optional fragments, etc.

But of course, just allowing stderr into the log makes sense, since the fragment messages themselves are stdout. That way we can show any future traps/failures in the single log, and the audit step can offer some more nuanced processing of the output.

I pulled together a patch that splits the output, and another value for the log level that will allow everything into logs .. and am running them through tests now.

Cheers,

Bruce

Quoted 37 lines

I already have the various audit level variables in use for linux-yocto style kernels, at the highest level it could be dumped, but absolutely not by default.

Cheers,

Bruce

Quoted 20 lines
---
 meta/classes/kernel-yocto.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/kernel-yocto.bbclass +1 -2
index 141eeafeb9..d2796b01bf 100644
@@ -345,7 +345,7 @@ do_kernel_configme() {
                bbfatal_log "Could not find configuration queue (${meta_dir}/config.queue)"
        fi
 
-       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}" CC="${KERNEL_CC}" ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} > ${meta_dir}/cfg/merge_config_build.log 2>&1
+       CFLAGS="${CFLAGS} ${TOOLCHAIN_OPTIONS}" HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="${BUILD_CPP}" CC="${KERNEL_CC}" LD="${KERNEL_LD}" ARCH=${ARCH} merge_config.sh -O ${B} ${config_flags} ${configs} > ${meta_dir}/cfg/merge_config_build.log 2>&1
        if [ $? -ne 0 ]; then
                bbfatal_log "Could not configure ${KMACHINE}-${LINUX_KERNEL_TYPE}"
        fi
--
Quoted 2 lines

2.20.1

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II