Skip to contentSkip to conversation

module.bbclass: swap AR and LD order

PATCH In review Jason Wessel 3 participants 6 messages Jan 23, 2020 – Jan 24, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 module.bbclass: swap AR and LD order

The 5.x kernels seem to have made a change to the linker command line processing.

When trying to build out of tree kernel modules, such as the virtualbox guest additions, the following error is printed:

| make[1]: Entering directory '/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo'
| make[3]: Entering directory '/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
| make V= CONFIG_MODULE_SIG= CONFIG_MODULE_SIG_ALL= -C /opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/5.2.28-yocto-standard/build M=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest SRCROOT=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest modules
| make[3]: Entering directory '/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
|   LD [M]  /opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo/vboxvideo.o
| x86_64-poky-linux-ld.bfd: cannot find AR=x86_64-poky-linux-ar: No such file or directory

The AR arguments passed to the make command are being processed as command line arguments to the linker due to some changes in the kernel's Makefile. For now the easiest fix is to just swap the arguments in the module.bbclass.

---
 meta/classes/module.bbclass | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
meta/classes/module.bbclass +1 -2
index c0dfa35061..e1e5fdaa72 100644
@@ -38,8 +38,7 @@ module_do_compile() {
 	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
 	oe_runmake KERNEL_PATH=${STAGING_KERNEL_DIR}   \
 		   KERNEL_VERSION=${KERNEL_VERSION}    \
-		   CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
-		   AR="${KERNEL_AR}" \
+		   CC="${KERNEL_CC}" AR="${KERNEL_AR}" LD="${KERNEL_LD}" \
 	           O=${STAGING_KERNEL_BUILDDIR} \
 		   KBUILD_EXTRA_SYMBOLS="${KBUILD_EXTRA_SYMBOLS}" \
 		   ${MAKE_TARGETS}
Signature · 1 line
2.23.0
#1 lore raw

What makefile change caused this? That behavior doesn't make much sense given how make processes its command-line arguments.

Quoted 55 lines · Jason Wessel

The 5.x kernels seem to have made a change to the linker command line processing.

When trying to build out of tree kernel modules, such as the virtualbox guest additions, the following error is printed:

| make[1]: Entering directory '/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo' | make[3]: Entering directory '/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
| make V= CONFIG_MODULE_SIG= CONFIG_MODULE_SIG_ALL= -C /opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/5.2.28-yocto-standard/build M=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest SRCROOT=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest modules
| make[3]: Entering directory '/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
| LD [M]
/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo/vboxvideo.o | x86_64-poky-linux-ld.bfd: cannot find AR=x86_64-poky-linux-ar: No such
file or directory

The AR arguments passed to the make command are being processed as command line arguments to the linker due to some changes in the kernel's Makefile. For now the easiest fix is to just swap the arguments in the module.bbclass.

---
 meta/classes/module.bbclass | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
meta/classes/module.bbclass +1 -3
index c0dfa35061..e1e5fdaa72 100644
@@ -38,8 +38,7 @@ module_do_compile() {
        unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
        oe_runmake KERNEL_PATH=${STAGING_KERNEL_DIR}   \
                   KERNEL_VERSION=${KERNEL_VERSION}    \
-                  CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
-                  AR="${KERNEL_AR}" \
+                  CC="${KERNEL_CC}" AR="${KERNEL_AR}" LD="${KERNEL_LD}" \
                   O=${STAGING_KERNEL_BUILDDIR} \
                   KBUILD_EXTRA_SYMBOLS="${KBUILD_EXTRA_SYMBOLS}" \
                   ${MAKE_TARGETS}
--

2.23.0

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Signature · 4 lines
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Senior Software Engineer, Mentor Graphics
#2 lore raw
Quoted 2 lines · Christopher Larson

What makefile change caused this? That behavior doesn't make much sense given how make processes its command-line arguments.

I agree with you here, it could be a rare check where one want to define what collect progam should be used ( ld or ar )

besides, recently we moved AR to be gcc-ar/llvm-ar by default in config metadata, which is not going to work out of box for compiling kernel and modules, so overriding it with KERNEL_AR in module_do_compile/install/configure tasks is good change.

Quoted 68 lines · On Thu, Jan 23, 2020 at 3:34 PM Jason Wessel <[email protected]…
    The 5.x kernels seem to have made a change to the linker command
    line processing.

    When trying to build out of tree kernel modules, such as the
    virtualbox guest additions, the following error is printed:

    | make[1]: Entering directory
    '/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo'
    | make[3]: Entering directory
    '/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
    | make V= CONFIG_MODULE_SIG= CONFIG_MODULE_SIG_ALL= -C
    /opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/5.2.28-yocto-standard/build
    M=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest
    SRCROOT=/opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxguest
    modules
    | make[3]: Entering directory
    '/opt/build-x86-64/tmp/work-shared/qemux86-64/kernel-build-artifacts'
    |   LD [M] 
    /opt/build-x86-64/tmp/work/qemux86_64-poky-linux/vboxguestdrivers/6.1.2-r0/vbox_module/vboxvideo/vboxvideo.o
    | x86_64-poky-linux-ld.bfd: cannot find AR=x86_64-poky-linux-ar: No
    such file or directory

    The AR arguments passed to the make command are being processed as
    command line arguments to the linker due to some changes in the
    kernel's Makefile.  For now the easiest fix is to just swap the
    arguments in the module.bbclass.

    Signed-off-by: Jason Wessel <[email protected]
    <mailto:[email protected]>>
    ---
      meta/classes/module.bbclass | 3 +--
      1 file changed, 1 insertion(+), 2 deletions(-)

    diff --git a/meta/classes/module.bbclass b/meta/classes/module.bbclass
    index c0dfa35061..e1e5fdaa72 100644
    --- a/meta/classes/module.bbclass
    +++ b/meta/classes/module.bbclass
    @@ -38,8 +38,7 @@ module_do_compile() {
             unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
             oe_runmake KERNEL_PATH=${STAGING_KERNEL_DIR}   \
                        KERNEL_VERSION=${KERNEL_VERSION}    \
    -                  CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
    -                  AR="${KERNEL_AR}" \
    +                  CC="${KERNEL_CC}" AR="${KERNEL_AR}"
    LD="${KERNEL_LD}" \
                        O=${STAGING_KERNEL_BUILDDIR} \
                        KBUILD_EXTRA_SYMBOLS="${KBUILD_EXTRA_SYMBOLS}" \
                        ${MAKE_TARGETS}
    -- 
    2.23.0

    -- 
    _______________________________________________
    Openembedded-core mailing list
    [email protected]
    <mailto:[email protected]>
    http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 4 lines
Christopher Larson
kergoth at gmail dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Senior Software Engineer, Mentor Graphics
#3 lore raw
Quoted 5 lines · Khem Raj
Quoted 3 lines · Christopher Larson

What makefile change caused this? That behavior doesn't make much sense given how make processes its command-line arguments.

Quoted 7 lines

I agree with you here, it could be a rare check where one want to define what collect progam should be used ( ld or ar )

besides, recently we moved AR to be gcc-ar/llvm-ar by default in config metadata, which is not going to work out of box for compiling kernel and modules, so overriding it with KERNEL_AR in module_do_compile/install/configure tasks is good change.

I am not actually sure which part of kbuild is parsing the LD arguments. My best guess is that it is actually all the nested gnumake filter calls causing the issue.

With some further experimentation I determined that a space at the end of the LD= is what causes it to think the next argument is part of the prior one. The poky distro is defining KERNEL_LD as:

% bitbake vboxguestdrivers -e |grep ^KERNEL_LD
KERNEL_LD="x86_64-poky-linux-ld.bfd "

The kbuild really doesn't like that space at the end. You can trash out a host native kernel compile module build by adding the arguments:

    LD="ld " JUNK_VAR="junk"

Attempting to debug/fix kbuild doesn't seem like the right answer.

The next version of this patch looks like what is below. Would that be ok?

meta/classes/module.bbclass +3 -0
index c0dfa35061..7b0a85a06d 100644
@@ -12,6 +12,9 @@ python __anonymous () {
         if dep.startswith("kernel-module-"):
             extra_symbols.append("${STAGING_INCDIR}/" + dep + "/Module.symvers")
     d.setVar('KBUILD_EXTRA_SYMBOLS', " ".join(extra_symbols))
+    d.setVar('KERNEL_CC', d.getVar('KERNEL_CC').rstrip())
+    d.setVar('KERNEL_LD', d.getVar('KERNEL_LD').rstrip())
+    d.setVar('KERNEL_AR', d.getVar('KERNEL_AR').rstrip())
 }
 
 
#4 lore raw
Quoted 40 lines · Jason Wessel
Quoted 5 lines · Khem Raj
Quoted 3 lines · Christopher Larson

What makefile change caused this? That behavior doesn't make much sense given how make processes its command-line arguments.

Quoted 7 lines

I agree with you here, it could be a rare check where one want to define what collect progam should be used ( ld or ar )

besides, recently we moved AR to be gcc-ar/llvm-ar by default in config metadata, which is not going to work out of box for compiling kernel and modules, so overriding it with KERNEL_AR in module_do_compile/install/configure tasks is good change.

I am not actually sure which part of kbuild is parsing the LD arguments. My best guess is that it is actually all the nested gnumake filter calls causing the issue.

With some further experimentation I determined that a space at the end of the LD= is what causes it to think the next argument is part of the prior one. The poky distro is defining KERNEL_LD as:

% bitbake vboxguestdrivers -e |grep ^KERNEL_LD
KERNEL_LD="x86_64-poky-linux-ld.bfd "

The kbuild really doesn't like that space at the end. You can trash out a host native kernel compile module build by adding the arguments:

    LD="ld " JUNK_VAR="junk"

Attempting to debug/fix kbuild doesn't seem like the right answer.

The next version of this patch looks like what is below. Would that be ok?

meta/classes/module.bbclass +0 -0
index c0dfa35061..7b0a85a06d 100644

meta/classes/module.bbclass python __anonymous () { · line 19

Quoted 8 lines
meta/classes/module.bbclass +3 -0
@@ -12,6 +12,9 @@ python __anonymous () {
         if dep.startswith("kernel-module-"):
             extra_symbols.append("${STAGING_INCDIR}/" + dep + "/Module.symvers")
     d.setVar('KBUILD_EXTRA_SYMBOLS', " ".join(extra_symbols))
+    d.setVar('KERNEL_CC', d.getVar('KERNEL_CC').rstrip())
+    d.setVar('KERNEL_LD', d.getVar('KERNEL_LD').rstrip())
+    d.setVar('KERNEL_AR', d.getVar('KERNEL_AR').rstrip())
 }
 

no one else uses KERNEL_* vars, so if modules cant use it then we have an issue, perhaps just delete it in base class where KERNEL_* vars are set

#5 lore raw
Quoted 36 lines · Khem Raj
Quoted 35 lines · Jason Wessel
Quoted 5 lines · Khem Raj
Quoted 3 lines · Christopher Larson

What makefile change caused this? That behavior doesn't make much sense given how make processes its command-line arguments.

Quoted 7 lines

I agree with you here, it could be a rare check where one want to define what collect progam should be used ( ld or ar )

besides, recently we moved AR to be gcc-ar/llvm-ar by default in config metadata, which is not going to work out of box for compiling kernel and modules, so overriding it with KERNEL_AR in module_do_compile/install/configure tasks is good change.

I am not actually sure which part of kbuild is parsing the LD arguments. My best guess is that it is actually all the nested gnumake filter calls causing the issue.

With some further experimentation I determined that a space at the end of the LD= is what causes it to think the next argument is part of the prior one. The poky distro is defining KERNEL_LD as:

% bitbake vboxguestdrivers -e |grep ^KERNEL_LD
KERNEL_LD="x86_64-poky-linux-ld.bfd "

The kbuild really doesn't like that space at the end. You can trash out a host native kernel compile module build by adding the arguments:

    LD="ld " JUNK_VAR="junk"

Attempting to debug/fix kbuild doesn't seem like the right answer.

The next version of this patch looks like what is below. Would that be ok?

Quoted 17 lines

meta/classes/module.bbclass python __anonymous () { · line 19

Quoted 13 lines
meta/classes/module.bbclass +3 -0
index c0dfa35061..7b0a85a06d 100644
@@ -12,6 +12,9 @@ python __anonymous () {
         if dep.startswith("kernel-module-"):
             extra_symbols.append("${STAGING_INCDIR}/" + dep + "/Module.symvers")
     d.setVar('KBUILD_EXTRA_SYMBOLS', " ".join(extra_symbols))
+    d.setVar('KERNEL_CC', d.getVar('KERNEL_CC').rstrip())
+    d.setVar('KERNEL_LD', d.getVar('KERNEL_LD').rstrip())
+    d.setVar('KERNEL_AR', d.getVar('KERNEL_AR').rstrip())
 }
 

no one else uses KERNEL_* vars, so if modules cant use it then we have an issue, perhaps just delete it in base class where KERNEL_* vars are set

I am not sure I follow what you want to see changed. Do you want the to try and fix up the problem where the variables are initially set? Example:

classes/kernel-arch.bbclass:KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd ${HOST_LD_KERNEL_ARCH}"

I see there are some BSPs that append to KERNEL_LD, so this problem could happen at a later point.

The other possibility is adding an echo to force the trimming without using the python.

@@ -41,8 +38,8 @@ module_do_compile() {
        unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
        oe_runmake KERNEL_PATH=${STAGING_KERNEL_DIR}   \
                   KERNEL_VERSION=${KERNEL_VERSION}    \
-                  CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
-                  AR="${KERNEL_AR}" \
+                  CC="$(echo ${KERNEL_CC})" LD="$(echo ${KERNEL_LD})" \
+                  AR="$(echo ${KERNEL_AR})" \
                   O=${STAGING_KERNEL_BUILDDIR} \
                   KBUILD_EXTRA_SYMBOLS="${KBUILD_EXTRA_SYMBOLS}" \
                   ${MAKE_TARGETS}
@@ -52,7 +49,7 @@ module_do_install() {
        unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
        oe_runmake DEPMOD=echo MODLIB="${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}" \
                   INSTALL_FW_PATH="${D}${nonarch_base_libdir}/firmware" \
-                  CC="${KERNEL_CC}" LD="${KERNEL_LD}" \
+                  CC="$(echo ${KERNEL_CC})" LD="$(echo ${KERNEL_LD})" \
                   O=${STAGING_KERNEL_BUILDDIR} \
                   ${MODULES_INSTALL_TARGET}