make-mod-scripts: Provide the correct objcopy to kernel make
PATCH 1/1 make-mod-scripts: Provide the correct objcopy to kernel make
When cross-compiling with v5.12-rc3, prepare fails[1] build of vdso at the objcopy stage since it seems to be using the local host's objcopy rather than the cross-compile version we want it to use.
This can be trivially reproduced in a localbuild of the kernel following the build parameters provided in the process[2]
Lets fix this by passing OBJCOPY over to the kernel.
[1] https://pastebin.ubuntu.com/p/pNcQtb93wr/
[2] https://pastebin.ubuntu.com/p/vZGqgh9Sq5/
---
NOTE:
1. This is a different problem and is not resolved with
https://lists.openembedded.org/g/openembedded-core/message/149943
2. Though reproduced during a dunfell build, this should probably apply else where as well.
3. I have'nt been able to cleanly bisect things, but my best guess was that this might be a consequence of kernel fixup exposing this.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a5b8ca97fbf8300a5e21c393df25ce6f521e7939
meta/classes/kernel-arch.bbclass | 3 +++ meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-)
meta/classes/kernel-arch.bbclass +3 -0
@@ -60,9 +60,12 @@ TARGET_LD_KERNEL_ARCH ?= "" HOST_LD_KERNEL_ARCH ?= "${TARGET_LD_KERNEL_ARCH}" TARGET_AR_KERNEL_ARCH ?= "" HOST_AR_KERNEL_ARCH ?= "${TARGET_AR_KERNEL_ARCH}" +TARGET_OBJCOPY_KERNEL_ARCH ?= "" +HOST_OBJCOPY_KERNEL_ARCH ?= "${TARGET_OBJCOPY_KERNEL_ARCH}" KERNEL_CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_KERNEL_ARCH} -fuse-ld=bfd ${DEBUG_PREFIX_MAP} -fdebug-prefix-map=${STAGING_KERNEL_DIR}=${KERNEL_SRC_PATH}" KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd ${HOST_LD_KERNEL_ARCH}" KERNEL_AR = "${CCACHE}${HOST_PREFIX}ar ${HOST_AR_KERNEL_ARCH}" +KERNEL_OBJCOPY = "${CCACHE}${HOST_PREFIX}objcopy ${HOST_OBJCOPY_KERNEL_ARCH}" TOOLCHAIN = "gcc"
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +1 -1
@@ -25,7 +25,7 @@ EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP=" do_configure() { unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS for t in prepare scripts_basic scripts; do - oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \ + oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" OBJCOPY="${KERNEL_OBJCOPY}"\ -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t done }
Signature · 1 line
2.31.0
Hi, Bruce,
Would you mind taking a look at this? Have you seen any issues building latest kernel 5.12-rc for aarch64 and seeing host objcopy being used instead of cross compile one? Please let us know. Thanks!
Signature · 1 line
Denys
Quoted 62 lines · Nishanth Menon
When cross-compiling with v5.12-rc3, prepare fails[1] build of vdso at the objcopy stage since it seems to be using the local host's objcopy rather than the cross-compile version we want it to use.
This can be trivially reproduced in a localbuild of the kernel following the build parameters provided in the process[2]
Lets fix this by passing OBJCOPY over to the kernel.
[1] https://pastebin.ubuntu.com/p/pNcQtb93wr/
[2] https://pastebin.ubuntu.com/p/vZGqgh9Sq5/Signed-off-by Nishanth Menon <[email protected]>---
NOTE:
1. This is a different problem and is not resolved with
https://lists.openembedded.org/g/openembedded-core/message/1499432. Though reproduced during a dunfell build, this should probably apply else where as well.
3. I have'nt been able to cleanly bisect things, but my best guess was that this might be a consequence of kernel fixup exposing this.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a5b8ca97fbf8300a5e21c393df25ce6f521e7939meta/classes/kernel-arch.bbclass | 3 +++ meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-)meta/classes/kernel-arch.bbclass +3
-0@@ -60,9 +60,12 @@ TARGET_LD_KERNEL_ARCH ?= "" HOST_LD_KERNEL_ARCH ?= "${TARGET_LD_KERNEL_ARCH}" TARGET_AR_KERNEL_ARCH ?= "" HOST_AR_KERNEL_ARCH ?= "${TARGET_AR_KERNEL_ARCH}" +TARGET_OBJCOPY_KERNEL_ARCH ?= "" +HOST_OBJCOPY_KERNEL_ARCH ?= "${TARGET_OBJCOPY_KERNEL_ARCH}" KERNEL_CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_KERNEL_ARCH} -fuse-ld=bfd ${DEBUG_PREFIX_MAP} -fdebug-prefix-map=${STAGING_KERNEL_DIR}=${KERNEL_SRC_PATH}" KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd ${HOST_LD_KERNEL_ARCH}" KERNEL_AR = "${CCACHE}${HOST_PREFIX}ar ${HOST_AR_KERNEL_ARCH}" +KERNEL_OBJCOPY = "${CCACHE}${HOST_PREFIX}objcopy ${HOST_OBJCOPY_KERNEL_ARCH}" TOOLCHAIN = "gcc"meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +1
-1@@ -25,7 +25,7 @@ EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP=" do_configure() { unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS for t in prepare scripts_basic scripts; do - oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \ + oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" OBJCOPY="${KERNEL_OBJCOPY}"\ -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t done }Signature · 1 line
2.31.0
Quoted 6 lines · Denys Dmytriyenko
Hi, Bruce,
Would you mind taking a look at this? Have you seen any issues building latest kernel 5.12-rc for aarch64 and seeing host objcopy being used instead of cross compile one? Please let us know. Thanks!
I've been able to build and boot all arches on v5.12-rc4, and I did run into something similar on 5.4 kernel's, but it was addressed in -stable backports.
That being said, let me test this over the next couple of days and follow up. I may not have had Ross' changes in my tree when I was sanity testing 5.12.
Bruce
Quoted 52 lines
--
DenysQuoted 46 lines · Nishanth Menon
When cross-compiling with v5.12-rc3, prepare fails[1] build of vdso at the objcopy stage since it seems to be using the local host's objcopy rather than the cross-compile version we want it to use.
This can be trivially reproduced in a localbuild of the kernel following the build parameters provided in the process[2]
Lets fix this by passing OBJCOPY over to the kernel.
[1] https://pastebin.ubuntu.com/p/pNcQtb93wr/
[2] https://pastebin.ubuntu.com/p/vZGqgh9Sq5/Signed-off-by Nishanth Menon <[email protected]>---
NOTE:
1. This is a different problem and is not resolved with
https://lists.openembedded.org/g/openembedded-core/message/1499432. Though reproduced during a dunfell build, this should probably apply else where as well.
3. I have'nt been able to cleanly bisect things, but my best guess was that this might be a consequence of kernel fixup exposing this.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a5b8ca97fbf8300a5e21c393df25ce6f521e7939meta/classes/kernel-arch.bbclass | 3 +++ meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-)meta/classes/kernel-arch.bbclass +3
-0@@ -60,9 +60,12 @@ TARGET_LD_KERNEL_ARCH ?= "" HOST_LD_KERNEL_ARCH ?= "${TARGET_LD_KERNEL_ARCH}" TARGET_AR_KERNEL_ARCH ?= "" HOST_AR_KERNEL_ARCH ?= "${TARGET_AR_KERNEL_ARCH}" +TARGET_OBJCOPY_KERNEL_ARCH ?= "" +HOST_OBJCOPY_KERNEL_ARCH ?= "${TARGET_OBJCOPY_KERNEL_ARCH}" KERNEL_CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_KERNEL_ARCH} -fuse-ld=bfd ${DEBUG_PREFIX_MAP} -fdebug-prefix-map=${STAGING_KERNEL_DIR}=${KERNEL_SRC_PATH}" KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd ${HOST_LD_KERNEL_ARCH}" KERNEL_AR = "${CCACHE}${HOST_PREFIX}ar ${HOST_AR_KERNEL_ARCH}" +KERNEL_OBJCOPY = "${CCACHE}${HOST_PREFIX}objcopy ${HOST_OBJCOPY_KERNEL_ARCH}" TOOLCHAIN = "gcc"
Quoted 16 lines
Quoted 14 lines
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +1
-2@@ -25,7 +25,7 @@ EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP=" do_configure() { unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS for t in prepare scripts_basic scripts; do - oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \ + oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" OBJCOPY="${KERNEL_OBJCOPY}"\ -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t done } --Quoted 2 lines
2.31.0
--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
Quoted 10 lines · Nishanth Menon via lists.openembedded.org
When cross-compiling with v5.12-rc3, prepare fails[1] build of vdso at the objcopy stage since it seems to be using the local host's objcopy rather than the cross-compile version we want it to use.
This can be trivially reproduced in a localbuild of the kernel following the build parameters provided in the process[2]
Lets fix this by passing OBJCOPY over to the kernel.
As I mentioned to Denys, I hadn't seen this. Consulting the maintainers file would have found me as a good addition to the cc'.
I'm doing some other work on make-mod-scripts dependencies right now, so I've pulled this in and will re-test against all of the active kernel versions in master.
But I don't think that make-mod-scripts is the only place we'd need this, if it is indeed happening on the tip of all the supported versions. There are routes to have the prepare steps run, without a make-mod-scripts dependency.
We've already made the mistake of letting the make-mod-scrips and kernel build parameters diverge (see AR=${KERNEL_AR}), so I need to spend a few minutes getting them back in sync.
I was just able to build and boot qemuarm64 on 5.12-rc4, so there's something different about your config than my setup. We need to figure that out. It could be a .config triggering different components to be built, but unlikely given vdso being involved.
qemuarm64 login: root
root@qemuarm64:~# uname -a Linux qemuarm64 5.12.0-rc4-yoctodev-standard #1 SMP PREEMPT Mon Mar 22 18:46:22 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux root@qemuarm64:~#
Quoted 25 lines
[1] https://pastebin.ubuntu.com/p/pNcQtb93wr/
[2] https://pastebin.ubuntu.com/p/vZGqgh9Sq5/Signed-off-by Nishanth Menon <[email protected]>---
NOTE:
1. This is a different problem and is not resolved with
https://lists.openembedded.org/g/openembedded-core/message/1499432. Though reproduced during a dunfell build, this should probably apply else where as well.
3. I have'nt been able to cleanly bisect things, but my best guess was that this might be a consequence of kernel fixup exposing this.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a5b8ca97fbf8300a5e21c393df25ce6f521e7939meta/classes/kernel-arch.bbclass | 3 +++ meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-)meta/classes/kernel-arch.bbclass +0
-0
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 65
Quoted 6 lines
meta/classes/kernel-arch.bbclass +2
-0@@ -60,9 +60,12 @@ TARGET_LD_KERNEL_ARCH ?= "" HOST_LD_KERNEL_ARCH ?= "${TARGET_LD_KERNEL_ARCH}" TARGET_AR_KERNEL_ARCH ?= "" HOST_AR_KERNEL_ARCH ?= "${TARGET_AR_KERNEL_ARCH}" +TARGET_OBJCOPY_KERNEL_ARCH ?= "" +HOST_OBJCOPY_KERNEL_ARCH ?= "${TARGET_OBJCOPY_KERNEL_ARCH}"
We shouldn't need this TARGET_OBJCOPY_KERNEL_ARCH -> HOST_OBJCOPY_KERNEL_ARCH, I can't think of how objcopy would be using them.
Are you setting them to some value in your builds ?
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 69
Quoted 4 lines
meta/classes/kernel-arch.bbclass +1
-0KERNEL_CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_KERNEL_ARCH} -fuse-ld=bfd ${DEBUG_PREFIX_MAP} -fdebug-prefix-map=${STAGING_KERNEL_DIR}=${KERNEL_SRC_PATH}" KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd ${HOST_LD_KERNEL_ARCH}" KERNEL_AR = "${CCACHE}${HOST_PREFIX}ar ${HOST_AR_KERNEL_ARCH}" +KERNEL_OBJCOPY = "${CCACHE}${HOST_PREFIX}objcopy ${HOST_OBJCOPY_KERNEL_ARCH}"
The main kernel Makefile already defines, and the standard env should already have both CROSS_COMPILE and OBJCOPY defined to be the target version.
Makefile:OBJCOPY = $(CROSS_COMPILE)objcopy
So we really have to pass this, when fundamentally it is the same definition as what the defaults give us.
What does that resolve to in your builds ? In mine, when I dump the objcopy value from within the kernel build, I get:
| /opt/poky/build/tmp/work-shared/qemuarm64/kernel-source/Makefile:443:
*** objcopy: aarch64-poky-linux-objcopy. Stop.
Which should be doing the job.
Are you building arm on arm ? or something else like that ?
Bruce
Quoted 21 lines
meta/classes/kernel-arch.bbclass +0
-0TOOLCHAIN = "gcc"meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +1
-2@@ -25,7 +25,7 @@ EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP=" do_configure() { unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS for t in prepare scripts_basic scripts; do - oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \ + oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" OBJCOPY="${KERNEL_OBJCOPY}"\ -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t done } --2.31.0
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
Quoted 7 lines · Bruce Ashfield
Quoted 5 lines · Nishanth Menon via lists.openembedded.org
When cross-compiling with v5.12-rc3, prepare fails[1] build of vdso at the objcopy stage since it seems to be using the local host's objcopy rather than the cross-compile version we want it to use.
Quoted 9 lines
Quoted 5 lines
This can be trivially reproduced in a localbuild of the kernel following the build parameters provided in the process[2]
Lets fix this by passing OBJCOPY over to the kernel.
As I mentioned to Denys, I hadn't seen this. Consulting the maintainers file would have found me as a good addition to the cc'.
Sure. understood. Thanks..
Quoted 4 lines
I'm doing some other work on make-mod-scripts dependencies right now, so I've pulled this in and will re-test against all of the active kernel versions in master.
Thanks.
Quoted 14 lines
But I don't think that make-mod-scripts is the only place we'd need this, if it is indeed happening on the tip of all the supported versions. There are routes to have the prepare steps run, without a make-mod-scripts dependency.
We've already made the mistake of letting the make-mod-scrips and kernel build parameters diverge (see AR=${KERNEL_AR}), so I need to spend a few minutes getting them back in sync.
I was just able to build and boot qemuarm64 on 5.12-rc4, so there's something different about your config than my setup. We need to figure that out. It could be a .config triggering different components to be built, but unlikely given vdso being involved.
Quoted 5 lines
qemuarm64 login: root
root@qemuarm64:~# uname -a Linux qemuarm64 5.12.0-rc4-yoctodev-standard #1 SMP PREEMPT Mon Mar 22 18:46:22 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux root@qemuarm64:~#
Hmm... only thing I have done is cross compile - see the pastebins.
Quoted 4 lines
Quoted 4 lines
[1] https://pastebin.ubuntu.com/p/pNcQtb93wr/
[2] https://pastebin.ubuntu.com/p/vZGqgh9Sq5/Signed-off-by Nishanth Menon <[email protected]>
[...]
Quoted 17 lines
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 65
Quoted 11 lines
meta/classes/kernel-arch.bbclass +2
-0@@ -60,9 +60,12 @@ TARGET_LD_KERNEL_ARCH ?= "" HOST_LD_KERNEL_ARCH ?= "${TARGET_LD_KERNEL_ARCH}" TARGET_AR_KERNEL_ARCH ?= "" HOST_AR_KERNEL_ARCH ?= "${TARGET_AR_KERNEL_ARCH}" +TARGET_OBJCOPY_KERNEL_ARCH ?= "" +HOST_OBJCOPY_KERNEL_ARCH ?= "${TARGET_OBJCOPY_KERNEL_ARCH}"We shouldn't need this
TARGET_OBJCOPY_KERNEL_ARCH->HOST_OBJCOPY_KERNEL_ARCH, I can't think of how objcopy would be using them.
Quoted 1 line
Are you setting them to some value in your builds ?
No, I am not. I decided to maintain consistency of usage from LD and AR. I could'nt think of a usage either, true.
Quoted 11 lines
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 69
Quoted 5 lines
meta/classes/kernel-arch.bbclass +1
-0KERNEL_CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_KERNEL_ARCH} -fuse-ld=bfd ${DEBUG_PREFIX_MAP} -fdebug-prefix-map=${STAGING_KERNEL_DIR}=${KERNEL_SRC_PATH}" KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd ${HOST_LD_KERNEL_ARCH}" KERNEL_AR = "${CCACHE}${HOST_PREFIX}ar ${HOST_AR_KERNEL_ARCH}" +KERNEL_OBJCOPY = "${CCACHE}${HOST_PREFIX}objcopy ${HOST_OBJCOPY_KERNEL_ARCH}"The main kernel
Makefilealready defines, and the standard env should already have bothCROSS_COMPILEand OBJCOPY defined to be the target version.Makefile:OBJCOPY = $(CROSS_COMPILE)objcopy
OK this is what is confusing me -> I dont see CROSS_COMPILE being set - which is what I had expected in the first place. other than meta/recipes-kernel/perf/perf.bb, I dont see it set else where -> I was really wondering why all the specific usage, when CROSS_COMPILE would have just done the job.. Then I was guessing maybe the rationale is for some ancient kernel where CROSS_COMPILE is'nt set right?
Quoted 11 lines
So we really have to pass this, when fundamentally it is the same definition as what the defaults give us.
What does that resolve to in your builds ? In mine, when I dump the objcopy value from within the kernel build, I get:
|
/opt/poky/build/tmp/work-shared/qemuarm64/kernel-source/Makefile:443:
*** objcopy: aarch64-poky-linux-objcopy. Stop.Which should be doing the job.
x86's objcopy - which is why I was trying to figure out what the heck went on..
Quoted 2 lines
Are you building arm on arm ? or something else like that ?
Nothing fancy.. x86 PC cross compiling for arm. honestly, 5.11 build went fine. What makes me wonder is how does it even build for you? how does OBJCOPY variable even point to aarch64-poky-linux-objcopy
Signature · 3 lines
Regards, Nishanth Menon Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3 1A34 DDB5 849D 1736 249D
Quoted 73 lines · Nishanth Menon
Quoted 7 lines · Bruce Ashfield
Quoted 5 lines · Nishanth Menon via lists.openembedded.org
When cross-compiling with v5.12-rc3, prepare fails[1] build of vdso at the objcopy stage since it seems to be using the local host's objcopy rather than the cross-compile version we want it to use.
Quoted 9 lines
Quoted 5 lines
This can be trivially reproduced in a localbuild of the kernel following the build parameters provided in the process[2]
Lets fix this by passing OBJCOPY over to the kernel.
As I mentioned to Denys, I hadn't seen this. Consulting the maintainers file would have found me as a good addition to the cc'.
Sure. understood. Thanks..
Quoted 4 lines
I'm doing some other work on make-mod-scripts dependencies right now, so I've pulled this in and will re-test against all of the active kernel versions in master.
Thanks.
Quoted 14 lines
But I don't think that make-mod-scripts is the only place we'd need this, if it is indeed happening on the tip of all the supported versions. There are routes to have the prepare steps run, without a make-mod-scripts dependency.
We've already made the mistake of letting the make-mod-scrips and kernel build parameters diverge (see AR=${KERNEL_AR}), so I need to spend a few minutes getting them back in sync.
I was just able to build and boot qemuarm64 on 5.12-rc4, so there's something different about your config than my setup. We need to figure that out. It could be a .config triggering different components to be built, but unlikely given vdso being involved.
Quoted 5 lines
qemuarm64 login: root
root@qemuarm64:~# uname -a Linux qemuarm64 5.12.0-rc4-yoctodev-standard #1 SMP PREEMPT Mon Mar 22 18:46:22 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux root@qemuarm64:~#Hmm... only thing I have done is cross compile - see the pastebins.
Quoted 4 lines
Quoted 4 lines
[1] https://pastebin.ubuntu.com/p/pNcQtb93wr/
[2] https://pastebin.ubuntu.com/p/vZGqgh9Sq5/Signed-off-by Nishanth Menon <[email protected]>[...]
Quoted 17 lines
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 65
Quoted 11 lines
meta/classes/kernel-arch.bbclass +2
-0@@ -60,9 +60,12 @@ TARGET_LD_KERNEL_ARCH ?= "" HOST_LD_KERNEL_ARCH ?= "${TARGET_LD_KERNEL_ARCH}" TARGET_AR_KERNEL_ARCH ?= "" HOST_AR_KERNEL_ARCH ?= "${TARGET_AR_KERNEL_ARCH}" +TARGET_OBJCOPY_KERNEL_ARCH ?= "" +HOST_OBJCOPY_KERNEL_ARCH ?= "${TARGET_OBJCOPY_KERNEL_ARCH}"We shouldn't need this
TARGET_OBJCOPY_KERNEL_ARCH->HOST_OBJCOPY_KERNEL_ARCH, I can't think of how objcopy would be using them.
Quoted 4 lines
Quoted 1 line
Are you setting them to some value in your builds ?
No, I am not. I decided to maintain consistency of usage from LD and AR. I could'nt think of a usage either, true.
That's what I figured. I was worried I was missing some sort of usecase. No harm in copying the existing pattern, I didn't introduce it either, so I wanted to make sure there wasn't something going on that I didn't see.
Quoted 12 lines
Quoted 12 lines
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 69
Quoted 7 lines
meta/classes/kernel-arch.bbclass +1
-0KERNEL_CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_KERNEL_ARCH} -fuse-ld=bfd ${DEBUG_PREFIX_MAP} -fdebug-prefix-map=${STAGING_KERNEL_DIR}=${KERNEL_SRC_PATH}" KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd ${HOST_LD_KERNEL_ARCH}" KERNEL_AR = "${CCACHE}${HOST_PREFIX}ar ${HOST_AR_KERNEL_ARCH}" +KERNEL_OBJCOPY = "${CCACHE}${HOST_PREFIX}objcopy ${HOST_OBJCOPY_KERNEL_ARCH}"The main kernel
Makefilealready defines, and the standard env should already have bothCROSS_COMPILEand OBJCOPY defined to be the target version.
Quoted 9 lines
Quoted 1 line
Makefile:OBJCOPY = $(CROSS_COMPILE)objcopy
OK this is what is confusing me -> I dont see
CROSS_COMPILEbeing set - which is what I had expected in the first place. other than meta/recipes-kernel/perf/perf.bb, I dont see it set else where -> I was really wondering why all the specific usage, whenCROSS_COMPILEwould have just done the job.. Then I was guessing maybe the rationale is for some ancient kernel whereCROSS_COMPILEis'nt set right?
It could be, we also wanted to tack on the CCACHE stuff, but generally speaking .. it is all kind of redundant with new kernels.
Quoted 7 lines
Quoted 7 lines
So we really have to pass this, when fundamentally it is the same definition as what the defaults give us.
What does that resolve to in your builds ? In mine, when I dump the objcopy value from within the kernel build, I get:
Quoted 7 lines
Quoted 4 lines
|
/opt/poky/build/tmp/work-shared/qemuarm64/kernel-source/Makefile:443:
*** objcopy: aarch64-poky-linux-objcopy. Stop.Which should be doing the job.
x86's objcopy - which is why I was trying to figure out what the heck went on..
That is indeed strange.
What does bitbake -e tell you ?
CROSS_COMPILE should be exported by kernel.bbclass itself, it definitely is in my environment dump.
Quoted 8 lines
Quoted 3 lines
Are you building arm on arm ? or something else like that ?
Nothing fancy.. x86 PC cross compiling for arm. honestly, 5.11 build went fine. What makes me wonder is how does it even build for you? how does OBJCOPY variable even point to aarch64-poky-linux-objcopy
Indeed. I've done a full set of tests on all the arches for 5.12-rc+ (as you can see by the lttng/perf/devsrc patches that I've been sending in the past week), so something odd is going on here.
Are the layers you are building public ? if so, I can try with your exact setup and see if I can reproduce the issue.
Bruce
Quoted 5 lines
--
Regards,
Nishanth Menon
Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3 1A34 DDB5 849D 1736 249D
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
On 11:14-20210329, Bruce Ashfield wrote:
[..]
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
So we really have to pass this, when fundamentally it is the same definition as what the defaults give us.
What does that resolve to in your builds ? In mine, when I dump the objcopy value from within the kernel build, I get:
|
/opt/poky/build/tmp/work-shared/qemuarm64/kernel-source/Makefile:443:
*** objcopy: aarch64-poky-linux-objcopy. Stop.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Which should be doing the job.
x86's objcopy - which is why I was trying to figure out what the heck went on..
That is indeed strange.
What does bitbake -e tell you ?
Not sure if it helps.. but..
https://pastebin.ubuntu.com/p/JJHtNkwSwY/
Quoted 3 lines
CROSS_COMPILEshould be exported by kernel.bbclass itself, it definitely is in my environment dump.
Hmm.. ok - there is something going on.. I see the export in bbclass..
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Are you building arm on arm ? or something else like that ?
Quoted 6 lines
Quoted 3 lines
Nothing fancy.. x86 PC cross compiling for arm. honestly, 5.11 build went fine. What makes me wonder is how does it even build for you? how does OBJCOPY variable even point to aarch64-poky-linux-objcopy
Indeed. I've done a full set of tests on all the arches for 5.12-rc+ (as you can see by the lttng/perf/devsrc patches that I've been sending
Awesome.
Quoted 1 line
in the past week), so something odd is going on here.
Yes, indeed. I think I will need to look closer at what is going on our end.. As you pointed out CROSS_COMPILE should really be available in the environment.. Rest of specific exports tend to be a never-ending patchup job..
Quoted 3 lines
Are the layers you are building public ? if so, I can try with your exact setup and see if I can reproduce the issue.
unfortunately, no. Denys and I are trying to figure our how to make our private layer public in a manner that others can work with as well.. but not yet.. we are still a few weeks or months away..
Let me try the reverse and try and reproduce your setup on my end and see where the deltas are.. That might be more practical, I think..
Thanks for your time and hints.. I need to do more homework here..
Signature · 3 lines
Regards, Nishanth Menon Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3 1A34 DDB5 849D 1736 249D
Quoted 124 lines · Bruce Ashfield
Quoted 73 lines · Nishanth Menon
Quoted 7 lines · Bruce Ashfield
Quoted 5 lines · Nishanth Menon via lists.openembedded.org
When cross-compiling with v5.12-rc3, prepare fails[1] build of vdso at the objcopy stage since it seems to be using the local host's objcopy rather than the cross-compile version we want it to use.
Quoted 9 lines
Quoted 5 lines
This can be trivially reproduced in a localbuild of the kernel following the build parameters provided in the process[2]
Lets fix this by passing OBJCOPY over to the kernel.
As I mentioned to Denys, I hadn't seen this. Consulting the maintainers file would have found me as a good addition to the cc'.
Sure. understood. Thanks..
Quoted 4 lines
I'm doing some other work on make-mod-scripts dependencies right now, so I've pulled this in and will re-test against all of the active kernel versions in master.
Thanks.
Quoted 14 lines
But I don't think that make-mod-scripts is the only place we'd need this, if it is indeed happening on the tip of all the supported versions. There are routes to have the prepare steps run, without a make-mod-scripts dependency.
We've already made the mistake of letting the make-mod-scrips and kernel build parameters diverge (see AR=${KERNEL_AR}), so I need to spend a few minutes getting them back in sync.
I was just able to build and boot qemuarm64 on 5.12-rc4, so there's something different about your config than my setup. We need to figure that out. It could be a .config triggering different components to be built, but unlikely given vdso being involved.
Quoted 5 lines
qemuarm64 login: root
root@qemuarm64:~# uname -a Linux qemuarm64 5.12.0-rc4-yoctodev-standard #1 SMP PREEMPT Mon Mar 22 18:46:22 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux root@qemuarm64:~#Hmm... only thing I have done is cross compile - see the pastebins.
Quoted 4 lines
Quoted 4 lines
[1] https://pastebin.ubuntu.com/p/pNcQtb93wr/
[2] https://pastebin.ubuntu.com/p/vZGqgh9Sq5/Signed-off-by Nishanth Menon <[email protected]>[...]
Quoted 17 lines
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 65
Quoted 11 lines
meta/classes/kernel-arch.bbclass +2
-0@@ -60,9 +60,12 @@ TARGET_LD_KERNEL_ARCH ?= "" HOST_LD_KERNEL_ARCH ?= "${TARGET_LD_KERNEL_ARCH}" TARGET_AR_KERNEL_ARCH ?= "" HOST_AR_KERNEL_ARCH ?= "${TARGET_AR_KERNEL_ARCH}" +TARGET_OBJCOPY_KERNEL_ARCH ?= "" +HOST_OBJCOPY_KERNEL_ARCH ?= "${TARGET_OBJCOPY_KERNEL_ARCH}"We shouldn't need this
TARGET_OBJCOPY_KERNEL_ARCH->HOST_OBJCOPY_KERNEL_ARCH, I can't think of how objcopy would be using them.Quoted 4 lines
Quoted 1 line
Are you setting them to some value in your builds ?
No, I am not. I decided to maintain consistency of usage from LD and AR. I could'nt think of a usage either, true.
That's what I figured. I was worried I was missing some sort of usecase. No harm in copying the existing pattern, I didn't introduce it either, so I wanted to make sure there wasn't something going on that I didn't see.
Quoted 12 lines
Quoted 12 lines
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 69
Quoted 7 lines
meta/classes/kernel-arch.bbclass +1
-0KERNEL_CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_KERNEL_ARCH} -fuse-ld=bfd ${DEBUG_PREFIX_MAP} -fdebug-prefix-map=${STAGING_KERNEL_DIR}=${KERNEL_SRC_PATH}" KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd ${HOST_LD_KERNEL_ARCH}" KERNEL_AR = "${CCACHE}${HOST_PREFIX}ar ${HOST_AR_KERNEL_ARCH}" +KERNEL_OBJCOPY = "${CCACHE}${HOST_PREFIX}objcopy ${HOST_OBJCOPY_KERNEL_ARCH}"The main kernel
Makefilealready defines, and the standard env should already have bothCROSS_COMPILEand OBJCOPY defined to be the target version.Quoted 9 lines
Quoted 1 line
Makefile:OBJCOPY = $(CROSS_COMPILE)objcopy
OK this is what is confusing me -> I dont see
CROSS_COMPILEbeing set - which is what I had expected in the first place. other than meta/recipes-kernel/perf/perf.bb, I dont see it set else where -> I was really wondering why all the specific usage, whenCROSS_COMPILEwould have just done the job.. Then I was guessing maybe the rationale is for some ancient kernel whereCROSS_COMPILEis'nt set right?It could be, we also wanted to tack on the CCACHE stuff, but generally speaking .. it is all kind of redundant with new kernels.
Quoted 7 lines
Quoted 7 lines
So we really have to pass this, when fundamentally it is the same definition as what the defaults give us.
What does that resolve to in your builds ? In mine, when I dump the objcopy value from within the kernel build, I get:
Quoted 7 lines
Quoted 4 lines
|
/opt/poky/build/tmp/work-shared/qemuarm64/kernel-source/Makefile:443:
*** objcopy: aarch64-poky-linux-objcopy. Stop.Which should be doing the job.
x86's objcopy - which is why I was trying to figure out what the heck went on..
Quoted 6 lines
That is indeed strange.
What does bitbake -e tell you ?
CROSS_COMPILEshould be exported by kernel.bbclass itself, it definitely is in my environment dump.
Bruce,
Looks like make-mod-scripts does not inherit kernel.bbclass, but only kernel-arch.bbclass and hence doesn't do "export CROSS_COMPILE" that is in kernel.bbclass
Since make-mod-scripts is not used for the kernel, only for out-of-tree modules, you have to either build it directly or e.g. cryptodev-module.
I got it easily reproducible with Poky master by adding these to local.conf:
MACHINE = "qemuarm64" PREFERRED_PROVIDER_virtual/kernel = "linux-yocto-dev"
$ bitbake make-mod-scripts
One easy way to fix it is to pass CROSS_COMPILE explicitly in make-mod-scripts. See the patch I just sent.
Quoted 15 lines
Quoted 6 lines
Quoted 1 line
Are you building arm on arm ? or something else like that ?
Nothing fancy.. x86 PC cross compiling for arm. honestly, 5.11 build went fine. What makes me wonder is how does it even build for you? how does OBJCOPY variable even point to aarch64-poky-linux-objcopy
Indeed. I've done a full set of tests on all the arches for 5.12-rc+ (as you can see by the lttng/perf/devsrc patches that I've been sending in the past week), so something odd is going on here.
Are the layers you are building public ? if so, I can try with your exact setup and see if I can reproduce the issue.
Bruce
Signature · 4 lines
Regards, Denys Dmytriyenko <[email protected]> PGP: 0x420902729A92C964 - https://denix.org/0x420902729A92C964 Fingerprint: 25FC E4A5 8A72 2F69 1186 6D76 4209 0272 9A92 C964
Quoted 145 lines · Denys Dmytriyenko
Quoted 124 lines · Bruce Ashfield
Quoted 73 lines · Nishanth Menon
Quoted 7 lines · Bruce Ashfield
Quoted 5 lines · Nishanth Menon via lists.openembedded.org
When cross-compiling with v5.12-rc3, prepare fails[1] build of vdso at the objcopy stage since it seems to be using the local host's objcopy rather than the cross-compile version we want it to use.
Quoted 9 lines
Quoted 5 lines
This can be trivially reproduced in a localbuild of the kernel following the build parameters provided in the process[2]
Lets fix this by passing OBJCOPY over to the kernel.
As I mentioned to Denys, I hadn't seen this. Consulting the maintainers file would have found me as a good addition to the cc'.
Sure. understood. Thanks..
Quoted 4 lines
I'm doing some other work on make-mod-scripts dependencies right now, so I've pulled this in and will re-test against all of the active kernel versions in master.
Thanks.
Quoted 14 lines
But I don't think that make-mod-scripts is the only place we'd need this, if it is indeed happening on the tip of all the supported versions. There are routes to have the prepare steps run, without a make-mod-scripts dependency.
We've already made the mistake of letting the make-mod-scrips and kernel build parameters diverge (see AR=${KERNEL_AR}), so I need to spend a few minutes getting them back in sync.
I was just able to build and boot qemuarm64 on 5.12-rc4, so there's something different about your config than my setup. We need to figure that out. It could be a .config triggering different components to be built, but unlikely given vdso being involved.
Quoted 5 lines
qemuarm64 login: root
root@qemuarm64:~# uname -a Linux qemuarm64 5.12.0-rc4-yoctodev-standard #1 SMP PREEMPT Mon Mar 22 18:46:22 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux root@qemuarm64:~#Hmm... only thing I have done is cross compile - see the pastebins.
Quoted 4 lines
Quoted 4 lines
[1] https://pastebin.ubuntu.com/p/pNcQtb93wr/
[2] https://pastebin.ubuntu.com/p/vZGqgh9Sq5/Signed-off-by Nishanth Menon <[email protected]>[...]
Quoted 17 lines
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 65
Quoted 11 lines
meta/classes/kernel-arch.bbclass +2
-0@@ -60,9 +60,12 @@ TARGET_LD_KERNEL_ARCH ?= "" HOST_LD_KERNEL_ARCH ?= "${TARGET_LD_KERNEL_ARCH}" TARGET_AR_KERNEL_ARCH ?= "" HOST_AR_KERNEL_ARCH ?= "${TARGET_AR_KERNEL_ARCH}" +TARGET_OBJCOPY_KERNEL_ARCH ?= "" +HOST_OBJCOPY_KERNEL_ARCH ?= "${TARGET_OBJCOPY_KERNEL_ARCH}"We shouldn't need this
TARGET_OBJCOPY_KERNEL_ARCH->HOST_OBJCOPY_KERNEL_ARCH, I can't think of how objcopy would be using them.Quoted 4 lines
Quoted 1 line
Are you setting them to some value in your builds ?
No, I am not. I decided to maintain consistency of usage from LD and AR. I could'nt think of a usage either, true.
That's what I figured. I was worried I was missing some sort of usecase. No harm in copying the existing pattern, I didn't introduce it either, so I wanted to make sure there wasn't something going on that I didn't see.
Quoted 12 lines
Quoted 12 lines
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 69
Quoted 7 lines
meta/classes/kernel-arch.bbclass +1
-0KERNEL_CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_KERNEL_ARCH} -fuse-ld=bfd ${DEBUG_PREFIX_MAP} -fdebug-prefix-map=${STAGING_KERNEL_DIR}=${KERNEL_SRC_PATH}" KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd ${HOST_LD_KERNEL_ARCH}" KERNEL_AR = "${CCACHE}${HOST_PREFIX}ar ${HOST_AR_KERNEL_ARCH}" +KERNEL_OBJCOPY = "${CCACHE}${HOST_PREFIX}objcopy ${HOST_OBJCOPY_KERNEL_ARCH}"The main kernel
Makefilealready defines, and the standard env should already have bothCROSS_COMPILEand OBJCOPY defined to be the target version.Quoted 9 lines
Quoted 1 line
Makefile:OBJCOPY = $(CROSS_COMPILE)objcopy
OK this is what is confusing me -> I dont see
CROSS_COMPILEbeing set - which is what I had expected in the first place. other than meta/recipes-kernel/perf/perf.bb, I dont see it set else where -> I was really wondering why all the specific usage, whenCROSS_COMPILEwould have just done the job.. Then I was guessing maybe the rationale is for some ancient kernel whereCROSS_COMPILEis'nt set right?It could be, we also wanted to tack on the CCACHE stuff, but generally speaking .. it is all kind of redundant with new kernels.
Quoted 7 lines
Quoted 7 lines
So we really have to pass this, when fundamentally it is the same definition as what the defaults give us.
What does that resolve to in your builds ? In mine, when I dump the objcopy value from within the kernel build, I get:
Quoted 7 lines
Quoted 4 lines
|
/opt/poky/build/tmp/work-shared/qemuarm64/kernel-source/Makefile:443:
*** objcopy: aarch64-poky-linux-objcopy. Stop.Which should be doing the job.
x86's objcopy - which is why I was trying to figure out what the heck went on..
Quoted 6 lines
That is indeed strange.
What does bitbake -e tell you ?
CROSS_COMPILEshould be exported by kernel.bbclass itself, it definitely is in my environment dump.Bruce,
Looks like make-mod-scripts does not inherit kernel.bbclass, but only kernel-arch.bbclass and hence doesn't do "export
CROSS_COMPILE" that is in kernel.bbclassSince make-mod-scripts is not used for the kernel, only for out-of-tree modules, you have to either build it directly or e.g. cryptodev-module.
I got it easily reproducible with Poky master by adding these to local.conf:
Quoted 7 lines
MACHINE = "qemuarm64" PREFERRED_PROVIDER_virtual/kernel = "linux-yocto-dev"
$ bitbake make-mod-scriptsOne easy way to fix it is to pass
CROSS_COMPILEexplicitly in make-mod-scripts. See the patch I just sent.
I agree that make-mod-scripts doesn't have CROSS_COMPILE set (I just dumped the env to confirm).
It is still odd that I can build for that very same machine without any issues though. There's still something lurking that is filling in the required dependencies and making the build not necessary.
I've built and booted qemuarm64 on the tip of poky master with 5.12+ hundreds of times already, and have never seen that.
BUT .. I switched to my second builder, used my exact same configuration and it did in fact show the issue. I have some local patches on my primary builder, so one of them may have been bolting everything together (not relevant here, but I'll go back and look at them later).
I like this simpler fix, and will follow up directly on the patch you sent.
Bruce
Quoted 16 lines
Quoted 16 lines
Quoted 8 lines
Quoted 3 lines
Are you building arm on arm ? or something else like that ?
Nothing fancy.. x86 PC cross compiling for arm. honestly, 5.11 build went fine. What makes me wonder is how does it even build for you? how does OBJCOPY variable even point to aarch64-poky-linux-objcopy
Indeed. I've done a full set of tests on all the arches for 5.12-rc+ (as you can see by the lttng/perf/devsrc patches that I've been sending in the past week), so something odd is going on here.
Are the layers you are building public ? if so, I can try with your exact setup and see if I can reproduce the issue.
Quoted 7 lines
Quoted 1 line
Bruce
--
Regards,
Denys Dmytriyenko <[email protected]>
PGP:0x420902729A92C964- https://denix.org/0x420902729A92C964
Fingerprint: 25FC E4A5 8A72 2F69 1186 6D76 4209 0272 9A92 C964
--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
Quoted 154 lines · Bruce Ashfield
Quoted 145 lines · Denys Dmytriyenko
Quoted 124 lines · Bruce Ashfield
Quoted 73 lines · Nishanth Menon
Quoted 7 lines · Bruce Ashfield
Quoted 5 lines · Nishanth Menon via lists.openembedded.org
When cross-compiling with v5.12-rc3, prepare fails[1] build of vdso at the objcopy stage since it seems to be using the local host's objcopy rather than the cross-compile version we want it to use.
Quoted 9 lines
Quoted 5 lines
This can be trivially reproduced in a localbuild of the kernel following the build parameters provided in the process[2]
Lets fix this by passing OBJCOPY over to the kernel.
As I mentioned to Denys, I hadn't seen this. Consulting the maintainers file would have found me as a good addition to the cc'.
Sure. understood. Thanks..
Quoted 4 lines
I'm doing some other work on make-mod-scripts dependencies right now, so I've pulled this in and will re-test against all of the active kernel versions in master.
Thanks.
Quoted 14 lines
But I don't think that make-mod-scripts is the only place we'd need this, if it is indeed happening on the tip of all the supported versions. There are routes to have the prepare steps run, without a make-mod-scripts dependency.
We've already made the mistake of letting the make-mod-scrips and kernel build parameters diverge (see AR=${KERNEL_AR}), so I need to spend a few minutes getting them back in sync.
I was just able to build and boot qemuarm64 on 5.12-rc4, so there's something different about your config than my setup. We need to figure that out. It could be a .config triggering different components to be built, but unlikely given vdso being involved.
Quoted 5 lines
qemuarm64 login: root
root@qemuarm64:~# uname -a Linux qemuarm64 5.12.0-rc4-yoctodev-standard #1 SMP PREEMPT Mon Mar 22 18:46:22 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux root@qemuarm64:~#Hmm... only thing I have done is cross compile - see the pastebins.
Quoted 4 lines
Quoted 4 lines
[1] https://pastebin.ubuntu.com/p/pNcQtb93wr/
[2] https://pastebin.ubuntu.com/p/vZGqgh9Sq5/Signed-off-by Nishanth Menon <[email protected]>[...]
Quoted 17 lines
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 65
Quoted 11 lines
meta/classes/kernel-arch.bbclass +2
-0@@ -60,9 +60,12 @@ TARGET_LD_KERNEL_ARCH ?= "" HOST_LD_KERNEL_ARCH ?= "${TARGET_LD_KERNEL_ARCH}" TARGET_AR_KERNEL_ARCH ?= "" HOST_AR_KERNEL_ARCH ?= "${TARGET_AR_KERNEL_ARCH}" +TARGET_OBJCOPY_KERNEL_ARCH ?= "" +HOST_OBJCOPY_KERNEL_ARCH ?= "${TARGET_OBJCOPY_KERNEL_ARCH}"We shouldn't need this
TARGET_OBJCOPY_KERNEL_ARCH->HOST_OBJCOPY_KERNEL_ARCH, I can't think of how objcopy would be using them.Quoted 4 lines
Quoted 1 line
Are you setting them to some value in your builds ?
No, I am not. I decided to maintain consistency of usage from LD and AR. I could'nt think of a usage either, true.
That's what I figured. I was worried I was missing some sort of usecase. No harm in copying the existing pattern, I didn't introduce it either, so I wanted to make sure there wasn't something going on that I didn't see.
Quoted 12 lines
Quoted 12 lines
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 69
Quoted 7 lines
meta/classes/kernel-arch.bbclass +1
-0KERNEL_CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_KERNEL_ARCH} -fuse-ld=bfd ${DEBUG_PREFIX_MAP} -fdebug-prefix-map=${STAGING_KERNEL_DIR}=${KERNEL_SRC_PATH}" KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd ${HOST_LD_KERNEL_ARCH}" KERNEL_AR = "${CCACHE}${HOST_PREFIX}ar ${HOST_AR_KERNEL_ARCH}" +KERNEL_OBJCOPY = "${CCACHE}${HOST_PREFIX}objcopy ${HOST_OBJCOPY_KERNEL_ARCH}"The main kernel
Makefilealready defines, and the standard env should already have bothCROSS_COMPILEand OBJCOPY defined to be the target version.Quoted 9 lines
Quoted 1 line
Makefile:OBJCOPY = $(CROSS_COMPILE)objcopy
OK this is what is confusing me -> I dont see
CROSS_COMPILEbeing set - which is what I had expected in the first place. other than meta/recipes-kernel/perf/perf.bb, I dont see it set else where -> I was really wondering why all the specific usage, whenCROSS_COMPILEwould have just done the job.. Then I was guessing maybe the rationale is for some ancient kernel whereCROSS_COMPILEis'nt set right?It could be, we also wanted to tack on the CCACHE stuff, but generally speaking .. it is all kind of redundant with new kernels.
Quoted 7 lines
Quoted 7 lines
So we really have to pass this, when fundamentally it is the same definition as what the defaults give us.
What does that resolve to in your builds ? In mine, when I dump the objcopy value from within the kernel build, I get:
Quoted 7 lines
Quoted 4 lines
|
/opt/poky/build/tmp/work-shared/qemuarm64/kernel-source/Makefile:443:
*** objcopy: aarch64-poky-linux-objcopy. Stop.Which should be doing the job.
x86's objcopy - which is why I was trying to figure out what the heck went on..
Quoted 6 lines
That is indeed strange.
What does bitbake -e tell you ?
CROSS_COMPILEshould be exported by kernel.bbclass itself, it definitely is in my environment dump.Bruce,
Looks like make-mod-scripts does not inherit kernel.bbclass, but only kernel-arch.bbclass and hence doesn't do "export
CROSS_COMPILE" that is in kernel.bbclassSince make-mod-scripts is not used for the kernel, only for out-of-tree modules, you have to either build it directly or e.g. cryptodev-module.
I got it easily reproducible with Poky master by adding these to local.conf:
Quoted 7 lines
MACHINE = "qemuarm64" PREFERRED_PROVIDER_virtual/kernel = "linux-yocto-dev"
$ bitbake make-mod-scriptsOne easy way to fix it is to pass
CROSS_COMPILEexplicitly in make-mod-scripts. See the patch I just sent.
Quoted 6 lines
I agree that make-mod-scripts doesn't have
CROSS_COMPILEset (I just dumped the env to confirm).It is still odd that I can build for that very same machine without any issues though. There's still something lurking that is filling in the required dependencies and making the build not necessary.
it will use objcopy from host at this point instead of what you built with binutils, perhaps a host issue creeps in who knows. cross objcopy is not strictly required but its good to use a captured tool.
Quoted 48 lines
I've built and booted qemuarm64 on the tip of poky master with 5.12+ hundreds of times already, and have never seen that.
BUT .. I switched to my second builder, used my exact same configuration and it did in fact show the issue. I have some local patches on my primary builder, so one of them may have been bolting everything together (not relevant here, but I'll go back and look at them later).
I like this simpler fix, and will follow up directly on the patch you sent.
Bruce
Quoted 16 lines
Quoted 16 lines
Quoted 8 lines
Quoted 3 lines
Are you building arm on arm ? or something else like that ?
Nothing fancy.. x86 PC cross compiling for arm. honestly, 5.11 build went fine. What makes me wonder is how does it even build for you? how does OBJCOPY variable even point to aarch64-poky-linux-objcopy
Indeed. I've done a full set of tests on all the arches for 5.12-rc+ (as you can see by the lttng/perf/devsrc patches that I've been sending in the past week), so something odd is going on here.
Are the layers you are building public ? if so, I can try with your exact setup and see if I can reproduce the issue.
Quoted 7 lines
Quoted 1 line
Bruce
--
Regards,
Denys Dmytriyenko <[email protected]>
PGP:0x420902729A92C964- https://denix.org/0x420902729A92C964
Fingerprint: 25FC E4A5 8A72 2F69 1186 6D76 4209 0272 9A92 C964--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
Quoted 161 lines · Khem Raj
Quoted 157 lines · Bruce Ashfield
Quoted 145 lines · Denys Dmytriyenko
Quoted 124 lines · Bruce Ashfield
Quoted 73 lines · Nishanth Menon
Quoted 7 lines · Bruce Ashfield
Quoted 5 lines · Nishanth Menon via lists.openembedded.org
When cross-compiling with v5.12-rc3, prepare fails[1] build of vdso at the objcopy stage since it seems to be using the local host's objcopy rather than the cross-compile version we want it to use.
Quoted 9 lines
Quoted 5 lines
This can be trivially reproduced in a localbuild of the kernel following the build parameters provided in the process[2]
Lets fix this by passing OBJCOPY over to the kernel.
As I mentioned to Denys, I hadn't seen this. Consulting the maintainers file would have found me as a good addition to the cc'.
Sure. understood. Thanks..
Quoted 4 lines
I'm doing some other work on make-mod-scripts dependencies right now, so I've pulled this in and will re-test against all of the active kernel versions in master.
Thanks.
Quoted 14 lines
But I don't think that make-mod-scripts is the only place we'd need this, if it is indeed happening on the tip of all the supported versions. There are routes to have the prepare steps run, without a make-mod-scripts dependency.
We've already made the mistake of letting the make-mod-scrips and kernel build parameters diverge (see AR=${KERNEL_AR}), so I need to spend a few minutes getting them back in sync.
I was just able to build and boot qemuarm64 on 5.12-rc4, so there's something different about your config than my setup. We need to figure that out. It could be a .config triggering different components to be built, but unlikely given vdso being involved.
Quoted 5 lines
qemuarm64 login: root
root@qemuarm64:~# uname -a Linux qemuarm64 5.12.0-rc4-yoctodev-standard #1 SMP PREEMPT Mon Mar 22 18:46:22 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux root@qemuarm64:~#Hmm... only thing I have done is cross compile - see the pastebins.
Quoted 4 lines
Quoted 4 lines
[1] https://pastebin.ubuntu.com/p/pNcQtb93wr/
[2] https://pastebin.ubuntu.com/p/vZGqgh9Sq5/Signed-off-by Nishanth Menon <[email protected]>[...]
Quoted 17 lines
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 65
Quoted 11 lines
meta/classes/kernel-arch.bbclass +2
-0@@ -60,9 +60,12 @@ TARGET_LD_KERNEL_ARCH ?= "" HOST_LD_KERNEL_ARCH ?= "${TARGET_LD_KERNEL_ARCH}" TARGET_AR_KERNEL_ARCH ?= "" HOST_AR_KERNEL_ARCH ?= "${TARGET_AR_KERNEL_ARCH}" +TARGET_OBJCOPY_KERNEL_ARCH ?= "" +HOST_OBJCOPY_KERNEL_ARCH ?= "${TARGET_OBJCOPY_KERNEL_ARCH}"We shouldn't need this
TARGET_OBJCOPY_KERNEL_ARCH->HOST_OBJCOPY_KERNEL_ARCH, I can't think of how objcopy would be using them.Quoted 4 lines
Quoted 1 line
Are you setting them to some value in your builds ?
No, I am not. I decided to maintain consistency of usage from LD and AR. I could'nt think of a usage either, true.
That's what I figured. I was worried I was missing some sort of usecase. No harm in copying the existing pattern, I didn't introduce it either, so I wanted to make sure there wasn't something going on that I didn't see.
Quoted 12 lines
Quoted 12 lines
meta/classes/kernel-arch.bbclass TARGET_LD_KERNEL_ARCH ?= "" · line 69
Quoted 7 lines
meta/classes/kernel-arch.bbclass +1
-0KERNEL_CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_KERNEL_ARCH} -fuse-ld=bfd ${DEBUG_PREFIX_MAP} -fdebug-prefix-map=${STAGING_KERNEL_DIR}=${KERNEL_SRC_PATH}" KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd ${HOST_LD_KERNEL_ARCH}" KERNEL_AR = "${CCACHE}${HOST_PREFIX}ar ${HOST_AR_KERNEL_ARCH}" +KERNEL_OBJCOPY = "${CCACHE}${HOST_PREFIX}objcopy ${HOST_OBJCOPY_KERNEL_ARCH}"The main kernel
Makefilealready defines, and the standard env should already have bothCROSS_COMPILEand OBJCOPY defined to be the target version.Quoted 9 lines
Quoted 1 line
Makefile:OBJCOPY = $(CROSS_COMPILE)objcopy
OK this is what is confusing me -> I dont see
CROSS_COMPILEbeing set - which is what I had expected in the first place. other than meta/recipes-kernel/perf/perf.bb, I dont see it set else where -> I was really wondering why all the specific usage, whenCROSS_COMPILEwould have just done the job.. Then I was guessing maybe the rationale is for some ancient kernel whereCROSS_COMPILEis'nt set right?It could be, we also wanted to tack on the CCACHE stuff, but generally speaking .. it is all kind of redundant with new kernels.
Quoted 7 lines
Quoted 7 lines
So we really have to pass this, when fundamentally it is the same definition as what the defaults give us.
What does that resolve to in your builds ? In mine, when I dump the objcopy value from within the kernel build, I get:
Quoted 7 lines
Quoted 4 lines
|
/opt/poky/build/tmp/work-shared/qemuarm64/kernel-source/Makefile:443:
*** objcopy: aarch64-poky-linux-objcopy. Stop.Which should be doing the job.
x86's objcopy - which is why I was trying to figure out what the heck went on..
Quoted 6 lines
That is indeed strange.
What does bitbake -e tell you ?
CROSS_COMPILEshould be exported by kernel.bbclass itself, it definitely is in my environment dump.Bruce,
Looks like make-mod-scripts does not inherit kernel.bbclass, but only kernel-arch.bbclass and hence doesn't do "export
CROSS_COMPILE" that is in kernel.bbclassSince make-mod-scripts is not used for the kernel, only for out-of-tree modules, you have to either build it directly or e.g. cryptodev-module.
I got it easily reproducible with Poky master by adding these to local.conf:
Quoted 7 lines
MACHINE = "qemuarm64" PREFERRED_PROVIDER_virtual/kernel = "linux-yocto-dev"
$ bitbake make-mod-scriptsOne easy way to fix it is to pass
CROSS_COMPILEexplicitly in make-mod-scripts. See the patch I just sent.I agree that make-mod-scripts doesn't have
CROSS_COMPILEset (I just dumped the env to confirm).
Quoted 7 lines
Quoted 3 lines
It is still odd that I can build for that very same machine without any issues though. There's still something lurking that is filling in the required dependencies and making the build not necessary.
it will use objcopy from host at this point instead of what you built with binutils, perhaps a host issue creeps in who knows. cross objcopy is not strictly required but its good to use a captured tool.
No it won't.
The host tool is a completely different arch and blows up. I'm saying that with some of the local (to be submitted in the future) patches I have, the cross objtool was picked up in that same dependency chain.
Bruce
Quoted 49 lines
Quoted 49 lines
I've built and booted qemuarm64 on the tip of poky master with 5.12+ hundreds of times already, and have never seen that.
BUT .. I switched to my second builder, used my exact same configuration and it did in fact show the issue. I have some local patches on my primary builder, so one of them may have been bolting everything together (not relevant here, but I'll go back and look at them later).
I like this simpler fix, and will follow up directly on the patch you sent.
Bruce
Quoted 16 lines
Quoted 16 lines
Quoted 8 lines
Quoted 3 lines
Are you building arm on arm ? or something else like that ?
Nothing fancy.. x86 PC cross compiling for arm. honestly, 5.11 build went fine. What makes me wonder is how does it even build for you? how does OBJCOPY variable even point to aarch64-poky-linux-objcopy
Indeed. I've done a full set of tests on all the arches for 5.12-rc+ (as you can see by the lttng/perf/devsrc patches that I've been sending in the past week), so something odd is going on here.
Are the layers you are building public ? if so, I can try with your exact setup and see if I can reproduce the issue.
Quoted 7 lines
Quoted 1 line
Bruce
--
Regards,
Denys Dmytriyenko <[email protected]>
PGP:0x420902729A92C964- https://denix.org/0x420902729A92C964
Fingerprint: 25FC E4A5 8A72 2F69 1186 6D76 4209 0272 9A92 C964--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II