Skip to contentSkip to conversation

make-mod-scripts: Fix a rare build race condition

PATCH Superseded Khem Raj 3 participants 7 messages May 27, 2020

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Khem Raj

author
#0 lore raw

PATCH 1/1 make-mod-scripts: Fix a rare build race condition

There is a build break which often happens whem using 4.1 kernel

/bin/sh: 1: scripts/basic/fixdep: Permission denied scripts/Makefile.host:124: recipe for target 'scripts/dtc/srcpos.o' failed make[3]: *** [scripts/dtc/srcpos.o] Error 126

this patch sequences the build targets so it can work reliably with different kernel versions

Cc Bruce Ashfield <[email protected]>
---
 .../make-mod-scripts/make-mod-scripts_1.0.bb               | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +4 -3
index 0be1422a24..c7edb20ee4 100644
@@ -23,7 +23,8 @@ EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="
 #
 do_configure() {
 	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
-	oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
-	           -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} scripts prepare
-
+	for t in prepare scripts_basic scripts; do
+		oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
+		-C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t
+	done
 }
Signature · 1 line
2.26.2
#1 lore raw
Quoted 8 lines · Khem Raj

There is a build break which often happens whem using 4.1 kernel

/bin/sh: 1: scripts/basic/fixdep: Permission denied scripts/Makefile.host:124: recipe for target 'scripts/dtc/srcpos.o' failed make[3]: *** [scripts/dtc/srcpos.o] Error 126

this patch sequences the build targets so it can work reliably with different kernel versions

Oh, I remember this failure from few years ago. Was it fixed upstream? Any specifics?

Quoted 27 lines
Cc Bruce Ashfield <[email protected]>
---
 .../make-mod-scripts/make-mod-scripts_1.0.bb               | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +4 -3
index 0be1422a24..c7edb20ee4 100644
@@ -23,7 +23,8 @@ EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="
 #
 do_configure() {
 	unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
-	oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
-	           -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} scripts prepare
-
+	for t in prepare scripts_basic scripts; do
+		oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
+		-C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t
+	done
 }
Signature · 1 line
2.26.2
#2 lore raw
In reply to Khem Raj
Quoted 21 lines · Khem Raj

There is a build break which often happens whem using 4.1 kernel

/bin/sh: 1: scripts/basic/fixdep: Permission denied scripts/Makefile.host:124: recipe for target 'scripts/dtc/srcpos.o' failed make[3]: *** [scripts/dtc/srcpos.o] Error 126

this patch sequences the build targets so it can work reliably with different kernel versions

Cc Bruce Ashfield <[email protected]>
---
 .../make-mod-scripts/make-mod-scripts_1.0.bb               | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +0 -0
index 0be1422a24..c7edb20ee4 100644

meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP=" · line 26

Quoted 7 lines
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +1 -3
@@ -23,7 +23,8 @@ EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="
 #
 do_configure() {
        unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
-       oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
-                  -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} scripts prepare
-
+       for t in prepare scripts_basic scripts; do

Last I checked, we don't actually need scripts_basic.

Isn't just sequencing prepare, followed by 'scripts' enough ?

If not, we really should put in the long log why we need that new target when sequencing, when we didn't need it before.

Bruce

Quoted 7 lines
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +3 -1
+               oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
+               -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t
+       done
 }
--

2.26.2

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
author
#3 lore raw
Quoted 33 lines · Bruce Ashfield
Quoted 21 lines · Khem Raj

There is a build break which often happens whem using 4.1 kernel

/bin/sh: 1: scripts/basic/fixdep: Permission denied scripts/Makefile.host:124: recipe for target 'scripts/dtc/srcpos.o' failed make[3]: *** [scripts/dtc/srcpos.o] Error 126

this patch sequences the build targets so it can work reliably with different kernel versions

Cc Bruce Ashfield <[email protected]>
---
 .../make-mod-scripts/make-mod-scripts_1.0.bb               | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +0 -0
index 0be1422a24..c7edb20ee4 100644

meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP=" · line 26

Quoted 7 lines
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +1 -3
@@ -23,7 +23,8 @@ EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="
 #
 do_configure() {
        unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
-       oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
-                  -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} scripts prepare
-
+       for t in prepare scripts_basic scripts; do

Last I checked, we don't actually need scripts_basic.

Quoted 5 lines

Isn't just sequencing prepare, followed by 'scripts' enough ?

If not, we really should put in the long log why we need that new target when sequencing, when we didn't need it before.

in a way you are right that scripts should cover it, but calling it out explicitly avoids any weird races if any and it does not hurt until this target is removed. I really have not debugged kernel build to see what is causing this, so there is not much details to add other than this that we are handing some race conditions which ideally should be in the makery

Quoted 15 lines

Bruce

Quoted 7 lines
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +3 -1
+               oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
+               -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t
+       done
 }
--

2.26.2

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

#4 lore raw
Quoted 41 lines · Khem Raj
Quoted 33 lines · Bruce Ashfield
Quoted 21 lines · Khem Raj

There is a build break which often happens whem using 4.1 kernel

/bin/sh: 1: scripts/basic/fixdep: Permission denied scripts/Makefile.host:124: recipe for target 'scripts/dtc/srcpos.o' failed make[3]: *** [scripts/dtc/srcpos.o] Error 126

this patch sequences the build targets so it can work reliably with different kernel versions

Cc Bruce Ashfield <[email protected]>
---
 .../make-mod-scripts/make-mod-scripts_1.0.bb               | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +0 -0
index 0be1422a24..c7edb20ee4 100644

meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP=" · line 26

Quoted 7 lines
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +1 -3
@@ -23,7 +23,8 @@ EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="
 #
 do_configure() {
        unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
-       oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
-                  -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} scripts prepare
-
+       for t in prepare scripts_basic scripts; do

Last I checked, we don't actually need scripts_basic.

Quoted 5 lines

Isn't just sequencing prepare, followed by 'scripts' enough ?

If not, we really should put in the long log why we need that new target when sequencing, when we didn't need it before.

Quoted 7 lines

in a way you are right that scripts should cover it, but calling it out explicitly avoids any weird races if any and it does not hurt until this target is removed. I really have not debugged kernel build to see what is causing this, so there is not much details to add other than this that we are handing some race conditions which ideally should be in the makery

But the race condition is handled with the two targets, no ? I realize it doesn't hurt, but you are implying that you need all three to be sure, but that really isn't the case from what I've seen.

I just think simpler is better, but don't take my comments as an objection.

Cheers,

Bruce

Quoted 15 lines
Quoted 15 lines

Bruce

Quoted 7 lines
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +3 -1
+               oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
+               -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t
+       done
 }
--

2.26.2

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
author
#5 lore raw
Quoted 51 lines · Bruce Ashfield
Quoted 41 lines · Khem Raj
Quoted 33 lines · Bruce Ashfield
Quoted 21 lines · Khem Raj

There is a build break which often happens whem using 4.1 kernel

/bin/sh: 1: scripts/basic/fixdep: Permission denied scripts/Makefile.host:124: recipe for target 'scripts/dtc/srcpos.o' failed make[3]: *** [scripts/dtc/srcpos.o] Error 126

this patch sequences the build targets so it can work reliably with different kernel versions

Cc Bruce Ashfield <[email protected]>
---
 .../make-mod-scripts/make-mod-scripts_1.0.bb               | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +0 -0
index 0be1422a24..c7edb20ee4 100644

meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP=" · line 26

Quoted 7 lines
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +1 -3
@@ -23,7 +23,8 @@ EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="
 #
 do_configure() {
        unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
-       oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
-                  -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} scripts prepare
-
+       for t in prepare scripts_basic scripts; do

Last I checked, we don't actually need scripts_basic.

Quoted 5 lines

Isn't just sequencing prepare, followed by 'scripts' enough ?

If not, we really should put in the long log why we need that new target when sequencing, when we didn't need it before.

Quoted 7 lines

in a way you are right that scripts should cover it, but calling it out explicitly avoids any weird races if any and it does not hurt until this target is removed. I really have not debugged kernel build to see what is causing this, so there is not much details to add other than this that we are handing some race conditions which ideally should be in the makery

Quoted 5 lines

But the race condition is handled with the two targets, no ? I realize it doesn't hurt, but you are implying that you need all three to be sure, but that really isn't the case from what I've seen.

I just think simpler is better, but don't take my comments as an objection.

Since it is a random failure that means validating any changes to fix it is as hard as reproducing it.
This solution has worked over a number of builds, testing anything else would take as much long and I am reluctant to do that without a potential gain.

Quoted 27 lines

Cheers,

Bruce

Quoted 11 lines
Quoted 11 lines

Bruce

Quoted 7 lines
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +3 -1
+               oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
+               -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t
+       done
 }
--

2.26.2

Quoted 4 lines
Quoted 4 lines

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

#6 lore raw
Quoted 58 lines · Khem Raj
Quoted 55 lines · Bruce Ashfield
Quoted 41 lines · Khem Raj
Quoted 33 lines · Bruce Ashfield
Quoted 21 lines · Khem Raj

There is a build break which often happens whem using 4.1 kernel

/bin/sh: 1: scripts/basic/fixdep: Permission denied scripts/Makefile.host:124: recipe for target 'scripts/dtc/srcpos.o' failed make[3]: *** [scripts/dtc/srcpos.o] Error 126

this patch sequences the build targets so it can work reliably with different kernel versions

Cc Bruce Ashfield <[email protected]>
---
 .../make-mod-scripts/make-mod-scripts_1.0.bb               | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +0 -0
index 0be1422a24..c7edb20ee4 100644

meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP=" · line 26

Quoted 7 lines
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +1 -3
@@ -23,7 +23,8 @@ EXTRA_OEMAKE = " HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}" HOSTCPP="
 #
 do_configure() {
        unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
-       oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
-                  -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} scripts prepare
-
+       for t in prepare scripts_basic scripts; do

Last I checked, we don't actually need scripts_basic.

Quoted 5 lines

Isn't just sequencing prepare, followed by 'scripts' enough ?

If not, we really should put in the long log why we need that new target when sequencing, when we didn't need it before.

Quoted 7 lines

in a way you are right that scripts should cover it, but calling it out explicitly avoids any weird races if any and it does not hurt until this target is removed. I really have not debugged kernel build to see what is causing this, so there is not much details to add other than this that we are handing some race conditions which ideally should be in the makery

But the race condition is handled with the two targets, no ? I realize it doesn't hurt, but you are implying that you need all three to be sure, but that really isn't the case from what I've seen.

Quoted 7 lines
Quoted 1 line

I just think simpler is better, but don't take my comments as an objection.

Since it is a random failure that means validating any changes to fix it is as hard as reproducing it.
This solution has worked over a number of builds, testing anything else would take as much long and I am reluctant to do that without a potential gain.

Minimizing the amount of change is definitely consider a gain by many (me included).

But I'm happy enough if you can just amend the message to indicate that it isn't strictly necessary and was simply what was used for testing.

Since I guarantee, someday, I'll be staring at that change wondering why it was added :D

Bruce

Quoted 28 lines
Quoted 28 lines

Cheers,

Bruce

Quoted 11 lines
Quoted 11 lines

Bruce

Quoted 7 lines
meta/recipes-kernel/make-mod-scripts/make-mod-scripts_1.0.bb +3 -1
+               oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" AR="${KERNEL_AR}" \
+               -C ${STAGING_KERNEL_DIR} O=${STAGING_KERNEL_BUILDDIR} $t
+       done
 }
--

2.26.2

Quoted 4 lines
Quoted 4 lines

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II