Skip to contentSkip to conversation

package/pkg-generic: additional dependencies for rebuild/reconfigure

PATCH In review Charles Hardin 2 participants 5 messages Jan 9, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 package/pkg-generic: additional dependencies for rebuild/reconfigure

While testing incremental changes for a kernel build using the LINUX_OVERRIDE_SRCDIR, the following behavior was observed.

- edit a file in the kernel tree and save
- make linux-rebuild for buildroot to pickup the changes
- rsync occurrs and then the install target is run (no build)
- run make linux-rebuild a second time and the build is compiled and run

To get the changes in on the first invocation of the rebuild requires a dependency on the build target to re-evaluate the stamp dependencies after the remove. This is a function of make dependency order evaluation so that the install steps will see the missing the install stamp but that evaluation doesn't require the rebuild because of the file being removed internally on the evaluation.

Explicit $(1)-build and $(1)-configure deps are added that will evaluate after the cleans remove the stamps and cause them to be re-invoked.

---
 package/pkg-generic.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
package/pkg-generic.mk +2 -2
index f2bea01d7d..da99d9622a 100644
@@ -1062,12 +1062,12 @@ $(1)-reinstall:		$(1)-clean-for-reinstall $(1)
 $(1)-clean-for-rebuild: $(1)-clean-for-reinstall
 			rm -f $$($(2)_TARGET_BUILD)
 
-$(1)-rebuild:		$(1)-clean-for-rebuild $(1)
+$(1)-rebuild:		$(1)-clean-for-rebuild $(1)-build $(1)
 
 $(1)-clean-for-reconfigure: $(1)-clean-for-rebuild
 			rm -f $$($(2)_TARGET_CONFIGURE)
 
-$(1)-reconfigure:	$(1)-clean-for-reconfigure $(1)
+$(1)-reconfigure:	$(1)-clean-for-reconfigure $(1)-configure $(1)-build $(1)
 
 # define the PKG variable for all targets, containing the
 # uppercase package variable prefix
Signature · 6 lines
2.24.3 (Apple Git-128)

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw
Quoted 16 lines · Charles Hardin

While testing incremental changes for a kernel build using the LINUX_OVERRIDE_SRCDIR, the following behavior was observed.

- edit a file in the kernel tree and save
- make linux-rebuild for buildroot to pickup the changes
- rsync occurrs and then the install target is run (no build)
- run make linux-rebuild a second time and the build is compiled and run

To get the changes in on the first invocation of the rebuild requires a dependency on the build target to re-evaluate the stamp dependencies after the remove. This is a function of make dependency order evaluation so that the install steps will see the missing the install stamp but that evaluation doesn't require the rebuild because of the file being removed internally on the evaluation.

Quoted 2 lines

Explicit $(1)-build and $(1)-configure deps are added that will evaluate after the cleans remove the stamps and cause them to be re-invoked.

This isn't correct from my understanding as you aren't actually specifying the above described execution order.

I think this should more correctly fix the issue:
https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/

Quoted 11 lines
---
 package/pkg-generic.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
package/pkg-generic.mk +0 -0
index f2bea01d7d..da99d9622a 100644

package/pkg-generic.mk $(1)-reinstall: $(1)-clean-for-reinstall $(1) · line 1065

Quoted 5 lines
package/pkg-generic.mk +1 -1
@@ -1062,12 +1062,12 @@ $(1)-reinstall:         $(1)-clean-for-reinstall $(1)
 $(1)-clean-for-rebuild: $(1)-clean-for-reinstall
                        rm -f $$($(2)_TARGET_BUILD)
 
-$(1)-rebuild:          $(1)-clean-for-rebuild $(1)
+$(1)-rebuild:          $(1)-clean-for-rebuild $(1)-build $(1)

You aren't actually specifying that $(1)-clean-for-rebuild runs before $(1)-build but rather are specifying that the $(1)-build target can be executed in parallel to the $(1)-clean-for-rebuild and $(1) targets.

Parallel targets can effectively be executed in any arbitrary order and/or at the same time.

Quoted 12 lines
package/pkg-generic.mk +1 -2
 $(1)-clean-for-reconfigure: $(1)-clean-for-rebuild
                        rm -f $$($(2)_TARGET_CONFIGURE)
 
-$(1)-reconfigure:      $(1)-clean-for-reconfigure $(1)
+$(1)-reconfigure:      $(1)-clean-for-reconfigure $(1)-configure $(1)-build $(1)
 
 # define the PKG variable for all targets, containing the
 # uppercase package variable prefix
--

2.24.3 (Apple Git-128)

Quoted 4 lines

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#2 lore raw

inline

Quoted 20 lines · James Hilliard
Quoted 16 lines · Charles Hardin

While testing incremental changes for a kernel build using the LINUX_OVERRIDE_SRCDIR, the following behavior was observed.

- edit a file in the kernel tree and save
- make linux-rebuild for buildroot to pickup the changes
- rsync occurrs and then the install target is run (no build)
- run make linux-rebuild a second time and the build is compiled and run

To get the changes in on the first invocation of the rebuild requires a dependency on the build target to re-evaluate the stamp dependencies after the remove. This is a function of make dependency order evaluation so that the install steps will see the missing the install stamp but that evaluation doesn't require the rebuild because of the file being removed internally on the evaluation.

Quoted 2 lines

Explicit $(1)-build and $(1)-configure deps are added that will evaluate after the cleans remove the stamps and cause them to be re-invoked.

Quoted 8 lines

This isn't correct from my understanding as you aren't actually specifying the above described execution order.

I think this should more correctly fix the issue:

https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/

Yup, the evaluation is dependent on the make behavior in the parallel prereq - tried the double colon rules and those work as expected - the rebuild will trigger the build and install and the reconfigure will do the configure, build, install steps.

Quoted 26 lines

package/pkg-generic.mk $(1)-reinstall: $(1)-clean-for-reinstall

Quoted 11 lines
---
 package/pkg-generic.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
package/pkg-generic.mk +0 -0
index f2bea01d7d..da99d9622a 100644
@@ -1062,12 +1062,12 @@ $(1)-reinstall:         $(1)-clean-for-reinstall

$(1)

package/pkg-generic.mk $(1)-reinstall: $(1)-clean-for-reinstall $(1) · line 1065

Quoted 5 lines
package/pkg-generic.mk +1 -1
 $(1)-clean-for-rebuild: $(1)-clean-for-reinstall
                        rm -f $$($(2)_TARGET_BUILD)
 
-$(1)-rebuild:          $(1)-clean-for-rebuild $(1)
+$(1)-rebuild:          $(1)-clean-for-rebuild $(1)-build $(1)

You aren't actually specifying that $(1)-clean-for-rebuild runs before $(1)-build but rather are specifying that the $(1)-build target can be executed in parallel to the $(1)-clean-for-rebuild and $(1) targets.

Parallel targets can effectively be executed in any arbitrary order and/or at the same time.

Quoted 17 lines

package/pkg-generic.mk $(1)-reinstall: $(1)-clean-for-reinstall $(1)

Quoted 5 lines
package/pkg-generic.mk +1 -1
 $(1)-clean-for-reconfigure: $(1)-clean-for-rebuild
                        rm -f $$($(2)_TARGET_CONFIGURE)
 
-$(1)-reconfigure:      $(1)-clean-for-reconfigure $(1)
+$(1)-reconfigure:      $(1)-clean-for-reconfigure $(1)-configure

$(1)-build $(1)

Quoted 10 lines
package/pkg-generic.mk +0 -1
 # define the PKG variable for all targets, containing the
 # uppercase package variable prefix
--
 2.24.3 (Apple Git-128)

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#3 lore raw
Quoted 30 lines · Charles Hardin

inline

Quoted 25 lines · James Hilliard
Quoted 16 lines · Charles Hardin

While testing incremental changes for a kernel build using the LINUX_OVERRIDE_SRCDIR, the following behavior was observed.

- edit a file in the kernel tree and save
- make linux-rebuild for buildroot to pickup the changes
- rsync occurrs and then the install target is run (no build)
- run make linux-rebuild a second time and the build is compiled and run

To get the changes in on the first invocation of the rebuild requires a dependency on the build target to re-evaluate the stamp dependencies after the remove. This is a function of make dependency order evaluation so that the install steps will see the missing the install stamp but that evaluation doesn't require the rebuild because of the file being removed internally on the evaluation.

Quoted 2 lines

Explicit $(1)-build and $(1)-configure deps are added that will evaluate after the cleans remove the stamps and cause them to be re-invoked.

This isn't correct from my understanding as you aren't actually specifying the above described execution order.

I think this should more correctly fix the issue:

Quoted 9 lines
Quoted 3 lines

https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/

Yup, the evaluation is dependent on the make behavior in the parallel prereq - tried the double colon rules and those work as expected - the rebuild will trigger the build and install and the reconfigure will do the configure, build, install steps.

Oh - I thought that double colon patch was already merged into mainline. It is pretty useful to be able to use rebuild and reconfigure on systems for a bit quicker turnaround.

The testing we use is pretty simple.

touch arm64/kernel/setup.c in the local linux directory make linux-rebuild

Will see the "Building" notification with that patch - without it will just rsync and then "Installing"
Watch the CC arm64/kernel/setup in the build

The rest proceeds as expected with Installing - etc.

Quoted 38 lines
Quoted 38 lines

package/pkg-generic.mk $(1)-reinstall:

Quoted 11 lines
---
 package/pkg-generic.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
package/pkg-generic.mk +0 -0
index f2bea01d7d..da99d9622a 100644
@@ -1062,12 +1062,12 @@ $(1)-reinstall:

$(1)-clean-for-reinstall $(1)

package/pkg-generic.mk $(1)-reinstall: $(1)-clean-for-reinstall $(1) · line 1065

Quoted 5 lines
package/pkg-generic.mk +1 -1
 $(1)-clean-for-rebuild: $(1)-clean-for-reinstall
                        rm -f $$($(2)_TARGET_BUILD)
 
-$(1)-rebuild:          $(1)-clean-for-rebuild $(1)
+$(1)-rebuild:          $(1)-clean-for-rebuild $(1)-build $(1)

You aren't actually specifying that $(1)-clean-for-rebuild runs before $(1)-build but rather are specifying that the $(1)-build target can be executed in parallel to the $(1)-clean-for-rebuild and $(1) targets.

Parallel targets can effectively be executed in any arbitrary order and/or at the same time.

package/pkg-generic.mk $(1)-reinstall: $(1)-clean-for-reinstall $(1)

Quoted 5 lines
package/pkg-generic.mk +1 -1
 $(1)-clean-for-reconfigure: $(1)-clean-for-rebuild
                        rm -f $$($(2)_TARGET_CONFIGURE)
 
-$(1)-reconfigure:      $(1)-clean-for-reconfigure $(1)
+$(1)-reconfigure:      $(1)-clean-for-reconfigure $(1)-configure

$(1)-build $(1)

Quoted 6 lines
package/pkg-generic.mk +0 -1
 # define the PKG variable for all targets, containing the
 # uppercase package variable prefix
--
 2.24.3 (Apple Git-128)
Quoted 6 lines
Quoted 5 lines
Quoted 4 lines

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#4 lore raw
Quoted 36 lines · Charles Hardin
Quoted 32 lines · Charles Hardin

inline

Quoted 27 lines · James Hilliard
Quoted 16 lines · Charles Hardin

While testing incremental changes for a kernel build using the LINUX_OVERRIDE_SRCDIR, the following behavior was observed.

- edit a file in the kernel tree and save
- make linux-rebuild for buildroot to pickup the changes
- rsync occurrs and then the install target is run (no build)
- run make linux-rebuild a second time and the build is compiled and run

To get the changes in on the first invocation of the rebuild requires a dependency on the build target to re-evaluate the stamp dependencies after the remove. This is a function of make dependency order evaluation so that the install steps will see the missing the install stamp but that evaluation doesn't require the rebuild because of the file being removed internally on the evaluation.

Quoted 2 lines

Explicit $(1)-build and $(1)-configure deps are added that will evaluate after the cleans remove the stamps and cause them to be re-invoked.

This isn't correct from my understanding as you aren't actually specifying the above described execution order.

I think this should more correctly fix the issue:
https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/

Quoted 5 lines
Quoted 2 lines

Yup, the evaluation is dependent on the make behavior in the parallel prereq - tried the double colon rules and those work as expected - the rebuild will trigger the build and install and the reconfigure will do the configure, build, install steps.

Oh - I thought that double colon patch was already merged into mainline. It is pretty useful to be able to use rebuild and reconfigure on systems for a bit quicker turnaround.

Not yet.

I had discovered the issue when testing with make 4.4's shuffle mode which is designed to make dependency ordering issues like this much more obvious.

Quoted 10 lines

The testing we use is pretty simple.

touch arm64/kernel/setup.c in the local linux directory make linux-rebuild

Will see the "Building" notification with that patch - without it will just rsync and then "Installing" Watch the CC arm64/kernel/setup in the build

The rest proceeds as expected with Installing - etc.

If my patch does indeed fully fix the issue can you reply to it with a tested-by?

Quoted 38 lines
Quoted 38 lines
Quoted 38 lines
Quoted 13 lines
---
 package/pkg-generic.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
package/pkg-generic.mk +0 -0
index f2bea01d7d..da99d9622a 100644

package/pkg-generic.mk $(1)-reinstall: $(1)-clean-for-reinstall $(1) · line 1065

Quoted 5 lines
package/pkg-generic.mk +1 -1
@@ -1062,12 +1062,12 @@ $(1)-reinstall:         $(1)-clean-for-reinstall $(1)
 $(1)-clean-for-rebuild: $(1)-clean-for-reinstall
                        rm -f $$($(2)_TARGET_BUILD)
 
-$(1)-rebuild:          $(1)-clean-for-rebuild $(1)
+$(1)-rebuild:          $(1)-clean-for-rebuild $(1)-build $(1)

You aren't actually specifying that $(1)-clean-for-rebuild runs before $(1)-build but rather are specifying that the $(1)-build target can be executed in parallel to the $(1)-clean-for-rebuild and $(1) targets.

Parallel targets can effectively be executed in any arbitrary order and/or at the same time.

Quoted 12 lines
package/pkg-generic.mk +1 -2
 $(1)-clean-for-reconfigure: $(1)-clean-for-rebuild
                        rm -f $$($(2)_TARGET_CONFIGURE)
 
-$(1)-reconfigure:      $(1)-clean-for-reconfigure $(1)
+$(1)-reconfigure:      $(1)-clean-for-reconfigure $(1)-configure $(1)-build $(1)
 
 # define the PKG variable for all targets, containing the
 # uppercase package variable prefix
--

2.24.3 (Apple Git-128)

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot