package/pkg-generic.mk: fix rule order for re{install, build, configure}
PATCH 1/1 package/pkg-generic.mk: fix rule order for re{install, build, configure}
These command rely on the clean operations being first so that the stamp files being deleted will rebuild the targets.
The execution ordering of the clean and rebuild operations may change, for example if --shuffle=reversed is set.
To ensure the evaluation order is always correct use double colon rules to make the evaluation order explicit as per make docs:
The double-colon rules for a target are executed in the order they appear in the makefile.
--- package/pkg-generic.mk | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
package/pkg-generic.mk +6 -3
@@ -1057,17 +1057,20 @@ endif rm -f $$($(2)_TARGET_INSTALL_IMAGES) rm -f $$($(2)_TARGET_INSTALL_HOST) -$(1)-reinstall: $(1)-clean-for-reinstall $(1) +$(1)-reinstall:: $(1)-clean-for-reinstall +$(1)-reinstall:: $(1) $(1)-clean-for-rebuild: $(1)-clean-for-reinstall rm -f $$($(2)_TARGET_BUILD) -$(1)-rebuild: $(1)-clean-for-rebuild $(1) +$(1)-rebuild:: $(1)-clean-for-rebuild +$(1)-rebuild:: $(1) $(1)-clean-for-reconfigure: $(1)-clean-for-rebuild rm -f $$($(2)_TARGET_CONFIGURE) -$(1)-reconfigure: $(1)-clean-for-reconfigure $(1) +$(1)-reconfigure:: $(1)-clean-for-reconfigure +$(1)-reconfigure:: $(1) # define the PKG variable for all targets, containing the # uppercase package variable prefix
Signature · 6 lines
2.34.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 6 lines · James Hilliard
These command rely on the clean operations being first so that the stamp files being deleted will rebuild the targets.
The execution ordering of the clean and rebuild operations may change, for example if --shuffle=reversed is set.
Quoted 8 lines
To ensure the evaluation order is always correct use double colon rules to make the evaluation order explicit as per make docs:
The double-colon rules for a target are executed in the order they appear in the makefile.
Signed-off-by James Hilliard <[email protected]>
Set LINUX_OVERRIDE_SRCDIR in local.mk
touch arm64/kernel/setup.c
make linux-rebuild
See Building steps with CC on the changed file setup.c
Quoted 35 lines
--- package/pkg-generic.mk | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)package/pkg-generic.mk +6
-4@@ -1057,17 +1057,20 @@ endif rm -f $$($(2)_TARGET_INSTALL_IMAGES) rm -f $$($(2)_TARGET_INSTALL_HOST) -$(1)-reinstall: $(1)-clean-for-reinstall $(1) +$(1)-reinstall:: $(1)-clean-for-reinstall +$(1)-reinstall:: $(1) $(1)-clean-for-rebuild: $(1)-clean-for-reinstall rm -f $$($(2)_TARGET_BUILD) -$(1)-rebuild: $(1)-clean-for-rebuild $(1) +$(1)-rebuild:: $(1)-clean-for-rebuild +$(1)-rebuild:: $(1) $(1)-clean-for-reconfigure: $(1)-clean-for-rebuild rm -f $$($(2)_TARGET_CONFIGURE) -$(1)-reconfigure: $(1)-clean-for-reconfigure $(1) +$(1)-reconfigure:: $(1)-clean-for-reconfigure +$(1)-reconfigure:: $(1) # define the PKG variable for all targets, containing the # uppercase package variable prefix --2.34.1
Quoted 5 lines
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
James, All,
Thanks again for working on such grunt work, it's much appreaciated!
On 2022-10-17 21:46 -0600, James Hilliard spake thusly:
Quoted 11 lines
These command rely on the clean operations being first so that the stamp files being deleted will rebuild the targets.
The execution ordering of the clean and rebuild operations may change, for example if --shuffle=reversed is set.
To ensure the evaluation order is always correct use double colon rules to make the evaluation order explicit as per make docs:
The double-colon rules for a target are executed in the order they appear in the makefile.
I am not convinced that we should switch over to using double-colons.
As I reviewed your other related patch about legal-info [0], the make manual [1] suggests that double-colons are rally intended for cases where "the method used to update a target differs depending on which prerequisite files caused the update" and "double-colon rules really make sense are those where the order of executing the recipes would not matter".
So, we'd use a mechanism where order should not matter, to solve an issue with ordering. That'd be weird...
Yes, the manual states that "double-colon rules for a target are executed in the order they appear in the makefile", but that really looks like a side-effect of another goal.
Also, we'd need to carefully review all our Makefile to ensure that there are no other cases wehre ordering of prerequisites matter. We'd also have to be careful every time we introduce a new rule and remember to check whether it depends on the ordering prereauisites or not, adn worse yet, also whether we can afford the rule to be parallel-safe or not (see the explanation for that in the legal-info review).
Having colons and double-colons is going to be quite a headache... We already have issues with remembering that we need to use $$ to expand variables in some places, and the rules there are relatively clean and explicit, but for the :: case, I'm afraid that's going to be a bit more complex.
We currently have a single location where we use double-colon:
support/kconfig/Makefile.br
which is added by our patch against kconfig:
support/kconfig/patches/10-br-build-system.patch
which dates back to 2010:
7c524dd0b683 Clean up our patches against kconfig
which got in three in 2007:
a665ed34960b - pull kconfig from linux-2.6.21.5
So, I'd like that we get another way to fix that, if possible...
Again, thanks a lot for spending efforts in that area, this is really great!
[0] https://lore.kernel.org/buildroot/20230212091914.GJ2796@scaer/
[1] https://www.gnu.org/software/make/manual/make.html#Double_002dColon
Regards,
Yann E. MORIN.
Quoted 40 lines
Signed-off-by James Hilliard <[email protected]>--- package/pkg-generic.mk | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)package/pkg-generic.mk +6
-3@@ -1057,17 +1057,20 @@ endif rm -f $$($(2)_TARGET_INSTALL_IMAGES) rm -f $$($(2)_TARGET_INSTALL_HOST) -$(1)-reinstall: $(1)-clean-for-reinstall $(1) +$(1)-reinstall:: $(1)-clean-for-reinstall +$(1)-reinstall:: $(1) $(1)-clean-for-rebuild: $(1)-clean-for-reinstall rm -f $$($(2)_TARGET_BUILD) -$(1)-rebuild: $(1)-clean-for-rebuild $(1) +$(1)-rebuild:: $(1)-clean-for-rebuild +$(1)-rebuild:: $(1) $(1)-clean-for-reconfigure: $(1)-clean-for-rebuild rm -f $$($(2)_TARGET_CONFIGURE) -$(1)-reconfigure: $(1)-clean-for-reconfigure $(1) +$(1)-reconfigure:: $(1)-clean-for-reconfigure +$(1)-reconfigure:: $(1) # define the PKG variable for all targets, containing the # uppercase package variable prefixSignature · 6 lines
2.34.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 18 lines · Yann E. MORIN
James, All,
Thanks again for working on such grunt work, it's much appreaciated!
On 2022-10-17 21:46 -0600, James Hilliard spake thusly:
Quoted 11 lines
These command rely on the clean operations being first so that the stamp files being deleted will rebuild the targets.
The execution ordering of the clean and rebuild operations may change, for example if --shuffle=reversed is set.
To ensure the evaluation order is always correct use double colon rules to make the evaluation order explicit as per make docs:
The double-colon rules for a target are executed in the order they appear in the makefile.
Quoted 8 lines
I am not convinced that we should switch over to using double-colons.
As I reviewed your other related patch about legal-info [0], the make manual [1] suggests that double-colons are rally intended for cases where "the method used to update a target differs depending on which prerequisite files caused the update" and "double-colon rules really make sense are those where the order of executing the recipes would not matter".
This appears to be the only documented method of enforcing execution order without disabling parallel builds...that I've come across at least.
Quoted 7 lines
So, we'd use a mechanism where order should not matter, to solve an issue with ordering. That'd be weird...
Yes, the manual states that "double-colon rules for a target are executed in the order they appear in the makefile", but that really looks like a side-effect of another goal.
Yeah, I agree it's not really the intended use case for double-colon rules, however the ordering behavior is documented so I would not expect it to break.
Quoted 7 lines
Also, we'd need to carefully review all our
Makefileto ensure that there are no other cases wehre ordering of prerequisites matter. We'd also have to be careful every time we introduce a new rule and remember to check whether it depends on the ordering prereauisites or not, adn worse yet, also whether we can afford the rule to be parallel-safe or not (see the explanation for that in the legal-info review).
So unlike the legal-info issue using .NOTPARALLEL would not work here as these make targets do need to work with parallel builds.
Quoted 6 lines
Having colons and double-colons is going to be quite a headache... We already have issues with remembering that we need to use $$ to expand variables in some places, and the rules there are relatively clean and explicit, but for the :: case, I'm afraid that's going to be a bit more complex.
I don't think we have all that many of these issues, but I could be wrong.
The clean-for-re{install, build, configure} make targets are a bit hacky to begin with so I don't think make was intended to handle those cases in the first place. We're already in unintended feature use case territory here so this doesn't seem to make things much worse.
Quoted 7 lines
We currently have a single location where we use double-colon:
support/kconfig/Makefile.brwhich is added by our patch against kconfig:
support/kconfig/patches/10-br-build-system.patch
Quoted 7 lines
which dates back to 2010:
7c524dd0b683Clean up our patches against kconfigwhich got in three in 2007:
a665ed34960b - pull kconfig from linux-2.6.21.5So, I'd like that we get another way to fix that, if possible...
I have unfortunately not found any other method to fix the execution ordering...
Quoted 53 lines
Again, thanks a lot for spending efforts in that area, this is really great!
[0] https://lore.kernel.org/buildroot/20230212091914.GJ2796@scaer/
[1] https://www.gnu.org/software/make/manual/make.html#Double_002dColonRegards,
Yann E. MORIN.Quoted 36 lines
Signed-off-by James Hilliard <[email protected]>--- package/pkg-generic.mk | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)package/pkg-generic.mk +6
-4@@ -1057,17 +1057,20 @@ endif rm -f $$($(2)_TARGET_INSTALL_IMAGES) rm -f $$($(2)_TARGET_INSTALL_HOST) -$(1)-reinstall: $(1)-clean-for-reinstall $(1) +$(1)-reinstall:: $(1)-clean-for-reinstall +$(1)-reinstall:: $(1) $(1)-clean-for-rebuild: $(1)-clean-for-reinstall rm -f $$($(2)_TARGET_BUILD) -$(1)-rebuild: $(1)-clean-for-rebuild $(1) +$(1)-rebuild:: $(1)-clean-for-rebuild +$(1)-rebuild:: $(1) $(1)-clean-for-reconfigure: $(1)-clean-for-rebuild rm -f $$($(2)_TARGET_CONFIGURE) -$(1)-reconfigure: $(1)-clean-for-reconfigure $(1) +$(1)-reconfigure:: $(1)-clean-for-reconfigure +$(1)-reconfigure:: $(1) # define the PKG variable for all targets, containing the # uppercase package variable prefix --2.34.1
Quoted 4 lines
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot--
.-----------------.--------------------.------------------.--------------------.
Quoted 5 lines
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
James, All,
On 2023-02-12 03:02 -0700, James Hilliard spake thusly:
Quoted 7 lines · Yann E. MORIN
Quoted 6 lines
On 2022-10-17 21:46 -0600, James Hilliard spake thusly:
Quoted 5 lines
To ensure the evaluation order is always correct use double colon rules to make the evaluation order explicit as per make docs:
The double-colon rules for a target are executed in the order they appear in the makefile.
[--SNIP--]
Quoted 3 lines
Quoted 1 line
So, I'd like that we get another way to fix that, if possible...
I have unfortunately not found any other method to fix the execution ordering...
What about something along the lines of (untestd!):
$(1)-rebuild: $(1): $(1)-clean-for-rebuild
$(1)-rebuild: $(1)We already use such a construct in package/pkg-kconfig.mk, but only once, so this is not as if it were widespread either...
Still, I think it is more easy to read...
Regards,
Yann E. MORIN.
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 12 lines · Yann E. MORIN
James, All,
On 2023-02-12 03:02 -0700, James Hilliard spake thusly:
Quoted 7 lines · Yann E. MORIN
Quoted 6 lines
On 2022-10-17 21:46 -0600, James Hilliard spake thusly:
Quoted 5 lines
To ensure the evaluation order is always correct use double colon rules to make the evaluation order explicit as per make docs:
The double-colon rules for a target are executed in the order they appear in the makefile.
[--SNIP--]
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
So, I'd like that we get another way to fix that, if possible...
I have unfortunately not found any other method to fix the execution ordering...
What about something along the lines of (untestd!):
$(1)-rebuild: $(1): $(1)-clean-for-rebuild $(1)-rebuild: $(1)
That would appear to be invalid syntax.
Quoted 3 lines
We already use such a construct in
package/pkg-kconfig.mk, but only once, so this is not as if it were widespread either...
I don't see anything like that there, just this which isn't doing anything unusual with multiple prerequisites that require a specific execution order like the re{install,build,configure} rules in pkg-generic.mk:
# Force olddefconfig again on -reconfigure $(1)-clean-for-reconfigure: $(1)-clean-kconfig-for-reconfigure
$(1)-clean-kconfig-for-reconfigure:
rm -f $$($(2)_DIR)/$$($(2)_KCONFIG_STAMP_DOTCONFIG)
Quoted 6 lines
Still, I think it is more easy to read...
Regards,
Yann E. MORIN.
Quoted 7 lines
--
.-----------------.--------------------.------------------.--------------------.| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
James, All,
On 2023-02-12 03:22 -0700, James Hilliard spake thusly:
Quoted 22 lines · Yann E. MORIN
Quoted 12 lines
James, All,
On 2023-02-12 03:02 -0700, James Hilliard spake thusly:
Quoted 7 lines · Yann E. MORIN
Quoted 6 lines
On 2022-10-17 21:46 -0600, James Hilliard spake thusly:
Quoted 5 lines
To ensure the evaluation order is always correct use double colon rules to make the evaluation order explicit as per make docs:
The double-colon rules for a target are executed in the order they appear in the makefile.
[--SNIP--]
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
So, I'd like that we get another way to fix that, if possible...
I have unfortunately not found any other method to fix the execution ordering...
What about something along the lines of (untestd!):
$(1)-rebuild: $(1): $(1)-clean-for-rebuild $(1)-rebuild: $(1)
Quoted 7 lines
That would appear to be invalid syntax.
Quoted 3 lines
We already use such a construct in
package/pkg-kconfig.mk, but only once, so this is not as if it were widespread either...I don't see anything like that there,
240: $$(addprefix $(1)-,$$($(2)_KCONFIG_EDITORS)): $(1)-%: $$($(2)_DIR)/.kconfig_editor_%
So, this looks like it is named "target pattern", and it works with a pattern (eh!).
$ cat Makefile
1 all: a
2
3 a:
4 @echo a
5
6 a-clean:
7 @sleep 1
8 @echo a-clean
9
10 a-rebuild: %-rebuild: %-clean
11 a-rebuild: aHowever, that does not work with parallel builds:
$ make
a $ make a
a $ make a-rebuild
a-clean
a $ make -j a-rebuild
a
a-cleanHowever, you stated in a previous reply:
So unlike the legal-info issue using .NOTPARALLEL would not work here as these make targets do need to work with parallel builds.
I don't think that's true. Indeed, foo-rebuild is usually done after a ful build, when iterating over the foo package, iether to test a new source tree (with ocerride-ssrcdir) or to test fixes in foo.mk. In either case, we do not need a top-level parallel build, because only one package, foo, will be built.
So, even in this case, we should be able to use .NOTPARALLEL, no?
It also means that the kconfig rule I mentionned above, does not work with top-level parallel builds, but that's OK: it's only used to spawn kconfig configurators (menuconfig et al.), so it should not be used with top-level parallel, and thus should require .NOTPARALLEL as well.
Meh, TLPB and shuffling are really uncovering issues everywhere... :-(
Regards,
Yann E. MORIN.
Quoted 23 lines
just this which isn't doing anything
unusual with multiple prerequisites that require a specific execution order like the re{install,build,configure} rules in pkg-generic.mk:# Force olddefconfig again on -reconfigure $(1)-clean-for-reconfigure: $(1)-clean-kconfig-for-reconfigure
$(1)-clean-kconfig-for-reconfigure:
rm -f $$($(2)_DIR)/$$($(2)_KCONFIG_STAMP_DOTCONFIG)Quoted 13 lines
Still, I think it is more easy to read...
Regards,
Yann E. MORIN.--
.-----------------.--------------------.------------------.--------------------.| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |'------------------------------^-------^------------------^--------------------'
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 73 lines · Yann E. MORIN
James, All,
On 2023-02-12 03:22 -0700, James Hilliard spake thusly:
Quoted 22 lines · Yann E. MORIN
Quoted 12 lines
James, All,
On 2023-02-12 03:02 -0700, James Hilliard spake thusly:
Quoted 7 lines · Yann E. MORIN
Quoted 6 lines
On 2022-10-17 21:46 -0600, James Hilliard spake thusly:
Quoted 5 lines
To ensure the evaluation order is always correct use double colon rules to make the evaluation order explicit as per make docs:
The double-colon rules for a target are executed in the order they appear in the makefile.
[--SNIP--]
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
So, I'd like that we get another way to fix that, if possible...
I have unfortunately not found any other method to fix the execution ordering...
What about something along the lines of (untestd!):
$(1)-rebuild: $(1): $(1)-clean-for-rebuild $(1)-rebuild: $(1)Quoted 7 lines
That would appear to be invalid syntax.
Quoted 3 lines
We already use such a construct in
package/pkg-kconfig.mk, but only once, so this is not as if it were widespread either...I don't see anything like that there,
240: $$(addprefix $(1)-,$$($(2)_KCONFIG_EDITORS)): $(1)-%: $$($(2)_DIR)/.kconfig_editor_%So, this looks like it is named "target pattern", and it works with a pattern (eh!).
$ cat Makefile 1 all: a 2 3 a: 4 @echo a 5 6 a-clean: 7 @sleep 1 8 @echo a-clean 9 10 a-rebuild: %-rebuild: %-clean 11 a-rebuild: aHowever, that does not work with parallel builds:
$ make a$ make a a$ make a-rebuild a-clean a$ make -j a-rebuild a a-cleanHowever, you stated in a previous reply:
So unlike the legal-info issue using .NOTPARALLEL would not work here as these make targets do need to work with parallel builds.
Quoted 7 lines
I don't think that's true. Indeed, foo-rebuild is usually done after a ful build, when iterating over the foo package, iether to test a new source tree (with ocerride-ssrcdir) or to test fixes in foo.mk. In either case, we do not need a top-level parallel build, because only one package, foo, will be built.
So, even in this case, we should be able to use .NOTPARALLEL, no?
Breaking parallel build for foo-rebuild would be rather unexpected behavior as foo-rebuild does work for full builds as well and one may default to using foo-rebuild instead of a foo target for various reasons(to ensure that a package is always rsynced from an overridden source directory for example).
I see no good reason to disable parallel build on it when simply ordering the dependency execution order fixes the issue.
Quoted 5 lines
It also means that the kconfig rule I mentionned above, does not work with top-level parallel builds, but that's OK: it's only used to spawn kconfig configurators (menuconfig et al.), so it should not be used with top-level parallel, and thus should require .NOTPARALLEL as well.
Might be better to use double colon rules there as well, they are more explicit than using .NOTPARALLEL IMO as .NOTPARALLEL doesn't specify an execution order, also it can potentially introduce accidental performance regressions.
Quoted 2 lines
Meh, TLPB and shuffling are really uncovering issues everywhere... :-(
Better to make them very obvious so they can be fixed IMO.
Once the known issues are fixed I can run shuffle mode on my autobuilders to uncover remaining issues.
Quoted 28 lines
Regards,
Yann E. MORIN.Quoted 4 lines
just this which isn't doing anything
unusual with multiple prerequisites that require a specific execution order like the re{install,build,configure} rules in pkg-generic.mk:Quoted 19 lines
# Force olddefconfig again on -reconfigure $(1)-clean-for-reconfigure: $(1)-clean-kconfig-for-reconfigure
$(1)-clean-kconfig-for-reconfigure:
rm -f $$($(2)_DIR)/$$($(2)_KCONFIG_STAMP_DOTCONFIG)Quoted 13 lines
Still, I think it is more easy to read...
Regards,
Yann E. MORIN.--
.-----------------.--------------------.------------------.--------------------.| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |'------------------------------^-------^------------------^--------------------'
Quoted 7 lines
--
.-----------------.--------------------.------------------.--------------------.| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 11 lines · Yann E. MORIN
James, All,
Thanks again for working on such grunt work, it's much appreaciated!
On 2022-10-17 21:46 -0600, James Hilliard spake thusly:
Quoted 6 lines
These command rely on the clean operations being first so that the stamp files being deleted will rebuild the targets.
The execution ordering of the clean and rebuild operations may change, for example if --shuffle=reversed is set.
Quoted 7 lines
Quoted 5 lines
To ensure the evaluation order is always correct use double colon rules to make the evaluation order explicit as per make docs:
The double-colon rules for a target are executed in the order they appear in the makefile.
I am not convinced that we should switch over to using double-colons.
Me neither.
Quoted 10 lines
As I reviewed your other related patch about legal-info [0], the make manual [1] suggests that double-colons are rally intended for cases where "the method used to update a target differs depending on which prerequisite files caused the update" and "double-colon rules really make sense are those where the order of executing the recipes would not matter".
So, we'd use a mechanism where order should not matter, to solve an issue with ordering. That'd be weird...
Quoted 6 lines
Yes, the manual states that "double-colon rules for a target are executed in the order they appear in the makefile", but that really looks like a side-effect of another goal.
Also, we'd need to carefully review all our
Makefileto ensure that there are no other cases wehre ordering of prerequisites matter.
Well, we need to do that regardless of which solution is found, right? Unless
we just disable parallel builds and --shuffle entirely.
Quoted 8 lines
We'd
also have to be careful every time we introduce a new rule and remember to check whether it depends on the ordering prereauisites or not, adn worse yet, also whether we can afford the rule to be parallel-safe or not (see the explanation for that in the legal-info review).Having colons and double-colons is going to be quite a headache... We already have issues with remembering that we need to use $$ to expand
Yeah, maybe we should have some check-package thing to verify that $$ is used
everywhere in a macro definition...
Quoted 40 lines
variables in some places, and the rules there are relatively clean and explicit, but for the :: case, I'm afraid that's going to be a bit more complex.
We currently have a single location where we use double-colon:
support/kconfig/Makefile.brwhich is added by our patch against kconfig:
support/kconfig/patches/10-br-build-system.patchwhich dates back to 2010:
7c524dd0b683Clean up our patches against kconfigwhich got in three in 2007:
a665ed34960b - pull kconfig from linux-2.6.21.5So, I'd like that we get another way to fix that, if possible...
Again, thanks a lot for spending efforts in that area, this is really great!
[0] https://lore.kernel.org/buildroot/20230212091914.GJ2796@scaer/
[1] https://www.gnu.org/software/make/manual/make.html#Double_002dColonRegards,
Yann E. MORIN.Quoted 13 lines
Signed-off-by James Hilliard <[email protected]>--- package/pkg-generic.mk | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)package/pkg-generic.mk +0
-0@@ -1057,17 +1057,20 @@ endif rm -f $$($(2)_TARGET_INSTALL_IMAGES) rm -f $$($(2)_TARGET_INSTALL_HOST)
Quoted 3 lines
package/pkg-generic.mk endif · line 1061
Quoted 3 lines
package/pkg-generic.mk +2
-1-$(1)-reinstall: $(1)-clean-for-reinstall $(1) +$(1)-reinstall:: $(1)-clean-for-reinstall +$(1)-reinstall:: $(1)
I may be stupid, but what we want is to have an ordering between
clean-for-reinstall and $(1), right? So, let's use an ordering dependency?
$(1)-reinstall: $(1)-clean-for-reinstall $(1)
$(1): $(1)-clean-for-reinstall
Regards,
Arnout
Quoted 20 lines
Quoted 20 lines
package/pkg-generic.mk +4
-2$(1)-clean-for-rebuild: $(1)-clean-for-reinstall rm -f $$($(2)_TARGET_BUILD) -$(1)-rebuild: $(1)-clean-for-rebuild $(1) +$(1)-rebuild:: $(1)-clean-for-rebuild +$(1)-rebuild:: $(1) $(1)-clean-for-reconfigure: $(1)-clean-for-rebuild rm -f $$($(2)_TARGET_CONFIGURE) -$(1)-reconfigure: $(1)-clean-for-reconfigure $(1) +$(1)-reconfigure:: $(1)-clean-for-reconfigure +$(1)-reconfigure:: $(1) # define the PKG variable for all targets, containing the # uppercase package variable prefixSignature · 1 line
2.34.1
Quoted 5 lines
Quoted 4 lines
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
On 14/02/2023 22:22, Arnout Vandecappelle wrote:
[1]
Quoted 4 lines
I may be stupid, but what we want is to have an ordering between clean-for-reinstall and $(1), right? So, let's use an ordering dependency?
$(1)-reinstall: $(1)-clean-for-reinstall $(1) $(1): $(1)-clean-for-reinstall
Gah, I forgot the actual order-only bit...
$(1)-reinstall: $(1)-clean-for-reinstall $(1)
$(1): | $(1)-clean-for-reinstall
Regards,
Arnout
Quoted 25 lines
Regards,
ArnoutQuoted 20 lines
Quoted 19 lines
$(1)-clean-for-rebuild: $(1)-clean-for-reinstall rm -f $$($(2)_TARGET_BUILD) -$(1)-rebuild: $(1)-clean-for-rebuild $(1) +$(1)-rebuild:: $(1)-clean-for-rebuild +$(1)-rebuild:: $(1) $(1)-clean-for-reconfigure: $(1)-clean-for-rebuild rm -f $$($(2)_TARGET_CONFIGURE) -$(1)-reconfigure: $(1)-clean-for-reconfigure $(1) +$(1)-reconfigure:: $(1)-clean-for-reconfigure +$(1)-reconfigure:: $(1)
# define the PKG variable for all targets, containing the # uppercase package variable prefixSignature · 6 lines
2.34.1 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 7 lines · Arnout Vandecappelle
On 14/02/2023 22:22, Arnout Vandecappelle wrote:
[1]Quoted 3 lines
I may be stupid, but what we want is to have an ordering between
clean-for-reinstall and $(1), right? So, let's use an ordering dependency?
Quoted 8 lines
Quoted 2 lines
$(1)-reinstall: $(1)-clean-for-reinstall $(1)
$(1): $(1)-clean-for-reinstallGah, I forgot the actual order-only bit...
$(1)-reinstall: $(1)-clean-for-reinstall $(1)
$(1): | $(1)-clean-for-reinstall
This doesn't seem to work, it ends up cleaning a bunch of additional targets like the skeleton and toolchain that shouldn't be cleaned when doing a reinstall for say a normal python package.
Quoted 31 lines
Regards,
ArnoutQuoted 25 lines
Regards,
ArnoutQuoted 20 lines
Quoted 19 lines
$(1)-clean-for-rebuild: $(1)-clean-for-reinstall rm -f $$($(2)_TARGET_BUILD) -$(1)-rebuild: $(1)-clean-for-rebuild $(1) +$(1)-rebuild:: $(1)-clean-for-rebuild +$(1)-rebuild:: $(1) $(1)-clean-for-reconfigure: $(1)-clean-for-rebuild rm -f $$($(2)_TARGET_CONFIGURE) -$(1)-reconfigure: $(1)-clean-for-reconfigure $(1) +$(1)-reconfigure:: $(1)-clean-for-reconfigure +$(1)-reconfigure:: $(1) # define the PKG variable for all targets, containing the # uppercase package variable prefix -- 2.34.1_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Hi James,
Quoted 11 lines · James Hilliard
These command rely on the clean operations being first so that the stamp files being deleted will rebuild the targets.
The execution ordering of the clean and rebuild operations may change, for example if --shuffle=reversed is set.
To ensure the evaluation order is always correct use double colon rules to make the evaluation order explicit as per make docs:
The double-colon rules for a target are executed in the order they appear in the makefile.
As noted in earlier discussions, this feels a bit hackish and it's not clear
if this behaviour is going to be kept in future versions of make.
Since make 4.4, however, there's a better solution: it introduced a .WAIT
phony target that enforces the ordering of dependencies. So I instead made a commit[1] that uses that approach.
Regards,
Arnout
[1] https://gitlab.com/buildroot.org/buildroot/-/commit/7e3105d5c8bd9ab31e728af9732a67d203c0a2e4
Quoted 24 lines
Signed-off-by James Hilliard <[email protected]>--- package/pkg-generic.mk | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)package/pkg-generic.mk +4
-2@@ -1057,17 +1057,20 @@ endif rm -f $$($(2)_TARGET_INSTALL_IMAGES) rm -f $$($(2)_TARGET_INSTALL_HOST) -$(1)-reinstall: $(1)-clean-for-reinstall $(1) +$(1)-reinstall:: $(1)-clean-for-reinstall +$(1)-reinstall:: $(1) $(1)-clean-for-rebuild: $(1)-clean-for-reinstall rm -f $$($(2)_TARGET_BUILD) -$(1)-rebuild: $(1)-clean-for-rebuild $(1) +$(1)-rebuild:: $(1)-clean-for-rebuild +$(1)-rebuild:: $(1)
package/pkg-generic.mk endif · line 1076
Quoted 10 lines
package/pkg-generic.mk +2
-1$(1)-clean-for-reconfigure: $(1)-clean-for-rebuild rm -f $$($(2)_TARGET_CONFIGURE) -$(1)-reconfigure: $(1)-clean-for-reconfigure $(1) +$(1)-reconfigure:: $(1)-clean-for-reconfigure +$(1)-reconfigure:: $(1) # define the PKG variable for all targets, containing the # uppercase package variable prefix
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 13 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Arnout" == Arnout Vandecappelle via buildroot <[email protected]> writes:
Hi James,
Quoted 8 lines · James Hilliard
These command rely on the clean operations being first so that the stamp files being deleted will rebuild the targets.
The execution ordering of the clean and rebuild operations may change, for example if --shuffle=reversed is set.
To ensure the evaluation order is always correct use double colon rules to make the evaluation order explicit as per make docs:
The double-colon rules for a target are executed in the order they appear in the makefile.
Quoted 7 lines
As noted in earlier discussions, this feels a bit hackish and it's not clear if this behaviour is going to be kept in future versions of
make.Since make 4.4, however, there's a better solution: it introduced a .WAIT phony target that enforces the ordering of dependencies. So I
instead made a commit[1] that uses that approach.
Committed to 2023.02.x and 2023.08.x, thanks.
Signature · 5 lines
Bye, Peter Korsgaard _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot