Skip to contentSkip to conversation

build: retain intermediate .aux.o targets

PATCH In review Nicholas Piggin 5 participants 9 messages Jun 12, 2024 – Jul 26, 2024

Subsystems (5)
KVM RISC-V s390 KVM arm64 +1 more subsystems
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 build: retain intermediate .aux.o targets

arm, powerpc, riscv, build .aux.o targets with implicit pattern rules in dependency chains that cause them to be made as intermediate files, which get removed when make finishes. This results in unnecessary partial rebuilds. If make is run again, this time the .aux.o targets are not intermediate, possibly due to being made via different dependencies.

Adding .aux.o files to .PRECIOUS prevents them being removed and solves the rebuild problem.

s390x does not have the problem because .SECONDARY prevents dependancies from being built as intermediate. However the same change is made for s390x, for consistency.

---
 arm/Makefile.common     | 2 +-
 powerpc/Makefile.common | 2 +-
 riscv/Makefile          | 2 +-
 s390x/Makefile          | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
arm/Makefile.common +1 -1
index f828dbe01..0b26a92a6 100644
@@ -31,7 +31,7 @@ CFLAGS += -O2
 CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
 
 # We want to keep intermediate files
-.PRECIOUS: %.elf %.o
+.PRECIOUS: %.elf %.o %.aux.o
 
 asm-offsets = lib/$(ARCH)/asm-offsets.h
 include $(SRCDIR)/scripts/asm-offsets.mak
powerpc/Makefile.common +1 -1
index b98f71c2f..16f14577e 100644
@@ -30,7 +30,7 @@ CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
 CFLAGS += -Wa,-mregnames
 
 # We want to keep intermediate files
-.PRECIOUS: %.o
+.PRECIOUS: %.o %.aux.o
 
 asm-offsets = lib/$(ARCH)/asm-offsets.h
 include $(SRCDIR)/scripts/asm-offsets.mak
riscv/Makefile +1 -1
index 919a3ebb5..7207ff988 100644
@@ -53,7 +53,7 @@ AUXFLAGS ?= 0x0
 KEEP_FRAME_POINTER := y
 
 # We want to keep intermediate files
-.PRECIOUS: %.elf %.o
+.PRECIOUS: %.elf %.o %.aux.o
 
 define arch_elf_check =
 	$(if $(shell ! $(READELF) -rW $(1) >&/dev/null && echo "nok"),
s390x/Makefile +1 -1
index 23342bd64..d436c6e9a 100644
@@ -85,7 +85,7 @@ CFLAGS += -fno-delete-null-pointer-checks
 LDFLAGS += -Wl,--build-id=none
 
 # We want to keep intermediate files
-.PRECIOUS: %.o %.lds
+.PRECIOUS: %.o %.aux.o %.lds
 
 asm-offsets = lib/$(ARCH)/asm-offsets.h
 include $(SRCDIR)/scripts/asm-offsets.mak
Signature · 1 line
2.45.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 10 lines · Nicholas Piggin

arm, powerpc, riscv, build .aux.o targets with implicit pattern rules in dependency chains that cause them to be made as intermediate files, which get removed when make finishes. This results in unnecessary partial rebuilds. If make is run again, this time the .aux.o targets are not intermediate, possibly due to being made via different dependencies.

Adding .aux.o files to .PRECIOUS prevents them being removed and solves the rebuild problem.

Quoted 7 lines

s390x does not have the problem because .SECONDARY prevents dependancies from being built as intermediate. However the same change is made for s390x, for consistency.

---

[?snip?]

Quoted 13 lines
s390x/Makefile +1 -1
@@ -85,7 +85,7 @@ CFLAGS += -fno-delete-null-pointer-checks
 LDFLAGS += -Wl,--build-id=none
 
 # We want to keep intermediate files
-.PRECIOUS: %.o %.lds
+.PRECIOUS: %.o %.aux.o %.lds
 
 asm-offsets = lib/$(ARCH)/asm-offsets.h
 include $(SRCDIR)/scripts/asm-offsets.mak
Signature · 1 line
2.45.1

Thanks for fixing this!

Signature · 8 lines
Kind regards / Beste Gr??e
   Marc Hartmayer

IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Wolfgang Wendt
Gesch?ftsf?hrung: David Faller
Sitz der Gesellschaft: B?blingen
Registergericht: Amtsgericht Stuttgart, HRB 243294
Acked-by on v1 1/1
#2 lore raw
In reply to Nicholas Piggin
Quoted 14 lines · Nicholas Piggin

arm, powerpc, riscv, build .aux.o targets with implicit pattern rules in dependency chains that cause them to be made as intermediate files, which get removed when make finishes. This results in unnecessary partial rebuilds. If make is run again, this time the .aux.o targets are not intermediate, possibly due to being made via different dependencies.

Adding .aux.o files to .PRECIOUS prevents them being removed and solves the rebuild problem.

s390x does not have the problem because .SECONDARY prevents dependancies from being built as intermediate. However the same change is made for s390x, for consistency.

Quoted 9 lines
---
 arm/Makefile.common     | 2 +-
 powerpc/Makefile.common | 2 +-
 riscv/Makefile          | 2 +-
 s390x/Makefile          | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
#3 lore raw
In reply to Nicholas Piggin
Quoted 7 lines · Nicholas Piggin

arm, powerpc, riscv, build .aux.o targets with implicit pattern rules in dependency chains that cause them to be made as intermediate files, which get removed when make finishes. This results in unnecessary partial rebuilds. If make is run again, this time the .aux.o targets are not intermediate, possibly due to being made via different dependencies.

Quoted 6 lines

Adding .aux.o files to .PRECIOUS prevents them being removed and solves the rebuild problem.

s390x does not have the problem because .SECONDARY prevents dependancies from being built as intermediate. However the same change is made for s390x, for consistency.

This is exactly what .SECONDARY is for, as its documentation says, even. Wouldn't it be better to just add a .SECONDARY to the other targets as well?

Segher

#4 lore raw
Quoted 11 lines · Segher Boessenkool
Quoted 10 lines · Nicholas Piggin

arm, powerpc, riscv, build .aux.o targets with implicit pattern rules in dependency chains that cause them to be made as intermediate files, which get removed when make finishes. This results in unnecessary partial rebuilds. If make is run again, this time the .aux.o targets are not intermediate, possibly due to being made via different dependencies.

Adding .aux.o files to .PRECIOUS prevents them being removed and solves the rebuild problem.

Quoted 7 lines
Quoted 3 lines

s390x does not have the problem because .SECONDARY prevents dependancies from being built as intermediate. However the same change is made for s390x, for consistency.

This is exactly what .SECONDARY is for, as its documentation says, even. Wouldn't it be better to just add a .SECONDARY to the other targets as well?

Yeah we were debating that and agreed .PRECIOUS may not be the cleanest fix but since we already use that it's okay for a minimal fix.

Thanks,
Nick

#5 lore raw
Quoted 16 lines · Nicholas Piggin
Quoted 15 lines · Segher Boessenkool
Quoted 13 lines · Nicholas Piggin

arm, powerpc, riscv, build .aux.o targets with implicit pattern rules in dependency chains that cause them to be made as intermediate files, which get removed when make finishes. This results in unnecessary partial rebuilds. If make is run again, this time the .aux.o targets are not intermediate, possibly due to being made via different dependencies.

Adding .aux.o files to .PRECIOUS prevents them being removed and solves the rebuild problem.

s390x does not have the problem because .SECONDARY prevents dependancies from being built as intermediate. However the same change is made for s390x, for consistency.

Quoted 7 lines
Quoted 3 lines

This is exactly what .SECONDARY is for, as its documentation says, even. Wouldn't it be better to just add a .SECONDARY to the other targets as well?

Yeah we were debating that and agreed .PRECIOUS may not be the cleanest fix but since we already use that it's okay for a minimal fix.

But why add it to s390x then? It is not a fix there at all!

Segher

#6 lore raw
Quoted 21 lines · Segher Boessenkool
Quoted 20 lines · Nicholas Piggin
Quoted 18 lines · Segher Boessenkool
Quoted 7 lines · Nicholas Piggin

arm, powerpc, riscv, build .aux.o targets with implicit pattern rules in dependency chains that cause them to be made as intermediate files, which get removed when make finishes. This results in unnecessary partial rebuilds. If make is run again, this time the .aux.o targets are not intermediate, possibly due to being made via different dependencies.

Quoted 6 lines

Adding .aux.o files to .PRECIOUS prevents them being removed and solves the rebuild problem.

s390x does not have the problem because .SECONDARY prevents dependancies from being built as intermediate. However the same change is made for s390x, for consistency.

This is exactly what .SECONDARY is for, as its documentation says, even. Wouldn't it be better to just add a .SECONDARY to the other targets as well?

Quoted 5 lines
Quoted 3 lines

Yeah we were debating that and agreed .PRECIOUS may not be the cleanest fix but since we already use that it's okay for a minimal fix.

But why add it to s390x then? It is not a fix there at all!

Eh, not a big deal. I mentioned that in the changelog it doesn't seem to pracicaly fix something. And I rather the makefiles converge as much as possible rather than diverge more.

.SECONDARY was added independently and not to fix this problem in s390x. And s390x has .SECONDARY slightly wrong AFAIKS. It mentions .SECONDARY: twice in a way that looks like it was meant to depend on specific targets, it actually gives it no dependencies and the resulting semantics are that all intermediate files in the build are treated as secondary. So somethig there should be cleaned up. If the .SECONDARY was changed to only depend on the .gobj and .hdr.obj then suddenly that would break .aux.o if I don't make the change.

So I'm meaning to work out what to do with all that, i.e., whether to add blanket .SECONDARY for all and trim or remove the .PRECIOUS files, or remove s390x's secondary, or make it more specific, or something else. But it takes a while for me to do makefile work.

Thanks,
Nick

#7 lore raw
Quoted 41 lines · Nicholas Piggin
Quoted 21 lines · Segher Boessenkool
Quoted 20 lines · Nicholas Piggin
Quoted 18 lines · Segher Boessenkool
Quoted 7 lines · Nicholas Piggin

arm, powerpc, riscv, build .aux.o targets with implicit pattern rules in dependency chains that cause them to be made as intermediate files, which get removed when make finishes. This results in unnecessary partial rebuilds. If make is run again, this time the .aux.o targets are not intermediate, possibly due to being made via different dependencies.

Quoted 6 lines

Adding .aux.o files to .PRECIOUS prevents them being removed and solves the rebuild problem.

s390x does not have the problem because .SECONDARY prevents dependancies from being built as intermediate. However the same change is made for s390x, for consistency.

This is exactly what .SECONDARY is for, as its documentation says, even. Wouldn't it be better to just add a .SECONDARY to the other targets as well?

Quoted 5 lines
Quoted 3 lines

Yeah we were debating that and agreed .PRECIOUS may not be the cleanest fix but since we already use that it's okay for a minimal fix.

But why add it to s390x then? It is not a fix there at all!

Eh, not a big deal. I mentioned that in the changelog it doesn't seem to pracicaly fix something. And I rather the makefiles converge as much as possible rather than diverge more.

.SECONDARY was added independently and not to fix this problem in s390x. And s390x has .SECONDARY slightly wrong AFAIKS. It mentions .SECONDARY: twice in a way that looks like it was meant to depend on specific targets, it actually gives it no dependencies and the resulting semantics are that all intermediate files in the build are treated as secondary. So somethig there should be cleaned up. If the .SECONDARY was changed to only depend on the .gobj and .hdr.obj then suddenly that would break .aux.o if I don't make the change.

Quoted 4 lines

So I'm meaning to work out what to do with all that, i.e., whether to add blanket .SECONDARY for all and trim or remove the .PRECIOUS files, or remove s390x's secondary, or make it more specific, or something else. But it takes a while for me to do makefile work.

Hi Thomas,

Ping on this patch?

Thanks,
Nick

#8 lore raw
Quoted 42 lines · Nicholas Piggin
Quoted 41 lines · Nicholas Piggin
Quoted 21 lines · Segher Boessenkool
Quoted 20 lines · Nicholas Piggin
Quoted 18 lines · Segher Boessenkool
Quoted 7 lines · Nicholas Piggin

arm, powerpc, riscv, build .aux.o targets with implicit pattern rules in dependency chains that cause them to be made as intermediate files, which get removed when make finishes. This results in unnecessary partial rebuilds. If make is run again, this time the .aux.o targets are not intermediate, possibly due to being made via different dependencies.

Quoted 6 lines

Adding .aux.o files to .PRECIOUS prevents them being removed and solves the rebuild problem.

s390x does not have the problem because .SECONDARY prevents dependancies from being built as intermediate. However the same change is made for s390x, for consistency.

This is exactly what .SECONDARY is for, as its documentation says, even. Wouldn't it be better to just add a .SECONDARY to the other targets as well?

Quoted 5 lines
Quoted 3 lines

Yeah we were debating that and agreed .PRECIOUS may not be the cleanest fix but since we already use that it's okay for a minimal fix.

But why add it to s390x then? It is not a fix there at all!

Eh, not a big deal. I mentioned that in the changelog it doesn't seem to pracicaly fix something. And I rather the makefiles converge as much as possible rather than diverge more.

.SECONDARY was added independently and not to fix this problem in s390x. And s390x has .SECONDARY slightly wrong AFAIKS. It mentions .SECONDARY: twice in a way that looks like it was meant to depend on specific targets, it actually gives it no dependencies and the resulting semantics are that all intermediate files in the build are treated as secondary. So somethig there should be cleaned up. If the .SECONDARY was changed to only depend on the .gobj and .hdr.obj then suddenly that would break .aux.o if I don't make the change.

Quoted 8 lines
Quoted 4 lines

So I'm meaning to work out what to do with all that, i.e., whether to add blanket .SECONDARY for all and trim or remove the .PRECIOUS files, or remove s390x's secondary, or make it more specific, or something else. But it takes a while for me to do makefile work.

Hi Thomas,

Ping on this patch?

I assumed that Marc would chime in on the s390x part here?

Thomas