Skip to contentSkip to conversation

libacpi: use temporary files for generated files

PATCH In review Olaf Hering 4 participants 11 messages Oct 26, 2020 – Oct 29, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 libacpi: use temporary files for generated files

Use a temporay file, and move it in place once done. The same pattern exists already for other dependencies.

---
 tools/libacpi/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
tools/libacpi/Makefile +2 -1
index c17f3924cc..2cc4cc585b 100644
@@ -43,7 +43,8 @@ all: $(C_SRC) $(H_SRC)
 
 $(H_SRC): $(ACPI_BUILD_DIR)/%.h: %.asl iasl
 	iasl -vs -p $(ACPI_BUILD_DIR)/$*.$(TMP_SUFFIX) -tc $<
-	sed -e 's/AmlCode/$*/g' -e 's/_aml_code//g' $(ACPI_BUILD_DIR)/$*.hex >$@
+	sed -e 's/AmlCode/$*/g' -e 's/_aml_code//g' $(ACPI_BUILD_DIR)/$*.hex >$@.$(TMP_SUFFIX)
+	mv -f $@.$(TMP_SUFFIX) $@
 	rm -f $(addprefix $(ACPI_BUILD_DIR)/, $*.aml $*.hex)
  
 $(MK_DSDT): mk_dsdt.c
#1 lore raw
Quoted 2 lines · Olaf Hering

Use a temporay file, and move it in place once done. The same pattern exists already for other dependencies.

This pattern is used when a rule consists of multiple commands having their output appended to one another's. This isn't the case here, so I'm afraid I'd like it to be made more obvious what the gain / improvement is. That is - I don't mind the change, if indeed it is for a good reason.

Jan

Quoted 20 lines
---
 tools/libacpi/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
tools/libacpi/Makefile +2 -1
index c17f3924cc..2cc4cc585b 100644
@@ -43,7 +43,8 @@ all: $(C_SRC) $(H_SRC)
 
 $(H_SRC): $(ACPI_BUILD_DIR)/%.h: %.asl iasl
 	iasl -vs -p $(ACPI_BUILD_DIR)/$*.$(TMP_SUFFIX) -tc $<
-	sed -e 's/AmlCode/$*/g' -e 's/_aml_code//g' $(ACPI_BUILD_DIR)/$*.hex >$@
+	sed -e 's/AmlCode/$*/g' -e 's/_aml_code//g' $(ACPI_BUILD_DIR)/$*.hex >$@.$(TMP_SUFFIX)
+	mv -f $@.$(TMP_SUFFIX) $@
 	rm -f $(addprefix $(ACPI_BUILD_DIR)/, $*.aml $*.hex)
 
 $(MK_DSDT): mk_dsdt.c
#2 lore raw

Am Tue, 27 Oct 2020 11:16:04 +0100
schrieb Jan Beulich <[email protected]>:

Quoted 2 lines

This pattern is used when a rule consists of multiple commands having their output appended to one another's.

My understanding is: a rule is satisfied as soon as the file exists. In this case once the invoked shell opens the file for writing. If this is the case, the change should be applied.

Olaf

#3 lore raw
Quoted 7 lines · Olaf Hering

Am Tue, 27 Oct 2020 11:16:04 +0100
schrieb Jan Beulich <[email protected]>:

Quoted 2 lines

This pattern is used when a rule consists of multiple commands having their output appended to one another's.

My understanding is: a rule is satisfied as soon as the file exists.

No - once make has found that a rule's commands need running, it'll run the full set and only check again afterwards. If there was a racing parallel make, things would be different, but such a race would need addressing elsewhere: No two possibly racing threads of the make process ought to be creating the same files. The creation of the files needs synchronizing in such a case.

Jan

#4 lore raw
Quoted 9 lines · Jan Beulich
Quoted 6 lines · Olaf Hering

Am Tue, 27 Oct 2020 11:16:04 +0100
schrieb Jan Beulich <[email protected]>:

Quoted 2 lines

This pattern is used when a rule consists of multiple commands having their output appended to one another's.

My understanding is: a rule is satisfied as soon as the file exists.

No - once make has found that a rule's commands need running, it'll run the full set and only check again afterwards.

It stops at the first command which fails.

Olaf is correct, but the problem here is an incremental build issue, not a parallel build issue.

Intermediate files must not use the name of the target, or a failure and re-build will use the (bogus) intermediate state rather than rebuilding it.

~Andrew

#5 lore raw
Quoted 11 lines · Andrew Cooper
Quoted 9 lines · Jan Beulich
Quoted 6 lines · Olaf Hering

Am Tue, 27 Oct 2020 11:16:04 +0100
schrieb Jan Beulich <[email protected]>:

Quoted 2 lines

This pattern is used when a rule consists of multiple commands having their output appended to one another's.

My understanding is: a rule is satisfied as soon as the file exists.

No - once make has found that a rule's commands need running, it'll run the full set and only check again afterwards.

Quoted 7 lines

It stops at the first command which fails.

Olaf is correct, but the problem here is an incremental build issue, not a parallel build issue.

Intermediate files must not use the name of the target, or a failure and re-build will use the (bogus) intermediate state rather than rebuilding it.

But there's no intermediate file here - the file gets created in one go. Furthermore doesn't make delete the target file(s) when a rule fails? (One may not want to rely on this, and hence indeed keep multi- part rules update intermediate files of different names.)

Jan

#6 lore raw
Quoted 17 lines · Jan Beulich
Quoted 16 lines · Andrew Cooper
Quoted 9 lines · Jan Beulich
Quoted 6 lines · Olaf Hering

Am Tue, 27 Oct 2020 11:16:04 +0100
schrieb Jan Beulich <[email protected]>:

Quoted 2 lines

This pattern is used when a rule consists of multiple commands having their output appended to one another's.

My understanding is: a rule is satisfied as soon as the file exists.

No - once make has found that a rule's commands need running, it'll run the full set and only check again afterwards.

It stops at the first command which fails.

Olaf is correct, but the problem here is an incremental build issue, not a parallel build issue.

Quoted 7 lines
Quoted 2 lines

Intermediate files must not use the name of the target, or a failure and re-build will use the (bogus) intermediate state rather than rebuilding it.

But there's no intermediate file here - the file gets created in one go. Furthermore doesn't make delete the target file(s) when a rule fails? (One may not want to rely on this, and hence indeed keep multi- part rules update intermediate files of different names.)

What if something went badly enough and sed didn't managed to write the whole file and make never had a chance to remove the bogus file?

Surely, this patch is a strict improvement to the build system.

Cheers,

Signature · 1 line
Anthony PERARD
#7 lore raw
Quoted 21 lines · Anthony PERARD
Quoted 20 lines · Jan Beulich
Quoted 18 lines · Andrew Cooper
Quoted 9 lines · Jan Beulich
Quoted 6 lines · Olaf Hering

Am Tue, 27 Oct 2020 11:16:04 +0100
schrieb Jan Beulich <[email protected]>:

Quoted 2 lines

This pattern is used when a rule consists of multiple commands having their output appended to one another's.

My understanding is: a rule is satisfied as soon as the file exists.

No - once make has found that a rule's commands need running, it'll run the full set and only check again afterwards.

It stops at the first command which fails.

Olaf is correct, but the problem here is an incremental build issue, not a parallel build issue.

Intermediate files must not use the name of the target, or a failure and re-build will use the (bogus) intermediate state rather than rebuilding it.

Quoted 7 lines
Quoted 4 lines

But there's no intermediate file here - the file gets created in one go. Furthermore doesn't make delete the target file(s) when a rule fails? (One may not want to rely on this, and hence indeed keep multi- part rules update intermediate files of different names.)

What if something went badly enough and sed didn't managed to write the whole file and make never had a chance to remove the bogus file?

How's this different from an object file getting only partly written and not deleted? We'd have to use the temporary file approach in literally every rule if we wanted to cater for this.

Jan

#8 lore raw
Quoted 27 lines · Jan Beulich
Quoted 26 lines · Anthony PERARD
Quoted 24 lines · Jan Beulich
Quoted 11 lines · Andrew Cooper
Quoted 9 lines · Jan Beulich
Quoted 6 lines · Olaf Hering

Am Tue, 27 Oct 2020 11:16:04 +0100
schrieb Jan Beulich <[email protected]>:

Quoted 2 lines

This pattern is used when a rule consists of multiple commands having their output appended to one another's.

My understanding is: a rule is satisfied as soon as the file exists.

No - once make has found that a rule's commands need running, it'll run the full set and only check again afterwards.

Quoted 7 lines

It stops at the first command which fails.

Olaf is correct, but the problem here is an incremental build issue, not a parallel build issue.

Intermediate files must not use the name of the target, or a failure and re-build will use the (bogus) intermediate state rather than rebuilding it.

But there's no intermediate file here - the file gets created in one go. Furthermore doesn't make delete the target file(s) when a rule fails? (One may not want to rely on this, and hence indeed keep multi- part rules update intermediate files of different names.)

Quoted 6 lines
Quoted 2 lines

What if something went badly enough and sed didn't managed to write the whole file and make never had a chance to remove the bogus file?

How's this different from an object file getting only partly written and not deleted? We'd have to use the temporary file approach in literally every rule if we wanted to cater for this.

I though that things like `gcc' would write the final object to a temporary place then rename it to the final destination, but that doesn't seems to be the case.

I tried to see what happens if the `sed' command fails, and the target is created, empty, and doesn't gets deleted by make. So an incremental build uses a broken file without trying to rebuild it.

If we want `make' to delete target when a rule fails, I think we need to add '.DELETE_ON_ERROR:' somewhere. Or avoid creating files before the command is likely to succeed.

Cheers,

Signature · 1 line
Anthony PERARD
#10 lore raw
In reply to Anthony PERARD
Quoted 35 lines · Anthony PERARD
Quoted 27 lines · Jan Beulich
Quoted 26 lines · Anthony PERARD
Quoted 24 lines · Jan Beulich
Quoted 11 lines · Andrew Cooper
Quoted 9 lines · Jan Beulich
Quoted 6 lines · Olaf Hering

Am Tue, 27 Oct 2020 11:16:04 +0100
schrieb Jan Beulich <[email protected]>:

Quoted 2 lines

This pattern is used when a rule consists of multiple commands having their output appended to one another's.

My understanding is: a rule is satisfied as soon as the file exists.

No - once make has found that a rule's commands need running, it'll run the full set and only check again afterwards.

Quoted 7 lines

It stops at the first command which fails.

Olaf is correct, but the problem here is an incremental build issue, not a parallel build issue.

Intermediate files must not use the name of the target, or a failure and re-build will use the (bogus) intermediate state rather than rebuilding it.

But there's no intermediate file here - the file gets created in one go. Furthermore doesn't make delete the target file(s) when a rule fails? (One may not want to rely on this, and hence indeed keep multi- part rules update intermediate files of different names.)

Quoted 6 lines
Quoted 2 lines

What if something went badly enough and sed didn't managed to write the whole file and make never had a chance to remove the bogus file?

How's this different from an object file getting only partly written and not deleted? We'd have to use the temporary file approach in literally every rule if we wanted to cater for this.

Quoted 7 lines

I though that things like `gcc' would write the final object to a temporary place then rename it to the final destination, but that doesn't seems to be the case.

I tried to see what happens if the `sed' command fails, and the target is created, empty, and doesn't gets deleted by make. So an incremental build uses a broken file without trying to rebuild it.

IOW it's rather a courtesy of the compiler / assembler / linker to delete their output files on error.

Quoted 2 lines

If we want `make' to delete target when a rule fails, I think we need to add '.DELETE_ON_ERROR:' somewhere.

Ah, indeed. I thought this was the default nowadays, but the doc says it isn't. I think this would be preferable over touching individual rules to go through temporary files.

Jan