readd build dependency for startup_raw.S
PATCH 1/1 readd build dependency for startup_raw.S
startup_raw.S includes a generated file, and used to have a dependency in the Makefile to make sure the generated file exists prior compilation. This dependency was removed, and as a result building startup_raw.S randomly fails:
../../20230814T181249.4fdcb339b/grub-core/boot/i386/pc/startup_raw.S:129:10: fatal error: rs_decoder.h: No such file or directory
129 | #include <rs_decoder.h>
| ^~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:34654: boot/i386/pc/lzma_decompress_image-startup_raw.o] Error 1
Fixes cc2841129652 ("Fix make dist on non-pc.")
---
The referenced commit removed also a dependency for mips. I can not test mips, so I left mips as it is.
grub-core/Makefile.am | 2 ++ 1 file changed, 2 insertions(+)
grub-core/Makefile.am +2 -0
@@ -62,6 +62,8 @@ grub_script.yy.c: grub_script.yy.h rs_decoder.h: $(srcdir)/lib/reed_solomon.c $(TARGET_CC) $(TARGET_CPPFLAGS) $(TARGET_CFLAGS) -Os -I$(top_builddir) -S -DSTANDALONE -o $@ $< -g0 -mregparm=3 -ffreestanding +boot/i386/pc/startup_raw.S: $(builddir)/rs_decoder.h + CLEANFILES += grub_script.yy.c grub_script.yy.h include $(srcdir)/Makefile.core.am
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
Hi Olaf,
There's an extra 'd' in read in the subject line.
Quoted 11 lines · On Wed, 16 Aug 2023 11:55:01 +0200 Olaf Hering <[email protected]> wrote:
startup_raw.Sincludes a generated file, and used to have a dependency in theMakefileto make sure the generated file exists prior compilation. This dependency was removed, and as a result buildingstartup_raw.Srandomly fails:
../../20230814T181249.4fdcb339b/grub-core/boot/i386/pc/startup_raw.S:129:10: fatal error:rs_decoder.h: No such file or directory
129 | #include <rs_decoder.h>| ^~~~~~~~~~~~~~compilation terminated.
make: *** [Makefile:34654:boot/i386/pc/lzma_decompress_image-startup_raw.o] Error 1
I don't see this error when building any of the targets that I test. This includes i386-pc, which I'm assuming this error comes from (if not which?). Are you running make if -jN where N > 1? Does rs_decoder.h exist in the fail build directory? (its a generated file) What compiler and version are you using?
When I look at the contents of ./grub-core/boot/i386/pc/.deps-core/lzma_decompress_image-startup_raw.Po in a successful build directory, I see that GCC figured out that rs_decoder.h is a dependency. So I'm not seeing yet why your build is failing, but the below change *should* be unnecessary.
Glenn
Quoted 27 lines
Fixes
cc2841129652("Fix make dist on non-pc.")Signed-off-by Olaf Hering <[email protected]>---
The referenced commit removed also a dependency for mips. I can not test mips, so I left mips as it is.
grub-core/Makefile.am | 2 ++ 1 file changed, 2 insertions(+)grub-core/Makefile.am +2
-0@@ -62,6 +62,8 @@ grub_script.yy.c: grub_script.yy.h rs_decoder.h: $(srcdir)/lib/reed_solomon.c $(TARGET_CC) $(TARGET_CPPFLAGS) $(TARGET_CFLAGS) -Os -I$(top_builddir) -S -DSTANDALONE -o $@ $< -g0 -mregparm=3 -ffreestanding +boot/i386/pc/startup_raw.S: $(builddir)/rs_decoder.h + CLEANFILES += grub_script.yy.c grub_script.yy.h include $(srcdir)/Makefile.core.am
Quoted 4 lines
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
Quoted 20 lines · Glenn Washburn
Hi Olaf,
There's an extra 'd' in read in the subject line.
Quoted 11 lines · On Wed, 16 Aug 2023 11:55:01 +0200 Olaf Hering <[email protected]> wrote:
startup_raw.Sincludes a generated file, and used to have a dependency in theMakefileto make sure the generated file exists prior compilation. This dependency was removed, and as a result buildingstartup_raw.Srandomly fails:
../../20230814T181249.4fdcb339b/grub-core/boot/i386/pc/startup_raw.S:129:10: fatal error:rs_decoder.h: No such file or directory
129 | #include <rs_decoder.h>| ^~~~~~~~~~~~~~compilation terminated.
make: *** [Makefile:34654:boot/i386/pc/lzma_decompress_image-startup_raw.o] Error 1
Quoted 5 lines
I don't see this error when building any of the targets that I test. This includes i386-pc, which I'm assuming this error comes from (if not which?). Are you running make if -jN where N > 1? Does
rs_decoder.hexist in the fail build directory? (its a generated file) What compiler and version are you using?
This is almost certainly a race condition with parallel make jobs. It might also be possible to trigger it with make --shuffle so that the build order of targets is randomized.
Quoted 5 lines
When I look at the contents of ./grub-core/boot/i386/pc/.deps-core/lzma_decompress_image-startup_raw.Po in a successful build directory, I see that GCC figured out that
rs_decoder.his a dependency. So I'm not seeing yet why your build is failing, but the below change *should* be unnecessary.
I think the .Po files are relevant only for incremental builds -- they let the build system detect which files need to be updated when headers are modified.
For an initial clean build, the .Po files will not exist. Any dependency on generated source files would need to be declared explicitly.
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
Quoted 28 lines · On Wed, 16 Aug 2023 21:43:25 -0400 Mike Gilbert <[email protected]> wrote:
Quoted 20 lines · Glenn Washburn
Hi Olaf,
There's an extra 'd' in read in the subject line.
Quoted 11 lines · On Wed, 16 Aug 2023 11:55:01 +0200 Olaf Hering <[email protected]> wrote:
startup_raw.Sincludes a generated file, and used to have a dependency in theMakefileto make sure the generated file exists prior compilation. This dependency was removed, and as a result buildingstartup_raw.Srandomly fails:
../../20230814T181249.4fdcb339b/grub-core/boot/i386/pc/startup_raw.S:129:10: fatal error:rs_decoder.h: No such file or directory
129 | #include <rs_decoder.h>| ^~~~~~~~~~~~~~compilation terminated.
make: *** [Makefile:34654:boot/i386/pc/lzma_decompress_image-startup_raw.o] Error 1Quoted 5 lines
I don't see this error when building any of the targets that I test. This includes i386-pc, which I'm assuming this error comes from (if not which?). Are you running make if -jN where N > 1? Does
rs_decoder.hexist in the fail build directory? (its a generated file) What compiler and version are you using?
Quoted 3 lines
This is almost certainly a race condition with parallel make jobs. It might also be possible to trigger it with make --shuffle so that the build order of targets is randomized.
Looks like I'm on v4.3 and this was introduced in v4.4. Perhaps you can test this? It certainly did sound like a race condition, which is why I was curious about the level of parallelism used. However, it appears as though the default -j1 was used. The two machines I test on use -j9 and -j25 and I've not seen this yet in the hundreds of runs I've done. So I'm skeptical of this. All my testing is done one Debian 11.
Quoted 6 lines
Quoted 5 lines
When I look at the contents of ./grub-core/boot/i386/pc/.deps-core/lzma_decompress_image-startup_raw.Po in a successful build directory, I see that GCC figured out that
rs_decoder.his a dependency. So I'm not seeing yet why your build is failing, but the below change *should* be unnecessary.
Quoted 7 lines
I think the .Po files are relevant only for incremental builds -- they let the build system detect which files need to be updated when headers are modified.
For an initial clean build, the .Po files will not exist. Any dependency on generated source files would need to be declared explicitly.
Yes, I now believe you're right, though its not 100% clear why to me. The .Po files are included via the include make directive, and so I would think make should fail if they do not exist. But it does seem true that the .Po files are only generated after compilation. So perhaps these rules are being included after compilation, which is why a clean build wouldn't see them but also why make doesn't fail[1].
What I'm seeing is that the generated grub-core/Makefile has rs_decoder.h in BUILT_SOURCES (via several levels of variable expansions). This should mean that rs_decoder.h is generated before any sources are compiled[2]. So it looks good to me. I wonder if Olaf has an old version of automake that has a bug in it?
[1] https://www.gnu.org/software/make/manual/html_node/Include.html
[2] https://www.gnu.org/software/automake/manual/html_node/Sources.html
Quoted 5 lines
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
Dear Glenn,
Am 16.08.23 um 21:33 schrieb Glenn Washburn:
Quoted 1 line
There's an extra 'd' in read in the subject line.
I think it’s supposed to be “re-add”, that means to add it back.
Kind regards,
Paul
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
Wed, 16 Aug 2023 14:33:43 -0500 Glenn Washburn <[email protected]>:
Quoted 1 line
There's an extra 'd' in read in the subject line.
Perhaps there is a better phrase to describe the act of adding something back?
Quoted 1 line
I don't see this error when building any of the targets that I test.
It is the result of my attempt to build just the required binaries, without docs. I came up with this:
git clean -dffx
bash bootstrap
mkdir .b
cd .b
../configure [options]
make check
make -C grub-core install-platformDATA install-platformPROGRAMS
But thinking further about it, and as others have noted, the real fix would be to wade through all targets and express their dependencies properly.
So, please disregard this change.
Olaf
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
Quoted 5 lines · On Thu, 17 Aug 2023 11:00:44 +0200 Olaf Hering <[email protected]> wrote:
Wed, 16 Aug 2023 14:33:43 -0500 Glenn Washburn <[email protected]>:
Quoted 1 line
There's an extra 'd' in read in the subject line.
Perhaps there is a better phrase to describe the act of adding something back?
Ah, it needs a dash, "re-add". I misinterpreted your intent.
Quoted 11 lines
Quoted 1 line
I don't see this error when building any of the targets that I test.
It is the result of my attempt to build just the required binaries, without docs. I came up with this:
git clean -dffx
bash bootstrap
mkdir .b
cd .b
../configure [options]
make check
make -C grub-core install-platformDATA install-platformPROGRAMS
And this avoids the compile error for startup_raw.S?
Quoted 3 lines
But thinking further about it, and as others have noted, the real fix would be to wade through all targets and express their dependencies properly.
What version of automake are you using? I'm on 1.16.3-2 from Debian 11. rs_decoder.h is ending up in BUILT_SOURCES, so it should get built before anything is compiled. Can you check BUILT_SOURCES? For me rs_decoder.h is included via a few levels of variable expansion, so it won't be literally in the definition of BUILT_SOURCES. I'm wondering if maybe you're using an older version of automake with a bug.
Glenn
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
Quoted 6 lines · Glenn Washburn
What version of automake are you using? I'm on 1.16.3-2 from Debian 11.
rs_decoder.his ending up inBUILT_SOURCES, so it should get built before anything is compiled. Can you checkBUILT_SOURCES? For mers_decoder.his included via a few levels of variable expansion, so it won't be literally in the definition ofBUILT_SOURCES. I'm wondering if maybe you're using an older version of automake with a bug.
Ah, I forgot about BUILT_SOURCES. That should cover it indeed.
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
Thu, 17 Aug 2023 15:21:44 -0500 Glenn Washburn <[email protected]>:
Quoted 11 lines
Quoted 9 lines · On Thu, 17 Aug 2023 11:00:44 +0200 Olaf Hering <[email protected]> wrote:
It is the result of my attempt to build just the required binaries, without docs. I came up with this:
git clean -dffx
bash bootstrap
mkdir .b
cd .b
../configure [options]
make check
make -C grub-core install-platformDATA install-platformPROGRAMS
Quoted 1 line
And this avoids the compile error for
startup_raw.S?
No, the other way around: this triggers it. Previously I had a simple 'make', followed by the install targets shown above. This builds all dependencies, and also the doc target. To get rid of the doc target, I was under the assumption that install-something already has proper dependencies, but only a subset is built. gnulib.a was missing, and likely others. I did not dig deep.
Quoted 9 lines
Quoted 2 lines
But thinking further about it, and as others have noted, the real fix would be to wade through all targets and express their dependencies properly.
What version of automake are you using? I'm on 1.16.3-2 from Debian 11.
rs_decoder.his ending up inBUILT_SOURCES, so it should get built before anything is compiled. Can you check ,? For mers_decoder.his included via a few levels of variable expansion, so it won't be literally in the definition ofBUILT_SOURCES. I'm wondering if maybe you're using an older version of automake with a bug.
I have seen it with automake 1.15.1 and 1.16.5. "nodist_lzma_decompress_image_SOURCES" should be part of "BUILT_SOURCES", via "am__append_6232". But another local build, without this change, does indeed fail. The 'check' target does have "BUILT_SOURCES" as dependency, not sure why it does not build the target.
With some debug, the content during 'make check' is just:BUILT_SOURCES grub_fstest_init.c grub_script.tab.c grub_script.tab.h grub_script.yy.c grub_script.yy.h libgrub_a_init.c
And the real reason is an incorrect invocation of make:
'make check' works in the toplevel, while 'make -C grub-core install-something' works in the given directory. So for me the fix (to avoid the docs target) is:
make check
make -C grub-core check
make -C grub-core install-platformDATA install-platformPROGRAMS
Sorry for the noise.
Olaf
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel
Quoted 40 lines · On Fri, 18 Aug 2023 16:33:19 +0200 Olaf Hering <[email protected]> wrote:
Thu, 17 Aug 2023 15:21:44 -0500 Glenn Washburn <[email protected]>:
Quoted 11 lines
Quoted 9 lines · On Thu, 17 Aug 2023 11:00:44 +0200 Olaf Hering <[email protected]> wrote:
It is the result of my attempt to build just the required binaries, without docs. I came up with this:
git clean -dffx
bash bootstrap
mkdir .b
cd .b
../configure [options]
make check
make -C grub-core install-platformDATA install-platformPROGRAMSQuoted 1 line
And this avoids the compile error for
startup_raw.S?No, the other way around: this triggers it. Previously I had a simple 'make', followed by the install targets shown above. This builds all dependencies, and also the doc target. To get rid of the doc target, I was under the assumption that install-something already has proper dependencies, but only a subset is built. gnulib.a was missing, and likely others. I did not dig deep.
Quoted 9 lines
Quoted 2 lines
But thinking further about it, and as others have noted, the real fix would be to wade through all targets and express their dependencies properly.
What version of automake are you using? I'm on 1.16.3-2 from Debian 11.
rs_decoder.his ending up inBUILT_SOURCES, so it should get built before anything is compiled. Can you check ,? For mers_decoder.his included via a few levels of variable expansion, so it won't be literally in the definition ofBUILT_SOURCES. I'm wondering if maybe you're using an older version of automake with a bug.I have seen it with automake 1.15.1 and 1.16.5. "
nodist_lzma_decompress_image_SOURCES" should be part of "BUILT_SOURCES", via "am__append_6232". But another local build, without this change, does indeed fail. The 'check' target does have "BUILT_SOURCES" as dependency, not sure why it does not build the target.With some debug, the content during 'make check' is just:
BUILT_SOURCESgrub_fstest_init.cgrub_script.tab.cgrub_script.tab.hgrub_script.yy.cgrub_script.yy.hlibgrub_a_init.c
Quoted 8 lines
And the real reason is an incorrect invocation of make:
'make check' works in the toplevel, while 'make -C grub-core install-something' works in the given directory. So for me the fix (to avoid the docs target) is:
make check
make -C grub-core check
make -C grub-core install-platformDATA install-platformPROGRAMSSorry for the noise.
Ok, I see now the issue. "A source file listed in BUILT_SOURCES is made when ‘make all’, ‘make check’, ‘make install’, ‘make install-exec’ (or make dist) is run, before other targets are processed"[1]. This is why the previous was failing and this one now works (ie. make check uses BUILT_SOURCES while the other targets do not).
Glenn
[1] https://www.gnu.org/software/automake/manual/html_node/Sources.html
_______________________________________________
Grub-devel mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/grub-devel