bison: Reset load average settings if specified in environment
PATCH 1/1 bison: Reset load average settings if specified in environment
In some cases, we run into parallel build failures where BUILT_SOURCES is skipped, as a result required header files are not generated and the build fails with missing header errors like
../bison-3.5.2/lib/uniwidth/width.c:21:10: fatal error: uniwidth.h: No such file or directory
#include "uniwidth.h"
^~~~~~~~~~~~compilation terminated.
BUILT_SOURCES should be built automatically with make all [1] therefore ensure that make is invoked with all target
bison-native parallel build fails when -l<n> is passed globally from build environment, errors like below due to race starts to show up
Therefore removes a previous load limit if set
--- meta/recipes-devtools/bison/bison_3.5.3.bb | 6 ++++++ 1 file changed, 6 insertions(+)
meta/recipes-devtools/bison/bison_3.5.3.bb +6 -0
@@ -18,6 +18,12 @@ SRC_URI[sha256sum] = "2bf85b5f88a5f2fa8069aed2a2dfc3a9f8d15a97e59c713e3906e5fdd9 # No point in hardcoding path to m4, just use PATH EXTRA_OECONF += "M4=m4" +# Reset any loadavg set via environment, it breaks parallel build +# | ../bison-3.5.2/lib/uniwidth/width.c:21:10: fatal error: uniwidth.h: No such file or directory +# | #include "uniwidth.h" +# | ^~~~~~~~~~~~ +EXTRA_OEMAKE_append = " -l" + inherit autotools gettext texinfo # The automatic m4 path detection gets confused, so force the right value
Signature · 1 line
2.25.1
✗ patchtest: failure for bison: Reset load average settings if specified in environment
== Series Details ==
Series: bison: Reset load average settings if specified in environment
Revision: 1
URL : https://patchwork.openembedded.org/series/23244/
State : failure
== Summary ==
Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:
* Issue Series does not apply on top of target branch [test_series_merge_on_head]
Suggested fix Rebase your series on top of targeted branch
Targeted branch master (currently at b1ae2178fa)
If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).
---
Guidelines: https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
Shouldn’t the race be fixed? This might still happen because -j still parallelises the build,no?
Alex
Quoted 55 lines · Khem Raj
In some cases, we run into parallel build failures where
BUILT_SOURCESis skipped, as a result required header files are not generated and the build fails with missing header errors like
../bison-3.5.2/lib/uniwidth/width.c:21:10: fatal error:uniwidth.h: No such file or directory#include "uniwidth.h" ^~~~~~~~~~~~compilation terminated.
BUILT_SOURCESshould be built automatically withmake all[1] therefore ensure that make is invoked withalltargetbison-native parallel build fails when -l<n> is passed globally from build environment, errors like below due to race starts to show up
Therefore removes a previous load limit if set
Signed-off-by Khem Raj <[email protected]>--- meta/recipes-devtools/bison/bison_3.5.3.bb | 6 ++++++ 1 file changed, 6 insertions(+)meta/recipes-devtools/bison/bison_3.5.3.bb +0
-0@@ -18,6 +18,12 @@ SRC_URI[sha256sum] ="
2bf85b5f88a5f2fa8069aed2a2dfc3a9f8d15a97e59c713e3906e5fdd9
# No point in hardcoding path to m4, just use PATHEXTRA_OECONF+= "M4=m4"meta/recipes-devtools/bison/bison_3.5.3.bb +2
-0+# Reset any loadavg set via environment, it breaks parallel build +# | ../bison-3.5.2/lib/uniwidth/width.c:21:10: fatal error: uniwidth.h:No such file or directory
meta/recipes-devtools/bison/bison_3.5.3.bb +4
-1+# | #include "uniwidth.h" +# | ^~~~~~~~~~~~ +EXTRA_OEMAKE_append = " -l" + inherit autotools gettext texinfo # The automatic m4 path detection gets confused, so force the right value --2.25.1
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Quoted 3 lines · Alexander Kanavin
Shouldn’t the race be fixed? This might still happen because -j still parallelises the build,no?
It perhaps is a make bug which only triggers with -j and -l being both present it’s perhaps easily reproducible using these together on bison builds
As such I don’t see anything wrong with bison makery
Quoted 60 lines
Alex
Quoted 55 lines · Khem Raj
In some cases, we run into parallel build failures where
BUILT_SOURCESis skipped, as a result required header files are not generated and the build fails with missing header errors like
../bison-3.5.2/lib/uniwidth/width.c:21:10: fatal error:uniwidth.h: No such file or directory#include "uniwidth.h" ^~~~~~~~~~~~compilation terminated.
BUILT_SOURCESshould be built automatically withmake all[1] therefore ensure that make is invoked withalltargetbison-native parallel build fails when -l<n> is passed globally from build environment, errors like below due to race starts to show up
Therefore removes a previous load limit if set
Signed-off-by Khem Raj <[email protected]>--- meta/recipes-devtools/bison/bison_3.5.3.bb | 6 ++++++ 1 file changed, 6 insertions(+)meta/recipes-devtools/bison/bison_3.5.3.bb +0
-0@@ -18,6 +18,12 @@ SRC_URI[sha256sum] ="
2bf85b5f88a5f2fa8069aed2a2dfc3a9f8d15a97e59c713e3906e5fdd9
# No point in hardcoding path to m4, just use PATHEXTRA_OECONF+= "M4=m4"meta/recipes-devtools/bison/bison_3.5.3.bb +2
-0+# Reset any loadavg set via environment, it breaks parallel build +# | ../bison-3.5.2/lib/uniwidth/width.c:21:10: fatal error: uniwidth.h:No such file or directory
meta/recipes-devtools/bison/bison_3.5.3.bb +4
-1+# | #include "uniwidth.h" +# | ^~~~~~~~~~~~ +EXTRA_OEMAKE_append = " -l" + inherit autotools gettext texinfo # The automatic m4 path detection gets confused, so force the right value --2.25.1
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Quoted 10 lines · Khem Raj
Quoted 2 lines · Alexander Kanavin
Shouldn’t the race be fixed? This might still happen because -j still parallelises the build,no?
It perhaps is a make bug which only triggers with -j and -l being both present it’s perhaps easily reproducible using these together on bison builds
As such I don’t see anything wrong with bison makery
This is the second bison-native problem in addition to
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13825
It is possible that the gnulib issue was a red herring and it is actually is a parallel build issue, but both being in bison-native does point more towards a problem specific to bison.
cu
Adrian
Quoted 16 lines · Khem Raj
In some cases, we run into parallel build failures where
BUILT_SOURCES
is skipped, as a result required header files are not generated and the
build fails with missing header errors like
../bison-3.5.2/lib/uniwidth/width.c:21:10: fatal error:uniwidth.h:
No such file or directory#include "uniwidth.h" ^~~~~~~~~~~~compilation terminated.
BUILT_SOURCESshould be built automatically withmake all[1] therefore
ensure that make is invoked withalltarget
Quoted 7 lines
bison-native parallel build fails when -l<n> is passed globally from build environment, errors like below due to race starts to show up
Therefore removes a previous load limit if set
I'm not sure I understand this. We've seen a failure on our ubuntu1604 performance tester which I think is the same bug as this. The environment on that machine doesn't change, so how/where is this -l option coming from?
I suspect there is some issue we're not quite getting to still :(
Cheers,
Richard
Quoted 23 lines · Richard Purdie
Quoted 21 lines · Khem Raj
In some cases, we run into parallel build failures where
BUILT_SOURCES
is skipped, as a result required header files are not generated and the
build fails with missing header errors like
../bison-3.5.2/lib/uniwidth/width.c:21:10: fatal error:uniwidth.h:
No such file or directory#include "uniwidth.h" ^~~~~~~~~~~~compilation terminated.
BUILT_SOURCESshould be built automatically withmake all[1] therefore
ensure that make is invoked withalltargetbison-native parallel build fails when -l<n> is passed globally from build environment, errors like below due to race starts to show up
Therefore removes a previous load limit if set
Quoted 8 lines
Quoted 2 lines
I'm not sure I understand this. We've seen a failure on our ubuntu1604 performance tester which I think is the same bug as this. The environment on that machine doesn't change, so how/where is this -l option coming from?
From custom distro global settings for EXTRA_OEMAKE
Quoted 2 lines
I suspect there is some issue we're not quite getting to still :(
its well reproducible now with -j and -l together on our builders, From what
I see in makefiles bison is doing right thing in using BUILT_SOURCES so I suspect it could be make acting in a certain way when those options are present.
our custom settings use -l options globally in EXTRA_OEMAKE this is to let machines with certain configurations build when machines are shared, we don't want a given build to use all resources on box. Yes one could say why not do something else to limit the resources build uses but this is a global setting that works well
This patch just ensures that any -l setting passed from environment is cleaned up and reset. So for general case it will be a noop.
Quoted 4 lines
Cheers,
Richard
Quoted 42 lines · Khem Raj
Quoted 22 lines · Richard Purdie
Quoted 21 lines · Khem Raj
In some cases, we run into parallel build failures where
BUILT_SOURCES
is skipped, as a result required header files are not generated and the
build fails with missing header errors like
../bison-3.5.2/lib/uniwidth/width.c:21:10: fatal error:uniwidth.h:
No such file or directory#include "uniwidth.h" ^~~~~~~~~~~~compilation terminated.
BUILT_SOURCESshould be built automatically withmake all[1] therefore
ensure that make is invoked withalltargetbison-native parallel build fails when -l<n> is passed globally from build environment, errors like below due to race starts to show up
Therefore removes a previous load limit if set
Quoted 8 lines
Quoted 2 lines
I'm not sure I understand this. We've seen a failure on our ubuntu1604 performance tester which I think is the same bug as this. The environment on that machine doesn't change, so how/where is this -l option coming from?
From custom distro global settings for
EXTRA_OEMAKEQuoted 2 lines
I suspect there is some issue we're not quite getting to still :(
its well reproducible now with -j and -l together on our builders, From what
I see in makefiles bison is doing right thing in usingBUILT_SOURCESso I suspect it could be make acting in a certain way when those options are present.
Quoted 8 lines
our custom settings use -l options globally in
EXTRA_OEMAKEthis is to let machines with certain configurations build when machines are shared, we don't want a given build to use all resources on box. Yes one could say why not do something else to limit the resources build uses but this is a global setting that works wellThis patch just ensures that any -l setting passed from environment is cleaned up and reset. So for general case it will be a noop.
Ok, that explains your failure but not the one on our performance testing worker :/.
Cheers,
Richard
Quoted 52 lines · Richard Purdie
Quoted 19 lines · Khem Raj
Quoted 17 lines · Richard Purdie
Quoted 16 lines · Khem Raj
In some cases, we run into parallel build failures where
BUILT_SOURCES
is skipped, as a result required header files are not generated and the
build fails with missing header errors like
../bison-3.5.2/lib/uniwidth/width.c:21:10: fatal error:uniwidth.h:
No such file or directory#include "uniwidth.h" ^~~~~~~~~~~~compilation terminated.
BUILT_SOURCESshould be built automatically withmake all[1] therefore
ensure that make is invoked withalltargetQuoted 7 lines
Quoted 7 lines
Quoted 7 lines
bison-native parallel build fails when -l<n> is passed globally from build environment, errors like below due to race starts to show up
Therefore removes a previous load limit if set
[1]
Quoted 7 lines
Quoted 6 lines
I'm not sure I understand this. We've seen a failure on our ubuntu1604 performance tester which I think is the same bug as this. The environment on that machine doesn't change, so how/where is this -l option coming from?
Quoted 5 lines
From custom distro global settings for
EXTRA_OEMAKEQuoted 2 lines
I suspect there is some issue we're not quite getting to still :(
its well reproducible now with -j and -l together on our builders, From
what
Quoted 6 lines
I see in makefiles bison is doing right thing in using
BUILT_SOURCESso I suspect it could be make acting in a certain way when those options are present.our custom settings use -l options globally in
EXTRA_OEMAKEthis is to let machines with certain configurations build when machinesare
Quoted 4 lines
shared, we don't want a given build to use all resources on box. Yes one could say why not do something else to limit the resources build uses but this is a global setting that works well
Quoted 5 lines
Quoted 2 lines
This patch just ensures that any -l setting passed from environment is cleaned up and reset. So for general case it will be a noop.
Ok, that explains your failure but not the one on our performance testing worker :/.
What errors do you see on perf testing worker is it exactly same ?
Quoted 7 lines
Cheers,
Richard
Quoted 49 lines · Khem Raj
Quoted 48 lines · Richard Purdie
Quoted 47 lines · Khem Raj
Quoted 22 lines · Richard Purdie
Quoted 21 lines · Khem Raj
In some cases, we run into parallel build failures where
BUILT_SOURCES
is skipped, as a result required header files are not generated and the
build fails with missing header errors like
../bison-3.5.2/lib/uniwidth/width.c:21:10: fatal error:uniwidth.h:
No such file or directory#include "uniwidth.h" ^~~~~~~~~~~~compilation terminated.
BUILT_SOURCESshould be built automatically withmake all[1] therefore
ensure that make is invoked withalltargetbison-native parallel build fails when -l<n> is passed globally from build environment, errors like below due to race starts to show up
Therefore removes a previous load limit if set
Quoted 7 lines
Quoted 2 lines
I'm not sure I understand this. We've seen a failure on our ubuntu1604 performance tester which I think is the same bug as this. The environment on that machine doesn't change, so how/where is this -l option coming from?
From custom distro global settings for
EXTRA_OEMAKEQuoted 2 lines
I suspect there is some issue we're not quite getting to still :(
its well reproducible now with -j and -l together on our builders, From what
I see in makefiles bison is doing right thing in usingBUILT_SOURCESso I suspect it could be make acting in a certain way when those options are present.our custom settings use -l options globally in
EXTRA_OEMAKEthis is to let machines with certain configurations build when machines are shared, we don't want a given build to use all resources on box. Yes one could say why not do something else to limit the resources build uses but this is a global setting that works well
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
This patch just ensures that any -l setting passed from environment is cleaned up and reset. So for general case it will be a noop.
Ok, that explains your failure but not the one on our performance testing worker :/.
I've heard reports of this still being an issue with latest OE 3.1 when -j is set high enough, without any -l options in EXTRA_OEMAKE (so Khem's workaround to over-ride -l doesn't help).
Disabling PARALLEL_MAKE completely might be the best workaround for now until we understand the real root cause.