cooker: Ensure BB_CONSOLE remains correct over server resets
PATCH 1/1 cooker: Ensure BB_CONSOLE remains correct over server resets
The console log data is written to is created at console initialisation time and does not change over reset events. This ensures the BB_CONSOLELOG value is correct over such resets by preserving it.
--- bitbake/lib/bb/cooker.py | 7 +++++++ 1 file changed, 7 insertions(+)
bitbake/lib/bb/cooker.py +7 -0
@@ -244,6 +244,11 @@ class BBCooker: self.state = state.initial self.caches_array = [] + # Need to preserve BB_CONSOLELOG over resets + consolelog = None + if hasattr(self, "data"): + consolelog = self.data.getVar("BB_CONSOLELOG", True) + if CookerFeatures.BASEDATASTORE_TRACKING in self.featureset: self.enableDataTracking()
@@ -270,6 +275,8 @@ class BBCooker: self.data = self.databuilder.data self.data_hash = self.databuilder.data_hash + if consolelog: + self.data.setVar("BB_CONSOLELOG", consolelog) # we log all events to a file if so directed if self.configuration.writeeventlog:
Thanks for these, Richard.
Quoted 4 lines · Richard Purdie
The console log data is written to is created at console initialisation time and does not change over reset events. This ensures the
BB_CONSOLELOGvalue is correct over such resets by preserving it.
I've put your three patches on top of master and ran a few builds. The result is that all of the logs for those builds are going into a single timestamped file. Before, they went to separate files. Is this intentional, or was the old behaviour incorrect? (Ideally, toaster needs a separate log file for each build.)
Thanks.
Elliot
Quoted 34 lines
Signed-off-by Richard Purdie <[email protected]>--- bitbake/lib/bb/cooker.py | 7 +++++++ 1 file changed, 7 insertions(+)bitbake/lib/bb/cooker.py +7
-0@@ -244,6 +244,11 @@ class BBCooker: self.state = state.initial self.caches_array = [] + # Need to preserve BB_CONSOLELOG over resets + consolelog = None + if hasattr(self, "data"): + consolelog = self.data.getVar("BB_CONSOLELOG", True) + if CookerFeatures.BASEDATASTORE_TRACKING in self.featureset: self.enableDataTracking()@@ -270,6 +275,8 @@ class BBCooker: self.data = self.databuilder.data self.data_hash = self.databuilder.data_hash + if consolelog: + self.data.setVar("BB_CONSOLELOG", consolelog) # we log all events to a file if so directed if self.configuration.writeeventlog:
Signature · 3 lines
Elliot Smith Software Engineer Intel Open Source Technology Centre
Quoted 10 lines · Elliot
Thanks for these, Richard.
On 11 October 2015 at 10:22, Richard Purdie <[email protected]> wrote:
The console log data is written to is created at console initialisation
time and does not change over reset events. This ensures theBB_CONSOLELOGvalue is correct over such resets by preserving it.
Quoted 6 lines
I've put your three patches on top of master and ran a few builds. The result is that all of the logs for those builds are going into a single timestamped file. Before, they went to separate files. Is this intentional, or was the old behaviour incorrect? (Ideally, toaster needs a separate log file for each build.)
It should generate one log file for each "session". A session would be one logfile for all bitbake memory resident transactions and a log file for each commandline transaction.
It is a "console log", not an individual build log.
Cheers,
Richard
Quoted 17 lines · On 16/10/2015 16:59, "[email protected] on behalf…
Quoted 10 lines · Elliot
Thanks for these, Richard.
On 11 October 2015 at 10:22, Richard Purdie <[email protected]> wrote:
The console log data is written to is created at console initialisation
time and does not change over reset events. This ensures theBB_CONSOLELOGvalue is correct over such resets by preserving it.Quoted 6 lines
I've put your three patches on top of master and ran a few builds. The result is that all of the logs for those builds are going into a single timestamped file. Before, they went to separate files. Is this intentional, or was the old behaviour incorrect? (Ideally, toaster needs a separate log file for each build.)
Quoted 5 lines
It should generate one log file for each "session". A session would be one logfile for all bitbake memory resident transactions and a log file for each commandline transaction.
It is a "console log", not an individual build log.
Mmmm, for Toaster we need an individual build log, not a console log :/
Quoted 12 lines
Cheers,
Richard
Signature · 4 lines
_______________________________________________ bitbake-devel mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/bitbake-devel
Quoted 16 lines · Belen
Quoted 11 lines · On 16/10/2015 16:59, "[email protected] on behalf…
Quoted 10 lines · Elliot
Thanks for these, Richard.
On 11 October 2015 at 10:22, Richard Purdie <[email protected]> wrote:
The console log data is written to is created at console initialisation
time and does not change over reset events. This ensures theBB_CONSOLELOGvalue is correct over such resets by preserving it.
Quoted 1 line
Quoted 1 line
Quoted 1 line
I've put your three patches on top of master and ran a few builds.
When you say "ran a few builds", was this with toaster? I'd like to double check we are seeing the right behaviour but it depends how you actually ran the builds as to whether this was the right result.
Quoted 6 lines
The
Quoted 5 lines
Quoted 5 lines
result is that all of the logs for those builds are going into a single timestamped file. Before, they went to separate files. Is this intentional, or was the old behaviour incorrect? (Ideally, toaster needs a separate log file for each build.)
Quoted 7 lines
Quoted 5 lines
It should generate one log file for each "session". A session would be one logfile for all bitbake memory resident transactions and a log file for each commandline transaction.
It is a "console log", not an individual build log.
Mmmm, for Toaster we need an individual build log, not a console log :/
Since toaster records all the log information, is it not able to compute an individual build log from that data?
The reason I'm reluctant to just add some kind of "build log" functionality to bitbake is that at the commandline level, I think it would mislead people. In theory, a "build" would be the output between BuildStarted and BuildCompleted, the question is then what happens to the output before and after these events.
Cheers,
Richard
Quoted 15 lines · Richard Purdie
Quoted 8 lines · Elliot
On 11 October 2015 at 10:22, Richard Purdie <[email protected]> wrote:
The console log data is written to is created at console initialisation
time and does not change over reset events. This ensures theBB_CONSOLELOGvalue is correct over such resets by preserving it.Quoted 6 lines
I've put your three patches on top of master and ran a few builds. The result is that all of the logs for those builds are going into a single timestamped file. Before, they went to separate files. Is this intentional, or was the old behaviour incorrect? (Ideally, toaster needs a separate log file for each build.)
Quoted 6 lines
It should generate one log file for each "session". A session would be one logfile for all bitbake memory resident transactions and a log file for each commandline transaction.
It is a "console log", not an individual build log.
Thanks for clarifying.
Will these patches be in 2.0? From the toaster point of view, it would be better if the current behaviour remained for now, even though it is broken.
We are working on a solution which will write separate build logs from within Toaster itself, which will mean we can keep our log download working when your patches do go in.
However, we may not get this finished for 2.0, and it would be good if we could keep the Toaster log download feature working in the meantime.
Thanks.
Elliot
Signature · 3 lines
Elliot Smith Software Engineer Intel Open Source Technology Centre
Quoted 6 lines · Elliot
Thanks for clarifying.
Will these patches be in 2.0? From the toaster point of view, it would be better if the current behaviour remained for now, even though it is broken.
Quoted 8 lines
We are working on a solution which will write separate build logs from within Toaster itself, which will mean we can keep our log download working when your patches do go in.
However, we may not get this finished for 2.0, and it would be good if we could keep the Toaster log download feature working in the meantime.
Given the other discussion I've seen, I'm assuming it would be ok to merge these now?
Cheers,
Richard
Quoted 11 lines · On 24/10/2015 12:26, "[email protected] on behalf…
Quoted 10 lines · Elliot
Thanks for clarifying.
Will these patches be in 2.0? From the toaster point of view, it would be better if the current behaviour remained for now, even though it is broken.
We are working on a solution which will write separate build logs from within Toaster itself, which will mean we can keep our log download working when your patches do go in.
Quoted 6 lines
Quoted 4 lines
However, we may not get this finished for 2.0, and it would be good if we could keep the Toaster log download feature working in the meantime.
Given the other discussion I've seen, I'm assuming it would be ok to merge these now?
I think it would be ok to merge them only if this
https://lists.yoctoproject.org/pipermail/toaster/2015-October/003196.html
Makes it to 2.0. Right now is still under review in the Toaster mailing list.
Elliot: correct me if I am wrong.
Cheers
Belén
Quoted 12 lines
Cheers,
Richard
Signature · 4 lines
_______________________________________________ bitbake-devel mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/bitbake-devel
Quoted 11 lines · Richard Purdie
Quoted 10 lines · Elliot
Thanks for clarifying.
Will these patches be in 2.0? From the toaster point of view, it would be better if the current behaviour remained for now, even though it is broken.
We are working on a solution which will write separate build logs from within Toaster itself, which will mean we can keep our log download working when your patches do go in.
Quoted 7 lines
Quoted 4 lines
However, we may not get this finished for 2.0, and it would be good if we could keep the Toaster log download feature working in the meantime.
Given the other discussion I've seen, I'm assuming it would be ok to merge these now?
Yes, as we're now writing our own logs for Toaster.
Thanks for merging.
Elliot
Quoted 7 lines
Cheers,
Richard
Signature · 3 lines
Elliot Smith Software Engineer Intel Open Source Technology Centre
Quoted 12 lines · Elliot
Quoted 4 lines · Richard Purdie
On Mon, 2015-10-19 at 13:27 +0100, Smith, Elliot wrote:
Given the other discussion I've seen, I'm assuming it would be ok to merge these now?Yes, as we're now writing our own logs for Toaster.
Thanks for merging.
Richard, apologies, I just realised we have one patch outstanding:
https://lists.yoctoproject.org/pipermail/toaster/2015-October/003196.html
(This is the patch which actually does the work on the Toaster side.)
I thought this had been reviewed and submitted to bitbake-devel, but it still hasn't. I will try to get this done today.
If we can get this in, as well as your patches to how bitbake sets BB_CONSOLELOG, we won't lose any functionality in Toaster.
Thanks.
Elliot
Quoted 17 lines
Elliot
Quoted 7 lines
Cheers,
Richard
--
Elliot Smith
Software Engineer
Intel Open Source Technology Centre
Signature · 3 lines
Elliot Smith Software Engineer Intel Open Source Technology Centre