Skip to contentSkip to conversation

cmake: Displace VERBOSE setting

PATCH In review Khem Raj 3 participants 5 messages Dec 18, 2019 – Dec 19, 2019

lore
Share & tools
lore

Khem Raj

author
#0 lore raw

PATCH 1/1 cmake: Displace VERBOSE setting

Setting VERBOSE always, while is fine on one hand for debugging, its coming at an expense of creating lots and lots of logs, e.g. qtwebkit compile logs alone with VERBOSE is 163MB, there are many other large packages which use cmake e.g. WPE, webkitgtk etc which are in same range with out this option on, the logs reduce to 861K and also speeds up build a notch

If user needs to enable this logs for targetted debugging debugging that could be added via

EXTRA_OECMAKE_BUILD += "--verbose"

in recipe

---
 meta/classes/cmake.bbclass | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
meta/classes/cmake.bbclass +7 -2
index 8ccb1eefc7..8a44a0c216 100644
@@ -181,9 +181,14 @@ cmake_do_configure() {
 	  -Wno-dev
 }
 
+# To enable VERBOSE option to emit all kind of logs for a given recipe add following in it
+#
+# EXTRA_OECMAKE_BUILD += "--verbose"
+#
+# Then run do_compile again
 cmake_runcmake_build() {
-	bbnote ${DESTDIR:+DESTDIR=${DESTDIR} }VERBOSE=1 cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD}
-	eval ${DESTDIR:+DESTDIR=${DESTDIR} }VERBOSE=1 cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD}
+	bbnote ${DESTDIR:+DESTDIR=${DESTDIR} } cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD}
+	eval ${DESTDIR:+DESTDIR=${DESTDIR} } cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD}
 }
 
 cmake_do_compile()  {
Signature · 1 line
2.24.1
#1 lore raw

Hi Khem,

Quoted 10 lines · Khem Raj

Setting VERBOSE always, while is fine on one hand for debugging, its coming at an expense of creating lots and lots of logs, e.g. qtwebkit compile logs alone with VERBOSE is 163MB, there are many other large packages which use cmake e.g. WPE, webkitgtk etc which are in same range with out this option on, the logs reduce to 861K and also speeds up build a notch

If user needs to enable this logs for targetted debugging debugging that could be added via

Quoted 6 lines

EXTRA_OECMAKE_BUILD += "--verbose"

in recipe

I don't agree with the reasoning to always disable this by default. - it's way too useful in general. I can't count the number of times it was enough to look at the log.do_compile to figure out something is wrong.

Now you have to recompile everything with this merged, and something that took 5 minutes to debug becomes an arduous task.

If the Webkit build is too verbose, you should make that specific recipe less verbose. E.g. you could always set
EXTRA_OECMAKE_BUILD, and just clear
it out in the webkit recipe.

Otherwise, based on the same reasoning, you should disable all logs, not just cmake logs: kernel, make output in general, bitbake logs. They all take up space.

Seriously, those logs are useful, keeping in mind that often logs are collected on build-machines, and having easy access is a huge time-saver.

Cheers,
Andre'

Quoted 28 lines
---
 meta/classes/cmake.bbclass | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
meta/classes/cmake.bbclass +7 -2
index 8ccb1eefc7..8a44a0c216 100644
@@ -181,9 +181,14 @@ cmake_do_configure() {
 	  -Wno-dev
 }
 
+# To enable VERBOSE option to emit all kind of logs for a given recipe add following in it
+#
+# EXTRA_OECMAKE_BUILD += "--verbose"
+#
+# Then run do_compile again
 cmake_runcmake_build() {
-	bbnote ${DESTDIR:+DESTDIR=${DESTDIR} }VERBOSE=1 cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD}
-	eval ${DESTDIR:+DESTDIR=${DESTDIR} }VERBOSE=1 cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD}
+	bbnote ${DESTDIR:+DESTDIR=${DESTDIR} } cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD}
+	eval ${DESTDIR:+DESTDIR=${DESTDIR} } cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD}
 }
 
 cmake_do_compile()  {
Signature · 1 line
2.24.1
#2 lore raw
Quoted 32 lines · André Draszik

Hi Khem,

Quoted 10 lines · Khem Raj

Setting VERBOSE always, while is fine on one hand for debugging, its coming at an expense of creating lots and lots of logs, e.g. qtwebkit compile logs alone with VERBOSE is 163MB, there are many other large packages which use cmake e.g. WPE, webkitgtk etc which are in same range with out this option on, the logs reduce to 861K and also speeds up build a notch

If user needs to enable this logs for targetted debugging debugging that could be added via

Quoted 6 lines

EXTRA_OECMAKE_BUILD += "--verbose"

in recipe

I don't agree with the reasoning to always disable this by default. - it's way too useful in general. I can't count the number of times it was enough to look at the log.do_compile to figure out something is wrong.

Now you have to recompile everything with this merged, and something that took 5 minutes to debug becomes an arduous task.

If the Webkit build is too verbose, you should make that specific recipe less verbose. E.g. you could always set
EXTRA_OECMAKE_BUILD, and just clear
it out in the webkit recipe.

Quoted 5 lines

Otherwise, based on the same reasoning, you should disable all logs, not just cmake logs: kernel, make output in general, bitbake logs. They all take up space.

Seriously, those logs are useful, keeping in mind that often logs are collected on build-machines, and having easy access is a huge time-saver.

Personally I agree with André. We have verbose logs for autotools and meson, so why is cmake special. As this is a webkit-specific issue for you, the neater fix would be to override this in webkit.

Stepping back though, why are 163MB log files a problem? Are you archiving every log maybe? If so then there's an argument to be made for a global "logs should be verbose" toggle, that defaults to on but you can set to off. That way the kernel, autotools, etc can also follow the same logic and *all* logs can be smaller.

Ross

author
#3 lore raw
Quoted 53 lines · Ross Burton
Quoted 47 lines · André Draszik

Hi Khem,

Quoted 16 lines · Khem Raj

Setting VERBOSE always, while is fine on one hand for debugging, its
coming at an expense of creating lots and lots of logs, e.g. qtwebkit
compile logs alone with VERBOSE is 163MB, there are many other large
packages which use cmake e.g. WPE, webkitgtk etc which are in same range
with out this option on, the logs reduce to 861K and also speeds up
build a notch

If user needs to enable this logs for targetted debugging debugging that
could be added via

Quoted 6 lines

EXTRA_OECMAKE_BUILD += "--verbose"

in recipe

I don't agree with the reasoning to always disable this by default.
- it's
way too useful in general. I can't count the number of times it was enough
to look at the log.do_compile to figure out something is wrong.

Now you have to recompile everything with this merged, and something that
took 5 minutes to debug becomes an arduous task.

If the Webkit build is too verbose, you should make that specific recipe
less verbose. E.g. you could always set EXTRA_OECMAKE_BUILD, and just clear
it out in the webkit recipe.

Otherwise, based on the same reasoning, you should disable all logs, not just
cmake logs: kernel, make output in general, bitbake logs. They all take up space.

Quoted 4 lines

Seriously, those logs are useful, keeping in mind that often logs are
collected on build-machines, and having easy access is a huge time- saver.

Quoted 6 lines

Personally I agree with André. We have verbose logs for autotools and
meson, so why is cmake special. As this is a webkit-specific issue for
you, the neater fix would be to override this in webkit.

perhaps there should be a global production mode and curb these logs too. Since now we are citing webkit as an example solving it just for webkit does'nt seem right option.

Quoted 8 lines

Stepping back though, why are 163MB log files a problem? Are you archiving every log maybe? If so then there's an argument to be made
for a global "logs should be verbose" toggle, that defaults to on but
you can set to off. That way the kernel, autotools, etc can also follow
the same logic and *all* logs can be smaller.

I agree on this point. Although I wonder if 80% problems can be solved by fixing cmake verbosity then that not a bad start. After this patch one could infact enable it globally too by setting it in config metadata, so we are just toggling defaults.

Quoted 3 lines

Ross

author
#4 lore raw
In reply to André Draszik
Quoted 22 lines · André Draszik

Hi Khem,

Quoted 19 lines · Khem Raj

Setting VERBOSE always, while is fine on one hand for debugging, its
coming at an expense of creating lots and lots of logs, e.g. qtwebkit
compile logs alone with VERBOSE is 163MB, there are many other large
packages which use cmake e.g. WPE, webkitgtk etc which are in same range
with out this option on, the logs reduce to 861K and also speeds up build a notch

If user needs to enable this logs for targetted debugging debugging that
could be added via

EXTRA_OECMAKE_BUILD += "--verbose"

in recipe

Quoted 9 lines
Quoted 2 lines

I don't agree with the reasoning to always disable this by default. - it's
way too useful in general. I can't count the number of times it was enough
to look at the log.do_compile to figure out something is wrong.

What details do you look into when builds dont break ? and how often? perhaps that will help in understanding the case.

Quoted 4 lines

Now you have to recompile everything with this merged, and something that
took 5 minutes to debug becomes an arduous task.

there are two aspects to it, one you described and another one is where users and infra is just building it. They are not component developers rather consumers. I think latter is more common by far.

Quoted 6 lines

If the Webkit build is too verbose, you should make that specific recipe
less verbose. E.g. you could always set EXTRA_OECMAKE_BUILD, and just clear
it out in the webkit recipe.

perhaps that is doable and after current patch it can be set globally too. However, for an OE user its a bad experience

1. its not default cmake option to enable verbosity, so when someone builds their stuff with OE, they don't get good experience, I consider that added impediment to use OE.

2. People who enable it know what they are doing.

Quoted 5 lines

Otherwise, based on the same reasoning, you should disable all logs, not just
cmake logs: kernel, make output in general, bitbake logs. They all take up space.

if they are large perhaps they should be too.
163M is a lot of logs, with slow disks it degrades the compile time, helps killing the disks quicker, all post processing it delayed, you pay high price for something you will occasionally use.

Quoted 4 lines

Seriously, those logs are useful, keeping in mind that often logs are collected on build-machines, and having easy access is a huge time- saver.

One case where people look into these logs are when something breaks and all error messages are logged without this option as well.

Quoted 40 lines

Cheers,
Andre'

Quoted 34 lines
---
 meta/classes/cmake.bbclass | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
meta/classes/cmake.bbclass +1 -0
index 8ccb1eefc7..8a44a0c216 100644
@@ -181,9 +181,14 @@ cmake_do_configure() {
 	  -Wno-dev
 }
 
+# To enable VERBOSE option to emit all kind of logs for a given

recipe add following in it

meta/classes/cmake.bbclass +4 -1
+#
+# EXTRA_OECMAKE_BUILD += "--verbose"
+#
+# Then run do_compile again
 cmake_runcmake_build() {
-	bbnote ${DESTDIR:+DESTDIR=${DESTDIR} }VERBOSE=1 cmake --build

'${B}' "$@" -- ${EXTRA_OECMAKE_BUILD}

meta/classes/cmake.bbclass +0 -1
-	eval ${DESTDIR:+DESTDIR=${DESTDIR} }VERBOSE=1 cmake --build

'${B}' "$@" -- ${EXTRA_OECMAKE_BUILD}

meta/classes/cmake.bbclass +1 -0
+	bbnote ${DESTDIR:+DESTDIR=${DESTDIR} } cmake --build '${B}'

"$@" -- ${EXTRA_OECMAKE_BUILD}

meta/classes/cmake.bbclass +1 -1
+	eval ${DESTDIR:+DESTDIR=${DESTDIR} } cmake --build '${B}' "$@"
-- ${EXTRA_OECMAKE_BUILD}
 }
 
 cmake_do_compile()  {
Signature · 1 line
2.24.1