Skip to contentSkip to conversation

infra/ccache: avoid color diagnostics with GCC older than v4.9

PATCH Superseded Carlos Santos 3 participants 17 messages Feb 10, 2017 – Feb 14, 2017

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but this flag requires GCC v4.9 or later. Older versions fail, complaining about the unrecognized command line option.

The problem is not detectable in the autobuilders because they redirect the output to files/pipes and ccache adds the offending flag only when stderr is a terminal.

---
Change-Id: I0574c7527b24d9e21119961adfa1c7aa17337314

---
 package/Makefile.in | 12 ++++++++++++
 1 file changed, 12 insertions(+)
package/Makefile.in +12 -0
index 5d591e9..6a010b1 100644
@@ -237,6 +237,13 @@ export PERL5LIB=$(HOST_DIR)/usr/lib/perl
 
 TARGET_MAKE_ENV = PATH=$(BR_PATH)
 
+# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
+# this flag requires GCC v4.9 or later. Older versions fail, complaining
+# about unrecognized command line option.
+ifneq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),y)
+	TARGET_MAKE_ENV += GCC_COLORS=""
+endif
+
 TARGET_CONFIGURE_OPTS = \
 	$(TARGET_MAKE_ENV) \
 	AR="$(TARGET_AR)" \
@@ -286,6 +293,11 @@ HOST_MAKE_ENV = \
 	PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \
 	PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig:$(HOST_DIR)/usr/share/pkgconfig"
 
+# Prevent ccache from passing '-fdiagnostics-color' (see above).
+ifneq ($(BR2_HOST_GCC_AT_LEAST_4_9),y)
+	HOST_MAKE_ENV += GCC_COLORS=""
+endif
+
 HOST_CONFIGURE_OPTS = \
 	$(HOST_MAKE_ENV) \
 	AR="$(HOSTAR)" \
Signature · 1 line
2.7.4
#1 lore raw

PATCH 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

Hello,

Adding Arnout, Yann and Gustavo in Cc.

Quoted 9 lines · Carlos Santos

When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but this flag requires GCC v4.9 or later. Older versions fail, complaining about the unrecognized command line option.

The problem is not detectable in the autobuilders because they redirect the output to files/pipes and ccache adds the offending flag only when stderr is a terminal.

This looks good to me in principle, so if Arnout, Yann and Gustavo don't complain, I'm going to soon apply this to master.

Another option would be to adjust this in ccache itself at compile time (i.e when ccache gets built) rather than having to pass this additional GCC_COLORS variable in the environment when building things. This way we would be absolutely sure -fdiagnostics-color will never be passed. Hence why I'd like the feedback from other developers.

A few nits below (but I can fix while applying).

package/Makefile.in export PERL5LIB=$(HOST_DIR)/usr/lib/perl · line 243

Quoted 4 lines
package/Makefile.in +4 -0
+# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
+# this flag requires GCC v4.9 or later. Older versions fail, complaining
+# about unrecognized command line option.
+ifneq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),y)

I think we tend to use:

ifeq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),)

instead.

package/Makefile.in export PERL5LIB=$(HOST_DIR)/usr/lib/perl · line 244

Quoted 1 line
package/Makefile.in +1 -0
+	TARGET_MAKE_ENV += GCC_COLORS=""

And we don't indent such lines.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#2 lore raw

PATCH 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

Thomas, Carlos, All,

On 2017-02-11 16:38 +0100, Thomas Petazzoni spake thusly:

Quoted 15 lines

Hello,

Adding Arnout, Yann and Gustavo in Cc.

Quoted 9 lines · Carlos Santos

When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but this flag requires GCC v4.9 or later. Older versions fail, complaining about the unrecognized command line option.

The problem is not detectable in the autobuilders because they redirect the output to files/pipes and ccache adds the offending flag only when stderr is a terminal.

Quoted 8 lines

This looks good to me in principle, so if Arnout, Yann and Gustavo don't complain, I'm going to soon apply this to master.

Another option would be to adjust this in ccache itself at compile time (i.e when ccache gets built) rather than having to pass this additional GCC_COLORS variable in the environment when building things. This way we would be absolutely sure -fdiagnostics-color will never be passed. Hence why I'd like the feedback from other developers.

But there is no config option to disable that in ccache. The behaviour is hard-coded; we would need a patch to disable that.

Quoted 9 lines

A few nits below (but I can fix while applying).

package/Makefile.in export PERL5LIB=$(HOST_DIR)/usr/lib/perl · line 243

Quoted 4 lines
package/Makefile.in +4 -0
+# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
+# this flag requires GCC v4.9 or later. Older versions fail, complaining
+# about unrecognized command line option.
+ifneq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),y)

I think we tend to use:

Quoted 7 lines

ifeq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),)

instead.

package/Makefile.in export PERL5LIB=$(HOST_DIR)/usr/lib/perl · line 244

Quoted 1 line
package/Makefile.in +1 -0
+	TARGET_MAKE_ENV += GCC_COLORS=""

And we don't indent such lines.

And we might need that for the host as well, no?

    ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),)
    HOST_MAKE_ENV += GCC_COLORS=""
    endif

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#3 lore raw

PATCH 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

Hello,

Quoted 8 lines · Yann E. MORIN
Quoted 5 lines

Another option would be to adjust this in ccache itself at compile time (i.e when ccache gets built) rather than having to pass this additional GCC_COLORS variable in the environment when building things. This way we would be absolutely sure -fdiagnostics-color will never be passed. Hence why I'd like the feedback from other developers.

But there is no config option to disable that in ccache. The behaviour is hard-coded; we would need a patch to disable that.

We could sed it like we already do in package/ccache/ccache.mk for other things.

Quoted 5 lines

And we might need that for the host as well, no?

    ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),)
    HOST_MAKE_ENV += GCC_COLORS=""
    endif

Check Carlos original patch, it was there.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#4 lore raw

PATCH 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

Thomas, All,

On 2017-02-12 13:14 +0100, Thomas Petazzoni spake thusly:

Quoted 8 lines · Yann E. MORIN
Quoted 7 lines
Quoted 6 lines

Another option would be to adjust this in ccache itself at compile time (i.e when ccache gets built) rather than having to pass this additional GCC_COLORS variable in the environment when building things. This way we would be absolutely sure -fdiagnostics-color will never be passed. Hence why I'd like the feedback from other developers.

Quoted 5 lines
Quoted 2 lines

But there is no config option to disable that in ccache. The behaviour is hard-coded; we would need a patch to disable that.

We could sed it like we already do in package/ccache/ccache.mk for other things.

Something like, maybe:

    diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
    index f174a5d..35d7a7b 100644
    --- a/package/ccache/ccache.mk
    +++ b/package/ccache/ccache.mk
    @@ -31,6 +31,7 @@ HOST_CCACHE_CONF_OPTS += --with-bundled-zlib
     define HOST_CCACHE_PATCH_CONFIGURATION
     	sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/ccache.c
     	sed -i 's,"%s/.ccache","$(BR_CACHE_DIR)",' $(@D)/conf.c
    +	sed -i  '/if (getenv("GCC_COLORS")/,+4d' $(@D)/ccache.c
     endef
     
     HOST_CCACHE_POST_PATCH_HOOKS += HOST_CCACHE_PATCH_CONFIGURATION

Or would we want it to be conditional on gcc < 4.9? In anycase, I think overridign the environment variable is much easier...

Quoted 6 lines
Quoted 5 lines

And we might need that for the host as well, no?

    ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),)
    HOST_MAKE_ENV += GCC_COLORS=""
    endif

Check Carlos original patch, it was there.

I didn't look; I only saw your own reply...

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#5 lore raw

PATCH 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

Hello,

Quoted 15 lines · Yann E. MORIN

Something like, maybe:

    diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
    index f174a5d..35d7a7b 100644
    --- a/package/ccache/ccache.mk
    +++ b/package/ccache/ccache.mk
    @@ -31,6 +31,7 @@ HOST_CCACHE_CONF_OPTS += --with-bundled-zlib
     define HOST_CCACHE_PATCH_CONFIGURATION
     	sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/ccache.c
     	sed -i 's,"%s/.ccache","$(BR_CACHE_DIR)",' $(@D)/conf.c
    +	sed -i  '/if (getenv("GCC_COLORS")/,+4d' $(@D)/ccache.c
     endef
     
     HOST_CCACHE_POST_PATCH_HOOKS += HOST_CCACHE_PATCH_CONFIGURATION
Quoted 1 line

Or would we want it to be conditional on gcc < 4.9?

Yes, it should be conditional on gcc < 4.9.

Quoted 2 lines

In anycase, I think overridign the environment variable is much easier...

Right, *but* if we forget to pass TARGET_MAKE_ENV or HOST_MAKE_ENV somewhere, then we will get a build failure. As a trivial example, package/makedevs/makedevs.mk directly calls $(HOSTCC), which uses ccache, but does not use $(HOST_MAKE_ENV). So that one would not pass GCC_COLORS="", and would therefore fail to build on machines with gcc < 4.9.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#6 lore raw

PATCH 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

Thomas, All,

On 2017-02-12 13:59 +0100, Thomas Petazzoni spake thusly:

Quoted 24 lines · Yann E. MORIN
Quoted 16 lines

Something like, maybe:

    diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk
    index f174a5d..35d7a7b 100644
    --- a/package/ccache/ccache.mk
    +++ b/package/ccache/ccache.mk
    @@ -31,6 +31,7 @@ HOST_CCACHE_CONF_OPTS += --with-bundled-zlib
     define HOST_CCACHE_PATCH_CONFIGURATION
     	sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/ccache.c
     	sed -i 's,"%s/.ccache","$(BR_CACHE_DIR)",' $(@D)/conf.c
    +	sed -i  '/if (getenv("GCC_COLORS")/,+4d' $(@D)/ccache.c
     endef
     
     HOST_CCACHE_POST_PATCH_HOOKS += HOST_CCACHE_PATCH_CONFIGURATION
Quoted 1 line

Or would we want it to be conditional on gcc < 4.9?

Yes, it should be conditional on gcc < 4.9.

Quoted 2 lines

In anycase, I think overridign the environment variable is much easier...

Quoted 6 lines

Right, *but* if we forget to pass TARGET_MAKE_ENV or HOST_MAKE_ENV somewhere, then we will get a build failure. As a trivial example, package/makedevs/makedevs.mk directly calls $(HOSTCC), which uses ccache, but does not use $(HOST_MAKE_ENV). So that one would not pass GCC_COLORS="", and would therefore fail to build on machines with gcc < 4.9.

But we do a single build of ccache which gets used for both the target and host compiler. So if we patch ccache, we have to do if any of them if older than 4.9.

Which is not impossible, by far, but means that an older host gcc will disable colouring of the target gcc output, and vice-versa.

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#7 lore raw

PATCH 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

Hello,

Quoted 6 lines · Yann E. MORIN

But we do a single build of ccache which gets used for both the target and host compiler. So if we patch ccache, we have to do if any of them if older than 4.9.

Which is not impossible, by far, but means that an older host gcc will disable colouring of the target gcc output, and vice-versa.

Correct, but it still seems better than failing completely.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#8 lore raw

PATCH v2 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

In reply to Carlos Santos

Ensure that the GCC_COLORS environment variable is empty when calling ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but this flag requires GCC v4.9 or later. Older versions complain about the unrecognized command line option.

That behavior is hard-coded in ccache; we would need a patch to either make it optional or disable it at all. Notice also that the problem is not detectable in the autobuilders because they redirect the output to files/pipes and ccache adds the offending flag only when stderr is a terminal.

---
Changes v1->v2
Fix indentation, use positive logic and improve the commit message, acording to comments from Thomas Petazzoni and Yann E. MORIN.

---
 package/Makefile.in | 12 ++++++++++++
 1 file changed, 12 insertions(+)
package/Makefile.in +12 -0
index 4a3eb26..e5e595a 100644
@@ -236,6 +236,13 @@ export PERL5LIB=$(HOST_DIR)/usr/lib/perl
 
 TARGET_MAKE_ENV = PATH=$(BR_PATH)
 
+# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
+# this flag requires GCC v4.9 or later. Older versions fail, complaining
+# about unrecognized command line option.
+ifeq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),)
+TARGET_MAKE_ENV += GCC_COLORS=""
+endif
+
 TARGET_CONFIGURE_OPTS = \
 	$(TARGET_MAKE_ENV) \
 	AR="$(TARGET_AR)" \
@@ -285,6 +292,11 @@ HOST_MAKE_ENV = \
 	PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \
 	PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig:$(HOST_DIR)/usr/share/pkgconfig"
 
+# Prevent ccache from passing '-fdiagnostics-color' (see above).
+ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),)
+HOST_MAKE_ENV += GCC_COLORS=""
+endif
+
 HOST_CONFIGURE_OPTS = \
 	$(HOST_MAKE_ENV) \
 	AR="$(HOSTAR)" \
Signature · 1 line
2.7.4
#9 lore raw

PATCH v2 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

Carlos, All,

On 2017-02-12 09:33 -0200, Carlos Santos spake thusly:

Quoted 5 lines

Ensure that the GCC_COLORS environment variable is empty when calling ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but this flag requires GCC v4.9 or later. Older versions complain about the unrecognized command line option.

Quoted 7 lines

That behavior is hard-coded in ccache; we would need a patch to either make it optional or disable it at all. Notice also that the problem is not detectable in the autobuilders because they redirect the output to files/pipes and ccache adds the offending flag only when stderr is a terminal.

Should go to master.

Regards,
Yann E. MORIN.

Quoted 43 lines

---
Changes v1->v2
Fix indentation, use positive logic and improve the commit message, acording to comments from Thomas Petazzoni and Yann E. MORIN.

---
 package/Makefile.in | 12 ++++++++++++
 1 file changed, 12 insertions(+)
package/Makefile.in +12 -0
index 4a3eb26..e5e595a 100644
@@ -236,6 +236,13 @@ export PERL5LIB=$(HOST_DIR)/usr/lib/perl
 
 TARGET_MAKE_ENV = PATH=$(BR_PATH)
 
+# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
+# this flag requires GCC v4.9 or later. Older versions fail, complaining
+# about unrecognized command line option.
+ifeq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),)
+TARGET_MAKE_ENV += GCC_COLORS=""
+endif
+
 TARGET_CONFIGURE_OPTS = \
 	$(TARGET_MAKE_ENV) \
 	AR="$(TARGET_AR)" \
@@ -285,6 +292,11 @@ HOST_MAKE_ENV = \
 	PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \
 	PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig:$(HOST_DIR)/usr/share/pkgconfig"
 
+# Prevent ccache from passing '-fdiagnostics-color' (see above).
+ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),)
+HOST_MAKE_ENV += GCC_COLORS=""
+endif
+
 HOST_CONFIGURE_OPTS = \
 	$(HOST_MAKE_ENV) \
 	AR="$(HOSTAR)" \
Signature · 1 line
2.7.4
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#10 lore raw

PATCH v2 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

In reply to Carlos Santos

Carlos, All,

On 2017-02-12 09:33 -0200, Carlos Santos spake thusly:

Quoted 4 lines

Ensure that the GCC_COLORS environment variable is empty when calling ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but this flag requires GCC v4.9 or later. Older versions complain about the unrecognized command line option.

So, while fixing my own patch about the issue [0], I noticed that I was not even able to make it fail in the first place...

I'm using this defconfig, which uses a gcc-4.7:

    BR2_x86_i686=y
    BR2_CCACHE=y
    BR2_TOOLCHAIN_EXTERNAL=y

and Thomas reported he was also unable to reproduce the failure with:

    BR2_sh=y
    BR2_sh4a=y
    BR2_CCACHE=y
    BR2_TOOLCHAIN_EXTERNAL=y
    BR2_INIT_NONE=y
    BR2_SYSTEM_BIN_SH_NONE=y
    # BR2_PACKAGE_BUSYBOX is not set
    BR2_PACKAGE_DEVMEM2=y
    # BR2_TARGET_ROOTFS_TAR is not set

which also uses gcc-4.7.

Can you provide a reliable way to reproduce the issue, please?

Regards,
Yann E. MORIN.

Quoted 50 lines

That behavior is hard-coded in ccache; we would need a patch to either make it optional or disable it at all. Notice also that the problem is not detectable in the autobuilders because they redirect the output to files/pipes and ccache adds the offending flag only when stderr is a terminal.

---
Changes v1->v2
Fix indentation, use positive logic and improve the commit message, acording to comments from Thomas Petazzoni and Yann E. MORIN.

---
 package/Makefile.in | 12 ++++++++++++
 1 file changed, 12 insertions(+)
package/Makefile.in +12 -0
index 4a3eb26..e5e595a 100644
@@ -236,6 +236,13 @@ export PERL5LIB=$(HOST_DIR)/usr/lib/perl
 
 TARGET_MAKE_ENV = PATH=$(BR_PATH)
 
+# When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but
+# this flag requires GCC v4.9 or later. Older versions fail, complaining
+# about unrecognized command line option.
+ifeq ($(BR2_TOOLCHAIN_GCC_AT_LEAST_4_9),)
+TARGET_MAKE_ENV += GCC_COLORS=""
+endif
+
 TARGET_CONFIGURE_OPTS = \
 	$(TARGET_MAKE_ENV) \
 	AR="$(TARGET_AR)" \
@@ -285,6 +292,11 @@ HOST_MAKE_ENV = \
 	PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 \
 	PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig:$(HOST_DIR)/usr/share/pkgconfig"
 
+# Prevent ccache from passing '-fdiagnostics-color' (see above).
+ifeq ($(BR2_HOST_GCC_AT_LEAST_4_9),)
+HOST_MAKE_ENV += GCC_COLORS=""
+endif
+
 HOST_CONFIGURE_OPTS = \
 	$(HOST_MAKE_ENV) \
 	AR="$(HOSTAR)" \
Signature · 1 line
2.7.4
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#11 lore raw

PATCH v2 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

Quoted 36 lines

From: "Yann E. MORIN" <[email protected]>
To: "Carlos Santos" <[email protected]>
Cc: buildroot at buildroot.org, "Thomas Petazzoni" <[email protected]>, arnout at mind.be, gustavo at zacarias.com.ar
Sent: Sunday, February 12, 2017 12:14:25 PM
Subject: Re: [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

Carlos, All,

On 2017-02-12 09:33 -0200, Carlos Santos spake thusly:

Quoted 4 lines

Ensure that the GCC_COLORS environment variable is empty when calling ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but this flag requires GCC v4.9 or later. Older versions complain about the unrecognized command line option.

So, while fixing my own patch about the issue [0], I noticed that I was not even able to make it fail in the first place...

I'm using this defconfig, which uses a gcc-4.7:

   BR2_x86_i686=y
   BR2_CCACHE=y
   BR2_TOOLCHAIN_EXTERNAL=y

and Thomas reported he was also unable to reproduce the failure with:

   BR2_sh=y
   BR2_sh4a=y
   BR2_CCACHE=y
   BR2_TOOLCHAIN_EXTERNAL=y
   BR2_INIT_NONE=y
   BR2_SYSTEM_BIN_SH_NONE=y
   # BR2_PACKAGE_BUSYBOX is not set
   BR2_PACKAGE_DEVMEM2=y
   # BR2_TARGET_ROOTFS_TAR is not set
Quoted 3 lines

which also uses gcc-4.7.

Can you provide a reliable way to reproduce the issue, please?

What does "echo $GCC_COLORS" say? And remember that you must not redirect the output to a file or pipe.

Carlos Santos (Casantos) - DATACOM, P&D
?Something must be done. This is something. Therefore we must do it.? (from the BBC series ?Yes, Prime Minister?, ?Power to the people?)

#12 lore raw

PATCH v2 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

Carlos, All,

On 2017-02-12 12:55 -0200, Carlos Santos spake thusly:

Quoted 36 lines
Quoted 36 lines

From: "Yann E. MORIN" <[email protected]>
To: "Carlos Santos" <[email protected]>
Cc: buildroot at buildroot.org, "Thomas Petazzoni" <[email protected]>, arnout at mind.be, gustavo at zacarias.com.ar
Sent: Sunday, February 12, 2017 12:14:25 PM
Subject: Re: [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

Carlos, All,

On 2017-02-12 09:33 -0200, Carlos Santos spake thusly:

Quoted 4 lines

Ensure that the GCC_COLORS environment variable is empty when calling ccache. When GCC_COLORS is set, ccache passes '-fdiagnostics-color' to GCC but this flag requires GCC v4.9 or later. Older versions complain about the unrecognized command line option.

So, while fixing my own patch about the issue [0], I noticed that I was not even able to make it fail in the first place...

I'm using this defconfig, which uses a gcc-4.7:

   BR2_x86_i686=y
   BR2_CCACHE=y
   BR2_TOOLCHAIN_EXTERNAL=y

and Thomas reported he was also unable to reproduce the failure with:

   BR2_sh=y
   BR2_sh4a=y
   BR2_CCACHE=y
   BR2_TOOLCHAIN_EXTERNAL=y
   BR2_INIT_NONE=y
   BR2_SYSTEM_BIN_SH_NONE=y
   # BR2_PACKAGE_BUSYBOX is not set
   BR2_PACKAGE_DEVMEM2=y
   # BR2_TARGET_ROOTFS_TAR is not set
Quoted 6 lines
Quoted 3 lines

which also uses gcc-4.7.

Can you provide a reliable way to reproduce the issue, please?

What does "echo $GCC_COLORS" say? And remember that you must not redirect the output to a file or pipe.

The GCC_COLORS are the default as dopcumented in the man page for gcc-5.

    $ cat defconfig
    BR2_x86_i686=y
    BR2_CCACHE=y
    BR2_TOOLCHAIN_EXTERNAL=y
    BR2_PACKAGE_LIBCEC=y
    $ export GCC_COLORS='error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01'
    $ make BR2_DEFCONFIG=$(pwd)/defconfig defconfig
    $ make busybox libcec

And the build does not fail... :-/

So I did a few other tentative:

    $ rm -rf ~/.buildroot-ccache/*/
    $ echo log_file = /home/ymorin/meh >> ~/.buildroot-ccache/ccache.conf
    $ touch meh.c
    $ ./host/usr/bin/i686-pc-linux-gnu-gcc -c -o meh.o meh.c
    $ ~/meh
    [2017-02-12T18:19:31.365731 7668 ] === CCACHE 3.3.3 STARTED =========================================
    [2017-02-12T18:19:31.365731 7668 ] Config: (environment) base_dir = /home/ymorin/dev/buildroot/O
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) cache_dir = /home/ymorin/.buildroot-ccache
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) cache_dir_levels = 2
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) compiler = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) compiler_check = mtime
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) compression = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) compression_level = 6
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) cpp_extension = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) direct_mode = true
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) disable = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) extra_files_to_hash = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) hard_link = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) hash_dir = true
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) ignore_headers_in_manifest = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) keep_comments_cpp = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) limit_multiple = 0.8
    [2017-02-12T18:19:31.365731 7668 ] Config: (/home/ymorin/.buildroot-ccache/ccache.conf) log_file = /home/ymorin/meh
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) max_files = 0
    [2017-02-12T18:19:31.365731 7668 ] Config: (/home/ymorin/.buildroot-ccache/ccache.conf) max_size = 5.0G
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) path = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) prefix_command = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) prefix_command_cpp = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) read_only = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) read_only_direct = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) recache = false
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) run_second_cpp = true
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) sloppiness = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) stats = true
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) temporary_dir = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) umask = 
    [2017-02-12T18:19:31.365731 7668 ] Config: (default) unify = false
    [2017-02-12T18:19:31.365999 7668 ] Command line: /home/ymorin/dev/buildroot/O/host/usr/bin/ccache /home/ymorin/dev/buildroot/O/host/opt/ext-toolchain/bin/i686-pc-linux-gnu-gcc --sysroot /home/ymorin/dev/buildroot/O/host/usr/i686-buildroot-linux-gnu/sysroot -march=i686 -c -o meh.o meh.c
    [2017-02-12T18:19:31.366080 7668 ] Hostname: gourin
    [2017-02-12T18:19:31.366157 7668 ] Working directory: /home/ymorin/dev/buildroot/O
    [2017-02-12T18:19:31.366280 7668 ] Automatically enabling colors
    [2017-02-12T18:19:31.366312 7668 ] Source file: meh.c
    [2017-02-12T18:19:31.366331 7668 ] Object file: meh.o
    [2017-02-12T18:19:31.366367 7668 ] Trying direct lookup
    [2017-02-12T18:19:31.366441 7668 ] Looking for object file hash in /home/ymorin/.buildroot-ccache/e/2/095355fd431a7dfa30fcd2c18aa6c7-359.manifest
    [2017-02-12T18:19:31.366468 7668 ] No such manifest file
    [2017-02-12T18:19:31.366486 7668 ] Did not find object file hash in manifest
    [2017-02-12T18:19:31.366695 7668 ] Running preprocessor
    [2017-02-12T18:19:31.366720 7668 ] Executing /home/ymorin/dev/buildroot/O/host/opt/ext-toolchain/bin/i686-pc-linux-gnu-gcc --sysroot host/usr/i686-buildroot-linux-gnu/sysroot -march=i686 -fdiagnostics-color -E meh.c
    [2017-02-12T18:19:31.368786 7668 ] Preprocessor gave exit status 1
    [2017-02-12T18:19:31.368834 7668 ] Failed; falling back to running the real compiler
    [2017-02-12T18:19:31.368847 7668 ] Executing /home/ymorin/dev/buildroot/O/host/opt/ext-toolchain/bin/i686-pc-linux-gnu-gcc --sysroot /home/ymorin/dev/buildroot/O/host/usr/i686-buildroot-linux-gnu/sysroot -march=i686 -c -o meh.o meh.c
    [2017-02-12T18:19:31.368943 7668 ] lockfile_acquire: symlink /home/ymorin/.buildroot-ccache/c/stats.lock: No such file or directory
    [2017-02-12T18:19:31.369031 7668 ] Acquired lock /home/ymorin/.buildroot-ccache/c/stats.lock
    [2017-02-12T18:19:31.369147 7668 ] Releasing lock /home/ymorin/.buildroot-ccache/c/stats.lock
    [2017-02-12T18:19:31.369163 7668 ] Unlink /home/ymorin/.buildroot-ccache/c/stats.lock
    [2017-02-12T18:19:31.369189 7668 ] Result: preprocessor error

Hope you can make something out of this.

BTW, I never trusted ccache; it only ever brought me more headaches on which I spent more time than it ever saved me...

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#13 lore raw

PATCH v2 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

In reply to Carlos Santos

Hello,

Quoted 3 lines · Carlos Santos
Quoted 1 line

Can you provide a reliable way to reproduce the issue, please?

What does "echo $GCC_COLORS" say?

thomas at skate:~$ echo $GCC_COLORS

thomas at skate:~$

So the issue only arise when GCC_COLORS is defined non-empty in the user's environment?

Quoted 1 line

And remember that you must not redirect the output to a file or pipe.

I just ran "make" without redirecting the output to a file or pipe.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#14 lore raw

PATCH v2 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

On 2017-02-12 18:22 +0100, Thomas Petazzoni spake thusly:

Quoted 6 lines

Hello,

Quoted 2 lines · Carlos Santos
Quoted 1 line

Can you provide a reliable way to reproduce the issue, please?

Quoted 8 lines
Quoted 1 line

What does "echo $GCC_COLORS" say?

thomas at skate:~$ echo $GCC_COLORS

thomas at skate:~$

So the issue only arise when GCC_COLORS is defined non-empty in the user's environment?

Well, from the point of view of ccache, any value whould have triggered the issue, because ccache only ever checks if the variables exists in the environment.

But see my other reply: even with a valid value, I could not trigger the bug either...

Quoted 2 lines
Quoted 1 line

And remember that you must not redirect the output to a file or pipe.

I just ran "make" without redirecting the output to a file or pipe.

Same as I did.

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#15 lore raw

PATCH v2 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

In reply to Yann E. MORIN
Quoted 8 lines

From: "Yann E. MORIN" <[email protected]>
To: "Carlos Santos" <[email protected]>
Cc: buildroot at buildroot.org, "Thomas Petazzoni" <[email protected]>, arnout at mind.be, gustavo at zacarias.com.ar
Sent: Sunday, February 12, 2017 12:14:25 PM
Subject: Re: [PATCH v2] infra/ccache: avoid color diagnostics with GCC older than v4.9

Carlos, All,

[...]

Quoted 5 lines

Can you provide a reliable way to reproduce the issue, please?

Regards,
Yann E. MORIN.

Sorry, I forgot so send this yesterday. Here is a reliable reproducer:

$ cat defconfig 
BR2_powerpc=y
BR2_powerpc_8548=y
BR2_CCACHE=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/powerpc-ctng_e500v2-linux-gnuspe.tar.xz"
BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="powerpc-ctng_e500v2-linux-gnuspe"
BR2_TOOLCHAIN_EXTERNAL_GCC_4_7=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_12=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_PACKAGE_OPENSSL=y
$ 
$ echo $GCC_COLORS 
error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01
$
$ make V=1 BR2_JLEVEL=1 openssl-dirclean openssl
[...]
/work/ccache-colors/host/usr/bin/powerpc-ctng_e500v2-linux-gnuspe-gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DB_ENDIAN -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -mabi=spe -mfloat-gprs=double -Wa,-me500x2 -Os  -Wall -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DVPAES_ASM -c  -o ppccpuid.o ppccpuid.s
powerpc-ctng_e500v2-linux-gnuspe-gcc: error: unrecognized command line option '-fdiagnostics-color'
<builtin>: recipe for target 'ppccpuid.o' failed
make[3]: *** [ppccpuid.o] Error 1
Makefile:287: recipe for target 'build_crypto' failed
make[2]: *** [build_crypto] Error 1
package/pkg-generic.mk:216: recipe for target '/work/ccache-colors/build/openssl-1.0.2k/.stamp_built' failed
make[1]: *** [/work/ccache-colors/build/openssl-1.0.2k/.stamp_built] Error 2
Makefile:16: recipe for target '_all' failed
make: *** [_all] Error 2

Carlos Santos (Casantos) - DATACOM, P&D
?Something must be done. This is something. Therefore we must do it.? (from the BBC series ?Yes, Prime Minister?, ?Power to the people?)

#16 lore raw

PATCH v2 1/1 infra/ccache: avoid color diagnostics with GCC older than v4.9

Carlos, All,

On 2017-02-13 20:41 -0200, Carlos Santos spake thusly:

Quoted 12 lines

Sorry, I forgot so send this yesterday. Here is a reliable reproducer:

$ cat defconfig 
BR2_powerpc=y
BR2_powerpc_8548=y
BR2_CCACHE=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/powerpc-ctng_e500v2-linux-gnuspe.tar.xz"
BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="powerpc-ctng_e500v2-linux-gnuspe"
BR2_TOOLCHAIN_EXTERNAL_GCC_4_7=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_12=y
Quoted 8 lines
BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_PACKAGE_OPENSSL=y
$ 
$ echo $GCC_COLORS 
error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01
$
$ make V=1 BR2_JLEVEL=1 openssl-dirclean openssl

Indeed, I could reproduce. Thanks.

So, I think Arnout is right in saying we should just globally unexport GCC_COLORS and be done with that.

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'