Skip to contentSkip to conversation

On overriding make variables from the environment...

PATCH Superseded SZEDER Gábor 5 participants 33 messages Oct 16, 2018 – Jan 18, 2019

lore
Share & tools
lore
#0 lore raw

Our Makefile has lines like these:

CFLAGS = -g -O2 -Wall
CC = cc
AR = ar
SPATCH = spatch

Note the use of '=', not '?='. This means that these variables can be overridden from the command line, i.e. 'make CC=gcc-X.Y' will build with that particular GCC version, but not from the environment, i.e. 'CC=gcc-X.Y make' will still build with 'cc'.

This can be confusing for developers who come from other projects where they used to run 'CC=whatever make'.

And our build jobs on Travis CI are badly affected by this. We have dedicated build jobs to build Git with GCC and Clang both on Linux and OSX from the very beginning (522354d70f (Add Travis CI support, 2015-11-27)). But guess how Travis CI specifies which compiler to use! With 'export CC=gcc' and 'export CC=clang', respectively. Consequently, our Clang Linux build job has always used gcc, because that's where 'cc' points at on Linux by default, while the GCC OSX build job has always used Clang. Oh, well. Furthermore, see 37fa4b3c78 (travis-ci: run gcc-8 on linux-gcc jobs, 2018-05-19), where Duy added an 'export CC=gcc-8' in an attempt to use a more modern compiler, but this had no effect either.

I'm not sure what to do. I'm fine with updating our 'ci/' scripts to explicitly respect CC in the environment (either by running 'make CC=$CC' or by writing $CC into 'config.mak'). Or I could update our Makefile to use '?=' for specific variables, but:

- I'm afraid to break somebody's setup relying on the current behavior and CC having different values in the environment and in 'config.mak'.

- Where to stop, IOW which variables should be set with '?='? CFLAGS, LDFLAGS, CC, AR, ..., SPATCH, SPATCH_FLAGS? Dunno. We already have 'STRIP ?= strip' and there are variables that are checked explicitly (e.g. 'DEVELOPER=y make' works).

Note also that prior to b05701c5b4 (Make CFLAGS overridable from make command line., 2005-08-06) our Makefile used 'CC?=gcc' as well.

#1 lore raw

Hi,

Quoted 8 lines · SZEDER Gábor

Our Makefile has lines like these:

CFLAGS = -g -O2 -Wall
CC = cc
AR = ar
SPATCH = spatch

Note the use of '=', not '?='.

[...]

Quoted 4 lines

I'm not sure what to do. I'm fine with updating our 'ci/' scripts to explicitly respect CC in the environment (either by running 'make CC=$CC' or by writing $CC into 'config.mak'). Or I could update our Makefile to use '?=' for specific variables, but:

That's a good question. I don't have a strong opinion myself, so I tend to trust larger projects like Linux to have thought this through more, and they use 'CC = cc' as well. So I'd lean toward the updating 'ci/' scripts approach, to do something like

	make ${CC:+"CC=$CC"} ...

(or the equivalent multi-line construction).

That also has the bonus of being explicit.

Just my two cents,
Jonathan

#2 lore raw
Quoted 10 lines · Jonathan Nieder
Quoted 8 lines · SZEDER Gábor

Our Makefile has lines like these:

CFLAGS = -g -O2 -Wall
CC = cc
AR = ar
SPATCH = spatch

Note the use of '=', not '?='.

[...]

Quoted 8 lines
Quoted 4 lines

I'm not sure what to do. I'm fine with updating our 'ci/' scripts to explicitly respect CC in the environment (either by running 'make CC=$CC' or by writing $CC into 'config.mak'). Or I could update our Makefile to use '?=' for specific variables, but:

That's a good question. I don't have a strong opinion myself, so I tend to trust larger projects like Linux to have thought this through more, and they use 'CC = cc' as well.

I don't think Linux is a good example to follow in this case, see e.g. 6d62c983f7 (Makefile: Change the default compiler from "gcc" to "cc", 2011-12-20) (in short: Git is supposed to be buildable with compilers other than GCC as well, while Linux not really, so they can hardcode 'CC = gcc').

Also, the projects I have on hand usually have 'CC = gcc' hardcoded in their Makefiles, too, but those Makefiles were generated by their ./configure script (which in turn by ./autogen.sh...), and those tend to write CC from the environment into the generated Makefiles.

#3 lore raw
Quoted 9 lines · SZEDER Gábor
Quoted 8 lines · Jonathan Nieder
Quoted 6 lines · SZEDER Gábor

Our Makefile has lines like these:

CFLAGS = -g -O2 -Wall
CC = cc
AR = ar
SPATCH = spatch

[...]

Quoted 9 lines
Quoted 8 lines
Quoted 4 lines

I'm not sure what to do. I'm fine with updating our 'ci/' scripts to explicitly respect CC in the environment (either by running 'make CC=$CC' or by writing $CC into 'config.mak'). Or I could update our Makefile to use '?=' for specific variables, but:

That's a good question. I don't have a strong opinion myself, so I tend to trust larger projects like Linux to have thought this through more, and they use 'CC = cc' as well.

Quoted 5 lines

I don't think Linux is a good example to follow in this case, see e.g. 6d62c983f7 (Makefile: Change the default compiler from "gcc" to "cc", 2011-12-20) (in short: Git is supposed to be buildable with compilers other than GCC as well, while Linux not really, so they can hardcode 'CC = gcc').

Nowadays Linux builds with clang as well. People also have other reasons to override the CC setting (cross-compiling, etc) and to override other settings like CFLAGS.

Quoted 4 lines

Also, the projects I have on hand usually have 'CC = gcc' hardcoded in their Makefiles, too, but those Makefiles were generated by their ./configure script (which in turn by ./autogen.sh...), and those tend to write CC from the environment into the generated Makefiles.

Yes, I think that's what makes travis's setup normally work. It makes sense to me since ./configure is expected to have more implicit or automatic behavior. For "make", I kind of like that it doesn't depend on environment variables that I am not thinking about when debugging a reported build problem.

When building Git without autoconf, the corresponding place for settings is config.mak. Would it make sense for the ci scripts to write a config.mak file?

Thanks,
Jonathan

#4 lore raw
Quoted 41 lines · Jonathan Nieder
Quoted 9 lines · SZEDER Gábor
Quoted 8 lines · Jonathan Nieder
Quoted 6 lines · SZEDER Gábor

Our Makefile has lines like these:

CFLAGS = -g -O2 -Wall
CC = cc
AR = ar
SPATCH = spatch

[...]

Quoted 9 lines
Quoted 8 lines
Quoted 4 lines

I'm not sure what to do. I'm fine with updating our 'ci/' scripts to explicitly respect CC in the environment (either by running 'make CC=$CC' or by writing $CC into 'config.mak'). Or I could update our Makefile to use '?=' for specific variables, but:

That's a good question. I don't have a strong opinion myself, so I tend to trust larger projects like Linux to have thought this through more, and they use 'CC = cc' as well.

Quoted 5 lines

I don't think Linux is a good example to follow in this case, see e.g. 6d62c983f7 (Makefile: Change the default compiler from "gcc" to "cc", 2011-12-20) (in short: Git is supposed to be buildable with compilers other than GCC as well, while Linux not really, so they can hardcode 'CC = gcc').

Nowadays Linux builds with clang as well. People also have other reasons to override the CC setting (cross-compiling, etc) and to override other settings like CFLAGS.

Quoted 4 lines

Also, the projects I have on hand usually have 'CC = gcc' hardcoded in their Makefiles, too, but those Makefiles were generated by their ./configure script (which in turn by ./autogen.sh...), and those tend to write CC from the environment into the generated Makefiles.

Yes, I think that's what makes travis's setup normally work. It makes sense to me since ./configure is expected to have more implicit or automatic behavior. For "make", I kind of like that it doesn't depend on environment variables that I am not thinking about when debugging a reported build problem.

Quoted 3 lines

When building Git without autoconf, the corresponding place for settings is config.mak. Would it make sense for the ci scripts to write a config.mak file?

A first I though it doesn't, but it turns out it acually does.

'ci/run-build-and-tests.sh' basically runs:

make
make test

I naively put a 'CC=$CC' argument at the end of the first command, thinking it should Just Work... but then that second 'make test' got all clever on me, said "* new build flags", and then proceeded to rebuild everything with the default 'cc'. (With the additional complication, that on Travis CI we actually run 'make --quiet test', which rebuilds everything, well, quietly... so the rebuild itself is not even visible in the build logs.)

So, then it's either 'config.mak', or passing a 'CC=$CC' argument to _all_ make commands, including those that are not supposed to build anything, but only run the tests. I find the latter aesthetically not particularly pleasing.

#5 lore raw

Hi Gábor,

Quoted 61 lines · SZEDER Gábor
Quoted 41 lines · Jonathan Nieder
Quoted 9 lines · SZEDER Gábor
Quoted 8 lines · Jonathan Nieder
Quoted 6 lines · SZEDER Gábor

Our Makefile has lines like these:

CFLAGS = -g -O2 -Wall
CC = cc
AR = ar
SPATCH = spatch

[...]

Quoted 9 lines
Quoted 8 lines
Quoted 4 lines

I'm not sure what to do. I'm fine with updating our 'ci/' scripts to explicitly respect CC in the environment (either by running 'make CC=$CC' or by writing $CC into 'config.mak'). Or I could update our Makefile to use '?=' for specific variables, but:

That's a good question. I don't have a strong opinion myself, so I tend to trust larger projects like Linux to have thought this through more, and they use 'CC = cc' as well.

Quoted 5 lines

I don't think Linux is a good example to follow in this case, see e.g. 6d62c983f7 (Makefile: Change the default compiler from "gcc" to "cc", 2011-12-20) (in short: Git is supposed to be buildable with compilers other than GCC as well, while Linux not really, so they can hardcode 'CC = gcc').

Nowadays Linux builds with clang as well. People also have other reasons to override the CC setting (cross-compiling, etc) and to override other settings like CFLAGS.

Quoted 4 lines

Also, the projects I have on hand usually have 'CC = gcc' hardcoded in their Makefiles, too, but those Makefiles were generated by their ./configure script (which in turn by ./autogen.sh...), and those tend to write CC from the environment into the generated Makefiles.

Yes, I think that's what makes travis's setup normally work. It makes sense to me since ./configure is expected to have more implicit or automatic behavior. For "make", I kind of like that it doesn't depend on environment variables that I am not thinking about when debugging a reported build problem.

Quoted 3 lines

When building Git without autoconf, the corresponding place for settings is config.mak. Would it make sense for the ci scripts to write a config.mak file?

A first I though it doesn't, but it turns out it acually does.

'ci/run-build-and-tests.sh' basically runs:

make
make test

I naively put a 'CC=$CC' argument at the end of the first command, thinking it should Just Work... but then that second 'make test' got all clever on me, said "* new build flags", and then proceeded to rebuild everything with the default 'cc'. (With the additional complication, that on Travis CI we actually run 'make --quiet test', which rebuilds everything, well, quietly... so the rebuild itself is not even visible in the build logs.)

Quoted 4 lines

So, then it's either 'config.mak', or passing a 'CC=$CC' argument to _all_ make commands, including those that are not supposed to build anything, but only run the tests. I find the latter aesthetically not particularly pleasing.

How about using MAKEFLAGS? I ran a quick test:

MAKEFLAGS='CC=blub' make -C .. git.o
make: Entering directory '/usr/src/git/wip'
* new build flags
CC git.o
/bin/sh: blub: command not found

In other words, you could add something like this to the ci/ script:

	MAKEFLAGS="${MAKEFLAGS:+$MAKEFLAGS }CC=$CC"
	export MAKEFLAGS

Ciao,
Dscho

#6 lore raw
In reply to SZEDER Gábor
Quoted 4 lines · SZEDER Gábor

So, then it's either 'config.mak', or passing a 'CC=$CC' argument to _all_ make commands, including those that are not supposed to build anything, but only run the tests. I find the latter aesthetically not particularly pleasing.

The config.mak file is available for individual builder-testers to customize their build, and in the context of this discussion, I think the CI builder is just one particular individual who happens to be non human. If it is easy to throw suitable settings from within the CI configuration .yaml files into config.mak, I'd think that is exactly how the mechanism was invented to be used, so...

#7 lore raw

cover travis-ci: build with the right compiler

In reply to SZEDER Gábor
Quoted 27 lines · SZEDER Gábor

Our Makefile has lines like these:

CFLAGS = -g -O2 -Wall
CC = cc
AR = ar
SPATCH = spatch

Note the use of '=', not '?='. This means that these variables can be overridden from the command line, i.e. 'make CC=gcc-X.Y' will build with that particular GCC version, but not from the environment, i.e. 'CC=gcc-X.Y make' will still build with 'cc'.

This can be confusing for developers who come from other projects where they used to run 'CC=whatever make'.

And our build jobs on Travis CI are badly affected by this. We have dedicated build jobs to build Git with GCC and Clang both on Linux and OSX from the very beginning (522354d70f (Add Travis CI support, 2015-11-27)). But guess how Travis CI specifies which compiler to use! With 'export CC=gcc' and 'export CC=clang', respectively. Consequently, our Clang Linux build job has always used gcc, because that's where 'cc' points at on Linux by default, while the GCC OSX build job has always used Clang. Oh, well. Furthermore, see 37fa4b3c78 (travis-ci: run gcc-8 on linux-gcc jobs, 2018-05-19), where Duy added an 'export CC=gcc-8' in an attempt to use a more modern compiler, but this had no effect either.

Quoted 3 lines

I'm not sure what to do. I'm fine with updating our 'ci/' scripts to explicitly respect CC in the environment (either by running 'make CC=$CC' or by writing $CC into 'config.mak').

So, this patch series, in particular the last patch fixes this issue by setting MAKEFLAGS to contain the right CC from the environment.

The first four patches are necessary cleanups/fixes to make it work, though, arguably, the third patch is neither strictly necessary nor that closely related to this series, but it just happened to bite me while working on this series.

SZEDER Gábor (5):
compat/obstack: fix -Wcast-function-type warnings .gitignore: ignore external debug symbols from GCC on macOS
travis-ci: don't be '--quiet' when running the tests
travis-ci: switch to Xcode 10.1 macOS image
travis-ci: build with the right compiler

 .gitignore                 |  1 +
 .travis.yml                |  2 ++
 ci/install-dependencies.sh |  5 +++++
 ci/lib-travisci.sh         | 15 ++++++++++++---
 ci/run-build-and-tests.sh  |  4 ++--
 ci/run-linux32-build.sh    |  2 +-
 compat/obstack.c           | 17 +++++++++--------
 compat/obstack.h           | 18 +++++++++++-------
 8 files changed, 43 insertions(+), 21 deletions(-)
Signature · 1 line
2.20.1.151.gec613c4b75
#8 lore raw

PATCH 1/5 compat/obstack: fix -Wcast-function-type warnings

When building Git with GCC 8.2.0 (at least from Homebrew on macOS, DEVELOPER flags enabled) one is greeted with a screenful of compiler errors:

compat/obstack.c: In function '_obstack_begin':
compat/obstack.c:162:17: error: cast between incompatible function types from 'void * (*)(long int)' to 'struct _obstack_chunk * (*)(void *, long int)' [-Werror=cast-function-type]

     h->chunkfun = (struct _obstack_chunk * (*)(void *, long)) chunkfun;
                   ^
  compat/obstack.c:163:16: error: cast between incompatible function types from 'void (*)(void *)' to 'void (*)(void *, struct _obstack_chunk *)' [-Werror=cast-function-type]
     h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun;
                  ^
  compat/obstack.c:116:8: error: cast between incompatible function types from 'struct _obstack_chunk * (*)(void *, long int)' to 'struct _obstack_chunk * (*)(long int)' [-Werror=cast-function-type]
      : (*(struct _obstack_chunk *(*) (long)) (h)->chunkfun) ((size)))
          ^
  compat/obstack.c:168:22: note: in expansion of macro 'CALL_CHUNKFUN'
     chunk = h->chunk = CALL_CHUNKFUN (h, h -> chunk_size);
                        ^~~~~~~~~~~~~

<snip>

'struct obstack' stores pointers to two functions to allocate and free "chunks", and depending on how obstack is used, these functions take either one parameter (like standard malloc() and free() do; this is how we use it) or two parameters. Presumably to reduce memory footprint, a single field is used to store the function pointer for both signatures, and then it's casted to the appropriate signature when the function pointer is accessed. These casts between function pointers with different number of parameters are what trigger those compiler errors.

Modify 'struct obstack' to use unions to store function pointers with different signatures, and then use the union member with the appropriate signature when accessing these function pointers. This eliminates the need for those casts, and thus avoids this compiler error.

---
 compat/obstack.c | 17 +++++++++--------
 compat/obstack.h | 18 +++++++++++-------
 2 files changed, 20 insertions(+), 15 deletions(-)
compat/obstack.c +9 -8
index 4d1d95beeb..dd9504c684 100644
@@ -112,15 +112,15 @@ compat_symbol (libc, _obstack_compat, _obstack, GLIBC_2_0);
 
 # define CALL_CHUNKFUN(h, size) \
   (((h) -> use_extra_arg) \
-   ? (*(h)->chunkfun) ((h)->extra_arg, (size)) \
-   : (*(struct _obstack_chunk *(*) (long)) (h)->chunkfun) ((size)))
+   ? (*(h)->chunkfun.fn_extra_arg) ((h)->extra_arg, (size)) \
+   : (*(h)->chunkfun.fn) ((size)))
 
 # define CALL_FREEFUN(h, old_chunk) \
   do { \
     if ((h) -> use_extra_arg) \
-      (*(h)->freefun) ((h)->extra_arg, (old_chunk)); \
+      (*(h)->freefun.fn_extra_arg) ((h)->extra_arg, (old_chunk)); \
     else \
-      (*(void (*) (void *)) (h)->freefun) ((old_chunk)); \
+      (*(h)->freefun.fn) ((old_chunk)); \
   } while (0)
 
 
@@ -159,8 +159,8 @@ _obstack_begin (struct obstack *h,
       size = 4096 - extra;
     }
 
-  h->chunkfun = (struct _obstack_chunk * (*)(void *, long)) chunkfun;
-  h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun;
+  h->chunkfun.fn = chunkfun;
+  h->freefun.fn = freefun;
   h->chunk_size = size;
   h->alignment_mask = alignment - 1;
   h->use_extra_arg = 0;
@@ -206,8 +206,9 @@ _obstack_begin_1 (struct obstack *h, int size, int alignment,
       size = 4096 - extra;
     }
 
-  h->chunkfun = (struct _obstack_chunk * (*)(void *,long)) chunkfun;
-  h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun;
+  h->chunkfun.fn_extra_arg = (struct _obstack_chunk * (*)(void *,long)) chunkfun;
+  h->freefun.fn_extra_arg = (void (*) (void *, struct _obstack_chunk *)) freefun;
+
   h->chunk_size = size;
   h->alignment_mask = alignment - 1;
   h->extra_arg = arg;
compat/obstack.h +11 -7
index 6bc24b7644..0f9b2232a9 100644
@@ -160,11 +160,15 @@ struct obstack		/* control current object in current chunk */
     void *tempptr;
   } temp;			/* Temporary for some macros.  */
   int   alignment_mask;		/* Mask of alignment for each object. */
-  /* These prototypes vary based on `use_extra_arg', and we use
-     casts to the prototypeless function type in all assignments,
-     but having prototypes here quiets -Wstrict-prototypes.  */
-  struct _obstack_chunk *(*chunkfun) (void *, long);
-  void (*freefun) (void *, struct _obstack_chunk *);
+  /* These prototypes vary based on `use_extra_arg'. */
+  union {
+    struct _obstack_chunk *(*fn_extra_arg) (void *, long);
+    void *(*fn) (long);
+  } chunkfun;
+  union {
+    void (*fn_extra_arg) (void *, struct _obstack_chunk *);
+    void (*fn) (void *);
+  } freefun;
   void *extra_arg;		/* first arg for chunk alloc/dealloc funcs */
   unsigned use_extra_arg:1;	/* chunk alloc/dealloc funcs take extra arg */
   unsigned maybe_empty_object:1;/* There is a possibility that the current
@@ -235,10 +239,10 @@ extern void (*obstack_alloc_failed_handler) (void);
 		    (void (*) (void *, void *)) (freefun), (arg))
 
 #define obstack_chunkfun(h, newchunkfun) \
-  ((h) -> chunkfun = (struct _obstack_chunk *(*)(void *, long)) (newchunkfun))
+  ((h)->chunkfun.fn_extra_arg = (struct _obstack_chunk *(*)(void *, long)) (newchunkfun))
 
 #define obstack_freefun(h, newfreefun) \
-  ((h) -> freefun = (void (*)(void *, struct _obstack_chunk *)) (newfreefun))
+  ((h)->freefun.fn_extra_arg = (void (*)(void *, struct _obstack_chunk *)) (newfreefun))
 
 #define obstack_1grow_fast(h,achar) (*((h)->next_free)++ = (achar))
 
Signature · 1 line
2.20.1.151.gec613c4b75
#18 lore raw

PATCH 2/5 .gitignore: ignore external debug symbols from GCC on macOS

In reply to SZEDER Gábor

When Git is build with a "real" GCC on macOS [1], or at least with GCC installed via Homebrew, and CFLAGS includes the '-g' option (and our default CFLAGS does), then by default GCC writes the debug symbols into external files under '<binary>.dSYM/' directories (e.g. 'git-daemon.dSYM/', 'git.dSYM/', etc.).

Update '.gitignore' to ignore these directories, so they don't clutter the output of 'git status'. Furthermore, these build artifacts then won't trigger build failures on Travis CI via b92cb86ea1 (travis-ci:
check that all build artifacts are .gitignore-d, 2017-12-31) once one of the following patches updates our CI build scripts to use a real GCC in the 'osx-gcc' build job.

[1] On macOS the default '/usr/bin/gcc' executable is not a real GCC,

    but merely a compatibility wrapper around Clang:

      $ gcc --version
      Configured with: --prefix=<...>
      Apple LLVM version 9.0.0 (clang-900.0.39.2)
      <...>

    So even though 'make CC=gcc' does indeed execute a command called
    'gcc', in the end Git will be built with Clang all the same.
---
 .gitignore | 1 +
 1 file changed, 1 insertion(+)
.gitignore +1 -0
index 0d77ea5894..a9db568712 100644
@@ -229,3 +229,4 @@
 *.pdb
 /Debug/
 /Release/
+*.dSYM
Signature · 1 line
2.20.1.151.gec613c4b75
#19 lore raw

PATCH 3/5 travis-ci: don't be '--quiet' when running the tests

In reply to SZEDER Gábor

All Travis CI build jobs run the test suite with 'make --quiet test'.

On one hand, being quiet doesn't save us from much clutter in the output:

  $ make test |wc -l
  861
  $ make --quiet test |wc -l
  848

It only spares 13 lines, mostly the output of entering the 't/' directory and the pre- and post-cleanup commands, which is negligible compared to the ~700 lines printed while building Git and the ~850 lines of 'prove' output.

On the other hand, it's asking for trouble. In our CI build scripts we build Git and run the test suite in two separate 'make' invocations. In a prelimiary version of one of the later patches in this series, to explicitly specify which compiler to use, I changed them to basically run:

  make CC=$CC
  make --quiet test

naively thinking that it should Just Work... but then that 'make --quiet test' got all clever on me, noticed the changed build flags, and then proceeded to rebuild everything with the default 'cc'. And because of that '--quiet' option, it did so, well, quietly, only saying "* new build flags", and it was by mere luck that I happened to notice that something is amiss.

Let's just drop that '--quiet' option when running the test suite in all build scripts.

---
 ci/run-build-and-tests.sh | 4 ++--
 ci/run-linux32-build.sh   | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
ci/run-build-and-tests.sh +2 -2
index cda170d5c2..84431c097e 100755
@@ -8,7 +8,7 @@
 ln -s "$cache_dir/.prove" t/.prove
 
 make --jobs=2
-make --quiet test
+make test
 if test "$jobname" = "linux-gcc"
 then
 	export GIT_TEST_SPLIT_INDEX=yes
@@ -17,7 +17,7 @@ then
 	export GIT_TEST_OE_DELTA_SIZE=5
 	export GIT_TEST_COMMIT_GRAPH=1
 	export GIT_TEST_MULTI_PACK_INDEX=1
-	make --quiet test
+	make test
 fi
 
 check_unignored_build_artifacts
ci/run-linux32-build.sh +1 -1
index 2c60d2e70a..26c168a016 100755
@@ -56,5 +56,5 @@ linux32 --32bit i386 su -m -l $CI_USER -c '
 	cd /usr/src/git
 	test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove
 	make --jobs=2
-	make --quiet test
+	make test
 '
Signature · 1 line
2.20.1.151.gec613c4b75
#21 lore raw

PATCH 5/5 travis-ci: build with the right compiler

In reply to SZEDER Gábor

Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'. This can be overridden from the command line, i.e. 'make CC=gcc-X.Y' will build with that particular GCC version, but not from the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever 'cc' happens to be on the platform.

Our build jobs on Travis CI are badly affected by this. In the build matrix we have dedicated build jobs to build Git with GCC and Clang both on Linux and macOS from the very beginning (522354d70f (Add Travis CI support, 2015-11-27)). Alas, this never really worked as supposed to, because Travis CI specifies the compiler for those build jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for projects built with './configure && make'). Consequently, our 'linux-clang' build job has always used GCC, because that's where 'cc' points at in Travis CI's Linux images, while the 'osx-gcc' build job has always used Clang. Furthermore, 37fa4b3c78 (travis-ci: run gcc-8 on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an attempt to build with a more modern compiler, but to no avail.

Set MAKEFLAGS with CC based on the $CC environment variable, so 'make' will run the "right" compiler. The Xcode 10.1 macOS image on Travis CI already contains the gcc@8 package from Homebrew, but we have to 'brew link' it first to be able to use it.

So with this patch our build jobs will build Git with the following compiler versions:

linux-clang: clang version 5.0.0 (tags/RELEASE_500/final)
linux-gcc: gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0

osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
osx-gcc: gcc-8 (Homebrew GCC 8.2.0) 8.2.0

GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

---
 ci/install-dependencies.sh |  5 +++++
 ci/lib-travisci.sh         | 15 ++++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)
ci/install-dependencies.sh +5 -0
index 06c3546e1e..dc719876bb 100755
@@ -40,6 +40,11 @@ osx-clang|osx-gcc)
 	brew install git-lfs gettext
 	brew link --force gettext
 	brew install caskroom/cask/perforce
+	case "$jobname" in
+	osx-gcc)
+		brew link gcc@8
+		;;
+	esac
 	;;
 StaticAnalysis)
 	sudo apt-get -q update
ci/lib-travisci.sh +12 -3
index 69dff4d1ec..a479613a57 100755
@@ -99,12 +99,14 @@ export DEFAULT_TEST_TARGET=prove
 export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
 export GIT_TEST_OPTS="--verbose-log -x --immediate"
 export GIT_TEST_CLONE_2GB=YesPlease
-if [ "$jobname" = linux-gcc ]; then
-	export CC=gcc-8
-fi
 
 case "$jobname" in
 linux-clang|linux-gcc)
+	if [ "$jobname" = linux-gcc ]
+	then
+		export CC=gcc-8
+	fi
+
 	export GIT_TEST_HTTPD=YesPlease
 
 	# The Linux build installs the defined dependency versions below.
@@ -118,6 +120,11 @@ linux-clang|linux-gcc)
 	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
 	;;
 osx-clang|osx-gcc)
+	if [ "$jobname" = osx-gcc ]
+	then
+		export CC=gcc-8
+	fi
+
 	# t9810 occasionally fails on Travis CI OS X
 	# t9816 occasionally fails with "TAP out of sequence errors" on
 	# Travis CI OS X
@@ -127,3 +134,5 @@ GIT_TEST_GETTEXT_POISON)
 	export GIT_TEST_GETTEXT_POISON=YesPlease
 	;;
 esac
+
+export MAKEFLAGS="CC=${CC:-cc}"
Signature · 1 line
2.20.1.151.gec613c4b75
#20 lore raw

PATCH 4/5 travis-ci: switch to Xcode 10.1 macOS image

In reply to SZEDER Gábor

When building something with GCC installed from Homebrew in the default macOS (with Xcode 9.4) image on Travis CI, it errors out with something like this:

/usr/local/Cellar/gcc/8.1.0/lib/gcc/8/gcc/x86_64-apple-darwin17.5.0/8.1.0/include-fixed/stdio.h:78:10: fatal error: _stdio.h: No such file or directory

   #include <_stdio.h>
            ^~~~~~~~~~

This seems to be a common problem affecting several projects, and the common solution is to use a Travis CI macOS image with more recent Xcode version, e.g. 10 or 10.1.

While we don't use such a GCC yet, in the very next patch we will, so switch our OSX build jobs to use the Xcode 10.1 image. Compared to the Xcode 10 image, this has the benefit that it comes with GCC (v8.2) preinstalled from Homebrew.

---
 .travis.yml | 2 ++
 1 file changed, 2 insertions(+)
.travis.yml +2 -0
index 03c8e4c613..36cbdea7f4 100644
@@ -8,6 +8,8 @@ os:
   - linux
   - osx
 
+osx_image: xcode10.1
+
 compiler:
   - clang
   - gcc
Signature · 1 line
2.20.1.151.gec613c4b75
#9 lore raw

Re: [PATCH 1/5] compat/obstack: fix -Wcast-function-type warnings

In reply to SZEDER Gábor
Quoted 17 lines · SZEDER Gábor

When building Git with GCC 8.2.0 (at least from Homebrew on macOS, DEVELOPER flags enabled) one is greeted with a screenful of compiler errors:

compat/obstack.c: In function '_obstack_begin':
compat/obstack.c:162:17: error: cast between incompatible function types from 'void * (*)(long int)' to 'struct _obstack_chunk * (*)(void *, long int)' [-Werror=cast-function-type]

     h->chunkfun = (struct _obstack_chunk * (*)(void *, long)) chunkfun;
                   ^
  compat/obstack.c:163:16: error: cast between incompatible function types from 'void (*)(void *)' to 'void (*)(void *, struct _obstack_chunk *)' [-Werror=cast-function-type]
     h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun;
                  ^
  compat/obstack.c:116:8: error: cast between incompatible function types from 'struct _obstack_chunk * (*)(void *, long int)' to 'struct _obstack_chunk * (*)(long int)' [-Werror=cast-function-type]
      : (*(struct _obstack_chunk *(*) (long)) (h)->chunkfun) ((size)))
          ^
  compat/obstack.c:168:22: note: in expansion of macro 'CALL_CHUNKFUN'
     chunk = h->chunk = CALL_CHUNKFUN (h, h -> chunk_size);
                        ^~~~~~~~~~~~~
Quoted 1 line

<snip>

We originally got this from now-discontinued eglibc, but I notice that glibc.git's malloc/obstack.[ch]'s diff also changes these lines. If you backport those do does that fix this warning?

I.e. is this another case where we're blindly fixing bugs but should just re-import upstream's code instead?

#22 lore raw

Re: [PATCH 5/5] travis-ci: build with the right compiler

In reply to SZEDER Gábor

Hi Gábor,

Quoted 1 line · SZEDER Gábor

Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'. This

... This CC variable ...

Quoted 4 lines

can be overridden from the command line, i.e. 'make CC=gcc-X.Y' will build with that particular GCC version, but not from the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever 'cc' happens to be on the platform.

Without this edit, it read to me as if the commit message claimed that CC cannot be overridden via the environment *at all*, even with MAKEFLAGS.

The rest of the entire patch series looks good to me, I did not dig as deeply as Ævar about that obstack patch, but if there is *some* sort of upstream from where we can get a fix, I think we should try to go for that (rather than risking to diverge even further).

Thanks,
Dscho

Quoted 96 lines

Our build jobs on Travis CI are badly affected by this. In the build matrix we have dedicated build jobs to build Git with GCC and Clang both on Linux and macOS from the very beginning (522354d70f (Add Travis CI support, 2015-11-27)). Alas, this never really worked as supposed to, because Travis CI specifies the compiler for those build jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for projects built with './configure && make'). Consequently, our 'linux-clang' build job has always used GCC, because that's where 'cc' points at in Travis CI's Linux images, while the 'osx-gcc' build job has always used Clang. Furthermore, 37fa4b3c78 (travis-ci: run gcc-8 on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an attempt to build with a more modern compiler, but to no avail.

Set MAKEFLAGS with CC based on the $CC environment variable, so 'make' will run the "right" compiler. The Xcode 10.1 macOS image on Travis CI already contains the gcc@8 package from Homebrew, but we have to 'brew link' it first to be able to use it.

So with this patch our build jobs will build Git with the following compiler versions:

linux-clang: clang version 5.0.0 (tags/RELEASE_500/final)
linux-gcc: gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0

osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
osx-gcc: gcc-8 (Homebrew GCC 8.2.0) 8.2.0

GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

---
 ci/install-dependencies.sh |  5 +++++
 ci/lib-travisci.sh         | 15 ++++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)
ci/install-dependencies.sh +5 -0
index 06c3546e1e..dc719876bb 100755
@@ -40,6 +40,11 @@ osx-clang|osx-gcc)
 	brew install git-lfs gettext
 	brew link --force gettext
 	brew install caskroom/cask/perforce
+	case "$jobname" in
+	osx-gcc)
+		brew link gcc@8
+		;;
+	esac
 	;;
 StaticAnalysis)
 	sudo apt-get -q update
ci/lib-travisci.sh +12 -3
index 69dff4d1ec..a479613a57 100755
@@ -99,12 +99,14 @@ export DEFAULT_TEST_TARGET=prove
 export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
 export GIT_TEST_OPTS="--verbose-log -x --immediate"
 export GIT_TEST_CLONE_2GB=YesPlease
-if [ "$jobname" = linux-gcc ]; then
-	export CC=gcc-8
-fi
 
 case "$jobname" in
 linux-clang|linux-gcc)
+	if [ "$jobname" = linux-gcc ]
+	then
+		export CC=gcc-8
+	fi
+
 	export GIT_TEST_HTTPD=YesPlease
 
 	# The Linux build installs the defined dependency versions below.
@@ -118,6 +120,11 @@ linux-clang|linux-gcc)
 	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
 	;;
 osx-clang|osx-gcc)
+	if [ "$jobname" = osx-gcc ]
+	then
+		export CC=gcc-8
+	fi
+
 	# t9810 occasionally fails on Travis CI OS X
 	# t9816 occasionally fails with "TAP out of sequence errors" on
 	# Travis CI OS X
@@ -127,3 +134,5 @@ GIT_TEST_GETTEXT_POISON)
 	export GIT_TEST_GETTEXT_POISON=YesPlease
 	;;
 esac
+
+export MAKEFLAGS="CC=${CC:-cc}"
Signature · 1 line
2.20.1.151.gec613c4b75
#10 lore raw

Re: [PATCH 1/5] compat/obstack: fix -Wcast-function-type warnings

In reply to Ævar Arnfjörð Bjarmason
Quoted 21 lines · Ævar Arnfjörð Bjarmason
Quoted 17 lines · SZEDER Gábor

When building Git with GCC 8.2.0 (at least from Homebrew on macOS, DEVELOPER flags enabled) one is greeted with a screenful of compiler errors:

compat/obstack.c: In function '_obstack_begin':
compat/obstack.c:162:17: error: cast between incompatible function types from 'void * (*)(long int)' to 'struct _obstack_chunk * (*)(void *, long int)' [-Werror=cast-function-type]

     h->chunkfun = (struct _obstack_chunk * (*)(void *, long)) chunkfun;
                   ^
  compat/obstack.c:163:16: error: cast between incompatible function types from 'void (*)(void *)' to 'void (*)(void *, struct _obstack_chunk *)' [-Werror=cast-function-type]
     h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun;
                  ^
  compat/obstack.c:116:8: error: cast between incompatible function types from 'struct _obstack_chunk * (*)(void *, long int)' to 'struct _obstack_chunk * (*)(long int)' [-Werror=cast-function-type]
      : (*(struct _obstack_chunk *(*) (long)) (h)->chunkfun) ((size)))
          ^
  compat/obstack.c:168:22: note: in expansion of macro 'CALL_CHUNKFUN'
     chunk = h->chunk = CALL_CHUNKFUN (h, h -> chunk_size);
                        ^~~~~~~~~~~~~
Quoted 1 line

<snip>

Quoted 6 lines

We originally got this from now-discontinued eglibc, but I notice that glibc.git's malloc/obstack.[ch]'s diff also changes these lines. If you backport those do does that fix this warning?

I.e. is this another case where we're blindly fixing bugs but should just re-import upstream's code instead?

Good point. I am inclined to queue the remainder of the series without this one for now.

Thanks.

#11 lore raw

Re: [PATCH 1/5] compat/obstack: fix -Wcast-function-type warnings

Quoted 27 lines · Junio C Hamano
Quoted 25 lines · Ævar Arnfjörð Bjarmason
Quoted 17 lines · SZEDER Gábor

When building Git with GCC 8.2.0 (at least from Homebrew on macOS, DEVELOPER flags enabled) one is greeted with a screenful of compiler errors:

compat/obstack.c: In function '_obstack_begin':
compat/obstack.c:162:17: error: cast between incompatible function types from 'void * (*)(long int)' to 'struct _obstack_chunk * (*)(void *, long int)' [-Werror=cast-function-type]

     h->chunkfun = (struct _obstack_chunk * (*)(void *, long)) chunkfun;
                   ^
  compat/obstack.c:163:16: error: cast between incompatible function types from 'void (*)(void *)' to 'void (*)(void *, struct _obstack_chunk *)' [-Werror=cast-function-type]
     h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun;
                  ^
  compat/obstack.c:116:8: error: cast between incompatible function types from 'struct _obstack_chunk * (*)(void *, long int)' to 'struct _obstack_chunk * (*)(long int)' [-Werror=cast-function-type]
      : (*(struct _obstack_chunk *(*) (long)) (h)->chunkfun) ((size)))
          ^
  compat/obstack.c:168:22: note: in expansion of macro 'CALL_CHUNKFUN'
     chunk = h->chunk = CALL_CHUNKFUN (h, h -> chunk_size);
                        ^~~~~~~~~~~~~
Quoted 1 line

<snip>

We originally got this from now-discontinued eglibc, but I notice that glibc.git's malloc/obstack.[ch]'s diff also changes these lines. If you backport those do does that fix this warning?

Quoted 5 lines
Quoted 2 lines

I.e. is this another case where we're blindly fixing bugs but should just re-import upstream's code instead?

Good point. I am inclined to queue the remainder of the series without this one for now.

Note that without this first patch the linux-gcc build job will fail with the above compiler error, and that's the only build job that runs the test suite with all the misc test knobs (split-index, commit-graph, etc.) enabled.

#12 lore raw

Re: [PATCH 1/5] compat/obstack: fix -Wcast-function-type warnings

Quoted 6 lines · SZEDER Gábor
Quoted 5 lines · Junio C Hamano

Ævar Arnfjörð Bjarmason <[email protected]> writes:
...

Quoted 2 lines

I.e. is this another case where we're blindly fixing bugs but should just re-import upstream's code instead?

Quoted 7 lines
Quoted 2 lines

Good point. I am inclined to queue the remainder of the series without this one for now.

Note that without this first patch the linux-gcc build job will fail with the above compiler error, and that's the only build job that runs the test suite with all the misc test knobs (split-index, commit-graph, etc.) enabled.

I know.

The point is to give more incentive to try what was suggested earlier by Ævar (in short, "try to do the right thing, before hacking around locally in this project" ;-)

#13 lore raw

Re: [PATCH 1/5] compat/obstack: fix -Wcast-function-type warnings

Quoted 16 lines · Junio C Hamano
Quoted 6 lines · SZEDER Gábor
Quoted 5 lines · Junio C Hamano

Ævar Arnfjörð Bjarmason <[email protected]> writes:
...

Quoted 2 lines

I.e. is this another case where we're blindly fixing bugs but should just re-import upstream's code instead?

Quoted 7 lines
Quoted 2 lines

Good point. I am inclined to queue the remainder of the series without this one for now.

Note that without this first patch the linux-gcc build job will fail with the above compiler error, and that's the only build job that runs the test suite with all the misc test knobs (split-index, commit-graph, etc.) enabled.

Quoted 5 lines

I know.

The point is to give more incentive to try what was suggested earlier by Ævar (in short, "try to do the right thing, before hacking around locally in this project" ;-)

Well, first I'm not sure what changes Ævar meant to be backported. Back then I briefly glanced at glibc's gitweb [1], but didn't see anything remotely relevant to these compiler errors.

As to re-importing obstack.{c,h} from upstream, we've made some portability fixes to these files, and neither of the commit messages of those fixes mention that they are backports from upstream. OTOH, one of those commits mentions platforms like "i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1, SunOS 5.10", which makes me suspect that the re-import will be susceptible to those portability issues again. Therefore, I think re-importing these files from upstream is beyond the scope of this patch series (and might not be the right thing at all).

[1] https://sourceware.org/git/?p=glibc.git;a=history;f=malloc/obstack.c;h=1669641983512d64f66c1ad659562f77ef48adfd;hb=refs/heads/master

#14 lore raw

Re: [PATCH 1/5] compat/obstack: fix -Wcast-function-type warnings

Quoted 19 lines · SZEDER Gábor
Quoted 18 lines · Junio C Hamano
Quoted 6 lines · SZEDER Gábor
Quoted 5 lines · Junio C Hamano

Ævar Arnfjörð Bjarmason <[email protected]> writes:
...

Quoted 2 lines

I.e. is this another case where we're blindly fixing bugs but should just re-import upstream's code instead?

Quoted 7 lines
Quoted 2 lines

Good point. I am inclined to queue the remainder of the series without this one for now.

Note that without this first patch the linux-gcc build job will fail with the above compiler error, and that's the only build job that runs the test suite with all the misc test knobs (split-index, commit-graph, etc.) enabled.

I know.

Quoted 7 lines
Quoted 3 lines

The point is to give more incentive to try what was suggested earlier by Ævar (in short, "try to do the right thing, before hacking around locally in this project" ;-)

Well, first I'm not sure what changes Ævar meant to be backported. Back then I briefly glanced at glibc's gitweb [1], but didn't see anything remotely relevant to these compiler errors.

So, I looked at the gnulib repository, where glibc got it's obstack.{c,h} from, and it does have a fix for this issue in commit 127ed6a3e (obstack: avoid potentially-nonportable function casts, 2014-11-04):

http://git.savannah.gnu.org/cgit/gnulib.git/commit?id=127ed6a3ea9c46452f079dee50382dc1f70ea796

It chose basically the same approach as my fix, i.e. storing pointers to functions with different signatures in an union. However,, the differences between our and their obstack.{c,h} are way too big to backport their patch.

Quoted 9 lines

As to re-importing obstack.{c,h} from upstream, we've made some portability fixes to these files, and neither of the commit messages of those fixes mention that they are backports from upstream. OTOH, one of those commits mentions platforms like "i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1, SunOS 5.10", which makes me suspect that the re-import will be susceptible to those portability issues again. Therefore, I think re-importing these files from upstream is beyond the scope of this patch series (and might not be the right thing at all).

gnulib's obstack.{c,h} doesn't fix the issues that we've fixed in 3254310863 (obstack.c: Fix some sparse warnings, 2011-09-11) and d190a0875f (obstack: Fix portability issues, 2011-08-28). So if we were to re-import from gnulib, then these two patches would have to be applied on top yet again.

Quoted 2 lines

[1] https://sourceware.org/git/?p=glibc.git;a=history;f=malloc/obstack.c;h=1669641983512d64f66c1ad659562f77ef48adfd;hb=refs/heads/master

#15 lore raw

Re: [PATCH 1/5] compat/obstack: fix -Wcast-function-type warnings

Hi,

Quoted 12 lines · SZEDER Gábor
Quoted 9 lines · SZEDER Gábor

As to re-importing obstack.{c,h} from upstream, we've made some portability fixes to these files, and neither of the commit messages of those fixes mention that they are backports from upstream. OTOH, one of those commits mentions platforms like "i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1, SunOS 5.10", which makes me suspect that the re-import will be susceptible to those portability issues again. Therefore, I think re-importing these files from upstream is beyond the scope of this patch series (and might not be the right thing at all).

Quoted 5 lines

gnulib's obstack.{c,h} doesn't fix the issues that we've fixed in 3254310863 (obstack.c: Fix some sparse warnings, 2011-09-11) and d190a0875f (obstack: Fix portability issues, 2011-08-28). So if we were to re-import from gnulib, then these two patches would have to be applied on top yet again.

Thanks for looking into it. The former looks applicable to upstream, while the latter appears to do some Git-specific things (e.g. relying on git-compat-util.h).

Mind if I send the former upstream? I believe gnulib upstream relies on copyright assignment, so it would help if you have a copyright assignment for the project on file, but if not, they may consider it a small enough change to take without.

Thanks,
Jonathan

#17 lore raw

Re: [PATCH 1/5] compat/obstack: fix -Wcast-function-type warnings

In reply to SZEDER Gábor
Quoted 9 lines · SZEDER Gábor
Quoted 3 lines

Well, first I'm not sure what changes Ævar meant to be backported. Back then I briefly glanced at glibc's gitweb [1], but didn't see anything remotely relevant to these compiler errors.

So, I looked at the gnulib repository, where glibc got it's obstack.{c,h} from, and it does have a fix for this issue in commit 127ed6a3e (obstack: avoid potentially-nonportable function casts, 2014-11-04):

Quoted 6 lines

http://git.savannah.gnu.org/cgit/gnulib.git/commit?id=127ed6a3ea9c46452f079dee50382dc1f70ea796

It chose basically the same approach as my fix, i.e. storing pointers to functions with different signatures in an union. However,, the differences between our and their obstack.{c,h} are way too big to backport their patch.

Thanks. In the meantime, I've queued your fix in my tree, and with your digging, we can say we did due diligence and apply a local fix with conscience ;-).

#23 lore raw

v2 cover travis-ci: build with the right compiler

In reply to SZEDER Gábor

Ever since we started using Travis CI, some of our Travis CI build jobs don't use the compiler they are supposed to. The last patch explains what's going on and fixes this issue. The first four patches are necessary cleanups/fixes to make the fix work.

Changes since v1:

- In patch 1, renamed the fields in the unions holding those function pointers to match how gnulib's obstack calls them

http://git.savannah.gnu.org/cgit/gnulib.git/commit?id=127ed6a3ea9c46452f079dee50382dc1f70ea796

- Clarified the commit messages of patches 1 and 5.

v1: https://public-inbox.org/git/[email protected]/T/#u

SZEDER Gábor (5):
compat/obstack: fix -Wcast-function-type warnings .gitignore: ignore external debug symbols from GCC on macOS
travis-ci: don't be '--quiet' when running the tests
travis-ci: switch to Xcode 10.1 macOS image
travis-ci: build with the right compiler

 .gitignore                 |  1 +
 .travis.yml                |  2 ++
 ci/install-dependencies.sh |  5 +++++
 ci/lib-travisci.sh         | 15 ++++++++++++---
 ci/run-build-and-tests.sh  |  4 ++--
 ci/run-linux32-build.sh    |  2 +-
 compat/obstack.c           | 17 +++++++++--------
 compat/obstack.h           | 18 +++++++++++-------
 8 files changed, 43 insertions(+), 21 deletions(-)

Range-diff:
1: da8f709650 ! 1: be132420c0 compat/obstack: fix -Wcast-function-type warnings

    @@ -2,9 +2,11 @@
     
         compat/obstack: fix -Wcast-function-type warnings
     
    -    When building Git with GCC 8.2.0 (at least from Homebrew on macOS,
    -    DEVELOPER flags enabled) one is greeted with a screenful of compiler
    -    errors:
    +    GCC 8 introduced the new -Wcast-function-type warning, which is
    +    implied by -Wextra (which, in turn is enabled in our DEVELOPER flags).
    +    When building Git with GCC 8 and this warning enabled on a non-glibc
    +    platform [1], one is greeted with a screenful of compiler
    +    warnings/errors:
     
           compat/obstack.c: In function '_obstack_begin':
           compat/obstack.c:162:17: error: cast between incompatible function types from 'void * (*)(long int)' to 'struct _obstack_chunk * (*)(void *, long int)' [-Werror=cast-function-type]
    @@ -24,9 +26,9 @@
         'struct obstack' stores pointers to two functions to allocate and free
         "chunks", and depending on how obstack is used, these functions take
         either one parameter (like standard malloc() and free() do; this is
    -    how we use it) or two parameters.  Presumably to reduce memory
    -    footprint, a single field is used to store the function pointer for
    -    both signatures, and then it's casted to the appropriate signature
    +    how we use it in 'kwset.c') or two parameters.  Presumably to reduce
    +    memory footprint, a single field is used to store the function pointer
    +    for both signatures, and then it's casted to the appropriate signature
         when the function pointer is accessed.  These casts between function
         pointers with different number of parameters are what trigger those
         compiler errors.
    @@ -37,6 +39,10 @@
         eliminates the need for those casts, and thus avoids this compiler
         error.
     
    +    [1] Compiling 'compat/obstack.c' on a platform with glibc is sort of
    +        a noop, see the comment before '#  define ELIDE_CODE', so this is
    +        not an issue on common Linux distros.
    +
         Signed-off-by: SZEDER Gábor <[email protected]>
     
      diff --git a/compat/obstack.c b/compat/obstack.c
    @@ -48,17 +54,17 @@
        (((h) -> use_extra_arg) \
     -   ? (*(h)->chunkfun) ((h)->extra_arg, (size)) \
     -   : (*(struct _obstack_chunk *(*) (long)) (h)->chunkfun) ((size)))
    -+   ? (*(h)->chunkfun.fn_extra_arg) ((h)->extra_arg, (size)) \
    -+   : (*(h)->chunkfun.fn) ((size)))
    ++   ? (*(h)->chunkfun.extra) ((h)->extra_arg, (size)) \
    ++   : (*(h)->chunkfun.plain) ((size)))
      
      # define CALL_FREEFUN(h, old_chunk) \
        do { \
          if ((h) -> use_extra_arg) \
     -      (*(h)->freefun) ((h)->extra_arg, (old_chunk)); \
    -+      (*(h)->freefun.fn_extra_arg) ((h)->extra_arg, (old_chunk)); \
    ++      (*(h)->freefun.extra) ((h)->extra_arg, (old_chunk)); \
          else \
     -      (*(void (*) (void *)) (h)->freefun) ((old_chunk)); \
    -+      (*(h)->freefun.fn) ((old_chunk)); \
    ++      (*(h)->freefun.plain) ((old_chunk)); \
        } while (0)
      
      
    @@ -68,8 +74,8 @@
      
     -  h->chunkfun = (struct _obstack_chunk * (*)(void *, long)) chunkfun;
     -  h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun;
    -+  h->chunkfun.fn = chunkfun;
    -+  h->freefun.fn = freefun;
    ++  h->chunkfun.plain = chunkfun;
    ++  h->freefun.plain = freefun;
        h->chunk_size = size;
        h->alignment_mask = alignment - 1;
        h->use_extra_arg = 0;
    @@ -79,8 +85,8 @@
      
     -  h->chunkfun = (struct _obstack_chunk * (*)(void *,long)) chunkfun;
     -  h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun;
    -+  h->chunkfun.fn_extra_arg = (struct _obstack_chunk * (*)(void *,long)) chunkfun;
    -+  h->freefun.fn_extra_arg = (void (*) (void *, struct _obstack_chunk *)) freefun;
    ++  h->chunkfun.extra = (struct _obstack_chunk * (*)(void *,long)) chunkfun;
    ++  h->freefun.extra = (void (*) (void *, struct _obstack_chunk *)) freefun;
     +
        h->chunk_size = size;
        h->alignment_mask = alignment - 1;
    @@ -100,12 +106,12 @@
     -  void (*freefun) (void *, struct _obstack_chunk *);
     +  /* These prototypes vary based on `use_extra_arg'. */
     +  union {
    -+    struct _obstack_chunk *(*fn_extra_arg) (void *, long);
    -+    void *(*fn) (long);
    ++    void *(*plain) (long);
    ++    struct _obstack_chunk *(*extra) (void *, long);
     +  } chunkfun;
     +  union {
    -+    void (*fn_extra_arg) (void *, struct _obstack_chunk *);
    -+    void (*fn) (void *);
    ++    void (*plain) (void *);
    ++    void (*extra) (void *, struct _obstack_chunk *);
     +  } freefun;
        void *extra_arg;		/* first arg for chunk alloc/dealloc funcs */
        unsigned use_extra_arg:1;	/* chunk alloc/dealloc funcs take extra arg */
    @@ -115,11 +121,11 @@
      
      #define obstack_chunkfun(h, newchunkfun) \
     -  ((h) -> chunkfun = (struct _obstack_chunk *(*)(void *, long)) (newchunkfun))
    -+  ((h)->chunkfun.fn_extra_arg = (struct _obstack_chunk *(*)(void *, long)) (newchunkfun))
    ++  ((h)->chunkfun.extra = (struct _obstack_chunk *(*)(void *, long)) (newchunkfun))
      
      #define obstack_freefun(h, newfreefun) \
     -  ((h) -> freefun = (void (*)(void *, struct _obstack_chunk *)) (newfreefun))
    -+  ((h)->freefun.fn_extra_arg = (void (*)(void *, struct _obstack_chunk *)) (newfreefun))
    ++  ((h)->freefun.extra = (void (*)(void *, struct _obstack_chunk *)) (newfreefun))
      
      #define obstack_1grow_fast(h,achar) (*((h)->next_free)++ = (achar))
2:  1f6be06841 = 2:  983e5a4107 .gitignore: ignore external debug symbols from GCC on macOS
3:  6f6ece8872 = 3:  cc3eb12441 travis-ci: don't be '--quiet' when running the tests
4:  a4179caeba = 4:  2155876a85 travis-ci: switch to Xcode 10.1 macOS image
5:  36323a7c8a ! 5:  c91a5bc7ee travis-ci: build with the right compiler

@@ -3,10 +3,10 @@
travis-ci: build with the right compiler

Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'. This
- can be overridden from the command line, i.e. 'make CC=gcc-X.Y' will
- build with that particular GCC version, but not from the environment,
- i.e. 'CC=gcc-X.Y make' will still build with whatever 'cc' happens to
- be on the platform.
+ CC variable can be overridden from the command line, i.e. 'make
+ CC=gcc-X.Y' will build with that particular GCC version, but not from
+ the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever
+ 'cc' happens to be on the platform.

Our build jobs on Travis CI are badly affected by this. In the build matrix we have dedicated build jobs to build Git with GCC and Clang

Signature · 1 line
2.20.1.499.gf60de1223c
#24 lore raw

PATCH v2 1/5 compat/obstack: fix -Wcast-function-type warnings

GCC 8 introduced the new -Wcast-function-type warning, which is implied by -Wextra (which, in turn is enabled in our DEVELOPER flags). When building Git with GCC 8 and this warning enabled on a non-glibc platform [1], one is greeted with a screenful of compiler warnings/errors:

compat/obstack.c: In function '_obstack_begin':
compat/obstack.c:162:17: error: cast between incompatible function types from 'void * (*)(long int)' to 'struct _obstack_chunk * (*)(void *, long int)' [-Werror=cast-function-type]

     h->chunkfun = (struct _obstack_chunk * (*)(void *, long)) chunkfun;
                   ^
  compat/obstack.c:163:16: error: cast between incompatible function types from 'void (*)(void *)' to 'void (*)(void *, struct _obstack_chunk *)' [-Werror=cast-function-type]
     h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun;
                  ^
  compat/obstack.c:116:8: error: cast between incompatible function types from 'struct _obstack_chunk * (*)(void *, long int)' to 'struct _obstack_chunk * (*)(long int)' [-Werror=cast-function-type]
      : (*(struct _obstack_chunk *(*) (long)) (h)->chunkfun) ((size)))
          ^
  compat/obstack.c:168:22: note: in expansion of macro 'CALL_CHUNKFUN'
     chunk = h->chunk = CALL_CHUNKFUN (h, h -> chunk_size);
                        ^~~~~~~~~~~~~

<snip>

'struct obstack' stores pointers to two functions to allocate and free "chunks", and depending on how obstack is used, these functions take either one parameter (like standard malloc() and free() do; this is how we use it in 'kwset.c') or two parameters. Presumably to reduce memory footprint, a single field is used to store the function pointer for both signatures, and then it's casted to the appropriate signature when the function pointer is accessed. These casts between function pointers with different number of parameters are what trigger those compiler errors.

Modify 'struct obstack' to use unions to store function pointers with different signatures, and then use the union member with the appropriate signature when accessing these function pointers. This eliminates the need for those casts, and thus avoids this compiler error.

[1] Compiling 'compat/obstack.c' on a platform with glibc is sort of a noop, see the comment before '# define ELIDE_CODE', so this is not an issue on common Linux distros.

---
 compat/obstack.c | 17 +++++++++--------
 compat/obstack.h | 18 +++++++++++-------
 2 files changed, 20 insertions(+), 15 deletions(-)
compat/obstack.c +9 -8
index 4d1d95beeb..27cd5c1ea1 100644
@@ -112,15 +112,15 @@ compat_symbol (libc, _obstack_compat, _obstack, GLIBC_2_0);
 
 # define CALL_CHUNKFUN(h, size) \
   (((h) -> use_extra_arg) \
-   ? (*(h)->chunkfun) ((h)->extra_arg, (size)) \
-   : (*(struct _obstack_chunk *(*) (long)) (h)->chunkfun) ((size)))
+   ? (*(h)->chunkfun.extra) ((h)->extra_arg, (size)) \
+   : (*(h)->chunkfun.plain) ((size)))
 
 # define CALL_FREEFUN(h, old_chunk) \
   do { \
     if ((h) -> use_extra_arg) \
-      (*(h)->freefun) ((h)->extra_arg, (old_chunk)); \
+      (*(h)->freefun.extra) ((h)->extra_arg, (old_chunk)); \
     else \
-      (*(void (*) (void *)) (h)->freefun) ((old_chunk)); \
+      (*(h)->freefun.plain) ((old_chunk)); \
   } while (0)
 
 
@@ -159,8 +159,8 @@ _obstack_begin (struct obstack *h,
       size = 4096 - extra;
     }
 
-  h->chunkfun = (struct _obstack_chunk * (*)(void *, long)) chunkfun;
-  h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun;
+  h->chunkfun.plain = chunkfun;
+  h->freefun.plain = freefun;
   h->chunk_size = size;
   h->alignment_mask = alignment - 1;
   h->use_extra_arg = 0;
@@ -206,8 +206,9 @@ _obstack_begin_1 (struct obstack *h, int size, int alignment,
       size = 4096 - extra;
     }
 
-  h->chunkfun = (struct _obstack_chunk * (*)(void *,long)) chunkfun;
-  h->freefun = (void (*) (void *, struct _obstack_chunk *)) freefun;
+  h->chunkfun.extra = (struct _obstack_chunk * (*)(void *,long)) chunkfun;
+  h->freefun.extra = (void (*) (void *, struct _obstack_chunk *)) freefun;
+
   h->chunk_size = size;
   h->alignment_mask = alignment - 1;
   h->extra_arg = arg;
compat/obstack.h +11 -7
index 6bc24b7644..ced94d0118 100644
@@ -160,11 +160,15 @@ struct obstack		/* control current object in current chunk */
     void *tempptr;
   } temp;			/* Temporary for some macros.  */
   int   alignment_mask;		/* Mask of alignment for each object. */
-  /* These prototypes vary based on `use_extra_arg', and we use
-     casts to the prototypeless function type in all assignments,
-     but having prototypes here quiets -Wstrict-prototypes.  */
-  struct _obstack_chunk *(*chunkfun) (void *, long);
-  void (*freefun) (void *, struct _obstack_chunk *);
+  /* These prototypes vary based on `use_extra_arg'. */
+  union {
+    void *(*plain) (long);
+    struct _obstack_chunk *(*extra) (void *, long);
+  } chunkfun;
+  union {
+    void (*plain) (void *);
+    void (*extra) (void *, struct _obstack_chunk *);
+  } freefun;
   void *extra_arg;		/* first arg for chunk alloc/dealloc funcs */
   unsigned use_extra_arg:1;	/* chunk alloc/dealloc funcs take extra arg */
   unsigned maybe_empty_object:1;/* There is a possibility that the current
@@ -235,10 +239,10 @@ extern void (*obstack_alloc_failed_handler) (void);
 		    (void (*) (void *, void *)) (freefun), (arg))
 
 #define obstack_chunkfun(h, newchunkfun) \
-  ((h) -> chunkfun = (struct _obstack_chunk *(*)(void *, long)) (newchunkfun))
+  ((h)->chunkfun.extra = (struct _obstack_chunk *(*)(void *, long)) (newchunkfun))
 
 #define obstack_freefun(h, newfreefun) \
-  ((h) -> freefun = (void (*)(void *, struct _obstack_chunk *)) (newfreefun))
+  ((h)->freefun.extra = (void (*)(void *, struct _obstack_chunk *)) (newfreefun))
 
 #define obstack_1grow_fast(h,achar) (*((h)->next_free)++ = (achar))
 
Signature · 1 line
2.20.1.499.gf60de1223c
#25 lore raw

PATCH v2 2/5 .gitignore: ignore external debug symbols from GCC on macOS

In reply to SZEDER Gábor

When Git is build with a "real" GCC on macOS [1], or at least with GCC installed via Homebrew, and CFLAGS includes the '-g' option (and our default CFLAGS does), then by default GCC writes the debug symbols into external files under '<binary>.dSYM/' directories (e.g. 'git-daemon.dSYM/', 'git.dSYM/', etc.).

Update '.gitignore' to ignore these directories, so they don't clutter the output of 'git status'. Furthermore, these build artifacts then won't trigger build failures on Travis CI via b92cb86ea1 (travis-ci:
check that all build artifacts are .gitignore-d, 2017-12-31) once one of the following patches updates our CI build scripts to use a real GCC in the 'osx-gcc' build job.

[1] On macOS the default '/usr/bin/gcc' executable is not a real GCC,

    but merely a compatibility wrapper around Clang:

      $ gcc --version
      Configured with: --prefix=<...>
      Apple LLVM version 9.0.0 (clang-900.0.39.2)
      <...>

    So even though 'make CC=gcc' does indeed execute a command called
    'gcc', in the end Git will be built with Clang all the same.
---
 .gitignore | 1 +
 1 file changed, 1 insertion(+)
.gitignore +1 -0
index 0d77ea5894..a9db568712 100644
@@ -229,3 +229,4 @@
 *.pdb
 /Debug/
 /Release/
+*.dSYM
Signature · 1 line
2.20.1.499.gf60de1223c
#26 lore raw

PATCH v2 3/5 travis-ci: don't be '--quiet' when running the tests

In reply to SZEDER Gábor

All Travis CI build jobs run the test suite with 'make --quiet test'.

On one hand, being quiet doesn't save us from much clutter in the output:

  $ make test |wc -l
  861
  $ make --quiet test |wc -l
  848

It only spares 13 lines, mostly the output of entering the 't/' directory and the pre- and post-cleanup commands, which is negligible compared to the ~700 lines printed while building Git and the ~850 lines of 'prove' output.

On the other hand, it's asking for trouble. In our CI build scripts we build Git and run the test suite in two separate 'make' invocations. In a prelimiary version of one of the later patches in this series, to explicitly specify which compiler to use, I changed them to basically run:

  make CC=$CC
  make --quiet test

naively thinking that it should Just Work... but then that 'make --quiet test' got all clever on me, noticed the changed build flags, and then proceeded to rebuild everything with the default 'cc'. And because of that '--quiet' option, it did so, well, quietly, only saying "* new build flags", and it was by mere luck that I happened to notice that something is amiss.

Let's just drop that '--quiet' option when running the test suite in all build scripts.

---
 ci/run-build-and-tests.sh | 4 ++--
 ci/run-linux32-build.sh   | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
ci/run-build-and-tests.sh +2 -2
index cda170d5c2..84431c097e 100755
@@ -8,7 +8,7 @@
 ln -s "$cache_dir/.prove" t/.prove
 
 make --jobs=2
-make --quiet test
+make test
 if test "$jobname" = "linux-gcc"
 then
 	export GIT_TEST_SPLIT_INDEX=yes
@@ -17,7 +17,7 @@ then
 	export GIT_TEST_OE_DELTA_SIZE=5
 	export GIT_TEST_COMMIT_GRAPH=1
 	export GIT_TEST_MULTI_PACK_INDEX=1
-	make --quiet test
+	make test
 fi
 
 check_unignored_build_artifacts
ci/run-linux32-build.sh +1 -1
index 2c60d2e70a..26c168a016 100755
@@ -56,5 +56,5 @@ linux32 --32bit i386 su -m -l $CI_USER -c '
 	cd /usr/src/git
 	test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove
 	make --jobs=2
-	make --quiet test
+	make test
 '
Signature · 1 line
2.20.1.499.gf60de1223c
#28 lore raw

PATCH v2 4/5 travis-ci: switch to Xcode 10.1 macOS image

In reply to SZEDER Gábor

When building something with GCC installed from Homebrew in the default macOS (with Xcode 9.4) image on Travis CI, it errors out with something like this:

/usr/local/Cellar/gcc/8.1.0/lib/gcc/8/gcc/x86_64-apple-darwin17.5.0/8.1.0/include-fixed/stdio.h:78:10: fatal error: _stdio.h: No such file or directory

   #include <_stdio.h>
            ^~~~~~~~~~

This seems to be a common problem affecting several projects, and the common solution is to use a Travis CI macOS image with more recent Xcode version, e.g. 10 or 10.1.

While we don't use such a GCC yet, in the very next patch we will, so switch our OSX build jobs to use the Xcode 10.1 image. Compared to the Xcode 10 image, this has the benefit that it comes with GCC (v8.2) preinstalled from Homebrew.

---
 .travis.yml | 2 ++
 1 file changed, 2 insertions(+)
.travis.yml +2 -0
index 03c8e4c613..36cbdea7f4 100644
@@ -8,6 +8,8 @@ os:
   - linux
   - osx
 
+osx_image: xcode10.1
+
 compiler:
   - clang
   - gcc
Signature · 1 line
2.20.1.499.gf60de1223c
#29 lore raw

PATCH v2 5/5 travis-ci: build with the right compiler

In reply to SZEDER Gábor

Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'. This CC variable can be overridden from the command line, i.e. 'make CC=gcc-X.Y' will build with that particular GCC version, but not from the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever 'cc' happens to be on the platform.

Our build jobs on Travis CI are badly affected by this. In the build matrix we have dedicated build jobs to build Git with GCC and Clang both on Linux and macOS from the very beginning (522354d70f (Add Travis CI support, 2015-11-27)). Alas, this never really worked as supposed to, because Travis CI specifies the compiler for those build jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for projects built with './configure && make'). Consequently, our 'linux-clang' build job has always used GCC, because that's where 'cc' points at in Travis CI's Linux images, while the 'osx-gcc' build job has always used Clang. Furthermore, 37fa4b3c78 (travis-ci: run gcc-8 on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an attempt to build with a more modern compiler, but to no avail.

Set MAKEFLAGS with CC based on the $CC environment variable, so 'make' will run the "right" compiler. The Xcode 10.1 macOS image on Travis CI already contains the gcc@8 package from Homebrew, but we have to 'brew link' it first to be able to use it.

So with this patch our build jobs will build Git with the following compiler versions:

linux-clang: clang version 5.0.0 (tags/RELEASE_500/final)
linux-gcc: gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0

osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
osx-gcc: gcc-8 (Homebrew GCC 8.2.0) 8.2.0

GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

---
 ci/install-dependencies.sh |  5 +++++
 ci/lib-travisci.sh         | 15 ++++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)
ci/install-dependencies.sh +5 -0
index 06c3546e1e..dc719876bb 100755
@@ -40,6 +40,11 @@ osx-clang|osx-gcc)
 	brew install git-lfs gettext
 	brew link --force gettext
 	brew install caskroom/cask/perforce
+	case "$jobname" in
+	osx-gcc)
+		brew link gcc@8
+		;;
+	esac
 	;;
 StaticAnalysis)
 	sudo apt-get -q update
ci/lib-travisci.sh +12 -3
index 69dff4d1ec..a479613a57 100755
@@ -99,12 +99,14 @@ export DEFAULT_TEST_TARGET=prove
 export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
 export GIT_TEST_OPTS="--verbose-log -x --immediate"
 export GIT_TEST_CLONE_2GB=YesPlease
-if [ "$jobname" = linux-gcc ]; then
-	export CC=gcc-8
-fi
 
 case "$jobname" in
 linux-clang|linux-gcc)
+	if [ "$jobname" = linux-gcc ]
+	then
+		export CC=gcc-8
+	fi
+
 	export GIT_TEST_HTTPD=YesPlease
 
 	# The Linux build installs the defined dependency versions below.
@@ -118,6 +120,11 @@ linux-clang|linux-gcc)
 	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
 	;;
 osx-clang|osx-gcc)
+	if [ "$jobname" = osx-gcc ]
+	then
+		export CC=gcc-8
+	fi
+
 	# t9810 occasionally fails on Travis CI OS X
 	# t9816 occasionally fails with "TAP out of sequence errors" on
 	# Travis CI OS X
@@ -127,3 +134,5 @@ GIT_TEST_GETTEXT_POISON)
 	export GIT_TEST_GETTEXT_POISON=YesPlease
 	;;
 esac
+
+export MAKEFLAGS="CC=${CC:-cc}"
Signature · 1 line
2.20.1.499.gf60de1223c
#16 lore raw

Re: [PATCH 1/5] compat/obstack: fix -Wcast-function-type warnings

In reply to Jonathan Nieder
Quoted 9 lines · Jonathan Nieder

Hi,

Quoted 5 lines · SZEDER Gábor

gnulib's obstack.{c,h} doesn't fix the issues that we've fixed in 3254310863 (obstack.c: Fix some sparse warnings, 2011-09-11) and d190a0875f (obstack: Fix portability issues, 2011-08-28). So if we were to re-import from gnulib, then these two patches would have to be applied on top yet again.

Quoted 5 lines

Thanks for looking into it. The former looks applicable to upstream, while the latter appears to do some Git-specific things (e.g. relying on git-compat-util.h).

Mind if I send the former upstream?

That's not my patch, so it doesn't matter whether I mind :) Cc-ing Ramsay, to make sure.

Quoted 7 lines

I believe gnulib upstream relies
on copyright assignment, so it would help if you have a copyright assignment for the project on file, but if not, they may consider it a small enough change to take without.

Thanks,
Jonathan

#27 lore raw

Re: [PATCH v2 3/5] travis-ci: don't be '--quiet' when running the tests

In reply to SZEDER Gábor

Hi Gábor,

Quoted 10 lines · SZEDER Gábor

All Travis CI build jobs run the test suite with 'make --quiet test'.

On one hand, being quiet doesn't save us from much clutter in the output:

  $ make test |wc -l
  861
  $ make --quiet test |wc -l
  848
Quoted 8 lines

It only spares 13 lines, mostly the output of entering the 't/' directory and the pre- and post-cleanup commands, which is negligible compared to the ~700 lines printed while building Git and the ~850 lines of 'prove' output.

On the other hand, it's asking for trouble. In our CI build scripts we build Git and run the test suite in two separate 'make' invocations. In a prelimiary version of one of the later patches in

s/prelimiary/preliminary/

Quoted 13 lines

this series, to explicitly specify which compiler to use, I changed them to basically run:

  make CC=$CC
  make --quiet test

naively thinking that it should Just Work... but then that 'make --quiet test' got all clever on me, noticed the changed build flags, and then proceeded to rebuild everything with the default 'cc'. And because of that '--quiet' option, it did so, well, quietly, only saying "* new build flags", and it was by mere luck that I happened to notice that something is amiss.

Quoted 4 lines

Let's just drop that '--quiet' option when running the test suite in all build scripts.

Just like the first two patches, this makes tons of sense to me.

I thank you and will read on,
Dscho

Quoted 42 lines
---
 ci/run-build-and-tests.sh | 4 ++--
 ci/run-linux32-build.sh   | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
ci/run-build-and-tests.sh +2 -2
index cda170d5c2..84431c097e 100755
@@ -8,7 +8,7 @@
 ln -s "$cache_dir/.prove" t/.prove
 
 make --jobs=2
-make --quiet test
+make test
 if test "$jobname" = "linux-gcc"
 then
 	export GIT_TEST_SPLIT_INDEX=yes
@@ -17,7 +17,7 @@ then
 	export GIT_TEST_OE_DELTA_SIZE=5
 	export GIT_TEST_COMMIT_GRAPH=1
 	export GIT_TEST_MULTI_PACK_INDEX=1
-	make --quiet test
+	make test
 fi
 
 check_unignored_build_artifacts
ci/run-linux32-build.sh +1 -1
index 2c60d2e70a..26c168a016 100755
@@ -56,5 +56,5 @@ linux32 --32bit i386 su -m -l $CI_USER -c '
 	cd /usr/src/git
 	test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove
 	make --jobs=2
-	make --quiet test
+	make test
 '
Signature · 1 line
2.20.1.499.gf60de1223c
Reviewed-by on v2 5/5
#30 lore raw

Re: [PATCH v2 5/5] travis-ci: build with the right compiler

In reply to SZEDER Gábor

Hi Gábor,

Quoted 30 lines · SZEDER Gábor

Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'. This CC variable can be overridden from the command line, i.e. 'make CC=gcc-X.Y' will build with that particular GCC version, but not from the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever 'cc' happens to be on the platform.

Our build jobs on Travis CI are badly affected by this. In the build matrix we have dedicated build jobs to build Git with GCC and Clang both on Linux and macOS from the very beginning (522354d70f (Add Travis CI support, 2015-11-27)). Alas, this never really worked as supposed to, because Travis CI specifies the compiler for those build jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for projects built with './configure && make'). Consequently, our 'linux-clang' build job has always used GCC, because that's where 'cc' points at in Travis CI's Linux images, while the 'osx-gcc' build job has always used Clang. Furthermore, 37fa4b3c78 (travis-ci: run gcc-8 on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an attempt to build with a more modern compiler, but to no avail.

Set MAKEFLAGS with CC based on the $CC environment variable, so 'make' will run the "right" compiler. The Xcode 10.1 macOS image on Travis CI already contains the gcc@8 package from Homebrew, but we have to 'brew link' it first to be able to use it.

So with this patch our build jobs will build Git with the following compiler versions:

linux-clang: clang version 5.0.0 (tags/RELEASE_500/final)
linux-gcc: gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0

Quoted 6 lines

osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
osx-gcc: gcc-8 (Homebrew GCC 8.2.0) 8.2.0

GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

Nicely explained and implemented. Thank you!

Apart from the single typo I stumbled over (quite honestly, I do not really care about typos, I just point them out when I see them to beat Eric to it), I have just one more wish: I would *love* to see a Travis run.

Surely you opened a PR at https://github.com/git/git. Oh wait, I can look for it! But no, there does not seem to be one. So probably in your fork,
https://github.com/szeder/git. No PRs. Ah, but there are branches! 60 of them. And yes, I guess I found a Travis build:

	https://travis-ci.org/szeder/git/builds/480654256

But no, the associated branch does not look like it reflects this patch series...

In any case, this series is:

Ciao,
Dscho

Quoted 65 lines
---
 ci/install-dependencies.sh |  5 +++++
 ci/lib-travisci.sh         | 15 ++++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)
ci/install-dependencies.sh +5 -0
index 06c3546e1e..dc719876bb 100755
@@ -40,6 +40,11 @@ osx-clang|osx-gcc)
 	brew install git-lfs gettext
 	brew link --force gettext
 	brew install caskroom/cask/perforce
+	case "$jobname" in
+	osx-gcc)
+		brew link gcc@8
+		;;
+	esac
 	;;
 StaticAnalysis)
 	sudo apt-get -q update
ci/lib-travisci.sh +12 -3
index 69dff4d1ec..a479613a57 100755
@@ -99,12 +99,14 @@ export DEFAULT_TEST_TARGET=prove
 export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
 export GIT_TEST_OPTS="--verbose-log -x --immediate"
 export GIT_TEST_CLONE_2GB=YesPlease
-if [ "$jobname" = linux-gcc ]; then
-	export CC=gcc-8
-fi
 
 case "$jobname" in
 linux-clang|linux-gcc)
+	if [ "$jobname" = linux-gcc ]
+	then
+		export CC=gcc-8
+	fi
+
 	export GIT_TEST_HTTPD=YesPlease
 
 	# The Linux build installs the defined dependency versions below.
@@ -118,6 +120,11 @@ linux-clang|linux-gcc)
 	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
 	;;
 osx-clang|osx-gcc)
+	if [ "$jobname" = osx-gcc ]
+	then
+		export CC=gcc-8
+	fi
+
 	# t9810 occasionally fails on Travis CI OS X
 	# t9816 occasionally fails with "TAP out of sequence errors" on
 	# Travis CI OS X
@@ -127,3 +134,5 @@ GIT_TEST_GETTEXT_POISON)
 	export GIT_TEST_GETTEXT_POISON=YesPlease
 	;;
 esac
+
+export MAKEFLAGS="CC=${CC:-cc}"
Signature · 1 line
2.20.1.499.gf60de1223c
#31 lore raw

Re: [PATCH v2 5/5] travis-ci: build with the right compiler

Quoted 39 lines · Johannes Schindelin

Hi Gábor,

Quoted 35 lines · SZEDER Gábor

Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'. This CC variable can be overridden from the command line, i.e. 'make CC=gcc-X.Y' will build with that particular GCC version, but not from the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever 'cc' happens to be on the platform.

Our build jobs on Travis CI are badly affected by this. In the build matrix we have dedicated build jobs to build Git with GCC and Clang both on Linux and macOS from the very beginning (522354d70f (Add Travis CI support, 2015-11-27)). Alas, this never really worked as supposed to, because Travis CI specifies the compiler for those build jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for projects built with './configure && make'). Consequently, our 'linux-clang' build job has always used GCC, because that's where 'cc' points at in Travis CI's Linux images, while the 'osx-gcc' build job has always used Clang. Furthermore, 37fa4b3c78 (travis-ci: run gcc-8 on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an attempt to build with a more modern compiler, but to no avail.

Set MAKEFLAGS with CC based on the $CC environment variable, so 'make' will run the "right" compiler. The Xcode 10.1 macOS image on Travis CI already contains the gcc@8 package from Homebrew, but we have to 'brew link' it first to be able to use it.

So with this patch our build jobs will build Git with the following compiler versions:

linux-clang: clang version 5.0.0 (tags/RELEASE_500/final)
linux-gcc: gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0

osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
osx-gcc: gcc-8 (Homebrew GCC 8.2.0) 8.2.0

GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

Quoted 7 lines
Quoted 1 line

Nicely explained and implemented. Thank you!

Apart from the single typo I stumbled over (quite honestly, I do not really care about typos, I just point them out when I see them to beat Eric to it)

Heh ;)

Quoted 5 lines

I have just one more wish: I would *love* to see a Travis run.

Surely you opened a PR at https://github.com/git/git. Oh wait, I can look for it! But no, there does not seem to be one.

Nope, "all pull requests are ignored", so I don't bother.

Quoted 3 lines

So probably in your fork,
https://github.com/szeder/git. No PRs. Ah, but there are branches! 60 of them.

Oh, wow, 60, that really needs to be cleaned up and all the already upstreamed, obsolete, or test branches removed. And, more importantly, my WIP branches finished and finally upstreamed... Sigh, maybe someday...

Quoted 3 lines

And yes, I guess I found a Travis build:

	https://travis-ci.org/szeder/git/builds/480654256

That would be the right build, but it was the result of a rebase squashing a fixup commit and editing commit messages while leaving the resulting tree intact, and then 9cc2c76f5e (travis-ci: record and skip successfully built trees, 2017-12-31) kicked in and optimized away the build.

Anyway, I deleted the branch's cache and restarted the build, it will most likely be finished by the time I send this email.

Quoted 2 lines

But no, the associated branch does not look like it reflects this patch series...

That's the right branch, though, except that it contains a debug patch on top (that I didn't submitted) that shows the used compiler and its version from within the Makefile, to make sure that the build jobs do indeed use the compilers that I think they should be using.

The other patches on that branch are the same that were submitted in this patch series.

Here are the most interesting bits of the problematic build jobs, i.e. the compiler versions from that debug patch:

linux-clang: https://travis-ci.org/szeder/git/jobs/480654257#L824
osx-gcc: https://travis-ci.org/szeder/git/jobs/480654264#L824

Quoted 3 lines

In any case, this series is:

Thanks.