Skip to contentSkip to conversation

kbuild: Implicit dependence on the C compiler

DISCUSSION H. Peter Anvin 4 participants 12 messages Jan 17, 2005 – Jan 30, 2005 Kbuild

lore
Share & tools
lore
#0 lore raw

Okay, this is driving me utterly crazy...

How the heck do I get kbuild to *not* think that because I'm using a different C compiler (including "gcc" versus "distcc"), or I'm on a different host, that it has to rebuild every single object file in my directory? This is an unbelievable headache.

-hpa

#3 lore raw
In reply to H. Peter Anvin
Quoted 2 lines · H. Peter Anvin

I don't mind the current default, but saying I shouldn't be able to override it is asinine.

No-one asked for it until now.
Any preferred syntax to disable this dependency check?

Quoted 2 lines

It also means "make install" is largely unusable.

Maybe we should not let install be dependent on vmlinux then?

Sam

#2 lore raw
In reply to Sam Ravnborg
Quoted 4 lines · Sam Ravnborg

It better be difficult. You want to recompile when changing gcc. Try this untested patch.

Sorry, but that's baloney. Saying "it better be difficult" is equivalent to saying "kbuild is smarter than you."

I don't mind the current default, but saying I shouldn't be able to override it is asinine.

It also means "make install" is largely unusable.

Quoted 1 line

There is no way to tell kbuild "ignore gcc change"

There really needs to be one.

-hpa

#1 lore raw
In reply to H. Peter Anvin
Quoted 6 lines · H. Peter Anvin

Okay, this is driving me utterly crazy...

How the heck do I get kbuild to *not* think that because I'm using a different C compiler (including "gcc" versus "distcc"), or I'm on a different host, that it has to rebuild every single object file in my directory? This is an unbelievable headache.

It better be difficult. You want to recompile when changing gcc. Try this untested patch.

There is no way to tell kbuild "ignore gcc change"

Sam

===== Makefile.lib 1.27 vs edited =====

Makefile.lib renamed from scripts/Makefile.lib +3 -9
@@ -187,9 +187,7 @@
 # >'< substitution is for echo to work, >$< substitution to preserve $ when reloading .cmd file
 # note: when using inline perl scripts [perl -e '...$$t=1;...'] in $(cmd_xxx) double $$ your perl vars
 
-if_changed = $(if $(strip $? \
-		          $(filter-out $(cmd_$(1)),$(cmd_$@))\
-			  $(filter-out $(cmd_$@),$(cmd_$(1)))),\
+if_changed = \
 	@set -e; \
 	$(if $($(quiet)cmd_$(1)),echo '  $(subst ','\'',$($(quiet)cmd_$(1)))';) \
 	$(cmd_$(1)); \
@@ -199,9 +197,7 @@
 # execute the command and also postprocess generated .d dependencies
 # file
 
-if_changed_dep = $(if $(strip $? $(filter-out FORCE $(wildcard $^),$^)\
-		          $(filter-out $(cmd_$(1)),$(cmd_$@))\
-			  $(filter-out $(cmd_$@),$(cmd_$(1)))),\
+if_changed_dep = \ 
 	@set -e; \
 	$(if $($(quiet)cmd_$(1)),echo '  $(subst ','\'',$($(quiet)cmd_$(1)))';) \
 	$(cmd_$(1)); \
@@ -213,9 +209,7 @@
 # will check if $(cmd_foo) changed, or any of the prequisites changed,
 # and if so will execute $(rule_foo)
 
-if_changed_rule = $(if $(strip $? \
-		               $(filter-out $(cmd_$(1)),$(cmd_$@))\
-			       $(filter-out $(cmd_$@),$(cmd_$(1)))),\
+if_changed_rule = \
 			@set -e; \
 			$(rule_$(1)))
 
#4 lore raw
In reply to Sam Ravnborg
Quoted 5 lines · Sam Ravnborg
Quoted 1 line · H. Peter Anvin

I don't mind the current default, but saying I shouldn't be able to override it is asinine.

No-one asked for it until now.

That's of course perfectly fair, and I'm not flaming you for saying that noone had asked for it. Saying that it *shouldn't* be done is another matter, that's all.

Quoted 1 line

Any preferred syntax to disable this dependency check?

How about "make CCDEP=0"?

Quoted 4 lines
Quoted 2 lines

It also means "make install" is largely unusable.

Maybe we should not let install be dependent on vmlinux then?

Maybe not. I don't think "modules_install" have any explicit dependencies, it's up to the user to make sure things are already built, so we might as well do the same thing with "make install".

-hpa

#5 lore raw
In reply to H. Peter Anvin

Hi hpa

Quoted 4 lines
Quoted 2 lines

It better be difficult. You want to recompile when changing gcc. Try this untested patch.

Sorry, but that's baloney. Saying "it better be difficult" is equivalent to saying "kbuild is smarter than you."

To give some background info about why kbuild does what it does. A kernel being compiled partly with and partly without say -regparm=3 will result in a non-workable kernel.

The same goes for a kernel that is partly built using gcc 2.96, partly using 3.3.4 for example.

So kbuild pr. default will force a recompile for any .o file where opions to gcc differ, or name of gcc has changed. Today no check has been implemented to check the actual gcc executable timestamp - and neither is this planned.

Default behaviour today is to recompile if anything change.

But as hpa points outs this hits us with nfs mounted kernel tree when performing a make install - because install has vmlinux as prerequsite. So this leaves us with at least two possibilitites:
1) Unconditionally execute make install assuming vmlinux is up-to-date. make modules_install run unconditionally, so this is already know practice
2) Detect that aother user is running the build - and therefore skip the kernel and the module build.
This is a rather intrusive change since with current kbuild structure it is rather difficult to stop this in all relevant cases.

Sam

#6 lore raw
Followup to:  <[email protected]>
By author:    Sam Ravnborg <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 5 lines

To give some background info about why kbuild does what it does. A kernel being compiled partly with and partly without say -regparm=3 will result in a non-workable kernel.

Quoted 8 lines

The same goes for a kernel that is partly built using gcc 2.96, partly using 3.3.4 for example.

So kbuild pr. default will force a recompile for any .o file where opions to gcc differ, or name of gcc has changed. Today no check has been implemented to check the actual gcc executable timestamp - and neither is this planned.

I would argue that "name of gcc has changed" is possibly a condition that does more harm than good. It is just as frequently used to have wrappers, like distcc, as it is to have different versions.

(FWIW, nothing is more obnoxious than having the kernel tree blown away when you try to compile in the one missing driver needed to talk to the rest of your distcc cluster.)

Quoted 12 lines

Default behaviour today is to recompile if anything change.

But as hpa points outs this hits us with nfs mounted kernel tree when performing a make install - because install has vmlinux as prerequsite. So this leaves us with at least two possibilitites:
1) Unconditionally execute make install assuming vmlinux is up-to-date. make modules_install run unconditionally, so this is already know practice
2) Detect that aother user is running the build - and therefore skip the kernel and the module build.
This is a rather intrusive change since with current kbuild structure it is rather difficult to stop this in all relevant cases.

I say unconditionally do make install, but there really, REALLY, need to be a way to override this check manually.

-hpa

#7 lore raw
Quoted 17 lines · H. Peter Anvin
Followup to:  <[email protected]>
By author:    Sam Ravnborg <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 5 lines

To give some background info about why kbuild does what it does. A kernel being compiled partly with and partly without say -regparm=3 will result in a non-workable kernel.

Quoted 8 lines

The same goes for a kernel that is partly built using gcc 2.96, partly using 3.3.4 for example.

So kbuild pr. default will force a recompile for any .o file where opions to gcc differ, or name of gcc has changed. Today no check has been implemented to check the actual gcc executable timestamp - and neither is this planned.

Quoted 3 lines

I would argue that "name of gcc has changed" is possibly a condition that does more harm than good. It is just as frequently used to have wrappers, like distcc, as it is to have different versions.

Disagree. I switch compilers all the time and kbuild does the right thing for me.

I do occassionally feel your 'make install' pain and some sort of 'make __install' might be called for.

Signature · 1 line
Mathematics is the supreme nostalgia of our time.
#8 lore raw
Quoted 11 lines · Matt Mackall
Quoted 4 lines

I would argue that "name of gcc has changed" is possibly a condition that does more harm than good. It is just as frequently used to have wrappers, like distcc, as it is to have different versions.

Disagree. I switch compilers all the time and kbuild does the right thing for me.

I do occassionally feel your 'make install' pain and some sort of 'make __install' might be called for.

As I said, I don't mind the default being there, it's certainly consistent with the default being safe. However, not being able to override it is bad.

-hpa

#9 lore raw
Quoted 11 lines · H. Peter Anvin
Quoted 8 lines · Matt Mackall
Quoted 4 lines

I would argue that "name of gcc has changed" is possibly a condition that does more harm than good. It is just as frequently used to have wrappers, like distcc, as it is to have different versions.

Disagree. I switch compilers all the time and kbuild does the right thing for me.
I do occassionally feel your 'make install' pain and some sort of 'make __install' might be called for.

As I said, I don't mind the default being there, it's certainly consistent with the default being safe. However, not being able to override it is bad.

Just please consider

CC ?= gcc

instead of

CC = gcc

in Makefiles. I assume it does precisely what you want. So I think anybody arguing against you is just arguing about a single ASCII character...

#10 lore raw
In reply to H. Peter Anvin
Quoted 3 lines · H. Peter Anvin
Quoted 1 line

There is no way to tell kbuild "ignore gcc change"

There really needs to be one.

make KBUILD_NOCMDDEP=1
will do what you want - at least I have it in my tree now.
I could not just ignore 'gcc' - but had to ignore the full commandline.

This is due to more complex commands like:
rm -f file; $(LD) ...

Within the Makefile.lib when I check KBUILD_NOCMDDEP there is no knowledge of the actual command being executed. And an implmentation that just filtered out $(CC) was too ugly.
And due to the above mentioned command I could not just skip the first word on the command line.

I will push my bk tree soon and it will show up in next -mm.

It is not perfect in the sense that the last part of the build will get redone (GEN .version and onwards). This is fixable but not worth it right now.

So with current implmentation executing:

make

make KBUILD_NOCMDDEP=1 CROSS_COMPILE=i586-pc-linux-gnu-

will result in only a few files being rebuild - and not the whole kernel as before.

Sam

#11 lore raw
Quoted 8 lines · Sam Ravnborg

make KBUILD_NOCMDDEP=1
will do what you want - at least I have it in my tree now.
I could not just ignore 'gcc' - but had to ignore the full commandline.

This is due to more complex commands like:
rm -f file; $(LD) ...

That's probably the right behaviour actually.

Quoted 13 lines

Within the Makefile.lib when I check KBUILD_NOCMDDEP there is no knowledge of the actual command being executed. And an implmentation that just filtered out $(CC) was too ugly.
And due to the above mentioned command I could not just skip the first word on the command line.

I will push my bk tree soon and it will show up in next -mm.

It is not perfect in the sense that the last part of the build will get redone (GEN .version and onwards). This is fixable but not worth it right now.

Quoted 8 lines

So with current implmentation executing:

make

make KBUILD_NOCMDDEP=1 CROSS_COMPILE=i586-pc-linux-gnu-

will result in only a few files being rebuild - and not the whole kernel as before.

That's fair. You got what you asked for.

-hpa