Fix/workaround broken dependency handling with make 3.81

This patch fixes the following problem:

make[1]: *** No rule to make target hello_world.srec', needed by all'.

The problem has been reported several times before, but so far no patches have been accepted and no one has fixed make. In fact, I can't even find any relevant bug reports against make.

This patch takes a slightly different approach than the other patches I've seen on the mailing list, as it lists the targets explicitly before each of the pattern rules for the elf-, srec- and bin-files. The dependencies for each target are correctly specified, so we don't end up depending on the build order (i.e. the ordering of prerequisites for the `all' rule)

As an added bonus, this gets rid of some duplication as each architecture, cpu, board, etc. only needs to add stuff to the ELF variable. The SREC and BIN variables are calculated automatically from the ELF variable.

---
 examples/Makefile |   32 +++++++++++++-------------------
 1 files changed, 13 insertions(+), 19 deletions(-)

index a342d75..8814f13 100644
@@ -59,51 +59,45 @@ endif
 
 include $(TOPDIR)/config.mk
 
-SREC	= hello_world.srec
-BIN	= hello_world.bin hello_world
+ELF	= hello_world
 
 ifeq ($(CPU),mpc8xx)
-SREC	= test_burst.srec
-BIN	= test_burst.bin test_burst
+ELF	= test_burst
 endif
 
 ifeq ($(ARCH),i386)
-SREC   += 82559_eeprom.srec
-BIN    += 82559_eeprom.bin 82559_eeprom
+ELF    += 82559_eeprom
 endif
 
 ifeq ($(ARCH),ppc)
-SREC   += sched.srec
-BIN    += sched.bin sched
+ELF    += sched
 endif
 
 ifeq ($(ARCH),blackfin)
-SREC	+= smc91111_eeprom.srec
-BIN 	+= smc91111_eeprom.bin smc91111_eeprom
+ELF	+= smc91111
 endif
 
 # The following example is pretty 8xx specific...
 ifeq ($(CPU),mpc8xx)
-SREC   += timer.srec
-BIN    += timer.bin timer
+ELF    += timer
 endif
 
 # The following example is 8260 specific...
 ifeq ($(CPU),mpc8260)
-SREC   += mem_to_mem_idma2intr.srec
-BIN    += mem_to_mem_idma2intr.bin mem_to_mem_idma2intr
+ELF    += mem_to_mem_idma2intr
 endif
 
 # Utility for resetting i82559 EEPROM
 ifeq ($(BOARD),oxc)
-SREC   += eepro100_eeprom.srec
-BIN    += eepro100_eeprom.bin eepro100_eeprom
+ELF    += eepro100_eeprom
 endif
 
 ifeq ($(BIG_ENDIAN),y)
 EX_LDFLAGS += -EB
 endif
 
+SREC	= $(addsuffix .srec,$(ELF))
+BIN	= $(addsuffix .bin,$(ELF))
 OBJS	= $(SREC:.srec=.o)
 
 LIB	= libstubs.a

@@ -128,14 +122,14 @@ ########################################
 $(LIB): .depend $(LIBOBJS)
 	$(AR) crv $@ $(LIBOBJS)
 
-%:	%.o $(LIB)
+$(ELF): %: %.o $(LIB)
 	$(LD) -g $(EX_LDFLAGS) -Ttext $(LOAD_ADDR) \
 		-o $@ -e $(<:.o=) $< $(LIB) \
 		-L$(gcclibdir) -lgcc
-%.srec:	%
+$(SREC): %.srec: %
 	$(OBJCOPY) -O srec $< $@ 2>/dev/null
 
-%.bin:	%
+$(BIN): %.bin: %
 	$(OBJCOPY) -O binary $< $@ 2>/dev/null
 
 #########################################################################

1.4.1.1

Dear Haavard,

I'm sorry, but you're addressing the rong end of the problem here. Please submit a bug report to the make folks, and try to fix make.

Does this mean that you think the current Makefile definition contains any "incorrectly specified" build rules? IF so, I would appreciate if you could point out where you see problems.

On the other hand you lose the ability to have different settings for SREC and BIN targets.

I'm not going to check in this patch, as it only hides a serious problem. If "make" is broken on a level where it's operation depends on the order of target names, then this should be noted, so the use of this broken version can be avoided, and eventually "make" be fixed. Even if we fix this place, we've seen a couple of other cases where similar problems happen with this broken version of make. It IMHO makes no sense to add work arounds for a broken tool.

Let's have the tool fixed instead.

Best regards,

Wolfgang Denk

Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Our OS who art in CPU, UNIX be thy name.
Thy programs run, thy syscalls done,
In kernel as it is in user!

If it's truly a bug in make, sure.

I'm pretty sure the problem is the "catch-all" rule for generating ELF files:

%: %.o $(LIB)

This target matches absolutely anything, and it's not clear from the make documentation that this is a valid rule. In fact, a target which consists only of "%" has a special meaning, as explained in "10.6 Defining Last-Resort Default Rules" in the make manual. It does say that a "catch-all" rule should have no prerequisites, however, so I could be wrong about this.

What kind of settings? I didn't see anything special in the existing makefile.

If you mean different CFLAGS and such, you can still use target- or pattern-specific variables.

This patch does not change the order of target names. If anything, it makes the makefile more robust by specifying exactly what targets you want to apply the pattern rules to.

Sure, if the tool indeed is broken. Now, it's your turn to say what you think `make' is doing wrong ;)

Haavard

Any "make" gurus around to comment?

Section "Overriding Part of Another Makefile" has an example with %:
which looks not too much different to what we do.

I can accept to get error messages in case of Makefile errors, but I think no matter what is wrong ion the Makefile it should never (a) just silently do something wrong without even a warning, and (b) the behaviour of Make must never depend on the order in which targets are given - this alone is for me a pretty strong hint that there is a make bug.

I might want to build foo, foo.srec and foo.bin, but only bar and bar.srec (no bar.bin), or baz and baz.bin (no baz.srec).

It refuses to do what I tell it. That's a major offense ;-)

Seriously: the make result must never depend on the order in which targets are given; and if there is a bug in the Makefile, I expect to see a (helpful) error message instand of random behaviour.

Best regards,

Wolfgang Denk

Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Systems programmers are the high priests of a low cult.
                                                       -- R.S. Barton

That section is about catch-all rules, which I believe is actually quite different from what we intend to do.

It doesn't fail silently -- it complains that it can't find any rules to make hello_world.srec. Although I agree it's a bit worrying that everything is suddenly fine when hello_world is a prerequisite of "all"...

Why? To shave half a second off the build time? ;-)

Anyway, I can make a new patch that assign the ELF, SREC and BIN explicitly. Would that be acceptable?

Now, what _do_ you tell it, though? Basically, you're saying `make' is free to build _anything_ from a .o file and $(LIB). This can actually be quite dangerous -- consider the following Makefile (NOTE: don't run it with files you care about in the same directory, including the Makefile itself):

all: hello_world.tgt

%.tgt: %

        cp $< $@

%: %.src

        cp $< $@

%.src: FORCE
touch $@

.PHONY: FORCE
FORCE:

When executing make, this is what happens (same result with make 3.80 and 3.81):

touch Makefile.src
cp Makefile.src Makefile
rm Makefile.src
make: *** No targets. Stop.

What prevents make from doing this with your Makefile?

Yeah, but the jungle of internal implicit rules, remaking of Makefiles and such is not easy to understand. I for one don't.

I'll send a mail to the make mailinglist tomorrow to see if anyone can give an explanation.

Haavard

You're trying to tenderize me, aren't you? ;-) Yes, probably I'd give in.

Yes, and IMHO you can easily show more problems with make. To me it's really frightening to see so many issues with a tool in so wide use like GNU make.

OK. Let's give it a try:

-> cat Makefile
all: hello_world.tgt

%.tgt: %

        cp $< $@

%: %.src

        cp $< $@

%.src: FORCE
touch $@

.PHONY: FORCE
FORCE:
-> make -n
touch Makefile.src
cp Makefile.src Makefile
rm Makefile.src
make: *** No targets. Stop.
-> ls -l
total 0
-rw-r--r-- 1 wd users 0 Aug 28 23:08 Makefile -rw-r--r-- 1 wd users 0 Aug 28 23:08 Makefile.src -> make --version
GNU Make 3.80
...

So yes, your Makefile is dangerous.

But wait. I used "-n" on the command line. And the man page says:

-n Print the commands that would be executed, but do not
execute them.

Do not execute them? So who did execute the "touch" and the "cp", but skip the "rm"? I'm frightened.

Actually: where is the "rm" coming from?

Neither do I> I don't even understand simple command line options like "-n" :-(

Thanks.

Best regards,

Wolfgang Denk

Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"It's when they say 2 + 2 = 5 that I begin to argue."    - Eric Pepke

Well, make is powerful, so expect it to be complex :-)

I don't have a solution, but some remarks:

from 'info make':

----------8<----------

3.7 How Makefiles Are Remade
============================

[...]

To this end, after reading in all makefiles, `make' will consider
each as a goal target and attempt to update it. If a makefile has a rule which says how to update it (found either in that very makefile or in another one) or if an implicit rule applies to it (*note Using Implicit Rules: Implicit Rules.), it will be updated if necessary.

[...]

When you use the -t' or --touch' option (*note Instead of Executing
the Commands: Instead of Execution.), you would not want to use an out-of-date makefile to decide which targets to touch. So the -t' option has no effect on updating makefiles; they are really updated even if -t' is specified. Likewise, -q' (or --question') and -n' (or --just-print') do not prevent updating of makefiles, because an out-of-date makefile would result in the wrong output for other targets.

----------8<----------

So -n is being ignored for Makefile-remaking, and the "%: %.src" rule is found as a goal target to update the Makefile from Makefile.src, which is not there and thus being touched by using the "%.src: FORCE" rule.

The file you refer to with "%" is an intermediate file, as it is part of a chain. So it is being deleted automatically. If you don't want to do that, declare it .INTERMEDIATE.

Here's a modified version which doens't work as well, but does if you 'make hello_world.src' manually (not the intended behaviour, but it shows some things):

----------8<----------
all: hello_world.tgt

%.tgt: %

	cp $< $@

% :: %.src

	cp $< $@

%.src: FORCE
touch $@

.INTERMEDIATE: hello_world

.PHONY: FORCE
FORCE:
----------8<----------

Note that I've declared % to be terminal, so Makefile is not rebuilt, because there is no Makefile.src.

rsc at isonoe:~/wd$ ll
total 4
-rw-r--r-- 1 rsc ptx 138 Aug 29 01:52 Makefile
-rw-r--r-- 1 rsc ptx 0 Aug 29 01:56 hello_world.src
rsc at isonoe:~/wd$ make
cp hello_world.src hello_world
cp hello_world hello_world.tgt

rsc@isonoe:~/wd$ ls -l 
total 4
-rw-r--r-- 1 rsc ptx 138 Aug 29 01:52 Makefile
-rw-r--r-- 1 rsc ptx   0 Aug 29 01:56 hello_world
-rw-r--r-- 1 rsc ptx   0 Aug 29 01:56 hello_world.src
-rw-r--r-- 1 rsc ptx   0 Aug 29 01:56 hello_world.tgt

Anyway, can't you avoid the match-anything rule somehow?

Robert

 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
     Hannoversche Str. 2, 31134 Hildesheim, Germany
   Phone: +49-5121-206917-0 |  Fax: +49-5121-206917-9

No, I'm trying to solve a problem. You said you were having a problem with that particular aspect of the patch, so I'm offering to fix it.

The real problem I'm trying to fix is that make 3.81 fails to build u-boot. I don't know whether that is caused by a bug in make or a bug in the Makefile, but I think my patch has merit either way, as catch-all rules may do unexpected things with earlier versions of make as well.

Updated patch below. I messed up the blackfin case in the previous patch, so that has been fixed. I've also added $(ELF) to the list of prerequisites for "all", as it was previously part of $(BIN).

IMHO many of the problems with GNU make are caused by the built-in rules. That's probably part of the old make legacy, and is not possible to fix without breaking lots of old Makefiles...

I'll let you know when I receive a reply.

Haavard

Applied, thanks.

Best regards,

Wolfgang Denk

Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Es ist nicht genug zu wissen, man mu? auch anwenden; es ist nicht ge-
nug zu wollen, man mu? auch tun.   -- Goethe, Maximen und Reflexionen