RFC: auto-generate ARM mach-types.h file from ARM machine database

Makefile +0 -0

@@ -469,7 +469,7 @@ $(obj)System.map: ? $(obj)u-boot

?# This target actually generates 2 files; autoconf.mk and autoconf.mk.dep.
?# the dep file is only include in this top level makefile to determine when
?# to regenerate the autoconf.mk file.
-$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h +$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h $(obj)include/asm/mach-types.h ? ? ? ?@$(XECHO) Generating $@ ; \
? ? ? ?set -e ; \
? ? ? ?: Generate the dependancies ; \

Makefile +0 -0

@@ -530,13 +530,18 @@ unconfig:
 ? ? ? ? ? ? ? ?$(obj)board/*/config.tmp $(obj)board/*/*/config.tmp \

? ? ? ? ? ? ? ?$(obj)include/autoconf.mk $(obj)include/autoconf.mk.dep

-%_config:: ? ? unconfig
+%_config:: ? ? unconfig $(obj)include/asm/mach-types.h ? ? ? ?@$(MKCONFIG) -A $(@:_config=)

?sinclude $(obj).boards.depend
?$(obj).boards.depend: ?boards.cfg
? ? ? ?awk '(NF && $$1 !~ /^#/) { print $$1 ": " $$1 "_config; $$(MAKE)" }' $< > $@

Makefile +5 -0

+
+$(obj)include/asm/mach-types.h: arch/arm/tools/gen-mach-types arch/arm/tools/mach-types
+ ? ? ? @mkdir -p $(obj)include/asm
+ ? ? ? awk -f $^ > $@ || { rm -f $@; /bin/false; }
+

?#
?# Functions to generate common board directory names ?#

Quoted 23 lines

Makefile +2 -2

@@ -469,7 +469,7 @@ $(obj)System.map:   $(obj)u-boot
 # This target actually generates 2 files; autoconf.mk and autoconf.mk.dep.
 # the dep file is only include in this top level makefile to determine when
 # to regenerate the autoconf.mk file.
-$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h
+$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h $(obj)include/asm/mach-types.h
       @$(XECHO) Generating $@ ; \
       set -e ; \
       : Generate the dependancies ; \

@@ -530,13 +530,18 @@ unconfig:
               $(obj)board/*/config.tmp $(obj)board/*/*/config.tmp \
               $(obj)include/autoconf.mk $(obj)include/autoconf.mk.dep
 
-%_config::     unconfig
+%_config::     unconfig $(obj)include/asm/mach-types.h
       @$(MKCONFIG) -A $(@:_config=)
 
 sinclude $(obj).boards.depend
 $(obj).boards.depend:  boards.cfg
       awk '(NF && $$1 !~ /^#/) { print $$1 ": " $$1 "_config; $$(MAKE)" }' $< > $@
 

Makefile unconfig: · line 547

Quoted 8 lines

Makefile +5 -0

+
+$(obj)include/asm/mach-types.h: arch/arm/tools/gen-mach-types arch/arm/tools/mach-types
+       @mkdir -p $(obj)include/asm
+       awk -f $^ > $@ || { rm -f $@; /bin/false; }
+
 #
 # Functions to generate common board directory names
 #

this all belongs in arch/arm/config.mk and not the toplevel makefile

also, dont hardcode full paths to things. there's no reason for it.

might want to add an "update-mach-types" target so people can type make update-mach-types and it'll automatically wget the right file to the right place ...

If I put rules in arch/arm/config.mk, then the first of these rules becomes the default rule which is executed in subdir makes (like "make -C arch"), which breaks compilation completely, since config.mk is included before the rules are defined in the subdir Makefiles.

It seems the current scheme allows only variable definitions in config.mk files, which is not sufficient here.

rules.mk would be fine, however, there is no provision to include rules from lower directories, since all the building in subdirectories is handled by recursively calling make (this is one of the problems that arise by recursively calling make for each directory, but that is a different topic).

Dear Michael Schwingen,

Quoted 4 lines · In message <[email protected]> you wrote:

If I put rules in arch/arm/config.mk, then the first of these rules becomes the default rule which is executed in subdir makes (like "make -C arch"), which breaks compilation completely, since config.mk is included before the rules are defined in the subdir Makefiles.

You are not suppoed to put any make rules in config.mk files.

Quoted 2 lines

It seems the current scheme allows only variable definitions in config.mk files, which is not sufficient here.

As the name implies, these files contain configuration (= variable) settings. Nothing else.

Quoted 5 lines

rules.mk would be fine, however, there is no provision to include rules from lower directories, since all the building in subdirectories is handled by recursively calling make (this is one of the problems that arise by recursively calling make for each directory, but that is a different topic).

But lower level directories inherit all settings from the top level Makefile?

mach-types.h needs to be built before autoconf.mk can be generated, and the rules for autoconf.mk are in the top-level Makefile.

If I put rules in arch/arm/config.mk, then the first of these rules becomes the default rule which is executed in subdir makes (like "make -C arch"), which breaks compilation completely, since config.mk is included before the rules are defined in the subdir Makefiles.

It seems the current scheme allows only variable definitions in config.mk files, which is not sufficient here.

rules.mk would be fine, however, there is no provision to include rules from lower directories, since all the building in subdirectories is handled by recursively calling make (this is one of the problems that arise by recursively calling make for each directory, but that is a different topic).

Any ideas? Using the current Makefile structure, I see no other solution than defining the mach-types.h generation rules in the toplevel Makefile.

Quoted 1 line

also, dont hardcode full paths to things. there's no reason for it.

Which of these can be omitted?

Quoted 3 lines

might want to add an "update-mach-types" target so people can type make update-mach-types and it'll automatically wget the right file to the right place ...

Good idea.
I agree with Wolfgang that this is intended to be used by the maintainer mainly, so the mach-types file should be included in the source so that a normal user does not need to download the file.

Quoted 17 lines · On Saturday, May 07, 2011 06:57:32 Michael Schwingen wrote:

mach-types.h needs to be built before autoconf.mk can be generated, and the rules for autoconf.mk are in the top-level Makefile.

If I put rules in arch/arm/config.mk, then the first of these rules becomes the default rule which is executed in subdir makes (like "make -C arch"), which breaks compilation completely, since config.mk is included before the rules are defined in the subdir Makefiles.

It seems the current scheme allows only variable definitions in config.mk files, which is not sufficient here.

rules.mk would be fine, however, there is no provision to include rules from lower directories, since all the building in subdirectories is handled by recursively calling make (this is one of the problems that arise by recursively calling make for each directory, but that is a different topic).

Quoted 3 lines

Any ideas? Using the current Makefile structure, I see no other solution than defining the mach-types.h generation rules in the toplevel Makefile.

do you need the mach-types file for anything else ? if not, dont keep that in git, keep the generated header.

Then we are back at basically the current state.
If I do this, we need no Makefile support at all - we just need a script that is run by the maintainer that downloads the current mach-types from the web, and generates mach-types.h, which is then checked in.

Quoted 4 lines

Then we are back at basically the current state.
If I do this, we need no Makefile support at all - we just need a script that is run by the maintainer that downloads the current mach-types from the web, and generates mach-types.h, which is then checked in.

i'm ok with that, but i'm not an arm maintainer ;) -mike

to conclude the discussion in the thread "Re: [U-Boot] Update and Cut down mach types", I tried a short patch that demonstrates how to automatically generate the mach-types.h file from a database dump (from
http://www.arm.linux.org.uk/developer/machines/?action=new).

This has multiple advantages:
- pulling in new machine types is easier (drop in a new downloaded database dump), and produces a much smaller diff.
- adding new machines is decoupled from the time they appear in Linux.
- boards that are not in mainline Linux will not be break due to removal of their mach types from the Linux headers.

The AWK and Makefile fragment script is taken verbatim from Linux 2.6.38.3. I think the AWK script is simple enough that it will not require big maintenance efforts (unless the machine database format changes).

The patch is edited down - I removed the diff for the deletion of the old mach-types.h file, and shortened the new mach-types file to a few entries just to show the concept - otherwise, the patch would be much too big for the list.

Is this an acceptable solution? Should I go on and produce a full-fledged patch?

Dear Michael Schwingen,

Quoted 17 lines · In message<[email protected]> you wrote:

to conclude the discussion in the thread "Re: [U-Boot] Update and Cut down mach types", I tried a short patch that demonstrates how to automatically generate the mach-types.h file from a database dump (from
http://www.arm.linux.org.uk/developer/machines/?action=new).

This has multiple advantages:
- pulling in new machine types is easier (drop in a new downloaded database dump), and produces a much smaller diff.
- adding new machines is decoupled from the time they appear in Linux.
- boards that are not in mainline Linux will not be break due to removal of their mach types from the Linux headers.

The AWK and Makefile fragment script is taken verbatim from Linux 2.6.38.3. I think the AWK script is simple enough that it will not require big maintenance efforts (unless the machine database format changes).

Quoted 7 lines

The patch is edited down - I removed the diff for the deletion of the old mach-types.h file, and shortened the new mach-types file to a few entries just to show the concept - otherwise, the patch would be much too big for the list.

Is this an acceptable solution? Should I go on and produce a full-fledged patch?

I agree with the comments alreay made by Mike Frysinger.

In addition, I would like to point out that I consider it mandatory thatthe normal build process does NOT involve any network download. Instead, it must be run completely based on files checked into the git repository only.

That means, that the "update-mach-types" target Mike mentions is something that the ARM maintainer can use to update the in-tree copy of the mach-types.h file, which then gets commited into the git repository. Also users can run this if they want to use the latest version, but it MUST NOT be used for a regular build.

Best regards,

Wolfgang Denk