Skip to contentSkip to conversation

handling of _shipped files

PATCH In review Nicholas Mc Guire 2 participants 3 messages Oct 1, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/3 handling of _shipped files

handling of _shipped files

This adds a section on handling of _shipped files in kbuild Makefiles and the build process.

---
 Documentation/kbuild/makefiles.txt |   57 +++++++++++++++++++++++++++++++++++-
 1 file changed, 56 insertions(+), 1 deletion(-)
Documentation/kbuild/makefiles.txt +56 -1
index 764f599..d0bd91a 100644
@@ -19,6 +19,7 @@ This document describes the Linux kernel Makefiles.
 	   --- 3.10 Special Rules
 	   --- 3.11 $(CC) support functions
 	   --- 3.12 $(LD) support functions
+	   --- 3.13 _shipped - generated files
 
 	=== 4 Host Program support
 	   --- 4.1 Simple Host Program
@@ -595,6 +596,60 @@ more details, with real examples.
 		LDFLAGS_vmlinux += $(call ld-option, -X)
 
 
+--- 3.13 _shipped - generated files
+
+   Some sources in the kernel are generated by tools or special
+   purpose scrips so that the build process dependencies can be kept
+   at a resonable level. Also in cases where the translation is not
+   configuration dependent but might impact build time _shipped files
+   may be added along with the actual sources to speed up the build
+   process.
+
+   For standard tools like lex and yacc the make process will actually
+   generate the _shipped files (see scripts/Makefile.lib) and retain
+   them by marking them .PRECIOUS so they are not removed by the clean
+   targets.
+
+   For non standard tools the generation of the _shipped file it self
+   may be manually achived by building and then renaming. A makefile
+   target for generate the actual target file should to be provided
+   though and can be conditioned on an environment variable.
+
+	Example:
+		ifdef GENERATE_KEYMAP
+
+		$(obj)/defkeymap.c: $(obj)/%.c: $(src)/%.map
+			loadkeys --mktable $< > $@
+
+		endif
+
+   The _shipped file does not appear in the actual Makefile, this is
+   handled by the rules found in scripts/Makefile.lib.
+
+		<snip>
+		cmd_shipped = cat $< > $@
+
+		$(obj)/%: $(src)/%_shipped
+			$(call cmd,shipped)
+		<snip>
+
+   which simply will copy the target_shipped to the target
+
+   If a file of a make target is found in the build directory with the
+   extension _shipped make will used it, rather than generating the
+   respective target file from source.
+
+   The build will indicate the use of the _shipped file by the SHIPPED
+   quiet command in the make output.
+
+	Example:
+		LD      drivers/scsi/aic7xxx/built-in.o
+		SHIPPED drivers/scsi/aic7xxx/aic79xx_seq.h
+		SHIPPED drivers/scsi/aic7xxx/aic79xx_reg.h
+		CC [M]  drivers/scsi/aic7xxx/aic79xx_core.o
+		CC [M]  drivers/scsi/aic7xxx/aic79xx_pci.o
+		SHIPPED drivers/scsi/aic7xxx/aic79xx_reg_print.c
+		CC [M]  drivers/scsi/aic7xxx/aic79xx_reg_print.o
 === 4 Host Program support
 
 Kbuild supports building executables on the host for use during the
@@ -1423,10 +1478,10 @@ Original version made by Michael Elizabeth Chastain, <mailto:[email protected]>
 Updates by Kai Germaschewski <[email protected]>
 Updates by Sam Ravnborg <[email protected]>
 Language QA by Jan Engelhardt <[email protected]>
+Kbuild support for shipped files Nicholas Mc Guire <[email protected]>
 
 === 11 TODO
 
-- Describe how kbuild supports shipped files with _shipped.
 - Generating offset header files.
 - Add more variables to section 7?
 
Signature · 1 line
1.7.10.4
reviewer
#1 lore raw
Quoted 24 lines · Nicholas Mc Guire

handling of _shipped files

This adds a section on handling of _shipped files in kbuild Makefiles and the build process.

---
 Documentation/kbuild/makefiles.txt |   57 +++++++++++++++++++++++++++++++++++-
 1 file changed, 56 insertions(+), 1 deletion(-)
Documentation/kbuild/makefiles.txt +1 -0
index 764f599..d0bd91a 100644
@@ -19,6 +19,7 @@ This document describes the Linux kernel Makefiles.
 	   --- 3.10 Special Rules
 	   --- 3.11 $(CC) support functions
 	   --- 3.12 $(LD) support functions
+	   --- 3.13 _shipped - generated files
 
 	=== 4 Host Program support
 	   --- 4.1 Simple Host Program

Documentation/kbuild/makefiles.txt more details, with real examples. · line 602

Quoted 7 lines
Documentation/kbuild/makefiles.txt +4 -0
@@ -595,6 +596,60 @@ more details, with real examples.
 		LDFLAGS_vmlinux += $(call ld-option, -X)
 
 
+--- 3.13 _shipped - generated files
+
+   Some sources in the kernel are generated by tools or special
+   purpose scrips so that the build process dependencies can be kept

scripts

Documentation/kbuild/makefiles.txt more details, with real examples. · line 603

Quoted 1 line
Documentation/kbuild/makefiles.txt +1 -0
+   at a resonable level. Also in cases where the translation is not

reasonable

Documentation/kbuild/makefiles.txt more details, with real examples. · line 604

Quoted 1 line
Documentation/kbuild/makefiles.txt +1 -0
+   configuration dependent but might impact build time _shipped files

build time,

Documentation/kbuild/makefiles.txt more details, with real examples. · line 613

Quoted 9 lines
Documentation/kbuild/makefiles.txt +9 -0
+   may be added along with the actual sources to speed up the build
+   process.
+
+   For standard tools like lex and yacc the make process will actually
+   generate the _shipped files (see scripts/Makefile.lib) and retain
+   them by marking them .PRECIOUS so they are not removed by the clean
+   targets.
+
+   For non standard tools the generation of the _shipped file it self

non-standard itself

Documentation/kbuild/makefiles.txt more details, with real examples. · line 614

Quoted 1 line
Documentation/kbuild/makefiles.txt +1 -0
+   may be manually achived by building and then renaming. A makefile

achieved

Documentation/kbuild/makefiles.txt more details, with real examples. · line 615

Quoted 1 line
Documentation/kbuild/makefiles.txt +1 -0
+   target for generate the actual target file should to be provided

to generate

Documentation/kbuild/makefiles.txt more details, with real examples. · line 616

Quoted 1 line
Documentation/kbuild/makefiles.txt +1 -0
+   though and can be conditioned on an environment variable.

Most examples that I see just expect the Makefile to be modified to set (or define) the variable.

Quoted 13 lines
Documentation/kbuild/makefiles.txt +13 -0
+
+	Example:
+		ifdef GENERATE_KEYMAP
+
+		$(obj)/defkeymap.c: $(obj)/%.c: $(src)/%.map
+			loadkeys --mktable $< > $@
+
+		endif
+
+   The _shipped file does not appear in the actual Makefile, this is
+   handled by the rules found in scripts/Makefile.lib.
+
+		<snip>

Documentation/kbuild/makefiles.txt more details, with real examples. · line 639

Quoted 10 lines
Documentation/kbuild/makefiles.txt +10 -0
+		cmd_shipped = cat $< > $@
+
+		$(obj)/%: $(src)/%_shipped
+			$(call cmd,shipped)
+		<snip>
+
+   which simply will copy the target_shipped to the target
+
+   If a file of a make target is found in the build directory with the
+   extension _shipped make will used it, rather than generating the

_shipped, make will use it rather than generating the

Quoted 29 lines
Documentation/kbuild/makefiles.txt +14 -1
+   respective target file from source.
+
+   The build will indicate the use of the _shipped file by the SHIPPED
+   quiet command in the make output.
+
+	Example:
+		LD      drivers/scsi/aic7xxx/built-in.o
+		SHIPPED drivers/scsi/aic7xxx/aic79xx_seq.h
+		SHIPPED drivers/scsi/aic7xxx/aic79xx_reg.h
+		CC [M]  drivers/scsi/aic7xxx/aic79xx_core.o
+		CC [M]  drivers/scsi/aic7xxx/aic79xx_pci.o
+		SHIPPED drivers/scsi/aic7xxx/aic79xx_reg_print.c
+		CC [M]  drivers/scsi/aic7xxx/aic79xx_reg_print.o
 === 4 Host Program support
 
 Kbuild supports building executables on the host for use during the
@@ -1423,10 +1478,10 @@ Original version made by Michael Elizabeth Chastain, <mailto:[email protected]>
 Updates by Kai Germaschewski <[email protected]>
 Updates by Sam Ravnborg <[email protected]>
 Language QA by Jan Engelhardt <[email protected]>
+Kbuild support for shipped files Nicholas Mc Guire <[email protected]>
 
 === 11 TODO
 
-- Describe how kbuild supports shipped files with _shipped.
 - Generating offset header files.
 - Add more variables to section 7?
 
Signature · 1 line
~Randy
#2 lore raw
Quoted 59 lines · Randy Dunlap
Quoted 24 lines · Nicholas Mc Guire

handling of _shipped files

This adds a section on handling of _shipped files in kbuild Makefiles and the build process.

---
 Documentation/kbuild/makefiles.txt |   57 +++++++++++++++++++++++++++++++++++-
 1 file changed, 56 insertions(+), 1 deletion(-)
Documentation/kbuild/makefiles.txt +1 -0
index 764f599..d0bd91a 100644
@@ -19,6 +19,7 @@ This document describes the Linux kernel Makefiles.
 	   --- 3.10 Special Rules
 	   --- 3.11 $(CC) support functions
 	   --- 3.12 $(LD) support functions
+	   --- 3.13 _shipped - generated files
 
 	=== 4 Host Program support
 	   --- 4.1 Simple Host Program

Documentation/kbuild/makefiles.txt more details, with real examples. · line 602

Quoted 7 lines
Documentation/kbuild/makefiles.txt +4 -0
@@ -595,6 +596,60 @@ more details, with real examples.
 		LDFLAGS_vmlinux += $(call ld-option, -X)
 
 
+--- 3.13 _shipped - generated files
+
+   Some sources in the kernel are generated by tools or special
+   purpose scrips so that the build process dependencies can be kept

scripts

Documentation/kbuild/makefiles.txt more details, with real examples. · line 603

Quoted 1 line
Documentation/kbuild/makefiles.txt +1 -0
+   at a resonable level. Also in cases where the translation is not

reasonable

Documentation/kbuild/makefiles.txt more details, with real examples. · line 604

Quoted 1 line
Documentation/kbuild/makefiles.txt +1 -0
+   configuration dependent but might impact build time _shipped files

build time,

Documentation/kbuild/makefiles.txt more details, with real examples. · line 613

Quoted 9 lines
Documentation/kbuild/makefiles.txt +9 -0
+   may be added along with the actual sources to speed up the build
+   process.
+
+   For standard tools like lex and yacc the make process will actually
+   generate the _shipped files (see scripts/Makefile.lib) and retain
+   them by marking them .PRECIOUS so they are not removed by the clean
+   targets.
+
+   For non standard tools the generation of the _shipped file it self

non-standard itself

Documentation/kbuild/makefiles.txt more details, with real examples. · line 614

Quoted 1 line
Documentation/kbuild/makefiles.txt +1 -0
+   may be manually achived by building and then renaming. A makefile

achieved

Quoted 9 lines

Documentation/kbuild/makefiles.txt more details, with real examples. · line 615

Quoted 1 line
Documentation/kbuild/makefiles.txt +1 -0
+   target for generate the actual target file should to be provided

to generate

Documentation/kbuild/makefiles.txt more details, with real examples. · line 616

Quoted 1 line
Documentation/kbuild/makefiles.txt +1 -0
+   though and can be conditioned on an environment variable.

Most examples that I see just expect the Makefile to be modified to set (or define) the variable.

Yes - but is that the prefered way of doing it ? I guess it would be more clear to have a variable that is taken from the environment and not via .config to trigger a conditional rebuild as the defkeymap example does it.

Quoted 48 lines
Quoted 13 lines
Documentation/kbuild/makefiles.txt +13 -0
+
+	Example:
+		ifdef GENERATE_KEYMAP
+
+		$(obj)/defkeymap.c: $(obj)/%.c: $(src)/%.map
+			loadkeys --mktable $< > $@
+
+		endif
+
+   The _shipped file does not appear in the actual Makefile, this is
+   handled by the rules found in scripts/Makefile.lib.
+
+		<snip>

Documentation/kbuild/makefiles.txt more details, with real examples. · line 639

Quoted 10 lines
Documentation/kbuild/makefiles.txt +10 -0
+		cmd_shipped = cat $< > $@
+
+		$(obj)/%: $(src)/%_shipped
+			$(call cmd,shipped)
+		<snip>
+
+   which simply will copy the target_shipped to the target
+
+   If a file of a make target is found in the build directory with the
+   extension _shipped make will used it, rather than generating the

_shipped, make will use it rather than generating the

Quoted 22 lines
Documentation/kbuild/makefiles.txt +14 -0
+   respective target file from source.
+
+   The build will indicate the use of the _shipped file by the SHIPPED
+   quiet command in the make output.
+
+	Example:
+		LD      drivers/scsi/aic7xxx/built-in.o
+		SHIPPED drivers/scsi/aic7xxx/aic79xx_seq.h
+		SHIPPED drivers/scsi/aic7xxx/aic79xx_reg.h
+		CC [M]  drivers/scsi/aic7xxx/aic79xx_core.o
+		CC [M]  drivers/scsi/aic7xxx/aic79xx_pci.o
+		SHIPPED drivers/scsi/aic7xxx/aic79xx_reg_print.c
+		CC [M]  drivers/scsi/aic7xxx/aic79xx_reg_print.o
 === 4 Host Program support
 
 Kbuild supports building executables on the host for use during the
@@ -1423,10 +1478,10 @@ Original version made by Michael Elizabeth Chastain, <mailto:[email protected]>
 Updates by Kai Germaschewski <[email protected]>
 Updates by Sam Ravnborg <[email protected]>
 Language QA by Jan Engelhardt <[email protected]>
+Kbuild support for shipped files Nicholas Mc Guire <[email protected]>
 
Quoted 8 lines

Documentation/kbuild/makefiles.txt Original version made by Michael Elizabeth Chastain, <mailto:[email protected]> · line 1487

Quoted 7 lines
Documentation/kbuild/makefiles.txt +0 -1
 === 11 TODO
 
-- Describe how kbuild supports shipped files with _shipped.
 - Generating offset header files.
 - Add more variables to section 7?
 

and thanks for the language fixes - thought I had spellchecked it but obviously I did not

thx!
hofrat