x86 disk image and modules initramfs generation
H. Peter Anvin
reviewercover x86 disk image and modules initramfs generation
From: "H. Peter Anvin" (Intel) <[email protected]>
When compiling on a different machine than the runtime target, including but not limited to simulators, it is rather handy to be able to produce a bootable image. The scripts for that in x86 are relatively old, and assume a BIOS system.
This adds a build target to generate a hdimage which can be booted either from BIOS or EFI, and modernizes the genimage.sh script including adding the ability to add an arbitrary number of initramfs files (limited only by the length of the command line.)
Possibly more controversial, at least from a Kbuild design perspective (as usual I'm the guy who wants to do something with Kbuild which it seems it was never really designed to do), is add the ability to create an initramfs image which includes all the built modules. Some distributions cannot be easily booted without modules in initramfs, and this creates an image which can be added to initramfs to provide the kernel modules, as finalized by "make modules_install".
The final patch put these two together, and allows the modules initramfs to be included in the x86 boot image.
Makefile | 17 ++- arch/x86/Makefile | 8 +- arch/x86/boot/.gitignore | 1 + arch/x86/boot/Makefile | 55 +++++---- arch/x86/boot/genimage.sh | 284 +++++++++++++++++++++++++++++++------------ arch/x86/boot/mtools.conf.in | 3 + usr/.gitignore | 3 + usr/Kconfig | 31 ++--- usr/Makefile | 39 +++++- 9 files changed, 318 insertions(+), 123 deletions(-)
H. Peter Anvin
reviewerPATCH 1/3 x86/boot: Modernize genimage script; hdimage support; remove bzlilo
From: "H. Peter Anvin" (Intel) <[email protected]>
The image generation scripts in arch/x86/boot are pretty out of date, except for the isoimage target. Update and clean up the genimage.sh script, and make it support an arbitrary number of initramfs files in the image.
Add a "hdimage" target, which can be booted by either BIOS or EFI (if the kernel is compiled with the EFI stub.) For EFI to be able to pass the command line to the kernel, we need the EFI shell, but the firmware builtin EFI shell, if it even exists, is pretty much always the last resort boot option, so search for OVMF or EDK2 and explicitly include a copy of the EFI shell.
To make this all work, use bash features in the script. Furthermore, this version of the script makes use of some mtools features, especially mpartition, that might not exist in very old version of mtools, but given all the other dependencies on this script this doesn't seem such a big deal.
--- arch/x86/Makefile | 5 +- arch/x86/boot/.gitignore | 1 + arch/x86/boot/Makefile | 44 +++--- arch/x86/boot/genimage.sh | 284 +++++++++++++++++++++++++---------- arch/x86/boot/mtools.conf.in | 3 + 5 files changed, 238 insertions(+), 99 deletions(-)
arch/x86/Makefile +3 -2
@@ -247,7 +247,7 @@ drivers-$(CONFIG_FB) += arch/x86/video/ boot := arch/x86/boot -BOOT_TARGETS = bzdisk fdimage fdimage144 fdimage288 isoimage +BOOT_TARGETS = bzdisk fdimage fdimage144 fdimage288 hdimage isoimage PHONY += bzImage $(BOOT_TARGETS)
@@ -305,8 +305,9 @@ define archhelp echo ' fdimage - Create 1.4MB boot floppy image (arch/x86/boot/fdimage)' echo ' fdimage144 - Create 1.4MB boot floppy image (arch/x86/boot/fdimage)' echo ' fdimage288 - Create 2.8MB boot floppy image (arch/x86/boot/fdimage)' + echo ' hdimage - Create a BIOS/EFI hard disk image (arch/x86/boot/hdimage)' echo ' isoimage - Create a boot CD-ROM image (arch/x86/boot/image.iso)' - echo ' bzdisk/fdimage*/isoimage also accept:' + echo ' bzdisk/fdimage*/hdimage/isoimage also accept:' echo ' FDARGS="..." arguments for the booted kernel' echo ' FDINITRD=file initrd for the booted kernel' echo ''
arch/x86/boot/.gitignore +1 -0
@@ -11,3 +11,4 @@ setup.elf fdimage mtools.conf image.iso +hdimage
arch/x86/boot/Makefile +23 -21
77 lines not on this page · raw message
arch/x86/boot/genimage.sh +208 -76
332 lines not on this page · raw message
arch/x86/boot/mtools.conf.in +3 -0
@@ -14,4 +14,7 @@ drive v: drive w: file="@OBJ@/fdimage" cylinders=80 heads=2 sectors=36 filter +# Hard disk +drive h: + file="@OBJ@/hdimage" partition=1 mformat_only
Signature · 1 line
2.30.2
H. Peter Anvin
reviewerPATCH 2/3 usr, modules: Add build target for creating a modules initramfs image
In reply to H. Peter AnvinFrom: "H. Peter Anvin" (Intel) <[email protected]>
Some distributions really cannot be booted without modules anymore. To allow an externally built kernel to be run, it is handy to be able to create an initramfs image with all the modules, that can appended to an existing initramfs image (preferrably without any modules, but not necessarily, since the /lib/modules path includes the kernel version.)
This image is generated in usr/modules.img.
Since this image may end up being rather large, change the default for CONFIG_INITRAMFS_COMPRESSION_*. We already have a special case in the Makefile to not re-compress an existing initramfs image; add the very small default builtin initramfs as another case to suppress the compression. To avoid losing the default compression setting, add a separate variable for the possibly overridden builtin initramfs compression method.
--- Makefile | 17 ++++++++++++----- usr/.gitignore | 3 +++ usr/Kconfig | 31 +++++++++++++++++-------------- usr/Makefile | 39 +++++++++++++++++++++++++++++++++++---- 4 files changed, 67 insertions(+), 23 deletions(-)
Makefile +12 -5
@@ -622,7 +622,7 @@ KBUILD_MODULES := KBUILD_BUILTIN := 1 # If we have only "make modules", don't compile built-in objects. -ifeq ($(MAKECMDGOALS),modules) +ifeq (modules,$(MAKECMDGOALS)) KBUILD_BUILTIN := endif
@@ -630,7 +630,8 @@ endif # in addition to whatever we do anyway. # Just "make" or "make all" shall build modules as well -ifneq ($(filter all modules nsdeps %compile_commands.json clang-%,$(MAKECMDGOALS)),) +ifneq ($(filter all modules nsdeps %compile_commands.json clang-% \ + modules.img usr/modules.img,$(MAKECMDGOALS)),) KBUILD_MODULES := 1 endif
@@ -1461,15 +1462,17 @@ modules_prepare: prepare PHONY += modules_install modules_install: _modinst_ _modinst_post +INSTALL_MOD_LN ?= ln -s + PHONY += _modinst_ _modinst_: @rm -rf $(MODLIB)/kernel @rm -f $(MODLIB)/source @mkdir -p $(MODLIB)/kernel - @ln -s $(abspath $(srctree)) $(MODLIB)/source + @$(INSTALL_MOD_LN) $(abspath $(srctree)) $(MODLIB)/source @if [ ! $(objtree) -ef $(MODLIB)/build ]; then \ - rm -f $(MODLIB)/build ; \ - ln -s $(CURDIR) $(MODLIB)/build ; \ + rm -rf $(MODLIB)/build ; \ + $(INSTALL_MOD_LN) $(CURDIR) $(MODLIB)/build ; \ fi @sed 's:^:kernel/:' modules.order > $(MODLIB)/modules.order @cp -f modules.builtin $(MODLIB)/
@@ -1489,6 +1492,10 @@ modules_sign: $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modsign endif +# initramfs image containing all modules +modules.img usr/modules.img: modules + $(Q)$(MAKE) $(build)=usr usr/modules.img + else # CONFIG_MODULES # Modules not configured
usr/.gitignore +3 -0
@@ -2,3 +2,6 @@ gen_init_cpio initramfs_data.cpio /initramfs_inc_data +/modules +modules.cpio +modules.img
usr/Kconfig +17 -14
@@ -109,18 +109,23 @@ config RD_ZSTD If unsure, say N. choice - prompt "Built-in initramfs compression mode" - depends on INITRAMFS_SOURCE != "" - help - This option allows you to decide by which algorithm the builtin - initramfs will be compressed. Several compression algorithms are - available, which differ in efficiency, compression and - decompression speed. Compression speed is only relevant - when building a kernel. Decompression speed is relevant at - each boot. Also the memory usage during decompression may become - relevant on memory constrained systems. This is usually based on the - dictionary size of the algorithm with algorithms like XZ and LZMA - featuring large dictionary sizes. + prompt "Preferred initramfs compression mode" + default INITRAMFS_COMPRESSION_XZ if RD_XZ + default INITRAMFS_COMPRESSION_GZIP if RD_GZIP + default INITRAMFS_COMPRESSION_LZO if RD_LZO + default INITRAMFS_COMPRESSION_LZ4 if RD_LZ4 + default INITRAMFS_COMPRESSION_NONE + help + This option allows you to decide by which algorithm the + builtin initramfs and usr/modules.img will be compressed. + Several compression algorithms are available, which differ + in efficiency, compression and decompression speed. + Compression speed is only relevant when building a kernel. + Decompression speed is relevant at each boot. Also the + memory usage during decompression may become relevant on + memory constrained systems. This is usually based on the + dictionary size of the algorithm with algorithms like XZ and + LZMA featuring large dictionary sizes. High compression options are mostly useful for users who are low on RAM, since it reduces the memory consumption during
@@ -130,8 +135,6 @@ choice compression tool to compress the generated initram cpio file for embedding. - If in doubt, select 'None' - config INITRAMFS_COMPRESSION_GZIP bool "Gzip" depends on RD_GZIP
usr/Makefile +35 -4
@@ -22,12 +22,14 @@ obj-$(CONFIG_BLK_DEV_INITRD) := initramfs_data.o $(obj)/initramfs_data.o: $(obj)/initramfs_inc_data ramfs-input := $(strip $(shell echo $(CONFIG_INITRAMFS_SOURCE))) +ramfs-compress := $(compress-y) cpio-data := # If CONFIG_INITRAMFS_SOURCE is empty, generate a small initramfs with the -# default contents. +# default contents. No point in compressing it separately from the kernel. ifeq ($(ramfs-input),) ramfs-input := $(srctree)/$(src)/default_cpio_list +ramfs-compress := shipped endif ifeq ($(words $(ramfs-input)),1)
@@ -42,7 +44,7 @@ endif # .cpio.*, use it directly as an initramfs, and avoid double compression. ifeq ($(words $(subst .cpio.,$(space),$(ramfs-input))),2) cpio-data := $(ramfs-input) -compress-y := shipped +ramfs-compress := shipped endif endif
@@ -76,14 +78,43 @@ quiet_cmd_initfs = GEN $@ # 2) There are changes in which files are included (added or deleted) # 3) If gen_init_cpio are newer than initramfs_data.cpio # 4) Arguments to gen_initramfs.sh changes -$(obj)/initramfs_data.cpio: $(src)/gen_initramfs.sh $(obj)/gen_init_cpio $(deps_initramfs) FORCE +$(obj)/initramfs_data.cpio: $(src)/gen_initramfs.sh $(obj)/gen_init_cpio \ + $(deps_initramfs) FORCE $(call if_changed,initfs) endif $(obj)/initramfs_inc_data: $(cpio-data) FORCE - $(call if_changed,$(compress-y)) + $(call if_changed,$(ramfs-compress)) targets += initramfs_data.cpio initramfs_inc_data subdir-$(CONFIG_UAPI_HEADER_TEST) += include + +# modules.img: an initramfs image containing all the modules +quiet_cmd_modinst = MODINST $(obj)/modules + cmd_modinst = \ + rm -rf '$(obj)/modules' && \ + $(MAKE) -f '$(srctree)/Makefile' O='$(objtree)' \ + INSTALL_MOD_PATH='$(obj)/modules' \ + INSTALL_MOD_LN=: modules_install + +quiet_cmd_modcpio = MODCPIO $@ + cmd_modcpio = \ + $(CONFIG_SHELL) $< -o $@ \ + $(if $(CONFIG_INITRAMFS_ROOT_UID), -u $(CONFIG_INITRAMFS_ROOT_UID)) \ + $(if $(CONFIG_INITRAMFS_ROOT_GID), -g $(CONFIG_INITRAMFS_ROOT_GID)) \ + '$(obj)/modules' + +modules_order := \ + $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD),$(objtree))/modules.order + +$(obj)/modules.cpio: $(src)/gen_initramfs.sh $(obj)/gen_init_cpio \ + $(modules_order) FORCE + $(call if_changed,modinst) + $(call if_changed,modcpio) + +$(obj)/modules.img: $(obj)/modules.cpio FORCE + $(call if_changed,$(compress-y)) + +targets += modules.cpio modules.img modules
Signature · 1 line
2.30.2
H. Peter Anvin
reviewerPATCH 3/3 x86/boot: Add option to add modules.img to {fd,hd,iso}image
In reply to H. Peter AnvinFrom: "H. Peter Anvin" (Intel) <[email protected]>
Make it easy to generate a disk image which includes the all-modules initramfs image.
--- arch/x86/Makefile | 3 ++- arch/x86/boot/Makefile | 11 +++++++++++ 2 files changed, 13 insertions(+), 1 deletion(-)
arch/x86/Makefile +2 -1
@@ -309,7 +309,8 @@ define archhelp echo ' isoimage - Create a boot CD-ROM image (arch/x86/boot/image.iso)' echo ' bzdisk/fdimage*/hdimage/isoimage also accept:' echo ' FDARGS="..." arguments for the booted kernel' - echo ' FDINITRD=file initrd for the booted kernel' + echo ' FDINITRD=file initrd for the booted kernel' + echo ' FDMODS=1 to include all modules as an initrd' echo '' echo ' kvm_guest.config - Enable Kconfig items for running this kernel as a KVM guest' echo ' xen.config - Enable Kconfig items for running this kernel as a Xen guest'
arch/x86/boot/Makefile +11 -0
@@ -119,9 +119,20 @@ $(obj)/compressed/vmlinux: FORCE FDARGS = # Set this if you want one or more initrds included in the image FDINITRD = +# Set this to 1 if you want usr/modules.img included in the image +FDMODS = imgdeps = $(obj)/bzImage $(obj)/mtools.conf $(src)/genimage.sh +ifneq ($(FDMODS),) +imgdeps += $(objtree)/usr/modules.img +FDINITRD += $(objtree)/usr/modules.img + +$(objtree)/usr/modules.img: + $(Q)$(MAKE) -f $(srctree)/Makefile usr/modules.img +KBUILD_MODULES := 1 +endif + $(obj)/mtools.conf: $(src)/mtools.conf.in sed -e 's|@OBJ@|$(obj)|g' < $< > $@
Signature · 1 line
2.30.2