Skip to contentSkip to conversation

x86, kbuild: Some improvements for fdimage/isoimage generation

PATCH Superseded [email protected] 2 participants 11 messages Nov 5, 2017 – Nov 6, 2017 x86

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover x86, kbuild: Some improvements for fdimage/isoimage generation

From: Changbin Du <[email protected]>

This serias is a expansion of below original patch after discussion with Ingo and Yamada.
[PATCH] x86, build: Improve the isolinux searching of isoimage generation

Changbin Du (4):
x86, build: Fact out fdimage/isoimage generation commands to
standalone script
x86, build: Add new paths for isolinux.bin and ldlinux.c32 x86, build: Specify -input-charset=utf-8 for mkisofs x86, boot: Add some generated files to the .gitignore file

 arch/x86/boot/.gitignore  |   3 ++
 arch/x86/boot/Makefile    |  59 +++++------------------
 arch/x86/boot/genimage.sh | 119 ++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 133 insertions(+), 48 deletions(-)
 create mode 100644 arch/x86/boot/genimage.sh
Signature · 1 line
2.7.4
#1 lore raw

PATCH 1/4 x86, build: Fact out fdimage/isoimage generation commands to standalone script

From: Changbin Du <[email protected]>

The build message for fdimage/isoimage are pretty unstructured. The raw shell command blocks are printed. We can improve them as regular build system messages. Besides, writing commands in shell script is much more easier than in a Makefile.

See Ingo's suggestion here https://lkml.org/lkml/2017/10/31/124.

This patch fact out the commands used for fdimage/isoimage generation from arch/x86/boot/Makefile to new script arch/x86/boot/genimage.sh. Then add a new kbuild command 'genimage' which invokes the new script. All fdimages/isoimage now is generated by call to 'genimage' with different parameters.

Now 'make isoimage' becomes:
...
Kernel: arch/x86/boot/bzImage is ready (#30)
GENIMAGE arch/x86/boot/image.iso
Size of boot image is 4 sectors -> No emulation

 15.37% done, estimate finish Sun Nov  5 23:36:57 2017
 30.68% done, estimate finish Sun Nov  5 23:36:57 2017
 46.04% done, estimate finish Sun Nov  5 23:36:57 2017
 61.35% done, estimate finish Sun Nov  5 23:36:57 2017
 76.69% done, estimate finish Sun Nov  5 23:36:57 2017
 92.00% done, estimate finish Sun Nov  5 23:36:57 2017

Total translation table size: 2048
Total rockridge attributes bytes: 659
Total directory bytes: 0
Path table size(bytes): 10
Max brk space used 0
32608 extents written (63 MB)
Kernel: arch/x86/boot/image.iso is ready

Before:
Kernel: arch/x86/boot/bzImage is ready (#63)
rm -rf arch/x86/boot/isoimage
mkdir arch/x86/boot/isoimage
for i in lib lib64 share end ; do \

	if [ -f /usr/$i/syslinux/isolinux.bin ] ; then \
		cp /usr/$i/syslinux/isolinux.bin arch/x86/boot/isoimage ; \
		if [ -f /usr/$i/syslinux/ldlinux.c32 ]; then \
			cp /usr/$i/syslinux/ldlinux.c32 arch/x86/boot/isoimage ; \
		fi ; \
		break ; \
	fi ; \
	if [ $i = end ] ; then exit 1 ; fi ; \

done
...

---
 arch/x86/boot/Makefile    |  59 +++++---------------------
 arch/x86/boot/genimage.sh | 105 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 116 insertions(+), 48 deletions(-)
 create mode 100644 arch/x86/boot/genimage.sh
arch/x86/boot/Makefile +11 -48
index d88a2fd..9b5adae 100644
@@ -123,63 +123,26 @@ image_cmdline = default linux $(FDARGS) $(if $(FDINITRD),initrd=initrd.img,)
 $(obj)/mtools.conf: $(src)/mtools.conf.in
 	sed -e 's|@OBJ@|$(obj)|g' < $< > $@
 
+quiet_cmd_genimage = GENIMAGE $3
+cmd_genimage = sh $(srctree)/$(src)/genimage.sh $2 $3 $(obj)/bzImage \
+			$(obj)/mtools.conf '$(image_cmdline)' $(FDINITRD)
+
 # This requires write access to /dev/fd0
 bzdisk: $(obj)/bzImage $(obj)/mtools.conf
-	MTOOLSRC=$(obj)/mtools.conf mformat a:			; sync
-	syslinux /dev/fd0					; sync
-	echo '$(image_cmdline)' | \
-		MTOOLSRC=$(src)/mtools.conf mcopy - a:syslinux.cfg
-	if [ -f '$(FDINITRD)' ] ; then \
-		MTOOLSRC=$(obj)/mtools.conf mcopy '$(FDINITRD)' a:initrd.img ; \
-	fi
-	MTOOLSRC=$(obj)/mtools.conf mcopy $(obj)/bzImage a:linux	; sync
+	$(call cmd,genimage,bzdisk,/dev/fd0)
 
 # These require being root or having syslinux 2.02 or higher installed
 fdimage fdimage144: $(obj)/bzImage $(obj)/mtools.conf
-	dd if=/dev/zero of=$(obj)/fdimage bs=1024 count=1440
-	MTOOLSRC=$(obj)/mtools.conf mformat v:			; sync
-	syslinux $(obj)/fdimage					; sync
-	echo '$(image_cmdline)' | \
-		MTOOLSRC=$(obj)/mtools.conf mcopy - v:syslinux.cfg
-	if [ -f '$(FDINITRD)' ] ; then \
-		MTOOLSRC=$(obj)/mtools.conf mcopy '$(FDINITRD)' v:initrd.img ; \
-	fi
-	MTOOLSRC=$(obj)/mtools.conf mcopy $(obj)/bzImage v:linux	; sync
+	$(call cmd,genimage,fdimage144,$(obj)/fdimage)
+	@$(kecho) 'Kernel: $(obj)/fdimage is ready'
 
 fdimage288: $(obj)/bzImage $(obj)/mtools.conf
-	dd if=/dev/zero of=$(obj)/fdimage bs=1024 count=2880
-	MTOOLSRC=$(obj)/mtools.conf mformat w:			; sync
-	syslinux $(obj)/fdimage					; sync
-	echo '$(image_cmdline)' | \
-		MTOOLSRC=$(obj)/mtools.conf mcopy - w:syslinux.cfg
-	if [ -f '$(FDINITRD)' ] ; then \
-		MTOOLSRC=$(obj)/mtools.conf mcopy '$(FDINITRD)' w:initrd.img ; \
-	fi
-	MTOOLSRC=$(obj)/mtools.conf mcopy $(obj)/bzImage w:linux	; sync
+	$(call cmd,genimage,fdimage288,$(obj)/fdimage)
+	@$(kecho) 'Kernel: $(obj)/fdimage is ready'
 
 isoimage: $(obj)/bzImage
-	-rm -rf $(obj)/isoimage
-	mkdir $(obj)/isoimage
-	for i in lib lib64 share end ; do \
-		if [ -f /usr/$$i/syslinux/isolinux.bin ] ; then \
-			cp /usr/$$i/syslinux/isolinux.bin $(obj)/isoimage ; \
-			if [ -f /usr/$$i/syslinux/ldlinux.c32 ]; then \
-				cp /usr/$$i/syslinux/ldlinux.c32 $(obj)/isoimage ; \
-			fi ; \
-			break ; \
-		fi ; \
-		if [ $$i = end ] ; then exit 1 ; fi ; \
-	done
-	cp $(obj)/bzImage $(obj)/isoimage/linux
-	echo '$(image_cmdline)' > $(obj)/isoimage/isolinux.cfg
-	if [ -f '$(FDINITRD)' ] ; then \
-		cp '$(FDINITRD)' $(obj)/isoimage/initrd.img ; \
-	fi
-	mkisofs -J -r -o $(obj)/image.iso -b isolinux.bin -c boot.cat \
-		-no-emul-boot -boot-load-size 4 -boot-info-table \
-		$(obj)/isoimage
-	isohybrid $(obj)/image.iso 2>/dev/null || true
-	rm -rf $(obj)/isoimage
+	$(call cmd,genimage,isoimage,$(obj)/image.iso)
+	@$(kecho) 'Kernel: $(obj)/image.iso is ready'
 
 bzlilo: $(obj)/bzImage
 	if [ -f $(INSTALL_PATH)/vmlinuz ]; then mv $(INSTALL_PATH)/vmlinuz $(INSTALL_PATH)/vmlinuz.old; fi
arch/x86/boot/genimage.sh new file +105 -0

105 lines not on this page · raw message

Signature · 1 line
2.7.4
#2 lore raw

Re: [PATCH 1/4] x86, build: Fact out fdimage/isoimage generation commands to standalone script

A few spelling fixes:

in the title:

s/Fact out
/Factor out

Quoted 6 lines · * [email protected] <[email protected]> wrote:

From: Changbin Du <[email protected]>

The build message for fdimage/isoimage are pretty unstructured. The raw shell command blocks are printed. We can improve them as regular build system messages. Besides, writing commands in shell script is much more easier than in a Makefile.

s/much more easier
/much more easy

Quoted 8 lines

See Ingo's suggestion here https://lkml.org/lkml/2017/10/31/124.

This patch fact out the commands used for fdimage/isoimage generation from arch/x86/boot/Makefile to new script arch/x86/boot/genimage.sh. Then add a new kbuild command 'genimage' which invokes the new script. All fdimages/isoimage now is generated by call to 'genimage' with different parameters.

s/fact out
/factors out

s/to new script
to a new script

s/Then add
/Then it adds

s/a new kbuild command 'genimage'
/the new 'genimage' kbuild command

s/All fdimages/isoimage now is generated by call to
/All fdimage/isoimage files are now generated by a call to

arch/x86/boot/genimage.sh line 20

Quoted 5 lines
arch/x86/boot/genimage.sh new file +5 -0
+#   $3 - kernel bzImage file
+#   $4 - mtool configuration file
+#   $5 - kernel cmdline
+#   $6 - inird image file
+#

The new script is much easier to read!

Thanks,

Ingo

#3 lore raw

Re: [PATCH 1/4] x86, build: Fact out fdimage/isoimage generation commands to standalone script

Quoted 35 lines · Ingo Molnar

A few spelling fixes:

in the title:

s/Fact out
/Factor out

Quoted 6 lines · * [email protected] <[email protected]> wrote:

From: Changbin Du <[email protected]>

The build message for fdimage/isoimage are pretty unstructured. The raw shell command blocks are printed. We can improve them as regular build system messages. Besides, writing commands in shell script is much more easier than in a Makefile.

s/much more easier
/much more easy

Quoted 8 lines

See Ingo's suggestion here https://lkml.org/lkml/2017/10/31/124.

This patch fact out the commands used for fdimage/isoimage generation from arch/x86/boot/Makefile to new script arch/x86/boot/genimage.sh. Then add a new kbuild command 'genimage' which invokes the new script. All fdimages/isoimage now is generated by call to 'genimage' with different parameters.

s/fact out
/factors out

s/to new script
to a new script

Quoted 9 lines

s/Then add
/Then it adds

s/a new kbuild command 'genimage'
/the new 'genimage' kbuild command

s/All fdimages/isoimage now is generated by call to
/All fdimage/isoimage files are now generated by a call to

Sorry for these grammar errors. I alwyas forgot to write complete sentences in English. :)

Quoted 11 lines

arch/x86/boot/genimage.sh line 20

Quoted 5 lines
arch/x86/boot/genimage.sh new file +5 -0
+#   $3 - kernel bzImage file
+#   $4 - mtool configuration file
+#   $5 - kernel cmdline
+#   $6 - inird image file
+#

The new script is much easier to read!

Thanks,

Ingo

Signature · 2 lines
Thanks,
Changbin Du
#4 lore raw

PATCH 2/4 x86, build: Add new paths for isolinux.bin and ldlinux.c32

In reply to [email protected]

From: Changbin Du <[email protected]>

Recently I failed to build isoimage target, because the path of isolinux.bin changed to /usr/xxx/ISOLINUX/isolinux.bin, as well as ldlinux.c32 which changed to /usr/xxx/syslinux/modules/bios/ldlinux.c32.

This patch has a improvement of the file search:
- Show a error message instead of silent fail.
- Add above new paths.

---
 arch/x86/boot/genimage.sh | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)
arch/x86/boot/genimage.sh +18 -5
index 75a9de1..f7994ad 100644
@@ -76,14 +76,27 @@ genisoimage() {
 	rm -rf $tmp_dir
 	mkdir $tmp_dir
 	for i in lib lib64 share end ; do
-		if [ -f /usr/$i/syslinux/isolinux.bin ] ; then
-			cp /usr/$i/syslinux/isolinux.bin $tmp_dir
-			if [ -f /usr/$i/syslinux/ldlinux.c32 ]; then
-				cp /usr/$i/syslinux/ldlinux.c32 $tmp_dir
+		for j in syslinux ISOLINUX ; do
+			if [ -f /usr/$i/$j/isolinux.bin ] ; then
+				isolinux=/usr/$i/$j/isolinux.bin
+				echo "Using $isolinux"
+				cp $isolinux $tmp_dir
 			fi
+		done
+		for j in syslinux syslinux/modules/bios ; do
+			if [ -f /usr/$i/$j/ldlinux.c32 ]; then
+				ldlinux=/usr/$i/$j/ldlinux.c32
+				echo "Using $ldlinux"
+				cp $ldlinux $tmp_dir
+			fi
+		done
+		if [ -n "$isolinux" -a -n "$ldlinux" ] ; then
 			break
 		fi
-		if [ $i = end ] ; then exit 1 ; fi ;
+		if [ $i = end -a -z "$isolinux" ] ; then
+			echo 'Need isolinux.bin, please install syslinux/isolinux'
+			exit 1
+		fi
 	done
 	cp $FBZIMAGE $tmp_dir/linux
 	echo "$KCMDLINE" > $tmp_dir/isolinux.cfg
Signature · 1 line
2.7.4
#5 lore raw

Re: [PATCH 2/4] x86, build: Add new paths for isolinux.bin and ldlinux.c32

Quoted 9 lines · * [email protected] <[email protected]> wrote:

From: Changbin Du <[email protected]>

Recently I failed to build isoimage target, because the path of isolinux.bin changed to /usr/xxx/ISOLINUX/isolinux.bin, as well as ldlinux.c32 which changed to /usr/xxx/syslinux/modules/bios/ldlinux.c32.

This patch has a improvement of the file search:
- Show a error message instead of silent fail.
- Add above new paths.

How about:

This patch improves the file search logic:
- Show an error message instead of failing silently
- Add the new paths listed above.

arch/x86/boot/genimage.sh genisoimage() { · line 97

Quoted 2 lines
arch/x86/boot/genimage.sh +2 -0
+		if [ $i = end -a -z "$isolinux" ] ; then
+			echo 'Need isolinux.bin, please install syslinux/isolinux'

How about:

echo 'Need an isolinux.bin file, please install syslinux/isolinux'

Thanks,

Ingo

#6 lore raw

Re: [PATCH 2/4] x86, build: Add new paths for isolinux.bin and ldlinux.c32

Hi Ingo,

Quoted 20 lines · Ingo Molnar
Quoted 9 lines · * [email protected] <[email protected]> wrote:

From: Changbin Du <[email protected]>

Recently I failed to build isoimage target, because the path of isolinux.bin changed to /usr/xxx/ISOLINUX/isolinux.bin, as well as ldlinux.c32 which changed to /usr/xxx/syslinux/modules/bios/ldlinux.c32.

This patch has a improvement of the file search:
- Show a error message instead of silent fail.
- Add above new paths.

How about:

This patch improves the file search logic:
- Show an error message instead of failing silently
- Add the new paths listed above.

Quoted 7 lines

arch/x86/boot/genimage.sh genisoimage() { · line 97

Quoted 2 lines
arch/x86/boot/genimage.sh +2 -0
+		if [ $i = end -a -z "$isolinux" ] ; then
+			echo 'Need isolinux.bin, please install syslinux/isolinux'

How about:

echo 'Need an isolinux.bin file, please install syslinux/isolinux'

The new description is much better, will update. Thanks.

Quoted 3 lines

Thanks,

Ingo

Signature · 2 lines
Thanks,
Changbin Du
#7 lore raw

PATCH 3/4 x86, build: Specify -input-charset=utf-8 for mkisofs

In reply to [email protected]

From: Changbin Du <[email protected]>

This can omit this:
I: -input-charset not specified, using utf-8 (detected in locale settings)

---
 arch/x86/boot/genimage.sh | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
arch/x86/boot/genimage.sh +3 -2
index f7994ad..b411102 100644
@@ -103,8 +103,9 @@ genisoimage() {
 	if [ -f "$FDINITRD" ] ; then
 		cp "$FDINITRD" $tmp_dir/initrd.img
 	fi
-	mkisofs -J -r -o $FIMAGE -b isolinux.bin -c boot.cat \
-		-no-emul-boot -boot-load-size 4 -boot-info-table $tmp_dir
+	mkisofs -J -r -input-charset=utf-8 -o $FIMAGE -b isolinux.bin \
+		-c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table \
+		$tmp_dir
 	isohybrid $FIMAGE 2>/dev/null || true
 	rm -rf $tmp_dir
 }
Signature · 1 line
2.7.4
#8 lore raw

Re: [PATCH 3/4] x86, build: Specify -input-charset=utf-8 for mkisofs

Quoted 4 lines · * [email protected] <[email protected]> wrote:

From: Changbin Du <[email protected]>

This can omit this:
I: -input-charset not specified, using utf-8 (detected in locale settings)

How about:

Avoids the following warning triggered by newer versions of mkisofs:

I: -input-charset not specified, using utf-8 (detected in locale settings)

Thanks,

Ingo

#9 lore raw

PATCH 4/4 x86, boot: Add some generated files to the .gitignore file

In reply to [email protected]

From: Changbin Du <[email protected]>

Ignore generated files fdimage,mtools.conf and image.iso.

---
 arch/x86/boot/.gitignore | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/boot/.gitignore +3 -0
index e3cf9f6..09d25dd 100644
@@ -7,3 +7,6 @@ zoffset.h
 setup
 setup.bin
 setup.elf
+fdimage
+mtools.conf
+image.iso
Signature · 1 line
2.7.4
#10 lore raw
In reply to [email protected]
Quoted 13 lines · * [email protected] <[email protected]> wrote:

From: Changbin Du <[email protected]>

This serias is a expansion of below original patch after discussion with Ingo and Yamada.
[PATCH] x86, build: Improve the isolinux searching of isoimage generation

Changbin Du (4):
x86, build: Fact out fdimage/isoimage generation commands to
standalone script
x86, build: Add new paths for isolinux.bin and ldlinux.c32 x86, build: Specify -input-charset=utf-8 for mkisofs x86, boot: Add some generated files to the .gitignore file

Quoted 5 lines
 arch/x86/boot/.gitignore  |   3 ++
 arch/x86/boot/Makefile    |  59 +++++------------------
 arch/x86/boot/genimage.sh | 119 ++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 133 insertions(+), 48 deletions(-)
 create mode 100644 arch/x86/boot/genimage.sh

Very nice - it looks good to me with the edits I suggested.

Would be nice if the kbuild maintainers could take a look as well - is this how boot image construction should be done at the arch level?

Thanks,

Ingo