Skip to contentSkip to conversation

kbuild: buildtar: add comments about inconsistent package generation

PATCH In review Masahiro Yamada 3 participants 4 messages Apr 14, 2024 – Apr 17, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kbuild: buildtar: add comments about inconsistent package generation

scripts/package/buildtar checks some kernel packages, and copies the first image found. This may potentially produce an inconsistent (and possibly wrong) package.

For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo}, and vmlinuz.efi, then copies the first image found, which might be a stale image.

When CONFIG_EFI_ZBOOT is enabled in the pristine source tree, 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the expected behavior.

If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created, which will remain in the build directory unless you clean it. Even if CONFIG_EFI_ZBOOT is turned on later, 'make ARCH=arm64 tar-pkg' will copy stale Image.gz instead of the latest vmlinuz.efi, as Image.gz takes precedence over vmlinuz.efi.

In summary, the code "[ -f ... ] && cp" does not consistently produce the desired outcome.

The other package scripts are deterministic; scripts/package/mkdebian, for example, chooses a copied kernel image based on CONFIG options.

I removed [ -f ... ] checks from x86, alpha, parisc, and the default because they have a single kernel image to copy. If it is missing, it should be an error.

I did not modify the code for mips, arm64, riscv. Instead, I left some comments. Eventually, someone may fix the code, or at the very least, it may discourage the copy-pasting of incorrect code.

---

 scripts/package/buildtar | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
scripts/package/buildtar +12 -4
index 72c91a1b832f..ed8d9b496305 100755
@@ -53,18 +53,24 @@ cp -v -- "${objtree}/vmlinux" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
 #
 # Install arch-specific kernel image(s)
 #
+# Note:
+#   mips, arm64, and riscv copy the first image found. This may not produce
+#   the desired outcome because it may pick up a stale file remaining in the
+#   build tree.
+#
 case "${ARCH}" in
 	x86|i386|x86_64)
-		[ -f "${objtree}/arch/x86/boot/bzImage" ] && cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
+		cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
 		;;
 	alpha)
-		[ -f "${objtree}/arch/alpha/boot/vmlinux.gz" ] && cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
+		cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
 		;;
 	parisc*)
-		[ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
+		cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
 		[ -f "${objtree}/lifimage" ] && cp -v -- "${objtree}/lifimage" "${tmpdir}/boot/lifimage-${KERNELRELEASE}"
 		;;
 	mips)
+		# Please note the following code may copy a stale file.
 		if [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.bin" ]; then
 			cp -v -- "${objtree}/arch/mips/boot/compressed/vmlinux.bin" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
 		elif [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.ecoff" ]; then
@@ -86,6 +92,7 @@ case "${ARCH}" in
 		fi
 		;;
 	arm64)
+		# Please note the following code may copy a stale file.
 		for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo vmlinuz.efi ; do
 			if [ -f "${objtree}/arch/arm64/boot/${i}" ] ; then
 				cp -v -- "${objtree}/arch/arm64/boot/${i}" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
@@ -94,6 +101,7 @@ case "${ARCH}" in
 		done
 		;;
 	riscv)
+		# Please note the following code may copy a stale file.
 		for i in Image.bz2 Image.gz Image; do
 			if [ -f "${objtree}/arch/riscv/boot/${i}" ] ; then
 				cp -v -- "${objtree}/arch/riscv/boot/${i}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
@@ -102,7 +110,7 @@ case "${ARCH}" in
 		done
 		;;
 	*)
-		[ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
+		cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
 		echo "" >&2
 		echo '** ** **  WARNING  ** ** **' >&2
 		echo "" >&2
Signature · 1 line
2.40.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 28 lines · Masahiro Yamada

scripts/package/buildtar checks some kernel packages, and copies the first image found. This may potentially produce an inconsistent (and possibly wrong) package.

For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo}, and vmlinuz.efi, then copies the first image found, which might be a stale image.

When CONFIG_EFI_ZBOOT is enabled in the pristine source tree, 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the expected behavior.

If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created, which will remain in the build directory unless you clean it. Even if CONFIG_EFI_ZBOOT is turned on later, 'make ARCH=arm64 tar-pkg' will copy stale Image.gz instead of the latest vmlinuz.efi, as Image.gz takes precedence over vmlinuz.efi.

In summary, the code "[ -f ... ] && cp" does not consistently produce the desired outcome.

The other package scripts are deterministic; scripts/package/mkdebian, for example, chooses a copied kernel image based on CONFIG options.

I removed [ -f ... ] checks from x86, alpha, parisc, and the default because they have a single kernel image to copy. If it is missing, it should be an error.

Quoted 5 lines

I did not modify the code for mips, arm64, riscv. Instead, I left some comments. Eventually, someone may fix the code, or at the very least, it may discourage the copy-pasting of incorrect code.

Hi Masahiro,

Thanks for the patch. I wonder why arm64 and riscv is not just using KBUILD_IMAGE as the source path of the image to copy like some other architectures do. In any case:

Quoted 70 lines

---

 scripts/package/buildtar | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
scripts/package/buildtar +12 -5
index 72c91a1b832f..ed8d9b496305 100755
@@ -53,18 +53,24 @@ cp -v -- "${objtree}/vmlinux" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
 #
 # Install arch-specific kernel image(s)
 #
+# Note:
+#   mips, arm64, and riscv copy the first image found. This may not produce
+#   the desired outcome because it may pick up a stale file remaining in the
+#   build tree.
+#
 case "${ARCH}" in
 	x86|i386|x86_64)
-		[ -f "${objtree}/arch/x86/boot/bzImage" ] && cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
+		cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
 		;;
 	alpha)
-		[ -f "${objtree}/arch/alpha/boot/vmlinux.gz" ] && cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
+		cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
 		;;
 	parisc*)
-		[ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
+		cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
 		[ -f "${objtree}/lifimage" ] && cp -v -- "${objtree}/lifimage" "${tmpdir}/boot/lifimage-${KERNELRELEASE}"
 		;;
 	mips)
+		# Please note the following code may copy a stale file.
 		if [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.bin" ]; then
 			cp -v -- "${objtree}/arch/mips/boot/compressed/vmlinux.bin" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
 		elif [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.ecoff" ]; then
@@ -86,6 +92,7 @@ case "${ARCH}" in
 		fi
 		;;
 	arm64)
+		# Please note the following code may copy a stale file.
 		for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo vmlinuz.efi ; do
 			if [ -f "${objtree}/arch/arm64/boot/${i}" ] ; then
 				cp -v -- "${objtree}/arch/arm64/boot/${i}" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
@@ -94,6 +101,7 @@ case "${ARCH}" in
 		done
 		;;
 	riscv)
+		# Please note the following code may copy a stale file.
 		for i in Image.bz2 Image.gz Image; do
 			if [ -f "${objtree}/arch/riscv/boot/${i}" ] ; then
 				cp -v -- "${objtree}/arch/riscv/boot/${i}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
@@ -102,7 +110,7 @@ case "${ARCH}" in
 		done
 		;;
 	*)
-		[ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
+		cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
 		echo "" >&2
 		echo '** ** **  WARNING  ** ** **' >&2
 		echo "" >&2
--

2.40.1

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#2 lore raw
In reply to Masahiro Yamada
Quoted 9 lines · Masahiro Yamada

scripts/package/buildtar checks some kernel packages, and copies the first image found. This may potentially produce an inconsistent (and possibly wrong) package.

For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo}, and vmlinuz.efi, then copies the first image found, which might be a stale image.

Quoted 5 lines

When CONFIG_EFI_ZBOOT is enabled in the pristine source tree, 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the expected behavior.

If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created,

The correct sentence is

If you build the kernel with CONFIG_EFI_ZBOOT disabled, ...

Quoted 83 lines

which will remain in the build directory unless you clean it. Even if CONFIG_EFI_ZBOOT is turned on later, 'make ARCH=arm64 tar-pkg' will copy stale Image.gz instead of the latest vmlinuz.efi, as Image.gz takes precedence over vmlinuz.efi.

In summary, the code "[ -f ... ] && cp" does not consistently produce the desired outcome.

The other package scripts are deterministic; scripts/package/mkdebian, for example, chooses a copied kernel image based on CONFIG options.

I removed [ -f ... ] checks from x86, alpha, parisc, and the default because they have a single kernel image to copy. If it is missing, it should be an error.

I did not modify the code for mips, arm64, riscv. Instead, I left some comments. Eventually, someone may fix the code, or at the very least, it may discourage the copy-pasting of incorrect code.

---

 scripts/package/buildtar | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
scripts/package/buildtar +12 -5
index 72c91a1b832f..ed8d9b496305 100755
@@ -53,18 +53,24 @@ cp -v -- "${objtree}/vmlinux" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
 #
 # Install arch-specific kernel image(s)
 #
+# Note:
+#   mips, arm64, and riscv copy the first image found. This may not produce
+#   the desired outcome because it may pick up a stale file remaining in the
+#   build tree.
+#
 case "${ARCH}" in
        x86|i386|x86_64)
-               [ -f "${objtree}/arch/x86/boot/bzImage" ] && cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
+               cp -v -- "${objtree}/arch/x86/boot/bzImage" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
                ;;
        alpha)
-               [ -f "${objtree}/arch/alpha/boot/vmlinux.gz" ] && cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
+               cp -v -- "${objtree}/arch/alpha/boot/vmlinux.gz" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
                ;;
        parisc*)
-               [ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
+               cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
                [ -f "${objtree}/lifimage" ] && cp -v -- "${objtree}/lifimage" "${tmpdir}/boot/lifimage-${KERNELRELEASE}"
                ;;
        mips)
+               # Please note the following code may copy a stale file.
                if [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.bin" ]; then
                        cp -v -- "${objtree}/arch/mips/boot/compressed/vmlinux.bin" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
                elif [ -f "${objtree}/arch/mips/boot/compressed/vmlinux.ecoff" ]; then
@@ -86,6 +92,7 @@ case "${ARCH}" in
                fi
                ;;
        arm64)
+               # Please note the following code may copy a stale file.
                for i in Image.bz2 Image.gz Image.lz4 Image.lzma Image.lzo vmlinuz.efi ; do
                        if [ -f "${objtree}/arch/arm64/boot/${i}" ] ; then
                                cp -v -- "${objtree}/arch/arm64/boot/${i}" "${tmpdir}/boot/vmlinuz-${KERNELRELEASE}"
@@ -94,6 +101,7 @@ case "${ARCH}" in
                done
                ;;
        riscv)
+               # Please note the following code may copy a stale file.
                for i in Image.bz2 Image.gz Image; do
                        if [ -f "${objtree}/arch/riscv/boot/${i}" ] ; then
                                cp -v -- "${objtree}/arch/riscv/boot/${i}" "${tmpdir}/boot/vmlinux-${KERNELRELEASE}"
@@ -102,7 +110,7 @@ case "${ARCH}" in
                done
                ;;
        *)
-               [ -f "${KBUILD_IMAGE}" ] && cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
+               cp -v -- "${KBUILD_IMAGE}" "${tmpdir}/boot/vmlinux-kbuild-${KERNELRELEASE}"
                echo "" >&2
                echo '** ** **  WARNING  ** ** **' >&2
                echo "" >&2
--
Quoted 2 lines

2.40.1

--
Best Regards
Masahiro Yamada

Reviewed-by
#3 lore raw
Quoted 15 lines · Masahiro Yamada
Quoted 13 lines · Masahiro Yamada

scripts/package/buildtar checks some kernel packages, and copies the first image found. This may potentially produce an inconsistent (and possibly wrong) package.

For instance, the for-loop for arm64 checks Image.{bz2,gz,lz4,lzma,lzo}, and vmlinuz.efi, then copies the first image found, which might be a stale image.

When CONFIG_EFI_ZBOOT is enabled in the pristine source tree, 'make ARCH=arm64 tar-pkg' will build and copy vmlinuz.efi. This is the expected behavior.

Quoted 6 lines
Quoted 1 line

If you build the kernel with CONFIG_EFI_ZBOOT, Image.gz will be created,

The correct sentence is

If you build the kernel with CONFIG_EFI_ZBOOT disabled, ...

Thanks, I appreciate the detailed descriptions and comments.

Kind regards,
Nicolas

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv