scripts: Fix incremental build header re-generation
PATCH 1/1 scripts: Fix incremental build header re-generation
compile.h and autoconf.h are ignored when checking headers sha as they are always re-generated for every kernel compilation. However, these two headers are packaged into kheaders tar. During incremental compilation of kernel, kheaders tar file is always generated and re-packaged irrespective of the changes in headers.
--- kernel/gen_kheaders.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
kernel/gen_kheaders.sh +1 -2
@@ -51,8 +51,7 @@ this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)" if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi if [ -f kernel/kheaders.md5 ] && [ "$(head -n 1 kernel/kheaders.md5)" = "$headers_md5" ] && - [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ] && - [ "$(tail -n 1 kernel/kheaders.md5)" = "$tarfile_md5" ]; then + [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ]; then exit fi
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project
Quoted 6 lines · Jeevan Shriram
compile.handautoconf.hare ignored when checking headers sha as they are always re-generated for every kernel compilation. However, these two headers are packaged into kheaders tar. During incremental compilation of kernel, kheaders tar file is always generated and re-packaged irrespective of the changes in headers.
I do not see this problem.
Could you describe the steps to reproduce it, please?
Quoted 24 lines
Change-Id I7a64faebb81df44c32230b0fea1d6df09d7ce66fSigned-off-by Jeevan Shriram <[email protected]>--- kernel/gen_kheaders.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)kernel/gen_kheaders.sh +1
-3@@ -51,8 +51,7 @@ this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)" if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi if [ -f kernel/kheaders.md5 ] && [ "$(head -n 1 kernel/kheaders.md5)" = "$headers_md5" ] && - [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ] && - [ "$(tail -n 1 kernel/kheaders.md5)" = "$tarfile_md5" ]; then + [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ]; then exit fi --The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 10 lines · Masahiro Yamada
Quoted 5 lines · Jeevan Shriram
compile.handautoconf.hare ignored when checking headers sha as they are always re-generated for every kernel compilation. However, these two headers are packaged into kheaders tar. During incremental compilation of kernel, kheaders tar file is always generated and re-packaged irrespective of the changes in headers.I do not see this problem.
Could you describe the steps to reproduce it, please?
Without making any changes in the kernel or it's headers, re-compile the kernel. i.e.,incremental kernel build without any changes. I have added following log in gen_kheaders.sh script for confirming the hash differences.
kernel/gen_kheaders.sh +4 -0
@@ -40,6 +40,10 @@ obj_files_md5="$(find $dir_list -name "*.h" | # Any changes to this script will also cause a rebuild of the archive. this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)" if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi + +echo "Old Tar file $tarfile_md5" +echo "New Tar file hash $this_file_md5" + if [ -f kernel/kheaders.md5 ] && [ "$(cat kernel/kheaders.md5|head -1)" == "$src_files_md5" ] && [ "$(cat kernel/kheaders.md5|head -2|tail -1)" == "$obj_files_md5" ] &&
log output :
89306 19:29:02.109961 CHK kernel/kheaders_data.tar.xz
89307 19:29:02.109971 Old Tar file 2aa6990e4183c31a862951f4bcac037e 89308 19:29:02.109982 New Tar file hash ecf84e700c7cacfe8b35a0905859582d
Quoted 25 lines
Quoted 24 lines
Change-Id I7a64faebb81df44c32230b0fea1d6df09d7ce66fSigned-off-by Jeevan Shriram <[email protected]>--- kernel/gen_kheaders.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)kernel/gen_kheaders.sh +1
-3@@ -51,8 +51,7 @@ this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)" if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi if [ -f kernel/kheaders.md5 ] && [ "$(head -n 1 kernel/kheaders.md5)" = "$headers_md5" ] && - [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ] && - [ "$(tail -n 1 kernel/kheaders.md5)" = "$tarfile_md5" ]; then + [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ]; then exit fi --The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project
Quoted 10 lines · Jeevan Shriram
Quoted 7 lines · Masahiro Yamada
Quoted 5 lines · Jeevan Shriram
compile.handautoconf.hare ignored when checking headers sha as they are always re-generated for every kernel compilation. However, these two headers are packaged into kheaders tar. During incremental compilation of kernel, kheaders tar file is always generated and re-packaged irrespective of the changes in headers.
Quoted 5 lines
Quoted 3 lines
I do not see this problem.
Could you describe the steps to reproduce it, please?Without making any changes in the kernel or it's headers, re-compile the kernel. i.e.,incremental kernel build without any changes. I have added following log in
gen_kheaders.shscript for confirming the hash differences.
Of course they are different because you are comparing the hashes of two different files.
kernel/gen_kheaders.sh obj_files_md5="$(find $dir_list -name "*.h" | · line 44
Quoted 11 lines
kernel/gen_kheaders.sh +2
-0@@ -40,6 +40,10 @@ obj_files_md5="$(find $dir_list -name "*.h" | # Any changes to this script will also cause a rebuild of the archive. this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)" if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi + +echo "Old Tar file $tarfile_md5"
This is the hash of kernel/kheaders_data.tar.xz
kernel/gen_kheaders.sh obj_files_md5="$(find $dir_list -name "*.h" | · line 45
Quoted 1 line
kernel/gen_kheaders.sh +1
-0+echo "New Tar file hash $this_file_md5"
This is the hash of kernel/gen_kheaders.sh
Quoted 9 lines
kernel/gen_kheaders.sh +1
-0+ if [ -f kernel/kheaders.md5 ] && [ "$(cat kernel/kheaders.md5|head -1)" == "$src_files_md5" ] && [ "$(cat kernel/kheaders.md5|head -2|tail -1)" == "$obj_files_md5" ] &&log output :
89306 19:29:02.109961 CHKkernel/kheaders_data.tar.xz
89307 19:29:02.109971 Old Tar file2aa6990e4183c31a862951f4bcac037e89308 19:29:02.109982 New Tar file hashecf84e700c7cacfe8b35a0905859582d
I do not understand what your claim is.
CHK kernel/kheaders_data.tar.xz
is displayed when it is checking the hash in order to determine if the tarball should be updated.
GEN kernel/kheaders_data.tar.xz
is displayed when the tarball is really updated.
[Incremental build with no change]
masahiro@oscar:~/ref/linux$ make -j24 DESCEND objtool CALL scripts/atomic/check-atomics.sh CALL scripts/checksyscalls.sh CHK include/generated/compile.h CHK kernel/kheaders_data.tar.xz Kernel: arch/x86/boot/bzImage is ready (#2)
[Touch one header and rebuild]
masahiro@oscar:~/ref/linux$ touch include/uapi/drm/exynos_drm.h masahiro@oscar:~/ref/linux$ make -j24 DESCEND objtool CALL scripts/atomic/check-atomics.sh CALL scripts/checksyscalls.sh CHK include/generated/compile.h CHK kernel/kheaders_data.tar.xz GEN kernel/kheaders_data.tar.xz CC kernel/kheaders.o AR kernel/built-in.a GEN .version CHK include/generated/compile.h UPD include/generated/compile.h CC init/version.o AR init/built-in.a LD vmlinux.o MODPOST vmlinux.symvers MODINFO modules.builtin.modinfo GEN modules.builtin LD .tmp_vmlinux.kallsyms1 KSYMS .tmp_vmlinux.kallsyms1.S AS .tmp_vmlinux.kallsyms1.S LD .tmp_vmlinux.kallsyms2 KSYMS .tmp_vmlinux.kallsyms2.S AS .tmp_vmlinux.kallsyms2.S LD vmlinux SORTTAB vmlinux SYSMAP System.map MODPOST Module.symvers CC arch/x86/boot/version.o VOFFSET arch/x86/boot/compressed/../voffset.h OBJCOPY arch/x86/boot/compressed/vmlinux.bin RELOCS arch/x86/boot/compressed/vmlinux.relocs CC arch/x86/boot/compressed/kaslr.o CC arch/x86/boot/compressed/misc.o GZIP arch/x86/boot/compressed/vmlinux.bin.gz MKPIGGY arch/x86/boot/compressed/piggy.S AS arch/x86/boot/compressed/piggy.o LD arch/x86/boot/compressed/vmlinux ZOFFSET arch/x86/boot/zoffset.h OBJCOPY arch/x86/boot/vmlinux.bin AS arch/x86/boot/header.o LD arch/x86/boot/setup.elf OBJCOPY arch/x86/boot/setup.bin BUILD arch/x86/boot/bzImage Kernel: arch/x86/boot/bzImage is ready (#3)
Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
Change-Id I7a64faebb81df44c32230b0fea1d6df09d7ce66fSigned-off-by Jeevan Shriram <[email protected]>--- kernel/gen_kheaders.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)kernel/gen_kheaders.sh +1
-2@@ -51,8 +51,7 @@ this_file_md5="$(ls -l $sfile | md5sum | cut -d ' ' -f1)" if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi if [ -f kernel/kheaders.md5 ] && [ "$(head -n 1 kernel/kheaders.md5)" = "$headers_md5" ] && - [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ] && - [ "$(tail -n 1 kernel/kheaders.md5)" = "$tarfile_md5" ]; then + [ "$(head -n 2 kernel/kheaders.md5 | tail -n 1)" = "$this_file_md5" ]; then exit fi
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
--
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project
--
Best Regards
Masahiro Yamada