Skip to contentSkip to conversation

Documentation: kbuild: Add description of git for reproducible builds

RFC In review Dan Li 4 participants 5 messages Oct 20, 2022 – Oct 23, 2022

Subsystems (2)
lore
Share & tools
lore

Dan Li

author
#0 lore raw

RFC 1/1 Documentation: kbuild: Add description of git for reproducible builds

The status of git will affect the final compilation result, add it to the documentation of reproducible builds.

---
 Documentation/kbuild/reproducible-builds.rst | 10 ++++++++++
 1 file changed, 10 insertions(+)
Documentation/kbuild/reproducible-builds.rst +10 -0
index 071f0151a7a4..13397f38c358 100644
@@ -119,6 +119,16 @@ To avoid this, you can make the vDSO different for different
 kernel versions by including an arbitrary string of "salt" in it.
 This is specified by the Kconfig symbol ``CONFIG_BUILD_SALT``.
 
+Git
+-----------------------
+
+Uncommitted changes or different commit ids in git can also lead
+to different compilation results. For example, after executing
+``git reset HEAD^``, even if the code is the same, the
+``include/config/kernel.release`` generated during compilation
+will be different, which will eventually lead to binary differences.
+See ``scripts/setlocalversion`` for details.
+
 .. _KBUILD_BUILD_TIMESTAMP: kbuild.html#kbuild-build-timestamp
 .. _KBUILD_BUILD_USER and KBUILD_BUILD_HOST: kbuild.html#kbuild-build-user-kbuild-build-host
 .. _KCFLAGS: kbuild.html#kcflags
Signature · 1 line
2.17.1
#1 lore raw
Quoted 8 lines · Dan Li
Documentation/kbuild/reproducible-builds.rst +0 -0
index 071f0151a7a4..13397f38c358 100644
@@ -119,6 +119,16 @@ To avoid this, you can make the vDSO different for different
 kernel versions by including an arbitrary string of "salt" in it.
 This is specified by the Kconfig symbol ``CONFIG_BUILD_SALT``.
 

Documentation/kbuild/reproducible-builds.rst To avoid this, you can make the vDSO different for different · line 131

Quoted 10 lines
Documentation/kbuild/reproducible-builds.rst +10 -0
+Git
+-----------------------
+
+Uncommitted changes or different commit ids in git can also lead
+to different compilation results. For example, after executing
+``git reset HEAD^``, even if the code is the same, the
+``include/config/kernel.release`` generated during compilation
+will be different, which will eventually lead to binary differences.
+See ``scripts/setlocalversion`` for details.
+

Briefly read the script, I don't see what the correlation between git reset with LOCALVERSION thing is. Also, does the exact state of git repository required for reproducible builds?

Thanks.

Signature · 1 line
An old man doll... just what I always wanted! - Clara
author
#2 lore raw
Quoted 9 lines · Bagas Sanjaya
Quoted 8 lines · Dan Li
Documentation/kbuild/reproducible-builds.rst +0 -0
index 071f0151a7a4..13397f38c358 100644
@@ -119,6 +119,16 @@ To avoid this, you can make the vDSO different for different
  kernel versions by including an arbitrary string of "salt" in it.
  This is specified by the Kconfig symbol ``CONFIG_BUILD_SALT``.
 
Quoted 15 lines

Documentation/kbuild/reproducible-builds.rst To avoid this, you can make the vDSO different for different · line 131

Quoted 10 lines
Documentation/kbuild/reproducible-builds.rst +10 -0
+Git
+-----------------------
+
+Uncommitted changes or different commit ids in git can also lead
+to different compilation results. For example, after executing
+``git reset HEAD^``, even if the code is the same, the
+``include/config/kernel.release`` generated during compilation
+will be different, which will eventually lead to binary differences.
+See ``scripts/setlocalversion`` for details.
+

Briefly read the script, I don't see what the correlation between git reset with LOCALVERSION thing is. Also, does the exact state of git repository required for reproducible builds?

Hi Bagas,

The Makefile has the following code:

filechk_kernel.release = \
        echo "$(KERNELVERSION)$$($(CONFIG_SHELL) $(srctree)/scripts/setlocalversion $(srctree))"

The output of setlocalversion affects kernel.release, while the output of setlocalversion is related to the state of git when the git repository exists (see function scm_version).

So changes in git state will result in changes to kernel.release, and this information will be included in the final output vmlinux/modules and in turn affect reproducible builds.

For example:

$ git log
commit 4cd155a93eec......
$ make ...
$ cat include/config/kernel.release
6.0.0-rc4-00025-g4cd155a93eec
$ git reset HEAD^
$ git log
commit 7b4d266b0c41......
$ make ...
$ cat include/config/kernel.release
6.0.0-rc4-00024-g7b4d266b0c41-dirty

AFAICT, in the presence of a git repository, we can compile a reproducible build kernel in any git state, but we need to ensure that the git state is always the same between compilations (or the same from the perspective of the scm_version function).

Thanks, Dan.

Quoted 2 lines

Thanks.

#3 lore raw
Quoted 56 lines · Dan Li
Quoted 14 lines · Bagas Sanjaya
Quoted 13 lines · Dan Li
Documentation/kbuild/reproducible-builds.rst +6 -0
index 071f0151a7a4..13397f38c358 100644
@@ -119,6 +119,16 @@ To avoid this, you can make the vDSO different for different
  kernel versions by including an arbitrary string of "salt" in it.
  This is specified by the Kconfig symbol ``CONFIG_BUILD_SALT``.
+Git
+-----------------------
+
+Uncommitted changes or different commit ids in git can also lead
+to different compilation results. For example, after executing
+``git reset HEAD^``, even if the code is the same, the
Quoted 9 lines

Documentation/kbuild/reproducible-builds.rst To avoid this, you can make the vDSO different for different · line 131

Quoted 4 lines
Documentation/kbuild/reproducible-builds.rst +4 -0
+``include/config/kernel.release`` generated during compilation
+will be different, which will eventually lead to binary differences.
+See ``scripts/setlocalversion`` for details.
+

Briefly read the script, I don't see what the correlation between git reset with LOCALVERSION thing is. Also, does the exact state of git repository required for reproducible builds?

Hi Bagas,

The Makefile has the following code:

filechk_kernel.release = \
        echo "$(KERNELVERSION)$$($(CONFIG_SHELL) $(srctree)/scripts/setlocalversion $(srctree))"

The output of setlocalversion affects kernel.release, while the output of setlocalversion is related to the state of git when the git repository exists (see function scm_version).

So changes in git state will result in changes to kernel.release, and this information will be included in the final output vmlinux/modules and in turn affect reproducible builds.

For example:

$ git log
commit 4cd155a93eec......
$ make ...
$ cat include/config/kernel.release
6.0.0-rc4-00025-g4cd155a93eec
$ git reset HEAD^
$ git log
commit 7b4d266b0c41......
$ make ...
$ cat include/config/kernel.release
6.0.0-rc4-00024-g7b4d266b0c41-dirty
Quoted 4 lines

AFAICT, in the presence of a git repository, we can compile a reproducible build kernel in any git state, but we need to ensure that the git state is always the same between compilations (or the same from the perspective of the scm_version function).

yes, that definitely true. Absence or presence of git tags can change the output of setlocalversion even more drastically.

I think it is sensible to add a stanza about git in Documentation/kbuild/reproducible-builds.rst.

Kind regards,
Nicolas

#4 lore raw
Quoted 63 lines · Nicolas Schier
Quoted 56 lines · Dan Li
Quoted 14 lines · Bagas Sanjaya
Quoted 13 lines · Dan Li
Documentation/kbuild/reproducible-builds.rst +6 -0
index 071f0151a7a4..13397f38c358 100644
@@ -119,6 +119,16 @@ To avoid this, you can make the vDSO different for different
  kernel versions by including an arbitrary string of "salt" in it.
  This is specified by the Kconfig symbol ``CONFIG_BUILD_SALT``.
+Git
+-----------------------
+
+Uncommitted changes or different commit ids in git can also lead
+to different compilation results. For example, after executing
+``git reset HEAD^``, even if the code is the same, the
Quoted 9 lines

Documentation/kbuild/reproducible-builds.rst To avoid this, you can make the vDSO different for different · line 131

Quoted 4 lines
Documentation/kbuild/reproducible-builds.rst +4 -0
+``include/config/kernel.release`` generated during compilation
+will be different, which will eventually lead to binary differences.
+See ``scripts/setlocalversion`` for details.
+

Briefly read the script, I don't see what the correlation between git reset with LOCALVERSION thing is. Also, does the exact state of git repository required for reproducible builds?

Hi Bagas,

The Makefile has the following code:

filechk_kernel.release = \
        echo "$(KERNELVERSION)$$($(CONFIG_SHELL) $(srctree)/scripts/setlocalversion $(srctree))"

The output of setlocalversion affects kernel.release, while the output of setlocalversion is related to the state of git when the git repository exists (see function scm_version).

So changes in git state will result in changes to kernel.release, and this information will be included in the final output vmlinux/modules and in turn affect reproducible builds.

For example:

$ git log
commit 4cd155a93eec......
$ make ...
$ cat include/config/kernel.release
6.0.0-rc4-00025-g4cd155a93eec
$ git reset HEAD^
$ git log
commit 7b4d266b0c41......
$ make ...
$ cat include/config/kernel.release
6.0.0-rc4-00024-g7b4d266b0c41-dirty
Quoted 4 lines

AFAICT, in the presence of a git repository, we can compile a reproducible build kernel in any git state, but we need to ensure that the git state is always the same between compilations (or the same from the perspective of the scm_version function).

Quoted 5 lines

yes, that definitely true. Absence or presence of git tags can change the output of setlocalversion even more drastically.

I think it is sensible to add a stanza about git in Documentation/kbuild/reproducible-builds.rst.

Make sense.
Applied to linux-kbuild.
Thanks.

Signature · 2 lines
Best Regards
Masahiro Yamada