Skip to contentSkip to conversation

kbuild: mkcompile_h: consider timestamp if KBUILD_BUILD_TIMESTAMP is set

PATCH In review Matthias Maennich 2 participants 4 messages Jun 12, 2021 – Jun 17, 2021 Kbuild

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kbuild: mkcompile_h: consider timestamp if KBUILD_BUILD_TIMESTAMP is set

To avoid unnecessary recompilations, mkcompile_h does not regenerate compile.h if just the timestamp changed.
Though, if KBUILD_BUILD_TIMESTAMP is set, an explicit timestamp for the build was requested, in which case we should not ignore it.

If a user follows the documentation for reproducible builds [1] and defines KBUILD_BUILD_TIMESTAMP as the git commit timestamp, a clean build will have the correct timestamp. A subsequent cherry-pick (or amend) changes the commit timestamp and if an incremental build is done with a different KBUILD_BUILD_TIMESTAMP now, that new value is not taken into consideration. But it should for reproducibility.

Hence, whenever KBUILD_BUILD_TIMESTAMP is explicitly set, do not ignore UTS_VERSION when making a decision about whether the regenerated version of compile.h should be moved into place.

[1] https://www.kernel.org/doc/html/latest/kbuild/reproducible-builds.html

---
 scripts/mkcompile_h | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
scripts/mkcompile_h +11 -3
index 4ae735039daf..a72b154de7b0 100755
@@ -70,15 +70,23 @@ UTS_VERSION="$(echo $UTS_VERSION $CONFIG_FLAGS $TIMESTAMP | cut -b -$UTS_LEN)"
 # Only replace the real compile.h if the new one is different,
 # in order to preserve the timestamp and avoid unnecessary
 # recompilations.
-# We don't consider the file changed if only the date/time changed.
+# We don't consider the file changed if only the date/time changed,
+# unless KBUILD_BUILD_TIMESTAMP was explicitly set (e.g. for
+# reproducible builds with that value referring to a commit timestamp).
 # A kernel config change will increase the generation number, thus
 # causing compile.h to be updated (including date/time) due to the
 # changed comment in the
 # first line.
 
+if [ -z "$KBUILD_BUILD_TIMESTAMP" ]; then
+   IGNORE_PATTERN="UTS_VERSION"
+else
+   IGNORE_PATTERN="NOT_A_PATTERN_TO_BE_MATCHED"
+fi
+
 if [ -r $TARGET ] && \
-      grep -v 'UTS_VERSION' $TARGET > .tmpver.1 && \
-      grep -v 'UTS_VERSION' .tmpcompile > .tmpver.2 && \
+      grep -v $IGNORE_PATTERN $TARGET > .tmpver.1 && \
+      grep -v $IGNORE_PATTERN .tmpcompile > .tmpver.2 && \
       cmp -s .tmpver.1 .tmpver.2; then
    rm -f .tmpcompile
 else
Signature · 1 line
2.32.0.272.g935e593368-goog
#1 lore raw
Quoted 17 lines · Matthias Maennich

To avoid unnecessary recompilations, mkcompile_h does not regenerate compile.h if just the timestamp changed.
Though, if KBUILD_BUILD_TIMESTAMP is set, an explicit timestamp for the build was requested, in which case we should not ignore it.

If a user follows the documentation for reproducible builds [1] and defines KBUILD_BUILD_TIMESTAMP as the git commit timestamp, a clean build will have the correct timestamp. A subsequent cherry-pick (or amend) changes the commit timestamp and if an incremental build is done with a different KBUILD_BUILD_TIMESTAMP now, that new value is not taken into consideration. But it should for reproducibility.

Hence, whenever KBUILD_BUILD_TIMESTAMP is explicitly set, do not ignore UTS_VERSION when making a decision about whether the regenerated version of compile.h should be moved into place.

Quoted 6 lines

[1] https://www.kernel.org/doc/html/latest/kbuild/reproducible-builds.html

---

Applied to linux-kbuild. Thanks.

Quoted 37 lines
 scripts/mkcompile_h | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
scripts/mkcompile_h +11 -4
index 4ae735039daf..a72b154de7b0 100755
@@ -70,15 +70,23 @@ UTS_VERSION="$(echo $UTS_VERSION $CONFIG_FLAGS $TIMESTAMP | cut -b -$UTS_LEN)"
 # Only replace the real compile.h if the new one is different,
 # in order to preserve the timestamp and avoid unnecessary
 # recompilations.
-# We don't consider the file changed if only the date/time changed.
+# We don't consider the file changed if only the date/time changed,
+# unless KBUILD_BUILD_TIMESTAMP was explicitly set (e.g. for
+# reproducible builds with that value referring to a commit timestamp).
 # A kernel config change will increase the generation number, thus
 # causing compile.h to be updated (including date/time) due to the
 # changed comment in the
 # first line.
 
+if [ -z "$KBUILD_BUILD_TIMESTAMP" ]; then
+   IGNORE_PATTERN="UTS_VERSION"
+else
+   IGNORE_PATTERN="NOT_A_PATTERN_TO_BE_MATCHED"
+fi
+
 if [ -r $TARGET ] && \
-      grep -v 'UTS_VERSION' $TARGET > .tmpver.1 && \
-      grep -v 'UTS_VERSION' .tmpcompile > .tmpver.2 && \
+      grep -v $IGNORE_PATTERN $TARGET > .tmpver.1 && \
+      grep -v $IGNORE_PATTERN .tmpcompile > .tmpver.2 && \
       cmp -s .tmpver.1 .tmpver.2; then
    rm -f .tmpcompile
 else
--

2.32.0.272.g935e593368-goog

Signature · 2 lines
Best Regards
Masahiro Yamada
#2 lore raw
Quoted 21 lines · Masahiro Yamada
Quoted 19 lines · Matthias Maennich

To avoid unnecessary recompilations, mkcompile_h does not regenerate compile.h if just the timestamp changed.
Though, if KBUILD_BUILD_TIMESTAMP is set, an explicit timestamp for the build was requested, in which case we should not ignore it.

If a user follows the documentation for reproducible builds [1] and defines KBUILD_BUILD_TIMESTAMP as the git commit timestamp, a clean build will have the correct timestamp. A subsequent cherry-pick (or amend) changes the commit timestamp and if an incremental build is done with a different KBUILD_BUILD_TIMESTAMP now, that new value is not taken into consideration. But it should for reproducibility.

Hence, whenever KBUILD_BUILD_TIMESTAMP is explicitly set, do not ignore UTS_VERSION when making a decision about whether the regenerated version of compile.h should be moved into place.

[1] https://www.kernel.org/doc/html/latest/kbuild/reproducible-builds.html

Quoted 8 lines
Quoted 4 lines

---

Applied to linux-kbuild. Thanks.

This may not be a big deal, but when KBUILD_BUILD_TIMESTAMP is unset, the timestamp is not updated. It still has a user-specified string.

Signature · 2 lines
Best Regards
Masahiro Yamada
#3 lore raw
Quoted 22 lines · Masahiro Yamada
Quoted 21 lines · Masahiro Yamada
Quoted 19 lines · Matthias Maennich

To avoid unnecessary recompilations, mkcompile_h does not regenerate compile.h if just the timestamp changed.
Though, if KBUILD_BUILD_TIMESTAMP is set, an explicit timestamp for the build was requested, in which case we should not ignore it.

If a user follows the documentation for reproducible builds [1] and defines KBUILD_BUILD_TIMESTAMP as the git commit timestamp, a clean build will have the correct timestamp. A subsequent cherry-pick (or amend) changes the commit timestamp and if an incremental build is done with a different KBUILD_BUILD_TIMESTAMP now, that new value is not taken into consideration. But it should for reproducibility.

Hence, whenever KBUILD_BUILD_TIMESTAMP is explicitly set, do not ignore UTS_VERSION when making a decision about whether the regenerated version of compile.h should be moved into place.

[1] https://www.kernel.org/doc/html/latest/kbuild/reproducible-builds.html

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

---

Applied to linux-kbuild. Thanks.

Thanks!

Quoted 4 lines
Quoted 1 line

This may not be a big deal, but when KBUILD_BUILD_TIMESTAMP is unset, the timestamp is not updated. It still has a user-specified string.

I think treating this like any other 'random' prior timestamp is ok. So, I agree: not a big deal.

Cheers,
Matthias

Quoted 7 lines
Signature · 3 lines
-- 
Best Regards
Masahiro Yamada