scripts/setlocalversion: also consider annotated tags of the form vx.y.z-${file_localversion}

Commit 6ab7e1f95e96 ("setlocalversion: use only the correct release tag for git-describe") was absolutely correct to limit which annotated tags would be used to compute the -01234-gabcdef suffix. Otherwise, if some random annotated tag exists closer to HEAD than the vX.Y.Z one, the commit count would be too low.

However, since the version string always includes the ${file_localversion} part, now the problem is that the count can be too high. For example, building an 6.4.6-rt8 kernel with a few patches on top, I currently get

$ make -s kernelrelease
6.4.6-rt8-00128-gd78b7f406397

But those 128 commits include the 100 commits that are in v6.4.6..v6.4.6-rt8, so this is somewhat misleading.

Amend the logic so that, in addition to the linux-next consideration, the script also looks for a tag corresponding to the 6.4.6-rt8 part of what will become the uname -r string. With this patch (so 29 patches on top of v6.4.6-rt8), one instead gets

$ make -s kernelrelease
6.4.6-rt8-00029-gd533209291a2

While there, note that the line

  git describe --exact-match --match=$tag $tag 2>/dev/null

obviously asks if $tag is an annotated tag, but it does not actually tell if the commit pointed to has any relation to HEAD. So remove both uses of --exact-match, and instead just ask if the description generated is identical to the tag we provided. Since we then already have the result of

  git describe --match=$tag

we also end up reducing the number of times we invoke "git describe".

Signed-off-by Rasmus Villemoes <[email protected]>

---
v2: explicitly initialize desc to empty [Masahiro]

 scripts/setlocalversion | 36 ++++++++++++++++++++++++++----------
 1 file changed, 26 insertions(+), 10 deletions(-)

scripts/setlocalversion +26 -11

index 3d3babac8298..e2f49e237565 100755
@@ -57,21 +57,37 @@ scm_version()
                return
        fi
 
-       # If a localversion*' file and the corresponding annotated tag exist,
-       # use it. This is the case in linux-next.
+       # mainline kernel:  6.2.0-rc5  ->  v6.2-rc5
+       # stable kernel:    6.1.7      ->  v6.1.7
+       version_tag=v$(echo "${KERNELVERSION}" | sed -E 's/^([0-9]+\.[0-9]+)\.0(.*)$/\1\2/')
+
+       # If a localversion* file exists, and the corresponding
+       # annotated tag exists and is an ancestor of HEAD, use
+       # it. This is the case in linux-next.
        tag=${file_localversion#-}
-       tag=$(git describe --exact-match --match=$tag $tag 2>/dev/null)
+       desc=
+       if [ -n "${tag}" ]; then
+               desc=$(git describe --match=$tag 2>/dev/null)
+       fi
+
+       # Otherwise, if a localversion* file exists, and the tag
+       # obtained by appending it to the tag derived from
+       # KERNELVERSION exists and is an ancestor of HEAD, use
+       # it. This is e.g. the case in linux-rt.
+       if [ -z "${desc}" ] && [ -n "${file_localversion}" ]; then
+               tag="${version_tag}${file_localversion}"
+               desc=$(git describe --match=$tag 2>/dev/null)
+       fi
 
        # Otherwise, default to the annotated tag derived from KERNELVERSION.
-       #   mainline kernel:  6.2.0-rc5  ->  v6.2-rc5
-       #   stable kernel:    6.1.7      ->  v6.1.7
-       if [ -z "${tag}" ]; then
-               tag=v$(echo "${KERNELVERSION}" | sed -E 's/^([0-9]+\.[0-9]+)\.0(.*)$/\1\2/')
+       if [ -z "${desc}" ]; then
+               tag="${version_tag}"
+               desc=$(git describe --match=$tag 2>/dev/null)
        fi
 
        # If we are at the tagged commit, we ignore it because the version is
        # well-defined.
-       if [ -z "$(git describe --exact-match --match=$tag 2>/dev/null)" ]; then
+       if [ "${tag}" != "${desc}" ]; then
 
                # If only the short version is requested, don't bother
                # running further git commands

@@ -81,8 +97,8 @@ scm_version()
                fi
                # If we are past the tagged commit, we pretty print it.
                # (like 6.1.0-14595-g292a089d78d3)
-               if atag="$(git describe --match=$tag 2>/dev/null)"; then
-                       echo "$atag" | awk -F- '{printf("-%05d", $(NF-1))}'
+               if [ -n "${desc}" ]; then
+                       echo "${desc}" | awk -F- '{printf("-%05d", $(NF-1))}'
                fi
 
                # Add -g and exactly 12 hex chars.
--

2.37.2

Commit 6ab7e1f95e96 ("setlocalversion: use only the correct release tag for git-describe") was absolutely correct to limit which annotated tags would be used to compute the -01234-gabcdef suffix. Otherwise, if some random annotated tag exists closer to HEAD than the vX.Y.Z one, the commit count would be too low.

However, since the version string always includes the ${file_localversion} part, now the problem is that the count can be too high. For example, building an 6.4.6-rt8 kernel with a few patches on top, I currently get

$ make -s kernelrelease
6.4.6-rt8-00128-gd78b7f406397

But those 128 commits include the 100 commits that are in v6.4.6..v6.4.6-rt8, so this is somewhat misleading.

Amend the logic so that, in addition to the linux-next consideration, the script also looks for a tag corresponding to the 6.4.6-rt8 part of what will become the uname -r string. With this patch (so 29 patches on top of v6.4.6-rt8), one instead gets

$ make -s kernelrelease
6.4.6-rt8-00029-gd533209291a2

While there, note that the line

  git describe --exact-match --match=$tag $tag 2>/dev/null

obviously asks if $tag is an annotated tag, but it does not actually tell if the commit pointed to has any relation to HEAD. So remove both uses of --exact-match, and instead just ask if the description generated is identical to the tag we provided. Since we then already have the result of

  git describe --match=$tag

we also end up reducing the number of times we invoke "git describe".

Quoted 35 lines · Rasmus Villemoes

Commit 6ab7e1f95e96 ("setlocalversion: use only the correct release tag for git-describe") was absolutely correct to limit which annotated tags would be used to compute the -01234-gabcdef suffix. Otherwise, if some random annotated tag exists closer to HEAD than the vX.Y.Z one, the commit count would be too low.

However, since the version string always includes the ${file_localversion} part, now the problem is that the count can be too high. For example, building an 6.4.6-rt8 kernel with a few patches on top, I currently get

$ make -s kernelrelease
6.4.6-rt8-00128-gd78b7f406397

But those 128 commits include the 100 commits that are in v6.4.6..v6.4.6-rt8, so this is somewhat misleading.

Amend the logic so that, in addition to the linux-next consideration, the script also looks for a tag corresponding to the 6.4.6-rt8 part of what will become the uname -r string. With this patch (so 29 patches on top of v6.4.6-rt8), one instead gets

$ make -s kernelrelease
6.4.6-rt8-00029-gd533209291a2

While there, note that the line

  git describe --exact-match --match=$tag $tag 2>/dev/null

obviously asks if $tag is an annotated tag, but it does not actually tell if the commit pointed to has any relation to HEAD. So remove both uses of --exact-match, and instead just ask if the description generated is identical to the tag we provided. Since we then already have the result of

Quoted 3 lines

  git describe --match=$tag

we also end up reducing the number of times we invoke "git describe".

Dropping "--exact-match" is resulting in unnacceptable latencies for me. I don't understand what this is trying to do well enough to make a suggestion, but something has to change.

E.g. on my build box, a single git describe --match=v6.5 takes ~8.5 seconds,

whereas a complete from-scratch kernel build takes <30 seconds, and an incremental build takes <2 seconds. When build testing to-be-applied changes, I compile each commit ~15 times (different x86 configs plus one for each other KVM architecture), which makes the ~8.5 second delay beyond painful.

Quoted 7 lines · Sean Christopherson

Quoted 6 lines · Rasmus Villemoes

obviously asks if $tag is an annotated tag, but it does not actually tell if the commit pointed to has any relation to HEAD. So remove both uses of --exact-match, and instead just ask if the description generated is identical to the tag we provided. Since we then already have the result of

Quoted 7 lines

Quoted 3 lines

  git describe --match=$tag

we also end up reducing the number of times we invoke "git describe".

Dropping "--exact-match" is resulting in unnacceptable latencies for me. I don't understand what this is trying to do well enough to make a suggestion, but something has to change.

Hm, that's quite unexpected. I mean, before that commit, I think that setlocalversion, especially when run on some dev branch, would _also_ end up doing at least one 'git describe --match=v6.5'. <goes digging>

Ah, so I assume that in your case you always end up in the

# If only the short version is requested, don't bother # running further git commands
if $short; then
echo "+"
return
fi

case, i.e. you do not have CONFIG_LOCALVERSION_AUTO=y. Can you confirm that?

Quoted 1 line

E.g. on my build box, a single git describe --match=v6.5 takes ~8.5 seconds,

That's a long time indeed. I don't really know why it can take that long.

Quoted 4 lines

whereas a complete from-scratch kernel build takes <30 seconds, and an incremental build takes <2 seconds. When build testing to-be-applied changes, I compile each commit ~15 times (different x86 configs plus one for each other KVM architecture), which makes the ~8.5 second delay beyond painful.

Sorry about that. I agree something needs to be done, but the commit above fixed a real problem with -rt kernels, and the refactoring just made it much easier to follow the logic IMO - and, for the CONFIG_LOCALVERSION_AUTO=y case, did reduce the number of times git describe gets invoked.

Until I have a bit more time to think, could you try setting the env var LOCALVERSION when building - it can be set to anything, including empty. Then if I'm reading the script right, the scm_version() function won't be called at all.