Skip to contentSkip to conversation

test-lib: try harder to ensure a working jgit

PATCH Superseded Todd Zullinger 6 participants 11 messages May 14, 2019 – May 15, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 test-lib: try harder to ensure a working jgit

The JGIT prereq uses 'type jgit' to determine whether jgit is present. While this should be sufficient, if the jgit found is broken we'll waste time running tests which fail due to no fault of our own.

Use 'jgit --version' instead, to catch some badly broken jgit installations.

---
I ran into such a broken jgit on Fedora >= 30¹. This is clearly a problem in the Fedora jgit package which will hopefully be resolved soon. But it may be good to avoid wasting time debugging tests which fail due to a broken tool outside of our control.

¹ https://bugzilla.redhat.com/1709624

 t/test-lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/test-lib.sh +1 -1
index 908ddb9c46..599fd70e14 100644
@@ -1522,7 +1522,7 @@ test_lazy_prereq NOT_ROOT '
 '
 
 test_lazy_prereq JGIT '
-	type jgit
+	jgit --version
 '
 
 # SANITY is about "can you correctly predict what the filesystem would
Signature · 1 line
Todd
#1 lore raw

t/test-lib.sh test_lazy_prereq NOT_ROOT ' · line 1526

Quoted 11 lines · Todd Zullinger
t/test-lib.sh +1 -1
index 908ddb9c46..599fd70e14 100644
@@ -1522,7 +1522,7 @@ test_lazy_prereq NOT_ROOT '
 '
 
 test_lazy_prereq JGIT '
-	type jgit
+	jgit --version
 '

I think this is an improvement, not only because of the reasons you mentioned, but because we remove the use of "type", which is not guaranteed to be present in a POSIX shell.

Signature · 2 lines
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204
#2 lore raw
Quoted 8 lines · brian m. carlson
Quoted 7 lines · Todd Zullinger
t/test-lib.sh +0 -0
index 908ddb9c46..599fd70e14 100644
@@ -1522,7 +1522,7 @@ test_lazy_prereq NOT_ROOT '
 '
 
Quoted 8 lines

t/test-lib.sh test_lazy_prereq NOT_ROOT ' · line 1526

Quoted 4 lines
t/test-lib.sh +1 -1
 test_lazy_prereq JGIT '
-	type jgit
+	jgit --version
 '

I think this is an improvement, not only because of the reasons you mentioned, but because we remove the use of "type", which is not guaranteed to be present in a POSIX shell.

Isn't it? I have always treated it as the most-portable option for this (compared to, say, which). It is in POSIX as a utility (albeit marked with XSI), which even says (in APPLICATION USAGE):

Since type must be aware of the contents of the current shell execution environment (such as the lists of commands, functions, and built-ins processed by hash), it is always provided as a shell regular built-in.

All that said, I think Todd's patch makes perfect sense even without wanting to avoid "type".

-Peff

#3 lore raw
Quoted 2 lines · Jeff King

All that said, I think Todd's patch makes perfect sense even without wanting to avoid "type".

Same here. t/lib-bash.sh seems to use "if type bash" to see if one is available on $PATH; I've never felt the need to avoid "type".

#4 lore raw
In reply to Jeff King

Hi,

Quoted 19 lines · Jeff King
Quoted 8 lines · brian m. carlson
Quoted 7 lines · Todd Zullinger
t/test-lib.sh +0 -0
index 908ddb9c46..599fd70e14 100644
@@ -1522,7 +1522,7 @@ test_lazy_prereq NOT_ROOT '
 '
 
Quoted 8 lines

t/test-lib.sh test_lazy_prereq NOT_ROOT ' · line 1526

Quoted 4 lines
t/test-lib.sh +1 -1
 test_lazy_prereq JGIT '
-	type jgit
+	jgit --version
 '

I think this is an improvement, not only because of the reasons you mentioned, but because we remove the use of "type", which is not guaranteed to be present in a POSIX shell.

Quoted 1 line

Isn't it?

I wondered the same thing, but I know I am not nearly as familiar with the POSIX rules as any of you.

Quoted 11 lines

I have always treated it as the most-portable option for this (compared to, say, which). It is in POSIX as a utility (albeit marked with XSI), which even says (in APPLICATION USAGE):

Since type must be aware of the contents of the current shell execution environment (such as the lists of commands, functions, and built-ins processed by hash), it is always provided as a shell regular built-in.

All that said, I think Todd's patch makes perfect sense even without wanting to avoid "type".

Yeah, which surely isn't a portable option. I presumed type must be fairly widely available since it was in the test suite since you added it way back in 212f2ffbf0 ("t:
add basic bitmap functionality tests", 2013-12-21).

I usually make use of command -p -v $foo in scripts that need to be portable across systems. But I don't have access to many esoteric systems.

Based on Junio's follow-up, I think we can avoid adding anything to the commit message about the use of type here. That way no one will take it as a sign that we should remove other uses of it just for conformance. (I will send a follow-up with an update based on Jonathan and Ævar's comments.)

Thanks to all of you.

Signature · 1 line
Todd
#5 lore raw
In reply to Jeff King
Quoted 5 lines · Jeff King
Quoted 3 lines · brian m. carlson

I think this is an improvement, not only because of the reasons you mentioned, but because we remove the use of "type", which is not guaranteed to be present in a POSIX shell.

Quoted 8 lines

Isn't it? I have always treated it as the most-portable option for this (compared to, say, which). It is in POSIX as a utility (albeit marked with XSI), which even says (in APPLICATION USAGE):

Since type must be aware of the contents of the current shell execution environment (such as the lists of commands, functions, and built-ins processed by hash), it is always provided as a shell regular built-in.

It's an XSI extension, while "command -v" is not. "type" may be more common for historical reasons, but if you have systems that don't implement XSI, "command -v" is the way to go.

I suppose this doesn't matter unless we have people try to build with Debian's posh package, which only implements the minimum requirements for a Debian /bin/sh (which don't include XSI extensions, but do include local).

Quoted 2 lines

All that said, I think Todd's patch makes perfect sense even without wanting to avoid "type".

I agree this is an improvement either way.

Signature · 2 lines
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204
Reviewed-by on v1 1/1
#6 lore raw
In reply to Todd Zullinger
Quoted 7 lines · Todd Zullinger

The JGIT prereq uses 'type jgit' to determine whether jgit is present. While this should be sufficient, if the jgit found is broken we'll waste time running tests which fail due to no fault of our own.

Use 'jgit --version' instead, to catch some badly broken jgit installations.

Quoted 8 lines

---
I ran into such a broken jgit on Fedora >= 30¹. This is clearly a problem in the Fedora jgit package which will hopefully be resolved soon. But it may be good to avoid wasting time debugging tests which fail due to a broken tool outside of our control.

¹ https://bugzilla.redhat.com/1709624

It would be nice to describe that bug in the commit message, to save readers some head scratching.

Thanks,
Jonathan

#7 lore raw
Quoted 16 lines · Jonathan Nieder
Quoted 14 lines · Todd Zullinger

The JGIT prereq uses 'type jgit' to determine whether jgit is present. While this should be sufficient, if the jgit found is broken we'll waste time running tests which fail due to no fault of our own.

Use 'jgit --version' instead, to catch some badly broken jgit installations.

---
I ran into such a broken jgit on Fedora >= 30¹. This is clearly a problem in the Fedora jgit package which will hopefully be resolved soon. But it may be good to avoid wasting time debugging tests which fail due to a broken tool outside of our control.

Quoted 6 lines
Quoted 1 line

¹ https://bugzilla.redhat.com/1709624

It would be nice to describe that bug in the commit message, to save readers some head scratching.

FWIW the jgit in Debian testing/unstable is similarly broken right now:

    $ apt policy jgit-cli
    jgit-cli:
      Installed: 3.7.1-6
      Candidate: 3.7.1-6
      Version table:
     *** 3.7.1-6 900
            900 http://ftp.nl.debian.org/debian testing/main amd64 Packages
            800 http://ftp.nl.debian.org/debian unstable/main amd64 Packages
            100 /var/lib/dpkg/status
         3.7.1-4 700
            700 http://ftp.nl.debian.org/debian stable/main amd64 Packages
    $ jgit --version; echo $?
    Error: A JNI error has occurred, please check your installation and try again
    Exception in thread "main" java.lang.NoClassDefFoundError: org/kohsuke/args4j/CmdLineException
            at java.lang.Class.getDeclaredMethods0(Native Method)
            at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
            at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
            at java.lang.Class.getMethod0(Class.java:3018)
            at java.lang.Class.getMethod(Class.java:1784)
            at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
            at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
    Caused by: java.lang.ClassNotFoundException: org.kohsuke.args4j.CmdLineException
            at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
            at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
            ... 7 more
    1

So rather than describe specific bugs on RedHat/Debian maybe just say:

This guards against cases where jgit is present on the system, but will fail to run, e.g. because of some JRE issue, or missing Java dependencies. Seeing if it gets far enough to process the "--version" argument isn't perfect, but seems to be good enough in practice. It's also consistent with how we detect some other dependencies, see e.g. the CURL and UNZIP prerequisites.

#8 lore raw

Hi,

Quoted 19 lines · Ævar Arnfjörð Bjarmason
Quoted 16 lines · Jonathan Nieder
Quoted 14 lines · Todd Zullinger

The JGIT prereq uses 'type jgit' to determine whether jgit is present. While this should be sufficient, if the jgit found is broken we'll waste time running tests which fail due to no fault of our own.

Use 'jgit --version' instead, to catch some badly broken jgit installations.

---
I ran into such a broken jgit on Fedora >= 30¹. This is clearly a problem in the Fedora jgit package which will hopefully be resolved soon. But it may be good to avoid wasting time debugging tests which fail due to a broken tool outside of our control.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

¹ https://bugzilla.redhat.com/1709624

It would be nice to describe that bug in the commit message, to save readers some head scratching.

FWIW the jgit in Debian testing/unstable is similarly broken right now:

[...]

Hah, small world. :)

Quoted 8 lines

So rather than describe specific bugs on RedHat/Debian maybe just say:

This guards against cases where jgit is present on the system, but will fail to run, e.g. because of some JRE issue, or missing Java dependencies. Seeing if it gets far enough to process the "--version" argument isn't perfect, but seems to be good enough in practice. It's also consistent with how we detect some other dependencies, see e.g. the CURL and UNZIP prerequisites.

Well said. I indeed avoided putting the detail into the commit message because it was such a Fedora-specific bug. I'll update the commit message to add more details though, borrowing liberally from^W^W^Wperhaps stealing your suggested wording.

Thanks!

Signature · 1 line
Todd
#9 lore raw
Quoted 7 lines · Todd Zullinger
Quoted 6 lines

This guards against cases where jgit is present on the system, but will fail to run, e.g. because of some JRE issue, or missing Java dependencies. Seeing if it gets far enough to process the "--version" argument isn't perfect, but seems to be good enough in practice. It's also consistent with how we detect some other dependencies, see e.g. the CURL and UNZIP prerequisites.

Quoted 7 lines

Well said. I indeed avoided putting the detail into the commit message because it was such a Fedora-specific bug. I'll update the commit message to add more details though, borrowing liberally from^W^W^Wperhaps stealing your suggested wording.

Thanks!

Thanks, all.

#10 lore raw

PATCH v2 1/1 test-lib: try harder to ensure a working jgit

In reply to Todd Zullinger

The JGIT prereq uses type jgit to determine whether jgit is present. While this is usually sufficient, it won't help if the jgit found is badly broken. This wastes time running tests which fail due to no fault of our own.

Use jgit --version instead, to guard against cases where jgit is present on the system, but will fail to run, e.g. because of some JRE issue, or missing Java dependencies. Checking that it gets far enough to process the '--version' argument isn't perfect, but seems to be good enough in practice. It's also consistent with how we detect some other dependencies, see e.g. the CURL and UNZIP prerequisites.

---
As promised, I stole the second paragraph from Ævar nearly verbatim. :)

 t/test-lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/test-lib.sh +1 -1
index 908ddb9c46..599fd70e14 100644
@@ -1522,7 +1522,7 @@ test_lazy_prereq NOT_ROOT '
 '
 
 test_lazy_prereq JGIT '
-	type jgit
+	jgit --version
 '
 
 # SANITY is about "can you correctly predict what the filesystem would
Signature · 1 line
Todd