ci: install P4 from package to fix build error
PATCH 1/1 ci: install P4 from package to fix build error
To test 'git-p4' in the Linux Clang and GCC build jobs we used to install the 'p4' and 'p4d' binaries by directly downloading those binaries from a Perforce filehost. This has worked just fine ever since we started using Travis CI [1], but during the last day or so that filehost appeared to be gone: while its hostname still resolves, the host doesn't seem to reply to any download request, it doesn't even refuse the connection, and eventually our build jobs time out [2].
Now, this might be just a temporary glitch, but I'm afraid that it isn't. The "Helix Core Server Administrator Guide" [3] describes two ways to install these binaries on Linux, and none of them mentions the filehost that we've been downloading from in the past:
- non-package installation: open the website's download page in your web browser, select OS and platform, click on the download link, and eventually you get a .tar.gz archive containing, among other things, the necessary 'p4' and 'p4d' binaries.
Although we could use the URL of this archive to download it in our CI scripts with 'wget', nobody said that that URL remains stable, and we would still need to extract the archive and copy the binaries to $PATH.
- package installation for various distros, including Ubuntu 16.04 (i.e. the Ubuntu version used both in our Travis CI and Azure Pipelines builds): add a package repository and its pubkey, 'apt-get update && apt-get install', and ready to go.
Let's install P4 from the package repository, because this approach seems to be simpler and more future proof.
Note that we used to install an old P4 version (2016.2) in the Linux build jobs, but with this change we'll install the most recent version available in the Perforce package repository (currently 2019.1).
[1] 522354d70f (Add Travis CI support, 2015-11-27).
[2] https://travis-ci.org/git/git/jobs/581429927#L422
[3] https://www.perforce.com/manuals/p4sag/Content/P4SAG/chapter.install.html
--- ci/install-dependencies.sh | 14 +++++--------- ci/lib.sh | 8 +++----- 2 files changed, 8 insertions(+), 14 deletions(-)
ci/install-dependencies.sh +5 -9
@@ -5,27 +5,23 @@ . ${0%/*}/lib.sh -P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION case "$jobname" in linux-clang|linux-gcc) + wget -qO - https://package.perforce.com/perforce.pubkey | sudo apt-key add - + echo "deb http://package.perforce.com/apt/ubuntu xenial release" >perforce.list + sudo mv perforce.list /etc/apt/sources.list.d/ sudo apt-add-repository -y "ppa:ubuntu-toolchain-r/test" sudo apt-get -q update - sudo apt-get -q -y install language-pack-is libsvn-perl apache2 + sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ + helix-p4d case "$jobname" in linux-gcc) sudo apt-get -q -y install gcc-8 ;; esac - mkdir --parents "$P4_PATH" - pushd "$P4_PATH" - wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d" - wget --quiet "$P4WHENCE/bin.linux26x86_64/p4" - chmod u+x p4d - chmod u+x p4 - popd mkdir --parents "$GIT_LFS_PATH" pushd "$GIT_LFS_PATH" wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
ci/lib.sh +3 -5
@@ -162,17 +162,15 @@ linux-clang|linux-gcc) export GIT_TEST_HTTPD=YesPlease - # The Linux build installs the defined dependency versions below. - # The OS X build installs much more recent versions, whichever + # The Linux build installs the defined dependency version below. + # The OS X build installs much more recent version, whichever # were recorded in the Homebrew database upon creating the OS X # image. # Keep that in mind when you encounter a broken OS X build! - export LINUX_P4_VERSION="16.2" export LINUX_GIT_LFS_VERSION="1.5.2" - P4_PATH="$HOME/custom/p4" GIT_LFS_PATH="$HOME/custom/git-lfs" - export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" + export PATH="$GIT_LFS_PATH:$PATH" ;; osx-clang|osx-gcc) if [ "$jobname" = osx-gcc ]
Signature · 1 line
2.23.0.331.g4e51dcdf11
Quoted 11 lines · SZEDER Gábor
To test 'git-p4' in the Linux Clang and GCC build jobs we used to install the 'p4' and 'p4d' binaries by directly downloading those binaries from a Perforce filehost. This has worked just fine ever since we started using Travis CI [1], but during the last day or so that filehost appeared to be gone: while its hostname still resolves, the host doesn't seem to reply to any download request, it doesn't even refuse the connection, and eventually our build jobs time out [2].
Now, this might be just a temporary glitch, but I'm afraid that it isn't.
Well, now would you believe it, while I was testing this patch (I even made a gitgitgadget PR to run it on Azure Pipelines! :) and touching up its log message the good old Perforce filehost sprang back to life, and the CI build jobs now succeed again even without this patch.
Quoted 6 lines
Let's install P4 from the package repository, because this approach seems to be simpler and more future proof.
Note that we used to install an old P4 version (2016.2) in the Linux build jobs, but with this change we'll install the most recent version available in the Perforce package repository (currently 2019.1).
So I'm not quite sure whether we really want this patch. It depends on how important it is to test 'git-p4' with an old P4 version, but I don't really have an opinion on that.
Hi,
Quoted 10 lines · SZEDER Gábor
Quoted 9 lines · SZEDER Gábor
To test 'git-p4' in the Linux Clang and GCC build jobs we used to install the 'p4' and 'p4d' binaries by directly downloading those binaries from a Perforce filehost. This has worked just fine ever since we started using Travis CI [1], but during the last day or so that filehost appeared to be gone: while its hostname still resolves, the host doesn't seem to reply to any download request, it doesn't even refuse the connection, and eventually our build jobs time out [2].
Quoted 7 lines
Quoted 2 lines
Now, this might be just a temporary glitch, but I'm afraid that it isn't.
Well, now would you believe it, while I was testing this patch (I even made a gitgitgadget PR to run it on Azure Pipelines! :) and touching up its log message the good old Perforce filehost sprang back to life, and the CI build jobs now succeed again even without this patch.
Sorry for being so slow with granting you access to GitGitGadget. FWIW _anybody_ who already was granted access can issue /allow commands, it is not just me.
Quoted 10 lines
Quoted 6 lines
Let's install P4 from the package repository, because this approach seems to be simpler and more future proof.
Note that we used to install an old P4 version (2016.2) in the Linux build jobs, but with this change we'll install the most recent version available in the Perforce package repository (currently 2019.1).
So I'm not quite sure whether we really want this patch. It depends on how important it is to test 'git-p4' with an old P4 version, but I don't really have an opinion on that.
I'd rather have that patch. It seems to be a much better idea to use the package management system than to rely on one host, especially when said host already displayed hiccups.
Ciao,
Dscho
Quoted 15 lines · Johannes Schindelin
Quoted 13 lines · SZEDER Gábor
Quoted 11 lines · SZEDER Gábor
To test 'git-p4' in the Linux Clang and GCC build jobs we used to install the 'p4' and 'p4d' binaries by directly downloading those binaries from a Perforce filehost. This has worked just fine ever since we started using Travis CI [1], but during the last day or so that filehost appeared to be gone: while its hostname still resolves, the host doesn't seem to reply to any download request, it doesn't even refuse the connection, and eventually our build jobs time out [2].
Now, this might be just a temporary glitch, but I'm afraid that it isn't.
Quoted 8 lines
Quoted 4 lines
Well, now would you believe it, while I was testing this patch (I even made a gitgitgadget PR to run it on Azure Pipelines! :) and touching up its log message the good old Perforce filehost sprang back to life, and the CI build jobs now succeed again even without this patch.
Sorry for being so slow with granting you access to GitGitGadget. FWIW _anybody_ who already was granted access can issue
/allowcommands, it is not just me.
No problem; I was only interested in the results of the Azure Pipelines build, and that seemed to go well even without access.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Let's install P4 from the package repository, because this approach seems to be simpler and more future proof.
Note that we used to install an old P4 version (2016.2) in the Linux build jobs, but with this change we'll install the most recent version available in the Perforce package repository (currently 2019.1).
Quoted 7 lines
Quoted 3 lines
So I'm not quite sure whether we really want this patch. It depends on how important it is to test 'git-p4' with an old P4 version, but I don't really have an opinion on that.
I'd rather have that patch. It seems to be a much better idea to use the package management system than to rely on one host, especially when said host already displayed hiccups.
Well, I'm not so sure. As far as I remember this was the first time that this Perforce filehost was inaccessible and a simple "Restart job" could not rectify the situation, because it was inaccessible for about a day or more. OTOH, transient errors or timeouts from 'apt-get update' or 'install' from the official Ubuntu package repositories are not uncommon (at least on Travis CI), although in those cases it's usually enough to just restart the errored job.
Hi,
Quoted 17 lines · SZEDER Gábor
Quoted 16 lines · Johannes Schindelin
Quoted 13 lines · SZEDER Gábor
Quoted 7 lines · SZEDER Gábor
Let's install P4 from the package repository, because this approach seems to be simpler and more future proof.
Note that we used to install an old P4 version (2016.2) in the Linux build jobs, but with this change we'll install the most recent version available in the Perforce package repository (currently 2019.1).
So I'm not quite sure whether we really want this patch. It depends on how important it is to test 'git-p4' with an old P4 version, but I don't really have an opinion on that.
Quoted 8 lines
Quoted 3 lines
I'd rather have that patch. It seems to be a much better idea to use the package management system than to rely on one host, especially when said host already displayed hiccups.
Well, I'm not so sure. As far as I remember this was the first time that this Perforce filehost was inaccessible and a simple "Restart job" could not rectify the situation, because it was inaccessible for about a day or more.
Right.
Quoted 4 lines
OTOH, transient errors or timeouts from 'apt-get update' or 'install' from the official Ubuntu package repositories are not uncommon (at least on Travis CI), although in those cases it's usually enough to just restart the errored job.
My impression precisely. I trust the Ubuntu package servers to have transient, _short-lived_ problems ;-)
Ciao,
Dscho
Quoted 30 lines · Johannes Schindelin
Quoted 17 lines · SZEDER Gábor
Quoted 16 lines · Johannes Schindelin
Quoted 13 lines · SZEDER Gábor
Quoted 7 lines · SZEDER Gábor
Let's install P4 from the package repository, because this approach seems to be simpler and more future proof.
Note that we used to install an old P4 version (2016.2) in the Linux build jobs, but with this change we'll install the most recent version available in the Perforce package repository (currently 2019.1).
So I'm not quite sure whether we really want this patch. It depends on how important it is to test 'git-p4' with an old P4 version, but I don't really have an opinion on that.
Quoted 8 lines
Quoted 3 lines
I'd rather have that patch. It seems to be a much better idea to use the package management system than to rely on one host, especially when said host already displayed hiccups.
Well, I'm not so sure. As far as I remember this was the first time that this Perforce filehost was inaccessible and a simple "Restart job" could not rectify the situation, because it was inaccessible for about a day or more.
Right.
Quoted 7 lines
Quoted 4 lines
OTOH, transient errors or timeouts from 'apt-get update' or 'install' from the official Ubuntu package repositories are not uncommon (at least on Travis CI), although in those cases it's usually enough to just restart the errored job.
My impression precisely. I trust the Ubuntu package servers to have transient, _short-lived_ problems ;-)
Heh, that's true, but note that we won't install P4 from the Ubuntu package servers, but from the Perforce package repository, so I doubt we can expect any improvements in this regard.
OTOH, instead of issuing only two HTTP requests to download those two binaries we'll issue a total of 9 requests (1 pubkey, 2 package lists, 6 packages) that could go wrong.
I'm undecided, and at the very least the proposed commit message should be updated.