gitr: Make gitr work on RHEL/CentOS
PATCH 1/1 gitr: Make gitr work on RHEL/CentOS
RHEL and CentOS do not ship cvsps or perl-TAP-Harness-Archive. For the former, we'll have to build it from git. For the latter, we can install it from CPAN, but we'll also have to install the perl-App-cpanminus package to do so.
Also add two other dependencies. First is 'make', which is currently only installed if you have KDEVOPS_TRY_INSTALL_KDEV_TOOLS enabled in your config. Second is 'tar' which is not installed in the RHEL virt-builder images by default ('tar' isn't part of the @core dnf package group).
--- playbooks/roles/gitr/defaults/main.yml | 3 ++ playbooks/roles/gitr/tasks/main.yml | 67 ++++++++++++++++++++++++++ playbooks/roles/gitr/vars/RedHat.yml | 7 ++- 3 files changed, 75 insertions(+), 2 deletions(-)
playbooks/roles/gitr/defaults/main.yml +3 -0
@@ -18,3 +18,6 @@ gitr_thread_single: false gitr_thread_fast: false gitr_thread_stress: false gitr_thread_custom: false + +cvsps_git: https://github.com/andreyvit/cvsps.git +cvsps_data: "{{ data_path }}/cvsps"
playbooks/roles/gitr/tasks/main.yml +67 -0
85 lines not on this page · raw message
playbooks/roles/gitr/vars/RedHat.yml +5 -2
@@ -1,7 +1,6 @@ --- gitr_packages: - cvs - - cvsps - expat-devel - gcc - gettext
@@ -9,11 +8,15 @@ gitr_packages: - git-email - glibc-utils - libcurl-devel + - make - openssl-devel - perl-Memoize - - perl-TAP-Harness-Archive - perl-Test-Harness - perl-Test-Simple - subversion - subversion-perl + - tar - zlib-devel + +gitr_cpan_modules: + - TAP::Harness::Archive
Signature · 1 line
2.44.0
Quoted 8 lines · Scott Mayhew
RHEL and CentOS do not ship cvsps or perl-TAP-Harness-Archive. For the former, we'll have to build it from git. For the latter, we can install it from CPAN, but we'll also have to install the perl-App-cpanminus package to do so.
Also add two other dependencies. First is 'make', which is currently only installed if you have
KDEVOPS_TRY_INSTALL_KDEV_TOOLSenabled in your config.
That's a little surprising. I guess Ansible's "make" module does not install the base make utility?
Quoted 3 lines
Second is 'tar' which is not installed in the RHEL virt-builder images by default ('tar' isn't part of the @core dnf package group).
Another option might be to disable the CVS-related tests by not installing CVS on the target nodes. The git test suite should pick up that omission and then not run those tests.
Quoted 143 lines
Signed-off-by Scott Mayhew <[email protected]>--- playbooks/roles/gitr/defaults/main.yml | 3 ++ playbooks/roles/gitr/tasks/main.yml | 67 ++++++++++++++++++++++++++ playbooks/roles/gitr/vars/RedHat.yml | 7 ++- 3 files changed, 75 insertions(+), 2 deletions(-)playbooks/roles/gitr/defaults/main.yml +3
-0@@ -18,3 +18,6 @@ gitr_thread_single: false gitr_thread_fast: false gitr_thread_stress: false gitr_thread_custom: false + +cvsps_git: https://github.com/andreyvit/cvsps.git +cvsps_data: "{{ data_path }}/cvsps"playbooks/roles/gitr/tasks/main.yml +67
-0@@ -11,6 +11,11 @@ failed_when: false tags: vars +- name: Set up the /data mount point + tags: ['data_partition'] + ansible.builtin.include_role: + name: create_data_partition + - name: Set the name of the test group tags: vars ansible.builtin.set_fact:@@ -42,6 +47,21 @@ paths: - 'vars' +- name: Update gitr depencies for RHEL/Centos + tags: gitr + ansible.builtin.set_fact: + gitr_packages: "{{ gitr_packages + ['perl-App-cpanminus'] }}" + when: + - ansible_distribution == "RedHat" or + ansible_distribution == "CentOS" + +- name: Update gitr dependencies for Fedora + tags: gitr + ansible.builtin.set_fact: + gitr_packages: "{{ gitr_packages + ['cvsps', 'perl-TAP-Harness-Archive'] }}" + when: + - ansible_distribution == "Fedora" + - name: Install dependencies for gitr tags: gitr become: true@@ -51,6 +71,53 @@ name: "{{ gitr_packages }}" state: present +- name: Install CPAN modules for gitr + tags: gitr + become: true + become_flags: 'su - -c' + become_method: ansible.builtin.sudo + community.general.cpanm: + name: "{{ item }}" + with_items: "{{ gitr_cpan_modules }}" + when: + - ansible_distribution == "RedHat" or + ansible_distribution == "CentOS" + +- name: git clone cvsps + tags: gitr + ansible.builtin.git: + repo: "{{ cvsps_git }}" + dest: "{{ cvsps_data }}" + update: true + retries: 3 + delay: 5 + register: result + until: not result.failed + when: + - ansible_distribution == "RedHat" or + ansible_distribution == "CentOS" + +- name: Build cvsps + tags: gitr + community.general.make: + chdir: "{{ cvsps_data }}" + jobs: "{{ ansible_processor_nproc }}" + when: + - ansible_distribution == "RedHat" or + ansible_distribution == "CentOS" + +- name: Install cvsps + tags: gitr + become: true + become_flags: 'su - -c' + become_method: ansible.builtin.sudo + community.general.make: + target: install + chdir: "{{ cvsps_data }}" + when: + - ansible_distribution == "RedHat" or + ansible_distribution == "CentOS" + - name: Set up an iSCSI initiator on the target node ansible.builtin.include_role: name: iscsiplaybooks/roles/gitr/vars/RedHat.yml +5
-2@@ -1,7 +1,6 @@ --- gitr_packages: - cvs - - cvsps - expat-devel - gcc - gettext@@ -9,11 +8,15 @@ gitr_packages: - git-email - glibc-utils - libcurl-devel + - make - openssl-devel - perl-Memoize - - perl-TAP-Harness-Archive - perl-Test-Harness - perl-Test-Simple - subversion - subversion-perl + - tar - zlib-devel + +gitr_cpan_modules: + - TAP::Harness::ArchiveSignature · 1 line
2.44.0
Signature · 1 line
Chuck Lever
Quoted 6 lines · Chuck Lever
Quoted 5 lines · Scott Mayhew
RHEL and CentOS do not ship cvsps or perl-TAP-Harness-Archive. For the former, we'll have to build it from git. For the latter, we can install it from CPAN, but we'll also have to install the perl-App-cpanminus package to do so.
Quoted 6 lines
Quoted 3 lines
Also add two other dependencies. First is 'make', which is currently only installed if you have
KDEVOPS_TRY_INSTALL_KDEV_TOOLSenabled in your config.That's a little surprising. I guess Ansible's "make" module does not install the base make utility?
AFAIK ansible modules just run the commands. It's up to the user to make sure the software that provide those commands is already installed on the target systems.
Quoted 9 lines
Quoted 5 lines
Second is 'tar' which is not installed in the RHEL virt-builder images by default ('tar' isn't part of the @core dnf package group).
Another option might be to disable the CVS-related tests by not installing CVS on the target nodes. The git test suite should pick up that omission and then not run those tests.
You're saying that only the CVS-related tests are using tar? Sounds weird, but I can try it if you want. Personally I'd rather have the same set of tests running on my RHEL targets that are running on my Fedora targets.
<snip/>
BTW, I think this will probably break the test on Oracle Linux.
Quoted 3 lines
playbooks/roles/gitr/tasks/main.yml line 56
Quoted 3 lines
playbooks/roles/gitr/tasks/main.yml +3
-0+ when: + - ansible_distribution == "RedHat" or + ansible_distribution == "CentOS"
I think 'when: ansible_distribution != "Fedora"' will work better.
-Scott
Quoted 16 lines · Scott Mayhew
Quoted 10 lines · Chuck Lever
Quoted 8 lines · Scott Mayhew
RHEL and CentOS do not ship cvsps or perl-TAP-Harness-Archive. For the former, we'll have to build it from git. For the latter, we can install it from CPAN, but we'll also have to install the perl-App-cpanminus package to do so.
Also add two other dependencies. First is 'make', which is currently only installed if you have
KDEVOPS_TRY_INSTALL_KDEV_TOOLSenabled in your config.
Quoted 6 lines
Quoted 2 lines
That's a little surprising. I guess Ansible's "make" module does not install the base make utility?
AFAIK ansible modules just run the commands. It's up to the user to make sure the software that provide those commands is already installed on the target systems.
My Buildbot configurations do not have INSTALL_KDEV_TOOLS enabled, but they seem to have no problem running the gitr workflow. I'm thinking there might be some other reason why "make" is not available -- ie, my quibble is with the patch description, not with adding the dependency.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Second is 'tar' which is not installed in the RHEL virt-builder images by default ('tar' isn't part of the @core dnf package group).
Quoted 8 lines
Quoted 3 lines
Another option might be to disable the CVS-related tests by not installing CVS on the target nodes. The git test suite should pick up that omission and then not run those tests.
You're saying that only the CVS-related tests are using tar? Sounds weird, but I can try it if you want. Personally I'd rather have the same set of tests running on my RHEL targets that are running on my Fedora targets.
That's sensible -- let's keep them the same.
Quoted 8 lines
<snip/>
BTW, I think this will probably break the test on Oracle Linux.
Quoted 3 lines
playbooks/roles/gitr/tasks/main.yml line 56
Quoted 3 lines
playbooks/roles/gitr/tasks/main.yml +3
-0+ when: + - ansible_distribution == "RedHat" or + ansible_distribution == "CentOS"I think 'when:
ansible_distribution!= "Fedora"' will work better.
No argument from me. I probably copied that from somewhere else.
--
Chuck Lever
Quoted 23 lines · Chuck Lever III
Quoted 16 lines · Scott Mayhew
Quoted 10 lines · Chuck Lever
Quoted 8 lines · Scott Mayhew
RHEL and CentOS do not ship cvsps or perl-TAP-Harness-Archive. For the former, we'll have to build it from git. For the latter, we can install it from CPAN, but we'll also have to install the perl-App-cpanminus package to do so.
Also add two other dependencies. First is 'make', which is currently only installed if you have
KDEVOPS_TRY_INSTALL_KDEV_TOOLSenabled in your config.Quoted 6 lines
Quoted 2 lines
That's a little surprising. I guess Ansible's "make" module does not install the base make utility?
AFAIK ansible modules just run the commands. It's up to the user to make sure the software that provide those commands is already installed on the target systems.
Quoted 5 lines
My Buildbot configurations do not have
INSTALL_KDEV_TOOLSenabled, but they seem to have no problem running the gitr workflow. I'm thinking there might be some other reason why "make" is not available -- ie, my quibble is with the patch description, not with adding the dependency.
Ok, let me double-check the Fedora images to see where they're getting 'make' from.
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 5 lines
Second is 'tar' which is not installed in the RHEL virt-builder images by default ('tar' isn't part of the @core dnf package group).
Another option might be to disable the CVS-related tests by not installing CVS on the target nodes. The git test suite should pick up that omission and then not run those tests.
Quoted 6 lines
Quoted 4 lines
You're saying that only the CVS-related tests are using tar? Sounds weird, but I can try it if you want. Personally I'd rather have the same set of tests running on my RHEL targets that are running on my Fedora targets.
That's sensible -- let's keep them the same.
Sounds good.
Quoted 4 lines
Quoted 4 lines
<snip/>
Quoted 8 lines
Quoted 6 lines
BTW, I think this will probably break the test on Oracle Linux.
Quoted 3 lines
playbooks/roles/gitr/tasks/main.yml line 56
Quoted 3 lines
playbooks/roles/gitr/tasks/main.yml +3
-0+ when: + - ansible_distribution == "RedHat" or + ansible_distribution == "CentOS"I think 'when:
ansible_distribution!= "Fedora"' will work better.No argument from me. I probably copied that from somewhere else.
Those new 'when' clauses were added by my patch and I was just pointing out my mistake.
-Scott
Quoted 6 lines
--
Chuck Lever
Quoted 25 lines · Scott Mayhew
Quoted 23 lines · Chuck Lever III
Quoted 22 lines · Scott Mayhew
Quoted 6 lines · Chuck Lever
Quoted 5 lines · Scott Mayhew
RHEL and CentOS do not ship cvsps or perl-TAP-Harness-Archive. For the former, we'll have to build it from git. For the latter, we can install it from CPAN, but we'll also have to install the perl-App-cpanminus package to do so.
Quoted 6 lines
Quoted 3 lines
Also add two other dependencies. First is 'make', which is currently only installed if you have
KDEVOPS_TRY_INSTALL_KDEV_TOOLSenabled in your config.That's a little surprising. I guess Ansible's "make" module does not install the base make utility?
AFAIK ansible modules just run the commands. It's up to the user to make sure the software that provide those commands is already installed on the target systems.
Quoted 8 lines
Quoted 5 lines
My Buildbot configurations do not have
INSTALL_KDEV_TOOLSenabled, but they seem to have no problem running the gitr workflow. I'm thinking there might be some other reason why "make" is not available -- ie, my quibble is with the patch description, not with adding the dependency.Ok, let me double-check the Fedora images to see where they're getting 'make' from.
It looks like the real culprit is that the "Install btrfs-progs" play in playbooks/roles/devconfig/tasks/install-deps/redhat/main.yml doesn't really do what it claims. It's actually installing the entire set of packages from the "Build install package list" play right before it. That's how we're magically getting both the make and tar packages on Fedora... make included in that list explicitly, and tar is getting pulled in as a dependency of libtool.
Quoted 41 lines
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 5 lines
Second is 'tar' which is not installed in the RHEL virt-builder images by default ('tar' isn't part of the @core dnf package group).
Another option might be to disable the CVS-related tests by not installing CVS on the target nodes. The git test suite should pick up that omission and then not run those tests.
Quoted 6 lines
Quoted 4 lines
You're saying that only the CVS-related tests are using tar? Sounds weird, but I can try it if you want. Personally I'd rather have the same set of tests running on my RHEL targets that are running on my Fedora targets.
That's sensible -- let's keep them the same.
Sounds good.
Quoted 4 lines
Quoted 4 lines
<snip/>
Quoted 8 lines
Quoted 6 lines
BTW, I think this will probably break the test on Oracle Linux.
Quoted 3 lines
playbooks/roles/gitr/tasks/main.yml line 56
Quoted 3 lines
playbooks/roles/gitr/tasks/main.yml +3
-0+ when: + - ansible_distribution == "RedHat" or + ansible_distribution == "CentOS"I think 'when:
ansible_distribution!= "Fedora"' will work better.No argument from me. I probably copied that from somewhere else.
Those new 'when' clauses were added by my patch and I was just pointing out my mistake.
-Scott
Quoted 6 lines
--
Chuck Lever
Quoted 37 lines · Scott Mayhew
Quoted 23 lines · Scott Mayhew
Quoted 21 lines · Chuck Lever III
Quoted 20 lines · Scott Mayhew
Quoted 6 lines · Chuck Lever
Quoted 5 lines · Scott Mayhew
RHEL and CentOS do not ship cvsps or perl-TAP-Harness-Archive. For the former, we'll have to build it from git. For the latter, we can install it from CPAN, but we'll also have to install the perl-App-cpanminus package to do so.
Quoted 6 lines
Quoted 3 lines
Also add two other dependencies. First is 'make', which is currently only installed if you have
KDEVOPS_TRY_INSTALL_KDEV_TOOLSenabled in your config.That's a little surprising. I guess Ansible's "make" module does not install the base make utility?
AFAIK ansible modules just run the commands. It's up to the user to make sure the software that provide those commands is already installed on the target systems.
Quoted 8 lines
Quoted 5 lines
My Buildbot configurations do not have
INSTALL_KDEV_TOOLSenabled, but they seem to have no problem running the gitr workflow. I'm thinking there might be some other reason why "make" is not available -- ie, my quibble is with the patch description, not with adding the dependency.Ok, let me double-check the Fedora images to see where they're getting 'make' from.
Quoted 7 lines
It looks like the real culprit is that the "Install btrfs-progs" play in
playbooks/roles/devconfig/tasks/install-deps/redhat/main.ymldoesn't really do what it claims. It's actually installing the entire set of packages from the "Build install package list" play right before it. That's how we're magically getting both the make and tar packages on Fedora... make included in that list explicitly, and tar is getting pulled in as a dependency of libtool.
That might explain why that step was hanging for so long when the public package repositories are slow.
No objection from me to adding make/tar explicitly for the gitr workflow. Root cause now looks plausible to me.
--
Chuck Lever