Fixes for recent commits
cover Fixes for recent commits
From: Chuck Lever <[email protected]>
I rebased on linux-kdevops/kdevops : main yesterday and found some new issues when trying to run NFSD-related workflows. Many of the workflows did not work because host groups were missing from the Ansible inventory.
The "Replace pynfs hosts template" patch could replace the workflows/pynfs.j2 hunk in the recently posted patch entitled "gen_hosts: remove 'write-your-own-template' placeholder text".
Chuck Lever (8):
gen_nodes: Re-title the playbook
devconfig: Remove "dnf update" optimizations for terraform hosts
gen_hosts: Get the "separate kernel builder" workflow working again
gen_hosts: Restore missing groups to the fstests hosts template
gen_hosts: Restore missing groups to the gitr hosts template
gen_hosts: Restore missing groups to the ltp hosts template
gen_hosts: Restore missing groups to the nfstest hosts template
gen_hosts: Replace pynfs hosts template
playbooks/gen_nodes.yml | 2 +- .../tasks/install-deps/redhat/main.yml | 4 + .../gen_hosts/templates/workflows/fstests.j2 | 4 +- .../gen_hosts/templates/workflows/gitr.j2 | 29 +++++- .../gen_hosts/templates/workflows/linux.j2 | 97 +------------------ .../gen_hosts/templates/workflows/ltp.j2 | 8 +- .../gen_hosts/templates/workflows/nfstest.j2 | 28 +++++- .../gen_hosts/templates/workflows/pynfs.j2 | 36 ++++++- 8 files changed, 104 insertions(+), 104 deletions(-)
Signature · 1 line
2.51.0
PATCH 1/8 gen_nodes: Re-title the playbook
From: Chuck Lever <[email protected]>
The gen_nodes playbook is used for both guestfs and terraform configurations. Seeing "libvirt XML" in the title while I was using a cloud provider was... startling.
--- playbooks/gen_nodes.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
playbooks/gen_nodes.yml +1 -1
@@ -1,5 +1,5 @@ --- -- name: Generate guests node configuration (libvirt domain XML) +- name: Generate target node configuration hosts: localhost roles: - role: gen_nodes
Signature · 1 line
2.51.0
PATCH 2/8 devconfig: Remove "dnf update" optimizations for terraform hosts
In reply to Chuck LeverFrom: Chuck Lever <[email protected]>
Using the fastest mirror seems to have some negative consequences when done on cloud systems.
--- playbooks/roles/devconfig/tasks/install-deps/redhat/main.yml | 4 ++++ 1 file changed, 4 insertions(+)
playbooks/roles/devconfig/tasks/install-deps/redhat/main.yml +4 -0
@@ -40,6 +40,8 @@ line: "fastestmirror=true" path: "/etc/dnf/dnf.conf" state: present + when: + - not kdevops_enable_terraform|bool - name: Increase the maximum number of concurrent package downloads become: true
@@ -49,6 +51,8 @@ line: "max_parallel_downloads=10" path: "/etc/dnf/dnf.conf" state: present + when: + - not kdevops_enable_terraform|bool - name: Refresh cache and upgrade all present packages become: true
Signature · 1 line
2.51.0
PATCH 3/8 gen_hosts: Get the "separate kernel builder" workflow working again
In reply to Chuck LeverFrom: Chuck Lever <[email protected]>
The [all] group contained the wrong host name. This causes "-m wait_for_connection -i all" to hang indefinitely.
The [dev] group was missing, resulting in needless warnings.
The only thing the linux workflow does is run a separate kernel builder node. The separate logic for nixos and the service hosts isn't needed, AFAICT.
Let's start over with a simple template.
--- .../gen_hosts/templates/workflows/linux.j2 | 97 +------------------ 1 file changed, 3 insertions(+), 94 deletions(-)
playbooks/roles/gen_hosts/templates/workflows/linux.j2 +3 -94
@@ -1,110 +1,19 @@ -{# Template for Linux custom kernel workflow #} +{# Template for the kernel builder workflow #} [all] localhost ansible_connection=local -{% if kdevops_enable_nixos|default(false) %} -{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3 -{% else %} -{{ kdevops_host_prefix }} -{% endif %} -{% if kdevops_baseline_and_dev == True %} -{% if kdevops_enable_nixos|default(false) %} -{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3 -{% else %} -{{ kdevops_host_prefix }}-dev -{% endif %} -{% endif %} -{% if kdevops_enable_iscsi %} -{{ kdevops_host_prefix }}-iscsi -{% endif %} -{% if kdevops_nfsd_enable %} -{{ kdevops_host_prefix }}-nfsd -{% endif %} -{% if kdevops_smbd_enable|default(false) %} -{{ kdevops_host_prefix }}-smbd -{% endif %} -{% if kdevops_krb5kdc_enable|default(false) %} -{{ kdevops_host_prefix }}-kdc -{% endif %} - +{{ kdevops_hosts_prefix }}-builder [all:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" [baseline] -{% if kdevops_enable_nixos|default(false) %} -{{ kdevops_host_prefix }} ansible_python_interpreter=/run/current-system/sw/bin/python3 -{% else %} -{{ kdevops_host_prefix }} -{% endif %} - +{{ kdevops_hosts_prefix }}-builder [baseline:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% if kdevops_baseline_and_dev %} [dev] -{% if kdevops_enable_nixos|default(false) %} -{{ kdevops_host_prefix }}-dev ansible_python_interpreter=/run/current-system/sw/bin/python3 -{% else %} -{{ kdevops_host_prefix }}-dev -{% endif %} - [dev:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% endif %} - -[linux] -{{ kdevops_host_prefix }} -{% if kdevops_baseline_and_dev == True %} -{{ kdevops_host_prefix }}-dev -{% endif %} - -[linux:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" - -{% if kdevops_enable_iscsi %} -[iscsi] -{{ kdevops_host_prefix }}-iscsi - -[iscsi:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% endif %} - -{% if kdevops_nfsd_enable %} -[nfsd] -{{ kdevops_host_prefix }}-nfsd - -[nfsd:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% endif %} - -{% if kdevops_smbd_enable|default(false) %} -[smbd] -{{ kdevops_host_prefix }}-smbd - -[smbd:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% endif %} - -{% if kdevops_krb5kdc_enable|default(false) %} -[kdc] -{{ kdevops_host_prefix }}-kdc - -[kdc:vars] -ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% endif %} [service] -{% if kdevops_enable_iscsi %} -{{ kdevops_host_prefix }}-iscsi -{% endif %} -{% if kdevops_nfsd_enable %} -{{ kdevops_host_prefix }}-nfsd -{% endif %} -{% if kdevops_smbd_enable|default(false) %} -{{ kdevops_host_prefix }}-smbd -{% endif %} -{% if kdevops_krb5kdc_enable|default(false) %} -{{ kdevops_host_prefix }}-kdc -{% endif %} - [service:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
Signature · 1 line
2.51.0
PATCH 4/8 gen_hosts: Restore missing groups to the fstests hosts template
In reply to Chuck LeverFrom: Chuck Lever <[email protected]>
Eliminate this warning:
[WARNING]: Could not match supplied host pattern, ignoring: dev
The [dev] group has to exist because some Makefiles specify it explicitly. It can be empty, though.
--- playbooks/roles/gen_hosts/templates/workflows/fstests.j2 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
playbooks/roles/gen_hosts/templates/workflows/fstests.j2 +2 -2
@@ -19,16 +19,16 @@ ansible_python_interpreter = "{{ kdevops_python_interpreter }}" [baseline:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% if kdevops_baseline_and_dev %} [dev] +{% if kdevops_baseline_and_dev %} {% for node_section in fstests_enabled_test_types %} {{ kdevops_host_prefix }}-{{ node_section }}-dev {% endfor %} +{% endif %} [dev:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% endif %} [fstests] {% for node_section in fstests_enabled_test_types %} {{ kdevops_host_prefix }}-{{ node_section }}
Signature · 1 line
2.51.0
PATCH 5/8 gen_hosts: Restore missing groups to the gitr hosts template
In reply to Chuck LeverFrom: Chuck Lever <[email protected]>
Eliminate this warning:
[WARNING]: Could not match supplied host pattern, ignoring: dev
The [dev] group has to exist because some Makefiles specify it explicitly. It can be empty, though.
--- .../gen_hosts/templates/workflows/gitr.j2 | 29 +++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-)
playbooks/roles/gen_hosts/templates/workflows/gitr.j2 +27 -2
@@ -19,16 +19,16 @@ ansible_python_interpreter = "{{ kdevops_python_interpreter }}" [baseline:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% if kdevops_baseline_and_dev %} [dev] +{% if kdevops_baseline_and_dev %} {% for host in gitr_enabled_hosts %} {{ kdevops_host_prefix }}-{{ host }}-dev {% endfor %} +{% endif %} [dev:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% endif %} [gitr] {% for host in gitr_enabled_hosts %} {{ kdevops_host_prefix }}-{{ host }}
@@ -39,3 +39,28 @@ ansible_python_interpreter = "{{ kdevops_python_interpreter }}" [gitr:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[iscsi] +{% if kdevops_enable_iscsi %} +{{ kdevops_hosts_prefix }}-iscsi +{% endif %} +[iscsi:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + + +[nfsd] +{% if kdevops_nfsd_enable %} +{{ kdevops_hosts_prefix }}-nfsd +{% endif %} +[nfsd:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[service] +{% if kdevops_enable_iscsi %} +{{ kdevops_hosts_prefix }}-iscsi +{% endif %} +{% if kdevops_nfsd_enable %} +{{ kdevops_hosts_prefix }}-nfsd +{% endif %} +[service:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
Signature · 1 line
2.51.0
PATCH 6/8 gen_hosts: Restore missing groups to the ltp hosts template
In reply to Chuck LeverFrom: Chuck Lever <[email protected]>
Eliminate these warnings:
[WARNING]: Could not match supplied host pattern, ignoring: dev [WARNING]: Could not match supplied host pattern, ignoring: service
These groups have to exist because some Makefiles specify them explicitly by name. They can remain empty, though.
--- playbooks/roles/gen_hosts/templates/workflows/ltp.j2 | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
playbooks/roles/gen_hosts/templates/workflows/ltp.j2 +6 -2
@@ -19,16 +19,16 @@ ansible_python_interpreter = "{{ kdevops_python_interpreter }}" [baseline:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% if kdevops_baseline_and_dev %} [dev] +{% if kdevops_baseline_and_dev %} {% for host in ltp_enabled_hosts %} {{ kdevops_host_prefix }}-{{ host }}-dev {% endfor %} +{% endif %} [dev:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% endif %} [ltp] {% for host in ltp_enabled_hosts %} {{ kdevops_host_prefix }}-{{ host }}
@@ -39,3 +39,7 @@ ansible_python_interpreter = "{{ kdevops_python_interpreter }}" [ltp:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[service] +[service:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
Signature · 1 line
2.51.0
PATCH 7/8 gen_hosts: Restore missing groups to the nfstest hosts template
In reply to Chuck LeverFrom: Chuck Lever <[email protected]>
Eliminate this warning:
[WARNING]: Could not match supplied host pattern, ignoring: dev
The [dev] group has to exist because some Makefiles specify it explicitly. It can be empty, though.
--- .../gen_hosts/templates/workflows/nfstest.j2 | 28 +++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-)
playbooks/roles/gen_hosts/templates/workflows/nfstest.j2 +26 -2
@@ -19,16 +19,16 @@ ansible_python_interpreter = "{{ kdevops_python_interpreter }}" [baseline:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% if kdevops_baseline_and_dev %} [dev] +{% if kdevops_baseline_and_dev %} {% for host in nfstest_enabled_hosts %} {{ kdevops_host_prefix }}-{{ host }}-dev {% endfor %} +{% endif %} [dev:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" -{% endif %} [nfstest] {% for host in nfstest_enabled_hosts %} {{ kdevops_host_prefix }}-{{ host }}
@@ -39,3 +39,27 @@ ansible_python_interpreter = "{{ kdevops_python_interpreter }}" [nfstest:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[iscsi] +{% if kdevops_enable_iscsi %} +{{ kdevops_hosts_prefix }}-iscsi +{% endif %} +[iscsi:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[nfsd] +{% if kdevops_nfsd_enable %} +{{ kdevops_hosts_prefix }}-nfsd +{% endif %} +[nfsd:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[service] +{% if kdevops_enable_iscsi %} +{{ kdevops_hosts_prefix }}-iscsi +{% endif %} +{% if kdevops_nfsd_enable %} +{{ kdevops_hosts_prefix }}-nfsd +{% endif %} +[service:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
Signature · 1 line
2.51.0
PATCH 8/8 gen_hosts: Replace pynfs hosts template
In reply to Chuck LeverFrom: Chuck Lever <[email protected]>
Commit 40df5ef72d19 ("gen_hosts: use kdevops_workflow_name directly for template selection") appears to have removed the pynfs hosts template completely, replacing it with the "write your own" template.
--- .../gen_hosts/templates/workflows/pynfs.j2 | 36 ++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-)
playbooks/roles/gen_hosts/templates/workflows/pynfs.j2 +35 -1
@@ -1,7 +1,41 @@ {# Workflow template for pynfs #} [all] localhost ansible_connection=local -write-your-own-template-for-pynfs-workflow +{% for host in pynfs_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ host }}-dev +{% endif %} +{% endfor %} [all:vars] ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[baseline] +{% for host in pynfs_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }} +{% endfor %} + +[baseline:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[dev] +{% if kdevops_baseline_and_dev %} +{% for host in pynfs_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }}-dev +{% endfor %} +{% endif %} + +[dev:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}" + +[pynfs] +{% for host in pynfs_enabled_hosts %} +{{ kdevops_host_prefix }}-{{ host }} +{% if kdevops_baseline_and_dev %} +{{ kdevops_host_prefix }}-{{ host }}-dev +{% endif %} +{% endfor %} + +[pynfs:vars] +ansible_python_interpreter = "{{ kdevops_python_interpreter }}"
Signature · 1 line
2.51.0
Quoted 10 lines · Chuck Lever
From: Chuck Lever <[email protected]>
I rebased on linux-kdevops/kdevops : main yesterday and found some new issues when trying to run NFSD-related workflows. Many of the workflows did not work because host groups were missing from the Ansible inventory.
The "Replace pynfs hosts template" patch could replace the workflows/pynfs.j2 hunk in the recently posted patch entitled "
gen_hosts: remove 'write-your-own-template' placeholder text".
Sorry for the collateral, we gotta get some automated tests for these workflow going! Agreed, with the chnages, in the essence of time, applied and pushed.
Luis