Replace terraform update_ssh_config module
cover Replace terraform update_ssh_config module
From: Chuck Lever <[email protected]>
*** Request For Comment ***
Luis suggested merging the way guestfs and terraform handle ssh host config. Luis, please let me know if I've grossly misunderstood your suggestion.
Vagrant is deprecated, so it will not be modified by these changes. The two remaining virtualization mechanisms have the same underlying purpose:
- Set up ssh to enable Ansible to have passwordless access to the target nodes
- Make human ssh to the target nodes as painless as possible
But they have grown apart in implementation over the years. This series is an attempt to build shared infrastructure that can be triggered by the common "bringup" and "destroy" make targets.
Eventually IMO the SSH Kconfig options for terraform need to be merged with the options from kconfigs/Kconfig.ssh. This series does not attempt to do that yet.
Daniel had some thoughts about how combined infrastructure might work which I haven't integrated here as the discussion doesn't seem finished yet.
I have tested this series with guestfs and AWS. I have provisioned an Azure account, and a colleague of mine has an active OCI account. These should help with finalizing and testing the needed non-AWS terraform changes, which are not yet part of this series.
Chuck Lever (4):
guestfs: Rename the update_ssh_config_guestfs role
update_ssh_config: Add always-run ssh clean-up steps
terraform: Add ssh hosts to ~/.ssh/config_kdevops_{{ sha1sum }}
terraform: Remove the terrraform update_ssh_config module
Makefile | 1 + playbooks/add_ssh_hosts_terraform.yml | 5 + .../add_ssh_hosts_terraform/defaults/main.yml | 2 + .../add_ssh_hosts_terraform/tasks/main.yml | 57 +++++++ .../templates/ssh_config.j2 | 15 ++ .../roles/update_ssh_config/tasks/main.yml | 150 ++++++++++++++++++ .../update_ssh_config_guestfs/tasks/main.yml | 71 --------- playbooks/update_ssh_config.yml | 5 + playbooks/update_ssh_config_guestfs.yml | 4 - scripts/bringup.Makefile | 7 + scripts/destroy_guestfs.sh | 6 - scripts/guestfs.Makefile | 6 - scripts/ssh.Makefile | 13 ++ scripts/terraform.Makefile | 6 +- terraform/aws/output.tf | 30 +--- terraform/aws/update_ssh_config.tf | 1 - terraform/azure/update_ssh_config.tf | 1 - terraform/gce/update_ssh_config.tf | 1 - terraform/oci/update_ssh_config.tf | 1 - terraform/openstack/update_ssh_config.tf | 1 - terraform/update_ssh_config.tf | 17 -- 21 files changed, 266 insertions(+), 134 deletions(-) create mode 100644 playbooks/add_ssh_hosts_terraform.yml create mode 100644 playbooks/roles/add_ssh_hosts_terraform/defaults/main.yml create mode 100644 playbooks/roles/add_ssh_hosts_terraform/tasks/main.yml create mode 100644 playbooks/roles/add_ssh_hosts_terraform/templates/ssh_config.j2 create mode 100644 playbooks/roles/update_ssh_config/tasks/main.yml delete mode 100644 playbooks/roles/update_ssh_config_guestfs/tasks/main.yml create mode 100644 playbooks/update_ssh_config.yml delete mode 100644 playbooks/update_ssh_config_guestfs.yml delete mode 120000 terraform/aws/update_ssh_config.tf delete mode 120000 terraform/azure/update_ssh_config.tf delete mode 120000 terraform/gce/update_ssh_config.tf delete mode 120000 terraform/oci/update_ssh_config.tf delete mode 120000 terraform/openstack/update_ssh_config.tf delete mode 100644 terraform/update_ssh_config.tf
Signature · 1 line
2.48.1
RFC PATCH 1/4 guestfs: Rename the update_ssh_config_guestfs role
From: Chuck Lever <[email protected]>
The update_ssh_config_guestfs role inserts an "Include" directive into the user's .ssh/config file. The included file is managed solely by kdevops.
The plan is to use this same mechanism for terraform as well. So give this role a generic name, perform a few clean-ups, and run it during "make deps". This situates the Include directive into the control user's .ssh/config for all virtualization methods.
--- .../roles/update_ssh_config/tasks/main.yml | 106 ++++++++++++++++++ .../update_ssh_config_guestfs/tasks/main.yml | 71 ------------ playbooks/update_ssh_config.yml | 5 + playbooks/update_ssh_config_guestfs.yml | 4 - scripts/guestfs.Makefile | 5 - scripts/ssh.Makefile | 13 +++ 6 files changed, 124 insertions(+), 80 deletions(-) create mode 100644 playbooks/roles/update_ssh_config/tasks/main.yml delete mode 100644 playbooks/roles/update_ssh_config_guestfs/tasks/main.yml create mode 100644 playbooks/update_ssh_config.yml delete mode 100644 playbooks/update_ssh_config_guestfs.yml
playbooks/roles/update_ssh_config/tasks/main.yml new file +106 -0
@@ -0,0 +1,106 @@ +--- +- name: Set the pathname of the controller's .ssh directory + ansible.builtin.set_fact: + sshdir: "{{ lookup('ansible.builtin.env', 'HOME') }}/.ssh" + tags: + - vars + +- name: Check that the user's ssh config file exists + delegate_to: localhost + run_once: true + ansible.builtin.stat: + path: "{{ sshdir }}/config" + register: ssh_config + tags: + - deps + +- name: Check that the kdevops Include directive is present + delegate_to: localhost + run_once: true + ansible.builtin.lineinfile: + path: "{{ sshdir }}/config" + regexp: "Include ~/.ssh/config_kdevops_*" + state: absent + check_mode: true + changed_when: false + register: kdevops_ssh_include + when: + - ssh_config.stat.exists + tags: + - deps + +- name: Check that the Include directive has a kdevops_version comment + delegate_to: localhost + run_once: true + ansible.builtin.lineinfile: + path: "{{ sshdir }}/config" + regexp: "^#(.*)kdevops_version(.*)" + state: absent + check_mode: true + changed_when: false + register: fixed_ssh_entry + when: + - ssh_config.stat.exists + tags: + - deps + +- name: Check if the correct Include directive is present + ansible.builtin.meta: end_play + when: + - ssh_config.stat.exists + - kdevops_ssh_include.found + - fixed_ssh_entry.found + tags: + - deps + +- name: Remove the stale Include directive + delegate_to: localhost + run_once: true + ansible.builtin.lineinfile: + path: "{{ sshdir }}/config" + line: "Include ~/.ssh/config_kdevops_*" + state: absent + when: + - ssh_config.stat.exists + tags: + - deps + +- name: Remove stale kdevops comments + delegate_to: localhost + run_once: true + ansible.builtin.lineinfile: + path: "{{ sshdir }}/config" + regexp: "^#(.*)kdevops(.*)" + state: absent + when: + - ssh_config.stat.exists + tags: + - deps + +- name: Remove extraneous new lines + delegate_to: localhost + run_once: true + ansible.builtin.replace: + path: "{{ sshdir }}/config" + regexp: '(^\s*$)' + replace: '' + when: + - ssh_config.stat.exists + tags: + - deps + +- name: Add a proper Include directive to ~/.ssh/config + delegate_to: localhost + run_once: true + ansible.builtin.blockinfile: + path: "{{ sshdir }}/config" + insertbefore: BOF + marker: "{mark}" + marker_begin: "# Automatically added by kdevops\n# kdevops_version: {{ kdevops_version }}" + marker_end: "" + create: true + mode: "u=rw,g=r,o=r" + block: | + Include ~/.ssh/config_kdevops_* + tags: + - deps
playbooks/roles/update_ssh_config_guestfs/tasks/main.yml deleted +0 -71
71 lines not on this page · raw message
playbooks/update_ssh_config.yml new file +5 -0
@@ -0,0 +1,5 @@ +--- +- hosts: all + gather_facts: false + roles: + - role: update_ssh_config
playbooks/update_ssh_config_guestfs.yml deleted +0 -4
@@ -1,4 +0,0 @@ ---- -- hosts: localhost - roles: - - role: update_ssh_config_guestfs
scripts/guestfs.Makefile +0 -5
@@ -62,11 +62,6 @@ libvirt_pcie_passthrough_permissions: $(KDEVOPS_PROVISIONED_SSH): $(Q)if [[ "$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE)" == "y" ]]; then \ - ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \ - --inventory localhost, \ - playbooks/update_ssh_config_guestfs.yml \ - --extra-vars=@./extra_vars.yaml \ - -e 'ansible_python_interpreter=/usr/bin/python3' ;\ LIBVIRT_DEFAULT_URI=$(CONFIG_LIBVIRT_URI) $(TOPDIR)/scripts/update_ssh_config_guestfs.py; \ fi $(Q)ansible $(ANSIBLE_VERBOSE) -i hosts all -e 'ansible_python_interpreter=/usr/bin/python3' -m wait_for_connection
scripts/ssh.Makefile +13 -0
@@ -21,3 +21,16 @@ $(KDEVOPS_SSH_PRIVKEY): .config $(NQ) Generating new private key: $(KDEVOPS_SSH_PRIVKEY) $(NQ) Generating new public key: $(KDEVOPS_SSH_PUBKEY) $(Q)$(TOPDIR)/scripts/gen_ssh_key.sh + +PHONY += update-ssh-config +update-ssh-config: + $(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \ + --inventory localhost, \ + playbooks/update_ssh_config.yml \ + --extra-vars=@./extra_vars.yaml \ + -e 'ansible_python_interpreter=/usr/bin/python3' \ + --tags vars,deps + +ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE)) +LOCALHOST_SETUP_WORK += update-ssh-config +endif
Signature · 1 line
2.48.1
RFC PATCH 2/4 update_ssh_config: Add always-run ssh clean-up steps
In reply to [email protected]From: Chuck Lever <[email protected]>
This common bit of logic is by the "make destroy" target, so it affects all virtualization methods.
--- .../roles/update_ssh_config/tasks/main.yml | 44 +++++++++++++++++++ scripts/bringup.Makefile | 7 +++ scripts/destroy_guestfs.sh | 6 --- scripts/guestfs.Makefile | 1 - scripts/terraform.Makefile | 1 - 5 files changed, 51 insertions(+), 8 deletions(-)
playbooks/roles/update_ssh_config/tasks/main.yml +44 -0
@@ -5,6 +5,22 @@ tags: - vars +- name: Set the pathname of the ephemeral ssh config file + ansible.builtin.set_fact: + kdevops_ssh_config: "{{ sshdir }}/config_kdevops_{{ topdir_path_sha256sum }}" + when: + - topdir_path_sha256sum is defined + tags: + - vars + +- name: Set the pathname of the ephemeral ssh config file + ansible.builtin.set_fact: + kdevops_ssh_config: "{{ sshdir }}/config_kdevops_{{ kdevops_host_prefix }}" + when: + - topdir_path_sha256sum is not defined + tags: + - vars + - name: Check that the user's ssh config file exists delegate_to: localhost run_once: true
@@ -104,3 +120,31 @@ Include ~/.ssh/config_kdevops_* tags: - deps + +- name: Remove saved keys for the target nodes + delegate_to: localhost + throttle: 1 + ansible.builtin.command: + argv: + - "ssh-keygen" + - "-q" + - "-f" + - "{{ sshdir }}/known_hosts" + - "-R" + - "{{ inventory_hostname }}" + register: result + failed_when: false + changed_when: + - result is success + - not "not found in" in result.stderr + tags: + - clean + +- name: Remove the ephemeral ssh config file on the control host + delegate_to: localhost + run_once: true + ansible.builtin.file: + path: "{{ kdevops_ssh_config }}" + state: absent + tags: + - clean
scripts/bringup.Makefile +7 -0
@@ -23,6 +23,13 @@ endif bringup: $(KDEVOPS_BRING_UP_DEPS) destroy: $(KDEVOPS_DESTROY_DEPS) + $(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \ + -i hosts \ + playbooks/update_ssh_config.yml \ + --extra-vars=@./extra_vars.yaml \ + -e 'ansible_python_interpreter=/usr/bin/python3' \ + --tags vars,clean + $(Q)rm -f $(KDEVOPS_PROVISIONED_SSH) $(KDEVOPS_PROVISIONED_DEVCONFIG) bringup-help-menu: @echo "Bringup targets:"
scripts/destroy_guestfs.sh +0 -6
@@ -23,14 +23,8 @@ if [ -f "$GUESTFSDIR/kdevops_nodes.yaml" ]; then fi rm -rf "$GUESTFSDIR/$name" rm -rf "$STORAGEDIR/$name" - ssh-keygen -q -f ~/.ssh/known_hosts -R $name 1> /dev/null 2>&1 done fi -if [[ "$CONFIG_TOPDIR_PATH_HAS_SHA256SUM" == "y" ]]; then - rm -f ~/.ssh/config_kdevops_$CONFIG_TOPDIR_PATH_SHA256SUM -else - rm -f ~/.ssh/config_kdevops_$CONFIG_KDEVOPS_HOSTS_PREFIX -fi rm -f $GUESTFSDIR/.provisioned_once rm -f $GUESTFSDIR/kdevops_nodes.yaml
scripts/guestfs.Makefile +0 -1
@@ -85,7 +85,6 @@ PHONY += bringup_guestfs destroy_guestfs: $(Q)$(TOPDIR)/scripts/destroy_guestfs.sh - $(Q)rm -f $(KDEVOPS_PROVISIONED_SSH) $(KDEVOPS_PROVISIONED_DEVCONFIG) PHONY += destroy_guestfs
scripts/terraform.Makefile +0 -1
@@ -166,7 +166,6 @@ bringup_terraform: destroy_terraform: $(Q)$(TOPDIR)/scripts/destroy_terraform.sh - $(Q)rm -f $(KDEVOPS_PROVISIONED_DEVCONFIG) $(KDEVOPS_TFVARS): $(KDEVOPS_TFVARS_TEMPLATE) .config $(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
Signature · 1 line
2.48.1
RFC PATCH 3/4 terraform: Add ssh hosts to ~/.ssh/config_kdevops_{{ sha1sum }}
In reply to [email protected]From: Chuck Lever <[email protected]>
The fixed update_ssh_config module is still not removing ssh Host configuration information with "make destroy".
Also, we want to have more control over how the control host's ssh config is managed. Updating a separate terraform module is getting awkward.
Let's replace the independent terraform module that handles ssh configuration with a playbook that operates the same as guestfs:
the host config is stuffed into a common file under ~/.ssh that is included in ~/.ssh/config, and is easily located and deleted by "make destroy".
XXX: I'm not 100% sold on this organization: it might be better to fold the new playbook into scripts/bringup_terraform.sh somehow.
--- playbooks/add_ssh_hosts_terraform.yml | 5 ++ .../add_ssh_hosts_terraform/defaults/main.yml | 2 + .../add_ssh_hosts_terraform/tasks/main.yml | 57 +++++++++++++++++++ .../templates/ssh_config.j2 | 15 +++++ scripts/terraform.Makefile | 5 ++ terraform/aws/output.tf | 7 +++ 6 files changed, 91 insertions(+) create mode 100644 playbooks/add_ssh_hosts_terraform.yml create mode 100644 playbooks/roles/add_ssh_hosts_terraform/defaults/main.yml create mode 100644 playbooks/roles/add_ssh_hosts_terraform/tasks/main.yml create mode 100644 playbooks/roles/add_ssh_hosts_terraform/templates/ssh_config.j2
playbooks/add_ssh_hosts_terraform.yml new file +5 -0
@@ -0,0 +1,5 @@ +--- +- hosts: all + gather_facts: false + roles: + - role: add_ssh_hosts_terraform
playbooks/roles/add_ssh_hosts_terraform/defaults/main.yml new file +2 -0
@@ -0,0 +1,2 @@ +--- +ssh_config_kexalgorithms: ""
playbooks/roles/add_ssh_hosts_terraform/tasks/main.yml new file +57 -0
57 lines not on this page · raw message
playbooks/roles/add_ssh_hosts_terraform/templates/ssh_config.j2 new file +15 -0
@@ -0,0 +1,15 @@ +Host {{ hostname }} {{ ipaddr }} + HostName {{ ipaddr }} + User {{ user }} + Port {{ port }} + IdentityFile {{ sshkey }} +{% if kexalgorithms %} + KexAlgorithms {{ kexalgorithms }} +{% endif %} +{% if strict %} + UserKnownHostsFile /dev/null + StrictHostKeyChecking no + PasswordAuthentication no + IdentitiesOnly yes + LogLevel FATAL +{% endif %}
scripts/terraform.Makefile +5 -0
@@ -163,6 +163,11 @@ ANSIBLE_EXTRA_ARGS += $(TERRAFORM_EXTRA_VARS) bringup_terraform: $(Q)$(TOPDIR)/scripts/bringup_terraform.sh + $(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \ + --inventory hosts \ + playbooks/add_ssh_hosts_terraform.yml \ + --extra-vars=@./extra_vars.yaml \ + -e 'ansible_python_interpreter=/usr/bin/python3' destroy_terraform: $(Q)$(TOPDIR)/scripts/destroy_terraform.sh
terraform/aws/output.tf +7 -0
@@ -25,3 +25,10 @@ output "login_using" { value = data.null_data_source.group_hostnames_and_ips.*.outputs } +# Each provider's output.tf needs to define a public_ip_map. This +# map is used to build the Ansible controller's ssh configuration. +# Each map entry contains the node's hostname and public IP address. +output "public_ip_map" { + description = "The public IP addresses assigned to each instance" + value = "${zipmap(var.kdevops_nodes[*], aws_eip.kdevops_eip[*].public_ip)}" +}
Signature · 1 line
2.48.1
RFC PATCH 4/4 terraform: Remove the terrraform update_ssh_config module
In reply to [email protected]From: Chuck Lever <[email protected]>
The add_ssh_hosts_terraform playbook has taken its place.
XXX: Need to add a public_ip_map output for all cloud providers before removing the update_ssh_config terraform module.
--- Makefile | 1 + terraform/aws/output.tf | 25 ------------------------ terraform/aws/update_ssh_config.tf | 1 - terraform/azure/update_ssh_config.tf | 1 - terraform/gce/update_ssh_config.tf | 1 - terraform/oci/update_ssh_config.tf | 1 - terraform/openstack/update_ssh_config.tf | 1 - terraform/update_ssh_config.tf | 17 ---------------- 8 files changed, 1 insertion(+), 47 deletions(-) delete mode 120000 terraform/aws/update_ssh_config.tf delete mode 120000 terraform/azure/update_ssh_config.tf delete mode 120000 terraform/gce/update_ssh_config.tf delete mode 120000 terraform/oci/update_ssh_config.tf delete mode 120000 terraform/openstack/update_ssh_config.tf delete mode 100644 terraform/update_ssh_config.tf
Makefile +1 -0
@@ -252,6 +252,7 @@ mrproper: $(Q)$(MAKE) -f scripts/build.Makefile $@ $(Q)rm -f $(KDEVOPS_DEPCHECK) $(Q)rm -f terraform/*/terraform.tfvars + $(Q)rm -rf terraform/*/.terraform $(Q)rm -f $(KDEVOPS_NODES) $(Q)rm -f $(KDEVOPS_HOSTFILE) $(KDEVOPS_MRPROPER) $(Q)rm -f .config .config.old extra_vars.yaml $(KCONFIG_YAMLCFG)
terraform/aws/output.tf +0 -25
@@ -1,30 +1,5 @@ # All generic output goes here -locals { - ssh_key_i = format( - " %s%s ", - var.ssh_config_pubkey_file != "" ? "-i " : "", - var.ssh_config_pubkey_file != "" ? replace(var.ssh_config_pubkey_file, ".pub", "") : "", - ) -} - -data "null_data_source" "group_hostnames_and_ips" { - count = local.kdevops_num_boxes - inputs = { - value = format( - "%30s : ssh %s@%s %s ", - element(var.kdevops_nodes, count.index), - var.ssh_config_user, - element(aws_eip.kdevops_eip.*.public_ip, count.index), - local.ssh_key_i, - ) - } -} - -output "login_using" { - value = data.null_data_source.group_hostnames_and_ips.*.outputs -} - # Each provider's output.tf needs to define a public_ip_map. This # map is used to build the Ansible controller's ssh configuration. # Each map entry contains the node's hostname and public IP address.
terraform/aws/update_ssh_config.tf deleted +0 -1
@@ -1 +0,0 @@ -../update_ssh_config.tf
terraform/azure/update_ssh_config.tf deleted +0 -1
@@ -1 +0,0 @@ -../update_ssh_config.tf
terraform/gce/update_ssh_config.tf deleted +0 -1
@@ -1 +0,0 @@ -../update_ssh_config.tf
terraform/oci/update_ssh_config.tf deleted +0 -1
@@ -1 +0,0 @@ -../update_ssh_config.tf
terraform/openstack/update_ssh_config.tf deleted +0 -1
@@ -1 +0,0 @@ -../update_ssh_config.tf
terraform/update_ssh_config.tf deleted +0 -17
@@ -1,17 +0,0 @@ -module "ssh_config_update_host_entries" { - source = "linux-kdevops/add-host-ssh-config/kdevops" - version = "3.0.0" - - ssh_config = var.ssh_config - update_ssh_config_enable = var.ssh_config_update - cmd = "update" - shorthosts = join(",", slice(local.shorthosts, 0, local.kdevops_num_boxes)) - hostnames = join(",", slice(local.ipv4s, 0, local.kdevops_num_boxes)) - ports = "22" - user = var.ssh_config_user == "" ? "" : var.ssh_config_user - id = replace(var.ssh_config_pubkey_file, ".pub", "") - strict = var.ssh_config_use_strict_settings ? "true" : "" - use_backup = !var.ssh_config_backup || var.ssh_config == "/dev/null" ? "" : "true" - backup_postfix = "kdevops" - kexalgorithms = var.ssh_config_kexalgorithms == "" ? "" : var.ssh_config_kexalgorithms -}
Signature · 1 line
2.48.1