Set TOPDIR at run time
cover Set TOPDIR at run time
From: Chuck Lever <[email protected]>
A few recent changes have been difficult to integrate into my current CI infrastructure. Here are a couple of ideas on how to stretch in a couple of areas to help my particular arrangement.
Note that I would need to handle the inventory file location in a similar way: add a boolean Kconfig option to use a sensible default location rather than baking a fixed pathname into the kdevops .config. So, work in progress here.
Thoughts and alternatives are welcome. Even the Internet standard response ("you're doing it wrong") is OK.
Chuck Lever (2):
Set TOPDIR_PATH and generate its sha256sum at run time ansible.cfg: generate an ansible.cfg file in TOPDIR
Kconfig | 15 --------------- Makefile | 12 ++++++++++++ kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++ scripts/gen-hosts.Makefile | 1 + scripts/ssh.Makefile | 4 ++-- 5 files changed, 27 insertions(+), 17 deletions(-)
Signature · 1 line
2.51.0
RFC PATCH 1/2 Set TOPDIR_PATH and generate its sha256sum at run time
From: Chuck Lever <[email protected]>
Commit 42d27579a9d0 ("guestfs: enhance ssh config choice") added a uniquifier to the name of the ssh config to avoid multiple kdevops runs clobbering each other's ssh config entries. It uses the TOPDIR pathname set in the current .config to benefit from the nice convenience of generating an Ansible extra_var using "output yaml".
However, if the same user generates all her .configs (that will be run by other users under automation) while in the same directory (say, if they are different branches in the same git repository), the TOPDIR pathname and thus the sha256sum of that pathname will be exactly the same in all of those .config files, allowing the path names of the ephemeral ssh configs to collide.
The TOPDIR and its checksum need to be generated not by "make menuconfig" but rather by "make" in order that .config files (as saved in defconfigs/) can be truly portable.
--- Kconfig | 15 --------------- Makefile | 4 ++++ scripts/gen-hosts.Makefile | 1 + scripts/ssh.Makefile | 4 ++-- 4 files changed, 7 insertions(+), 17 deletions(-)
Kconfig +0 -15
@@ -5,21 +5,6 @@ mainmenu "$(PROJECT) $(PROJECTRELEASE)" -config TOPDIR_PATH - string - output yaml - default $(shell, echo $TOPDIR_PATH) - -config TOPDIR_PATH_HAS_SHA256SUM - bool - output yaml - default y - -config TOPDIR_PATH_SHA256SUM - string - output yaml - default $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH)) - config HAVE_KDEVOPS_CUSTOM_DEFAULTS bool default n
Makefile +4 -0
@@ -19,6 +19,8 @@ export KDEVOPS_NODES := export PYTHONUNBUFFERED=1 export TOPDIR=./ export TOPDIR_PATH = $(shell readlink -f $(TOPDIR)) +export TOPDIR_PATH_HAS_SHA256SUM=y +export TOPDIR_PATH_SHA256SUM = $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH)) include scripts/refs.Makefile KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/templates
@@ -82,6 +84,8 @@ INCLUDES = -I include/ CFLAGS += $(INCLUDES) ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)' +ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)' +ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)' export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
scripts/gen-hosts.Makefile +1 -0
@@ -3,6 +3,7 @@ KDEVOPS_PLAYBOOKS_DIR_FULL_PATH=$(TOPDIR_PATH)/$(KDEVOPS_PLAYBOOKS_DIR) KDEVOPS_HOSTS_TEMPLATE_DIR=$(KDEVOPS_PLAYBOOKS_DIR_FULL_PATH)/roles/gen_hosts/templates +GENHOSTS_EXTRA_ARGS += topdir_path='$(TOPDIR_PATH)' GENHOSTS_EXTRA_ARGS += kdevops_playbooks_dir='$(KDEVOPS_PLAYBOOKS_DIR)' # Relative path so that ansible can work with it
scripts/ssh.Makefile +2 -2
@@ -10,8 +10,8 @@ ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE)) SSH_CONFIG_FILE:=$(subst ",,$(CONFIG_KDEVOPS_SSH_CONFIG)) ANSIBLE_EXTRA_ARGS += sshconfig=$(shell realpath $(SSH_CONFIG_FILE)) -ifeq (y,$(CONFIG_TOPDIR_PATH_HAS_SHA256SUM)) -SSH_CONFIG_SUFFIX:=$(CONFIG_TOPDIR_PATH_SHA256SUM) +ifeq (y,$(TOPDIR_PATH_HAS_SHA256SUM)) +SSH_CONFIG_SUFFIX:=$(TOPDIR_PATH_SHA256SUM) else SSH_CONFIG_SUFFIX:=$(CONFIG_KDEVOPS_HOST_PREFIX) endif
Signature · 1 line
2.51.0
Re: [RFC PATCH 1/2] Set TOPDIR_PATH and generate its sha256sum at run time
Quoted 8 lines · Chuck Lever
From: Chuck Lever <[email protected]>
Commit
42d27579a9d0("guestfs: enhance ssh config choice") added a uniquifier to the name of the ssh config to avoid multiple kdevops runs clobbering each other's ssh config entries. It uses the TOPDIR pathname set in the current .config to benefit from the nice convenience of generating an Ansibleextra_varusing "output yaml".
Quoted 6 lines
However, if the same user generates all her .configs (that will be run by other users under automation) while in the same directory (say, if they are different branches in the same git repository), the TOPDIR pathname and thus the sha256sum of that pathname will be exactly the same in all of those .config files, allowing the path names of the ephemeral ssh configs to collide.
Both paragraphs above took me quite a bit of time to grasp, I think it was the parenthesis, as they make me try to switch contexts. So I think the explanation of the issue can be simplified. I *think* that the issue your are describing is essentially that we currently cannot support sharing .config to other users because TOPDIR_PATH expects the same path to be used. Would that be a summary of the issue?
Quoted 3 lines
The TOPDIR and its checksum need to be generated not by "make menuconfig" but rather by "make" in order that .config files (as saved in defconfigs/) can be truly portable.
This is nice.
Quoted 39 lines
Signed-off-by Chuck Lever <[email protected]>--- Kconfig | 15 --------------- Makefile | 4 ++++ scripts/gen-hosts.Makefile | 1 + scripts/ssh.Makefile | 4 ++-- 4 files changed, 7 insertions(+), 17 deletions(-)Kconfig +0
-15@@ -5,21 +5,6 @@ mainmenu "$(PROJECT) $(PROJECTRELEASE)" -config TOPDIR_PATH - string - output yaml - default $(shell, echo $TOPDIR_PATH) - -config TOPDIR_PATH_HAS_SHA256SUM - bool - output yaml - default y - -config TOPDIR_PATH_SHA256SUM - string - output yaml - default $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH)) - config HAVE_KDEVOPS_CUSTOM_DEFAULTS bool default nMakefile +0
-0
Makefile export KDEVOPS_NODES := · line 22
Quoted 4 lines
Makefile +1
-0@@ -19,6 +19,8 @@ export KDEVOPS_NODES := export PYTHONUNBUFFERED=1 export TOPDIR=./ export TOPDIR_PATH = $(shell readlink -f $(TOPDIR)) +export TOPDIR_PATH_HAS_SHA256SUM=y
We added this a long time ago, to remain compatible with users who may git pull to the latest kdevops which didn't yet have the symbol TOPDIR_PATH_HAS_SHA256SUM.
So I think a prep patch to just remove TOPDIR_PATH_HAS_SHA256SUM now make sense.
Otherwise this is looking nice.
Luis
Re: [RFC PATCH 1/2] Set TOPDIR_PATH and generate its sha256sum at run time
Quoted 16 lines · Luis Chamberlain
Quoted 8 lines · Chuck Lever
From: Chuck Lever <[email protected]>
Commit
42d27579a9d0("guestfs: enhance ssh config choice") added a uniquifier to the name of the ssh config to avoid multiple kdevops runs clobbering each other's ssh config entries. It uses the TOPDIR pathname set in the current .config to benefit from the nice convenience of generating an Ansibleextra_varusing "output yaml".Quoted 6 lines
However, if the same user generates all her .configs (that will be run by other users under automation) while in the same directory (say, if they are different branches in the same git repository), the TOPDIR pathname and thus the sha256sum of that pathname will be exactly the same in all of those .config files, allowing the path names of the ephemeral ssh configs to collide.
Quoted 7 lines
Both paragraphs above took me quite a bit of time to grasp, I think it was the parenthesis, as they make me try to switch contexts. So I think the explanation of the issue can be simplified. I *think* that the issue your are describing is essentially that we currently cannot support sharing .config to other users because
TOPDIR_PATHexpects the same path to be used. Would that be a summary of the issue?
I struggled with writing it. I wanted to capture my use case as the rationale for the change. I'll try to integrate your suggestion.
Quoted 54 lines
Quoted 3 lines
The TOPDIR and its checksum need to be generated not by "make menuconfig" but rather by "make" in order that .config files (as saved in defconfigs/) can be truly portable.
This is nice.
Quoted 39 lines
Signed-off-by Chuck Lever <[email protected]>--- Kconfig | 15 --------------- Makefile | 4 ++++ scripts/gen-hosts.Makefile | 1 + scripts/ssh.Makefile | 4 ++-- 4 files changed, 7 insertions(+), 17 deletions(-)Kconfig +0
-15@@ -5,21 +5,6 @@ mainmenu "$(PROJECT) $(PROJECTRELEASE)" -config TOPDIR_PATH - string - output yaml - default $(shell, echo $TOPDIR_PATH) - -config TOPDIR_PATH_HAS_SHA256SUM - bool - output yaml - default y - -config TOPDIR_PATH_SHA256SUM - string - output yaml - default $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH)) - config HAVE_KDEVOPS_CUSTOM_DEFAULTS bool default nMakefile +0
-0Makefile export KDEVOPS_NODES := · line 22
Quoted 4 lines
Makefile +1
-0@@ -19,6 +19,8 @@ export KDEVOPS_NODES := export PYTHONUNBUFFERED=1 export TOPDIR=./ export TOPDIR_PATH = $(shell readlink -f $(TOPDIR)) +export TOPDIR_PATH_HAS_SHA256SUM=yWe added this a long time ago, to remain compatible with users who may git pull to the latest kdevops which didn't yet have the symbol
TOPDIR_PATH_HAS_SHA256SUM.
Quoted 2 lines
So I think a prep patch to just remove
TOPDIR_PATH_HAS_SHA256SUMnow make sense.
Ha! I had a sentence in the patch description suggesting removal of HAS_SHA256SUM, but decided to keep things simple and took it out.
Quoted 3 lines
Otherwise this is looking nice.
Luis
Signature · 1 line
Chuck Lever
Re: [RFC PATCH 1/2] Set TOPDIR_PATH and generate its sha256sum at run time
In reply to Chuck LeverQuoted 8 lines · Chuck Lever
From: Chuck Lever <[email protected]>
Commit
42d27579a9d0("guestfs: enhance ssh config choice") added a uniquifier to the name of the ssh config to avoid multiple kdevops runs clobbering each other's ssh config entries. It uses the TOPDIR pathname set in the current .config to benefit from the nice convenience of generating an Ansibleextra_varusing "output yaml".
Quoted 7 lines
However, if the same user generates all her .configs (that will be run by other users under automation) while in the same directory (say, if they are different branches in the same git repository), the TOPDIR pathname and thus the sha256sum of that pathname will be exactly the same in all of those .config files, allowing the path names of the ephemeral ssh configs to collide.
IIUC, this aligns very well with the sandboxing effort. Definitely, TOPDIR needs to point to a sandbox directory.
Thanks for this series!
Quoted 65 lines
The TOPDIR and its checksum need to be generated not by "make menuconfig" but rather by "make" in order that .config files (as saved in defconfigs/) can be truly portable.
Signed-off-by Chuck Lever <[email protected]>--- Kconfig | 15 --------------- Makefile | 4 ++++ scripts/gen-hosts.Makefile | 1 + scripts/ssh.Makefile | 4 ++-- 4 files changed, 7 insertions(+), 17 deletions(-)Kconfig +0
-15@@ -5,21 +5,6 @@ mainmenu "$(PROJECT) $(PROJECTRELEASE)" -config TOPDIR_PATH - string - output yaml - default $(shell, echo $TOPDIR_PATH) - -config TOPDIR_PATH_HAS_SHA256SUM - bool - output yaml - default y - -config TOPDIR_PATH_SHA256SUM - string - output yaml - default $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH)) - config HAVE_KDEVOPS_CUSTOM_DEFAULTS bool default nMakefile +4
-0@@ -19,6 +19,8 @@ export KDEVOPS_NODES := export PYTHONUNBUFFERED=1 export TOPDIR=./ export TOPDIR_PATH = $(shell readlink -f $(TOPDIR)) +export TOPDIR_PATH_HAS_SHA256SUM=y +export TOPDIR_PATH_SHA256SUM = $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH)) include scripts/refs.Makefile KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/templates@@ -82,6 +84,8 @@ INCLUDES = -I include/ CFLAGS += $(INCLUDES) ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)' +ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)' +ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)' export KDEVOPS_HOSTS_TEMPLATE := hosts.j2scripts/gen-hosts.Makefile +0
-0
scripts/gen-hosts.Makefile line 6
Quoted 4 lines
scripts/gen-hosts.Makefile +1
-0@@ -3,6 +3,7 @@ KDEVOPS_PLAYBOOKS_DIR_FULL_PATH=$(TOPDIR_PATH)/$(KDEVOPS_PLAYBOOKS_DIR) KDEVOPS_HOSTS_TEMPLATE_DIR=$(KDEVOPS_PLAYBOOKS_DIR_FULL_PATH)/roles/gen_hosts/templates +GENHOSTS_EXTRA_ARGS += topdir_path='$(TOPDIR_PATH)'
We need to feed these vars as they are variables generated based on the directory the user is currently running kdevops on, is that correct?
Quoted 18 lines
scripts/gen-hosts.Makefile +0
-0GENHOSTS_EXTRA_ARGS += kdevops_playbooks_dir='$(KDEVOPS_PLAYBOOKS_DIR)' # Relative path so that ansible can work with itscripts/ssh.Makefile +2
-2@@ -10,8 +10,8 @@ ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE)) SSH_CONFIG_FILE:=$(subst ",,$(CONFIG_KDEVOPS_SSH_CONFIG)) ANSIBLE_EXTRA_ARGS += sshconfig=$(shell realpath $(SSH_CONFIG_FILE)) -ifeq (y,$(CONFIG_TOPDIR_PATH_HAS_SHA256SUM)) -SSH_CONFIG_SUFFIX:=$(CONFIG_TOPDIR_PATH_SHA256SUM) +ifeq (y,$(TOPDIR_PATH_HAS_SHA256SUM)) +SSH_CONFIG_SUFFIX:=$(TOPDIR_PATH_SHA256SUM) else SSH_CONFIG_SUFFIX:=$(CONFIG_KDEVOPS_HOST_PREFIX) endif
RFC PATCH 2/2 ansible.cfg: generate an ansible.cfg file in TOPDIR
In reply to Chuck LeverFrom: Chuck Lever <[email protected]>
I need an ansible.cfg that is generated by the Kconfig menu, but whose pathname is selected dynamically based on where kdevops is being run rather than having that path baked into the .config.
--- Makefile | 8 ++++++++ kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++ 2 files changed, 20 insertions(+)
Makefile +8 -0
@@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem export KDEVOPS_NODES_TEMPLATE := export KDEVOPS_MRPROPER := +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT)) +ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg +export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE) +else ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),) ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"') export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE) endif +endif ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"') KDEVOPS_INSTALL_TARGETS :=
@@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES) ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)' ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)' ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)' +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT)) +ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)' +endif export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
kconfigs/Kconfig.ansible_cfg +12 -0
@@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI bool default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS) +config ANSIBLE_CFG_FILE_DEFAULT + bool "Use a sensible default location for the ansible.cfg file" + default N + help + When this option is set to Y, kdevops creates an ansible.cfg + in a sensible default location, using the settings defined + in this Kconfig menu. + +if !ANSIBLE_CFG_FILE_DEFAULT + config ANSIBLE_CFG_FILE string "Ansible configuration file" output yaml
@@ -41,6 +51,8 @@ config ANSIBLE_CFG_FILE For more details, refer to the Ansible documentation: https://docs.ansible.com/ansible/latest/reference_appendices/config.html#the-configuration-file +endif # ANSIBLE_CFG_FILE_DEFAULT + menu "Ansible Callback Plugin Configuration" choice prompt "Ansible Callback Plugin"
Signature · 1 line
2.51.0
Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR
Quoted 47 lines · Chuck Lever
From: Chuck Lever <[email protected]>
I need an ansible.cfg that is generated by the
Kconfigmenu, but whose pathname is selected dynamically based on where kdevops is being run rather than having that path baked into the .config.Signed-off-by Chuck Lever <[email protected]>--- Makefile | 8 ++++++++ kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++ 2 files changed, 20 insertions(+)Makefile +8
-0@@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem export KDEVOPS_NODES_TEMPLATE := export KDEVOPS_MRPROPER := +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT)) +ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg +export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE) +else ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),) ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"') export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE) endif +endif ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"') KDEVOPS_INSTALL_TARGETS :=@@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES) ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)' ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)' ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)' +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT)) +ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)' +endif export KDEVOPS_HOSTS_TEMPLATE := hosts.j2kconfigs/Kconfig.ansible_cfg +0
-0
kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 24
Quoted 7 lines
kconfigs/Kconfig.ansible_cfg +3
-0@@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI bool default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS) +config ANSIBLE_CFG_FILE_DEFAULT + bool "Use a sensible default location for the ansible.cfg file" + default N
defaults are always n, so this line can be removed.
kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 27
Quoted 4 lines
kconfigs/Kconfig.ansible_cfg +4
-0+ help + When this option is set to Y, kdevops creates an ansible.cfg + in a sensible default location, using the settings defined + in this Kconfig menu.
Why would we not want this by default?
Luis
Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR
Quoted 58 lines · Luis Chamberlain
Quoted 47 lines · Chuck Lever
From: Chuck Lever <[email protected]>
I need an ansible.cfg that is generated by the
Kconfigmenu, but whose pathname is selected dynamically based on where kdevops is being run rather than having that path baked into the .config.Signed-off-by Chuck Lever <[email protected]>--- Makefile | 8 ++++++++ kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++ 2 files changed, 20 insertions(+)Makefile +8
-0@@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem export KDEVOPS_NODES_TEMPLATE := export KDEVOPS_MRPROPER := +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT)) +ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg +export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE) +else ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),) ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"') export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE) endif +endif ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"') KDEVOPS_INSTALL_TARGETS :=@@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES) ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)' ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)' ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)' +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT)) +ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)' +endif export KDEVOPS_HOSTS_TEMPLATE := hosts.j2kconfigs/Kconfig.ansible_cfg +0
-0kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 24
Quoted 7 lines
kconfigs/Kconfig.ansible_cfg +3
-0@@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI bool default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS) +config ANSIBLE_CFG_FILE_DEFAULT + bool "Use a sensible default location for the ansible.cfg file" + default Ndefaults are always n, so this line can be removed.
Quoted 6 lines
kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 27
Quoted 4 lines
kconfigs/Kconfig.ansible_cfg +4
-0+ help + When this option is set to Y, kdevops creates an ansible.cfg + in a sensible default location, using the settings defined + in this Kconfig menu.Why would we not want this by default?
Because N matches the current behavior.
But I would prefer Y, actually.
Signature · 1 line
Chuck Lever
Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR
Quoted 17 lines · Chuck Lever
Quoted 9 lines · Luis Chamberlain
kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 23
Quoted 5 lines · Chuck Lever
From: Chuck Lever <[email protected]>
kconfigs/Kconfig.ansible_cfg +3
-0+config ANSIBLE_CFG_FILE_DEFAULT + bool "Use a sensible default location for the ansible.cfg file" + default Ndefaults are always n, so this line can be removed.
Quoted 6 lines
kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 27
Quoted 4 lines
kconfigs/Kconfig.ansible_cfg +4
-0+ help + When this option is set to Y, kdevops creates an ansible.cfg + in a sensible default location, using the settings defined + in this Kconfig menu.Why would we not want this by default?
Quoted 3 lines
Because N matches the current behavior.
But I would prefer Y, actually.
I think its safe to use Y.
LUis
Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR
In reply to Chuck LeverQuoted 5 lines · Chuck Lever
From: Chuck Lever <[email protected]>
I need an ansible.cfg that is generated by the
Kconfigmenu, but whose pathname is selected dynamically based on where kdevops is being run rather than having that path baked into the .config.
I'm trying to review the previous conversation before these patches as well as this series. Please, let me know if I'm understanding something wrong:
I think what we need is not to have individual configuration paths (as we have now) for ansible.cfg, inventory, etc. What we want is to have one sandbox folder, our TOPDIR, and have our kdevops sandbox directory there. All files generated and consumed by kdevops should then use that path.
Would this work? See below:
kdevops-tree $ make defconfig-selftest-xarray O=path/to/sandbox0 # Now path/to/sandbox0 contains the generated .config, extra_vars.yml, etc kdevops-tree $ make menuconfig O=path/to/sandbox0
kdevops-tree $ make O=path/to/sandbox0
kdevops-tree $ make bringup O=path/to/sandbox0
...
kdevops-tree $ make defconfig-tmpfs O=path/to/sandbox1
...
path/to/sandbox0 $ make linux
path/to/sandbox0 $ make selftests
path/to/sandbox0 $ make selftests-xarray
...
path/to/sandbox1 $ make
path/to/sandbox1 $ make bringup
path/to/sandbox1 $ make linux
path/to/sandbox1 $ make fstests
path/to/sandbox1 $ make fstests-baseline
To clarify: instead of letting users override individual file paths, we allow customization of the topdir directory (with O=) where all files are created. If O= is not passed, then PWD is assumed. I think it would still be useful to support customizing individual files, but only relative to TOPDIR. That would keep the structure cleaner, for example, using subdirectories for Ansible files, kconfigs, SSH, virsh, and so on.
Quoted 11 lines
Signed-off-by Chuck Lever <[email protected]>--- Makefile | 8 ++++++++ kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++ 2 files changed, 20 insertions(+)Makefile +0
-0
Makefile KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem · line 33
Quoted 7 lines
Makefile +4
-0@@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem export KDEVOPS_NODES_TEMPLATE := export KDEVOPS_MRPROPER := +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT)) +ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg +export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE) +else
What I'm suggesting above is to always use this new option. And perhaps keep ANSIBLE_CFG_FILE configurable but inside TOPDIR. For example:
ANSIBLE_CFG_FILE=ansible/ansible.cfg.
Quoted 26 lines
Makefile +4
-0ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),) ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"') export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE) endif +endif ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"') KDEVOPS_INSTALL_TARGETS :=@@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES) ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)' ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)' ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)' +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT)) +ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)' +endif export KDEVOPS_HOSTS_TEMPLATE := hosts.j2kconfigs/Kconfig.ansible_cfg +0
-0@@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI bool default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS)
kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 30
Quoted 10 lines
kconfigs/Kconfig.ansible_cfg +10
-0+config ANSIBLE_CFG_FILE_DEFAULT + bool "Use a sensible default location for the ansible.cfg file" + default N + help + When this option is set to Y, kdevops creates an ansible.cfg + in a sensible default location, using the settings defined + in this Kconfig menu. + +if !ANSIBLE_CFG_FILE_DEFAULT +
I think we don't need this if you agree with the approach above.
Quoted 12 lines
kconfigs/Kconfig.ansible_cfg +2
-0config ANSIBLE_CFG_FILE string "Ansible configuration file" output yaml@@ -41,6 +51,8 @@ config ANSIBLE_CFG_FILE For more details, refer to the Ansible documentation: https://docs.ansible.com/ansible/latest/reference_appendices/config.html#the-configuration-file +endif # ANSIBLE_CFG_FILE_DEFAULT + menu "Ansible Callback Plugin Configuration" choice prompt "Ansible Callback Plugin"
Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR
Quoted 10 lines · Daniel Gomez
Quoted 5 lines · Chuck Lever
From: Chuck Lever <[email protected]>
I need an ansible.cfg that is generated by the
Kconfigmenu, but whose pathname is selected dynamically based on where kdevops is being run rather than having that path baked into the .config.I'm trying to review the previous conversation before these patches as well as this series. Please, let me know if I'm understanding something wrong:
Quoted 6 lines
I think what we need is not to have individual configuration paths (as we have now) for ansible.cfg, inventory, etc. What we want is to have one sandbox folder, our TOPDIR, and have our kdevops sandbox directory there. All files generated and consumed by kdevops should then use that path.
Would this work?
Well I think that would work for my particular usage, however: If I'm reading between the lines of the Ansible pathname help texts, it appears that these settings can specify search paths, not a single file.
In particular, the ANSIBLE_CFG_INVENTORY_FILE string is a comma- separated list. And, if ANSIBLE_CFG_FILE is an empty string, then there is a default set of locations /outside of/ the CWD that Ansible searches for a configuration.
I'm not against your suggestion of specifying a sandbox pathname at all; that would be a good UX simplification. There are these details, though, and I don't have any suggestions about that at the moment.
Quoted 101 lines
See below:
kdevops-tree $ make defconfig-selftest-xarray O=path/to/sandbox0 # Now path/to/sandbox0 contains the generated .config, extra_vars.yml, etc kdevops-tree $ make menuconfig O=path/to/sandbox0
kdevops-tree $ make O=path/to/sandbox0
kdevops-tree $ make bringup O=path/to/sandbox0
...
kdevops-tree $ make defconfig-tmpfs O=path/to/sandbox1
...
path/to/sandbox0 $ make linux
path/to/sandbox0 $ make selftests
path/to/sandbox0 $ make selftests-xarray
...
path/to/sandbox1 $ make
path/to/sandbox1 $ make bringup
path/to/sandbox1 $ make linux
path/to/sandbox1 $ make fstests
path/to/sandbox1 $ make fstests-baselineTo clarify: instead of letting users override individual file paths, we allow customization of the topdir directory (with O=) where all files are created. If O= is not passed, then PWD is assumed. I think it would still be useful to support customizing individual files, but only relative to TOPDIR. That would keep the structure cleaner, for example, using subdirectories for Ansible files, kconfigs, SSH, virsh, and so on.
Quoted 11 lines
Signed-off-by Chuck Lever <[email protected]>--- Makefile | 8 ++++++++ kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++ 2 files changed, 20 insertions(+)Makefile +0
-0Makefile KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem · line 33
Quoted 7 lines
Makefile +4
-0@@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem export KDEVOPS_NODES_TEMPLATE := export KDEVOPS_MRPROPER := +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT)) +ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg +export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE) +elseWhat I'm suggesting above is to always use this new option. And perhaps keep
ANSIBLE_CFG_FILEconfigurable but inside TOPDIR. For example:
ANSIBLE_CFG_FILE=ansible/ansible.cfg.Quoted 26 lines
Makefile +4
-0ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),) ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"') export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE) endif +endif ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"') KDEVOPS_INSTALL_TARGETS :=@@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES) ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)' ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)' ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)' +ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT)) +ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)' +endif export KDEVOPS_HOSTS_TEMPLATE := hosts.j2kconfigs/Kconfig.ansible_cfg +0
-0@@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI bool default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS)kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 30
Quoted 10 lines
kconfigs/Kconfig.ansible_cfg +10
-0+config ANSIBLE_CFG_FILE_DEFAULT + bool "Use a sensible default location for the ansible.cfg file" + default N + help + When this option is set to Y, kdevops creates an ansible.cfg + in a sensible default location, using the settings defined + in this Kconfig menu. + +if !ANSIBLE_CFG_FILE_DEFAULT +I think we don't need this if you agree with the approach above.
Quoted 12 lines
kconfigs/Kconfig.ansible_cfg +2
-0config ANSIBLE_CFG_FILE string "Ansible configuration file" output yaml@@ -41,6 +51,8 @@ config ANSIBLE_CFG_FILE For more details, refer to the Ansible documentation: https://docs.ansible.com/ansible/latest/reference_appendices/config.html#the-configuration-file +endif # ANSIBLE_CFG_FILE_DEFAULT + menu "Ansible Callback Plugin Configuration" choice prompt "Ansible Callback Plugin"
Signature · 1 line
Chuck Lever
Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR
Quoted 18 lines · Chuck Lever
Quoted 10 lines · Daniel Gomez
Quoted 5 lines · Chuck Lever
From: Chuck Lever <[email protected]>
I need an ansible.cfg that is generated by the
Kconfigmenu, but whose pathname is selected dynamically based on where kdevops is being run rather than having that path baked into the .config.I'm trying to review the previous conversation before these patches as well as this series. Please, let me know if I'm understanding something wrong:
Quoted 6 lines
I think what we need is not to have individual configuration paths (as we have now) for ansible.cfg, inventory, etc. What we want is to have one sandbox folder, our TOPDIR, and have our kdevops sandbox directory there. All files generated and consumed by kdevops should then use that path.
Would this work?
Quoted 8 lines
Well I think that would work for my particular usage, however: If I'm reading between the lines of the Ansible pathname help texts, it appears that these settings can specify search paths, not a single file.
In particular, the
ANSIBLE_CFG_INVENTORY_FILEstring is a comma- separated list. And, ifANSIBLE_CFG_FILEis an empty string, then there is a default set of locations /outside of/ the CWD that Ansible searches for a configuration.
That case will never happen, as we will ensure our sandbox var is always defined (either with O=, or with PWD) and therefore the location of the kdevops artifacts (.config, ansible.cfg, inventory, etc). That should preserve the current functionality while also allowing others to generate kdevops artifacts into separate directories.
Quoted 4 lines
I'm not against your suggestion of specifying a sandbox pathname at all; that would be a good UX simplification. There are these details, though, and I don't have any suggestions about that at the moment.
Great! And agreed. Even if my assessment above is wrong, I don't see any major blockers.