Skip to contentSkip to conversation

Set TOPDIR at run time

RFC PATCH In review Chuck Lever 3 participants 12 messages Aug 28, 2025 – Sep 1, 2025 Kbuild

lore
Share & tools
lore
#0 lore raw

cover Set TOPDIR at run time

From: Chuck Lever <[email protected]>

A few recent changes have been difficult to integrate into my current CI infrastructure. Here are a couple of ideas on how to stretch in a couple of areas to help my particular arrangement.

Note that I would need to handle the inventory file location in a similar way: add a boolean Kconfig option to use a sensible default location rather than baking a fixed pathname into the kdevops .config. So, work in progress here.

Thoughts and alternatives are welcome. Even the Internet standard response ("you're doing it wrong") is OK.

Chuck Lever (2):
Set TOPDIR_PATH and generate its sha256sum at run time ansible.cfg: generate an ansible.cfg file in TOPDIR

 Kconfig                      | 15 ---------------
 Makefile                     | 12 ++++++++++++
 kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
 scripts/gen-hosts.Makefile   |  1 +
 scripts/ssh.Makefile         |  4 ++--
 5 files changed, 27 insertions(+), 17 deletions(-)
Signature · 1 line
2.51.0
#1 lore raw

RFC PATCH 1/2 Set TOPDIR_PATH and generate its sha256sum at run time

From: Chuck Lever <[email protected]>

Commit 42d27579a9d0 ("guestfs: enhance ssh config choice") added a uniquifier to the name of the ssh config to avoid multiple kdevops runs clobbering each other's ssh config entries. It uses the TOPDIR pathname set in the current .config to benefit from the nice convenience of generating an Ansible extra_var using "output yaml".

However, if the same user generates all her .configs (that will be run by other users under automation) while in the same directory (say, if they are different branches in the same git repository), the TOPDIR pathname and thus the sha256sum of that pathname will be exactly the same in all of those .config files, allowing the path names of the ephemeral ssh configs to collide.

The TOPDIR and its checksum need to be generated not by "make menuconfig" but rather by "make" in order that .config files (as saved in defconfigs/) can be truly portable.

---
 Kconfig                    | 15 ---------------
 Makefile                   |  4 ++++
 scripts/gen-hosts.Makefile |  1 +
 scripts/ssh.Makefile       |  4 ++--
 4 files changed, 7 insertions(+), 17 deletions(-)
Kconfig +0 -15
index 3f2bc8cf019a..948befbce4de 100644
@@ -5,21 +5,6 @@
 
 mainmenu "$(PROJECT) $(PROJECTRELEASE)"
 
-config TOPDIR_PATH
-	string
-	output yaml
-	default $(shell, echo $TOPDIR_PATH)
-
-config TOPDIR_PATH_HAS_SHA256SUM
-	bool
-	output yaml
-	default y
-
-config TOPDIR_PATH_SHA256SUM
-	string
-	output yaml
-	default $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
-
 config HAVE_KDEVOPS_CUSTOM_DEFAULTS
 	bool
 	default n
Makefile +4 -0
index 8755577e4d85..b6fdbe67fc07 100644
@@ -19,6 +19,8 @@ export KDEVOPS_NODES :=
 export PYTHONUNBUFFERED=1
 export TOPDIR=./
 export TOPDIR_PATH = $(shell readlink -f $(TOPDIR))
+export TOPDIR_PATH_HAS_SHA256SUM=y
+export TOPDIR_PATH_SHA256SUM = $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
 include scripts/refs.Makefile
 
 KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/templates
@@ -82,6 +84,8 @@ INCLUDES = -I include/
 CFLAGS += $(INCLUDES)
 
 ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
+ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
+ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
 
 export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
 
scripts/gen-hosts.Makefile +1 -0
index 0d1e0cd77c2e..c809b1da18e5 100644
@@ -3,6 +3,7 @@
 KDEVOPS_PLAYBOOKS_DIR_FULL_PATH=$(TOPDIR_PATH)/$(KDEVOPS_PLAYBOOKS_DIR)
 KDEVOPS_HOSTS_TEMPLATE_DIR=$(KDEVOPS_PLAYBOOKS_DIR_FULL_PATH)/roles/gen_hosts/templates
 
+GENHOSTS_EXTRA_ARGS += topdir_path='$(TOPDIR_PATH)'
 GENHOSTS_EXTRA_ARGS += kdevops_playbooks_dir='$(KDEVOPS_PLAYBOOKS_DIR)'
 
 # Relative path so that ansible can work with it
scripts/ssh.Makefile +2 -2
index d18a03e094ee..d2a583f45da2 100644
@@ -10,8 +10,8 @@ ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE))
 SSH_CONFIG_FILE:=$(subst ",,$(CONFIG_KDEVOPS_SSH_CONFIG))
 ANSIBLE_EXTRA_ARGS += sshconfig=$(shell realpath $(SSH_CONFIG_FILE))
 
-ifeq (y,$(CONFIG_TOPDIR_PATH_HAS_SHA256SUM))
-SSH_CONFIG_SUFFIX:=$(CONFIG_TOPDIR_PATH_SHA256SUM)
+ifeq (y,$(TOPDIR_PATH_HAS_SHA256SUM))
+SSH_CONFIG_SUFFIX:=$(TOPDIR_PATH_SHA256SUM)
 else
 SSH_CONFIG_SUFFIX:=$(CONFIG_KDEVOPS_HOST_PREFIX)
 endif
Signature · 1 line
2.51.0
#2 lore raw

Re: [RFC PATCH 1/2] Set TOPDIR_PATH and generate its sha256sum at run time

Quoted 8 lines · Chuck Lever

From: Chuck Lever <[email protected]>

Commit 42d27579a9d0 ("guestfs: enhance ssh config choice") added a uniquifier to the name of the ssh config to avoid multiple kdevops runs clobbering each other's ssh config entries. It uses the TOPDIR pathname set in the current .config to benefit from the nice convenience of generating an Ansible extra_var using "output yaml".

Quoted 6 lines

However, if the same user generates all her .configs (that will be run by other users under automation) while in the same directory (say, if they are different branches in the same git repository), the TOPDIR pathname and thus the sha256sum of that pathname will be exactly the same in all of those .config files, allowing the path names of the ephemeral ssh configs to collide.

Both paragraphs above took me quite a bit of time to grasp, I think it was the parenthesis, as they make me try to switch contexts. So I think the explanation of the issue can be simplified. I *think* that the issue your are describing is essentially that we currently cannot support sharing .config to other users because TOPDIR_PATH expects the same path to be used. Would that be a summary of the issue?

Quoted 3 lines

The TOPDIR and its checksum need to be generated not by "make menuconfig" but rather by "make" in order that .config files (as saved in defconfigs/) can be truly portable.

This is nice.

Quoted 39 lines
---
 Kconfig                    | 15 ---------------
 Makefile                   |  4 ++++
 scripts/gen-hosts.Makefile |  1 +
 scripts/ssh.Makefile       |  4 ++--
 4 files changed, 7 insertions(+), 17 deletions(-)
Kconfig +0 -15
index 3f2bc8cf019a..948befbce4de 100644
@@ -5,21 +5,6 @@
 
 mainmenu "$(PROJECT) $(PROJECTRELEASE)"
 
-config TOPDIR_PATH
-	string
-	output yaml
-	default $(shell, echo $TOPDIR_PATH)
-
-config TOPDIR_PATH_HAS_SHA256SUM
-	bool
-	output yaml
-	default y
-
-config TOPDIR_PATH_SHA256SUM
-	string
-	output yaml
-	default $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
-
 config HAVE_KDEVOPS_CUSTOM_DEFAULTS
 	bool
 	default n
Makefile +0 -0
index 8755577e4d85..b6fdbe67fc07 100644

Makefile export KDEVOPS_NODES := · line 22

Quoted 4 lines
Makefile +1 -0
@@ -19,6 +19,8 @@ export KDEVOPS_NODES :=
 export PYTHONUNBUFFERED=1
 export TOPDIR=./
 export TOPDIR_PATH = $(shell readlink -f $(TOPDIR))
+export TOPDIR_PATH_HAS_SHA256SUM=y

We added this a long time ago, to remain compatible with users who may git pull to the latest kdevops which didn't yet have the symbol TOPDIR_PATH_HAS_SHA256SUM.

So I think a prep patch to just remove TOPDIR_PATH_HAS_SHA256SUM now make sense.

Otherwise this is looking nice.

Luis

#3 lore raw

Re: [RFC PATCH 1/2] Set TOPDIR_PATH and generate its sha256sum at run time

Quoted 16 lines · Luis Chamberlain
Quoted 8 lines · Chuck Lever

From: Chuck Lever <[email protected]>

Commit 42d27579a9d0 ("guestfs: enhance ssh config choice") added a uniquifier to the name of the ssh config to avoid multiple kdevops runs clobbering each other's ssh config entries. It uses the TOPDIR pathname set in the current .config to benefit from the nice convenience of generating an Ansible extra_var using "output yaml".

Quoted 6 lines

However, if the same user generates all her .configs (that will be run by other users under automation) while in the same directory (say, if they are different branches in the same git repository), the TOPDIR pathname and thus the sha256sum of that pathname will be exactly the same in all of those .config files, allowing the path names of the ephemeral ssh configs to collide.

Quoted 7 lines

Both paragraphs above took me quite a bit of time to grasp, I think it was the parenthesis, as they make me try to switch contexts. So I think the explanation of the issue can be simplified. I *think* that the issue your are describing is essentially that we currently cannot support sharing .config to other users because TOPDIR_PATH expects the same path to be used. Would that be a summary of the issue?

I struggled with writing it. I wanted to capture my use case as the rationale for the change. I'll try to integrate your suggestion.

Quoted 54 lines
Quoted 3 lines

The TOPDIR and its checksum need to be generated not by "make menuconfig" but rather by "make" in order that .config files (as saved in defconfigs/) can be truly portable.

This is nice.

Quoted 39 lines
---
 Kconfig                    | 15 ---------------
 Makefile                   |  4 ++++
 scripts/gen-hosts.Makefile |  1 +
 scripts/ssh.Makefile       |  4 ++--
 4 files changed, 7 insertions(+), 17 deletions(-)
Kconfig +0 -15
index 3f2bc8cf019a..948befbce4de 100644
@@ -5,21 +5,6 @@
 
 mainmenu "$(PROJECT) $(PROJECTRELEASE)"
 
-config TOPDIR_PATH
-	string
-	output yaml
-	default $(shell, echo $TOPDIR_PATH)
-
-config TOPDIR_PATH_HAS_SHA256SUM
-	bool
-	output yaml
-	default y
-
-config TOPDIR_PATH_SHA256SUM
-	string
-	output yaml
-	default $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
-
 config HAVE_KDEVOPS_CUSTOM_DEFAULTS
 	bool
 	default n
Makefile +0 -0
index 8755577e4d85..b6fdbe67fc07 100644

Makefile export KDEVOPS_NODES := · line 22

Quoted 4 lines
Makefile +1 -0
@@ -19,6 +19,8 @@ export KDEVOPS_NODES :=
 export PYTHONUNBUFFERED=1
 export TOPDIR=./
 export TOPDIR_PATH = $(shell readlink -f $(TOPDIR))
+export TOPDIR_PATH_HAS_SHA256SUM=y

We added this a long time ago, to remain compatible with users who may git pull to the latest kdevops which didn't yet have the symbol TOPDIR_PATH_HAS_SHA256SUM.

Quoted 2 lines

So I think a prep patch to just remove TOPDIR_PATH_HAS_SHA256SUM now make sense.

Ha! I had a sentence in the patch description suggesting removal of HAS_SHA256SUM, but decided to keep things simple and took it out.

Quoted 3 lines

Otherwise this is looking nice.

Luis

Signature · 1 line
Chuck Lever
#4 lore raw

Re: [RFC PATCH 1/2] Set TOPDIR_PATH and generate its sha256sum at run time

In reply to Chuck Lever
Quoted 8 lines · Chuck Lever

From: Chuck Lever <[email protected]>

Commit 42d27579a9d0 ("guestfs: enhance ssh config choice") added a uniquifier to the name of the ssh config to avoid multiple kdevops runs clobbering each other's ssh config entries. It uses the TOPDIR pathname set in the current .config to benefit from the nice convenience of generating an Ansible extra_var using "output yaml".

Quoted 7 lines

However, if the same user generates all her .configs (that will be run by other users under automation) while in the same directory (say, if they are different branches in the same git repository), the TOPDIR pathname and thus the sha256sum of that pathname will be exactly the same in all of those .config files, allowing the path names of the ephemeral ssh configs to collide.

IIUC, this aligns very well with the sandboxing effort. Definitely, TOPDIR needs to point to a sandbox directory.

Thanks for this series!

Quoted 65 lines

The TOPDIR and its checksum need to be generated not by "make menuconfig" but rather by "make" in order that .config files (as saved in defconfigs/) can be truly portable.

---
 Kconfig                    | 15 ---------------
 Makefile                   |  4 ++++
 scripts/gen-hosts.Makefile |  1 +
 scripts/ssh.Makefile       |  4 ++--
 4 files changed, 7 insertions(+), 17 deletions(-)
Kconfig +0 -15
index 3f2bc8cf019a..948befbce4de 100644
@@ -5,21 +5,6 @@
 
 mainmenu "$(PROJECT) $(PROJECTRELEASE)"
 
-config TOPDIR_PATH
-	string
-	output yaml
-	default $(shell, echo $TOPDIR_PATH)
-
-config TOPDIR_PATH_HAS_SHA256SUM
-	bool
-	output yaml
-	default y
-
-config TOPDIR_PATH_SHA256SUM
-	string
-	output yaml
-	default $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
-
 config HAVE_KDEVOPS_CUSTOM_DEFAULTS
 	bool
 	default n
Makefile +4 -0
index 8755577e4d85..b6fdbe67fc07 100644
@@ -19,6 +19,8 @@ export KDEVOPS_NODES :=
 export PYTHONUNBUFFERED=1
 export TOPDIR=./
 export TOPDIR_PATH = $(shell readlink -f $(TOPDIR))
+export TOPDIR_PATH_HAS_SHA256SUM=y
+export TOPDIR_PATH_SHA256SUM = $(shell, ./scripts/compute_sha256sum.sh $(TOPDIR_PATH))
 include scripts/refs.Makefile
 
 KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/templates
@@ -82,6 +84,8 @@ INCLUDES = -I include/
 CFLAGS += $(INCLUDES)
 
 ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
+ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
+ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
 
 export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
 
scripts/gen-hosts.Makefile +0 -0
index 0d1e0cd77c2e..c809b1da18e5 100644

scripts/gen-hosts.Makefile line 6

Quoted 4 lines
scripts/gen-hosts.Makefile +1 -0
@@ -3,6 +3,7 @@
 KDEVOPS_PLAYBOOKS_DIR_FULL_PATH=$(TOPDIR_PATH)/$(KDEVOPS_PLAYBOOKS_DIR)
 KDEVOPS_HOSTS_TEMPLATE_DIR=$(KDEVOPS_PLAYBOOKS_DIR_FULL_PATH)/roles/gen_hosts/templates
 
+GENHOSTS_EXTRA_ARGS += topdir_path='$(TOPDIR_PATH)'

We need to feed these vars as they are variables generated based on the directory the user is currently running kdevops on, is that correct?

Quoted 18 lines
scripts/gen-hosts.Makefile +0 -0
 GENHOSTS_EXTRA_ARGS += kdevops_playbooks_dir='$(KDEVOPS_PLAYBOOKS_DIR)'
 
 # Relative path so that ansible can work with it
scripts/ssh.Makefile +2 -2
index d18a03e094ee..d2a583f45da2 100644
@@ -10,8 +10,8 @@ ifeq (y,$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE))
 SSH_CONFIG_FILE:=$(subst ",,$(CONFIG_KDEVOPS_SSH_CONFIG))
 ANSIBLE_EXTRA_ARGS += sshconfig=$(shell realpath $(SSH_CONFIG_FILE))
 
-ifeq (y,$(CONFIG_TOPDIR_PATH_HAS_SHA256SUM))
-SSH_CONFIG_SUFFIX:=$(CONFIG_TOPDIR_PATH_SHA256SUM)
+ifeq (y,$(TOPDIR_PATH_HAS_SHA256SUM))
+SSH_CONFIG_SUFFIX:=$(TOPDIR_PATH_SHA256SUM)
 else
 SSH_CONFIG_SUFFIX:=$(CONFIG_KDEVOPS_HOST_PREFIX)
 endif
#5 lore raw

RFC PATCH 2/2 ansible.cfg: generate an ansible.cfg file in TOPDIR

In reply to Chuck Lever

From: Chuck Lever <[email protected]>

I need an ansible.cfg that is generated by the Kconfig menu, but whose pathname is selected dynamically based on where kdevops is being run rather than having that path baked into the .config.

---
 Makefile                     |  8 ++++++++
 kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
 2 files changed, 20 insertions(+)
Makefile +8 -0
index b6fdbe67fc07..b014faa5632e 100644
@@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem
 export KDEVOPS_NODES_TEMPLATE :=
 export KDEVOPS_MRPROPER :=
 
+ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
+ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg
+export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
+else
 ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),)
 ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"')
 export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
 endif
+endif
 ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"')
 
 KDEVOPS_INSTALL_TARGETS :=
@@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES)
 ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
 ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
 ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
+ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
+ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)'
+endif
 
 export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
 
kconfigs/Kconfig.ansible_cfg +12 -0
index dbd189f5d19e..a44f760912cf 100644
@@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI
 	bool
 	default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS)
 
+config ANSIBLE_CFG_FILE_DEFAULT
+	bool "Use a sensible default location for the ansible.cfg file"
+	default N
+	help
+	  When this option is set to Y, kdevops creates an ansible.cfg
+	  in a sensible default location, using the settings defined
+	  in this Kconfig menu.
+
+if !ANSIBLE_CFG_FILE_DEFAULT
+
 config ANSIBLE_CFG_FILE
 	string "Ansible configuration file"
 	output yaml
@@ -41,6 +51,8 @@ config ANSIBLE_CFG_FILE
 	  For more details, refer to the Ansible documentation:
 	  https://docs.ansible.com/ansible/latest/reference_appendices/config.html#the-configuration-file
 
+endif # ANSIBLE_CFG_FILE_DEFAULT
+
 menu "Ansible Callback Plugin Configuration"
 choice
 	prompt "Ansible Callback Plugin"
Signature · 1 line
2.51.0
#6 lore raw

Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR

Quoted 47 lines · Chuck Lever

From: Chuck Lever <[email protected]>

I need an ansible.cfg that is generated by the Kconfig menu, but whose pathname is selected dynamically based on where kdevops is being run rather than having that path baked into the .config.

---
 Makefile                     |  8 ++++++++
 kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
 2 files changed, 20 insertions(+)
Makefile +8 -0
index b6fdbe67fc07..b014faa5632e 100644
@@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem
 export KDEVOPS_NODES_TEMPLATE :=
 export KDEVOPS_MRPROPER :=
 
+ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
+ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg
+export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
+else
 ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),)
 ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"')
 export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
 endif
+endif
 ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"')
 
 KDEVOPS_INSTALL_TARGETS :=
@@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES)
 ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
 ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
 ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
+ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
+ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)'
+endif
 
 export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
 
kconfigs/Kconfig.ansible_cfg +0 -0
index dbd189f5d19e..a44f760912cf 100644

kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 24

Quoted 7 lines
kconfigs/Kconfig.ansible_cfg +3 -0
@@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI
 	bool
 	default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS)
 
+config ANSIBLE_CFG_FILE_DEFAULT
+	bool "Use a sensible default location for the ansible.cfg file"
 
+	default N

defaults are always n, so this line can be removed.

kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 27

Quoted 4 lines
kconfigs/Kconfig.ansible_cfg +4 -0
+	help
+	  When this option is set to Y, kdevops creates an ansible.cfg
+	  in a sensible default location, using the settings defined
+	  in this Kconfig menu.

Why would we not want this by default?

Luis

#7 lore raw

Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR

Quoted 58 lines · Luis Chamberlain
Quoted 47 lines · Chuck Lever

From: Chuck Lever <[email protected]>

I need an ansible.cfg that is generated by the Kconfig menu, but whose pathname is selected dynamically based on where kdevops is being run rather than having that path baked into the .config.

---
 Makefile                     |  8 ++++++++
 kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
 2 files changed, 20 insertions(+)
Makefile +8 -0
index b6fdbe67fc07..b014faa5632e 100644
@@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem
 export KDEVOPS_NODES_TEMPLATE :=
 export KDEVOPS_MRPROPER :=
 
+ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
+ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg
+export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
+else
 ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),)
 ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"')
 export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
 endif
+endif
 ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"')
 
 KDEVOPS_INSTALL_TARGETS :=
@@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES)
 ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
 ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
 ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
+ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
+ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)'
+endif
 
 export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
 
kconfigs/Kconfig.ansible_cfg +0 -0
index dbd189f5d19e..a44f760912cf 100644

kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 24

Quoted 7 lines
kconfigs/Kconfig.ansible_cfg +3 -0
@@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI
 	bool
 	default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS)
 
+config ANSIBLE_CFG_FILE_DEFAULT
+	bool "Use a sensible default location for the ansible.cfg file"
 
+	default N

defaults are always n, so this line can be removed.

Quoted 6 lines

kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 27

Quoted 4 lines
kconfigs/Kconfig.ansible_cfg +4 -0
+	help
+	  When this option is set to Y, kdevops creates an ansible.cfg
+	  in a sensible default location, using the settings defined
+	  in this Kconfig menu.

Why would we not want this by default?

Because N matches the current behavior.

But I would prefer Y, actually.

Signature · 1 line
Chuck Lever
#8 lore raw

Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR

Quoted 17 lines · Chuck Lever
Quoted 9 lines · Luis Chamberlain

kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 23

Quoted 5 lines · Chuck Lever

From: Chuck Lever <[email protected]>

kconfigs/Kconfig.ansible_cfg +3 -0
+config ANSIBLE_CFG_FILE_DEFAULT
+	bool "Use a sensible default location for the ansible.cfg file"
 
+	default N

defaults are always n, so this line can be removed.

Quoted 6 lines

kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 27

Quoted 4 lines
kconfigs/Kconfig.ansible_cfg +4 -0
+	help
+	  When this option is set to Y, kdevops creates an ansible.cfg
+	  in a sensible default location, using the settings defined
+	  in this Kconfig menu.

Why would we not want this by default?

Quoted 3 lines

Because N matches the current behavior.

But I would prefer Y, actually.

I think its safe to use Y.

LUis

#9 lore raw

Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR

In reply to Chuck Lever
Quoted 5 lines · Chuck Lever

From: Chuck Lever <[email protected]>

I need an ansible.cfg that is generated by the Kconfig menu, but whose pathname is selected dynamically based on where kdevops is being run rather than having that path baked into the .config.

I'm trying to review the previous conversation before these patches as well as this series. Please, let me know if I'm understanding something wrong:

I think what we need is not to have individual configuration paths (as we have now) for ansible.cfg, inventory, etc. What we want is to have one sandbox folder, our TOPDIR, and have our kdevops sandbox directory there. All files generated and consumed by kdevops should then use that path.

Would this work? See below:

kdevops-tree $ make defconfig-selftest-xarray O=path/to/sandbox0 # Now path/to/sandbox0 contains the generated .config, extra_vars.yml, etc kdevops-tree $ make menuconfig O=path/to/sandbox0
kdevops-tree $ make O=path/to/sandbox0
kdevops-tree $ make bringup O=path/to/sandbox0
...
kdevops-tree $ make defconfig-tmpfs O=path/to/sandbox1
...
path/to/sandbox0 $ make linux
path/to/sandbox0 $ make selftests
path/to/sandbox0 $ make selftests-xarray
...
path/to/sandbox1 $ make
path/to/sandbox1 $ make bringup
path/to/sandbox1 $ make linux
path/to/sandbox1 $ make fstests
path/to/sandbox1 $ make fstests-baseline

To clarify: instead of letting users override individual file paths, we allow customization of the topdir directory (with O=) where all files are created. If O= is not passed, then PWD is assumed. I think it would still be useful to support customizing individual files, but only relative to TOPDIR. That would keep the structure cleaner, for example, using subdirectories for Ansible files, kconfigs, SSH, virsh, and so on.

Quoted 11 lines
---
 Makefile                     |  8 ++++++++
 kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
 2 files changed, 20 insertions(+)
Makefile +0 -0
index b6fdbe67fc07..b014faa5632e 100644

Makefile KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem · line 33

Quoted 7 lines
Makefile +4 -0
@@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem
 export KDEVOPS_NODES_TEMPLATE :=
 export KDEVOPS_MRPROPER :=
 
+ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
+ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg
+export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
+else

What I'm suggesting above is to always use this new option. And perhaps keep ANSIBLE_CFG_FILE configurable but inside TOPDIR. For example:
ANSIBLE_CFG_FILE=ansible/ansible.cfg.

Quoted 26 lines
Makefile +4 -0
 ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),)
 ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"')
 export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
 endif
+endif
 ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"')
 
 KDEVOPS_INSTALL_TARGETS :=
@@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES)
 ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
 ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
 ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
+ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
+ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)'
+endif
 
 export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
 
kconfigs/Kconfig.ansible_cfg +0 -0
index dbd189f5d19e..a44f760912cf 100644
@@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI
 	bool
 	default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS)
 

kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 30

Quoted 10 lines
kconfigs/Kconfig.ansible_cfg +10 -0
+config ANSIBLE_CFG_FILE_DEFAULT
+	bool "Use a sensible default location for the ansible.cfg file"
+	default N
+	help
+	  When this option is set to Y, kdevops creates an ansible.cfg
+	  in a sensible default location, using the settings defined
+	  in this Kconfig menu.
+
+if !ANSIBLE_CFG_FILE_DEFAULT
+

I think we don't need this if you agree with the approach above.

Quoted 12 lines
kconfigs/Kconfig.ansible_cfg +2 -0
 config ANSIBLE_CFG_FILE
 	string "Ansible configuration file"
 	output yaml
@@ -41,6 +51,8 @@ config ANSIBLE_CFG_FILE
 	  For more details, refer to the Ansible documentation:
 	  https://docs.ansible.com/ansible/latest/reference_appendices/config.html#the-configuration-file
 
+endif # ANSIBLE_CFG_FILE_DEFAULT
+
 menu "Ansible Callback Plugin Configuration"
 choice
 	prompt "Ansible Callback Plugin"
#10 lore raw

Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR

Quoted 10 lines · Daniel Gomez
Quoted 5 lines · Chuck Lever

From: Chuck Lever <[email protected]>

I need an ansible.cfg that is generated by the Kconfig menu, but whose pathname is selected dynamically based on where kdevops is being run rather than having that path baked into the .config.

I'm trying to review the previous conversation before these patches as well as this series. Please, let me know if I'm understanding something wrong:

Quoted 6 lines

I think what we need is not to have individual configuration paths (as we have now) for ansible.cfg, inventory, etc. What we want is to have one sandbox folder, our TOPDIR, and have our kdevops sandbox directory there. All files generated and consumed by kdevops should then use that path.

Would this work?

Well I think that would work for my particular usage, however: If I'm reading between the lines of the Ansible pathname help texts, it appears that these settings can specify search paths, not a single file.

In particular, the ANSIBLE_CFG_INVENTORY_FILE string is a comma- separated list. And, if ANSIBLE_CFG_FILE is an empty string, then there is a default set of locations /outside of/ the CWD that Ansible searches for a configuration.

I'm not against your suggestion of specifying a sandbox pathname at all; that would be a good UX simplification. There are these details, though, and I don't have any suggestions about that at the moment.

Quoted 101 lines

See below:

kdevops-tree $ make defconfig-selftest-xarray O=path/to/sandbox0 # Now path/to/sandbox0 contains the generated .config, extra_vars.yml, etc kdevops-tree $ make menuconfig O=path/to/sandbox0
kdevops-tree $ make O=path/to/sandbox0
kdevops-tree $ make bringup O=path/to/sandbox0
...
kdevops-tree $ make defconfig-tmpfs O=path/to/sandbox1
...
path/to/sandbox0 $ make linux
path/to/sandbox0 $ make selftests
path/to/sandbox0 $ make selftests-xarray
...
path/to/sandbox1 $ make
path/to/sandbox1 $ make bringup
path/to/sandbox1 $ make linux
path/to/sandbox1 $ make fstests
path/to/sandbox1 $ make fstests-baseline

To clarify: instead of letting users override individual file paths, we allow customization of the topdir directory (with O=) where all files are created. If O= is not passed, then PWD is assumed. I think it would still be useful to support customizing individual files, but only relative to TOPDIR. That would keep the structure cleaner, for example, using subdirectories for Ansible files, kconfigs, SSH, virsh, and so on.

Quoted 11 lines
---
 Makefile                     |  8 ++++++++
 kconfigs/Kconfig.ansible_cfg | 12 ++++++++++++
 2 files changed, 20 insertions(+)
Makefile +0 -0
index b6fdbe67fc07..b014faa5632e 100644

Makefile KDEVOPS_NODES_ROLE_TEMPLATE_DIR := $(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem · line 33

Quoted 7 lines
Makefile +4 -0
@@ -27,10 +27,15 @@ KDEVOPS_NODES_ROLE_TEMPLATE_DIR :=		$(KDEVOPS_PLAYBOOKS_DIR)/roles/gen_nodes/tem
 export KDEVOPS_NODES_TEMPLATE :=
 export KDEVOPS_MRPROPER :=
 
+ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
+ANSIBLE_CFG_FILE := $(TOPDIR_PATH)/ansible.cfg
+export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
+else

What I'm suggesting above is to always use this new option. And perhaps keep ANSIBLE_CFG_FILE configurable but inside TOPDIR. For example:
ANSIBLE_CFG_FILE=ansible/ansible.cfg.

Quoted 26 lines
Makefile +4 -0
 ifneq ($(strip $(CONFIG_ANSIBLE_CFG_FILE)),)
 ANSIBLE_CFG_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_FILE) | tr --delete '"')
 export ANSIBLE_CONFIG := $(ANSIBLE_CFG_FILE)
 endif
+endif
 ANSIBLE_INVENTORY_FILE := $(shell echo $(CONFIG_ANSIBLE_CFG_INVENTORY) | tr --delete '"')
 
 KDEVOPS_INSTALL_TARGETS :=
@@ -86,6 +91,9 @@ CFLAGS += $(INCLUDES)
 ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
 ANSIBLE_EXTRA_ARGS += topdir_path_has_sha256sum='$(TOPDIR_PATH_HAS_SHA256SUM)'
 ANSIBLE_EXTRA_ARGS += topdir_path_sha256sum='$(TOPDIR_PATH_SHA256SUM)'
+ifeq (y,$(CONFIG_ANSIBLE_CFG_FILE_DEFAULT))
+ANSIBLE_EXTRA_ARGS += ansible_cfg_file='$(ANSIBLE_CFG_FILE)'
+endif
 
 export KDEVOPS_HOSTS_TEMPLATE := hosts.j2
 
kconfigs/Kconfig.ansible_cfg +0 -0
index dbd189f5d19e..a44f760912cf 100644
@@ -18,6 +18,16 @@ config ANSIBLE_CFG_INVENTORY_SET_BY_CLI
 	bool
 	default $(shell, scripts/check-cli-set-var.sh KDEVOPS_HOSTS)
 

kconfigs/Kconfig.ansible_cfg config ANSIBLE_CFG_INVENTORY_SET_BY_CLI · line 30

Quoted 10 lines
kconfigs/Kconfig.ansible_cfg +10 -0
+config ANSIBLE_CFG_FILE_DEFAULT
+	bool "Use a sensible default location for the ansible.cfg file"
+	default N
+	help
+	  When this option is set to Y, kdevops creates an ansible.cfg
+	  in a sensible default location, using the settings defined
+	  in this Kconfig menu.
+
+if !ANSIBLE_CFG_FILE_DEFAULT
+

I think we don't need this if you agree with the approach above.

Quoted 12 lines
kconfigs/Kconfig.ansible_cfg +2 -0
 config ANSIBLE_CFG_FILE
 	string "Ansible configuration file"
 	output yaml
@@ -41,6 +51,8 @@ config ANSIBLE_CFG_FILE
 	  For more details, refer to the Ansible documentation:
 	  https://docs.ansible.com/ansible/latest/reference_appendices/config.html#the-configuration-file
 
+endif # ANSIBLE_CFG_FILE_DEFAULT
+
 menu "Ansible Callback Plugin Configuration"
 choice
 	prompt "Ansible Callback Plugin"
Signature · 1 line
Chuck Lever
#11 lore raw

Re: [RFC PATCH 2/2] ansible.cfg: generate an ansible.cfg file in TOPDIR

Quoted 18 lines · Chuck Lever
Quoted 10 lines · Daniel Gomez
Quoted 5 lines · Chuck Lever

From: Chuck Lever <[email protected]>

I need an ansible.cfg that is generated by the Kconfig menu, but whose pathname is selected dynamically based on where kdevops is being run rather than having that path baked into the .config.

I'm trying to review the previous conversation before these patches as well as this series. Please, let me know if I'm understanding something wrong:

Quoted 6 lines

I think what we need is not to have individual configuration paths (as we have now) for ansible.cfg, inventory, etc. What we want is to have one sandbox folder, our TOPDIR, and have our kdevops sandbox directory there. All files generated and consumed by kdevops should then use that path.

Would this work?

Quoted 8 lines

Well I think that would work for my particular usage, however: If I'm reading between the lines of the Ansible pathname help texts, it appears that these settings can specify search paths, not a single file.

In particular, the ANSIBLE_CFG_INVENTORY_FILE string is a comma- separated list. And, if ANSIBLE_CFG_FILE is an empty string, then there is a default set of locations /outside of/ the CWD that Ansible searches for a configuration.

That case will never happen, as we will ensure our sandbox var is always defined (either with O=, or with PWD) and therefore the location of the kdevops artifacts (.config, ansible.cfg, inventory, etc). That should preserve the current functionality while also allowing others to generate kdevops artifacts into separate directories.

Quoted 4 lines

I'm not against your suggestion of specifying a sandbox pathname at all; that would be a good UX simplification. There are these details, though, and I don't have any suggestions about that at the moment.

Great! And agreed. Even if my assessment above is wrong, I don't see any major blockers.