Skip to contentSkip to conversation

Fix ordering for systemd remote journal support

PATCH New Luis Chamberlain 1 participant 7 messages Jan 25, 2024 Kbuild

lore
Share & tools
lore
#0 lore raw

cover Fix ordering for systemd remote journal support

For those that may have missed it, in December I added support for systemd remote journals. This allows us to:

1) reduce the workflow watchdog time, since it is now on the localhost
2) always lets us collect node logs in case they crash and the node is gone after so we can't insepct it
3) let's us use journalctl to look for data on the target nodes

The implementation however was to address a series of issues when one is using and enabling SOAK_DURATION of 2.5 hours (so that is CONFIG_FSTESTS_SOAK_DURATION=9900), and so it was designed to cure an existing running system I was fet up with not being able to collect logs for, as crashes are much more frequent to the point you really do often loose access to the system / console.

Since I implemented support for a running kdevops system I simply togggled on CONFIG_DEVCONFIG_ENABLE_SYSTEMD_JOURNAL_REMOTE=y and ran 'make' and then:

make journal-server
make journal-client

And then use it. The other commands are:

journal-restart - Restart client upload service
journal-status - Ensure systemd-journal-remote works
journal-ls - List journals available and sizes
journal-ln - Add symlinks with hostnames

The issue with this is I had not tested a new cluster bringup. After Daniel Gomez struggled with that, I now looked into that and have identified the issues. The fix is to enhance our semantics on bringup by defining clearly what a provider is and also add clear semantics to allow us to install things *prior* to running the generic devconfig playbook with all the bells and whistles.

The last few patches should demonstrate how easy it is now to add new install targets for things we need in the pipeline for provisioning.

This should bring using CONFIG_DEVCONFIG_ENABLE_SYSTEMD_JOURNAL_REMOTE closer to a reality for most folks. The only nagging thing I don't like about it is, is that we require sudo on the local system to query the client logs, and that pollutes your systems's logs with a ton of messages with sudo.

So the next thing is to verify we can remove sudo from the watchdog calls, since we already took care of making the systemd remote journal directory with a sticky bit, and then ensured our user is part of the group systemd-journal-remote so to allow our user to read the remote journals without sudo. We just gotta test this a bit more wider.

So other than patch review, it's now a good time to ask for wider testing with CONFIG_DEVCONFIG_ENABLE_SYSTEMD_JOURNAL_REMOTE=y and so we can default CONFIG_DEVCONFIG_ENABLE_SYSTEMD_JOURNAL_REMOTE=y later.

Please let me know what you think.

Luis Chamberlain (6):
mirror: add a smart git check
bringup: split provisioning into 2 steps
bringup: share bringup method targe and agument by two steps
provision: move all provisioning things to its own Makefile
bringup: move journal-server setup early
journal-server: fix by adjusting ordering

 .gitignore                               |  2 +-
 Makefile                                 | 31 ++--------
 kconfigs/Kconfig.mirror                  | 24 ++++++--
 playbooks/roles/devconfig/tasks/main.yml |  1 +
 scripts/bringup.Makefile                 | 56 ------------------
 scripts/guestfs.Makefile                 | 19 ++----
 scripts/journal-server.Makefile          | 56 ++++++++++++++++++
 scripts/provision.Makefile               | 74 ++++++++++++++++++++++++
 scripts/terraform.Makefile               | 10 ++--
 scripts/test_git_firewall.sh             | 17 ++++++
 scripts/vagrant.Makefile                 | 19 ++----
 11 files changed, 187 insertions(+), 122 deletions(-)
 create mode 100644 scripts/journal-server.Makefile
 create mode 100644 scripts/provision.Makefile
 create mode 100755 scripts/test_git_firewall.sh
Signature · 1 line
2.42.0
#1 lore raw

PATCH 1/6 mirror: add a smart git check

If you are on a firewall chances are you will fail miserably with the default mirroring options and this is a terrible experience as the git protocol will fail. The git protocol is nice and efficient so we don't want to change the default to https.

So add a smart git check on make menuconfig which will test if the git protocol will work, otherwise disable all git options. This incurs a penalty on checking if the git protocol works once on make menuconfig with a timeout of 3 seconds. This hack only works on systems which have netcat installed, otherwise we assume git will work too. Ie, if you want this smart heuristic install netcat.

---
 kconfigs/Kconfig.mirror      | 24 +++++++++++++++++++-----
 scripts/test_git_firewall.sh | 17 +++++++++++++++++
 2 files changed, 36 insertions(+), 5 deletions(-)
 create mode 100755 scripts/test_git_firewall.sh
kconfigs/Kconfig.mirror +19 -5
index 1b237c7c32c5..3f0b15859319 100644
@@ -47,9 +47,14 @@ config INSTALL_LOCAL_LINUX_MIRROR
 	  first linux-mirror setup and just use USE_LOCAL_LINUX_MIRROR so to
 	  make use of the mirror.
 
+config MIRROR_GIT_WORKS
+	bool
+	default $(shell, ./scripts/test_git_firewall.sh)
+
 choice
 	prompt "Linux mirror protocol"
-	default MIRROR_TORVALDS_GIT
+	default MIRROR_TORVALDS_GIT if MIRROR_GIT_WORKS
+	default MIRROR_TORVALDS_HTTPS if !MIRROR_GIT_WORKS
 	depends on INSTALL_LOCAL_LINUX_MIRROR
 
 config MIRROR_TORVALDS_HTTPS
@@ -70,6 +75,7 @@ config MIRROR_TORVALDS_HTTPS_GOOGLE
 
 config MIRROR_TORVALDS_GIT
 	bool "Git"
+	depends on MIRROR_GIT_WORKS
 	help
 	  If you enable this option then git:// protocol will be used as the
 	  source of the mirror. The URL is:
@@ -105,7 +111,8 @@ config MIRROR_KDEVOPS_LINUS_URL
 
 choice
 	prompt "Linux Next mirror protocol/source"
-	default MIRROR_NEXT_GIT
+	default MIRROR_NEXT_GIT if MIRROR_GIT_WORKS
+	default MIRROR_NEXT_HTTPS if !MIRROR_GIT_WORKS
 	depends on INSTALL_LOCAL_LINUX_MIRROR
 
 config MIRROR_NEXT_HTTPS
@@ -126,6 +133,7 @@ config MIRROR_NEXT_HTTPS_GOOGLE
 
 config MIRROR_NEXT_GIT
 	bool "Git"
+	depends on MIRROR_GIT_WORKS
 	help
 	  If you enable this option then Git protocol will be used as the
 	  source of the mirror. The URL is:
@@ -142,7 +150,8 @@ config MIRROR_NEXT_URL
 
 choice
 	prompt "Linux mcgrof's linux-next fork protocol/source"
-	default MIRROR_MCGROF_GIT
+	default MIRROR_MCGROF_GIT if MIRROR_GIT_WORKS
+	default MIRROR_MCGROF_HTTPS if !MIRROR_GIT_WORKS
 	depends on INSTALL_LOCAL_LINUX_MIRROR
 
 config MIRROR_MCGROF_HTTPS
@@ -163,6 +172,7 @@ config MIRROR_MCGROF_HTTPS_GOOGLE
 
 config MIRROR_MCGROF_GIT
 	bool "Git"
+	depends on MIRROR_GIT_WORKS
 	help
 	  If you enable this option then Git protocol will be used as the
 	  source of the mirror. The URL is:
@@ -179,7 +189,8 @@ config MIRROR_MCGROF_URL
 
 choice
 	prompt "Linux mcgrof's linux fork protocol/source"
-	default MIRROR_MCGROF_LINUS_GIT
+	default MIRROR_MCGROF_LINUS_GIT if MIRROR_GIT_WORKS
+	default MIRROR_MCGROF_LINUS_HTTPS if !MIRROR_GIT_WORKS
 	depends on INSTALL_LOCAL_LINUX_MIRROR
 
 config MIRROR_MCGROF_LINUS_HTTPS
@@ -200,6 +211,7 @@ config MIRROR_MCGROF_LINUS_HTTPS_GOOGLE
 
 config MIRROR_MCGROF_LINUS_GIT
 	bool "Git"
+	depends on MIRROR_GIT_WORKS
 	help
 	  If you enable this option then Git protocol will be used as the
 	  source of the mirror. The URL is:
@@ -216,7 +228,8 @@ config MIRROR_MCGROF_LINUS_URL
 
 choice
 	prompt "Linux stable mirror protocol/source"
-	default MIRROR_STABLE_GIT
+	default MIRROR_STABLE_GIT if MIRROR_GIT_WORKS
+	default MIRROR_STABLE_HTTPS if !MIRROR_GIT_WORKS
 	depends on INSTALL_LOCAL_LINUX_MIRROR
 
 config MIRROR_STABLE_HTTPS
@@ -237,6 +250,7 @@ config MIRROR_STABLE_HTTPS_GOOGLE
 
 config MIRROR_STABLE_GIT
 	bool "Git"
+	depends on MIRROR_GIT_WORKS
 	help
 	  If you enable this option then Git protocol will be used as the
 	  source of the mirror. The URL is:
scripts/test_git_firewall.sh new file +17 -0
new file mode 100755
index 000000000000..019151d5ab9d
@@ -0,0 +1,17 @@
+#!/bin/bash
+
+CUR_VAL=$1
+which nc &> /dev/null
+if [[ $? -ne 0 ]]; then
+	echo y
+	exit
+fi
+
+# change to port 9419 to verify this will fail if your firewall does not
+# allow git access.
+nc -v -z -w 3 git.kernel.org 9418 &> /dev/null
+if [[ $? -eq 0 ]]; then
+	echo y
+else
+	echo n
+fi
Signature · 1 line
2.42.0
#2 lore raw

PATCH 2/6 bringup: split provisioning into 2 steps

In reply to Luis Chamberlain

Right now we deal with provisioning as a post bringup work item and it does two things:

1) Update ssh config file
2) Runs the devconfig playbook, which optionally installs some deps other than do basic things like setup console, kdump, etc.

The fact that deps are optional on the devconfig playbook makes it hard to add dependencies we want to install without incurring a penalty to everyone as a default option to install all standard generic kernel development tools we define under CONFIG_KDEVOPS_TRY_INSTALL_KDEV_TOOLS. The reason we disable CONFIG_KDEVOPS_TRY_INSTALL_KDEV_TOOLS by default is it delays bringup.

However the devconfig playbook is also used to place random oddball generic tools may want to install on bringup. Some of these tools we may want to try to install early on, even before we run the devconfig playbook for generic stuff like setting up the console or kdump.

To help with this, and in order to allow us to keep relying on the devconfig playbook for random packages we may want to allow users to install, we split the above two provisioning tasks into two separate tasks. This will allow us to add another middle set of targets which get executed after 1) but before 2). We will add support for this in the next subsequent commit.

For now this should be a no-op for both vagrant and libguest. For terraform we actually enhance the experience by adding a guard to not run the devconfig playbook twice if a user by mistake tried to run 'make bringup' twice. This guard was not present before for terraform.

Maybe later on we should share these snippets between all 3 bringup methods. Note we have no generic task for bare metal yet, but we should -- we should just add the devconfig playbook. That would cases where bare metal systems which are already brought up and you have ssh configuration going for already working.

---
 .gitignore                 |  4 +++-
 Makefile                   |  5 +++--
 scripts/guestfs.Makefile   | 24 ++++++++++++++++++------
 scripts/terraform.Makefile | 18 ++++++++++++++++++
 scripts/vagrant.Makefile   | 22 +++++++++++++++++-----
 5 files changed, 59 insertions(+), 14 deletions(-)
.gitignore +3 -1
index 738d073e889a..1d2653c5b26c 100644
@@ -29,9 +29,11 @@ scripts/workflows/lib/__pycache__/
 vagrant/kdevops_nodes.yaml
 vagrant/.Vagrantfile.generated
 vagrant/Vagrantfile
-vagrant/.provisioned_once
+vagrant/.provisioned_once*
 .vagrant/
 
+terraform/.provisioned_once*
+
 include/
 
 # You can override role specific stuff on these
Makefile +3 -2
index ad34743e3e53..8ee02f2a9b6c 100644
@@ -86,9 +86,10 @@ endif # CONFIG_NEEDS_LOCAL_DEVELOPMENT_PATH
 ANSIBLE_EXTRA_ARGS += $(LOCAL_DEVELOPMENT_ARGS)
 
 # These should be set as non-empty if you want any generic bring up
-# targets to come up. We support 2 bring up methods:
+# targets to come up. We support 3 bring up methods:
 #
-#  - vagrant: for kvm/virtualbox
+#  - vagrant: for kvm/virtualbox - will eventually be deprecated
+#  - libguestfs: for kvm
 #  - terraform: for any cloud provider
 #
 # If you are using bare metal, you don't do bring up, or you'd
scripts/guestfs.Makefile +18 -6
index 211ac9ba688e..17c8cbd354c1 100644
@@ -5,9 +5,11 @@ GUESTFS_ARGS :=
 KDEVOPS_NODES_TEMPLATE :=	$(KDEVOPS_NODES_ROLE_TEMPLATE_DIR)/guestfs_nodes.j2
 KDEVOPS_NODES :=		guestfs/kdevops_nodes.yaml
 
-export KDEVOPS_GUESTFS_PROVISIONED :=	guestfs/.provisioned_once
+export KDEVOPS_GUESTFS_PROVISIONED_SSH		:=	guestfs/.provisioned_once_ssh
+export KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG	:=	guestfs/.provisioned_once_devconfig
 
-KDEVOPS_MRPROPER +=		$(KDEVOPS_GUESTFS_PROVISIONED)
+KDEVOPS_MRPROPER +=		$(KDEVOPS_GUESTFS_PROVISIONED_SSH)
+KDEVOPS_MRPROPER +=		$(KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG)
 
 GUESTFS_ARGS += kdevops_enable_guestfs=True
 GUESTFS_ARGS += guestfs_path='$(TOPDIR_PATH)/guestfs'
@@ -51,11 +53,18 @@ GUESTFS_BRINGUP_DEPS :=
 GUESTFS_BRINGUP_DEPS +=  $(9P_HOST_CLONE)
 GUESTFS_BRINGUP_DEPS +=  $(LIBVIRT_PCIE_PASSTHROUGH)
 
-KDEVOPS_BRING_UP_DEPS := bringup_guestfs
+	KDEVOPS_BRING_UP_DEPS := bringup_guestfs
 KDEVOPS_DESTROY_DEPS := destroy_guestfs
 
 # Provisioning goes last
-KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_GUESTFS_PROVISIONED)
+#
+# Provisioning split into 2 steps:
+# 1) Ensuring we can use ansible with ssh
+# 2) Generic devconfig final configuration (which may include extra tools)
+#
+# Anything deps after this is dealt with on each respective workflow.
+KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_GUESTFS_PROVISIONED_SSH)
+KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG)
 
 9p_linux_clone:
 	$(Q)make linux-clone
@@ -66,15 +75,18 @@ libvirt_pcie_passthrough_permissions:
 		playbooks/libvirt_pcie_passthrough.yml \
 		-e 'ansible_python_interpreter=/usr/bin/python3'
 
-$(KDEVOPS_GUESTFS_PROVISIONED):
+$(KDEVOPS_GUESTFS_PROVISIONED_SSH):
 	$(Q)if [[ "$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE)" == "y" ]]; then \
 		LIBVIRT_DEFAULT_URI=$(CONFIG_LIBVIRT_URI) $(TOPDIR)/scripts/update_ssh_config_guestfs.py; \
 	fi
+	$(Q)touch $(KDEVOPS_GUESTFS_PROVISIONED_SSH)
+
+$(KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG):
 	$(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \
 		ansible-playbook $(ANSIBLE_VERBOSE) -i \
 			$(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ; \
 	fi
-	$(Q)touch $(KDEVOPS_GUESTFS_PROVISIONED)
+	$(Q)touch $(KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG)
 
 bringup_guestfs: $(GUESTFS_BRINGUP_DEPS)
 	$(Q)$(TOPDIR)/scripts/bringup_guestfs.sh
scripts/terraform.Makefile +18 -0
index d4caa4dbb6d6..8e95e8cd822b 100644
@@ -2,7 +2,22 @@
 
 TERRAFORM_EXTRA_VARS :=
 
+export KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG    :=      terraform/.provisioned_once_devconfig
+KDEVOPS_MRPROPER += $(KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG)
+
 KDEVOPS_BRING_UP_DEPS := bringup_terraform
+# Provisioning goes last
+#
+# Provisioning split into 2 steps:
+# 1) Ensuring we can use ansible with ssh - this is implied by all
+#    terraform providers. That is, our terraform providers all have a last
+#    provisioning element task which does this update for us as part of
+#    the above bringup_terraform.
+# 2) Generic devconfig final configuration (which may include extra tools)
+#
+# Anything deps after this is dealt with on each respective workflow.
+KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG)
+
 KDEVOPS_DESTROY_DEPS := destroy_terraform
 
 KDEVOPS_NODES_TEMPLATE :=	$(KDEVOPS_NODES_ROLE_TEMPLATE_DIR)/terraform_nodes.tf.j2
@@ -163,10 +178,13 @@ ANSIBLE_EXTRA_ARGS += $(TERRAFORM_EXTRA_VARS)
 
 bringup_terraform:
 	$(Q)$(TOPDIR)/scripts/bringup_terraform.sh
+
+$(KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG):
 	$(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \
 		ansible-playbook $(ANSIBLE_VERBOSE) -i \
 			$(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ;\
 	fi
+	$(Q)touch $(KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG)
 
 destroy_terraform:
 	$(Q)$(TOPDIR)/scripts/destroy_terraform.sh
scripts/vagrant.Makefile +17 -5
index 0633f90da719..825cfd210c6b 100644
@@ -8,11 +8,13 @@ KDEVOPS_NODES :=		vagrant/kdevops_nodes.yaml
 KDEVOPS_VAGRANT_TEMPLATE :=	$(KDEVOPS_NODES_ROLE_TEMPLATE_DIR)/Vagrantfile.j2
 KDEVOPS_VAGRANT_GENERATED:=	vagrant/.Vagrantfile.generated
 KDEVOPS_VAGRANT :=		vagrant/Vagrantfile
-export KDEVOPS_VAGRANT_PROVISIONED :=	vagrant/.provisioned_once
+export KDEVOPS_VAGRANT_PROVISIONED_SSH		:=	vagrant/.provisioned_once_ssh
+export KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG	:=	vagrant/.provisioned_once_devconfig
 
 KDEVOPS_MRPROPER +=		$(KDEVOPS_VAGRANT_GENERATED)
 KDEVOPS_MRPROPER +=		$(KDEVOPS_VAGRANT)
-KDEVOPS_MRPROPER +=		$(KDEVOPS_VAGRANT_PROVISIONED)
+KDEVOPS_MRPROPER +=		$(KDEVOPS_VAGRANT_PROVISIONED_SSH)
+KDEVOPS_MRPROPER +=		$(KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG)
 
 VAGRANT_ARGS += kdevops_vagrant_template_full_path='$(TOPDIR_PATH)/$(KDEVOPS_VAGRANT_TEMPLATE)'
 
@@ -75,7 +77,14 @@ VAGRANT_BRINGUP_DEPS +=  $(LIBVIRT_PCIE_PASSTHROUGH)
 
 KDEVOPS_BRING_UP_DEPS := bringup_vagrant
 # Provisioning goes last
-KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_VAGRANT_PROVISIONED)
+#
+# Provisioning split into 2 steps:
+# 1) Ensuring we can use ansible with ssh
+# 2) Generic devconfig final configuration (which may include extra tools)
+#
+# Anything deps after this is dealt with on each respective workflow.
+KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_VAGRANT_PROVISIONED_SSH)
+KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG)
 
 KDEVOPS_DESTROY_DEPS := destroy_vagrant
 
@@ -99,18 +108,21 @@ libvirt_pcie_passthrough_permissions:
 		playbooks/libvirt_pcie_passthrough.yml \
 		-e 'ansible_python_interpreter=/usr/bin/python3'
 
-$(KDEVOPS_VAGRANT_PROVISIONED):
+$(KDEVOPS_VAGRANT_PROVISIONED_SSH):
 	$(Q)if [[ "$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE)" == "y" ]]; then \
 		ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
 			--inventory localhost, \
 			playbooks/update_ssh_config_vagrant.yml \
 			-e 'ansible_python_interpreter=/usr/bin/python3' ;\
 	fi
+	$(Q)touch $(KDEVOPS_VAGRANT_PROVISIONED_SSH)
+
+$(KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG):
 	$(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \
 		ansible-playbook $(ANSIBLE_VERBOSE) -i \
 			$(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ;\
 	fi
-	$(Q)touch $(KDEVOPS_VAGRANT_PROVISIONED)
+	$(Q)touch $(KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG)
 
 bringup_vagrant: $(VAGRANT_BRINGUP_DEPS)
 	$(Q)$(TOPDIR)/scripts/bringup_vagrant.sh
Signature · 1 line
2.42.0
#3 lore raw

PATCH 3/6 bringup: share bringup method targe and agument by two steps

In reply to Luis Chamberlain

First we formalize now a kdevops provisioner as for one which can bring up nodes for us, can optionally setup ssh for us, and handles destroying nodes.

Prior to this patch we defined provisioning as a two step process:

1) ssh update
2) devconfig playbook is run

However we implicitly were issuing each provisioning script. So let us augment this first by defining the provisionin method first. The devconfig playbook target is also common, so no need to keep that separate. By sharing the devconfig playbook run as a shared bringup goal we also implicitly fix bare metal bringup by ensuring we run the devconfig playbook once at bringup.

We also enhance this with a new step:

KDEVOPS_BRING_UP_DEPS_EARLY

This variable can be augmented by new features for so to be able to install things using the devconfig playbook before the general devconfig playbook is run.

The diff stat reveals sharing is best. It will allow us later to say, for example add ktest support or xfstests-bld, if we want to in a much easier smoother way, as yet another provisioner.

This should pretty much be a no-op for most deployments, but enhances bare metal setup so to ensure we run the devconfig playbook. It also allows makes it easier to add new tools which may be optional which we want installed early.

---
 .gitignore                 |  4 +--
 Makefile                   | 67 ++++++++++++++++++++++++++++++--------
 scripts/guestfs.Makefile   | 31 +++---------------
 scripts/terraform.Makefile | 28 +++-------------
 scripts/vagrant.Makefile   | 31 ++++--------------
 5 files changed, 70 insertions(+), 91 deletions(-)
.gitignore +1 -3
index 1d2653c5b26c..d0499ddb2348 100644
@@ -3,6 +3,7 @@
 *.retry
 
 .kdevops\.depcheck
+.provisioned_once*
 
 guestfs/
 
@@ -29,11 +30,8 @@ scripts/workflows/lib/__pycache__/
 vagrant/kdevops_nodes.yaml
 vagrant/.Vagrantfile.generated
 vagrant/Vagrantfile
-vagrant/.provisioned_once*
 .vagrant/
 
-terraform/.provisioned_once*
-
 include/
 
 # You can override role specific stuff on these
Makefile +54 -13

79 lines not on this page · raw message

scripts/guestfs.Makefile +5 -26
index 17c8cbd354c1..054ce7a7a3ea 100644
@@ -5,11 +5,7 @@ GUESTFS_ARGS :=
 KDEVOPS_NODES_TEMPLATE :=	$(KDEVOPS_NODES_ROLE_TEMPLATE_DIR)/guestfs_nodes.j2
 KDEVOPS_NODES :=		guestfs/kdevops_nodes.yaml
 
-export KDEVOPS_GUESTFS_PROVISIONED_SSH		:=	guestfs/.provisioned_once_ssh
-export KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG	:=	guestfs/.provisioned_once_devconfig
-
-KDEVOPS_MRPROPER +=		$(KDEVOPS_GUESTFS_PROVISIONED_SSH)
-KDEVOPS_MRPROPER +=		$(KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG)
+export KDEVOPS_PROVISIONED_SSH := $(KDEVOPS_PROVISIONED_SSH_DEFAULT_GUARD)
 
 GUESTFS_ARGS += kdevops_enable_guestfs=True
 GUESTFS_ARGS += guestfs_path='$(TOPDIR_PATH)/guestfs'
@@ -53,18 +49,8 @@ GUESTFS_BRINGUP_DEPS :=
 GUESTFS_BRINGUP_DEPS +=  $(9P_HOST_CLONE)
 GUESTFS_BRINGUP_DEPS +=  $(LIBVIRT_PCIE_PASSTHROUGH)
 
-	KDEVOPS_BRING_UP_DEPS := bringup_guestfs
-KDEVOPS_DESTROY_DEPS := destroy_guestfs
-
-# Provisioning goes last
-#
-# Provisioning split into 2 steps:
-# 1) Ensuring we can use ansible with ssh
-# 2) Generic devconfig final configuration (which may include extra tools)
-#
-# Anything deps after this is dealt with on each respective workflow.
-KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_GUESTFS_PROVISIONED_SSH)
-KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG)
+KDEVOPS_PROVISION_METHOD		:= bringup_guestfs
+KDEVOPS_PROVISION_DESTROY_METHOD	:= destroy_guestfs
 
 9p_linux_clone:
 	$(Q)make linux-clone
@@ -75,18 +61,11 @@ libvirt_pcie_passthrough_permissions:
 		playbooks/libvirt_pcie_passthrough.yml \
 		-e 'ansible_python_interpreter=/usr/bin/python3'
 
-$(KDEVOPS_GUESTFS_PROVISIONED_SSH):
+$(KDEVOPS_PROVISIONED_SSH):
 	$(Q)if [[ "$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE)" == "y" ]]; then \
 		LIBVIRT_DEFAULT_URI=$(CONFIG_LIBVIRT_URI) $(TOPDIR)/scripts/update_ssh_config_guestfs.py; \
 	fi
-	$(Q)touch $(KDEVOPS_GUESTFS_PROVISIONED_SSH)
-
-$(KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG):
-	$(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \
-		ansible-playbook $(ANSIBLE_VERBOSE) -i \
-			$(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ; \
-	fi
-	$(Q)touch $(KDEVOPS_GUESTFS_PROVISIONED_DEVCONFIG)
+	$(Q)touch $(KDEVOPS_PROVISIONED_SSH)
 
 bringup_guestfs: $(GUESTFS_BRINGUP_DEPS)
 	$(Q)$(TOPDIR)/scripts/bringup_guestfs.sh
scripts/terraform.Makefile +4 -24
index 8e95e8cd822b..9c04a89d5a83 100644
@@ -2,23 +2,10 @@
 
 TERRAFORM_EXTRA_VARS :=
 
-export KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG    :=      terraform/.provisioned_once_devconfig
-KDEVOPS_MRPROPER += $(KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG)
-
-KDEVOPS_BRING_UP_DEPS := bringup_terraform
-# Provisioning goes last
-#
-# Provisioning split into 2 steps:
-# 1) Ensuring we can use ansible with ssh - this is implied by all
-#    terraform providers. That is, our terraform providers all have a last
-#    provisioning element task which does this update for us as part of
-#    the above bringup_terraform.
-# 2) Generic devconfig final configuration (which may include extra tools)
-#
-# Anything deps after this is dealt with on each respective workflow.
-KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG)
-
-KDEVOPS_DESTROY_DEPS := destroy_terraform
+export KDEVOPS_PROVISIONED_SSH := $(KDEVOPS_PROVISIONED_SSH_DEFAULT_GUARD)
+
+KDEVOPS_PROVISION_METHOD		:= bringup_terraform
+KDEVOPS_PROVISION_DESTROY_METHOD	:= destroy_terraform
 
 KDEVOPS_NODES_TEMPLATE :=	$(KDEVOPS_NODES_ROLE_TEMPLATE_DIR)/terraform_nodes.tf.j2
 KDEVOPS_NODES :=		terraform/nodes.tf
@@ -179,13 +166,6 @@ ANSIBLE_EXTRA_ARGS += $(TERRAFORM_EXTRA_VARS)
 bringup_terraform:
 	$(Q)$(TOPDIR)/scripts/bringup_terraform.sh
 
-$(KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG):
-	$(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \
-		ansible-playbook $(ANSIBLE_VERBOSE) -i \
-			$(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ;\
-	fi
-	$(Q)touch $(KDEVOPS_TERRAFORM_PROVISIONED_DEVCONFIG)
-
 destroy_terraform:
 	$(Q)$(TOPDIR)/scripts/destroy_terraform.sh
 
scripts/vagrant.Makefile +6 -25
index 825cfd210c6b..85e1c57356de 100644
@@ -8,13 +8,11 @@ KDEVOPS_NODES :=		vagrant/kdevops_nodes.yaml
 KDEVOPS_VAGRANT_TEMPLATE :=	$(KDEVOPS_NODES_ROLE_TEMPLATE_DIR)/Vagrantfile.j2
 KDEVOPS_VAGRANT_GENERATED:=	vagrant/.Vagrantfile.generated
 KDEVOPS_VAGRANT :=		vagrant/Vagrantfile
-export KDEVOPS_VAGRANT_PROVISIONED_SSH		:=	vagrant/.provisioned_once_ssh
-export KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG	:=	vagrant/.provisioned_once_devconfig
+
+export KDEVOPS_PROVISIONED_SSH := $(KDEVOPS_PROVISIONED_SSH_DEFAULT_GUARD)
 
 KDEVOPS_MRPROPER +=		$(KDEVOPS_VAGRANT_GENERATED)
 KDEVOPS_MRPROPER +=		$(KDEVOPS_VAGRANT)
-KDEVOPS_MRPROPER +=		$(KDEVOPS_VAGRANT_PROVISIONED_SSH)
-KDEVOPS_MRPROPER +=		$(KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG)
 
 VAGRANT_ARGS += kdevops_vagrant_template_full_path='$(TOPDIR_PATH)/$(KDEVOPS_VAGRANT_TEMPLATE)'
 
@@ -75,18 +73,8 @@ VAGRANT_BRINGUP_DEPS +=  $(VAGRANT_PRIVATE_BOX_DEPS)
 VAGRANT_BRINGUP_DEPS +=  $(VAGRANT_9P_HOST_CLONE)
 VAGRANT_BRINGUP_DEPS +=  $(LIBVIRT_PCIE_PASSTHROUGH)
 
-KDEVOPS_BRING_UP_DEPS := bringup_vagrant
-# Provisioning goes last
-#
-# Provisioning split into 2 steps:
-# 1) Ensuring we can use ansible with ssh
-# 2) Generic devconfig final configuration (which may include extra tools)
-#
-# Anything deps after this is dealt with on each respective workflow.
-KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_VAGRANT_PROVISIONED_SSH)
-KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG)
-
-KDEVOPS_DESTROY_DEPS := destroy_vagrant
+KDEVOPS_PROVISION_METHOD		:= bringup_vagrant
+KDEVOPS_PROVISION_DESTROY_METHOD	:= destroy_vagrant
 
 extend-extra-args-vagrant:
 	@if [[ "$(CONFIG_HAVE_VAGRANT_BOX_URL)" == "y" ]]; then \
@@ -108,21 +96,14 @@ libvirt_pcie_passthrough_permissions:
 		playbooks/libvirt_pcie_passthrough.yml \
 		-e 'ansible_python_interpreter=/usr/bin/python3'
 
-$(KDEVOPS_VAGRANT_PROVISIONED_SSH):
+$(KDEVOPS_PROVISIONED_SSH):
 	$(Q)if [[ "$(CONFIG_KDEVOPS_SSH_CONFIG_UPDATE)" == "y" ]]; then \
 		ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
 			--inventory localhost, \
 			playbooks/update_ssh_config_vagrant.yml \
 			-e 'ansible_python_interpreter=/usr/bin/python3' ;\
 	fi
-	$(Q)touch $(KDEVOPS_VAGRANT_PROVISIONED_SSH)
-
-$(KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG):
-	$(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \
-		ansible-playbook $(ANSIBLE_VERBOSE) -i \
-			$(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ;\
-	fi
-	$(Q)touch $(KDEVOPS_VAGRANT_PROVISIONED_DEVCONFIG)
+	$(Q)touch $(KDEVOPS_PROVISIONED_SSH)
 
 bringup_vagrant: $(VAGRANT_BRINGUP_DEPS)
 	$(Q)$(TOPDIR)/scripts/bringup_vagrant.sh
Signature · 1 line
2.42.0
#4 lore raw

PATCH 4/6 provision: move all provisioning things to its own Makefile

In reply to Luis Chamberlain

The provisioning aspect of kdevops is creating clutter at the top level Makefile. Move it all into its own file so to be able to easily identify related tasks and to easily extend it.

We keep augmenting KDEVOPS_BRING_UP_DEPS with KDEVOPS_BRING_UP_DEPS_EARLY in the top level Makefile so we can in between add makefiles for tools which we want to include in between and make it clear these requirements will be met before KDEVOPS_PROVISIONED_DEVCONFIG, ie running the devconfig playbook for general setup.

---
 Makefile                   | 69 +----------------------------------
 scripts/provision.Makefile | 74 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 75 insertions(+), 68 deletions(-)
 create mode 100644 scripts/provision.Makefile
Makefile +1 -68
index be686b279e2c..67e13edbf6ff 100644
@@ -85,77 +85,10 @@ endif # CONFIG_NEEDS_LOCAL_DEVELOPMENT_PATH
 
 ANSIBLE_EXTRA_ARGS += $(LOCAL_DEVELOPMENT_ARGS)
 
-# Provisioning methods should set this to their target which will ensure
-# systems will be up after this.
-KDEVOPS_PROVISION_METHOD :=
-
-# Provisioning methods should set this to their target which will ensure
-# the systems will be removed after this
-KDEVOPS_PROVISION_DESTROY_METHOD :=
-
-# The default guard for ssh provisioning. Provisioning methods can set the
-# KDEVOPS_PROVISIONED_SSH to this if they are OK with the default guard.
-KDEVOPS_PROVISIONED_SSH_DEFAULT_GUARD := .provisioned_once_ssh
-
-# This is empty on purpose so to support terraform which does this for us
-# on the terraform provider and bare metal where it is assumed you already
-# have ssh setup. Later we can grow this for other bare metal setups or to
-# consume / gather existing topologies. To be clear, terraform providers are
-# in charge of figuring out ssh configuration updates for you in kdevops.
-KDEVOPS_PROVISIONED_SSH :=
-
-# You should augment this with targets which should be after ssh is
-# ready to go to each node, and before devconfig playbook is run.
-# The devconfig playbook can be used with tags to ensure deps / setup
-# for early deps are run.
-KDEVOPS_BRING_UP_DEPS_EARLY  :=
-
-# This is shared task.
-KDEVOPS_PROVISIONED_DEVCONFIG := .provisioned_once_devconfig
-
-KDEVOPS_BRING_UP_DEPS :=
-KDEVOPS_DESTROY_DEPS :=
-
-include scripts/dynamic-kconfig.Makefile
-
-ifeq (y,$(CONFIG_TERRAFORM))
-include scripts/terraform.Makefile
-endif # CONFIG_TERRAFORM
-
-ifeq (y,$(CONFIG_VAGRANT))
-include scripts/vagrant.Makefile
-endif
-
-ifeq (y,$(CONFIG_GUESTFS))
-include scripts/guestfs.Makefile
-endif
-
-KDEVOPS_MRPROPER += $(KDEVOPS_PROVISIONED_SSH)
-KDEVOPS_MRPROPER += $(KDEVOPS_PROVISIONED_DEVCONFIG)
-
-$(KDEVOPS_PROVISIONED_DEVCONFIG):
-	$(Q)if [[ "$(CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK)" != "" ]]; then \
-		ansible-playbook $(ANSIBLE_VERBOSE) -i \
-			$(KDEVOPS_HOSTFILE) $(KDEVOPS_PLAYBOOKS_DIR)/$(KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK) ;\
-	fi
-	$(Q)touch $(KDEVOPS_PROVISIONED_DEVCONFIG)
-
-# Provisioning split into 4 steps:
-#
-# 1) Provisioning method, if one is defined
-# 2) Ensuring we can use ansible with ssh, if required
-# 3) Installing early dependencies, if any, which independent of any workflow
-# 4) Optionally run the devconfig playbook for configuration (which may include
-#    extra tools)
-#
-# Anything deps after this is dealt with on each respective workflow.
-KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_PROVISION_METHOD)
-KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_PROVISIONED_SSH)
+include scripts/provision.Makefile
 KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_BRING_UP_DEPS_EARLY)
 KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_PROVISIONED_DEVCONFIG)
 
-KDEVOPS_DESTROY_DEPS += $(KDEVOPS_PROVISION_DESTROY_METHOD)
-
 ifeq (y,$(CONFIG_WORKFLOWS))
 include workflows/Makefile
 endif # CONFIG_WORKFLOWS
scripts/provision.Makefile new file +74 -0

74 lines not on this page · raw message

Signature · 1 line
2.42.0
#5 lore raw

PATCH 5/6 bringup: move journal-server setup early

In reply to Luis Chamberlain

Now that we have done the cleaning up of the provisioners and have defined an early target task which we can piggy the back on top before the devconfig playbook is run, move the journal server setup early.

This does not make any functional changes to the file to make it easier to review the functional changes, those will be done next.

---
 Makefile                        |  2 ++
 scripts/bringup.Makefile        | 56 ---------------------------------
 scripts/journal-server.Makefile | 55 ++++++++++++++++++++++++++++++++
 3 files changed, 57 insertions(+), 56 deletions(-)
 create mode 100644 scripts/journal-server.Makefile
Makefile +2 -0
index 67e13edbf6ff..4ab805ae7ff5 100644
@@ -86,6 +86,8 @@ endif # CONFIG_NEEDS_LOCAL_DEVELOPMENT_PATH
 ANSIBLE_EXTRA_ARGS += $(LOCAL_DEVELOPMENT_ARGS)
 
 include scripts/provision.Makefile
+include scripts/journal-server.Makefile
+
 KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_BRING_UP_DEPS_EARLY)
 KDEVOPS_BRING_UP_DEPS += $(KDEVOPS_PROVISIONED_DEVCONFIG)
 
scripts/bringup.Makefile +0 -56

68 lines not on this page · raw message

scripts/journal-server.Makefile new file +55 -0

55 lines not on this page · raw message

Signature · 1 line
2.42.0
#6 lore raw

PATCH 6/6 journal-server: fix by adjusting ordering

In reply to Luis Chamberlain

Now that we have done all the legwork to address ordering for bringup and allow for items to be setup / installed with the devconfig playbook for specific features, prior to running all generic devconfig tasks, move the systemd remote journal server and client setup to run early.

Fix also ensuring we only try to setup the server once.

---
 playbooks/roles/devconfig/tasks/main.yml | 1 +
 scripts/journal-server.Makefile          | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
playbooks/roles/devconfig/tasks/main.yml +1 -0
index 061d05acdc80..3e38d8f49cdf 100644
@@ -485,6 +485,7 @@
   ansible.builtin.systemd_service:
     name: systemd-journal-remote.service
     state: started
+  run_once: true
   when:
     - devconfig_enable_systemd_journal_remote|bool
 
scripts/journal-server.Makefile +2 -1
index 3c32d5df982e..23a149bda87c 100644
@@ -40,7 +40,8 @@ journal-ln:
 		--tags vars_extra,journal_ln \
 		$(KDEVOPS_PLAYBOOKS_DIR)/devconfig.yml
 
-KDEVOPS_BRING_UP_DEPS += journal-server
+KDEVOPS_BRING_UP_DEPS_EARLY += journal-server
+KDEVOPS_BRING_UP_DEPS_EARLY += journal-client
 
 journal-help:
 	@echo "journal-server	   - Setup systemd-journal-remote on localhost"
Signature · 1 line
2.42.0