Skip to contentSkip to conversation

Fix kdevops startup

PATCH In review Daniel Gomez 3 participants 7 messages Aug 7, 2024 – Aug 9, 2024 Kbuild

lore
Share & tools
lore
#0 lore raw

cover Fix kdevops startup

This patch series solves the startup issues found when kdevops is first run in a new machine. Mainly when Ansible is not installed but you have sudo and make available.

Here the errors I found:

make kdevops-deps
make: *** No rule to make target 'kdevops-deps'. Stop.

make dynconfig-pci
/bin/bash: line 1: ansible-playbook: command not found
make[1]: *** [scripts/dynamic-pci-kconfig.Makefile:45: dynamic_pcipassthrough_kconfig] Error 127
make: *** [scripts/dynamic-pci-kconfig.Makefile:63: dynconfig-pci] Error 2

The changes below have been tested on Debian GNU/Linux trixie/sid. The packages for Red Hat and SUSE have not been verified. A double-check on those would be appreciated.

Besides fixing missing a minor target issue, I've added kdevops-install-deps target to solve basic dependencies. Here the workflow suggested:

make kdevops-deps # Warn the user missing dependencies and how to solve it make kdevops-install-deps # Installs Ansible full package with pipx make kdevops-deps # Makes sure menuconfig dependencies are met,
# libvirt, etc. (this already tests Ansible dependencies)
make dynconfig-pci # Run a simple test that runs Ansible and menuconfig

Note that running kdevops-deps, then finding missing dependencies and automatically install Ansible would be possible if preferred more an auto-solve solution. Pipx [1] is used for installing Ansible full package to get a more uptodate version than the one your distro supports.

[1] https://docs.ansible.com/ansible/latest/installation_guide/
intro_installation.html#installing-and-upgrading-ansible-with-pipx

And explicitly, the last patch also adds install-ansible-full and update-ansible-full target to manually handle Ansible installations via pipx.

Note that we need the full version of Ansible, not the minimal one (ansible-core). The latter is insufficient for kdevops as it depends on plugins only available in the full installation, such as the callback plugin configured in ansible.cfg (e.g., stdout_callback=debug).

Here the full log output:

make kdevops-deps
Verifying sudo ...
Verifying make ...
Verifying ansible-playbook ...
Missing dependency: ansible-playbook
Run make kdevops-install-deps to install missing dependencies
make: *** [Makefile.min_deps:38: ansible-playbook] Error 1

make kdevops-install-deps
Detecting Linux distribution...
Detected Debian-based distro.
Get:1 http://deb.debian.org/debian testing InRelease [169 kB] Get:2 http://deb.debian.org/debian testing-updates InRelease [49.6 kB] Get:3 http://deb.debian.org/debian testing/main Sources.diff/Index [27 [...]
Fetched 17.7 MB in 48s (366 kB/s)
359 packages can be upgraded. Run 'apt list --upgradable' to see them. sudo is already the newest version (1.9.15p5-3+b1).
make is already the newest version (4.3-4.1).
netcat-traditional is already the newest version (1.10-48.1).
The following packages were automatically installed and are no longer required:

    libpython3.11-minimal  libpython3.11-stdlib  libpython3.11t64  linux-image-6.1.0-22-amd64  python3-lib2to3

Use 'sudo apt autoremove' to remove them.

Upgrading:
libpython3-stdlib python3 python3-minimal python3-pkg-resources python3-setuptools

Installing:
pipx

Installing dependencies:
python3-argcomplete python3-pip-whl python3-platformdirs python3-setuptools-whl python3-userpath python3-venv python3.12-venv

REMOVING:
python3-distutils

Summary:
Upgrading: 5, Installing: 8, Removing: 1, Not Upgrading: 354 Download size: 4,086 kB
Space needed: 6,411 kB / 748 GB available

Get:1 http://deb.debian.org/debian testing/main amd64 python3-minimal amd64 3.12.4-1 [26.5 kB]
Get:2 http://deb.debian.org/debian testing/main amd64 python3 amd64 3.12.4-1 [27.6 kB]
[...]
Setting up python3-argcomplete (3.4.0-2) ...
Setting up python3-userpath (1.9.1-1) ...
Setting up python3-pkg-resources (70.3.0-2) ...
Setting up python3-setuptools (70.3.0-2) ...
Setting up pipx (1.6.0-1) ...
[...]
No VM guests are running outdated hypervisor (qemu) binaries on this host.
installed package ansible 10.2.0, installed using Python 3.12.4 These apps are now globally available
- ansible
- ansible-community
- ansible-config
- ansible-connection
- ansible-console
- ansible-doc
- ansible-galaxy
- ansible-inventory
- ansible-playbook
- ansible-pull
- ansible-test
- ansible-vault
⚠ Note: '/home/dagomez/.local/bin' is not on your PATH environment variable. These apps will not be globally accessible until your PATH is updated. Run pipx ensurepath to automatically add it, or manually modify your PATH in your shell's config file (e.g. ~/.bashrc). done! ✨ 🌟 ✨

source ~/.profile

  ansible-playbook --version
  ansible-playbook [core 2.17.2]
    config file = /home/dagomez/src/kdevops/ansible.cfg
    configured module search path = ['/home/dagomez/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
    ansible python module location = /home/dagomez/.local/share/pipx/venvs/ansible/lib/python3.12/site-packages/ansible
    ansible collection location = /home/dagomez/.ansible/collections:/usr/share/ansible/collections
    executable location = /home/dagomez/.local/bin/ansible-playbook
    python version = 3.12.4 (main, Jul 15 2024, 12:17:32) [GCC 13.3.0] (/home/dagomez/.local/share/pipx/venvs/ansible/bin/python)
    jinja version = 3.1.4
    libyaml = True

ansible-doc --type callback --list

  amazon.aws.aws_resource_actions        summarizes all "resource:actions" completed
  ansible.builtin.default                default Ansible screen output
  ansible.builtin.junit                  write playbook output to a JUnit file
  ansible.builtin.minimal                minimal Ansible screen output
  ansible.builtin.oneline                oneline Ansible screen output
  ansible.builtin.tree                   Save host events to files
  ansible.posix.cgroup_perf_recap        Profiles system activity of tasks and full execution using cgroups
  ansible.posix.debug                    formatted stdout/stderr display
  ansible.posix.json                     Ansible screen output as JSON
  ansible.posix.jsonl                    Ansible screen output as JSONL (lines in json format)
  ansible.posix.profile_roles            adds timing information to roles
  ansible.posix.profile_tasks            adds time information to tasks
  ansible.posix.skippy                   Ansible screen output that ignores skipped status
  ansible.posix.timer                    Adds time to play stats
  community.general.cgroup_memory_recap  Profiles maximum memory usage of tasks and full execution using cgroups
  community.general.context_demo         demo callback that adds play/task context
  community.general.counter_enabled      adds counters to the output items (tasks and hosts/task)
  community.general.default_without_diff The default ansible callback without diff output
  community.general.dense                minimal stdout output
  community.general.diy                  Customize the output
  community.general.elastic              Create distributed traces for each Ansible task in Elastic APM
  community.general.hipchat              post task events to hipchat
  community.general.jabber               post task events to a jabber server
  community.general.log_plays            write playbook output to log file
  community.general.loganalytics         Posts task results to Azure Log Analytics
  community.general.logdna               Sends playbook logs to LogDNA
  community.general.logentries           Sends events to Logentries
  community.general.logstash             Sends events to Logstash
  community.general.mail                 Sends failure events via email
  community.general.nrdp                 Post task results to a Nagios server through nrdp
  community.general.null                 Don't display stuff to screen
  community.general.opentelemetry        Create distributed traces with OpenTelemetry
  community.general.say                  notify using software speech synthesizer
  community.general.selective            only print certain tasks
  community.general.slack                Sends play events to a Slack channel
  community.general.splunk               Sends task result events to Splunk HTTP Event Collector
  community.general.sumologic            Sends task result events to Sumologic
  community.general.syslog_json          sends JSON events to syslog
  community.general.timestamp            Adds simple timestamp for each header
  community.general.unixy                condensed Ansible output
  community.general.yaml                 YAML-ized Ansible screen output
  community.grafana.grafana_annotations  send ansible events as annotations on charts to grafana over http api
  ovirt.ovirt.stdout                     Output the log of ansible
  theforeman.foreman.foreman             Sends events to Foreman

make kdevops-deps
Verifying sudo ...
Verifying make ...
Verifying ansible-playbook ...
Verifying pipx ...

PLAY [localhost] ********************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [install-menuconfig-deps : Install kdevops deps to run make menuconfig] [...]
Installed dependencies

make dynconfig-pci

PLAY [localhost] ********************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [gen_pci_kconfig : Dump pci output in machine-readible form] [...]

<menuconfig opened>
<Ctrl-C>

Your configuration changes were NOT saved.

To: [email protected]
To: Luis Chamberlain <[email protected]>
Cc: Daniel Gomez (Samsung) <[email protected]>
Cc: [email protected]

---
Daniel Gomez (3):
Makefile: fix kdevops-deps missing target Makefile.kdevops: add extra dependencies Makefile.min_deps: add kdevops-install-deps target

 Makefile                 |  3 +++
 Makefile.kdevops         |  2 +-
 Makefile.min_deps        | 43 ++++++++++++++++++++++++++++++++++++++-----
 scripts/bringup.Makefile |  4 +++-
 4 files changed, 45 insertions(+), 7 deletions(-)

---
base-commit: 8e31f894270b79ab109b08fe785a2b1a9c1d8431
change-id: 20240807-fix-startup-kdevops-8f5e58b73124

Best regards,

Signature · 1 line
Daniel Gomez <[email protected]>
#1 lore raw

PATCH 1/3 Makefile: fix kdevops-deps missing target

Fixes:

make kdevops-deps
make: *** No rule to make target 'kdevops-deps'. Stop

---
 Makefile | 3 +++
 1 file changed, 3 insertions(+)
Makefile +3 -0
index b961e44..0fb7ab3 100644
@@ -78,6 +78,9 @@ ANSIBLE_EXTRA_ARGS += kdevops_version='$(PROJECTRELEASE)'
 export KDEVOPS_HOSTS_TEMPLATE := $(KDEVOPS_HOSTFILE).j2
 export KDEVOPS_HOSTS := $(KDEVOPS_HOSTFILE)
 
+include Makefile.kdevops
+DEFAULT_DEPS += kdevops-deps
+
 LOCAL_DEVELOPMENT_ARGS	:=
 ifeq (y,$(CONFIG_NEEDS_LOCAL_DEVELOPMENT_PATH))
 include Makefile.local
Signature · 1 line
Git-146)
#2 lore raw

PATCH 2/3 Makefile.kdevops: add extra dependencies

In reply to Daniel Gomez

To be able to run kdevops project, we need to make sure all dependencies are met. For that, make sure the kdevops-deps includes min_deps target (checks for sudo, make, nc and ansible-playbook) and menuconfig-deps (installs ncurses).

Keep menuconfig-deps as target for users who do not want to run the full kdevops dependency list check.

---
 Makefile.kdevops | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Makefile.kdevops +1 -1
index ac83bd8..3edbeda 100644
@@ -68,7 +68,7 @@ kdevops_libvirt_storage_pool_create:
 PHONY += kdevops_libvirt_storage_pool_create
 KDEVOPS_VAGRANT_WORK += kdevops_libvirt_storage_pool_create
 
-kdevops_deps: kdevops_terraform_deps $(KDEVOPS_VAGRANT_WORK) $(KDEVOPS_DEPS)
+kdevops_deps: $(BINARY_DEPS) $(BINARY_DEP_VCHECKS) menuconfig-deps kdevops_terraform_deps $(KDEVOPS_VAGRANT_WORK) $(KDEVOPS_DEPS)
 	@echo Installed dependencies
 PHONY += kdevops_deps
 
Signature · 1 line
Git-146)
#3 lore raw

PATCH 3/3 Makefile.min_deps: add kdevops-install-deps target

In reply to Daniel Gomez

This ensures based packages are installed using distro package manager method and pipx for ansible.

Use pipx over distro package manager as the ansible package provided by distro my be outdated.

https://docs.ansible.com/ansible/latest/installation_guide/
intro_installation.html#installing-and-upgrading-ansible-with-pipx

Add targets to install and update full Ansible package using pipx. Full Ansible package is required to run kdevops (minimal installation lacks support for callback plugins and more).

---
 Makefile.min_deps        | 43 ++++++++++++++++++++++++++++++++++++++-----
 scripts/bringup.Makefile |  4 +++-
 2 files changed, 41 insertions(+), 6 deletions(-)
Makefile.min_deps +38 -5
index a1cb99a..c281379 100644
@@ -5,11 +5,12 @@
 # have something more elaborate, like a version check add it to its
 # own file as we do with scripts/version_check/ansible-playbook
 
-BINARY_DEPS :=
-BINARY_DEPS += sudo
-BINARY_DEPS += make
-BINARY_DEPS += nc
-BINARY_DEPS += ansible-playbook
+BINARY_DEPS := sudo make ansible-playbook pipx
+DEBIAN_PACKAGE_DEPS := sudo make netcat-traditional pipx
+SUSE_PACKAGE_DEPS := sudo make nc python-pipx
+REDHAT_PACKAGE_DEPS := sudo make ncat pipx
+PIPX_DEPS :=
+PIPX_DEPS += ansible-playbook
 ifneq (y,$(CONFIG_KDEVOPS_FIRST_RUN))
 ifeq (y,$(CONFIG_GUESTFS))
 BINARY_DEPS += virt-builder
@@ -36,8 +37,40 @@ $(BINARY_DEPS):
 	$(Q)echo Verifying $@ ...
 	$(Q)set -e ; which $@ > /dev/null || ( \
 		echo Missing dependency: $@ ;\
+		echo Run 'make kdevops-install-deps' to install missing dependencies ;\
 	false)
 
+install_package_deps:
+	@echo "Detecting Linux distribution..."
+	@if [ -f /etc/debian_version ]; then \
+		echo "Detected Debian-based distro."; \
+		sudo apt update && sudo apt install -y $(DEBIAN_PACKAGE_DEPS); \
+	elif [ -f /etc/redhat-release ]; then \
+		echo "Detected Red Hat-based distro."; \
+		sudo yum install -y $(REDHAT_PACKAGE_DEPS); \
+	elif [ -f /etc/SuSE-release ]; then \
+		echo "Detected SUSE-based distro."; \
+		sudo zypper install -y $(SUSE_PACKAGE_DEPS); \
+	else \
+		echo "Unsupported Linux distribution."; \
+		exit 1; \
+	fi
+
+pipx_install_ansible_full:
+	$(Q)pipx install --force --include-deps ansible
+
+pipx_update_ansible_full:
+	$(Q)pipx upgrade --include-injected ansible
+
+kdevops-install-deps: install_package_deps pipx_install_ansible_full
+PHONY += kdevops-install-deps
+
+install-ansible-full: pipx_install_ansible_full
+PHONY += install-ansible-full
+
+update-ansible-full: pipx_update_ansible_full
+PHONY += update-ansible-full
+
 $(KDEVOPS_DEPCHECK): $(DYNAMIC_DEPS)
 	$(Q)echo The current minimum kdevops expected dependencies are met.
 	$(Q)touch $(KDEVOPS_DEPCHECK)
scripts/bringup.Makefile +3 -1
index 148547c..2917218 100644
@@ -35,7 +35,9 @@ HELP_TARGETS+=bringup-help-menu
 
 bringup-setup-help-menu:
 	@echo "Generic bring up set up targets:"
-	@echo "kdevops-deps        - Installs what we need on the localhost"
+	@echo "kdevops-deps            - Installs what we need on the localhost"
+	@echo "install-ansible-full    - Force install full Ansible package with pipx"
+	@echo "update-ansible-full     - Updates full Ansible package with pipx"
 
 HELP_TARGETS += bringup-setup-help-menu
 
Signature · 1 line
Git-146)
#4 lore raw

Re: [PATCH 3/3] Makefile.min_deps: add kdevops-install-deps target

Quoted 3 lines · On Wed, Aug 07, 2024 at 12:40:44PM +0200, Daniel Gomez via B4 Relay wrote:

Add targets to install and update full Ansible package using pipx. Full Ansible package is required to run kdevops (minimal installation lacks support for callback plugins and more).

I've never had to use pipx and I if we're going down this road this surely requires more distro review. We stride to always use distro packages. Is there a distro package which provides suport for callback plugins / more?

Are other seeing this on their rolling distro other than debian testing?

Luis

#5 lore raw

Re: [PATCH 3/3] Makefile.min_deps: add kdevops-install-deps target

Quoted 10 lines · Luis Chamberlain
Quoted 3 lines · On Wed, Aug 07, 2024 at 12:40:44PM +0200, Daniel Gomez via B4 Relay wrote:

Add targets to install and update full Ansible package using pipx. Full Ansible package is required to run kdevops (minimal installation lacks support for callback plugins and more).

I've never had to use pipx and I if we're going down this road this surely requires more distro review. We stride to always use distro packages. Is there a distro package which provides suport for callback plugins / more?

I think the 'ansible' package in Debian provides the full Ansible package [1].

[1] https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#selecting-an-ansible-package-and-version-to-install

The motivation for using pipx over a distro package is that it provides the latest version from the Ansible project via PyPi, and it ensures consistent Ansible versions across all distributions that support pipx. It also allows you to pin specific Ansible versions if needed.

At the moment, pipx will provide the following:
package ansible 10.2.0
package ansible-core 2.17.2

Debian stable
ansible 7.3.0+dfsg-1
ansible-core (2.14.3-1)
https://packages.debian.org/bookworm/ansible
https://packages.debian.org/bookworm/ansible-core

Debian testing:
ansible (10.1.0+dfsg-1)
ansible-core (2.17.1-1)
https://packages.debian.org/trixie/ansible
https://packages.debian.org/trixie/ansible-core

Debian unstable:
ansible (10.1.0+dfsg-1)
ansible-core (2.17.2-1)
https://packages.debian.org/sid/ansible
https://packages.debian.org/sid/ansible-core

Note that the pipx commands will provide full Ansible packages only for the user. If we want to provide system wide installation via pipx we need the --global argument.

Distros are now suggesting the following:

root@localhost:~# python3 -m pip install daemon
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to install.

If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed.

If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed.

See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

On the Ansible documentation, they state:

The ansible or ansible-core packages may be available in your operating systems package manager, and you are free to install these packages with your preferred method. For more information, see the Installing Ansible on specific operating systems guide. These installation instructions only cover the officially supported means of installing the python packages with pip.

And because of PEP 668, they added:

On some systems, it may not be possible to install Ansible with pip, due to decisions made by the operating system developers. In such cases, pipx is a widely available alternative.

So, while they provide specific distro installation steps [2], they seem to encourage using PyPi via pip/pipx. However, pipx may also come with its own Python problems. In short, pipx depends on useradd, which introduced a functional regression for remote SSH executions in version 1.9.0. This is solved now upstream. Below you can find the links with all the details [3].

[2] Debian official Ansible installation steps:
https://docs.ansible.com/ansible/latest/installation_guide/installation_distros.html#installing-ansible-on-debian

[3] pipx/useradd problem:
https://github.com/pypa/pipx/pull/1034
https://github.com/pypa/pipx/pull/1075/files
https://github.com/ofek/userpath/issues/50
https://github.com/ofek/userpath/commit/a4c7c719099116ee0246997837e2022c7aad2ef8

Quoted 2 lines

Are other seeing this on their rolling distro other than debian testing?

I found this for Red Hat/Fedora:
https://discuss.python.org/t/pep-668-marking-python-base-environments-as-externally-managed/10302/10

And these links are the PEP 668:

https://peps.python.org/pep-0668/
https://packaging.python.org/en/latest/specifications/externally-managed-environments/#externally-managed-environments

I think distros still prefer users to use their Python packages, then use venv and/or pipx. I hope the above helps in evaluating the pros and cons of both approaches.

Quoted 2 lines

Luis

#6 lore raw

Re: [PATCH 3/3] Makefile.min_deps: add kdevops-install-deps target

Quoted 7 lines · Daniel Gomez (Samsung)
Quoted 6 lines · Luis Chamberlain
Quoted 3 lines · On Wed, Aug 07, 2024 at 12:40:44PM +0200, Daniel Gomez via B4 Relay wrote:

Add targets to install and update full Ansible package using pipx. Full Ansible package is required to run kdevops (minimal installation lacks support for callback plugins and more).

Quoted 8 lines
Quoted 4 lines

I've never had to use pipx and I if we're going down this road this surely requires more distro review. We stride to always use distro packages. Is there a distro package which provides suport for callback plugins / more?

I think the 'ansible' package in Debian provides the full Ansible package [1].

[1] https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#selecting-an-ansible-package-and-version-to-install

Ah OK, great.

Quoted 4 lines

The motivation for using pipx over a distro package is that it provides the latest version from the Ansible project via PyPi, and it ensures consistent Ansible versions across all distributions that support pipx.

That in no way justifies making this the default.

Just as when we supported vagrant, or today terraform, some of these software packages have a lot of dependencies and it takes a distro to get right.

Additionally you are also brushing off the possible risks over this without doing a full assessment.

But you're making it a default.

So no, if you want to have this mechanism in place to support, just add a new Kconfig option and let the folks who want to venture into that world do so, at their own risk.

The default should be we just use distro packages.

Additionally a more contained working environment and deterministic one is supporting NiXOS, that would force full on requirements from the get go. But kdevops has always supported regular distributions precisely because getting all this right for distros was hard, but with a bit of work, it actually is not if we stick to supporting rolling distributions.

So NixOS can be supported, and so can this pipx thing but by *deafult*, no.

Quoted 2 lines

I think distros still prefer users to use their Python packages, then use venv and/or pipx.

Right, exactly, so let's keep it that way. If you want a different work, Kconfig lets you do that. Just don't force it as default.

Luis