Random short subjects
cover Random short subjects
From: Chuck Lever <[email protected]>
Here is a set of changes I've been toying with since early December while working on getting AWS hosting to work again. I'm interested in community comments, objections, or other musings before I just merge these. There's no real rocket surgery here, but there could be some controversy.
Work on the nfsd role's logic to set up an LVM for exports, which was posted here previously, continues slowly. I have something that seems pretty nice for guestfs, but unfortunately it's complicated by latent bugs and old changes in terraform when it comes to AWS. I'm still working on it.
Chuck Lever (9):
bringup: prevent the use of vagrant
Remove defconfigs/nfsd
terraform/aws: Clean up a few nits
Clean up task messages
devconfig: Replace import_tasks with include_tasks terraform/aws: Enable selection of a Fedora (40) AMI terraform/aws: Add a couple more instance types terraform/aws: Add support for m7a instance types
terraform: Hoist aws_profile into the terraform Kconfig
defconfigs/nfsd | 396 ------------------ docs/kdevops-terraform.md | 8 +- kconfigs/Kconfig.bringup | 16 - kconfigs/workflows/Kconfig.shared | 2 + .../devconfig/tasks/install-deps/main.yml | 22 +- playbooks/roles/devconfig/tasks/main.yml | 10 +- playbooks/roles/gen_tfvars/defaults/main.yml | 1 + .../templates/aws/terraform.tfvars.j2 | 1 + .../roles/update_etc_hosts/tasks/main.yml | 2 +- scripts/terraform.Makefile | 1 + terraform/aws/Kconfig | 152 ++++++- 11 files changed, 170 insertions(+), 441 deletions(-) delete mode 100644 defconfigs/nfsd
Signature · 1 line
2.47.1
PATCH 1/9 bringup: prevent the use of vagrant
From: Chuck Lever <[email protected]>
Vagrant virtualization has been deprecated for some time, but its been left in kdevops to allow time for folks to move over to guestfs, which has been the default for a while.
It's time to start removing Vagrant, so remove the menu choice to select the use of Vagrant virtualization. This change will be quick to revert if there are loud complaints.
--- kconfigs/Kconfig.bringup | 16 ---------------- 1 file changed, 16 deletions(-)
kconfigs/Kconfig.bringup +0 -16
@@ -9,22 +9,6 @@ choice prompt "Node bring up method" default GUESTFS -config VAGRANT - bool "Vagrant for local virtualization (KVM / VirtualBox)" - select KDEVOPS_SSH_CONFIG_UPDATE_STRICT - select EXTRA_STORAGE_SUPPORTS_ZNS - select EXTRA_STORAGE_SUPPORTS_LARGEIO - select BRINGUP_SUPPORTS_CXL - depends on TARGET_ARCH_X86_64 - help - This option will enable use of Vagrant. Enable this if you want to - use local virtualization technologies such as KVM with libvirt or - VirtualBox. Enabling this feature will enable you to select which - target guests to use, and configures things for you. It will also - ensure dependencies are installed for Vagrant when running: - - make deps - config GUESTFS bool "Use guestfs-tools for local virtualization via KVM and libvirt" select KDEVOPS_ENABLE_GUESTFS
Signature · 1 line
2.47.1
PATCH 2/9 Remove defconfigs/nfsd
In reply to [email protected]From: Chuck Lever <[email protected]>
This file has become stale.
--- defconfigs/nfsd | 396 ------------------------------------------------ 1 file changed, 396 deletions(-) delete mode 100644 defconfigs/nfsd
defconfigs/nfsd deleted +0 -396
@@ -1,396 +0,0 @@ -# CONFIG_KDEVOPS_FIRST_RUN is not set -CONFIG_DISTRO_FEDORA=y - -# -# Target architecture -# -CONFIG_TARGET_ARCH_X86_64=y -# CONFIG_TARGET_ARCH_ARM64 is not set -# CONFIG_TARGET_ARCH_PPC64LE is not set -CONFIG_TARGET_ARCH="x86_64" -# end of Target architecture - -# -# SSH update configuration -# -CONFIG_KDEVOPS_SSH_CONFIG_UPDATE=y -CONFIG_KDEVOPS_SSH_CONFIG="~/.ssh/config" -CONFIG_KDEVOPS_SSH_CONFIG_UPDATE_STRICT=y -CONFIG_KDEVOPS_SSH_CONFIG_UPDATE_BACKUP=y -# end of SSH update configuration - -CONFIG_EXTRA_STORAGE_SUPPORTS_512=y -CONFIG_EXTRA_STORAGE_SUPPORTS_1K=y -CONFIG_EXTRA_STORAGE_SUPPORTS_2K=y -CONFIG_EXTRA_STORAGE_SUPPORTS_4K=y -CONFIG_GIT_ALTERNATIVES=y -CONFIG_GIT_LINUX_KDEVOPS_GITHUB=y -# CONFIG_GIT_LINUX_KDEVOPS_GITLAB is not set -# CONFIG_SETUP_POSTFIX_EMAIL_RELAY is not set -# CONFIG_HYPERVISOR_TUNING is not set -CONFIG_ENABLE_LOCAL_LINUX_MIRROR=y -CONFIG_USE_LOCAL_LINUX_MIRROR=y -CONFIG_INSTALL_LOCAL_LINUX_MIRROR=y -# CONFIG_MIRROR_TORVALDS_HTTPS is not set -# CONFIG_MIRROR_TORVALDS_HTTPS_GOOGLE is not set -CONFIG_MIRROR_TORVALDS_GIT=y -CONFIG_MIRROR_TORVALDS_URL="git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git" -CONFIG_MIRROR_KDEVOPS_LINUS_HTTPS=y -CONFIG_MIRROR_KDEVOPS_LINUS_URL="https://github.com/linux-kdevops/linux.git" -# CONFIG_MIRROR_NEXT_HTTPS is not set -# CONFIG_MIRROR_NEXT_HTTPS_GOOGLE is not set -CONFIG_MIRROR_NEXT_GIT=y -CONFIG_MIRROR_NEXT_URL="git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git" -# CONFIG_MIRROR_MCGROF_HTTPS is not set -# CONFIG_MIRROR_MCGROF_HTTPS_GOOGLE is not set -CONFIG_MIRROR_MCGROF_GIT=y -CONFIG_MIRROR_MCGROF_URL="git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux-next.git" -# CONFIG_MIRROR_MCGROF_LINUS_HTTPS is not set -# CONFIG_MIRROR_MCGROF_LINUS_HTTPS_GOOGLE is not set -CONFIG_MIRROR_MCGROF_LINUS_GIT=y -CONFIG_MIRROR_MCGROF_LINUS_URL="git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git" -# CONFIG_MIRROR_STABLE_HTTPS is not set -# CONFIG_MIRROR_STABLE_HTTPS_GOOGLE is not set -CONFIG_MIRROR_STABLE_GIT=y -CONFIG_MIRROR_STABLE_URL="git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git" -CONFIG_MIRROR_QEMU_HTTPS_GITLAB=y -# CONFIG_MIRROR_QEMU_SSH_GITLAB is not set -# CONFIG_MIRROR_QEMU_HTTPS_GITHUB is not set -# CONFIG_MIRROR_QEMU_SSH_GITHUB is not set -CONFIG_MIRROR_QEMU_URL="https://gitlab.com/qemu-project/qemu.git" -CONFIG_MIRROR_QEMU_JIC23_HTTPS_GITLAB=y -# CONFIG_MIRROR_QEMU_JIC23_SSH_GITLAB is not set -# CONFIG_MIRROR_QEMU_JIC23_HTTPS_GITHUB is not set -# CONFIG_MIRROR_QEMU_JIC23_SSH_GITHUB is not set -CONFIG_MIRROR_QEMU_JIC23_URL="https://gitlab.com/jic23/qemu.git" - -# -# Bring up methods -# -CONFIG_VAGRANT=y -# CONFIG_GUESTFS is not set -# CONFIG_TERRAFORM is not set -# CONFIG_SKIP_BRINGUP is not set -CONFIG_LIBVIRT=y -CONFIG_VAGRANT_LIBVIRT_SELECT=y -# CONFIG_VAGRANT_VIRTUALBOX is not set -# CONFIG_VAGRANT_DEBIAN is not set -# CONFIG_VAGRANT_OPENSUSE is not set -CONFIG_VAGRANT_FEDORA=y -# CONFIG_VAGRANT_KDEVOPS is not set -CONFIG_HAVE_VAGRANT_BOX_VERSION=y -CONFIG_VAGRANT_FEDORA_BOX_STRING="fedora/38-cloud-base" -# CONFIG_VAGRANT_FEDORA_OLD_RELEASE is not set -# CONFIG_RHEL_REGISTER is not set -CONFIG_VAGRANT_FEDORA_NEW_RELEASE=y -CONFIG_VAGRANT_BOX="fedora/38-cloud-base" -CONFIG_VAGRANT_BOX_UPDATE_ON_BRINGUP=y -# CONFIG_LIBVIRT_INSTALL is not set -# CONFIG_LIBVIRT_CONFIGURE is not set -# CONFIG_LIBVIRT_VERIFY is not set -# CONFIG_LIBVIRT_URI_SYSTEM is not set -CONFIG_LIBVIRT_URI_SESSION=y -# CONFIG_LIBVIRT_URI_CUSTOM is not set -CONFIG_LIBVIRT_URI_PATH="qemu:///session" -CONFIG_LIBVIRT_SYSTEM_URI_PATH="qemu:///system" -CONFIG_LIBVIRT_QEMU_GROUP="qemu" -CONFIG_KDEVOPS_STORAGE_POOL_PATH="/var/lib/libvirt/images" -CONFIG_QEMU_BIN_PATH="/usr/bin/qemu-system-x86_64" -CONFIG_LIBVIRT_URI="qemu:///session" -CONFIG_LIBVIRT_SYSTEM_URI="qemu:///system" -CONFIG_LIBVIRT_SESSION=y -CONFIG_LIBVIRT_SESSION_SOCKET="/run/libvirt/libvirt-sock-ro" -CONFIG_LIBVIRT_SESSION_MANAGEMENT_NETWORK_DEVICE="virbr0" -CONFIG_LIBVIRT_SESSION_PUBLIC_NETWORK_DEV="virbr0" -CONFIG_USE_LIBVIRT_MIRROR=y -# CONFIG_QEMU_BUILD is not set -# CONFIG_QEMU_USE_DEVELOPMENT_VERSION is not set -CONFIG_QEMU_BIN_PATH_LIBVIRT="/usr/bin/qemu-system-x86_64" -CONFIG_QEMU_INSTALL_DIR_LIBVIRT="/usr/bin" -# CONFIG_LIBVIRT_LARGE_CPU is not set -# CONFIG_LIBVIRT_VCPUS_2 is not set -# CONFIG_LIBVIRT_VCPUS_4 is not set -CONFIG_LIBVIRT_VCPUS_8=y -# CONFIG_LIBVIRT_VCPUS_16 is not set -# CONFIG_LIBVIRT_VCPUS_32 is not set -# CONFIG_LIBVIRT_VCPUS_64 is not set -# CONFIG_LIBVIRT_VCPUS_128 is not set -# CONFIG_LIBVIRT_VCPUS_255 is not set -CONFIG_LIBVIRT_VCPUS_COUNT=8 -# CONFIG_LIBVIRT_MEM_2G is not set -# CONFIG_LIBVIRT_MEM_3G is not set -# CONFIG_LIBVIRT_MEM_4G is not set -CONFIG_LIBVIRT_MEM_8G=y -# CONFIG_LIBVIRT_MEM_16G is not set -# CONFIG_LIBVIRT_MEM_32G is not set -CONFIG_LIBVIRT_MEM_MB=8192 -# CONFIG_LIBVIRT_MACHINE_TYPE_DEFAULT is not set -CONFIG_LIBVIRT_MACHINE_TYPE_Q35=y -CONFIG_LIBVIRT_HOST_PASSTHROUGH=y -# CONFIG_LIBVIRT_EXTRA_STORAGE_DRIVE_NVME is not set -CONFIG_LIBVIRT_EXTRA_STORAGE_DRIVE_VIRTIO=y -# CONFIG_LIBVIRT_EXTRA_STORAGE_DRIVE_IDE is not set -CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_512=y -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_1K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_2K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_4K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_8K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_16K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_32K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_64K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_128K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_256K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_512K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_1M is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_PHYSICAL_BLOCK_SIZE_2M is not set -CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_512=y -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_1K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_2K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_4K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_8K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_16K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_32K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_64K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_128K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_256K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_512K is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_1M is not set -# CONFIG_LIBVIRT_EXTRA_STORAGE_VIRTIO_LOGICAL_BLOCK_SIZE_2M is not set -# CONFIG_LIBVIRT_AIO_MODE_NATIVE is not set -# CONFIG_LIBVIRT_AIO_MODE_THREADS is not set -CONFIG_LIBVIRT_AIO_MODE_IO_URING=y -CONFIG_LIBVIRT_AIO_MODE="io_uring" -CONFIG_LIBVIRT_AIO_CACHE_MODE_NONE=y -# CONFIG_LIBVIRT_AIO_CACHE_MODE_WRITETHROUGH is not set -# CONFIG_LIBVIRT_AIO_CACHE_MODE_WRITEBACK is not set -# CONFIG_LIBVIRT_AIO_CACHE_MODE_DIRECTSYNC is not set -# CONFIG_LIBVIRT_AIO_CACHE_MODE_UNSAFE is not set -CONFIG_LIBVIRT_AIO_CACHE_MODE="none" -# CONFIG_LIBVIRT_EXTRA_DRIVE_FORMAT_QCOW2 is not set -CONFIG_LIBVIRT_EXTRA_DRIVE_FORMAT_RAW=y -# CONFIG_LIBVIRT_STORAGE_POOL_PATH_INFER_ADVANCED is not set -# CONFIG_LIBVIRT_STORAGE_POOL_PATH_CUSTOM_CWD is not set -CONFIG_LIBVIRT_STORAGE_POOL_PATH_CUSTOM_DEFAULT_DISTRO=y -# CONFIG_LIBVIRT_STORAGE_POOL_PATH_CUSTOM_MANUAL is not set -CONFIG_LIBVIRT_STORAGE_POOL_PATH_AUTO="/var/lib/libvirt/images" -# CONFIG_LIBVIRT_STORAGE_POOL_CREATE is not set -CONFIG_QEMU_NVME_ZONE_DRIVE_SIZE=102400 -CONFIG_QEMU_NVME_ZONE_ZASL=0 -CONFIG_QEMU_NVME_ZONE_SIZE="128M" -CONFIG_QEMU_NVME_ZONE_CAPACITY="0M" -CONFIG_QEMU_NVME_ZONE_MAX_ACTIVE=0 -CONFIG_QEMU_NVME_ZONE_MAX_OPEN=0 -CONFIG_QEMU_NVME_ZONE_PHYSICAL_BLOCK_SIZE=4096 -CONFIG_QEMU_NVME_ZONE_LOGICAL_BLOCK_SIZE=4096 -# CONFIG_QEMU_ENABLE_EXTRA_DRIVE_LARGEIO is not set -CONFIG_QEMU_LARGEIO_DRIVE_BASE_SIZE=10240 -CONFIG_QEMU_LARGEIO_COMPAT_SIZE=512 -CONFIG_QEMU_LARGEIO_MAX_POW_LIMIT=12 -# end of Bring up methods - -# -# Bring up goals -# -CONFIG_KDEVOPS_TRY_REFRESH_REPOS=y -CONFIG_KDEVOPS_CUSTOM_YUM_REPOFILE="" -CONFIG_KDEVOPS_TRY_UPDATE_SYSTEMS=y -CONFIG_KDEVOPS_TRY_INSTALL_KDEV_TOOLS=y -# CONFIG_KDEVOPS_SETUP_KTLS is not set -CONFIG_KDEVOPS_SETUP_NFSD=y - -# -# Configure the kernel NFS server -# -CONFIG_NFSD_EXPORT_DEVICE_0="/dev/disk/by-id/virtio-kdevops2" -CONFIG_NFSD_EXPORT_DEVICE_1="/dev/disk/by-id/virtio-kdevops3" -CONFIG_NFSD_EXPORT_FSTYPE_BTRFS=y -# CONFIG_NFSD_EXPORT_FSTYPE_XFS is not set -# CONFIG_NFSD_EXPORT_FSTYPE_EXT4 is not set -# CONFIG_NFSD_EXPORT_FSTYPE_TMPFS is not set -CONFIG_NFSD_EXPORT_FSTYPE="btrfs" -CONFIG_NFSD_EXPORT_PATH="/export" -CONFIG_NFSD_EXPORT_OPTIONS="rw,insecure,no_root_squash" -CONFIG_NFSD_THREADS=16 -# end of Configure the kernel NFS server -# end of Bring up goals - -# -# Node sysctl configuration -# -# CONFIG_SYSCTL_TUNING is not set -# end of Node sysctl configuration - -# -# Target workflows -# -CONFIG_WORKFLOWS=y - -# -# Shared workflow configuration -# - -# -# Shared workflow data partition -# -# CONFIG_WORKFLOW_DATA_DEVICE_ENABLE_CUSTOM is not set -CONFIG_WORKFLOW_DATA_DEVICE="/dev/disk/by-id/virtio-kdevops0" -CONFIG_WORKFLOW_DATA_PATH="/data" -CONFIG_WORKFLOW_INFER_USER_AND_GROUP=y -# CONFIG_WORKFLOW_DATA_FSTYPE_XFS is not set -# CONFIG_WORKFLOW_DATA_FSTYPE_EXT4 is not set -CONFIG_WORKFLOW_DATA_FSTYPE_BTRFS=y -CONFIG_WORKFLOW_DATA_FSTYPE="btrfs" -CONFIG_WORKFLOW_DATA_LABEL="data" -# end of Shared workflow data partition - -# CONFIG_WORKFLOW_MAKE_CMD_OVERRIDE is not set -# CONFIG_WORKFLOW_KDEVOPS_GIT_CUSTOM is not set -CONFIG_WORKFLOW_KDEVOPS_GIT="https://github.com/linux-kdevops/kdevops.git" -CONFIG_WORKFLOW_KDEVOPS_GIT_VERSION="HEAD" -CONFIG_WORKFLOW_KDEVOPS_GIT_DATA="{{data_path}}/kdevops" -CONFIG_WORKFLOW_KDEVOPS_DIR="{{data_path}}/kdevops" -# end of Shared workflow configuration - -# CONFIG_WORKFLOW_LINUX_DISTRO is not set -CONFIG_WORKFLOW_LINUX_CUSTOM=y - -# -# Get and install Linux from git -# -CONFIG_BOOTLINUX=y -CONFIG_BOOTLINUX_9P=y - -# -# Modify default 9p configuration -# -CONFIG_BOOTLINUX_9P_HOST_PATH="/home/cel/src/kdevops/linux" -CONFIG_BOOTLINUX_9P_MSIZE=131072 -CONFIG_BOOTLINUX_9P_FSDEV="kdevops_9p_fsdev" -CONFIG_BOOTLINUX_9P_SECURITY_MODEL="none" -CONFIG_BOOTLINUX_9P_DRIVER="virtio-9p-pci" -CONFIG_BOOTLINUX_9P_MOUNT_TAG="kdevops_9p_bootlinux" -# end of Modify default 9p configuration - -# CONFIG_BOOTLINUX_LINUS is not set -# CONFIG_BOOTLINUX_STABLE is not set -CONFIG_BOOTLINUX_DEV=y -# CONFIG_BOOTLINUX_CUSTOM is not set -CONFIG_BOOTLINUX_TREE_NEXT=y -# CONFIG_BOOTLINUX_TREE_BTRFS_DEVEL is not set -# CONFIG_BOOTLINUX_TREE_JLAYTON_LINUX is not set -# CONFIG_BOOTLINUX_TREE_MCGROF_NEXT is not set -CONFIG_BOOTLINUX_TREE_NEXT_REF="next-20240108" -CONFIG_BOOTLINUX_TREE_NEXT_NAME="linux-next" -CONFIG_BOOTLINUX_TREE_NEXT_URL="/mirror/linux-next.git" -CONFIG_BOOTLINUX_TREE_NAME="linux-next" -CONFIG_BOOTLINUX_TREE="/mirror/linux-next.git" -# CONFIG_BOOTLINUX_TEST_MESSAGE_ID is not set -CONFIG_BOOTLINUX_TREE_REF="next-20240108" -CONFIG_BOOTLINUX_TREE_LOCALVERSION="" -# CONFIG_BOOTLINUX_SHALLOW_CLONE is not set -# end of Get and install Linux from git - -CONFIG_WORKFLOWS_TESTS=y -# CONFIG_WORKFLOWS_TESTS_DEMOS is not set -CONFIG_WORKFLOWS_LINUX_TESTS=y -CONFIG_WORKFLOWS_DEDICATED_WORKFLOW=y -CONFIG_KDEVOPS_WORKFLOW_DEDICATE_FSTESTS=y -# CONFIG_KDEVOPS_WORKFLOW_DEDICATE_BLKTESTS is not set -# CONFIG_KDEVOPS_WORKFLOW_DEDICATE_CXL is not set -# CONFIG_KDEVOPS_WORKFLOW_DEDICATE_PYNFS is not set -# CONFIG_KDEVOPS_WORKFLOW_DEDICATE_SELFTESTS is not set -CONFIG_KDEVOPS_WORKFLOW_ENABLE_FSTESTS=y - -# -# Configure and run fstests -# -CONFIG_HAVE_DISTRO_PREFERS_FSTESTS_WATCHDOG=y -CONFIG_HAVE_DISTRO_PREFERS_FSTESTS_WATCHDOG_KILL=y -CONFIG_FSTESTS_NFS_ENABLE=y -# CONFIG_FSTESTS_XFS is not set -# CONFIG_FSTESTS_BTRFS is not set -# CONFIG_FSTESTS_EXT4 is not set -CONFIG_FSTESTS_NFS=y -# CONFIG_FSTESTS_TMPFS is not set -CONFIG_FSTESTS_FSTYP="nfs" -CONFIG_FSTESTS_TFB_COPY_ENABLE=y -CONFIG_FSTESTS_WATCHDOG=y -CONFIG_FSTESTS_WATCHDOG_CHECK_TIME=5 -CONFIG_FSTESTS_WATCHDOG_MAX_NEW_TEST_TIME=60 -CONFIG_FSTESTS_WATCHDOG_HUNG_MULTIPLIER_LONG_TESTS=10 -CONFIG_FSTESTS_WATCHDOG_HUNG_FAST_TEST_MAX_TIME=5 -CONFIG_FSTESTS_WATCHDOG_KILL_TASKS_ON_HANG=y -# CONFIG_FSTESTS_WATCHDOG_RESET_HUNG_SYSTEMS is not set - -# -# Configure how nfs should be tested -# -CONFIG_FSTESTS_USE_KDEVOPS_NFSD=y -CONFIG_FSTESTS_NFS_MANUAL_COVERAGE=y -# CONFIG_FSTESTS_NFS_SECTION_DEFAULT is not set -CONFIG_FSTESTS_NFS_SECTION_V42=y -CONFIG_FSTESTS_NFS_SECTION_V41=y -CONFIG_FSTESTS_NFS_SECTION_V40=y -CONFIG_FSTESTS_NFS_SECTION_V3=y -# end of Configure how nfs should be tested - -CONFIG_FSTESTS_GIT="https://github.com/linux-kdevops/fstests.git" -CONFIG_FSTESTS_DATA="{{data_path}}/fstests" -CONFIG_FSTESTS_GIT_VERSION="HEAD" -CONFIG_FSTESTS_DATA_TARGET="/var/lib/xfstests" -# CONFIG_FSTESTS_TESTDEV_UNDER_DEVELOPMENT is not set -CONFIG_FSTESTS_TEST_DIR="/media/test" -CONFIG_FSTESTS_SCRATCH_MNT="/media/scratch" -CONFIG_FSTESTS_SETUP_SYSTEM=y -CONFIG_FSTESTS_RUN_TESTS=y -CONFIG_FSTESTS_RUN_AUTO_GROUP_TESTS=y -CONFIG_FSTESTS_RUN_CUSTOM_GROUP_TESTS="auto" -CONFIG_FSTESTS_EXCLUDE_TEST_GROUPS="" -# CONFIG_FSTESTS_ENABLE_RUN_CUSTOM_TESTS is not set -CONFIG_FSTESTS_RUN_LARGE_DISK_TESTS=y -CONFIG_FSTESTS_SOAK_DURATION=0 -# end of Configure and run fstests - -CONFIG_KDEVOPS_WORKFLOW_GIT_CLONES_KDEVOPS_GIT=y -# end of Target workflows - -# -# Kdevops configuration -# -# CONFIG_KDEVOPS_USE_DISTRO_HOSTS_PREFIX is not set -CONFIG_KDEVOPS_HOSTS_PREFIX="cel" -# CONFIG_KDEVOPS_BASELINE_AND_DEV is not set - -# -# Ansible post-bring up provisioning configuration -# -CONFIG_KDEVOPS_PLAYBOOK_DIR="playbooks" -CONFIG_KDEVOPS_ANSIBLE_PROVISION_ENABLE=y -CONFIG_KDEVOPS_ANSIBLE_PROVISION_PLAYBOOK="devconfig.yml" -CONFIG_KDEVOPS_DEVCONFIG_ENABLE=y -CONFIG_KDEVOPS_DEVCONFIG_ENABLE_CONSOLE=y -CONFIG_KDEVOPS_DEVCONFIG_KERNEL_CONSOLE_SETTINGS="console=tty0 console=tty1 console=ttyS0,115200n8" -CONFIG_KDEVOPS_DEVCONFIG_GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --parity=no --stop=1" -CONFIG_KDEVOPS_GRUB_TIMEOUT=2 -CONFIG_KDEVOPS_DEVCONFIG_ENABLE_SYSTEMD_WATCHDOG=y -CONFIG_KDEVOPS_DEVCONFIG_SYSTEMD_WATCHDOG_TIMEOUT_RUNTIME="5min" -CONFIG_KDEVOPS_DEVCONFIG_SYSTEMD_WATCHDOG_TIMEOUT_REBOOT="10min" -CONFIG_KDEVOPS_DEVCONFIG_SYSTEMD_WATCHDOG_TIMEOUT_KEXEC="5min" -CONFIG_KDEVOPS_ANSIBLE_INVENTORY_FILE="hosts" -CONFIG_KDEVOPS_PYTHON_INTERPRETER="/usr/bin/python3" -CONFIG_KDEVOPS_PYTHON_OLD_INTERPRETER="/usr/bin/python2" -# CONFIG_DEVCONFIG_ENABLE_SYSTEMD_JOURNAL_REMOTE is not set -# CONFIG_KDEVOPS_DEVCONFIG_ENABLE_CUSTOM_REPOS is not set -# end of Ansible post-bring up provisioning configuration - -# -# Kernel continous integration configuration -# -CONFIG_KERNEL_CI_DEFAULT_STEADY_STATE_GOAL=100 -CONFIG_KERNEL_CI=y -CONFIG_KERNEL_CI_ENABLE_STEADY_STATE=y -CONFIG_KERNEL_CI_STEADY_STATE_GOAL=100 -CONFIG_KERNEL_CI_STEADY_STATE_INCREMENTAL=y -# CONFIG_KERNEL_CI_EMAIL_REPORT is not set -# end of Kernel continous integration configuration -# end of Kdevops configuration
Signature · 1 line
2.47.1
PATCH 3/9 terraform/aws: Clean up a few nits
In reply to [email protected]From: Chuck Lever <[email protected]>
--- terraform/aws/Kconfig | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
terraform/aws/Kconfig +2 -3
@@ -39,7 +39,7 @@ config TERRAFORM_AWS_US_WEST_2 "OwnerId": "137112412989" - And that is what vaule to use for that ami-0efa651876de2a5ce + And that is what value to use for ami-0efa651876de2a5ce for the TERRAFORM_AWS_AMI_OWNER. To get the ami-* for your regions just go to your EC2 console, that console will be associated with a region already. You have to change regions if you want to look
@@ -237,8 +237,7 @@ config TERRAFORM_AWS_NS default "debian-stretch-hvm-arm64-*" if TERRAFORM_AWS_NS_DEBIAN_STRETCH_ARM64 default "al2023-*-x86_64" if TARGET_ARCH_X86_64 && TERRAFORM_AWS_AMI_AMAZON_X86_64 default "al2023-*-arm64" if TARGET_ARCH_ARM64 && TERRAFORM_AWS_AMI_AMAZON_ARM64 - default "RHEL-9.*x86_64-*" if TARGET_ARCH_X86_64 && TERRAFORM_AWS_RHEL9_X86_64 - default "RHEL-9.*x86_64-*" if TARGET_ARCH_X86_64 && TERRAFORM_AWS_RHEL9_X86_64_EAST + default "RHEL-9.*x86_64-*" if TERRAFORM_AWS_RHEL9_X86_64 || TERRAFORM_AWS_RHEL9_X86_64_EAST default "RHEL-9.*arm64-*" if TARGET_ARCH_ARM64 && TERRAFORM_AWS_RHEL9_ARM64 default "suse-sles-15-sp4-*x86_64" if TARGET_ARCH_X86_64 && TERRAFORM_AWS_SLE15_SP4_X86_64 default "suse-sles-15-sp4-*arm64" if TARGET_ARCH_ARM64 && TERRAFORM_AWS_SLE15_SP4_ARM64
Signature · 1 line
2.47.1
PATCH 4/9 Clean up task messages
In reply to [email protected]From: Chuck Lever <[email protected]>
The "{{ ansible_host }}" variable is never evaluated in task messages, so fix the message. A similar message in the update_etc_hosts role is updated for consistency.
--- playbooks/roles/devconfig/tasks/main.yml | 2 +- playbooks/roles/update_etc_hosts/tasks/main.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
playbooks/roles/devconfig/tasks/main.yml +1 -1
@@ -10,7 +10,7 @@ skip: true tags: vars,vars_simple,vars_extra -- name: Wait for {{ ansible_host }} to be available +- name: Wait for target nodes to become reachable wait_for_connection: - name: Gathering facts
playbooks/roles/update_etc_hosts/tasks/main.yml +1 -1
@@ -10,7 +10,7 @@ skip: true tags: vars -- name: Wait for hosts to become available +- name: Wait for target nodes to become reachable wait_for_connection: - name: Gather network facts
Signature · 1 line
2.47.1
PATCH 5/9 devconfig: Replace import_tasks with include_tasks
In reply to [email protected]From: Chuck Lever <[email protected]>
On a fresh Fedora 40 system, under buildbot, the devconfig role fails:
statically imported: /var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/main.yml statically imported: /var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/debian/main.yml statically imported: /var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/suse/main.yml redirecting (type: modules) ansible.builtin.zypper to community.general.zypper
ERROR! couldn't resolve module/action 'zypper'. This often indicates a misspelling, missing collection, or incorrect module path.
It shouldn't care about zypper, that's a Suse thing. Somewhere the os_family sensing logic has gone hay-wire.
Handle this by using dynamic include_task instead of static import_task, and restructure the os_family checking task in install-deps/main.yml.
--- .../devconfig/tasks/install-deps/main.yml | 22 ++++++++++--------- playbooks/roles/devconfig/tasks/main.yml | 8 +++---- 2 files changed, 16 insertions(+), 14 deletions(-)
playbooks/roles/devconfig/tasks/install-deps/main.yml +12 -10
@@ -24,21 +24,23 @@ skip: true tags: vars -# tasks to setup up repos, register system if needed and install -# preferred devtools packages. -- name: Distribution specific setup - tags: vars_simple - import_tasks: debian/main.yml +- name: Debian-specific setup + ansible.builtin.include_tasks: debian/main.yml when: - - ansible_facts['os_family']|lower == 'debian' + - ansible_os_family == "Debian" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool -- import_tasks: suse/main.yml tags: vars_simple + +- name: SuSE-specific setup + ansible.builtin.include_tasks: suse/main.yml when: - - ansible_facts['os_family']|lower == 'suse' + - ansible_os_family == "Suse" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool -- import_tasks: redhat/main.yml tags: vars_simple + +- name: Red Hat-specific setup + ansible.builtin.include_tasks: redhat/main.yml when: - - ansible_facts['os_family']|lower == 'redhat' + - ansible_os_family == "RedHat" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool + tags: vars_simple
playbooks/roles/devconfig/tasks/main.yml +4 -4
@@ -31,10 +31,10 @@ # Distro specific - name: Install dependencies - import_tasks: install-deps/main.yml + ansible.builtin.include_tasks: install-deps/main.yml - name: Configure custom repositories and install packages - import_tasks: config-custom-repos-and-packages/main.yml + ansible.builtin.include_tasks: config-custom-repos-and-packages/main.yml when: - ansible_facts['os_family']|lower == 'redhat'
@@ -447,7 +447,7 @@ tags: [ 'console' ] - name: Update your boot GRUB file if necessary - import_tasks: update-grub/main.yml + ansible.builtin.include_tasks: update-grub/main.yml when: - grub2_config_file.stat.exists - devconfig_enable_console|bool
@@ -637,7 +637,7 @@ tags: [ 'sysctl' ] - name: Rev the kernel to the latest distribution kotd - import_tasks: kotd-rev-kernel/main.yml + ansible.builtin.include_tasks: kotd-rev-kernel/main.yml when: - devconfig_enable_kotd|bool tags: [ 'kotd' ]
Signature · 1 line
2.47.1
Re: [PATCH v1 5/9] devconfig: Replace import_tasks with include_tasks
Hi Chuck,
Quoted 29 lines · On Tue, Jan 14, 2025 at 03:03:44PM -0500, [email protected] wrote:
From: Chuck Lever <[email protected]>
On a fresh Fedora 40 system, under buildbot, the devconfig role fails:
statically imported:
/var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/main.ymlstatically imported:/var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/debian/main.ymlstatically imported:/var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/suse/main.ymlredirecting (type: modules) ansible.builtin.zypper to community.general.zypper
ERROR! couldn't resolve module/action 'zypper'. This often indicates a misspelling, missing collection, or incorrect module path.It shouldn't care about zypper, that's a Suse thing. Somewhere the
os_familysensing logic has gone hay-wire.Handle this by using dynamic
include_taskinstead of staticimport_task, and restructure theos_familychecking task ininstall-deps/main.yml.Signed-off-by Chuck Lever <[email protected]>--- .../devconfig/tasks/install-deps/main.yml | 22 ++++++++++--------- playbooks/roles/devconfig/tasks/main.yml | 8 +++---- 2 files changed, 16 insertions(+), 14 deletions(-)playbooks/roles/devconfig/tasks/install-deps/main.yml +0
-0
playbooks/roles/devconfig/tasks/install-deps/main.yml line 28
Quoted 10 lines
playbooks/roles/devconfig/tasks/install-deps/main.yml +2
-5@@ -24,21 +24,23 @@ skip: true tags: vars -# tasks to setup up repos, register system if needed and install -# preferred devtools packages. -- name: Distribution specific setup - tags: vars_simple - import_tasks: debian/main.yml +- name: Debian-specific setup + ansible.builtin.include_tasks: debian/main.yml
Replacing import_tasks in the ansible include_tasks module is making the kdevops CI fail as reported here [1].
https://lore.kernel.org/kdevops/[email protected]/T/#t
The CI failure indicates the tasks inside debian yml are not actually executed before the rest of the playbook.
I have other debian/non-debian workflows where the conditional does not conflict with the imported tasks when using ansible_facts['os_family']| lower == 'debian'. And I think this is what we need here. Any idea what the error might be on your end when using import_tasks instead of include_tasks?
Daniel
Quoted 61 lines
playbooks/roles/devconfig/tasks/install-deps/main.yml +10
-5when: - - ansible_facts['os_family']|lower == 'debian' + - ansible_os_family == "Debian" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool -- import_tasks: suse/main.yml tags: vars_simple + +- name: SuSE-specific setup + ansible.builtin.include_tasks: suse/main.yml when: - - ansible_facts['os_family']|lower == 'suse' + - ansible_os_family == "Suse" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool -- import_tasks: redhat/main.yml tags: vars_simple + +- name: Red Hat-specific setup + ansible.builtin.include_tasks: redhat/main.yml when: - - ansible_facts['os_family']|lower == 'redhat' + - ansible_os_family == "RedHat" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool + tags: vars_simpleplaybooks/roles/devconfig/tasks/main.yml +4
-4@@ -31,10 +31,10 @@ # Distro specific - name: Install dependencies - import_tasks: install-deps/main.yml + ansible.builtin.include_tasks: install-deps/main.yml - name: Configure custom repositories and install packages - import_tasks: config-custom-repos-and-packages/main.yml + ansible.builtin.include_tasks: config-custom-repos-and-packages/main.yml when: - ansible_facts['os_family']|lower == 'redhat'@@ -447,7 +447,7 @@ tags: [ 'console' ] - name: Update your boot GRUB file if necessary - import_tasks: update-grub/main.yml + ansible.builtin.include_tasks: update-grub/main.yml when: - grub2_config_file.stat.exists - devconfig_enable_console|bool@@ -637,7 +637,7 @@ tags: [ 'sysctl' ] - name: Rev the kernel to the latest distribution kotd - import_tasks: kotd-rev-kernel/main.yml + ansible.builtin.include_tasks: kotd-rev-kernel/main.yml when: - devconfig_enable_kotd|bool tags: [ 'kotd' ]Signature · 1 line
2.47.1
Re: [PATCH v1 5/9] devconfig: Replace import_tasks with include_tasks
Quoted 46 lines · Daniel Gomez
Hi Chuck,
Quoted 29 lines · On Tue, Jan 14, 2025 at 03:03:44PM -0500, [email protected] wrote:
From: Chuck Lever <[email protected]>
On a fresh Fedora 40 system, under buildbot, the devconfig role fails:
statically imported:
/var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/main.ymlstatically imported:/var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/debian/main.ymlstatically imported:/var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/suse/main.ymlredirecting (type: modules) ansible.builtin.zypper to community.general.zypper
ERROR! couldn't resolve module/action 'zypper'. This often indicates a misspelling, missing collection, or incorrect module path.It shouldn't care about zypper, that's a Suse thing. Somewhere the
os_familysensing logic has gone hay-wire.Handle this by using dynamic
include_taskinstead of staticimport_task, and restructure theos_familychecking task ininstall-deps/main.yml.Signed-off-by Chuck Lever <[email protected]>--- .../devconfig/tasks/install-deps/main.yml | 22 ++++++++++--------- playbooks/roles/devconfig/tasks/main.yml | 8 +++---- 2 files changed, 16 insertions(+), 14 deletions(-)playbooks/roles/devconfig/tasks/install-deps/main.yml +0
-0playbooks/roles/devconfig/tasks/install-deps/main.yml line 28
Quoted 10 lines
playbooks/roles/devconfig/tasks/install-deps/main.yml +2
-5@@ -24,21 +24,23 @@ skip: true tags: vars -# tasks to setup up repos, register system if needed and install -# preferred devtools packages. -- name: Distribution specific setup - tags: vars_simple - import_tasks: debian/main.yml +- name: Debian-specific setup + ansible.builtin.include_tasks: debian/main.ymlReplacing
import_tasksin the ansibleinclude_tasksmodule is making the kdevops CI fail as reported here [1].
Quoted 4 lines
https://lore.kernel.org/kdevops/[email protected]/T/#t
The CI failure indicates the tasks inside debian yml are not actually executed before the rest of the playbook.
I don't understand. The playbook steps are executed serially by all hosts. The steps in the included .yml file should happen before any subsequent steps in the devconfig.yml file.
Quoted 3 lines
I have other debian/non-debian workflows where the conditional does not conflict with the imported tasks when using ansible_facts['os_family']| lower == 'debian'.
So in previous cases where this came up, Debian was working as expected, but Red Hat (Fedora) was not. This happened in every one of these cases.
Those fixes are far enough in the past that they would appear only in the "historical" kdevops repo.
Quoted 3 lines
And I think this is what we need here. Any idea
what the error might be on your end when usingimport_tasksinstead ofinclude_tasks?
I can have a look. I don't understand the bug report above well enough to answer your question immediately.
"import_tasks" has some subtle behaviors -- it seems to be pretty brittle in my experience.
Quoted 61 lines
Quoted 61 lines
playbooks/roles/devconfig/tasks/install-deps/main.yml +10
-5when: - - ansible_facts['os_family']|lower == 'debian' + - ansible_os_family == "Debian" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool -- import_tasks: suse/main.yml tags: vars_simple + +- name: SuSE-specific setup + ansible.builtin.include_tasks: suse/main.yml when: - - ansible_facts['os_family']|lower == 'suse' + - ansible_os_family == "Suse" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool -- import_tasks: redhat/main.yml tags: vars_simple + +- name: Red Hat-specific setup + ansible.builtin.include_tasks: redhat/main.yml when: - - ansible_facts['os_family']|lower == 'redhat' + - ansible_os_family == "RedHat" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool + tags: vars_simpleplaybooks/roles/devconfig/tasks/main.yml +4
-4@@ -31,10 +31,10 @@ # Distro specific - name: Install dependencies - import_tasks: install-deps/main.yml + ansible.builtin.include_tasks: install-deps/main.yml - name: Configure custom repositories and install packages - import_tasks: config-custom-repos-and-packages/main.yml + ansible.builtin.include_tasks: config-custom-repos-and-packages/main.yml when: - ansible_facts['os_family']|lower == 'redhat'@@ -447,7 +447,7 @@ tags: [ 'console' ] - name: Update your boot GRUB file if necessary - import_tasks: update-grub/main.yml + ansible.builtin.include_tasks: update-grub/main.yml when: - grub2_config_file.stat.exists - devconfig_enable_console|bool@@ -637,7 +637,7 @@ tags: [ 'sysctl' ] - name: Rev the kernel to the latest distribution kotd - import_tasks: kotd-rev-kernel/main.yml + ansible.builtin.include_tasks: kotd-rev-kernel/main.yml when: - devconfig_enable_kotd|bool tags: [ 'kotd' ]Signature · 1 line
2.47.1
Signature · 1 line
Chuck Lever
Re: [PATCH v1 5/9] devconfig: Replace import_tasks with include_tasks
Quoted 46 lines · Chuck Lever
Quoted 45 lines · Daniel Gomez
Hi Chuck,
Quoted 29 lines · On Tue, Jan 14, 2025 at 03:03:44PM -0500, [email protected] wrote:
From: Chuck Lever <[email protected]>
On a fresh Fedora 40 system, under buildbot, the devconfig role fails:
statically imported:
/var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/main.ymlstatically imported:/var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/debian/main.ymlstatically imported:/var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/suse/main.ymlredirecting (type: modules) ansible.builtin.zypper to community.general.zypper
ERROR! couldn't resolve module/action 'zypper'. This often indicates a misspelling, missing collection, or incorrect module path.It shouldn't care about zypper, that's a Suse thing. Somewhere the
os_familysensing logic has gone hay-wire.Handle this by using dynamic
include_taskinstead of staticimport_task, and restructure theos_familychecking task ininstall-deps/main.yml.Signed-off-by Chuck Lever <[email protected]>--- .../devconfig/tasks/install-deps/main.yml | 22 ++++++++++--------- playbooks/roles/devconfig/tasks/main.yml | 8 +++---- 2 files changed, 16 insertions(+), 14 deletions(-)playbooks/roles/devconfig/tasks/install-deps/main.yml +0
-0playbooks/roles/devconfig/tasks/install-deps/main.yml line 27
Quoted 9 lines
playbooks/roles/devconfig/tasks/install-deps/main.yml +2
-5@@ -24,21 +24,23 @@ skip: true tags: vars -# tasks to setup up repos, register system if needed and install -# preferred devtools packages. -- name: Distribution specific setup - tags: vars_simple - import_tasks: debian/main.yml +- name: Debian-specific setup + ansible.builtin.include_tasks: debian/main.ymlReplacing
import_tasksin the ansibleinclude_tasksmodule is making the kdevops CI fail as reported here [1].
Quoted 8 lines
Quoted 4 lines
https://lore.kernel.org/kdevops/[email protected]/T/#t
The CI failure indicates the tasks inside debian yml are not actually executed before the rest of the playbook.
I don't understand. The playbook steps are executed serially by all hosts. The steps in the included .yml file should happen before any subsequent steps in the devconfig.yml file.
I don't see the install-deps being executed before the playbook checks if the journal client service is up un running.
Quoted 18 lines
Quoted 5 lines
I have other debian/non-debian workflows where the conditional does not conflict with the imported tasks when using ansible_facts['os_family']| lower == 'debian'.
So in previous cases where this came up, Debian was working as expected, but Red Hat (Fedora) was not. This happened in every one of these cases.
Those fixes are far enough in the past that they would appear only in the "historical" kdevops repo.
Quoted 3 lines
And I think this is what we need here. Any idea
what the error might be on your end when usingimport_tasksinstead ofinclude_tasks?
Quoted 5 lines
I can have a look. I don't understand the bug report above well enough to answer your question immediately.
"
import_tasks" has some subtle behaviors -- it seems to be pretty brittle in my experience.
Thanks for checking it!
FYI, I replicated the error in the CI server using the CI workflow. You will see the install-deps are not executed before we reach the point in the playbook where the journal client service is required.
Quoted 62 lines
Quoted 58 lines
Quoted 58 lines
playbooks/roles/devconfig/tasks/install-deps/main.yml +10
-5when: - - ansible_facts['os_family']|lower == 'debian' + - ansible_os_family == "Debian" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool -- import_tasks: suse/main.yml tags: vars_simple + +- name: SuSE-specific setup + ansible.builtin.include_tasks: suse/main.yml when: - - ansible_facts['os_family']|lower == 'suse' + - ansible_os_family == "Suse" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool -- import_tasks: redhat/main.yml tags: vars_simple + +- name: Red Hat-specific setup + ansible.builtin.include_tasks: redhat/main.yml when: - - ansible_facts['os_family']|lower == 'redhat' + - ansible_os_family == "RedHat" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool + tags: vars_simpleplaybooks/roles/devconfig/tasks/main.yml +4
-4@@ -31,10 +31,10 @@ # Distro specific - name: Install dependencies - import_tasks: install-deps/main.yml + ansible.builtin.include_tasks: install-deps/main.yml - name: Configure custom repositories and install packages - import_tasks: config-custom-repos-and-packages/main.yml + ansible.builtin.include_tasks: config-custom-repos-and-packages/main.yml when: - ansible_facts['os_family']|lower == 'redhat'@@ -447,7 +447,7 @@ tags: [ 'console' ] - name: Update your boot GRUB file if necessary - import_tasks: update-grub/main.yml + ansible.builtin.include_tasks: update-grub/main.yml when: - grub2_config_file.stat.exists - devconfig_enable_console|bool@@ -637,7 +637,7 @@ tags: [ 'sysctl' ] - name: Rev the kernel to the latest distribution kotd - import_tasks: kotd-rev-kernel/main.yml + ansible.builtin.include_tasks: kotd-rev-kernel/main.yml when: - devconfig_enable_kotd|bool tags: [ 'kotd' ]Signature · 1 line
2.47.1Signature · 1 line
Chuck Lever
Re: [PATCH v1 5/9] devconfig: Replace import_tasks with include_tasks
Quoted 52 lines · Daniel Gomez
Quoted 51 lines · Chuck Lever
Quoted 49 lines · Daniel Gomez
Hi Chuck,
Quoted 29 lines · On Tue, Jan 14, 2025 at 03:03:44PM -0500, [email protected] wrote:
From: Chuck Lever <[email protected]>
On a fresh Fedora 40 system, under buildbot, the devconfig role fails:
statically imported:
/var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/main.ymlstatically imported:/var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/debian/main.ymlstatically imported:/var/lib/buildbot/worker/renoir1/nfsd-fixes-nfsd-pynfs/build/playbooks/roles/devconfig/tasks/install-deps/suse/main.ymlredirecting (type: modules) ansible.builtin.zypper to community.general.zypper
ERROR! couldn't resolve module/action 'zypper'. This often indicates a misspelling, missing collection, or incorrect module path.It shouldn't care about zypper, that's a Suse thing. Somewhere the
os_familysensing logic has gone hay-wire.Handle this by using dynamic
include_taskinstead of staticimport_task, and restructure theos_familychecking task ininstall-deps/main.yml.Signed-off-by Chuck Lever <[email protected]>--- .../devconfig/tasks/install-deps/main.yml | 22 ++++++++++--------- playbooks/roles/devconfig/tasks/main.yml | 8 +++---- 2 files changed, 16 insertions(+), 14 deletions(-)playbooks/roles/devconfig/tasks/install-deps/main.yml +0
-0playbooks/roles/devconfig/tasks/install-deps/main.yml line 27
Quoted 9 lines
playbooks/roles/devconfig/tasks/install-deps/main.yml +2
-5@@ -24,21 +24,23 @@ skip: true tags: vars -# tasks to setup up repos, register system if needed and install -# preferred devtools packages. -- name: Distribution specific setup - tags: vars_simple - import_tasks: debian/main.yml +- name: Debian-specific setup + ansible.builtin.include_tasks: debian/main.ymlReplacing
import_tasksin the ansibleinclude_tasksmodule is making the kdevops CI fail as reported here [1].https://lore.kernel.org/kdevops/[email protected]/T/#t
The CI failure indicates the tasks inside debian yml are not actually executed before the rest of the playbook.
Quoted 6 lines
Quoted 3 lines
I don't understand. The playbook steps are executed serially by all hosts. The steps in the included .yml file should happen before any subsequent steps in the devconfig.yml file.
I don't see the install-deps being executed before the playbook checks if the journal client service is up un running.
Could this be a "tags" problem?
Quoted 90 lines
Quoted 16 lines
Quoted 3 lines
I have other debian/non-debian workflows where the conditional does not conflict with the imported tasks when using ansible_facts['os_family']| lower == 'debian'.
So in previous cases where this came up, Debian was working as expected, but Red Hat (Fedora) was not. This happened in every one of these cases.
Those fixes are far enough in the past that they would appear only in the "historical" kdevops repo.
Quoted 3 lines
And I think this is what we need here. Any idea
what the error might be on your end when usingimport_tasksinstead ofinclude_tasks?Quoted 5 lines
I can have a look. I don't understand the bug report above well enough to answer your question immediately.
"
import_tasks" has some subtle behaviors -- it seems to be pretty brittle in my experience.Thanks for checking it!
FYI, I replicated the error in the CI server using the CI workflow. You will see the install-deps are not executed before we reach the point in the playbook where the journal client service is required.
Quoted 62 lines
Quoted 58 lines
Quoted 58 lines
playbooks/roles/devconfig/tasks/install-deps/main.yml +10
-5when: - - ansible_facts['os_family']|lower == 'debian' + - ansible_os_family == "Debian" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool -- import_tasks: suse/main.yml tags: vars_simple + +- name: SuSE-specific setup + ansible.builtin.include_tasks: suse/main.yml when: - - ansible_facts['os_family']|lower == 'suse' + - ansible_os_family == "Suse" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool -- import_tasks: redhat/main.yml tags: vars_simple + +- name: Red Hat-specific setup + ansible.builtin.include_tasks: redhat/main.yml when: - - ansible_facts['os_family']|lower == 'redhat' + - ansible_os_family == "RedHat" - devconfig_try_refresh_repos|bool or devconfig_try_install_kdevtools|bool or kdevops_cli_install|bool + tags: vars_simpleplaybooks/roles/devconfig/tasks/main.yml +4
-4@@ -31,10 +31,10 @@ # Distro specific - name: Install dependencies - import_tasks: install-deps/main.yml + ansible.builtin.include_tasks: install-deps/main.yml - name: Configure custom repositories and install packages - import_tasks: config-custom-repos-and-packages/main.yml + ansible.builtin.include_tasks: config-custom-repos-and-packages/main.yml when: - ansible_facts['os_family']|lower == 'redhat'@@ -447,7 +447,7 @@ tags: [ 'console' ] - name: Update your boot GRUB file if necessary - import_tasks: update-grub/main.yml + ansible.builtin.include_tasks: update-grub/main.yml when: - grub2_config_file.stat.exists - devconfig_enable_console|bool@@ -637,7 +637,7 @@ tags: [ 'sysctl' ] - name: Rev the kernel to the latest distribution kotd - import_tasks: kotd-rev-kernel/main.yml + ansible.builtin.include_tasks: kotd-rev-kernel/main.yml when: - devconfig_enable_kotd|bool tags: [ 'kotd' ]Signature · 1 line
2.47.1Signature · 1 line
Chuck Lever
Signature · 1 line
Chuck Lever
PATCH 6/9 terraform/aws: Enable selection of a Fedora (40) AMI
In reply to [email protected]From: Chuck Lever <[email protected]>
For the RedHat-o-philes, add an option to allow selection of a flavor of Fedora for the target nodes.
--- terraform/aws/Kconfig | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)
terraform/aws/Kconfig +17 -2
@@ -124,6 +124,18 @@ config TERRAFORM_AWS_AMI_AMAZON_ARM64 help This option will set AWS AMI owner to Amazon EC2 images for arm64. +config TERRAFORM_AWS_FEDORA40_X86_64 + bool "Fedora 40 on x86_64" + depends on TARGET_ARCH_X86_64 + help + This option sets the AWS AMI owner to Fedora 40 for x86_64. + +config TERRAFORM_AWS_FEDORA40_ARM64 + bool "Fedora 40 on aarch64" + depends on TARGET_ARCH_ARM64 + help + This option sets the AWS AMI owner to Fedora 40 for ARM 64-bit. + config TERRAFORM_AWS_RHEL9_X86_64 bool "RHEL9 x86_64 - ami-0dda7e535b65b6469" depends on TARGET_ARCH_X86_64 && TERRAFORM_AWS_US_WEST_2
@@ -137,7 +149,7 @@ config TERRAFORM_AWS_RHEL9_X86_64_EAST This option will set AWS AMI owner to RHEL9 for x86_64. config TERRAFORM_AWS_RHEL9_ARM64 - bool "RHEL9 x86_64 - ami-08911268ee09cb08e" + bool "RHEL9 ARM 64 - ami-08911268ee09cb08e" depends on TARGET_ARCH_ARM64 && TERRAFORM_AWS_US_WEST_2 help This option will set AWS AMI owner to RHEL9 for ARM64
@@ -224,8 +236,9 @@ config TERRAFORM_AWS_AMI_OWNER default "137112412989" if TERRAFORM_AWS_AMI_AMAZON_X86_64 || TERRAFORM_AWS_AMI_AMAZON_ARM64 default "309956199498" if TERRAFORM_AWS_RHEL9_X86_64 || TERRAFORM_AWS_RHEL9_ARM64 || TERRAFORM_AWS_RHEL9_X86_64_EAST default "013907871322" if TERRAFORM_AWS_SLE15_SP4_X86_64 || TERRAFORM_AWS_SLE15_SP4_ARM64 + default "125523088429" if TERRAFORM_AWS_FEDORA40_X86_64 || TERRAFORM_AWS_FEDORA40_ARM64 help - This option will set AWS AMI owner. + This option specifies the AWS AMI owner. config TERRAFORM_AWS_NS string
@@ -239,6 +252,8 @@ config TERRAFORM_AWS_NS default "al2023-*-arm64" if TARGET_ARCH_ARM64 && TERRAFORM_AWS_AMI_AMAZON_ARM64 default "RHEL-9.*x86_64-*" if TERRAFORM_AWS_RHEL9_X86_64 || TERRAFORM_AWS_RHEL9_X86_64_EAST default "RHEL-9.*arm64-*" if TARGET_ARCH_ARM64 && TERRAFORM_AWS_RHEL9_ARM64 + default "Fedora-Cloud-Base-AmazonEC2.x86_64-40-*" if TERRAFORM_AWS_FEDORA40_X86_64 + default "Fedora-Cloud-Base-AmazonEC2.aarch64-40-*" if TERRAFORM_AWS_FEDORA40_ARM64 default "suse-sles-15-sp4-*x86_64" if TARGET_ARCH_X86_64 && TERRAFORM_AWS_SLE15_SP4_X86_64 default "suse-sles-15-sp4-*arm64" if TARGET_ARCH_ARM64 && TERRAFORM_AWS_SLE15_SP4_ARM64 help
Signature · 1 line
2.47.1
PATCH 7/9 terraform/aws: Add a couple more instance types
In reply to [email protected]From: Chuck Lever <[email protected]>
The existing m5ad selections are still more than is needed for some workflows, so introduce a couple of smaller ones.
Update the help text for these types to better reflect their current provisions.
--- kconfigs/workflows/Kconfig.shared | 2 ++ terraform/aws/Kconfig | 44 ++++++++++++++++++++++++++++--- 2 files changed, 42 insertions(+), 4 deletions(-)
kconfigs/workflows/Kconfig.shared +2 -0
@@ -35,6 +35,8 @@ if WORKFLOW_MAKE_CMD_OVERRIDE config WORKFLOW_MAKE_CMD string "The make command to use when building" + default "make -j 2" if TERRAFORM_AWS_INSTANCE_M5AD_LARGE + default "make -j 4" if TERRAFORM_AWS_INSTANCE_M5AD_XLARGE default "make -j 8" if TERRAFORM_AWS_INSTANCE_M5AD_2XLARGE default "make -j 16" if TERRAFORM_AWS_INSTANCE_M5AD_4XLARGE default "make -j 1" if TERRAFORM_AZURE_VM_SIZE_STANDARD_DS1_V2
terraform/aws/Kconfig +40 -4
65 lines not on this page · raw message
Signature · 1 line
2.47.1
PATCH 8/9 terraform/aws: Add support for m7a instance types
In reply to [email protected]From: Chuck Lever <[email protected]>
These are less expensive AMD-based types, recommended by the AWS instance type advisor for CI/CD workloads.
--- terraform/aws/Kconfig | 66 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+)
terraform/aws/Kconfig +66 -0
@@ -342,6 +342,66 @@ config TERRAFORM_AWS_INSTANCE_M5AD_4XLARGE See https://aws.amazon.com/ec2/instance-types/m5/ for details. +config TERRAFORM_AWS_INSTANCE_M7A_MEDIUM + bool "m7a.medium" + depends on TARGET_ARCH_X86_64 + help + 4 GiB RAM, 1 AMD Ryzen vcpu, 1 10 GiB main drive, and up to + 12.5 Gbs network speed. + + Add storage by selecting the TERRAFORM_AWS_ENABLE_EBS_VOLUMES + option and adjusting the size and number of additional devices. + + See https://aws.amazon.com/ec2/instance-types/m7a/ for details. + +config TERRAFORM_AWS_INSTANCE_M7A_LARGE + bool "m7a.large" + depends on TARGET_ARCH_X86_64 + help + 8 GiB RAM, 2 AMD Ryzen vcpus, 1 10 GiB main drive, and up to + 12.5 Gbs network speed. + + Add storage by selecting the TERRAFORM_AWS_ENABLE_EBS_VOLUMES + option and adjusting the size and number of additional devices. + + See https://aws.amazon.com/ec2/instance-types/m7a/ for details. + +config TERRAFORM_AWS_INSTANCE_M7A_XLARGE + bool "m7a.xlarge" + depends on TARGET_ARCH_X86_64 + help + 16 GiB RAM, 4 AMD Ryzen vcpus, 1 10 GiB main drive, and up to + 12.5 Gbs network speed. + + Add storage by selecting the TERRAFORM_AWS_ENABLE_EBS_VOLUMES + option and adjusting the size and number of additional devices. + + See https://aws.amazon.com/ec2/instance-types/m7a/ for details. + +config TERRAFORM_AWS_INSTANCE_M7A_2XLARGE + bool "m7a.2xlarge" + depends on TARGET_ARCH_X86_64 + help + 32 GiB RAM, 8 AMD Ryzen vcpus, 1 10 GiB main drive, and up to + 12.5 Gbs network speed. + + Add storage by selecting the TERRAFORM_AWS_ENABLE_EBS_VOLUMES + option and adjusting the size and number of additional devices. + + See https://aws.amazon.com/ec2/instance-types/m7a/ for details. + +config TERRAFORM_AWS_INSTANCE_M7A_4XLARGE + bool "m7a.4xlarge" + depends on TARGET_ARCH_X86_64 + help + 64 GiB RAM, 16 AMD Ryzen vcpus, 1 10 GiB main drive, and up to + 12.5 Gbs network speed. + + Add storage by selecting the TERRAFORM_AWS_ENABLE_EBS_VOLUMES + option and adjusting the size and number of additional devices. + + See https://aws.amazon.com/ec2/instance-types/m7a/ for details. + config TERRAFORM_AWS_INSTANCE_IS4GEN_MEDIUM bool "is4gen.medium" depends on TARGET_ARCH_ARM64
@@ -414,6 +474,12 @@ config TERRAFORM_AWS_INSTANCE_TYPE default "m5ad.xlarge" if TERRAFORM_AWS_INSTANCE_M5AD_XLARGE default "m5ad.2xlarge" if TERRAFORM_AWS_INSTANCE_M5AD_2XLARGE default "m5ad.4xlarge" if TERRAFORM_AWS_INSTANCE_M5AD_4XLARGE + default "m7a.medium" if TERRAFORM_AWS_INSTANCE_M7A_MEDIUM + default "m7a.large" if TERRAFORM_AWS_INSTANCE_M7A_LARGE + default "m7a.xlarge" if TERRAFORM_AWS_INSTANCE_M7A_XLARGE + default "m7a.xlarge" if TERRAFORM_AWS_INSTANCE_M7A_XLARGE + default "m7a.2xlarge" if TERRAFORM_AWS_INSTANCE_M7A_2XLARGE + default "m7a.4xlarge" if TERRAFORM_AWS_INSTANCE_M7A_4XLARGE default "is4gen.medium" if TERRAFORM_AWS_INSTANCE_IS4GEN_MEDIUM default "is4gen.8xlarge" if TERRAFORM_AWS_INSTANCE_IS4GEN_8XLARGE default "im4gn.4xlarge" if TERRAFORM_AWS_INSTANCE_IM4GN_4XLARGE
Signature · 1 line
2.47.1
PATCH 9/9 terraform: Hoist aws_profile into the terraform Kconfig
In reply to [email protected]From: Chuck Lever <[email protected]>
Enable automation to select among several AWS profiles by storing the AWS profile name in the kdevops Kconfig.
This allows each test runner to use a distinct IAM user. This is a way to parallelize the workflows.
--- docs/kdevops-terraform.md | 8 ++++---- playbooks/roles/gen_tfvars/defaults/main.yml | 1 + .../templates/aws/terraform.tfvars.j2 | 1 + scripts/terraform.Makefile | 1 + terraform/aws/Kconfig | 18 ++++++++++++++++++ 5 files changed, 25 insertions(+), 4 deletions(-)
docs/kdevops-terraform.md +4 -4
@@ -238,10 +238,10 @@ aws_access_key_id = SOME_ACCESS_KEY aws_secret_access_key = SECRET_KEY ``` -The profile above is "default", and you can multiple profiles. By default -our Terraform's AWS vars.tf assumes ~/.aws/credentials as the default -credentials location, and the profile as "default". If this is different -for you, you can override with the variables: +The profile above is "default", and you can have multiple profiles. By +default our Terraform's AWS vars.tf assumes ~/.aws/credentials as the +default credentials location, and the profile as "default". If this is +different for you, you can override with the variables: ``` aws_shared_credentials_file
playbooks/roles/gen_tfvars/defaults/main.yml +1 -0
@@ -17,6 +17,7 @@ terraform_private_net_enabled: "false" terraform_private_net_prefix: "" terraform_private_net_mask: 0 +terraform_aws_profile: "default" terraform_aws_region: "invalid" terraform_aws_av_region: "invalid" terraform_aws_ami_owner: "invalid"
playbooks/roles/gen_tfvars/templates/aws/terraform.tfvars.j2 +1 -0
@@ -1,3 +1,4 @@ +aws_profile = "{{ terraform_aws_profile }}" aws_region = "{{ terraform_aws_region }}" aws_availability_region = "{{ terraform_aws_av_region }}" aws_name_search = "{{ terraform_aws_ns }}"
scripts/terraform.Makefile +1 -0
@@ -46,6 +46,7 @@ KDEVOPS_MRPROPER += $(KDEVOPS_NODES) DEFAULT_DEPS_REQS_EXTRA_VARS += $(KDEVOPS_TFVARS) ifeq (y,$(CONFIG_TERRAFORM_AWS)) +TERRAFORM_EXTRA_VARS += terraform_aws_profile=$(subst ",,$(CONFIG_TERRAFORM_AWS_PROFILE)) TERRAFORM_EXTRA_VARS += terraform_aws_region=$(subst ",,$(CONFIG_TERRAFORM_AWS_REGION)) TERRAFORM_EXTRA_VARS += terraform_aws_av_region=$(subst ",,$(CONFIG_TERRAFORM_AWS_AV_REGION)) TERRAFORM_EXTRA_VARS += terraform_aws_ami_owner=$(subst ",,$(CONFIG_TERRAFORM_AWS_AMI_OWNER))
terraform/aws/Kconfig +18 -0
@@ -1,5 +1,23 @@ if TERRAFORM_AWS +config TERRAFORM_AWS_PROFILE + prompt "AWS profile to use" + string + default "default" + help + This option sets the value of the Terraform aws_profile + variable. The AWS profile specifies an IAM user and secret + key that Terraform uses to authenticate to AWS. + + AWS profiles are set up in ~/.aws/credentials using the + "aws configure" command. You may also edit this file by + hand, if you're daring. + + The default value of this Kconfig option selects the + "default" profile, which is fine when using only a single + AWS profile. Change the value of this option to select an + alternate profile / login credentials. + choice prompt "AWS region to use" default TERRAFORM_AWS_US_WEST_2
Signature · 1 line
2.47.1
Quoted 8 lines · On Tue, Jan 14, 2025 at 03:03:39PM -0500, [email protected] wrote:
From: Chuck Lever <[email protected]>
Here is a set of changes I've been toying with since early December while working on getting AWS hosting to work again. I'm interested in community comments, objections, or other musings before I just merge these. There's no real rocket surgery here, but there could be some controversy.
Quoted 5 lines
Work on the nfsd role's logic to set up an LVM for exports, which was posted here previously, continues slowly. I have something that seems pretty nice for guestfs, but unfortunately it's complicated by latent bugs and old changes in terraform when it comes to AWS. I'm still working on it.
Luis