RFC Build once, test everywhere
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest3 patches 5 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/3 | Add a guest/instance for building the test kernelplaybooks/ playbooks/ playbooks/ +4 more | +65 | 0 replies | |
| 2/3 | playbooks: Add a build_linux role.gitignore playbooks/ playbooks/ +6 more | +599 | 0 replies | |
| 3/3 | Experimental: Add a separate install_linux roleplaybooks/ playbooks/ playbooks/ +10 more | +750 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [RFC PATCH 0/3] Build once, test everywhere
From: Chuck Lever <[email protected]>
With apologies to the Java folks.
I've been looking for ways to make our workflow runs more efficient because my lab resources are limited, and moving to the cloud costs money per CPU second.
One thing that seems like an obvious win would be to build the test kernel one time (for example, after a merge/pull request), then re-use that kernel binary for all the workflows we want to run on it.
For libvirt, the usual way to do this is use the 9p option. There are a few cases that are important to me where that option does not work:
1. The control host is running late-model Fedora, and the test runners want to run old LTS kernels
2. The control host or test runners run a RHEL-based kernel, which does not support 9p, last I checked
3. Cloud, where there is no way for cloud-based test runners to mount the control host with 9p
That leaves me with building the test kernel in each test runner. That's a lot of kernel builds; and in the cloud, the kernel source code has to be cloned onto each runner from kernel.org, because we don't have repo mirroring there.
Ted pointed out another use case where build-once is great: when you want to under-provision the test runners to exercise code paths that handle resource exhaustion. Building the kernel on such test runners takes forever.
Cloud providers throw another curve: there is typically a per- tenant or per-availability zone limit on the number of CPU cores you can provision at once. So a workflow that starts several instances has to limit each instance to only one or two cores. Building the kernel on those instances also takes forever.
I really want something more like KOTD, where test kernels are built once, packaged, and placed somewhere that the runners can find and install them. I'm not real interested in setting up a persistent yum repo and builder, though; I'd like kdevops to manage all that for me.
So I got out my whittling knife and built this proof of concept where kdevops brings up a kernel builder node that is tailored for a fast kernel build (eg, it is one guest with 16 vCPUs). The test kernel is packaged (.rpm or .deb) and fetched to the control host, then the builder node is destroyed.
I added a top-level "make" target that uploads the test kernel packages to each test runner, and they install it.
This is not part of the patch series, but shows how to run it:
$ make mrproper defconfig-kernel-builder $ make && make bringup && make linux-packages && make destroy $ make defconfig-workflow-one $ make && make bringup && make linux-artifacts && make destroy $ make defconfig-workflow-two $ make && make bringup && make linux-artifacts && make destroy $ make defconfig-workflow-three $ make && make bringup && make linux-artifacts && make destroy $ make mrproper
All three workflows use the same kernel, built just once, for all of their test runners.
This is still band-aids and chewing gum, but it seems to work on both libvirt and cloud configurations. There is more than one way to skin this cat, though.
Chuck Lever (3):
Add a guest/instance for building the test kernel
playbooks: Add a build_linux role
Experimental: Add a separate install_linux role
.gitignore | 2 + playbooks/build_linux.yml | 4 + playbooks/install_linux.yml | 4 + playbooks/roles/build_linux/README.md | 74 +++++ playbooks/roles/build_linux/defaults/main.yml | 38 +++ .../tasks/install-deps/debian/main.yml | 46 +++ .../tasks/install-deps/redhat/main.yml | 102 ++++++ .../tasks/install-deps/suse/main.yml | 31 ++ playbooks/roles/build_linux/tasks/main.yml | 295 ++++++++++++++++++ playbooks/roles/gen_hosts/defaults/main.yml | 2 + playbooks/roles/gen_hosts/tasks/main.yml | 12 + .../roles/gen_hosts/templates/builder.j2 | 13 + playbooks/roles/gen_nodes/defaults/main.yml | 2 + playbooks/roles/gen_nodes/tasks/main.yml | 22 ++ playbooks/roles/install_linux/README.md | 136 ++++++++ .../roles/install_linux/defaults/main.yml | 43 +++ .../tasks/install-deps/debian/main.yml | 44 +++ .../tasks/install-deps/redhat/main.yml | 76 +++++ .../tasks/install-deps/suse/main.yml | 31 ++ playbooks/roles/install_linux/tasks/main.yml | 142 +++++++++ .../tasks/update-grub/debian.yml | 8 + .../tasks/update-grub/install.yml | 196 ++++++++++++ .../install_linux/tasks/update-grub/main.yml | 15 + .../tasks/update-grub/redhat.yml | 36 +++ .../install_linux/tasks/update-grub/suse.yml | 11 + workflows/linux/Kconfig | 10 + workflows/linux/Makefile | 19 ++ 27 files changed, 1414 insertions(+) create mode 100644 playbooks/build_linux.yml create mode 100644 playbooks/install_linux.yml create mode 100644 playbooks/roles/build_linux/README.md create mode 100644 playbooks/roles/build_linux/defaults/main.yml create mode 100644 playbooks/roles/build_linux/tasks/install-deps/debian/main.yml create mode 100644 playbooks/roles/build_linux/tasks/install-deps/redhat/main.yml create mode 100644 playbooks/roles/build_linux/tasks/install-deps/suse/main.yml create mode 100644 playbooks/roles/build_linux/tasks/main.yml create mode 100644 playbooks/roles/gen_hosts/templates/builder.j2 create mode 100644 playbooks/roles/install_linux/README.md create mode 100644 playbooks/roles/install_linux/defaults/main.yml create mode 100644 playbooks/roles/install_linux/tasks/install-deps/debian/main.yml create mode 100644 playbooks/roles/install_linux/tasks/install-deps/redhat/main.yml create mode 100644 playbooks/roles/install_linux/tasks/install-deps/suse/main.yml create mode 100644 playbooks/roles/install_linux/tasks/main.yml create mode 100644 playbooks/roles/install_linux/tasks/update-grub/debian.yml create mode 100644 playbooks/roles/install_linux/tasks/update-grub/install.yml create mode 100644 playbooks/roles/install_linux/tasks/update-grub/main.yml create mode 100644 playbooks/roles/install_linux/tasks/update-grub/redhat.yml create mode 100644 playbooks/roles/install_linux/tasks/update-grub/suse.yml
Signature · 1 line
2.49.0