Skip to contentSkip to conversation

kdevops: add support for A/B testing

PATCH In review Luis Chamberlain 1 participant 13 messages Aug 1, 2025 – Aug 2, 2025

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v3 cover kdevops: add support for A/B testing

This v2 just expands the debian-testing apt sources with deb822 support, we can now infer and detect both formats and prefer deb822 when possible.

Luis Chamberlain (11):
roles/guestfs: add missing bootlinux_9p: False
Makefile: suppress Ansible warnings during configuration generation
playbooks: few space cleanups
style: add extensive code formatting checks to make style
Makefile: move styling to scripts/style.Makefile CLAUDE.md: add instrucitons to verify commit
all: run black
scripts: enhance hop count detection to support DEB822 format
devconfig: add automatic APT mirror fallback with DEB822 modernization
devconfig: enhance hop1 detection to support traditional sources.list
bootlinux: add support for A/B kernel testing

 .github/workflows/linux-ab.yml                |  47 ++
 CLAUDE.md                                     |  10 +
 Makefile                                      |  18 +-
 PROMPTS.md                                    |  52 +++
 defconfigs/linux-ab-testing                   |  14 +
 defconfigs/linux-ab-testing-9p                |  15 +
 defconfigs/linux-ab-testing-builder           |  15 +
 defconfigs/linux-ab-testing-target            |  15 +
 docs/kdevops-make-linux.md                    | 158 +++++++
 playbooks/mmtests.yml                         |   2 +-
 .../blktests/augment_expunge_list.py          |  95 ++--
 .../workflows/blktests/gen-expunge-args.py    |  46 +-
 .../workflows/blktests/gen-results-dir.py     |  39 +-
 .../blktests/get_new_expunge_files.py         |  16 +-
 .../dynamic-kconfig/gen-dynamic-pci.py        |  89 ++--
 .../workflows/fstests/augment_expunge_list.py | 163 +++++--
 .../workflows/fstests/bad_files_summary.py    |  46 +-
 .../fstests/fstests-checktime-distribution.py |  46 +-
 .../workflows/fstests/gen_results_summary.py  | 132 +++---
 .../fstests/get_new_expunge_files.py          |  24 +-
 playbooks/python/workflows/fstests/lib/git.py |  21 +-
 .../workflows/fstests/xunit_merge_all.py      |  33 +-
 .../sysbench/sysbench-tps-compare.py          |  84 +++-
 .../workflows/sysbench/sysbench-tps-plot.py   |  36 +-
 .../sysbench/sysbench-tps-variance.py         | 435 +++++++++++++-----
 playbooks/roles/bootlinux/defaults/main.yml   |  14 +
 playbooks/roles/bootlinux/tasks/build/9p.yml  |  20 +-
 .../install-minimal-deps/debian/main.yml      |   2 +-
 .../tasks/install-minimal-deps/main.yml       |   2 +-
 .../install-minimal-deps/redhat/main.yml      |   2 +-
 .../tasks/install-minimal-deps/suse/main.yml  |   2 +-
 playbooks/roles/bootlinux/tasks/main.yml      | 112 +++++
 .../devconfig/tasks/check-apt-mirrors.yml     | 213 +++++++++
 playbooks/roles/devconfig/tasks/main.yml      |   8 +
 .../templates/debian-hop1-mirror.sources      |  13 +
 .../templates/debian-testing-fallback.sources |  13 +
 .../gen_pcie_passthrough_guestfs_xml.py       |  49 +-
 playbooks/roles/guestfs/defaults/main.yml     |   1 +
 .../linux-mirror/python/gen-mirror-files.py   | 131 +++---
 .../linux-mirror/python/start-mirroring.py    | 116 +++--
 .../roles/mmtests/tasks/install-deps/main.yml |   2 +-
 scripts/check_commit_format.py                |  28 +-
 .../generation/check_for_atomic_calls.py      |  71 +--
 .../generation/check_for_sleepy_calls.py      | 202 +++++---
 scripts/detect_indentation_issues.py          | 163 +++++++
 scripts/detect_whitespace_issues.py           |  38 +-
 scripts/ensure_newlines.py                    |  75 +++
 scripts/fix_indentation_issues.py             | 152 ++++++
 scripts/fix_whitespace_issues.py              |  44 +-
 scripts/generate_refs.py                      |   6 +-
 scripts/get-distro-has-hop-count-sources.sh   |  29 +-
 scripts/honey-badger.py                       | 103 +++--
 scripts/infer_last_stable_kernel.sh           |  35 ++
 scripts/linux-ab-testing.Makefile             |  51 ++
 scripts/spdxcheck.py                          | 201 ++++----
 scripts/style.Makefile                        |  11 +
 scripts/test-linux-ab-config.py               | 182 ++++++++
 scripts/test-linux-ab.sh                      | 213 +++++++++
 scripts/update_ssh_config_guestfs.py          |  49 +-
 .../workflows/blktests/blktests_watchdog.py   |  75 ++-
 scripts/workflows/cxl/gen_qemu_cxl.py         | 235 +++++++---
 scripts/workflows/fstests/fstests_watchdog.py |  99 ++--
 scripts/workflows/generic/crash_report.py     |   4 +-
 scripts/workflows/generic/crash_watchdog.py   |  78 +++-
 scripts/workflows/lib/blktests.py             |  47 +-
 scripts/workflows/lib/crash.py                |  12 +-
 scripts/workflows/lib/fstests.py              | 155 ++++---
 scripts/workflows/lib/kssh.py                 | 178 ++++---
 scripts/workflows/lib/systemd_remote.py       | 101 ++--
 .../workflows/pynfs/check_pynfs_results.py    |  17 +-
 workflows/linux/Kconfig                       | 102 +++-
 workflows/linux/Makefile                      |  39 ++
 72 files changed, 4022 insertions(+), 1124 deletions(-)
 create mode 100644 .github/workflows/linux-ab.yml
 create mode 100644 defconfigs/linux-ab-testing
 create mode 100644 defconfigs/linux-ab-testing-9p
 create mode 100644 defconfigs/linux-ab-testing-builder
 create mode 100644 defconfigs/linux-ab-testing-target
 create mode 100644 playbooks/roles/devconfig/tasks/check-apt-mirrors.yml
 create mode 100644 playbooks/roles/devconfig/templates/debian-hop1-mirror.sources
 create mode 100644 playbooks/roles/devconfig/templates/debian-testing-fallback.sources
 create mode 100755 scripts/detect_indentation_issues.py
 create mode 100755 scripts/ensure_newlines.py
 create mode 100755 scripts/fix_indentation_issues.py
 create mode 100755 scripts/infer_last_stable_kernel.sh
 create mode 100644 scripts/linux-ab-testing.Makefile
 create mode 100644 scripts/style.Makefile
 create mode 100755 scripts/test-linux-ab-config.py
 create mode 100755 scripts/test-linux-ab.sh
Signature · 1 line
2.47.2
#1 lore raw

PATCH v3 1/11 roles/guestfs: add missing bootlinux_9p: False

Commit 722d83a4871c41 ("bootlinux: Move 9p build tasks to a subrole") added a proactive directory creation as we no longer git clone right away, but forgot to ensure the variable is defined by default. When we don't enable building linux this variable is not defined. Fix this.

Fixes 722d83a4871c41 ("bootlinux: Move 9p build tasks to a subrole")
---
 playbooks/roles/guestfs/defaults/main.yml | 1 +
 1 file changed, 1 insertion(+)
playbooks/roles/guestfs/defaults/main.yml +1 -0
index eec137bd730a..76854d06582f 100644
@@ -3,3 +3,4 @@ distro_debian_based: false
 
 libvirt_uri_system: false
 libvirt_enable_largeio: false
+bootlinux_9p: False
Signature · 1 line
2.47.2
#2 lore raw

PATCH v3 2/11 Makefile: suppress Ansible warnings during configuration generation

In reply to Luis Chamberlain

The initial configuration generation playbooks (ansible_cfg.yml, gen_hosts.yml, and gen_nodes.yml) run without a proper inventory file by design, as they are responsible for creating the configuration files that will be used by subsequent playbooks.

This causes Ansible to emit warnings about:
- "No inventory was parsed, only implicit localhost is available"
- "provided hosts list is empty, only localhost is available"

These warnings are harmless but create noise in the build output, potentially confusing users who might think something is wrong.

Add ANSIBLE_LOCALHOST_WARNING=False and ANSIBLE_INVENTORY_UNPARSED_WARNING=False environment variables to these three ansible-playbook invocations to suppress these specific warnings. This makes the build output cleaner while maintaining the same functionality.

The warnings were appearing because:
1. ansible_cfg.yml creates the ansible.cfg file
2. gen_hosts.yml creates the hosts inventory file
3. gen_nodes.yml creates the kdevops_nodes.yaml file

All three intentionally run with "--connection=local" and minimal inventory since they're bootstrapping the configuration that other playbooks will use.

Generated-by Claude AI
---
 Makefile | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
Makefile +6 -3
index 37c2522be8be..31f544e9dd9c 100644
@@ -195,7 +195,8 @@ include scripts/gen-nodes.Makefile
 	false)
 
 $(ANSIBLE_CFG_FILE): .config
-	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
+	$(Q)ANSIBLE_LOCALHOST_WARNING=False ANSIBLE_INVENTORY_UNPARSED_WARNING=False \
+		ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
 		--inventory localhost, \
 		$(KDEVOPS_PLAYBOOKS_DIR)/ansible_cfg.yml \
 		--extra-vars=@./.extra_vars_auto.yaml
@@ -226,13 +227,15 @@ endif
 
 DEFAULT_DEPS += $(ANSIBLE_INVENTORY_FILE)
 $(ANSIBLE_INVENTORY_FILE): .config $(ANSIBLE_CFG_FILE) $(KDEVOPS_HOSTS_TEMPLATE)
-	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
+	$(Q)ANSIBLE_LOCALHOST_WARNING=False ANSIBLE_INVENTORY_UNPARSED_WARNING=False \
+		ansible-playbook $(ANSIBLE_VERBOSE) \
 		$(KDEVOPS_PLAYBOOKS_DIR)/gen_hosts.yml \
 		--extra-vars=@./extra_vars.yaml
 
 DEFAULT_DEPS += $(KDEVOPS_NODES)
 $(KDEVOPS_NODES): .config $(ANSIBLE_CFG_FILE) $(KDEVOPS_NODES_TEMPLATE)
-	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
+	$(Q)ANSIBLE_LOCALHOST_WARNING=False ANSIBLE_INVENTORY_UNPARSED_WARNING=False \
+		ansible-playbook $(ANSIBLE_VERBOSE) --connection=local \
 		--inventory localhost, \
 		$(KDEVOPS_PLAYBOOKS_DIR)/gen_nodes.yml \
 		--extra-vars=@./extra_vars.yaml
Signature · 1 line
2.47.2
#3 lore raw

PATCH v3 3/11 playbooks: few space cleanups

In reply to Luis Chamberlain

Since we're going to be using bots we sometimes forget to clean up after them. Fix a few minor spacing issues left behind from prior commits.

---
 Makefile                                                        | 2 +-
 playbooks/mmtests.yml                                           | 2 +-
 .../roles/bootlinux/tasks/install-minimal-deps/debian/main.yml  | 2 +-
 playbooks/roles/bootlinux/tasks/install-minimal-deps/main.yml   | 2 +-
 .../roles/bootlinux/tasks/install-minimal-deps/redhat/main.yml  | 2 +-
 .../roles/bootlinux/tasks/install-minimal-deps/suse/main.yml    | 2 +-
 playbooks/roles/mmtests/tasks/install-deps/main.yml             | 2 +-
 7 files changed, 7 insertions(+), 7 deletions(-)
Makefile +1 -1
index 31f544e9dd9c..a8b58eb7cb88 100644
@@ -122,7 +122,7 @@ include scripts/krb5.Makefile
 include scripts/devconfig.Makefile
 include scripts/ssh.Makefile
 
-ANSIBLE_CMD_KOTD_ENABLE := echo KOTD disabled so not running: 
+ANSIBLE_CMD_KOTD_ENABLE := echo KOTD disabled so not running:
 ifeq (y,$(CONFIG_WORKFLOW_KOTD_ENABLE))
 include scripts/kotd.Makefile
 endif # WORKFLOW_KOTD_ENABLE
playbooks/mmtests.yml +1 -1
index d341d246871a..f66e65db8b41 100644
@@ -1,4 +1,4 @@
 ---
 - hosts: all
   roles:
-    - role: mmtests
\ No newline at end of file
+    - role: mmtests
playbooks/roles/bootlinux/tasks/install-minimal-deps/debian/main.yml +1 -1
index 9fd6834f13fd..21d33331dbc2 100644
@@ -16,4 +16,4 @@
       - make
       - gcc
       - kmod
-    state: present
\ No newline at end of file
+    state: present
playbooks/roles/bootlinux/tasks/install-minimal-deps/main.yml +1 -1
index d1e9f6757785..75dbd9a46df2 100644
@@ -12,4 +12,4 @@
 - name: Red Hat-specific minimal setup
   ansible.builtin.import_tasks: redhat/main.yml
   when:
-    - ansible_os_family == "RedHat"
\ No newline at end of file
+    - ansible_os_family == "RedHat"
playbooks/roles/bootlinux/tasks/install-minimal-deps/redhat/main.yml +1 -1
index c5cd6fbf6241..e077f5e88a2f 100644
@@ -24,4 +24,4 @@
       - kmod
     state: present
   when:
-    - ansible_facts['distribution_major_version']|int >= 8
\ No newline at end of file
+    - ansible_facts['distribution_major_version']|int >= 8
playbooks/roles/bootlinux/tasks/install-minimal-deps/suse/main.yml +1 -1
index a17768f91c27..ce73ddd18de8 100644
@@ -10,4 +10,4 @@
       - make
       - gcc
       - kmod-compat
-    state: present
\ No newline at end of file
+    state: present
playbooks/roles/mmtests/tasks/install-deps/main.yml +1 -1
index d3972fae51d4..3ecd69825b8e 100644
@@ -13,4 +13,4 @@
 
 - name: RedHat distribution specific setup
   import_tasks: tasks/install-deps/redhat/main.yml
-  when: ansible_facts['os_family']|lower == 'redhat'
\ No newline at end of file
+  when: ansible_facts['os_family']|lower == 'redhat'
Signature · 1 line
2.47.2
#4 lore raw

PATCH v3 4/11 style: add extensive code formatting checks to make style

In reply to Luis Chamberlain

Extend the 'make style' target with comprehensive code formatting and style checks:

1. Add black formatter for Python files - runs 'black .' to check and format all Python files in the repository

2. Add indentation detection for mixed tabs/spaces:
- detect_indentation_issues.py: Detects incorrect indentation based on file type (YAML requires spaces, Makefiles need tabs for recipes, Python prefers spaces)
- fix_indentation_issues.py: Automatically fixes indentation issues

3. Keep existing checks:
- Whitespace issue detection (trailing spaces, missing newlines)
- Commit message format validation
- Ensure all text files end with newlines

The style checks are ordered to run black first as it provides the most comprehensive Python formatting validation. Indentation and whitespace checks default to checking only modified files to avoid being overwhelmed by pre-existing issues.

Generated-by Claude AI
---
 Makefile                             |  11 +-
 scripts/detect_indentation_issues.py | 163 +++++++++++++++++++++++++++
 scripts/ensure_newlines.py           |  75 ++++++++++++
 scripts/fix_indentation_issues.py    | 152 +++++++++++++++++++++++++
 4 files changed, 399 insertions(+), 2 deletions(-)
 create mode 100755 scripts/detect_indentation_issues.py
 create mode 100755 scripts/ensure_newlines.py
 create mode 100755 scripts/fix_indentation_issues.py
Makefile +9 -2
index a8b58eb7cb88..2c72042d57c8 100644
@@ -247,10 +247,17 @@ include scripts/ci.Makefile
 include scripts/archive.Makefile
 include scripts/defconfig.Makefile
 
+PHONY += fix-whitespace-last-commit
+fix-whitespace-last-commit:
+	$(Q)git diff --name-only --diff-filter=M HEAD~1..HEAD | xargs -r python3 scripts/fix_whitespace_issues.py
+
 PHONY += style
 style:
-	$(Q)python3 scripts/detect_whitespace_issues.py
-	$(Q)python3 scripts/check_commit_format.py
+	$(Q)if which black > /dev/null ; then black . || true; fi
+	$(Q)python3 scripts/detect_whitespace_issues.py || true
+	$(Q)python3 scripts/detect_indentation_issues.py || true
+	$(Q)python3 scripts/check_commit_format.py || true
+	$(Q)python3 scripts/ensure_newlines.py || true
 
 PHONY += clean
 clean:
scripts/detect_indentation_issues.py new file +163 -0

163 lines not on this page · raw message

scripts/ensure_newlines.py new file +75 -0

75 lines not on this page · raw message

scripts/fix_indentation_issues.py new file +152 -0

152 lines not on this page · raw message

Signature · 1 line
2.47.2
#5 lore raw

PATCH v3 5/11 Makefile: move styling to scripts/style.Makefile

In reply to Luis Chamberlain

Monitoring styling enhancements turn out to be its own thing, move this effort to its own file to keep the top level Makefile tidy.

---
 Makefile               | 13 +------------
 scripts/style.Makefile | 11 +++++++++++
 2 files changed, 12 insertions(+), 12 deletions(-)
 create mode 100644 scripts/style.Makefile
Makefile +1 -12
index 2c72042d57c8..c88637c2966f 100644
@@ -246,18 +246,7 @@ include scripts/tests.Makefile
 include scripts/ci.Makefile
 include scripts/archive.Makefile
 include scripts/defconfig.Makefile
-
-PHONY += fix-whitespace-last-commit
-fix-whitespace-last-commit:
-	$(Q)git diff --name-only --diff-filter=M HEAD~1..HEAD | xargs -r python3 scripts/fix_whitespace_issues.py
-
-PHONY += style
-style:
-	$(Q)if which black > /dev/null ; then black . || true; fi
-	$(Q)python3 scripts/detect_whitespace_issues.py || true
-	$(Q)python3 scripts/detect_indentation_issues.py || true
-	$(Q)python3 scripts/check_commit_format.py || true
-	$(Q)python3 scripts/ensure_newlines.py || true
+include scripts/style.Makefile
 
 PHONY += clean
 clean:
scripts/style.Makefile new file +11 -0
new file mode 100644
index 000000000000..37aed9af00b3
@@ -0,0 +1,11 @@
+PHONY += fix-whitespace-last-commit
+fix-whitespace-last-commit:
+	$(Q)git diff --name-only --diff-filter=M HEAD~1..HEAD | xargs -r python3 scripts/fix_whitespace_issues.py
+
+PHONY += style
+style:
+	$(Q)if which black > /dev/null ; then black . || true; fi
+	$(Q)python3 scripts/detect_whitespace_issues.py || true
+	$(Q)python3 scripts/detect_indentation_issues.py || true
+	$(Q)python3 scripts/check_commit_format.py || true
+	$(Q)python3 scripts/ensure_newlines.py || true
Signature · 1 line
2.47.2
#6 lore raw

PATCH v3 6/11 CLAUDE.md: add instrucitons to verify commit

In reply to Luis Chamberlain

White space damange is just a pain with bots, so we need to also give it a separate instruction to verify the commit is not adding new file with space damage.

---
 CLAUDE.md | 10 ++++++++++
 1 file changed, 10 insertions(+)
CLAUDE.md +10 -0
index c52774b513f4..2bcf94431671 100644
@@ -121,6 +121,7 @@ make V=1 [target]       # Verbose build output
 make AV=1-6 [target]    # Ansible verbose output (levels 0-6)
 make dynconfig          # Generate dynamic configuration
 make style              # Check for whitespace issues - ALWAYS run before completing work
+make fix-whitespace-last-commit # Fixes commit white space damage
 make mrproper           # Clean everything and restart from scratch
 ```
 
@@ -428,6 +429,15 @@ The fixer script will:
 
 Always run `make style` after using the fixer to verify all issues are resolved.
 
+### Verifying commit has no white space damage
+
+Run the following after you commit something:
+```bash
+make fix-whitespace-last-commit
+```
+
+This will fix all white space only for new files you add.
+
 ## Complex System Interactions
 
 kdevops integrates multiple subsystems (Ansible, Kconfig, Git, Make) that often
Signature · 1 line
2.47.2
#7 lore raw

PATCH v3 7/11 all: run black

In reply to Luis Chamberlain

Run black to fix tons of styling issues with tons of Python scripts. In order to help bots ensure they don't add odd styling we need a convention.

---
 .../blktests/augment_expunge_list.py          |  95 ++--
 .../workflows/blktests/gen-expunge-args.py    |  46 +-
 .../workflows/blktests/gen-results-dir.py     |  39 +-
 .../blktests/get_new_expunge_files.py         |  16 +-
 .../dynamic-kconfig/gen-dynamic-pci.py        |  89 ++--
 .../workflows/fstests/augment_expunge_list.py | 163 +++++--
 .../workflows/fstests/bad_files_summary.py    |  46 +-
 .../fstests/fstests-checktime-distribution.py |  46 +-
 .../workflows/fstests/gen_results_summary.py  | 132 +++---
 .../fstests/get_new_expunge_files.py          |  24 +-
 playbooks/python/workflows/fstests/lib/git.py |  21 +-
 .../workflows/fstests/xunit_merge_all.py      |  33 +-
 .../sysbench/sysbench-tps-compare.py          |  84 +++-
 .../workflows/sysbench/sysbench-tps-plot.py   |  36 +-
 .../sysbench/sysbench-tps-variance.py         | 435 +++++++++++++-----
 .../gen_pcie_passthrough_guestfs_xml.py       |  49 +-
 .../linux-mirror/python/gen-mirror-files.py   | 131 +++---
 .../linux-mirror/python/start-mirroring.py    | 116 +++--
 scripts/check_commit_format.py                |  28 +-
 .../generation/check_for_atomic_calls.py      |  71 +--
 .../generation/check_for_sleepy_calls.py      | 202 +++++---
 scripts/detect_whitespace_issues.py           |  38 +-
 scripts/fix_whitespace_issues.py              |  44 +-
 scripts/generate_refs.py                      |   6 +-
 scripts/honey-badger.py                       | 103 +++--
 scripts/spdxcheck.py                          | 201 ++++----
 scripts/update_ssh_config_guestfs.py          |  49 +-
 .../workflows/blktests/blktests_watchdog.py   |  75 ++-
 scripts/workflows/cxl/gen_qemu_cxl.py         | 235 +++++++---
 scripts/workflows/fstests/fstests_watchdog.py |  99 ++--
 scripts/workflows/generic/crash_report.py     |   4 +-
 scripts/workflows/generic/crash_watchdog.py   |  78 +++-
 scripts/workflows/lib/blktests.py             |  47 +-
 scripts/workflows/lib/crash.py                |  12 +-
 scripts/workflows/lib/fstests.py              | 155 ++++---
 scripts/workflows/lib/kssh.py                 | 178 ++++---
 scripts/workflows/lib/systemd_remote.py       | 101 ++--
 .../workflows/pynfs/check_pynfs_results.py    |  17 +-
 38 files changed, 2250 insertions(+), 1094 deletions(-)
playbooks/python/workflows/blktests/augment_expunge_list.py +65 -30
index 4fe856a3339f..a7b8eb42c874 100755
@@ -15,58 +15,81 @@ import configparser
 from itertools import chain
 
 oscheck_ansible_python_dir = os.path.dirname(os.path.abspath(__file__))
-oscheck_sort_expunge = oscheck_ansible_python_dir + "/../../../scripts/workflows/fstests/sort-expunges.sh"
+oscheck_sort_expunge = (
+    oscheck_ansible_python_dir + "/../../../scripts/workflows/fstests/sort-expunges.sh"
+)
 top_dir = oscheck_ansible_python_dir + "/../../../../"
-blktests_last_kernel = top_dir + 'workflows/blktests/results/last-kernel.txt'
+blktests_last_kernel = top_dir + "workflows/blktests/results/last-kernel.txt"
 expunge_name = "failures.txt"
 
+
 def append_line(output_file, test_failure_line):
     # We want to now add entries like block/xxx where xxx are digits
     output = open(output_file, "a+")
     output.write("%s\n" % test_failure_line)
     output.close()
 
+
 def is_config_bool_true(config, name):
-    if name in config and config[name].strip('\"') == "y":
+    if name in config and config[name].strip('"') == "y":
         return True
     return False
 
+
 def config_string(config, name):
     if name in config:
-        return config[name].strip('\"')
+        return config[name].strip('"')
     return None
 
+
 def get_config(dotconfig):
-    config = configparser.ConfigParser(allow_no_value=True, strict=False, interpolation=None)
+    config = configparser.ConfigParser(
+        allow_no_value=True, strict=False, interpolation=None
+    )
     with open(dotconfig) as lines:
         lines = chain(("[top]",), lines)
         config.read_file(lines)
         return config["top"]
     return None
 
+
 def read_blktest_last_kernel():
     if not os.path.isfile(blktests_last_kernel):
         return None
-    kfile = open(blktests_last_kernel, 'r')
+    kfile = open(blktests_last_kernel, "r")
     all_lines = kfile.readlines()
     kfile.close()
     for line in all_lines:
         return line.strip()
     return None
 
+
 def main():
-    parser = argparse.ArgumentParser(description='Augments expunge list for blktest')
-    parser.add_argument('results', metavar='<directory with results>', type=str,
-                        help='directory with results file')
-    parser.add_argument('outputdir', metavar='<output directory>', type=str,
-                        help='The directory where to generate the expunge failure.txt ')
-    parser.add_argument('--verbose', const=True, default=False, action="store_const",
-                        help='Print more verbose information')
+    parser = argparse.ArgumentParser(description="Augments expunge list for blktest")
+    parser.add_argument(
+        "results",
+        metavar="<directory with results>",
+        type=str,
+        help="directory with results file",
+    )
+    parser.add_argument(
+        "outputdir",
+        metavar="<output directory>",
+        type=str,
+        help="The directory where to generate the expunge failure.txt ",
+    )
+    parser.add_argument(
+        "--verbose",
+        const=True,
+        default=False,
+        action="store_const",
+        help="Print more verbose information",
+    )
     args = parser.parse_args()
 
     expunge_kernel_dir = ""
 
-    dotconfig = top_dir + '/.config'
+    dotconfig = top_dir + "/.config"
     config = get_config(dotconfig)
     if not config:
         sys.stdout.write("%s does not exist\n" % (dotconfig))
@@ -85,7 +108,7 @@ def main():
                 continue
             if not os.path.isfile(f):
                 continue
-            if not f.endswith('.bad') and not f.endswith('.dmesg'):
+            if not f.endswith(".bad") and not f.endswith(".dmesg"):
                 continue
 
             bad_files.append(f)
@@ -97,13 +120,13 @@ def main():
         # f may be results/last-run/nodev/meta/009.dmesg
         bad_file_list = f.split("/")
         bad_file_list_len = len(bad_file_list) - 1
-        bad_file =      bad_file_list[bad_file_list_len]
-        test_group =    bad_file_list[bad_file_list_len-1]
-        bdev =          bad_file_list[bad_file_list_len-2]
+        bad_file = bad_file_list[bad_file_list_len]
+        test_group = bad_file_list[bad_file_list_len - 1]
+        bdev = bad_file_list[bad_file_list_len - 2]
 
         if args.verbose:
             sys.stdout.write("%s\n" % bad_file_list)
-            sys.stdout.write("\tbad_file: %s\n" %bad_file)
+            sys.stdout.write("\tbad_file: %s\n" % bad_file)
             sys.stdout.write("\ttest_group: %s\n" % test_group)
             sys.stdout.write("\tkernel: %s\n" % kernel)
 
@@ -126,11 +149,11 @@ def main():
             sys.exit(1)
 
         # This is like for example block/xxx where xxx are digits
-        test_failure_line = test_group + '/' + bad_file_test_number
+        test_failure_line = test_group + "/" + bad_file_test_number
 
         # now to stuff this into expunge files such as:
         # expunges/sles/15.3/failures.txt
-        expunge_kernel_dir = args.outputdir + '/' + kernel + '/'
+        expunge_kernel_dir = args.outputdir + "/" + kernel + "/"
         output_dir = expunge_kernel_dir
         output_file = output_dir + expunge_name
         shortcut_kernel_dir = None
@@ -145,19 +168,23 @@ def main():
                 sles_release_name = config_string(config, "CONFIG_KDEVOPS_HOSTS_PREFIX")
                 sles_release_parts = sles_release_name.split("sp")
                 if len(sles_release_parts) <= 1:
-                    sys.stderr.write("Unexpected sles_release_name: %s\n" % sles_release_name)
+                    sys.stderr.write(
+                        "Unexpected sles_release_name: %s\n" % sles_release_name
+                    )
                     sys.exit(1)
-                sles_point_release = sles_release_parts[0].split("sles")[1] + "." + sles_release_parts[1]
+                sles_point_release = (
+                    sles_release_parts[0].split("sles")[1] + "." + sles_release_parts[1]
+                )
 
                 # This becomes generic release directory, not specific to any
                 # kernel.
-                shortcut_dir = args.outputdir + '/' + "sles/" + sles_point_release + '/'
+                shortcut_dir = args.outputdir + "/" + "sles/" + sles_point_release + "/"
                 shortcut_kernel_dir = shortcut_dir
                 shortcut_file = shortcut_dir + expunge_name
             else:
                 ksplit = kernel.split(".")
                 shortcut_kernel = ksplit[0] + "." + ksplit[1] + "." + ksplit[2]
-                shortcut_kernel_dir = args.outputdir + '/' + shortcut_kernel + '/'
+                shortcut_kernel_dir = args.outputdir + "/" + shortcut_kernel + "/"
                 shortcut_dir = shortcut_kernel_dir
                 shortcut_file = shortcut_dir + expunge_name
 
@@ -170,10 +197,13 @@ def main():
                 os.makedirs(output_dir)
 
         if not os.path.isfile(output_file):
-            sys.stdout.write("====%s/%s new failure found file was empty\n" % (test_group, test_failure_line))
+            sys.stdout.write(
+                "====%s/%s new failure found file was empty\n"
+                % (test_group, test_failure_line)
+            )
             append_line(output_file, test_failure_line)
         else:
-            existing_file = open(output_file, 'r')
+            existing_file = open(output_file, "r")
             all_lines = existing_file.readlines()
             existing_file.close()
             found = False
@@ -182,13 +212,18 @@ def main():
                     found = True
                     break
             if not found:
-                sys.stdout.write("%s %s new failure found\n" % (test_group, test_failure_line))
+                sys.stdout.write(
+                    "%s %s new failure found\n" % (test_group, test_failure_line)
+                )
                 append_line(output_file, test_failure_line)
 
     if expunge_kernel_dir != "":
         sys.stdout.write("Sorting %s ...\n" % (expunge_kernel_dir))
-        sys.stdout.write("Running %s %s...\n" % (oscheck_sort_expunge, expunge_kernel_dir))
+        sys.stdout.write(
+            "Running %s %s...\n" % (oscheck_sort_expunge, expunge_kernel_dir)
+        )
         subprocess.call([oscheck_sort_expunge, expunge_kernel_dir])
 
-if __name__ == '__main__':
+
+if __name__ == "__main__":
     main()
playbooks/python/workflows/blktests/gen-expunge-args.py +34 -12

69 lines not on this page · raw message

playbooks/python/workflows/blktests/gen-results-dir.py +27 -12
index cb3e76aabce6..e18504bcffd0 100755
@@ -18,7 +18,8 @@ oscheck_ansible_python_dir = os.path.dirname(os.path.abspath(__file__))
 top_dir = oscheck_ansible_python_dir + "/../../../../"
 results_dir = top_dir + "workflows/blktests/results/"
 last_run_dir = results_dir + "last-run/"
-blktests_last_kernel = top_dir + 'workflows/blktests/results/last-kernel.txt'
+blktests_last_kernel = top_dir + "workflows/blktests/results/last-kernel.txt"
+
 
 def clean_empty_dir(target_results):
     for i in range(1, 3):
@@ -31,12 +32,23 @@ def clean_empty_dir(target_results):
                 else:
                     clean_empty_dir(f)
 
+
 def main():
-    parser = argparse.ArgumentParser(description='Get list of expunge files not yet committed in git')
-    parser.add_argument('--clean-dir-only', metavar='<clean_dir_only>', type=str, default='none',
-                        help='Do not perform an evaluation, just clean empty directories on the specified directory')
-    parser.add_argument('--copy-all', action='store_true',
-                        help='Copy all test results without filtering')
+    parser = argparse.ArgumentParser(
+        description="Get list of expunge files not yet committed in git"
+    )
+    parser.add_argument(
+        "--clean-dir-only",
+        metavar="<clean_dir_only>",
+        type=str,
+        default="none",
+        help="Do not perform an evaluation, just clean empty directories on the specified directory",
+    )
+    parser.add_argument(
+        "--copy-all",
+        action="store_true",
+        help="Copy all test results without filtering",
+    )
     args = parser.parse_args()
 
     if not os.path.isfile(blktests_last_kernel):
@@ -44,7 +56,7 @@ def main():
         sys.exit(1)
 
     kernel = None
-    f = open(blktests_last_kernel, 'r')
+    f = open(blktests_last_kernel, "r")
     for line in f:
         kernel = line.strip()
     if not line:
@@ -56,9 +68,11 @@ def main():
         clean_empty_dir(args.clean_dir_only)
         sys.exit(0)
 
-    target_results = results_dir + kernel + '/'
+    target_results = results_dir + kernel + "/"
     if not os.path.isdir(last_run_dir):
-        sys.stdout.write("Ignoring last-run directory %s as it is empty ...\n" % (last_run_dir))
+        sys.stdout.write(
+            "Ignoring last-run directory %s as it is empty ...\n" % (last_run_dir)
+        )
         sys.exit(0)
     sys.stdout.write("Copying %s to %s ...\n" % (last_run_dir, target_results))
     copytree(last_run_dir, target_results, dirs_exist_ok=True)
@@ -89,8 +103,8 @@ def main():
                     test_name = test_name_file_list[0]
 
                 test_dir = os.path.dirname(f)
-                name_lookup_base = test_dir + '/' + test_name + '*'
-                name_lookup = test_dir + '/' + test_name + '.*'
+                name_lookup_base = test_dir + "/" + test_name + "*"
+                name_lookup = test_dir + "/" + test_name + ".*"
                 listing = glob.glob(name_lookup)
                 bad_ext_found = False
                 if len(listing) > 0:
@@ -102,5 +116,6 @@ def main():
                         os.unlink(r)
     clean_empty_dir(target_results)
 
-if __name__ == '__main__':
+
+if __name__ == "__main__":
     main()
playbooks/python/workflows/blktests/get_new_expunge_files.py +12 -4
index 45f0e1a3c3d8..f1f1241bfc7b 100755
@@ -14,10 +14,17 @@ import sys
 import subprocess
 from lib import git
 
+
 def main():
-    parser = argparse.ArgumentParser(description='Get list of expunge files not yet committed in git')
-    parser.add_argument('expunge_dir', metavar='<directory with expunge files>', type=str,
-                        help='directory with expunge files')
+    parser = argparse.ArgumentParser(
+        description="Get list of expunge files not yet committed in git"
+    )
+    parser.add_argument(
+        "expunge_dir",
+        metavar="<directory with expunge files>",
+        type=str,
+        help="directory with expunge files",
+    )
     args = parser.parse_args()
 
     block_expunge_dir = args.expunge_dir
@@ -36,5 +43,6 @@ def main():
                     short_file = f.split("../")[1]
                 sys.stdout.write("%s\n" % (short_file))
 
-if __name__ == '__main__':
+
+if __name__ == "__main__":
     main()
playbooks/python/workflows/dynamic-kconfig/gen-dynamic-pci.py +60 -29

201 lines not on this page · raw message

playbooks/python/workflows/fstests/augment_expunge_list.py +114 -49

276 lines not on this page · raw message

playbooks/python/workflows/fstests/bad_files_summary.py +31 -15

90 lines not on this page · raw message

playbooks/python/workflows/fstests/fstests-checktime-distribution.py +29 -17

88 lines not on this page · raw message

playbooks/python/workflows/fstests/gen_results_summary.py +77 -55

271 lines not on this page · raw message

playbooks/python/workflows/fstests/get_new_expunge_files.py +18 -6
index a01330a2d32a..5b164044e1da 100755
@@ -14,12 +14,23 @@ import sys
 import subprocess
 from lib import git
 
+
 def main():
-    parser = argparse.ArgumentParser(description='Get list of expunge files not yet committed in git')
-    parser.add_argument('filesystem', metavar='<filesystem name>', type=str,
-                        help='filesystem which was tested')
-    parser.add_argument('expunge_dir', metavar='<directory with expunge files>', type=str,
-                        help='directory with expunge files')
+    parser = argparse.ArgumentParser(
+        description="Get list of expunge files not yet committed in git"
+    )
+    parser.add_argument(
+        "filesystem",
+        metavar="<filesystem name>",
+        type=str,
+        help="filesystem which was tested",
+    )
+    parser.add_argument(
+        "expunge_dir",
+        metavar="<directory with expunge files>",
+        type=str,
+        help="directory with expunge files",
+    )
     args = parser.parse_args()
 
     fs_expunge_dir = args.expunge_dir
@@ -41,5 +52,6 @@ def main():
                     short_file = f.split("../")[1]
                 sys.stdout.write("%s\n" % (short_file))
 
-if __name__ == '__main__':
+
+if __name__ == "__main__":
     main()
playbooks/python/workflows/fstests/lib/git.py +16 -5
index 1884f565d613..fe8cf857da7e 100644
@@ -2,25 +2,36 @@
 
 import subprocess, os
 
+
 class GitError(Exception):
     pass
+
+
 class ExecutionError(GitError):
     def __init__(self, errcode):
         self.error_code = errcode
+
+
 class TimeoutExpired(GitError):
     def __init__(self, errcode):
         self.error_code = errcode
         return "timeout"
 
+
 def _check(process):
     if process.returncode != 0:
         raise ExecutionError(process.returncode)
 
+
 def is_new_file(file):
-    cmd = ['git', 'status', '-s', file ]
-    process = subprocess.Popen(cmd,
-                               stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
-                               close_fds=True, universal_newlines=True)
+    cmd = ["git", "status", "-s", file]
+    process = subprocess.Popen(
+        cmd,
+        stdout=subprocess.PIPE,
+        stderr=subprocess.STDOUT,
+        close_fds=True,
+        universal_newlines=True,
+    )
     data = None
     try:
         data = process.communicate(timeout=1)
@@ -31,6 +42,6 @@ def is_new_file(file):
         process.wait()
         if process.returncode != 0:
             return False
-        if stdout.startswith('??'):
+        if stdout.startswith("??"):
             return True
         return False
playbooks/python/workflows/fstests/xunit_merge_all.py +24 -9
index 6af6e9817ab2..d2d2b64c100a 100755
@@ -11,6 +11,7 @@ import os
 import sys
 from junitparser import JUnitXml, TestSuite
 
+
 def get_test_suite(filename):
     try:
         ts = JUnitXml.fromfile(filename)
@@ -18,21 +19,31 @@ def get_test_suite(filename):
         sys.exit("Couldn't open %s: %s" % (filename, e[1]))
 
     if type(ts) != TestSuite:
-        sys.exit('%s is not a xUnit report file' % filename)
+        sys.exit("%s is not a xUnit report file" % filename)
     return ts
 
+
 def merge_ts(old_ts, new_ts):
     for tc in new_ts:
         old_ts.add_testcase(tc)
     old_ts.update_statistics()
     return old_ts
 
+
 def main():
-    parser = argparse.ArgumentParser(description='Merge all xunit files into one')
-    parser.add_argument('results', metavar='<directory with results>', type=str,
-                        help='directory with results file')
-    parser.add_argument('outputfile', metavar='<output file>', type=str,
-                        help='The file to generate output to')
+    parser = argparse.ArgumentParser(description="Merge all xunit files into one")
+    parser.add_argument(
+        "results",
+        metavar="<directory with results>",
+        type=str,
+        help="directory with results file",
+    )
+    parser.add_argument(
+        "outputfile",
+        metavar="<output file>",
+        type=str,
+        help="The file to generate output to",
+    )
     args = parser.parse_args()
 
     all_xunit_ts = None
@@ -46,7 +57,7 @@ def main():
                 continue
             if not os.path.isfile(f):
                 continue
-            if not f.endswith('.xml'):
+            if not f.endswith(".xml"):
                 continue
 
             sys.stdout.write("Processing %s ...\n" % f)
@@ -60,7 +71,11 @@ def main():
 
     if all_xunit_ts:
         all_xunit_ts.write(args.outputfile)
-        sys.stdout.write("%s generated by merging all the above %d xunit files successfully\n" % (args.outputfile, num))
+        sys.stdout.write(
+            "%s generated by merging all the above %d xunit files successfully\n"
+            % (args.outputfile, num)
+        )
+
 
-if __name__ == '__main__':
+if __name__ == "__main__":
     main()
playbooks/python/workflows/sysbench/sysbench-tps-compare.py +62 -22

140 lines not on this page · raw message

playbooks/python/workflows/sysbench/sysbench-tps-plot.py +24 -12
index 5505ced88ce2..ee56b1f56e3e 100755
@@ -9,27 +9,38 @@ import re
 import argparse
 from concurrent.futures import ThreadPoolExecutor
 
+
 # Function to parse a line and extract time and TPS from text
 def parse_line(line):
-    match = re.search(r'\[\s*(\d+)s\s*\].*?tps:\s*([\d.]+)', line)
+    match = re.search(r"\[\s*(\d+)s\s*\].*?tps:\s*([\d.]+)", line)
     if match:
         time_in_seconds = int(match.group(1))
         tps = float(match.group(2))
         return time_in_seconds, tps
     return None
 
+
 def main():
     # Setup argument parser
-    parser = argparse.ArgumentParser(description="Generate TPS plot from sysbench output")
-    parser.add_argument('input_file', type=str, help="Path to the input file (text or JSON)")
-    parser.add_argument('--output', type=str, default='tps_over_time.png', help="Output image file (default: tps_over_time.png)")
+    parser = argparse.ArgumentParser(
+        description="Generate TPS plot from sysbench output"
+    )
+    parser.add_argument(
+        "input_file", type=str, help="Path to the input file (text or JSON)"
+    )
+    parser.add_argument(
+        "--output",
+        type=str,
+        default="tps_over_time.png",
+        help="Output image file (default: tps_over_time.png)",
+    )
 
     # Parse arguments
     args = parser.parse_args()
 
     # Read the input file
     try:
-        with open(args.input_file, 'r') as file:
+        with open(args.input_file, "r") as file:
             # Read text lines and parse them concurrently
             lines = file.readlines()
             with ThreadPoolExecutor() as executor:
@@ -51,19 +62,19 @@ def main():
     # Convert times if necessary
     if use_hours:
         tps_data = [(time / 3600, tps) for time, tps in tps_data]
-        time_label = 'Time (hours)'
+        time_label = "Time (hours)"
     else:
-        time_label = 'Time (seconds)'
+        time_label = "Time (seconds)"
 
     # Create a pandas DataFrame
-    df = pd.DataFrame(tps_data, columns=[time_label, 'TPS'])
+    df = pd.DataFrame(tps_data, columns=[time_label, "TPS"])
 
     # Plot the TPS values
     plt.figure(figsize=(30, 12))
-    plt.plot(df[time_label], df['TPS'], 'o', markersize=2)
-    plt.title('Transactions Per Second (TPS) Over Time')
+    plt.plot(df[time_label], df["TPS"], "o", markersize=2)
+    plt.title("Transactions Per Second (TPS) Over Time")
     plt.xlabel(time_label)
-    plt.ylabel('TPS')
+    plt.ylabel("TPS")
     plt.grid(True)
     plt.ylim(0)
     plt.tight_layout()
@@ -72,5 +83,6 @@ def main():
     plt.savefig(args.output)
     print(f"TPS plot saved to {args.output}")
 
-if __name__ == '__main__':
+
+if __name__ == "__main__":
     main()
playbooks/python/workflows/sysbench/sysbench-tps-variance.py +318 -117

560 lines not on this page · raw message

playbooks/roles/gen_nodes/python/gen_pcie_passthrough_guestfs_xml.py +31 -18

78 lines not on this page · raw message

playbooks/roles/linux-mirror/python/gen-mirror-files.py +79 -52

193 lines not on this page · raw message

playbooks/roles/linux-mirror/python/start-mirroring.py +69 -47

172 lines not on this page · raw message

scripts/check_commit_format.py +20 -8
index f72f9d1a0352..3ac37944418c 100755
@@ -11,11 +11,16 @@ import subprocess
 import sys
 import re
 
+
 def get_latest_commit_message():
     """Get the latest commit message"""
     try:
-        result = subprocess.run(['git', 'log', '-1', '--pretty=format:%B'],
-                              capture_output=True, text=True, check=True)
+        result = subprocess.run(
+            ["git", "log", "-1", "--pretty=format:%B"],
+            capture_output=True,
+            text=True,
+            check=True,
+        )
         return result.stdout
     except subprocess.CalledProcessError:
         print("Error: Failed to get commit message")
@@ -24,30 +29,35 @@ def get_latest_commit_message():
         print("Error: git command not found")
         return None
 
+
 def check_commit_format(commit_msg):
     """Check commit message formatting"""
     issues = []
     if not commit_msg:
         return ["No commit message found"]
-    lines = commit_msg.strip().split('\n')
+    lines = commit_msg.strip().split("\n")
     # Find Generated-by line
     generated_by_idx = None
     signed_off_by_idx = None
     for i, line in enumerate(lines):
-        if line.startswith('Generated-by: Claude AI'):
+        if line.startswith("Generated-by: Claude AI"):
             generated_by_idx = i
-        elif line.startswith('Signed-off-by:'):
+        elif line.startswith("Signed-off-by:"):
             signed_off_by_idx = i
     # If Generated-by is present, check formatting
     if generated_by_idx is not None:
         if signed_off_by_idx is None:
-            issues.append("Generated-by: Claude AI found but no Signed-off-by tag present")
+            issues.append(
+                "Generated-by: Claude AI found but no Signed-off-by tag present"
+            )
         else:
             # Check if Generated-by is immediately followed by Signed-off-by (no lines in between)
             if signed_off_by_idx != generated_by_idx + 1:
                 lines_between = signed_off_by_idx - generated_by_idx - 1
                 if lines_between > 0:
-                    issues.append(f"Generated-by: Claude AI must be immediately followed by Signed-off-by (found {lines_between} lines between them)")
+                    issues.append(
+                        f"Generated-by: Claude AI must be immediately followed by Signed-off-by (found {lines_between} lines between them)"
+                    )
                     for i in range(generated_by_idx + 1, signed_off_by_idx):
                         if lines[i].strip():
                             issues.append(f"  - Non-empty line at {i+1}: '{lines[i]}'")
@@ -55,6 +65,7 @@ def check_commit_format(commit_msg):
                             issues.append(f"  - Empty line at {i+1}")
     return issues
 
+
 def main():
     """Main function to check commit message format"""
     commit_msg = get_latest_commit_message()
@@ -81,5 +92,6 @@ def main():
         print("✅ Commit message formatting is correct!")
         return 0
 
-if __name__ == '__main__':
+
+if __name__ == "__main__":
     sys.exit(main())
scripts/coccinelle/generation/check_for_atomic_calls.py +45 -26

168 lines not on this page · raw message

scripts/coccinelle/generation/check_for_sleepy_calls.py +140 -62

102 lines not on this page · raw message

scripts/detect_whitespace_issues.py +25 -13
index 165a33e25bcb..de5fef707317 100755

Lines of this file are not shown. View the raw message.

scripts/fix_whitespace_issues.py +26 -18

44 lines not on this page · raw message

scripts/generate_refs.py +4 -2
index 4bd179b9ddfd..45b743481f79 100755

Lines of this file are not shown. View the raw message.

scripts/honey-badger.py +75 -28

103 lines not on this page · raw message

scripts/spdxcheck.py +119 -82

201 lines not on this page · raw message

scripts/update_ssh_config_guestfs.py +32 -17

49 lines not on this page · raw message

scripts/workflows/blktests/blktests_watchdog.py +59 -16

75 lines not on this page · raw message

scripts/workflows/cxl/gen_qemu_cxl.py +167 -68

235 lines not on this page · raw message

scripts/workflows/fstests/fstests_watchdog.py +68 -31

99 lines not on this page · raw message

scripts/workflows/generic/crash_report.py +3 -1
index 10b4f958042d..65a05343d2ce 100755

Lines of this file are not shown. View the raw message.

scripts/workflows/generic/crash_watchdog.py +58 -20

78 lines not on this page · raw message

scripts/workflows/lib/blktests.py +34 -13

47 lines not on this page · raw message

scripts/workflows/lib/crash.py +6 -6
index d9d121d10a24..5397d784b8a2 100755

Lines of this file are not shown. View the raw message.

scripts/workflows/lib/fstests.py +93 -62

155 lines not on this page · raw message

scripts/workflows/lib/kssh.py +123 -55

178 lines not on this page · raw message

scripts/workflows/lib/systemd_remote.py +54 -47

101 lines not on this page · raw message

scripts/workflows/pynfs/check_pynfs_results.py +9 -8
index 724d2ace3950..753d177f734a 100755

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.47.2

Part of the diff is not shown here. View the raw message.

#8 lore raw

PATCH v3 8/11 scripts: enhance hop count detection to support DEB822 format

In reply to Luis Chamberlain

Update get-distro-has-hop-count-sources.sh to detect and handle both legacy sources.list and modern DEB822 debian.sources formats.

The script now:
- Checks for /etc/apt/sources.list.d/debian.sources first (DEB822)
- Falls back to /etc/apt/sources.list if DEB822 not found
- Extracts mirror URLs appropriately for each format:
- DEB822: Parses URIs: field
- Legacy: Parses deb lines

This ensures hop count detection works correctly regardless of which APT sources format is in use.

Generated-by Claude AI
---
 scripts/get-distro-has-hop-count-sources.sh | 29 ++++++++++++++++++---
 1 file changed, 25 insertions(+), 4 deletions(-)
scripts/get-distro-has-hop-count-sources.sh +25 -4
index 98e422b0cc85..2a1f5aa49981 100755
@@ -15,10 +15,17 @@ if [[ ! -f $DEBIAN_VERSION_FILE ]]; then
 fi
 
 SOURCES_FILE="/etc/apt/sources.list"
+DEB822_SOURCES_FILE="/etc/apt/sources.list.d/debian.sources"
 
-if [[ ! -f $SOURCES_FILE ]]; then
+# Check for DEB822 format first
+if [[ -f $DEB822_SOURCES_FILE ]]; then
+	SOURCES_FILE=$DEB822_SOURCES_FILE
+	IS_DEB822=true
+elif [[ ! -f $SOURCES_FILE ]]; then
 	echo n
 	exit 0
+else
+	IS_DEB822=false
 fi
 
 which traceroute > /dev/null
@@ -27,9 +34,23 @@ if [[ $? -ne 0 ]]; then
 	exit 0
 fi
 
-LINE=$(grep -v "^#" $SOURCES_FILE | head -1)
-HOST_URL_LINE=$(echo $LINE | awk '{print $2}')
-HOST=$(echo $HOST_URL_LINE | awk -F[/:] '{print $4}')
+# Extract host depending on format
+if [[ "$IS_DEB822" == "true" ]]; then
+	# DEB822 format: URIs: https://deb.debian.org/debian
+	HOST_URL_LINE=$(grep -E "^URIs:" $SOURCES_FILE | head -1 | awk '{print $2}')
+	HOST=$(echo $HOST_URL_LINE | sed -E 's|https?://||' | cut -d'/' -f1)
+else
+	# Legacy format: deb http://deb.debian.org/debian ...
+	LINE=$(grep -v "^#" $SOURCES_FILE | head -1)
+	HOST_URL_LINE=$(echo $LINE | awk '{print $2}')
+	HOST=$(echo $HOST_URL_LINE | awk -F[/:] '{print $4}')
+fi
+
+if [[ -z "$HOST" ]]; then
+	echo n
+	exit 0
+fi
+
 HOP_COUNT=$(traceroute -n -w 1,1,1 $HOST | wc -l)
 HOP_COUNT=$((HOP_COUNT -1))
 
Signature · 1 line
2.47.2
#9 lore raw

PATCH v3 9/11 devconfig: add automatic APT mirror fallback with DEB822 modernization

In reply to Luis Chamberlain

Debian testing (trixie) VMs can fail to provision when configured APT mirrors become unavailable or unresponsive. This is particularly common with local or regional mirrors that may have intermittent connectivity issues.

This fix adds automatic mirror health checking specifically for Debian testing systems. The implementation:

1. Detects current APT format (legacy sources.list or DEB822)
2. Extracts the configured mirror hostname from either format
3. Tests connectivity to the mirror on port 80 with 10 second timeout
4. Falls back to official Debian mirrors if the test fails
5. Backs up the original sources before making changes
6. Applies the new sources in modern DEB822 format
7. Removes legacy sources.list when migrating from old format
8. Updates the APT cache after switching mirrors

The check only runs on Debian testing systems where devconfig_debian_testing is set to true, avoiding any impact on stable Debian or other distributions.

By modernizing to DEB822 format during fallback, we align with Debian's recommended configuration while ensuring VMs can successfully provision even when the initially configured mirror is unavailable.

Generated-by Claude AI
---
 .../devconfig/tasks/check-apt-mirrors.yml     | 99 +++++++++++++++++++
 playbooks/roles/devconfig/tasks/main.yml      |  8 ++
 .../templates/debian-testing-fallback.sources | 13 +++
 3 files changed, 120 insertions(+)
 create mode 100644 playbooks/roles/devconfig/tasks/check-apt-mirrors.yml
 create mode 100644 playbooks/roles/devconfig/templates/debian-testing-fallback.sources
playbooks/roles/devconfig/tasks/check-apt-mirrors.yml new file +99 -0
new file mode 100644
index 000000000000..96e4048015d1
@@ -0,0 +1,99 @@
+---
+# Only run mirror checks for Debian testing (trixie) where mirror issues are common
+- name: Check for DEB822-style sources
+  stat:
+    path: /etc/apt/sources.list.d/debian.sources
+  register: deb822_sources
+
+- name: Extract current APT mirror hostname (DEB822 format)
+  shell: |
+    grep -E "^URIs:" /etc/apt/sources.list.d/debian.sources | head -1 | awk '{print $2}' | sed -E 's|https?://||' | cut -d'/' -f1
+  register: apt_mirror_host_deb822
+  changed_when: false
+  ignore_errors: yes
+  when: deb822_sources.stat.exists
+
+- name: Extract current APT mirror hostname (legacy format)
+  shell: |
+    grep -E "^deb\s+http" /etc/apt/sources.list | head -1 | awk '{print $2}' | sed 's|http://||' | cut -d'/' -f1
+  register: apt_mirror_host_legacy
+  changed_when: false
+  ignore_errors: yes
+  when: not deb822_sources.stat.exists
+
+- name: Set unified mirror hostname
+  set_fact:
+    apt_mirror_host:
+      stdout: "{{ apt_mirror_host_deb822.stdout if deb822_sources.stat.exists else apt_mirror_host_legacy.stdout }}"
+
+- name: Check connectivity to current APT mirror
+  wait_for:
+    host: "{{ apt_mirror_host.stdout }}"
+    port: 80
+    timeout: 10
+  register: mirror_connectivity
+  ignore_errors: yes
+  when: apt_mirror_host.stdout != ""
+
+- name: Display mirror check results
+  debug:
+    msg: |
+      Current APT mirror: {{ apt_mirror_host.stdout | default('Not found') }}
+      Mirror connectivity: {{ 'OK' if mirror_connectivity is not failed else 'FAILED' }}
+  when: apt_mirror_host.stdout != ""
+
+- name: Fall back to official Debian mirrors if current mirror fails
+  block:
+    - name: Backup current sources (DEB822 format)
+      copy:
+        src: /etc/apt/sources.list.d/debian.sources
+        dest: /etc/apt/sources.list.d/debian.sources.backup
+        remote_src: yes
+      become: yes
+      when: deb822_sources.stat.exists
+
+    - name: Backup current sources (legacy format)
+      copy:
+        src: /etc/apt/sources.list
+        dest: /etc/apt/sources.list.backup
+        remote_src: yes
+      become: yes
+      when: not deb822_sources.stat.exists
+
+    - name: Apply Debian testing fallback sources using modern DEB822 format
+      template:
+        src: debian-testing-fallback.sources
+        dest: /etc/apt/sources.list.d/debian.sources
+        owner: root
+        group: root
+        mode: '0644'
+      become: yes
+
+    - name: Remove legacy sources.list if migrating to DEB822
+      file:
+        path: /etc/apt/sources.list
+        state: absent
+      become: yes
+      when: not deb822_sources.stat.exists
+
+    - name: Update APT cache after mirror change
+      apt:
+        update_cache: yes
+        cache_valid_time: 0
+      become: yes
+
+    - name: Inform user about mirror fallback
+      debug:
+        msg: |
+          WARNING: The configured APT mirror '{{ apt_mirror_host.stdout }}' is not accessible.
+          Falling back to official Debian testing mirrors using modern DEB822 format:
+          - deb.debian.org for main packages
+          - security.debian.org for security updates
+
+          Your sources have been migrated to /etc/apt/sources.list.d/debian.sources
+          This may result in slower package downloads depending on your location.
+          Consider configuring a local mirror for better performance.
+
+  when:
+    - apt_mirror_host.stdout != ""
+    - mirror_connectivity is failed
playbooks/roles/devconfig/tasks/main.yml +8 -0
index 656d5389f685..ceb0f2e8b29c 100644
@@ -30,6 +30,14 @@
   tags: hostname
 
 # Distro specific
+
+# Check and fix APT mirrors for Debian testing before installing dependencies
+- name: Check and fix APT mirrors for Debian testing
+  include_tasks: check-apt-mirrors.yml
+  when:
+    - devconfig_debian_testing is defined
+    - devconfig_debian_testing | bool
+
 - name: Install dependencies
   ansible.builtin.include_tasks: install-deps/main.yml
   tags: ['vars', 'vars_simple']
playbooks/roles/devconfig/templates/debian-testing-fallback.sources new file +13 -0
new file mode 100644
index 000000000000..5358e3e61bee
@@ -0,0 +1,13 @@
+Types: deb deb-src
+URIs: https://deb.debian.org/debian
+Suites: testing testing-updates
+Components: main contrib non-free non-free-firmware
+Enabled: yes
+Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
+
+Types: deb deb-src
+URIs: https://security.debian.org/debian-security
+Suites: testing-security
+Components: main contrib non-free non-free-firmware
+Enabled: yes
+Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
\ No newline at end of file
Signature · 1 line
2.47.2
#10 lore raw

PATCH v3 10/11 devconfig: enhance hop1 detection to support traditional sources.list

In reply to Luis Chamberlain

Enhance the hop1 mirror detection to work with both DEB822 and traditional sources.list formats. When a hop1 mirror is detected on the control host (regardless of format) and the guest is running Debian testing, automatically convert to DEB822 format for consistency and modern package management.

Key improvements:
- Detect hop1 servers from both /etc/apt/sources.list.d/debian.sources (DEB822) and /etc/apt/sources.list (traditional format)
- Extract mirror host and path correctly from both formats
- Automatically convert Debian testing guests to use DEB822 when a hop1 mirror is available and accessible
- Preserve the mirror path (e.g., /debian, /debian-mirror) from the control host configuration
- Add connectivity checks before using the hop1 mirror
- Fall back to official Debian mirrors only if hop1 is unavailable or inaccessible

This ensures that local hop1 mirrors are used efficiently even when the control host hasn't migrated to DEB822 format yet, while modernizing the guest configuration for Debian testing systems.

Generated-by Claude AI
---
 .../devconfig/tasks/check-apt-mirrors.yml     | 114 ++++++++++++++++++
 .../templates/debian-hop1-mirror.sources      |  13 ++
 2 files changed, 127 insertions(+)
 create mode 100644 playbooks/roles/devconfig/templates/debian-hop1-mirror.sources
playbooks/roles/devconfig/tasks/check-apt-mirrors.yml +114 -0
index 96e4048015d1..d86196f15ebe 100644
@@ -1,5 +1,55 @@
 ---
 # Only run mirror checks for Debian testing (trixie) where mirror issues are common
+- name: Check hop count on control host
+  delegate_to: localhost
+  shell: |
+    {{ topdir_path }}/scripts/get-distro-has-hop-count-sources.sh 1
+  register: localhost_hop_count
+  changed_when: false
+  ignore_errors: yes
+
+- name: Extract hop1 mirror info from control host (DEB822 format)
+  delegate_to: localhost
+  shell: |
+    if [ -f /etc/apt/sources.list.d/debian.sources ]; then
+      HOST=$(grep -E "^URIs:" /etc/apt/sources.list.d/debian.sources | head -1 | awk '{print $2}' | sed -E 's|https?://||')
+      echo "${HOST}"
+    fi
+  register: localhost_hop1_mirror_deb822
+  changed_when: false
+  ignore_errors: yes
+  when: localhost_hop_count.stdout == 'y'
+
+- name: Extract hop1 mirror info from control host (legacy format)
+  delegate_to: localhost
+  shell: |
+    if [ -f /etc/apt/sources.list ] && [ ! -f /etc/apt/sources.list.d/debian.sources ]; then
+      LINE=$(grep -v "^#" /etc/apt/sources.list | grep -E "^deb\s+http" | head -1)
+      if [ -n "$LINE" ]; then
+        URL=$(echo $LINE | awk '{print $2}')
+        echo "${URL}" | sed 's|http://||'
+      fi
+    fi
+  register: localhost_hop1_mirror_legacy
+  changed_when: false
+  ignore_errors: yes
+  when: localhost_hop_count.stdout == 'y'
+
+- name: Set hop1 mirror variables
+  set_fact:
+    has_hop1_mirror: "{{ localhost_hop_count.stdout == 'y' }}"
+    hop1_mirror_full: "{{ localhost_hop1_mirror_deb822.stdout if localhost_hop1_mirror_deb822.stdout != '' else localhost_hop1_mirror_legacy.stdout }}"
+  when: localhost_hop_count.stdout == 'y'
+
+- name: Parse hop1 mirror host and path
+  set_fact:
+    hop1_mirror_host: "{{ hop1_mirror_full.split('/')[0] }}"
+    hop1_mirror_path: "/{{ hop1_mirror_full.split('/', 1)[1] if '/' in hop1_mirror_full else 'debian' }}"
+  when:
+    - has_hop1_mirror is defined
+    - has_hop1_mirror | bool
+    - hop1_mirror_full != ''
+
 - name: Check for DEB822-style sources
   stat:
     path: /etc/apt/sources.list.d/debian.sources
@@ -42,6 +92,69 @@
       Mirror connectivity: {{ 'OK' if mirror_connectivity is not failed else 'FAILED' }}
   when: apt_mirror_host.stdout != ""
 
+- name: Configure APT sources based on hop1 availability and debian testing
+  block:
+    - name: Check connectivity to hop1 mirror if available
+      wait_for:
+        host: "{{ hop1_mirror_host }}"
+        port: 80
+        timeout: 10
+      register: hop1_mirror_connectivity
+      ignore_errors: yes
+      when:
+        - has_hop1_mirror is defined
+        - has_hop1_mirror | bool
+        - hop1_mirror_host is defined
+
+    - name: Use hop1 mirror with DEB822 format for debian testing
+      block:
+        - name: Backup current sources
+          copy:
+            src: "{{ item }}"
+            dest: "{{ item }}.backup"
+            remote_src: yes
+          become: yes
+          loop:
+            - /etc/apt/sources.list
+            - /etc/apt/sources.list.d/debian.sources
+          ignore_errors: yes
+
+        - name: Apply hop1 mirror configuration using DEB822 format
+          template:
+            src: debian-hop1-mirror.sources
+            dest: /etc/apt/sources.list.d/debian.sources
+            owner: root
+            group: root
+            mode: '0644'
+          become: yes
+
+        - name: Remove legacy sources.list if migrating to DEB822
+          file:
+            path: /etc/apt/sources.list
+            state: absent
+          become: yes
+          when: not deb822_sources.stat.exists
+
+        - name: Update APT cache with hop1 mirror
+          apt:
+            update_cache: yes
+            cache_valid_time: 0
+          become: yes
+
+        - name: Inform user about hop1 mirror usage
+          debug:
+            msg: |
+              Using local hop1 mirror with DEB822 format:
+              Mirror: {{ hop1_mirror_host }}{{ hop1_mirror_path }}
+
+              This provides faster package downloads from your local mirror.
+              Sources have been converted to modern DEB822 format.
+      when:
+        - has_hop1_mirror is defined
+        - has_hop1_mirror | bool
+        - hop1_mirror_connectivity is not failed
+        - hop1_mirror_host is defined
+
 - name: Fall back to official Debian mirrors if current mirror fails
   block:
     - name: Backup current sources (DEB822 format)
@@ -97,3 +210,4 @@
   when:
     - apt_mirror_host.stdout != ""
     - mirror_connectivity is failed
+    - not (has_hop1_mirror is defined and has_hop1_mirror | bool and hop1_mirror_connectivity is not failed)
playbooks/roles/devconfig/templates/debian-hop1-mirror.sources new file +13 -0
new file mode 100644
index 000000000000..eff66706396e
@@ -0,0 +1,13 @@
+Types: deb deb-src
+URIs: http://{{ hop1_mirror_host }}{{ hop1_mirror_path | default('/debian') }}
+Suites: testing testing-updates
+Components: main contrib non-free non-free-firmware
+Enabled: yes
+Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
+
+Types: deb deb-src
+URIs: https://security.debian.org/debian-security
+Suites: testing-security
+Components: main contrib non-free non-free-firmware
+Enabled: yes
+Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
\ No newline at end of file
Signature · 1 line
2.47.2
#11 lore raw

PATCH v3 11/11 bootlinux: add support for A/B kernel testing

In reply to Luis Chamberlain

Right now we use the same kernel for all target nodes. We want to compare and contrast different kenrels for different features. We add support for A/B testing by leveraging the baseline and dev groups provided to us by KDEVOPS_BASELINE_AND_DEV.

This extends the bootlinux playbook by enabling us to allow a different kernel tree / ref to be used for the dev group. This just becomes a configuration thing. The targets are intuitive:

  make linux                 # Handles A/B compilation transparently
  make linux-baseline        # Build and install baseline kernel only
  make linux-dev             # Build and install development kernel only

We also add a simple check to verify all different build types end up respecting different kernels if we so choose:

make check-linux-ab

This does not launch targets it just verifies we don't regress in the future with the different ref tags.

Generated-by Claude AI
---
 .github/workflows/linux-ab.yml               |  47 ++++
 Makefile                                     |   1 +
 PROMPTS.md                                   |  52 +++++
 defconfigs/linux-ab-testing                  |  14 ++
 defconfigs/linux-ab-testing-9p               |  15 ++
 defconfigs/linux-ab-testing-builder          |  15 ++
 defconfigs/linux-ab-testing-target           |  15 ++
 docs/kdevops-make-linux.md                   | 158 ++++++++++++++
 playbooks/roles/bootlinux/defaults/main.yml  |  14 ++
 playbooks/roles/bootlinux/tasks/build/9p.yml |  20 +-
 playbooks/roles/bootlinux/tasks/main.yml     | 112 ++++++++++
 scripts/infer_last_stable_kernel.sh          |  35 +++
 scripts/linux-ab-testing.Makefile            |  51 +++++
 scripts/test-linux-ab-config.py              | 182 ++++++++++++++++
 scripts/test-linux-ab.sh                     | 213 +++++++++++++++++++
 workflows/linux/Kconfig                      | 102 ++++++++-
 workflows/linux/Makefile                     |  39 ++++
 17 files changed, 1074 insertions(+), 11 deletions(-)
 create mode 100644 .github/workflows/linux-ab.yml
 create mode 100644 defconfigs/linux-ab-testing
 create mode 100644 defconfigs/linux-ab-testing-9p
 create mode 100644 defconfigs/linux-ab-testing-builder
 create mode 100644 defconfigs/linux-ab-testing-target
 create mode 100755 scripts/infer_last_stable_kernel.sh
 create mode 100644 scripts/linux-ab-testing.Makefile
 create mode 100755 scripts/test-linux-ab-config.py
 create mode 100755 scripts/test-linux-ab.sh
.github/workflows/linux-ab.yml new file +47 -0
new file mode 100644
index 000000000000..9162c8879e1e
@@ -0,0 +1,47 @@
+name: Run kdevops linux-ab tests on self-hosted runner
+
+on:
+  push:
+    branches:
+      - '**'
+  pull_request:
+    branches:
+      - '**'
+  workflow_dispatch:  # Add this for manual triggering of the workflow
+
+jobs:
+  run-kdevops:
+    name: Run kdevops CI
+    runs-on: [self-hosted, Linux, X64]
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set CI metadata for kdevops-results-archive
+        run: |
+          echo "$(basename ${{ github.repository }})" > ci.trigger
+          git log -1 --pretty=format:"%s" > ci.subject
+          # Start out pessimistic
+          echo "not ok" > ci.result
+          echo "Nothing to write home about." > ci.commit_extra
+
+      - name: Set kdevops path
+        run: echo "KDEVOPS_PATH=$GITHUB_WORKSPACE" >> $GITHUB_ENV
+
+      - name: Configure git
+        run: |
+          git config --global --add safe.directory '*'
+          git config --global user.name "kdevops"
+          git config --global user.email "[email protected]"
+
+      - name: Run kdevops check-linux-ab
+        run: |
+          make check-linux-ab
+          echo "ok" > ci.result
+
+      # Ensure make destroy always runs, even on failure
+      - name: Run kdevops make destroy
+        if: always()  # This ensures the step runs even if previous steps failed
+        run: |
+          make destroy
+          make mrproper
Makefile +1 -0
index c88637c2966f..8755577e4d85 100644
@@ -243,6 +243,7 @@ $(KDEVOPS_NODES): .config $(ANSIBLE_CFG_FILE) $(KDEVOPS_NODES_TEMPLATE)
 DEFAULT_DEPS += $(LOCALHOST_SETUP_WORK)
 
 include scripts/tests.Makefile
+include scripts/linux-ab-testing.Makefile
 include scripts/ci.Makefile
 include scripts/archive.Makefile
 include scripts/defconfig.Makefile
PROMPTS.md +52 -0

55 lines not on this page · raw message

defconfigs/linux-ab-testing new file +14 -0
new file mode 100644
index 000000000000..c752e6a99aa4
@@ -0,0 +1,14 @@
+CONFIG_GUESTFS=y
+CONFIG_LIBVIRT=y
+
+CONFIG_WORKFLOWS=y
+CONFIG_WORKFLOW_LINUX_CUSTOM=y
+
+CONFIG_BOOTLINUX=y
+CONFIG_BOOTLINUX_9P=y
+
+# Enable baseline and dev testing
+CONFIG_KDEVOPS_BASELINE_AND_DEV=y
+
+# Enable A/B testing with different kernel references
+CONFIG_BOOTLINUX_AB_DIFFERENT_REF=y
defconfigs/linux-ab-testing-9p new file +15 -0
new file mode 100644
index 000000000000..35d589aa35c6
@@ -0,0 +1,15 @@
+CONFIG_GUESTFS=y
+CONFIG_LIBVIRT=y
+
+CONFIG_WORKFLOWS=y
+CONFIG_WORKFLOW_LINUX_CUSTOM=y
+
+CONFIG_BOOTLINUX=y
+CONFIG_BOOTLINUX_9P=y
+# CONFIG_BOOTLINUX_BUILDER is not set
+
+# Enable baseline and dev testing
+CONFIG_KDEVOPS_BASELINE_AND_DEV=y
+
+# Enable A/B testing with different kernel references
+CONFIG_BOOTLINUX_AB_DIFFERENT_REF=y
defconfigs/linux-ab-testing-builder new file +15 -0
new file mode 100644
index 000000000000..0b8817098d60
@@ -0,0 +1,15 @@
+CONFIG_GUESTFS=y
+CONFIG_LIBVIRT=y
+
+CONFIG_WORKFLOWS=y
+CONFIG_WORKFLOW_LINUX_CUSTOM=y
+
+CONFIG_BOOTLINUX=y
+# CONFIG_BOOTLINUX_9P is not set
+CONFIG_BOOTLINUX_BUILDER=y
+
+# Enable baseline and dev testing
+CONFIG_KDEVOPS_BASELINE_AND_DEV=y
+
+# Enable A/B testing with different kernel references
+CONFIG_BOOTLINUX_AB_DIFFERENT_REF=y
defconfigs/linux-ab-testing-target new file +15 -0
new file mode 100644
index 000000000000..21c72b56654b
@@ -0,0 +1,15 @@
+CONFIG_GUESTFS=y
+CONFIG_LIBVIRT=y
+
+CONFIG_WORKFLOWS=y
+CONFIG_WORKFLOW_LINUX_CUSTOM=y
+
+CONFIG_BOOTLINUX=y
+# CONFIG_BOOTLINUX_9P is not set
+# CONFIG_BOOTLINUX_BUILDER is not set
+
+# Enable baseline and dev testing
+CONFIG_KDEVOPS_BASELINE_AND_DEV=y
+
+# Enable A/B testing with different kernel references
+CONFIG_BOOTLINUX_AB_DIFFERENT_REF=y
docs/kdevops-make-linux.md +158 -0

161 lines not on this page · raw message

playbooks/roles/bootlinux/defaults/main.yml +14 -0
index fc6bfec0be1c..bbb85f0050ba 100644
@@ -52,3 +52,17 @@ bootlinux_tree_set_by_cli: False
 bootlinux_artifacts_dir: "{{ topdir_path }}/workflows/linux/artifacts"
 kernel_packages: []
 workflow_linux_packaged: false
+
+# A/B testing defaults
+bootlinux_ab_same_ref: True
+bootlinux_ab_different_ref: False
+
+# Development kernel settings (used when bootlinux_ab_different_ref is True)
+bootlinux_dev_tree: ""
+target_linux_dev_ref: "master"
+target_linux_dev_kernelrelease: ""
+target_linux_dev_localversion: ""
+bootlinux_tree_custom_kernelrelease: False
+bootlinux_tree_custom_localversion: false
+bootlinux_is_dev_node: False
+bootlinux_debug_ref: "{{ lookup('env', 'DEBUG_REF') | default(false, true) | bool }}"
playbooks/roles/bootlinux/tasks/build/9p.yml +10 -10
index bc2a66b6449a..1951e50edff9 100644
@@ -50,7 +50,7 @@
     dest: "{{ bootlinux_9p_host_path }}"
     update: yes
     depth: "{{ target_linux_shallow_depth }}"
-    version: "{{ target_linux_ref }}"
+    version: "{{ active_linux_ref | default(target_linux_ref) }}"
   retries: 3
   delay: 5
   register: result
@@ -106,9 +106,9 @@
   delegate_to: localhost
 
 - name: Set kernel localversion if requested on the control node
-  shell: "echo {{ target_linux_localversion }} > {{ bootlinux_9p_host_path }}/localversion"
+  shell: "echo {{ active_linux_localversion | default(target_linux_localversion) }} > {{ bootlinux_9p_host_path }}/localversion"
   when:
-    - target_linux_localversion is defined and target_linux_localversion != ""
+    - (active_linux_localversion is defined and active_linux_localversion != "") or (target_linux_localversion is defined and target_linux_localversion != "")
   run_once: true
   delegate_to: localhost
 
@@ -139,16 +139,16 @@
   register: target_linux_kernelversion
   tags: [ 'build-linux' ]
   when:
-    - target_linux_kernelrelease | length > 0
+    - (active_linux_kernelrelease | default(target_linux_kernelrelease)) | length > 0
   run_once: true
   delegate_to: localhost
 
-- name: Generate user kernelrelease {{ target_linux_kernelversion.stdout }}-{{ target_linux_kernelrelease }}
+- name: Generate user kernelrelease {{ target_linux_kernelversion.stdout }}-{{ active_linux_kernelrelease | default(target_linux_kernelrelease) }}
   set_fact:
-    target_user_kernelrelease: "{{ target_linux_kernelversion.stdout }}-{{ target_linux_kernelrelease }}"
+    target_user_kernelrelease: "{{ target_linux_kernelversion.stdout }}-{{ active_linux_kernelrelease | default(target_linux_kernelrelease) }}"
   tags: [ 'build-linux' ]
   when:
-    - target_linux_kernelrelease | length > 0
+    - (active_linux_kernelrelease | default(target_linux_kernelrelease)) | length > 0
   run_once: true
   delegate_to: localhost
 
@@ -160,17 +160,17 @@
       KERNELRELEASE={{ target_user_kernelrelease }}
   tags: [ 'build-linux' ]
   when:
-    - target_linux_kernelrelease | length > 0
+    - (active_linux_kernelrelease | default(target_linux_kernelrelease)) | length > 0
   run_once: true
   delegate_to: localhost
 
-- name: Build {{ target_linux_tree }} {{ target_user_kernelrelease }} on the control node using {{ nproc_9p.stdout }} threads
+- name: Build {{ target_linux_tree }} on the control node using {{ nproc_9p.stdout }} threads
   make:
     jobs: "{{ nproc_9p.stdout }}"
     chdir: "{{ bootlinux_9p_host_path }}"
   tags: [ 'build-linux' ]
   when:
-    - target_linux_kernelrelease | length == 0
+    - (active_linux_kernelrelease | default(target_linux_kernelrelease)) | length == 0
   run_once: true
   delegate_to: localhost
 
playbooks/roles/bootlinux/tasks/main.yml +112 -0

118 lines not on this page · raw message

scripts/infer_last_stable_kernel.sh new file +35 -0
new file mode 100755
index 000000000000..8d97f88249f5
@@ -0,0 +1,35 @@
+#!/bin/bash
+# SPDX-License-Identifier: copyleft-next-0.3.1
+
+# This script infers the last stable kernel version from the git repository.
+# It looks for the most recent non-rc tag (e.g., v6.14, v6.13) that would
+# be a good default for A/B testing with different kernel references.
+
+GIT_TREE="${1:-/mirror/linux.git}"
+
+if [ ! -d "$GIT_TREE" ]; then
+    echo "v6.12"  # fallback if no git tree available
+    exit 0
+fi
+
+# Get all v6.x tags, excluding release candidates
+# Sort them by version and get the last stable release
+LAST_STABLE=$(git --git-dir="$GIT_TREE" tag --list 'v6.*' | \
+    grep -v -- '-rc' | \
+    sort -V | \
+    tail -2 | head -1)
+
+if [ -z "$LAST_STABLE" ]; then
+    # If no stable v6.x found, try v5.x as fallback
+    LAST_STABLE=$(git --git-dir="$GIT_TREE" tag --list 'v5.*' | \
+        grep -v -- '-rc' | \
+        sort -V | \
+        tail -2 | head -1)
+fi
+
+# Final fallback if nothing found
+if [ -z "$LAST_STABLE" ]; then
+    echo "v6.12"
+else
+    echo "$LAST_STABLE"
+fi
scripts/linux-ab-testing.Makefile new file +51 -0

51 lines not on this page · raw message

scripts/test-linux-ab-config.py new file +182 -0

182 lines not on this page · raw message

scripts/test-linux-ab.sh new file +213 -0

213 lines not on this page · raw message

workflows/linux/Kconfig +101 -1

134 lines not on this page · raw message

workflows/linux/Makefile +39 -0
index bbc2c3d44a80..30b123f93017 100644
@@ -74,6 +74,10 @@ PHONY +=  linux-help-menu
 linux-help-menu:
 	@echo "Linux git kernel development options"
 	@echo "linux              - Git clones a linux git tree, build Linux, installs and reboots into it"
+	@if [[ "$(CONFIG_KDEVOPS_BASELINE_AND_DEV)" == "y" ]]; then \
+		echo "linux-baseline     - Build and install kernel for baseline nodes only" ;\
+		echo "linux-dev          - Build and install kernel for dev nodes only" ;\
+	fi
 	@if [[ "$(CONFIG_BOOTLINUX_9P)" == "y" ]]; then \
 		echo "linux-mount        - Mounts 9p path on targets" ;\
 	fi
@@ -93,11 +97,46 @@ linux-help-end:
 LINUX_HELP_EXTRA :=
 
 PHONY += linux
+ifeq (y,$(CONFIG_KDEVOPS_BASELINE_AND_DEV))
+ifeq (y,$(CONFIG_BOOTLINUX_AB_DIFFERENT_REF))
+linux: linux-baseline linux-dev
+else
+linux: $(KDEVOPS_NODES)
+	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) -i \
+		hosts $(KDEVOPS_PLAYBOOKS_DIR)/bootlinux.yml \
+		--extra-vars="$(BOOTLINUX_ARGS)" $(LIMIT_HOSTS)
+endif
+else
 linux: $(KDEVOPS_NODES)
 	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) \
 		--limit 'baseline:dev' \
 		$(KDEVOPS_PLAYBOOKS_DIR)/bootlinux.yml \
 		--extra-vars="$(BOOTLINUX_ARGS)" $(LIMIT_HOSTS)
+endif
+
+PHONY += linux-baseline
+ifeq (y,$(CONFIG_KDEVOPS_BASELINE_AND_DEV))
+linux-baseline: $(KDEVOPS_NODES)
+	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) -i \
+		hosts $(KDEVOPS_PLAYBOOKS_DIR)/bootlinux.yml \
+		--extra-vars="$(BOOTLINUX_ARGS)" --limit baseline
+else
+linux-baseline:
+	@echo "linux-baseline requires KDEVOPS_BASELINE_AND_DEV=y"
+	@exit 1
+endif
+
+PHONY += linux-dev
+ifeq (y,$(CONFIG_KDEVOPS_BASELINE_AND_DEV))
+linux-dev: $(KDEVOPS_NODES)
+	$(Q)ansible-playbook $(ANSIBLE_VERBOSE) -i \
+		hosts $(KDEVOPS_PLAYBOOKS_DIR)/bootlinux.yml \
+		--extra-vars="$(BOOTLINUX_ARGS)" --limit dev
+else
+linux-dev:
+	@echo "linux-dev requires KDEVOPS_BASELINE_AND_DEV=y"
+	@exit 1
+endif
 
 PHONY += linux-mount
 linux-mount:
Signature · 1 line
2.47.2
#12 lore raw
In reply to Luis Chamberlain
Quoted 2 lines · Luis Chamberlain

This v2 just expands the debian-testing apt sources with deb822 support, we can now infer and detect both formats and prefer deb822 when possible.

Pushed.

Luis