linux: generate git refs automatically
A message that is not in the archive (sent off-list or not yet received): CGME20240603203059eucas1p24a51c04a7c2bb86fe325fba0ad4b28ba@eucas1p2.samsung.com
v3 cover linux: generate git refs automatically
This adds support to generate git references automatically using git-ls-remote and kernel.org/releases.json.
Questions:
* Default behaviour (make) will generate Kconfigs for linux, next and mainline repos. Using make dynconfig will force and regenerate these files with the latest 15 references. Do we need this 15 refs or should we target latest-and-greatest only for these 3 repos?
* This patch includes autogenerated files for all trees (mcgrof-linus, mcgrof-next...) including only the static configuration. Running make dynconfig will force an update of these files with latest 15 references. Since autogenerated files are part of the git tree, kdevops users will see that some files have changed with git status. Any suggestion on how to improve this path? Not including the files would make 'make' fail as it will try to include the missing files. An alternative could be to create empty files on the way (to avoid the file creation delay) and force the user to run 'make dynconfig' in case these refs need to be generated.
* Default behaviour will update linux, next and mainline after 24h hours since last file modification. If the user has overwritten these files with make dynconfig, the values will be lost. Do we want a different behaviour here?
Daniel
Changes since v2:
* Rebase changes to latest kdevops.
* Add parser support for kernel.org/releases.json
* Add generate_refs.py subcommand support: gitref and kreleases.
* gitref: Retrieves refs from upstream repository.
* kreleases: Retrieves latest-and-greatest refs from kernel.org/releases.
* Integrate yaml with static configuration for mcgrof, jlayton and cel trees.
* Generate a default configuration file for all trees except for linus, next and stable.
* Kconfig.refs-{linus,next,stable} refs will automatically retrieve the information from kernel.org/releases.json every time make is run.
* Kconfig.refs-* files will be generated/forced when make dynconfig is run.
* Minor script changes: e.g. special case handling for CUSTOM entries.
* Minor Kconfig/Makefile renaming/cleanup.
Changes since v1:
* Script generation moved and called from the main Makefile.
* scripts/gen-refs.Makefile to generate refs.
* Conversion to python.
* Added support for:
* Command-line arguments to control options.
* YAML parser to configure extra and static references. Example added for mcgrof and mcgrof-next.
* Filter for tags and heads. Example to filter semantic versioning would be --filter-tags "*.*.*".
* File generation based on time. Update it every 24h (hardcoded). Possibility to use --force to always generate the file.
* git-ls-remote sorting inverted using --sort option and git versionsort.suffix option (to avoid sort --reverse).
Daniel Gomez (1):
linux: generate refs automatically
.gitignore | 2 + Makefile | 3 + defconfigs/nfsd | 2 +- scripts/dynamic-kconfig.Makefile | 4 + scripts/gen-gitrefs.Makefile | 111 ++++++++ scripts/gen-kreleases.Makefile | 25 ++ scripts/generate_refs.py | 304 +++++++++++++++++++++ workflows/linux/Kconfig | 32 +-- workflows/linux/Kconfig.btrfs | 4 +- workflows/linux/Kconfig.cel | 31 +-- workflows/linux/Kconfig.jlayton | 31 +-- workflows/linux/Kconfig.kdevops | 17 +- workflows/linux/Kconfig.linus | 69 +---- workflows/linux/Kconfig.mcgrof | 48 +--- workflows/linux/Kconfig.next | 4 +- workflows/linux/Kconfig.refs-btrfs-devel | 15 + workflows/linux/Kconfig.refs-cel-linux | 27 ++ workflows/linux/Kconfig.refs-jlayton-linux | 27 ++ workflows/linux/Kconfig.refs-kdevops-linus | 15 + workflows/linux/Kconfig.refs-mcgrof-linus | 21 ++ workflows/linux/Kconfig.refs-mcgrof-next | 21 ++ workflows/linux/Kconfig.stable | 63 +---- workflows/linux/btrfs-devel-refs.yaml | 6 + workflows/linux/cel-linux-refs.yaml | 14 + workflows/linux/jlayton-linux-refs.yaml | 14 + workflows/linux/kdevops-linus-refs.yaml | 6 + workflows/linux/mcgrof-linus-refs.yaml | 10 + workflows/linux/mcgrof-next-refs.yaml | 10 + 28 files changed, 669 insertions(+), 267 deletions(-) create mode 100644 scripts/gen-gitrefs.Makefile create mode 100644 scripts/gen-kreleases.Makefile create mode 100755 scripts/generate_refs.py create mode 100644 workflows/linux/Kconfig.refs-btrfs-devel create mode 100644 workflows/linux/Kconfig.refs-cel-linux create mode 100644 workflows/linux/Kconfig.refs-jlayton-linux create mode 100644 workflows/linux/Kconfig.refs-kdevops-linus create mode 100644 workflows/linux/Kconfig.refs-mcgrof-linus create mode 100644 workflows/linux/Kconfig.refs-mcgrof-next create mode 100644 workflows/linux/btrfs-devel-refs.yaml create mode 100644 workflows/linux/cel-linux-refs.yaml create mode 100644 workflows/linux/jlayton-linux-refs.yaml create mode 100644 workflows/linux/kdevops-linus-refs.yaml create mode 100644 workflows/linux/mcgrof-linus-refs.yaml create mode 100644 workflows/linux/mcgrof-next-refs.yaml
Signature · 1 line
2.43.0
PATCH v3 1/1 linux: generate refs automatically
Generate a Kconfig choice/endchoice menu with a list of the latest git references for upstream repositories.
* Default behaviour: latest-and-greatest.
The script included (generate_refs.py) will create (if it does not exist) the Kconfig choice options with the latest and greatest kernel release references provided by kernel.org/releases.json. Every time make is run, the script will check for the date of the last modification, and will force an update if more than 24h has passed since last modification. The generated files are: Kconfig.refs-{next,stable,linus}.
* make dynconfig:
In addition, the script will generate and overwrite all the existing Kconfig.refs-* files (including next, stable and linus) when dynconfig is run. This mode will retrieve the information from the upstream git repoitories for all supported kernel trees in kdevops and will create a list with the latest 15 references found.
--- .gitignore | 2 + Makefile | 3 + defconfigs/nfsd | 2 +- scripts/dynamic-kconfig.Makefile | 4 + scripts/gen-gitrefs.Makefile | 111 ++++++++ scripts/gen-kreleases.Makefile | 25 ++ scripts/generate_refs.py | 304 +++++++++++++++++++++ workflows/linux/Kconfig | 32 +-- workflows/linux/Kconfig.btrfs | 4 +- workflows/linux/Kconfig.cel | 31 +-- workflows/linux/Kconfig.jlayton | 31 +-- workflows/linux/Kconfig.kdevops | 17 +- workflows/linux/Kconfig.linus | 69 +---- workflows/linux/Kconfig.mcgrof | 48 +--- workflows/linux/Kconfig.next | 4 +- workflows/linux/Kconfig.refs-btrfs-devel | 15 + workflows/linux/Kconfig.refs-cel-linux | 27 ++ workflows/linux/Kconfig.refs-jlayton-linux | 27 ++ workflows/linux/Kconfig.refs-kdevops-linus | 15 + workflows/linux/Kconfig.refs-mcgrof-linus | 21 ++ workflows/linux/Kconfig.refs-mcgrof-next | 21 ++ workflows/linux/Kconfig.stable | 63 +---- workflows/linux/btrfs-devel-refs.yaml | 6 + workflows/linux/cel-linux-refs.yaml | 14 + workflows/linux/jlayton-linux-refs.yaml | 14 + workflows/linux/kdevops-linus-refs.yaml | 6 + workflows/linux/mcgrof-linus-refs.yaml | 10 + workflows/linux/mcgrof-next-refs.yaml | 10 + 28 files changed, 669 insertions(+), 267 deletions(-) create mode 100644 scripts/gen-gitrefs.Makefile create mode 100644 scripts/gen-kreleases.Makefile create mode 100755 scripts/generate_refs.py create mode 100644 workflows/linux/Kconfig.refs-btrfs-devel create mode 100644 workflows/linux/Kconfig.refs-cel-linux create mode 100644 workflows/linux/Kconfig.refs-jlayton-linux create mode 100644 workflows/linux/Kconfig.refs-kdevops-linus create mode 100644 workflows/linux/Kconfig.refs-mcgrof-linus create mode 100644 workflows/linux/Kconfig.refs-mcgrof-next create mode 100644 workflows/linux/btrfs-devel-refs.yaml create mode 100644 workflows/linux/cel-linux-refs.yaml create mode 100644 workflows/linux/jlayton-linux-refs.yaml create mode 100644 workflows/linux/kdevops-linus-refs.yaml create mode 100644 workflows/linux/mcgrof-linus-refs.yaml create mode 100644 workflows/linux/mcgrof-next-refs.yaml
.gitignore +2 -0
@@ -80,6 +80,8 @@ playbooks/roles/linux-mirror/linux-mirror-systemd/xfsprogs-mirror.timer # yet. workflows/selftests/results/ +workflows/linux/Kconfig.refs-* + .kernel-ci.* .kotd.*
Makefile +3 -0
@@ -173,6 +173,9 @@ endif include scripts/gen-hosts.Makefile include scripts/gen-nodes.Makefile +ifneq ($(MAKECMDGOALS),dynconfig) + include scripts/gen-kreleases.Makefile +endif # disable built-in rules for this .SUFFIXES:
defconfigs/nfsd +1 -1
@@ -279,7 +279,7 @@ CONFIG_BOOTLINUX_TREE_NEXT=y # CONFIG_BOOTLINUX_TREE_BTRFS_DEVEL is not set # CONFIG_BOOTLINUX_TREE_JLAYTON_LINUX is not set # CONFIG_BOOTLINUX_TREE_MCGROF_NEXT is not set -CONFIG_BOOTLINUX_TREE_NEXT_TAG="next-20240108" +CONFIG_BOOTLINUX_TREE_NEXT_REF="next-20240108" CONFIG_BOOTLINUX_TREE_NEXT_NAME="linux-next" CONFIG_BOOTLINUX_TREE_NEXT_URL="/mirror/linux-next.git" CONFIG_BOOTLINUX_TREE_NAME="linux-next"
scripts/dynamic-kconfig.Makefile +4 -0
@@ -17,6 +17,10 @@ dynamic-kconfig-help: PHONY += dynamic-kconfig-help +ifeq ($(MAKECMDGOALS),dynconfig) + include scripts/gen-gitrefs.Makefile +endif + dynconfig: $(Q)$(MAKE) dynconfig-pci
scripts/gen-gitrefs.Makefile new file +111 -0
111 lines not on this page · raw message
scripts/gen-kreleases.Makefile new file +25 -0
@@ -0,0 +1,25 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 + +GENERATED_REFS_NEXT := $(shell \ + ./scripts/generate_refs.py \ + --prefix BOOTLINUX_TREE_NEXT \ + --output workflows/linux/Kconfig.refs-next \ + kreleases \ + --moniker linux-next \ + ) + +GENERATED_REFS_STABLE := $(shell \ + ./scripts/generate_refs.py \ + --prefix BOOTLINUX_TREE_STABLE \ + --output workflows/linux/Kconfig.refs-stable \ + kreleases \ + --moniker stable \ + ) + +GENERATED_REFS_LINUS := $(shell \ + ./scripts/generate_refs.py \ + --prefix BOOTLINUX_TREE_LINUS \ + --output workflows/linux/Kconfig.refs-linus \ + kreleases \ + --moniker mainline \ + )
scripts/generate_refs.py new file +304 -0
304 lines not on this page · raw message
workflows/linux/Kconfig +16 -16
@@ -171,10 +171,10 @@ config BOOTLINUX_TREE_NAME default BOOTLINUX_TREE_STABLE_NAME if BOOTLINUX_TREE_STABLE default BOOTLINUX_TREE_NEXT_NAME if BOOTLINUX_TREE_NEXT default BOOTLINUX_TREE_BTRFS_DEVEL_NAME if BOOTLINUX_TREE_BTRFS_DEVEL - default BOOTLINUX_KDEVOPS_TREE_NAME if BOOTLINUX_TREE_KDEVOPS_LINUS - default BOOTLINUX_MCGROF_TREE_NAME if BOOTLINUX_MCGROF_TREE - default BOOTLINUX_JLAYTON_TREE_NAME if BOOTLINUX_TREE_JLAYTON_LINUX - default BOOTLINUX_CEL_TREE_NAME if BOOTLINUX_TREE_CEL_LINUX + default BOOTLINUX_TREE_KDEVOPS_NAME if BOOTLINUX_TREE_KDEVOPS_LINUS + default BOOTLINUX_TREE_MCGROF_NAME if BOOTLINUX_TREE_MCGROF + default BOOTLINUX_TREE_JLAYTON_NAME if BOOTLINUX_TREE_JLAYTON_LINUX + default BOOTLINUX_TREE_CEL_NAME if BOOTLINUX_TREE_CEL_LINUX default BOOTLINUX_CUSTOM_NAME if BOOTLINUX_CUSTOM config BOOTLINUX_TREE
@@ -182,11 +182,11 @@ config BOOTLINUX_TREE default BOOTLINUX_TREE_LINUS_URL if BOOTLINUX_TREE_LINUS default BOOTLINUX_TREE_STABLE_URL if BOOTLINUX_TREE_STABLE default BOOTLINUX_TREE_NEXT_URL if BOOTLINUX_TREE_NEXT - default BOOTLINUX_MCGROF_TREE_URL if BOOTLINUX_MCGROF_TREE + default BOOTLINUX_TREE_MCGROF_URL if BOOTLINUX_TREE_MCGROF default BOOTLINUX_TREE_KDEVOPS_URL if BOOTLINUX_TREE_KDEVOPS_LINUS - default BOOTLINUX_TREE_BTRFS_TREE_URL if BOOTLINUX_TREE_BTRFS_DEVEL - default BOOTLINUX_JLAYTON_TREE_URL if BOOTLINUX_TREE_JLAYTON_LINUX - default BOOTLINUX_CEL_TREE_URL if BOOTLINUX_TREE_CEL_LINUX + default BOOTLINUX_TREE_BTRFS_URL if BOOTLINUX_TREE_BTRFS_DEVEL + default BOOTLINUX_TREE_JLAYTON_URL if BOOTLINUX_TREE_JLAYTON_LINUX + default BOOTLINUX_TREE_CEL_URL if BOOTLINUX_TREE_CEL_LINUX default BOOTLINUX_CUSTOM_URL if BOOTLINUX_CUSTOM config BOOTLINUX_TEST_MESSAGE_ID_SET_BY_CLI
@@ -224,14 +224,14 @@ config BOOTLINUX_TEST_MESSAGE_ID_INSTALL_B4 config BOOTLINUX_TREE_TAG string - default BOOTLINUX_TREE_LINUS_TAG if BOOTLINUX_TREE_LINUS - default BOOTLINUX_TREE_STABLE_TAG if BOOTLINUX_STABLE - default BOOTLINUX_TREE_NEXT_TAG if BOOTLINUX_TREE_NEXT - default BOOTLINUX_TREE_BTRFS_DEVEL_TAG if BOOTLINUX_TREE_BTRFS_DEVEL - default BOOTLINUX_KDEVOPS_TREE_TAG if BOOTLINUX_TREE_KDEVOPS_LINUS - default BOOTLINUX_MCGROF_TREE_TAG if BOOTLINUX_MCGROF_TREE - default BOOTLINUX_TREE_JLAYTON_LINUX_TAG if BOOTLINUX_TREE_JLAYTON_LINUX - default BOOTLINUX_CEL_TREE_TAG if BOOTLINUX_TREE_CEL_LINUX + default BOOTLINUX_TREE_LINUS_REF if BOOTLINUX_TREE_LINUS + default BOOTLINUX_TREE_STABLE_REF if BOOTLINUX_STABLE + default BOOTLINUX_TREE_NEXT_REF if BOOTLINUX_TREE_NEXT + default BOOTLINUX_TREE_BTRFS_DEVEL_REF if BOOTLINUX_TREE_BTRFS_DEVEL + default BOOTLINUX_KDEVOPS_TREE_REF if BOOTLINUX_TREE_KDEVOPS_LINUS + default BOOTLINUX_MCGROF_TREE_REF if BOOTLINUX_MCGROF_TREE + default BOOTLINUX_TREE_JLAYTON_LINUX_REF if BOOTLINUX_TREE_JLAYTON_LINUX + default BOOTLINUX_TREE_CEL_REF if BOOTLINUX_TREE_CEL_LINUX default BOOTLINUX_CUSTOM_TAG if BOOTLINUX_CUSTOM config BOOTLINUX_TREE_KERNELRELEASE
workflows/linux/Kconfig.btrfs +1 -3
@@ -1,8 +1,6 @@ if BOOTLINUX_TREE_BTRFS_DEVEL -config BOOTLINUX_TREE_BTRFS_DEVEL_TAG - string - default "btrfs-devel" +source "workflows/linux/Kconfig.refs-btrfs-devel" config BOOTLINUX_TREE_BTRFS_DEVEL_NAME string
workflows/linux/Kconfig.cel +3 -28
@@ -1,25 +1,6 @@ if BOOTLINUX_TREE_CEL_LINUX -choice - prompt "cel-linux branch to use" - default BOOTLINUX_CEL_LINUX_NEXT - -config BOOTLINUX_CEL_LINUX_NEXT - bool "nfsd-next" - help - Pull the nfsd-next branch. - -config BOOTLINUX_CEL_LINUX_FIXES - bool "nfsd-fixes" - help - Pull the nfsd-fixes branch. - -config BOOTLINUX_CEL_LINUX_CUSTOM - bool "Custom branch name" - help - Pull a custom branch name. - -endchoice +source "workflows/linux/Kconfig.refs-cel-linux" config BOOTLINUX_CEL_LINUX_CUSTOM_NAME string "Custom branch name"
@@ -28,17 +9,11 @@ config BOOTLINUX_CEL_LINUX_CUSTOM_NAME help Custom branch name to use for Chuck Lever's linux tree. -config BOOTLINUX_CEL_TREE_TAG - string - default "nfsd-next" if BOOTLINUX_CEL_LINUX_NEXT - default "nfsd-fixes" if BOOTLINUX_CEL_LINUX_FIXES - default BOOTLINUX_CEL_LINUX_CUSTOM_NAME if BOOTLINUX_CEL_LINUX_CUSTOM - -config BOOTLINUX_CEL_TREE_NAME +config BOOTLINUX_TREE_CEL_NAME string default "cel-linux" -config BOOTLINUX_CEL_TREE_URL +config BOOTLINUX_TREE_CEL_URL string default "https://git.kernel.org/pub/scm/linux/kernel/git/cel/linux.git" if BOOTLINUX_TREE_CEL_LINUX
workflows/linux/Kconfig.jlayton +3 -28
@@ -1,25 +1,6 @@ if BOOTLINUX_TREE_JLAYTON_LINUX -choice - prompt "jlayton-linux branch to use" - default BOOTLINUX_JLAYTON_LINUX_KDEVOPS - -config BOOTLINUX_JLAYTON_LINUX_KDEVOPS - bool "kdevops" - help - Use the kdevops branch. - -config BOOTLINUX_JLAYTON_LINUX_IVERSION - bool "iversion-next" - help - Use the iversion-next branch. - -config BOOTLINUX_JLAYTON_LINUX_CUSTOM - bool "Custom branch name" - help - Use a custom branch name. - -endchoice +source "workflows/linux/Kconfig.refs-jlayton-linux" config BOOTLINUX_JLAYTON_LINUX_CUSTOM_NAME string "Custom branch name"
@@ -28,17 +9,11 @@ config BOOTLINUX_JLAYTON_LINUX_CUSTOM_NAME help Custom branch name to use for Jeff Layton's linux tree. -config BOOTLINUX_JLAYTON_LINUX_TAG - string - default BOOTLINUX_JLAYTON_LINUX_CUSTOM_NAME if BOOTLINUX_JLAYTON_LINUX_CUSTOM - default "kdevops" if BOOTLINUX_JLAYTON_LINUX_KDEVOPS - default "iversion-next" if BOOTLINUX_JLAYTON_LINUX_IVERSION - -config BOOTLINUX_JLAYTON_TREE_NAME +config BOOTLINUX_TREE_JLAYTON_NAME string default "jlayton-linux" -config BOOTLINUX_JLAYTON_TREE_URL +config BOOTLINUX_TREE_JLAYTON_URL string default "git://git.kernel.org/pub/scm/linux/kernel/git/jlayton/linux.git" if BOOTLINUX_TREE_JLAYTON_LINUX
workflows/linux/Kconfig.kdevops +2 -15
@@ -1,26 +1,13 @@ if BOOTLINUX_TREE_KDEVOPS_LINUS -choice - prompt "kdevops-linus branch to use" - default BOOTLINUX_KDEVOPS_LINUS_LBS_MINORDER - -config BOOTLINUX_KDEVOPS_LINUS_LBS_MINORDER - bool "large-block-minorder" - help - Use the latest LBS branch based on Linus' tree. - -endchoice +source "workflows/linux/Kconfig.refs-kdevops-linus" config BOOTLINUX_TREE_KDEVOPS_URL string default "https://github.com/linux-kdevops/linux.git" if !USE_LOCAL_LINUX_MIRROR default "/mirror/kdevops-linus.git" if USE_LIBVIRT_MIRROR && BOOTLINUX_9P -config BOOTLINUX_KDEVOPS_TREE_TAG - string - default "large-block-minorder" - -config BOOTLINUX_KDEVOPS_TREE_NAME +config BOOTLINUX_TREE_KDEVOPS_NAME string default "kdevops-linus"
workflows/linux/Kconfig.linus +1 -68
75 lines not on this page · raw message
workflows/linux/Kconfig.mcgrof +5 -43
70 lines not on this page · raw message
workflows/linux/Kconfig.next +1 -3
@@ -1,8 +1,6 @@ if BOOTLINUX_TREE_NEXT -config BOOTLINUX_TREE_NEXT_TAG - string "linux-next tag to pull" - default "next-20240424" +source "workflows/linux/Kconfig.refs-next" config BOOTLINUX_TREE_NEXT_NAME string
workflows/linux/Kconfig.refs-btrfs-devel new file +15 -0
@@ -0,0 +1,15 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 +# Automatically generated file +choice + prompt "Tag or branch to use" + +config BOOTLINUX_BTRFS_DEVEL + bool "btrfs-devel" + help + Use development branch. + +endchoice + +config BOOTLINUX_TREE_BTRFS_DEVEL_REF + string + default "btrfs-devel" if BOOTLINUX_BTRFS_DEVEL
workflows/linux/Kconfig.refs-cel-linux new file +27 -0
@@ -0,0 +1,27 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 +# Automatically generated file +choice + prompt "Tag or branch to use" + +config BOOTLINUX_CEL_LINUX_NEXT + bool "nfsd-next" + help + Pull the nfsd-next branch. + +config BOOTLINUX_CEL_LINUX_FIXES + bool "nfsd-fixed" + help + Pull the nfsd-fixed branch. + +config BOOTLINUX_CEL_LINUX_CUSTOM + bool "custom" + help + Use a custom branch name. + +endchoice + +config BOOTLINUX_TREE_CEL_LINUX_REF + string + default "nfsd-next" if BOOTLINUX_CEL_LINUX_NEXT + default "nfsd-fixed" if BOOTLINUX_CEL_LINUX_FIXES + default BOOTLINUX_CEL_LINUX_CUSTOM_NAME if BOOTLINUX_CEL_LINUX_CUSTOM
workflows/linux/Kconfig.refs-jlayton-linux new file +27 -0
@@ -0,0 +1,27 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 +# Automatically generated file +choice + prompt "Tag or branch to use" + +config BOOTLINUX_JLAYTON_LINUX_KDEVOPS + bool "kdevops" + help + Use the latest kdevops branch. + +config BOOTLINUX_JLAYTON_LINUX_IVERSION + bool "iversion-next" + help + Use the iversion-next branch. + +config BOOTLINUX_JLAYTON_LINUX_CUSTOM + bool "custom" + help + Use a custom branch name. + +endchoice + +config BOOTLINUX_TREE_JLAYTON_LINUX_REF + string + default "kdevops" if BOOTLINUX_JLAYTON_LINUX_KDEVOPS + default "iversion-next" if BOOTLINUX_JLAYTON_LINUX_IVERSION + default BOOTLINUX_JLAYTON_LINUX_CUSTOM_NAME if BOOTLINUX_JLAYTON_LINUX_CUSTOM
workflows/linux/Kconfig.refs-kdevops-linus new file +15 -0
@@ -0,0 +1,15 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 +# Automatically generated file +choice + prompt "Tag or branch to use" + +config BOOTLINUX_KDEVOPS_LINUS_LBS_MINORDER + bool "large-block-minorder" + help + Use the latest LBS branch based on Linus's tree + +endchoice + +config BOOTLINUX_TREE_KDEVOPS_LINUS_REF + string + default "large-block-minorder" if BOOTLINUX_KDEVOPS_LINUS_LBS_MINORDER
workflows/linux/Kconfig.refs-mcgrof-linus new file +21 -0
@@ -0,0 +1,21 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 +# Automatically generated file +choice + prompt "Tag or branch to use" + +config BOOTLINUX_MCGROF_LINUS_LBS + bool "large-block-linus" + help + Use the latest LBS branch based on Linus' tree. + +config BOOTLINUX_MCGROF_LINUS_LBS_NOBDEV + bool "large-block-linus-nobdev" + help + Use the latest minimal branch for LBS which let's you use LBS devices with a sectorsize compatibility based on Linus' tree. + +endchoice + +config BOOTLINUX_TREE_MCGROF_LINUS_REF + string + default "large-block-linus" if BOOTLINUX_MCGROF_LINUS_LBS + default "large-block-linus-nobdev" if BOOTLINUX_MCGROF_LINUS_LBS_NOBDEV
workflows/linux/Kconfig.refs-mcgrof-next new file +21 -0
@@ -0,0 +1,21 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 +# Automatically generated file +choice + prompt "Tag or branch to use" + +config BOOTLINUX_MCGROF_NEXT_LBS + bool "large-block-next" + help + Use the latest stable LBS branch which enables all the bells and whistles for all types of LBS devices. + +config BOOTLINUX_MCGROF_NEXT_LBS_NOBDEV + bool "large-block-nobdev" + help + Use the latest minimal branch for LBS which let's you use LBS devices with a sectorsize compatibility + +endchoice + +config BOOTLINUX_TREE_MCGROF_NEXT_REF + string + default "large-block-next" if BOOTLINUX_MCGROF_NEXT_LBS + default "large-block-nobdev" if BOOTLINUX_MCGROF_NEXT_LBS_NOBDEV
workflows/linux/Kconfig.stable +1 -62
72 lines not on this page · raw message
workflows/linux/btrfs-devel-refs.yaml new file +6 -0
@@ -0,0 +1,6 @@ +--- +configs: + - btrfs: + config: BOOTLINUX_BTRFS_DEVEL + ref: "btrfs-devel" + help: "Use development branch."
workflows/linux/cel-linux-refs.yaml new file +14 -0
@@ -0,0 +1,14 @@ +--- +configs: + - next: + config: BOOTLINUX_CEL_LINUX_NEXT + ref: "nfsd-next" + help: "Pull the nfsd-next branch." + - fixes: + config: BOOTLINUX_CEL_LINUX_FIXES + ref: "nfsd-fixed" + help: "Pull the nfsd-fixed branch." + - custom: + config: BOOTLINUX_CEL_LINUX_CUSTOM + ref: BOOTLINUX_CEL_LINUX_CUSTOM_NAME + help: "Use a custom branch name."
workflows/linux/jlayton-linux-refs.yaml new file +14 -0
@@ -0,0 +1,14 @@ +--- +configs: + - kdevops: + config: BOOTLINUX_JLAYTON_LINUX_KDEVOPS + ref: "kdevops" + help: "Use the latest kdevops branch." + - iversion: + config: BOOTLINUX_JLAYTON_LINUX_IVERSION + ref: "iversion-next" + help: "Use the iversion-next branch." + - custom: + config: BOOTLINUX_JLAYTON_LINUX_CUSTOM + ref: BOOTLINUX_JLAYTON_LINUX_CUSTOM_NAME + help: "Use a custom branch name."
workflows/linux/kdevops-linus-refs.yaml new file +6 -0
@@ -0,0 +1,6 @@ +--- +configs: + - minorder: + config: BOOTLINUX_KDEVOPS_LINUS_LBS_MINORDER + ref: "large-block-minorder" + help: "Use the latest LBS branch based on Linus's tree"
workflows/linux/mcgrof-linus-refs.yaml new file +10 -0
@@ -0,0 +1,10 @@ +--- +configs: + - master: + config: BOOTLINUX_MCGROF_LINUS_LBS + ref: "large-block-linus" + help: "Use the latest LBS branch based on Linus' tree." + - main: + config: BOOTLINUX_MCGROF_LINUS_LBS_NOBDEV + ref: "large-block-linus-nobdev" + help: "Use the latest minimal branch for LBS which let's you use LBS devices with a sectorsize compatibility based on Linus' tree."
workflows/linux/mcgrof-next-refs.yaml new file +10 -0
@@ -0,0 +1,10 @@ +--- +configs: + - master: + config: BOOTLINUX_MCGROF_NEXT_LBS + ref: "large-block-next" + help: "Use the latest stable LBS branch which enables all the bells and whistles for all types of LBS devices." + - main: + config: BOOTLINUX_MCGROF_NEXT_LBS_NOBDEV + ref: "large-block-nobdev" + help: "Use the latest minimal branch for LBS which let's you use LBS devices with a sectorsize compatibility"
Signature · 1 line
2.43.0
Re: [PATCH v3 1/1] linux: generate refs automatically
Quoted 11 lines · Daniel Gomez
Generate a
Kconfigchoice/endchoice menu with a list of the latest git references for upstream repositories.* Default behaviour: latest-and-greatest.
The script included (generate_refs.py) will create (if it does not exist) theKconfigchoice options with the latest and greatest kernel release references provided bykernel.org/releases.json. Every time make is run, the script will check for the date of the last modification, and will force an update if more than 24h has passed since last modification. The generated files are:Kconfig.refs-{next,stable,linus}.
Quoted 6 lines
* make dynconfig:
In addition, the script will generate and overwrite all the existingKconfig.refs-*files (including next, stable and linus) when dynconfig is run. This mode will retrieve the information from the upstream git repoitories for all supported kernel trees in kdevops and will create a list with the latest 15 references found.
This is pretty fucking cool.
Quoted 5 lines
Signed-off-by Daniel Gomez <[email protected]>--- .gitignore | 2 + Makefile | 3 + defconfigs/nfsd | 2 +-
Oh wow, that's the only one that needed changing?
scripts/gen-gitrefs.Makefile line 7
Quoted 13 lines
scripts/gen-gitrefs.Makefile new file +7
-0@@ -0,0 +1,111 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 + +$(info Generating git references...) + +export SRC_URI_HTTPS_LINUS = https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git +export SRC_URI_HTTPS_NEXT = https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git +export SRC_URI_HTTPS_STABLE = https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
If Linux had a religion, the above would be our canon for development.
scripts/gen-gitrefs.Makefile line 14
Quoted 6 lines
scripts/gen-gitrefs.Makefile new file +6
-0+export SRC_URI_HTTPS_MCGROF_LINUS = https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git +export SRC_URI_HTTPS_MCGROF_NEXT = https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux-next.git +export SRC_URI_HTTPS_BTRFS_DEVEL = https://github.com/kdave/btrfs-devel.git +export SRC_URI_HTTPS_CEL_LINUX = https://git.kernel.org/pub/scm/linux/kernel/git/cel/linux.git +export SRC_URI_HTTPS_JLAYTON_LINUX = https://git.kernel.org/pub/scm/linux/kernel/git/jlayton/linux.git +export SRC_URI_HTTPS_KDEVOPS_LINUS = https://github.com/linux-kdevops/linux.git
While this is the apocrypha. And in thinking about scaling this, I wonder if we should just have this section later, the apocrypha, automatically generated. Doesn't have to happen now, I think this a *huge* improvement over what we had before, so I welcome this as-is now with just a few comments below:
scripts/gen-gitrefs.Makefile line 23
Quoted 8 lines
scripts/gen-gitrefs.Makefile new file +8
-0+GENERATED_REFS_LINUS := $(shell \ + ./scripts/generate_refs.py \ + --prefix BOOTLINUX_TREE_LINUS \ + --output workflows/linux/Kconfig.refs-linus \ + --force \ + gitref \ + --repo $(SRC_URI_HTTPS_LINUS) \ + --refs 15 \
These seem to only differ from the non dynconfig approach with a few differences, and I wonder if those differences could be set with Makefile variables, and then having the dynconfig include a shared file. Doing a cursory review of the arguments passed, I don't think it's a as easy as a few variable changes, so I'll leave it up to you to decide. I'd prefer to get this merged sooner rather than later and we deal with other enhancements later.
As for the 15 latest refs per tree, hell I'd say just last 5 is good but after checking the results, and thining that one is already querying git so asking for 5 Vs 15 doesn't cause less latencies on make dynconfig, so I think 15 seems pretty sensible to me right now. If anything a user environment variable could be set to allow this to go beyond 15 too.
What concerns me slightly is just the default canon only gets the latest and greatest, while I think it is right that our Kconfig should default to the the latest and greatest, it would not allow us to allow scripts with older .configs to exist which rely on older rc releases for example.
If using default refs of 15 even for canon fixes this up to the latest 15 releases, then I'd be in favor of us just adopting that. I'd like other's feedback on this.
Luis
Re: [PATCH v3 1/1] linux: generate refs automatically
Quoted 21 lines · Luis Chamberlain
Quoted 11 lines · Daniel Gomez
Generate a
Kconfigchoice/endchoice menu with a list of the latest git references for upstream repositories.* Default behaviour: latest-and-greatest.
The script included (generate_refs.py) will create (if it does not exist) theKconfigchoice options with the latest and greatest kernel release references provided bykernel.org/releases.json. Every time make is run, the script will check for the date of the last modification, and will force an update if more than 24h has passed since last modification. The generated files are:Kconfig.refs-{next,stable,linus}.Quoted 6 lines
* make dynconfig:
In addition, the script will generate and overwrite all the existingKconfig.refs-*files (including next, stable and linus) when dynconfig is run. This mode will retrieve the information from the upstream git repoitories for all supported kernel trees in kdevops and will create a list with the latest 15 references found.This is pretty fucking cool.
Quoted 7 lines
Quoted 5 lines
Signed-off-by Daniel Gomez <[email protected]>--- .gitignore | 2 + Makefile | 3 + defconfigs/nfsd | 2 +-Oh wow, that's the only one that needed changing?
You are right. I did some Kconfig renames and I'm missing few places that need update for example, to replace BOOTLINUX_TREE_TAG with he new BOOTLINUX_TREE_REF introduced in this series. I'll double check it. Thanks!
Quoted 30 lines
Quoted 7 lines
scripts/gen-gitrefs.Makefile new file +0
-0scripts/gen-gitrefs.Makefile line 7
Quoted 7 lines
scripts/gen-gitrefs.Makefile new file +7
-0@@ -0,0 +1,111 @@ +# SPDX-License-Identifier: copyleft-next-0.3.1 + +$(info Generating git references...) + +export SRC_URI_HTTPS_LINUS = https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git +export SRC_URI_HTTPS_NEXT = https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git +export SRC_URI_HTTPS_STABLE = https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.gitIf Linux had a religion, the above would be our canon for development.
scripts/gen-gitrefs.Makefile
Quoted 6 lines
scripts/gen-gitrefs.Makefile new file +6
-0+export SRC_URI_HTTPS_MCGROF_LINUS = https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git +export SRC_URI_HTTPS_MCGROF_NEXT = https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux-next.git +export SRC_URI_HTTPS_BTRFS_DEVEL = https://protect2.fireeye.com/v1/url?k=26391c8b-47b209b2-263897c4-000babffae10-69325ae4f77a3118&q=1&e=e0306a31-3fd8-4b61-a6db-8aad13329128&u=https%3A%2F%2Fgithub.com%2Fkdave%2Fbtrfs-devel.git +export SRC_URI_HTTPS_CEL_LINUX = https://git.kernel.org/pub/scm/linux/kernel/git/cel/linux.git +export SRC_URI_HTTPS_JLAYTON_LINUX = https://git.kernel.org/pub/scm/linux/kernel/git/jlayton/linux.git +export SRC_URI_HTTPS_KDEVOPS_LINUS = https://protect2.fireeye.com/v1/url?k=1fabbbfc-7e20aec5-1faa30b3-000babffae10-7fccea486acfbba5&q=1&e=e0306a31-3fd8-4b61-a6db-8aad13329128&u=https%3A%2F%2Fgithub.com%2Flinux-kdevops%2Flinux.gitWhile this is the apocrypha. And in thinking about scaling this, I wonder if we should just have this section later, the apocrypha, automatically generated. Doesn't have to happen now, I think this a *huge* improvement over what we had before, so I welcome this as-is now with just a few comments below:
Quoted 14 lines
scripts/gen-gitrefs.Makefile line 23
Quoted 8 lines
scripts/gen-gitrefs.Makefile new file +8
-0+GENERATED_REFS_LINUS := $(shell \ + ./scripts/generate_refs.py \ + --prefix BOOTLINUX_TREE_LINUS \ + --output workflows/linux/Kconfig.refs-linus \ + --force \ + gitref \ + --repo $(SRC_URI_HTTPS_LINUS) \ + --refs 15 \These seem to only differ from the non dynconfig approach with a few differences, and I wonder if those differences could be set with
Makefilevariables, and then having the dynconfig include a shared file. Doing a cursory review of the arguments passed, I don't think it's a as easy as a few variable changes, so I'll leave it up to you to decide.
The problem is to include the git refs differences in the already existing Kconfig file as you need to add not only the new choices but also to increase the _REF list. Perhaps the easiest is to regenerate Kconfig files with the kernel.org/releases.json refs found first.
Quoted 2 lines
I'd prefer to get this merged sooner rather than later and we deal with other enhancements later.
Ok.
Quoted 6 lines
As for the 15 latest refs per tree, hell I'd say just last 5 is good but after checking the results, and thining that one is already querying git so asking for 5 Vs 15 doesn't cause less latencies on
make dynconfig, so I think 15 seems pretty sensible to me right now. If anything a user environment variable could be set to allow this to go beyond 15 too.
I think user config is better option and 15 by default looks sensible considering we have typically 7/8 -rc versions so I was thinking to be able to go at least 1 release back (for mainline).
Quoted 6 lines
What concerns me slightly is just the default canon only gets the latest and greatest, while I think it is right that our
Kconfigshould default to the the latest and greatest, it would not allow us to allow scripts with older .configs to exist which rely on older rc releases for example.
Old .configs pushed to kdevops can also include the static references needed in the yaml file. Then, we can include them with kreleases command (with --extra).
Quoted 4 lines
If using default refs of 15 even for canon fixes this up to the latest 15 releases, then I'd be in favor of us just adopting that. I'd like other's feedback on this.
That would require quering kernel.org which I think we were trying to avoid in the canon path? Local git mirror can be used too but would require the user to have set them up.
I cannot find better options other than quering via http/git kernel.org or manually including them in the static yaml file. Any alternatives?
Maybe we can just go back from the latest version? So from 6.10-rc2 go back and generate all v6.x mainline stable releases. And we would have just to include the latest for each major version (e.g. v5.19, v4.20, v3.19...) and generate all the refs from there. Since we do not have a pattern with -rcX we would not be able to automatically generate them with this approach.
Quoted 2 lines
Luis