linux: generate git refs automatically
Get series with b4
b4 am [email protected] change-id 20240620-linux-refs-a26f57d5fa3cVersions newest first · linked by author and title
- v5 Latest3 patches 2 repliesIn review
- v31 patch 2 repliesSuperseded
- v21 patch 5 repliesSuperseded
Changes in v5 from the cover letter · 18 lines
* Split renaming into its own patch as per Luis request. Note that config reference renaming is dropped in patch 2 as they are be autogenerated.
* Add static refs support in Next tree for fs-current and fs-next as per Chuck's suggestion.
* Refs now can be either Default References or User References.
* Refs are now generated into workflows/linux/refs/{default,user}/ Kconfig.*.
* Kconfig file names are renamed and simplified.
* 'make dynconfig' no longer has support to generate references.
* 'refs-default' and 'refs-user' targets added to switch in between 2 reference generation.
* Detection of modes is done by detecting if user refs are generated or not. This uses check_file_empty.sh script to set the Kconfig HAVE_BOOTLINUX_TREE_<TREE>_USER_REFS value depending on refs/user/ Kconfig.* content.
* Default Mode is explicitly run through _refs-default and specified in menuconfig dependent targets.
* Add verbosity mode to refs.Makefile. This enables echo but keeps the running commands silent.
* Add refs helper menu to indicate how to switch between Git Reference Modes.
* Minor fixes in the Kconfig generation output (tabs/spaces, etc).
* Some YAML entries were renamed to match the refs names.
* Documentation is added to explain workflow.
* generate_refs check for connectivity before trying to reach kernel.org servers to read releases.json or running git-ls-remote.
* User refs configuration names are suffixed with _USER_REF to avoid static duplicates. This duplicates the confs but prevents the warning 'choice value used outside its choice group'.
* Minor codying style changes in generate_refs script.
* Generated Kconfig now includes the HAVE_BOOTLINUX_xxx or !HAVE_BOOTLINUX_xxx based on the mode the output file is targeted.
Patches in v5 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/3 | linux: rename refactordefconfigs/ defconfigs/ defconfigs/ +19 more | +148 | 0 replies | |
| 2/3 | linux: generate refs automatically.gitignore Makefile README.md +32 more | +1067 | 0 replies | |
| 3/3 | linux: add fs-current and fs-next configsworkflows/ | +8 | 0 replies |
Reviews trailers per patch
None of the 3 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.
| Patch | Luis Chamberlain |
|---|---|
| Series | R-b |
| 1/3 | |
| 2/3 | |
| 3/3 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH v5 0/3] linux: generate git refs automatically
This adds support to generate git references automatically using make menuconfig, make refs-default and make refs-user.
With these targets, we can switch between Default and User Git Reference Modes to generate different lists of references. Default mode uses static references for the Development trees, kernel.org/releases.json, and static references for Linus, Next, and Stable trees. User mode generates the last 15 references for all trees, including the static references.
Please, read the docs/kdevops-autorefs.md from the second patch for a more detailed workflow explanation. And send me any questions if clarification is needed.
With so many changes in version 4, I decided to copy the questions from Luis in version 3.
Luis:
2) The expected day to do is not clear from the commit log. What aspects of no-use of this will generate a 'git diff' visible output if one does not use 'make dynconfig'?
None in a normal workflow. But in reality, only when updating the static references in the Development trees YAML file and running 'make refs-user'. The developer should send a patch upstream.
Note that 'make dynconfig' integration is dropped in this version in favor of the new targets 'refs-user' and 'refs-default'.
What aspects will generate a diff which
*should* be commited to the tree, ie how are kdevops developers expected to contribute to say a new revision of Linus' bump of a new rc candidate? The same applies to their git trees.
Only when updating the static references in the Development trees YAML file and running 'make refs-user'. The developer should send a patch upstream with the new YAML content as well as the autogenerate file. This does not apply to the Linux, Next and Stable trees as they are all the time generated with make menuconfig and make refs-default.
Changes since v4:
* Split renaming into its own patch as per Luis request. Note that config reference renaming is dropped in patch 2 as they are be autogenerated.
* Add static refs support in Next tree for fs-current and fs-next as per Chuck's suggestion.
* Refs now can be either Default References or User References.
* Refs are now generated into workflows/linux/refs/{default,user}/ Kconfig.*.
* Kconfig file names are renamed and simplified.
* 'make dynconfig' no longer has support to generate references.
* 'refs-default' and 'refs-user' targets added to switch in between 2 reference generation.
* Detection of modes is done by detecting if user refs are generated or not. This uses check_file_empty.sh script to set the Kconfig HAVE_BOOTLINUX_TREE_<TREE>_USER_REFS value depending on refs/user/ Kconfig.* content.
* Default Mode is explicitly run through _refs-default and specified in menuconfig dependent targets.
* Add verbosity mode to refs.Makefile. This enables echo but keeps the running commands silent.
* Add refs helper menu to indicate how to switch between Git Reference Modes.
* Minor fixes in the Kconfig generation output (tabs/spaces, etc).
* Some YAML entries were renamed to match the refs names.
* Documentation is added to explain workflow.
* generate_refs check for connectivity before trying to reach kernel.org servers to read releases.json or running git-ls-remote.
* User refs configuration names are suffixed with _USER_REF to avoid static duplicates. This duplicates the confs but prevents the warning 'choice value used outside its choice group'.
* Minor codying style changes in generate_refs script.
* Generated Kconfig now includes the HAVE_BOOTLINUX_xxx or !HAVE_BOOTLINUX_xxx based on the mode the output file is targeted.
Changes since v3:
* Change the strategy for the canon path (linus, next and stable trees):
* Current git references used in Kdevops are now included in a YAML file.
* This static references will be appended to the latest-and-greatest found by kreleases (kernel.org/releases.json).
* Minor typing fixes in generate_refs.py.
* --extra (config yaml) is now supported for gitrefs and kreleases sub-commands.
* Review files where new vars/configs were required to update because of renaming.
* Minor CONFIG renames for consistency.
Changes since v2:
* Rebase changes to latest kdevops.
* Add parser support for kernel.org/releases.json
* Add generate_refs.py subcommand support: gitref and kreleases.
* gitref: Retrieves refs from upstream repository.
* kreleases: Retrieves latest-and-greatest refs from kernel.org/releases.
* Integrate yaml with static configuration for mcgrof, jlayton and cel trees.
* Generate a default configuration file for all trees except for linus, next and stable.
* Kconfig.refs-{linus,next,stable} refs will automatically retrieve the information from kernel.org/releases.json every time make is run.
* Kconfig.refs-* files will be generated/forced when make dynconfig is run.
* Minor script changes: e.g. special case handling for CUSTOM entries.
* Minor Kconfig/Makefile renaming/cleanup.
Changes since v1:
* Script generation moved and called from the main Makefile.
* scripts/gen-refs.Makefile to generate refs.
* Conversion to python.
* Added support for:
* Command-line arguments to control options.
* YAML parser to configure extra and static references. Example added for mcgrof and mcgrof-next.
* Filter for tags and heads. Example to filter semantic versioning would be --filter-tags "*.*.*".
* File generation based on time. Update it every 24h (hardcoded). Possibility to use --force to always generate the file.
* git-ls-remote sorting inverted using --sort option and git versionsort.suffix option (to avoid sort --reverse).
Daniel
---
Daniel Gomez (3):
linux: rename refactor
linux: generate refs automatically
linux: add fs-current and fs-next configs
.gitignore | 5 + Makefile | 1 + README.md | 1 + defconfigs/cxl-switch | 8 +- defconfigs/nfsd | 4 +- defconfigs/xfs | 2 +- docs/how-extra-vars-generated.md | 8 +- docs/kdevops-autorefs.md | 118 +++++++ kconfigs/workflows/Kconfig | 2 + kconfigs/workflows/Kconfig.bootlinux | 37 +++ playbooks/roles/bootlinux/README.md | 4 +- playbooks/roles/bootlinux/defaults/main.yml | 4 +- playbooks/roles/bootlinux/tasks/main.yml | 8 +- .../roles/bootlinux/tasks/update-grub/install.yml | 6 +- scripts/check_file_empty.sh | 17 + scripts/gen-refs-default.Makefile | 52 ++++ scripts/gen-refs-user.Makefile | 154 ++++++++++ scripts/generate_refs.py | 341 +++++++++++++++++++++ scripts/kconfig/kconfig.Makefile | 2 +- scripts/refs.Makefile | 22 ++ scripts/workflows/fstests/fstests_watchdog.py | 4 +- scripts/workflows/generic/run_kernel_ci.sh | 2 +- workflows/fstests/Makefile | 4 +- .../xfs/unassigned/fstests-next-20230515 | 2 +- workflows/linux/Kconfig | 46 +-- workflows/linux/Kconfig.btrfs | 5 +- workflows/linux/Kconfig.cel | 36 +-- workflows/linux/Kconfig.jlayton | 36 +-- workflows/linux/Kconfig.kdevops | 18 +- workflows/linux/Kconfig.linus | 70 +---- workflows/linux/Kconfig.mcgrof | 50 +-- workflows/linux/Kconfig.next | 5 +- workflows/linux/Kconfig.stable | 64 +--- workflows/linux/Makefile | 14 +- workflows/linux/refs/default/Kconfig.btrfs-devel | 20 ++ workflows/linux/refs/default/Kconfig.cel-linux | 32 ++ workflows/linux/refs/default/Kconfig.jlayton-linux | 32 ++ workflows/linux/refs/default/Kconfig.kdevops-linus | 20 ++ workflows/linux/refs/default/Kconfig.mcgrof-linus | 26 ++ workflows/linux/refs/default/Kconfig.mcgrof-next | 26 ++ workflows/linux/refs/static/btrfs-devel.yaml | 6 + workflows/linux/refs/static/cel-linux.yaml | 14 + workflows/linux/refs/static/jlayton-linux.yaml | 14 + workflows/linux/refs/static/kdevops-linus.yaml | 6 + workflows/linux/refs/static/linus.yaml | 38 +++ workflows/linux/refs/static/mcgrof-linus.yaml | 10 + workflows/linux/refs/static/mcgrof-next.yaml | 10 + workflows/linux/refs/static/next.yaml | 14 + workflows/linux/refs/static/stable.yaml | 38 +++ 49 files changed, 1145 insertions(+), 313 deletions(-)
---
base-commit: 9f42f09ae473e6ebb81c9094ac924ddc27c9e38e
change-id: 20240620-linux-refs-a26f57d5fa3c
Best regards,
Signature · 1 line
Daniel Gomez <[email protected]>