Update release.sh
v5 cover Update release.sh
The release.sh seems to be last updated in 2015. Every one seems to have their own script to release. This patchset updates release.sh to do the basic stuff as updating version files, committing them, creating tag, creating release tarball, uploading tarball (optional), generate for-next announce.
Just a few minor fixes from v4, see below.
Signature · 54 lines
Andrey --- Changes in v5: - Use --separator instead of --delimiter for git-contributors.py in release.sh: generate ANNOUNCE email - Check return code of gpg --verify-signature in release.sh: add signing and fix outdated commands - Add argument to $(basename) in help() in release.sh: add --kup to upload release tarball to kernel.org - Link to v4: https://lore.kernel.org/r/[email protected] Changes in v4: - Add a few from recently generated files to gitignore - Drop Cc: trailers in libxfs-apply - Better handling of trailers with multiple emails/hash marks/not quoted names - Link to v3: https://lore.kernel.org/r/[email protected] Changes in v3: - Add -f to generate for-next ANNOUNCE email - Update for-next when new released is pushed - Link to v2: https://lore.kernel.org/r/[email protected] Changes in v2: - Added git-contributors.py from Darrick - A bit better/shorter commands for maintainer to run - Link to v1: https://lore.kernel.org/r/[email protected] --- Andrey Albershteyn (10): release.sh: add signing and fix outdated commands release.sh: add --kup to upload release tarball to kernel.org release.sh: update version files make commit optional Add git-contributors script to notify about merges git-contributors: better handling of hash mark/multiple emails git-contributors: make revspec required and shebang fix release.sh: generate ANNOUNCE email release.sh: add -f to generate for-next update email libxfs-apply: drop Cc: to stable release list gitignore: ignore a few newly generated files .gitignore | 2 + release.sh | 189 +++++++++++++++++++++++++++++++++++++++++++--- tools/git-contributors.py | 168 +++++++++++++++++++++++++++++++++++++++++ tools/libxfs-apply | 1 + 4 files changed, 348 insertions(+), 12 deletions(-) --- base-commit: 4fd999332e19993fb7fd381f5fcd40ff943d98ac change-id: 20241111-update-release-13ef17c008a5 Best regards, -- Andrey Albershteyn <[email protected]>
PATCH v5 1/10 release.sh: add signing and fix outdated commands
--- release.sh | 33 ++++++++++++++++++++++++++++----- 1 file changed, 28 insertions(+), 5 deletions(-)
release.sh +28 -5
@@ -9,6 +9,8 @@ # configure.ac (with new version string) # debian/changelog (with new release entry, only for release version) +set -e + . ./VERSION version=${PKG_MAJOR}.${PKG_MINOR}.${PKG_REVISION}
@@ -16,21 +18,42 @@ date=`date +"%-d %B %Y"` echo "Cleaning up" make realclean +rm -rf "xfsprogs-${version}.tar" \ + "xfsprogs-${version}.tar.gz" \ + "xfsprogs-${version}.tar.asc" \ + "xfsprogs-${version}.tar.sign" echo "Updating CHANGES" sed -e "s/${version}.*/${version} (${date})/" doc/CHANGES > doc/CHANGES.tmp && \ mv doc/CHANGES.tmp doc/CHANGES echo "Commiting CHANGES update to git" -git commit -a -m "${version} release" +git commit --all --signoff --message="xfsprogs: Release v${version} + +Update all the necessary files for a v${version} release." echo "Tagging git repository" -git tag -a -m "${version} release" v${version} +git tag --annotate --sign --message="Release v${version}" v${version} echo "Making source tarball" make dist +gunzip -k "xfsprogs-${version}.tar.gz" -#echo "Sign the source tarball" -#gpg --detach-sign xfsprogs-${version}.tar.gz +echo "Sign the source tarball" +gpg \ + --detach-sign \ + --armor \ + "xfsprogs-${version}.tar" -echo "Done. Please remember to push out tags using \"git push --tags\"" +echo "Verify signature" +gpg \ + --verify \ + "xfsprogs-${version}.tar.asc" +if [ $? -ne 0 ]; then + echo "Can not verify signature of tarball" + exit 1 +fi + +mv "xfsprogs-${version}.tar.asc" "xfsprogs-${version}.tar.sign" + +echo "Done. Please remember to push out tags using \"git push origin v${version}\""
Signature · 1 line
2.47.2
PATCH v5 2/10 release.sh: add --kup to upload release tarball to kernel.org
In reply to Andrey AlbershteynAdd kup support so that the maintainer can push the newly formed release tarballs to kernel.org.
--- release.sh | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+)
release.sh +31 -0
@@ -16,6 +16,30 @@ set -e version=${PKG_MAJOR}.${PKG_MINOR}.${PKG_REVISION} date=`date +"%-d %B %Y"` +KUP=0 + +help() { + echo "$(basename $0) - prepare xfsprogs release tarball or for-next update" + printf "\t[--kup|-k] upload final tarball with KUP\n" +} + +while [ $# -gt 0 ]; do + case "$1" in + --kup|-k) + KUP=1 + ;; + --help|-h) + help + exit 0 + ;; + *) + >&2 printf "Error: Invalid argument\n" + exit 1 + ;; + esac + shift +done + echo "Cleaning up" make realclean rm -rf "xfsprogs-${version}.tar" \
@@ -56,4 +80,11 @@ fi mv "xfsprogs-${version}.tar.asc" "xfsprogs-${version}.tar.sign" +if [ $KUP -eq 1 ]; then + kup put \ + xfsprogs-${version}.tar.gz \ + xfsprogs-${version}.tar.sign \ + pub/linux/utils/fs/xfs/xfsprogs/ +fi; + echo "Done. Please remember to push out tags using \"git push origin v${version}\""
Signature · 1 line
2.47.2
PATCH v5 3/10 release.sh: update version files make commit optional
In reply to Andrey AlbershteynBased on ./VERSION script updates all other files. For ./doc/changelog script asks maintainer to fill it manually as not all changes goes into changelog.
--no-commit|-n flag is handy when something got into the version commit and need to be changed manually. Then ./release.sh -c will use fixed history
--- release.sh | 75 ++++++++++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 58 insertions(+), 17 deletions(-)
release.sh +58 -17
@@ -11,16 +11,33 @@ set -e -. ./VERSION - -version=${PKG_MAJOR}.${PKG_MINOR}.${PKG_REVISION} -date=`date +"%-d %B %Y"` - KUP=0 +COMMIT=1 help() { echo "$(basename $0) - prepare xfsprogs release tarball or for-next update" printf "\t[--kup|-k] upload final tarball with KUP\n" + printf "\t[--no-commit|-n] don't create release commit\n" +} + +update_version() { + echo "Updating version files" + # doc/CHANGES + header="xfsprogs-${version} ($(date +'%d %b %Y'))" + sed -i "1s/^/$header\n\t<TODO list user affecting changes>\n\n/" doc/CHANGES + $EDITOR doc/CHANGES + + # ./configure.ac + CONF_AC="AC_INIT([xfsprogs],[${version}],[[email protected]])" + sed -i "s/^AC_INIT.*/$CONF_AC/" ./configure.ac + + # ./debian/changelog + sed -i "1s/^/\n/" ./debian/changelog + sed -i "1s/^/ -- Nathan Scott <[email protected]> `date -R`\n/" ./debian/changelog + sed -i "1s/^/\n/" ./debian/changelog + sed -i "1s/^/ * New upstream release\n/" ./debian/changelog + sed -i "1s/^/\n/" ./debian/changelog + sed -i "1s/^/xfsprogs (${version}-1) unstable; urgency=low\n/" ./debian/changelog } while [ $# -gt 0 ]; do
@@ -28,6 +45,9 @@ while [ $# -gt 0 ]; do --kup|-k) KUP=1 ;; + --no-commit|-n) + COMMIT=0 + ;; --help|-h) help exit 0
@@ -40,6 +60,36 @@ while [ $# -gt 0 ]; do shift done +if [ -z "$EDITOR" ]; then + EDITOR=$(command -v vi) +fi + +if [ $COMMIT -eq 1 ]; then + if git diff --exit-code ./VERSION > /dev/null; then + $EDITOR ./VERSION + fi +fi + +. ./VERSION + +version=${PKG_MAJOR}.${PKG_MINOR}.${PKG_REVISION} +date=`date +"%-d %B %Y"` + +if [ $COMMIT -eq 1 ]; then + update_version + + git diff --color=always | less -r + [[ "$(read -e -p 'All good? [Y/n]> '; echo $REPLY)" == [Nn]* ]] && exit 0 + + echo "Commiting new version update to git" + git commit --all --signoff --message="xfsprogs: Release v${version} + +Update all the necessary files for a v${version} release." + + echo "Tagging git repository" + git tag --annotate --sign --message="Release v${version}" v${version} +fi + echo "Cleaning up" make realclean rm -rf "xfsprogs-${version}.tar" \
@@ -47,17 +97,6 @@ rm -rf "xfsprogs-${version}.tar" \ "xfsprogs-${version}.tar.asc" \ "xfsprogs-${version}.tar.sign" -echo "Updating CHANGES" -sed -e "s/${version}.*/${version} (${date})/" doc/CHANGES > doc/CHANGES.tmp && \ - mv doc/CHANGES.tmp doc/CHANGES - -echo "Commiting CHANGES update to git" -git commit --all --signoff --message="xfsprogs: Release v${version} - -Update all the necessary files for a v${version} release." - -echo "Tagging git repository" -git tag --annotate --sign --message="Release v${version}" v${version} echo "Making source tarball" make dist
@@ -87,4 +126,6 @@ if [ $KUP -eq 1 ]; then pub/linux/utils/fs/xfs/xfsprogs/ fi; -echo "Done. Please remember to push out tags using \"git push origin v${version}\"" +echo "" +echo "Done. Please remember to push out tags and the branch." +printf "\tgit push origin v${version} master:master master:for-next\n"
Signature · 1 line
2.47.2
PATCH v5 4/10 Add git-contributors script to notify about merges
In reply to Andrey AlbershteynAdd python script used to collect emails over all changes merged in the next release.
--- tools/git-contributors.py | 101 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+)
tools/git-contributors.py new file +101 -0
@@ -0,0 +1,101 @@ +#!/usr/bin/python3 + +# List all contributors to a series of git commits. +# Copyright(C) 2025 Oracle, All Rights Reserved. +# Licensed under GPL 2.0 or later + +import re +import subprocess +import io +import sys +import argparse +import email.utils + +DEBUG = False + +def backtick(args): + '''Generator function that yields lines of a program's stdout.''' + if DEBUG: + print(' '.join(args)) + p = subprocess.Popen(args, stdout = subprocess.PIPE) + for line in io.TextIOWrapper(p.stdout, encoding="utf-8"): + yield line + +class find_developers(object): + def __init__(self): + tags = '%s|%s|%s|%s|%s|%s|%s|%s' % ( + 'signed-off-by', + 'acked-by', + 'cc', + 'reviewed-by', + 'reported-by', + 'tested-by', + 'suggested-by', + 'reported-and-tested-by') + # some tag, a colon, a space, and everything after that + regex1 = r'^(%s):\s+(.+)$' % tags + + self.r1 = re.compile(regex1, re.I) + + def run(self, lines): + addr_list = [] + + for line in lines: + l = line.strip() + + # emailutils can handle abominations like: + # + # Reviewed-by: Bogus J. Simpson <[email protected]> + # Reviewed-by: "Bogus J. Simpson" <[email protected]> + # Reviewed-by: [email protected] + # Cc: <[email protected]> # v6.9 + # Tested-by: Moo Cow <[email protected]> # powerpc + m = self.r1.match(l) + if not m: + continue + (name, addr) = email.utils.parseaddr(m.expand(r'\g<2>')) + + # This last split removes anything after a hash mark, + # because someone could have provided an improperly + # formatted email address: + # + # Cc: [email protected] # v6.19+ + # + # emailutils doesn't seem to catch this, and I can't + # fully tell from RFC2822 that this isn't allowed. I + # think it is because dtext doesn't forbid spaces or + # hash marks. + addr_list.append(addr.split('#')[0]) + + return sorted(set(addr_list)) + +def main(): + global DEBUG + + parser = argparse.ArgumentParser(description = "List email addresses of contributors to a series of git commits.") + parser.add_argument("revspec", nargs = '?', default = None, \ + help = "git revisions to process.") + parser.add_argument("--separator", type = str, default = '\n', \ + help = "Separate each email address with this string.") + parser.add_argument('--debug', action = 'store_true', default = False, \ + help = argparse.SUPPRESS) + args = parser.parse_args() + + if args.debug: + DEBUG = True + + fd = find_developers() + if args.revspec: + # read git commits from repo + contributors = fd.run(backtick(['git', 'log', '--pretty=medium', + args.revspec])) + else: + # read patch from stdin + contributors = fd.run(sys.stdin.readlines()) + + print(args.separator.join(sorted(contributors))) + return 0 + +if __name__ == '__main__': + sys.exit(main()) +
Signature · 1 line
2.47.2
PATCH v5 5/10 git-contributors: better handling of hash mark/multiple emails
In reply to Andrey AlbershteynBetter handling of hash mark, tags with multiple emails and not quoted names in emails. See comments in the script.
--- tools/git-contributors.py | 109 ++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 90 insertions(+), 19 deletions(-)
tools/git-contributors.py +90 -19
@@ -37,35 +37,106 @@ class find_developers(object): self.r1 = re.compile(regex1, re.I) + # regex to guess if this is a list of multiple addresses. + # Not sure why the initial "^.*" is needed here. + self.r2 = re.compile(r'^.*,[^,]*@[^@]*,[^,]*@', re.I) + + # regex to match on anything inside a pair of angle brackets + self.r3 = re.compile(r'^.*<(.+)>', re.I) + + def _handle_addr(self, addr): + # The next split removes everything after an octothorpe (hash + # mark), because someone could have provided an improperly + # formatted email address: + # + # Cc: [email protected] # v6.19+ + # + # This, according to my reading of RFC5322, is allowed because + # octothorpes can be part of atom text. However, it is + # interepreted as if there weren't any whitespace + # ("[email protected]#v6.19+"). The grammar allows for + # this form, even though this is not a correct Internet domain + # name. + # + # Worse, if you follow the format specified in the kernel's + # SubmittingPatches file: + # + # Cc: <[email protected]> # v6.9 + # + # emailutils will not know how to parse this, and returns empty + # strings. I think this is because the angle-addr + # specification allows only whitespace between the closing + # angle bracket and the CRLF. + # + # Hack around both problems by ignoring everything after an + # octothorpe, no matter where it occurs in the string. If + # someone has one in their name or the email address, too bad. + a = addr.split('#')[0] + + # emailutils can extract email addresses from headers that + # roughly follow the destination address field format: + # + # Reviewed-by: Bogus J. Simpson <[email protected]> + # Reviewed-by: "Bogus J. Simpson" <[email protected]> + # Reviewed-by: [email protected] + # + # Use it to extract the email address, because we don't care + # about the display name. + (name, addr) = email.utils.parseaddr(a) + if DEBUG: + print(f'A:{a}:NAME:{name}:ADDR:{addr}:') + if len(addr) > 0: + return addr + + # If emailutils fails to find anything, let's see if there's + # a sequence of characters within angle brackets and hope that + # is an email address. This works around things like: + # + # Reported-by: Xu, Wen <[email protected]> + # + # Which should have had the name in quotations because there's + # a comma. + m = self.r3.match(a) + if m: + addr = m.expand(r'\g<1>') + if DEBUG: + print(f"M3:{addr}:M:{m}:") + return addr + + # No idea, just spit the whole thing out and hope for the best. + return a + def run(self, lines): addr_list = [] for line in lines: l = line.strip() - # emailutils can handle abominations like: - # - # Reviewed-by: Bogus J. Simpson <[email protected]> - # Reviewed-by: "Bogus J. Simpson" <[email protected]> - # Reviewed-by: [email protected] - # Cc: <[email protected]> # v6.9 - # Tested-by: Moo Cow <[email protected]> # powerpc + # First, does this line match any of the headers we + # know about? m = self.r1.match(l) if not m: continue - (name, addr) = email.utils.parseaddr(m.expand(r'\g<2>')) + rightside = m.expand(r'\g<2>') - # This last split removes anything after a hash mark, - # because someone could have provided an improperly - # formatted email address: - # - # Cc: [email protected] # v6.19+ - # - # emailutils doesn't seem to catch this, and I can't - # fully tell from RFC2822 that this isn't allowed. I - # think it is because dtext doesn't forbid spaces or - # hash marks. - addr_list.append(addr.split('#')[0]) + n = self.r2.match(rightside) + if n: + # Break the line into an array of addresses, + # delimited by commas, then handle each + # address. + addrs = rightside.split(',') + if DEBUG: + print(f"0LINE:{rightside}:ADDRS:{addrs}:M:{n}") + for addr in addrs: + a = self._handle_addr(addr) + addr_list.append(a) + else: + # Otherwise treat the line as a single email + # address. + if DEBUG: + print(f"1LINE:{rightside}:M:{n}") + a = self._handle_addr(rightside) + addr_list.append(a) return sorted(set(addr_list))
Signature · 1 line
2.47.2
PATCH v5 6/10 git-contributors: make revspec required and shebang fix
In reply to Andrey AlbershteynWithout default value script will show help instead of just hanging waiting for input on stdin.
Shebang fix for system with different python location than the /usr/bin one.
Cut leading delimiter from the final CC string.
--- tools/git-contributors.py | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-)
tools/git-contributors.py +2 -6
@@ -1,4 +1,4 @@ -#!/usr/bin/python3 +#!/usr/bin/env python3 # List all contributors to a series of git commits. # Copyright(C) 2025 Oracle, All Rights Reserved.
@@ -144,8 +144,7 @@ def main(): global DEBUG parser = argparse.ArgumentParser(description = "List email addresses of contributors to a series of git commits.") - parser.add_argument("revspec", nargs = '?', default = None, \ - help = "git revisions to process.") + parser.add_argument("revspec", help = "git revisions to process.") parser.add_argument("--separator", type = str, default = '\n', \ help = "Separate each email address with this string.") parser.add_argument('--debug', action = 'store_true', default = False, \
@@ -160,9 +159,6 @@ def main(): # read git commits from repo contributors = fd.run(backtick(['git', 'log', '--pretty=medium', args.revspec])) - else: - # read patch from stdin - contributors = fd.run(sys.stdin.readlines()) print(args.separator.join(sorted(contributors))) return 0
Signature · 1 line
2.47.2
PATCH v5 7/10 release.sh: generate ANNOUNCE email
In reply to Andrey Albershteyn--- release.sh | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+)
release.sh +54 -0
@@ -13,11 +13,13 @@ set -e KUP=0 COMMIT=1 +LAST_HEAD="" help() { echo "$(basename $0) - prepare xfsprogs release tarball or for-next update" printf "\t[--kup|-k] upload final tarball with KUP\n" printf "\t[--no-commit|-n] don't create release commit\n" + printf "\t[--last-head|-l] commit of the last release\n" } update_version() {
@@ -40,6 +42,48 @@ update_version() { sed -i "1s/^/xfsprogs (${version}-1) unstable; urgency=low\n/" ./debian/changelog } +prepare_mail() { + branch="$1" + mail_file=$(mktemp) + if [ -n "$LAST_HEAD" ]; then + if [ $branch == "master" ]; then + reason="$(git describe --abbrev=0 $branch) released" + else + reason="for-next updated to $(git log --oneline --format="%h" -1 $branch)" + fi; + cat << EOF > $mail_file +To: [email protected] +Cc: $(./tools/git-contributors.py $LAST_HEAD..$branch --separator ', ') +Subject: [ANNOUNCE] xfsprogs: $reason + +Hi folks, + +The xfsprogs $branch branch in repository at: + + git://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git + +has just been updated. + +Patches often get missed, so if your outstanding patches are properly reviewed +on the list and not included in this update, please let me know. + +The for-next branch has also been updated to match the state of master. + +The new head of the $branch branch is commit: + +$(git log --oneline --format="%H" -1 $branch) + +New commits: + +$(git shortlog --format="[%h] %s" $LAST_HEAD..$branch) + +Code Diffstat: + +$(git diff --stat --summary -C -M $LAST_HEAD..$branch) +EOF + fi +} + while [ $# -gt 0 ]; do case "$1" in --kup|-k)
@@ -48,6 +92,10 @@ while [ $# -gt 0 ]; do --no-commit|-n) COMMIT=0 ;; + --last-head|-l) + LAST_HEAD=$2 + shift + ;; --help|-h) help exit 0
@@ -126,6 +174,12 @@ if [ $KUP -eq 1 ]; then pub/linux/utils/fs/xfs/xfsprogs/ fi; +prepare_mail "master" + echo "" echo "Done. Please remember to push out tags and the branch." printf "\tgit push origin v${version} master:master master:for-next\n" +if [ -n "$LAST_HEAD" ]; then + echo "Command to send ANNOUNCE email" + printf "\tneomutt -H $mail_file\n" +fi
Signature · 1 line
2.47.2
PATCH v5 8/10 release.sh: add -f to generate for-next update email
In reply to Andrey AlbershteynAdd --for-next/-f to generate ANNOUNCE email for for-next branch update. This doesn't require new commit/tarball/tags, so skip it.
--- release.sh | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
release.sh +16 -0
@@ -14,12 +14,14 @@ set -e KUP=0 COMMIT=1 LAST_HEAD="" +FOR_NEXT=0 help() { echo "$(basename $0) - prepare xfsprogs release tarball or for-next update" printf "\t[--kup|-k] upload final tarball with KUP\n" printf "\t[--no-commit|-n] don't create release commit\n" printf "\t[--last-head|-l] commit of the last release\n" + printf "\t[--for-next|-f] generate announce email for for-next update\n" } update_version() {
@@ -96,6 +98,9 @@ while [ $# -gt 0 ]; do LAST_HEAD=$2 shift ;; + --for-next|-f) + FOR_NEXT=1 + ;; --help|-h) help exit 0
@@ -108,6 +113,17 @@ while [ $# -gt 0 ]; do shift done +if [ $FOR_NEXT -eq 1 ]; then + echo "Push your for-next branch:" + printf "\tgit push origin for-next:for-next\n" + prepare_mail "for-next" + if [ -n "$LAST_HEAD" ]; then + echo "Command to send ANNOUNCE email" + printf "\tneomutt -H $mail_file\n" + fi + exit 0 +fi + if [ -z "$EDITOR" ]; then EDITOR=$(command -v vi) fi
Signature · 1 line
2.47.2
PATCH v5 9/10 libxfs-apply: drop Cc: to stable release list
In reply to Andrey AlbershteynThese Cc: tags are intended for kernel commits which need to be backported to stable kernels. Maintainers of stable kernel aren't interested in xfsprogs syncs.
--- tools/libxfs-apply | 1 + 1 file changed, 1 insertion(+)
tools/libxfs-apply +1 -0
@@ -254,6 +254,7 @@ fixup_header_format() } /^Date:/ { date_seen=1; next } /^difflib/ { next } + /[Cc]{2}: <[email protected]>?.*/ { next } // { if (date_seen == 0)
Signature · 1 line
2.47.2
PATCH v5 10/10 gitignore: ignore a few newly generated files
In reply to Andrey AlbershteynThese files are generated from corresponding *.in templates.
--- .gitignore | 2 ++ 1 file changed, 2 insertions(+)
.gitignore +2 -0
@@ -65,12 +65,14 @@ cscope.* /mdrestore/xfs_mdrestore /mkfs/fstyp /mkfs/mkfs.xfs +/mkfs/xfs_protofile /quota/xfs_quota /repair/xfs_repair /rtcp/xfs_rtcp /spaceman/xfs_spaceman /scrub/xfs_scrub /scrub/xfs_scrub_all +/scrub/xfs_scrub_all.timer /scrub/xfs_scrub_fail /scrub/*.cron /scrub/*.service
Signature · 1 line
2.47.2