Skip to contentSkip to conversation

Add new add_maintainers.pl script to optimise the workflow when using git format-patch with get_maintainer.pl

PATCH Superseded Lars Kurth 3 participants 8 messages Apr 27, 2018

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Lars Kurth

author
#0 lore raw

cover Add new add_maintainers.pl script to optimise the workflow when using git format-patch with get_maintainer.pl

To avoid issues with duplicate e-mail sending, I also cleaned up MAINTAINERS

Lars Kurth (2):
Add new add_maintainers.pl script to optimise the workflow when using
git format-patch with get_maintainer.pl
Replace occurances of xen.org with xenproject.org

 MAINTAINERS                |  30 +++---
 scripts/add_maintainers.pl | 252 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 267 insertions(+), 15 deletions(-)
 create mode 100755 scripts/add_maintainers.pl
Signature · 6 lines
2.13.0

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#1 lore raw

PATCH 1/2 Add new add_maintainers.pl script to optimise the workflow when using git format-patch with get_maintainer.pl

The tool covers step 2 of the following workflow

Without --overwrite
Step 1: git format-patch ... -o <patchdir> ...
Step 2: ./scripts/add_maintainers.pl -d <patchdir>
This creates *.patchm files in <patchdir>
Step 3: git send-email -to [email protected] <patchdir>/*.patchm

With --overwrite
Step 1: git format-patch ... -o <patchdir> ...
Step 2: ./scripts/add_maintainers.pl -d <patchdir>
This overwrites *.patch files in <patchdir> but makes a backup
Step 3: git send-email -to [email protected] <patchdir>/*.patch

It is not clear to me what the preferred default would be, so I went for the safer option (ok without --overwrite).

I manually tested all options and the most common combinations on Mac.

Cc Andrew Cooper <[email protected]>
Cc George Dunlap <[email protected]>
Cc Ian Jackson <[email protected]>
Cc Jan Beulich <[email protected]>
Cc Julien Grall <[email protected]>
Cc Konrad Rzeszutek Wilk <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Tim Deegan <[email protected]>
Cc Wei Liu <[email protected]>
---
 scripts/add_maintainers.pl | 252 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 252 insertions(+)
 create mode 100755 scripts/add_maintainers.pl
scripts/add_maintainers.pl new file +252 -0
new file mode 100755
index 0000000000..27ac74f339
@@ -0,0 +1,252 @@
+#!/usr/bin/perl -w
+# (c) 2018, Lars Kurth <[email protected]>
+#
+# Add maintainers to patches generated with git format-patch
+#
+# Usage: perl scripts/add_maintainers.pl [OPTIONS] -patchdir <patchdir>
+#
+# Prerequisites: Execute
+#                git format-patch ... -o <patchdir> ...
+#
+#                ./scripts/get_maintainer.pl is present in the tree
+#
+# Licensed under the terms of the GNU GPL License version 2
+
+use strict;
+
+use Getopt::Long qw(:config no_auto_abbrev);
+use File::Basename;
+use List::MoreUtils qw(uniq);
+
+# Arguments / Options
+my $tool = $0;
+my $toolversion = "1.0";
+
+# Arguments / Options
+my $version = 0;
+my $help = 0;
+my $patch_dir = 0;
+my $patch_prefix = "";
+my $get_maintainer_args = "";
+my $verbose = 0;
+my $overwrite = 0;
+my $overwrite_ext = "m";
+
+# Constants
+my $CC                  = "Cc: "; # Note: git-send-mail requires Cc:
+my $TO                  = "To: ";
+my $AT                  = "@";
+my $cc_insert_before    = "Signed-off-by:";
+my $to_insert_before    = "Date:";
+my $cover_letter        = "0000-cover-letter.patch";
+my $get_maintainer      = "./scripts/get_maintainer.pl";
+my $patch_ext           = ".patch";
+my $mailing_lists       = $AT."lists.";
+
+my @lists; #Needed for <<EOT
+my $usage = <<EOT;
+USAGE: $tool [options] -patchdir <patchdir>
+VERSION: $toolversion
+
+OPTIONS:
+--------
+  --p|prefix (e.g. --prefix v2)
+    Choose patch files with a specific prefix
+  --a|args
+    Arguments passed on to $get_maintainer
+  --v|verbose
+    Show more output
+  --version
+    Show version
+  --o|overwrite
+    Overwrites *.patch file (but makes a backup), whereas teh default
+    creates a new *.mpatch file
+  --h|help|usage
+    Show this help information
+
+WORKFLOW:
+---------
+  This script is intended to be used as part of the following workflow
+
+  Without --overwrite
+  Step 1: git format-patch ... -o <patchdir> ...
+  Step 2: ./scripts/add_maintainers.pl -d <patchdir>
+          This creates *.patchm files in <patchdir>
+  Step 3: git send-email -to [email protected] <patchdir>/*.patchm
+
+  With --overwrite
+  Step 1: git format-patch ... -o <patchdir> ...
+  Step 2: ./scripts/add_maintainers.pl -d <patchdir>
+          This overwrites *.patch files in <patchdir> but makes a backup
+  Step 3: git send-email -to [email protected] <patchdir>/*.patch
+EOT
+
+if (!GetOptions(
+                'd|patchdir=s' => \$patch_dir,
+                'p|prefix=s'   => \$patch_prefix,
+                'o|overwrite'  => \$overwrite,
+                'a|args=s'     => \$get_maintainer_args,
+                'v|verbose'    => \$verbose,
+                'version'      => \$version,
+                'h|help|usage' => \$help,
+                )) {
+    die "$tool: invalid argument - use --help if necessary\n";
+}
+
+if ($help != 0) {
+    print $usage;
+    exit 0;
+}
+
+if ($version != 0) {
+    print("$tool: version $toolversion\n");
+    exit 0;
+}
+
+if (! -e $get_maintainer) {
+    die "$tool: The tool requires $get_maintainer\n";
+}
+
+if (!$patch_dir) {
+    die "$tool: Directory -d|--patchdir not specified\n";
+}
+if (! -e $patch_dir) {
+    die "$tool: Directory $patch_dir does not exist\n";
+}
+if ($overwrite) {
+    $overwrite_ext = "";
+}
+
+# Get the list of patches
+my $pattern = $patch_dir.'/'.$patch_prefix.'*'.$patch_ext;
+my @patches = glob($pattern);
+my $has_cover_letter = 0;
+my $cover_letter_file;
+
+if (!scalar @patches) {
+    die "$tool: Directory $patch_dir contains no patches\n";
+}
+
+# Do the actual processing
+my $file;
+my @combined_to;
+my @combined_cc;
+
+foreach my $file (@patches){
+    if (index($file, $cover_letter) != -1) {
+        $has_cover_letter = 1;
+        $cover_letter_file = $file;
+    } else {
+        print "Processing: ".basename($file)."\n";
+        my $fh;
+        my $cmd = $get_maintainer." ".$get_maintainer_args." < ".$file;
+        my @to;
+        my @cc;
+
+        open($fh, "$cmd|")
+            or die "Failed to execute '$cmd'\n";
+        while(<$fh>) {
+            chomp;
+            # Keep lists and CC's separately as we dont want them in
+            # the commit message under a Cc: line
+            if (index($_, $mailing_lists) != -1) {
+                push @to, $TO.$_;
+                push @combined_to, $TO.$_;
+            } else {
+                push @cc, $CC.$_;
+                push @combined_cc, $CC.$_;
+            }
+        }
+        close $fh;
+        
+        my $to = join("\n", uniq @to)."\n";
+        my $cc = join("\n", uniq @cc)."\n";
+        
+        # Insert snippets into files
+        # $cc before "Signed-off-by:"
+        insert_before($file , $cc, $cc_insert_before, 1);
+        # $to before suitable header line
+        insert_before($file , $to, $to_insert_before, 0);
+    }
+}
+
+# Deal with the cover letter
+if ($has_cover_letter) {
+    my $to = join("\n", uniq @combined_to)."\n";
+    my $cc = join("\n", uniq @combined_cc)."\n";
+
+    print "Processing: ".basename($cover_letter_file)."\n";
+
+    # Insert snippets into files
+    # $to and $cc before suitable header line
+    insert_before($cover_letter_file, $to.$cc, $to_insert_before, 1);
+
+    print "\nDon't forget to add the subject and message to ".
+          $cover_letter_file.$overwrite_ext."\n";
+}
+
+print "Then perform:\n".
+      "git send-email -to xen-devel".$AT."lists.xenproject.org ".
+      $patch_dir.'/'.$patch_prefix."*.patch".$overwrite_ext."\n";
+
+exit 1;
+
+sub readfile {
+    my ($file) = @_;
+    my $fh;
+    my $content;
+    open($fh, "<", $file)
+         or die "Could not open file '$file' $!";
+    $content = do { local $/; <$fh> };
+    close $fh;
+    
+    return $content;
+}
+
+sub writefile {
+    my ($content, $file) = @_;
+    my $fh;
+    open($fh, ">", $file)
+         or die "Could not open file '$file' $!";
+    print $fh $content;
+    close $fh;
+    
+    return 1;
+}
+
+sub insert_before {
+    my ($file, $ins, $before, $first) = @_;
+    my $content;
+
+    # $first must always be set to one for the first
+    # in a series of multiple transformations. This is
+    # a little ugly, but allows us to handle backups
+    # and filename issues correctly.
+    
+    # Also, the different filename handling via
+    # $overwrite is entirely handled in this function.
+    if ($first) {
+        $content = readfile($file);
+        if ($overwrite) {
+            # Make a backup
+            writefile($content, $file."~");
+        }
+    } else {
+        # Read from the previously modified file
+        $content = readfile($file.$overwrite_ext);
+    }
+    
+    # Split the string and generate new content
+    my $i  = index($content, $before);
+    my $p1 = substr $content, 0, $i;
+    my $p2 = substr $content, $i;
+    #
+    writefile($p1.$ins.$p2, $file.$overwrite_ext);
+        
+    if ($verbose) {
+        print "\nInserted into ".basename($file).' before "$before"'.
+              "\n-----\n".$ins."-----\n";
+    }
+
+    return 1;
+}
Signature · 7 lines
2.13.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#2 lore raw

Added Anthony as he highlighted the issue which led to the creation of the tool
Otherwise I played a little bit more on various series I worked on in the past and there are minor niggles in its usage

On 27/04/2018, 10:31, "Lars Kurth" <[email protected]> wrote:

+OPTIONS:
+--------
+ --p|prefix (e.g. --prefix v2)
+ Choose patch files with a specific prefix

I am thinking of renaming this to --r|reroll-count=<number> as this would make the tool more consistent with git format-patch and would make automation easier

+ --a|args
+ Arguments passed on to $get_maintainer
+ --v|verbose
+ Show more output
+ --version
+ Show version
+ --o|overwrite
+ Overwrites *.patch file (but makes a backup), whereas teh default
+ creates a new *.mpatch file

I think I will make -o the default, as once you get used to the tool the renaming of files is a little annoying. Also, instead of using --nooverwrite, I was thinking of canning --overwrite altogether
and replacing it by --dry-run as this is consistent with git send-mail. In that case, the *.patchm files get created and can be looked at, by those which are less confident.

Maybe add a warning or a user prompt, when an existing *.patch~ file would be overwritten, to prevent accidental application of the tool (when that had already been done). Typically, this could happen when you store several patch versions in one directory and forget the --prefix option. This would also prompt a warning if someone had manually edited the file with an editor that creates a backup (which many do via ~).

Any views?

Cheers
Lars

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#3 lore raw

PATCH 2/2 Replace occurances of xen.org with xenproject.org

In reply to Lars Kurth

This is a general clean-up activity. It also avoids mails being sent to [email protected] and [email protected] when used with add_maintainers.pl/git send-email

Cc Andrew Cooper <[email protected]>
Cc George Dunlap <[email protected]>
Cc Ian Jackson <[email protected]>
Cc Jan Beulich <[email protected]>
Cc Julien Grall <[email protected]>
Cc Konrad Rzeszutek Wilk <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Tim Deegan <[email protected]>
Cc Wei Liu <[email protected]>
---
 MAINTAINERS | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)
MAINTAINERS +15 -15
index fab76b0af4..c419c0d37a 100644
@@ -18,7 +18,7 @@ trivial patch so apply some common sense.
 	and variable names.  These aren't as silly as they seem. One
 	job the maintainers do is to keep things looking the same.
 
-	PLEASE see http://wiki.xen.org/wiki/Submitting_Xen_Patches for
+	PLEASE see http://wiki.xenproject.org/wiki/Submitting_Xen_Patches for
 	hints on how to submit a patch to xen-unstable in a suitable
 	form.
 
@@ -46,7 +46,7 @@ trivial patch so apply some common sense.
 The policy for inclusion in a Xen stable release is different to that
 for inclusion in xen-unstable.
 
-Please see http://wiki.xen.org/wiki/Xen_Maintenance_Releases for more
+Please see http://wiki.xenproject.org/wiki/Xen_Maintenance_Releases for more
 information.
 
 Backport requests should be made on the [email protected]
@@ -167,7 +167,7 @@ ARM (W/ VIRTUALISATION EXTENSIONS) ARCHITECTURE
 M:	Stefano Stabellini <[email protected]>
 M:	Julien Grall <[email protected]>
 S:	Supported
-L:	[email protected]
+L:	[email protected]
 F:	docs/misc/arm/
 F:	xen/arch/arm/
 F:	xen/drivers/char/arm-uart.c
@@ -265,7 +265,7 @@ F:	docs/misc/kconfig{,-language}.txt
 F:	xen/tools/kconfig/
 
 KDD DEBUGGER
-M:	Tim Deegan <[email protected]>
+M:	Tim Deegan <[email protected]>
 S:	Odd Fixes
 F:	tools/debugger/kdd/
 
@@ -294,7 +294,7 @@ MINI-OS
 M:	Samuel Thibault <[email protected]>
 S:	Supported
 L:	[email protected]
-T:	git git://xenbits.xen.org/mini-os.git
+T:	git git://xenbits.xenproject.org/mini-os.git
 F:	config/MiniOS.mk
 
 OCAML TOOLS
@@ -307,7 +307,7 @@ OVMF UPSTREAM
 M:	Anthony PERARD <[email protected]>
 M:	Wei Liu <[email protected]>
 S:	Supported
-T:	git git://xenbits.xen.org/ovmf.git
+T:	git git://xenbits.xenproject.org/ovmf.git
 
 POWER MANAGEMENT
 M:	Jan Beulich <[email protected]>
@@ -331,13 +331,13 @@ F:	tools/python
 QEMU-DM
 M:	Ian Jackson <[email protected]>
 S:	Supported
-T:	git git://xenbits.xen.org/qemu-xen-traditional.git
+T:	git git://xenbits.xenproject.org/qemu-xen-traditional.git
 
 QEMU UPSTREAM
 M:	Stefano Stabellini <[email protected]>
 M:	Anthony Perard <[email protected]>
 S:	Supported
-T:	git git://xenbits.xen.org/qemu-xen.git
+T:	git git://xenbits.xenproject.org/qemu-xen.git
 
 REMUS
 M:	Shriram Rajagopalan <[email protected]>
@@ -365,7 +365,7 @@ F:	xen/common/sched*
 SEABIOS UPSTREAM
 M:	Wei Liu <[email protected]>
 S:	Supported
-T:	git git://xenbits.xen.org/seabios.git
+T:	git git://xenbits.xenproject.org/seabios.git
 
 STUB DOMAINS
 M:	Samuel Thibault <[email protected]>
@@ -402,13 +402,13 @@ F:	docs/misc/tmem*
 UNMODIFIED LINUX PV DRIVERS
 M:	Jan Beulich <[email protected]>
 S:	Obsolete
-L:	[email protected]
+L:	[email protected]
 F:	unmodified_drivers/linux-2.6/
 
 USB PV DRIVERS
 M:	Noboru Iwamatsu <[email protected]>
 S:	Supported
-T:	hg http://xenbits.xen.org/linux-2.6.18-xen.hg
+T:	hg http://xenbits.xenproject.org/linux-2.6.18-xen.hg
 F:	drivers/xen/usb*/
 
 VM EVENT, MEM ACCESS and MONITOR
@@ -445,7 +445,7 @@ X86 ARCHITECTURE
 M:	Jan Beulich <[email protected]>
 M:	Andrew Cooper <[email protected]>
 S:	Supported
-L:	[email protected]
+L:	[email protected]
 F:	xen/arch/x86/
 F:	xen/include/asm-x86/
 F:	xen/include/public/arch-x86/
@@ -483,7 +483,7 @@ F:	xen/arch/x86/mm/mem_sharing.c
 F:	tools/memshr
 
 X86 SHADOW PAGETABLES
-M:	Tim Deegan <[email protected]>
+M:	Tim Deegan <[email protected]>
 S:	Maintained
 F:	xen/arch/x86/mm/shadow/
 
@@ -515,9 +515,9 @@ M:	Jan Beulich <[email protected]>
 M:	Julien Grall <[email protected]>
 M:	Konrad Rzeszutek Wilk <[email protected]>
 M:	Stefano Stabellini <[email protected]>
-M:	Tim Deegan <[email protected]>
+M:	Tim Deegan <[email protected]>
 M:	Wei Liu <[email protected]>
-L:	[email protected]
+L:	[email protected]
 S:	Supported
 F:	*
 F:	*/
Signature · 7 lines
2.13.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#5 lore raw

Re: [PATCH v1 2/2] Replace occurances of xen.org with xenproject.org

On 27/04/2018, 10:40, "Wei Liu" <[email protected]> wrote:

On Fri, Apr 27, 2018 at 10:30:51AM +0100, Lars Kurth wrote:
> KDD DEBUGGER
> -M: Tim Deegan <[email protected]>
> +M: Tim Deegan <[email protected]>

I think Tim should choose which domain name he prefers.

Sure: I did a global search and replace. But one is just an alias of the other, but of course mail filters could be impacted. Lars

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#6 lore raw

Re: [PATCH v1 2/2] Replace occurances of xen.org with xenproject.org

In reply to Wei Liu
Quoted 6 lines · Wei Liu

Review

Quoted 3 lines · Lars Kurth

KDD DEBUGGER

(quoted diff) +1 -1
-M:	Tim Deegan <[email protected]>
+M:	Tim Deegan <[email protected]>

I think Tim should choose which domain name he prefers.

I would prefer to keep using @xen.org, thank you.

Cheers,

Tim.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#7 lore raw

Re: [PATCH v1 2/2] Replace occurances of xen.org with xenproject.org

On 27/04/2018, 15:22, "Tim Deegan" <[email protected]> wrote:

At 10:40 +0100 on 27 Apr (1524825602), Wei Liu wrote:
> On Fri, Apr 27, 2018 at 10:30:51AM +0100, Lars Kurth wrote:
> > KDD DEBUGGER
> > -M: Tim Deegan <[email protected]>
> > +M: Tim Deegan <[email protected]>
>
> I think Tim should choose which domain name he prefers.

I would prefer to keep using @xen.org, thank you.

Will fix that in the next version
Lars

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel