Allow hand-editing of patches before sending

From: Karl Hasselström <[email protected]>

This adds a new flag to 'stg mail', allowing the user to hand-edit each patch e-mail before it is sent, just like it is currently possible to edit the cover message.

---

Intended use is to allow the user to write comments like this, when there is only one patch and a cover mail would be overkill.

 stgit/commands/mail.py |   59 ++++++++++++++++++++++++++++--------------------
 1 files changed, 35 insertions(+), 24 deletions(-)

index 970ae3e..26c711d 100644
@@ -108,6 +108,9 @@ options = [make_option('-a', '--all',
            make_option('-e', '--edit',
                        help = 'edit the cover message before sending',
                        action = 'store_true'),
+           make_option('-E', '--edit-patches',
+                       help = 'edit each patch before sending',
+                       action = 'store_true'),
            make_option('-s', '--sleep', type = 'int', metavar = 'SECONDS',
                        help = 'sleep for SECONDS between e-mails sending'),
            make_option('--refid',

@@ -276,6 +279,34 @@ def __build_extra_headers():
 
     return headers
 
+def edit_message(msg):
+    fname = '.stgitmail.txt'
+
+    # create the initial file
+    f = file(fname, 'w')
+    f.write(msg)
+    f.close()
+
+    # the editor
+    if config.has_option('stgit', 'editor'):
+        editor = config.get('stgit', 'editor')
+    elif 'EDITOR' in os.environ:
+        editor = os.environ['EDITOR']
+    else:
+        editor = 'vi'
+    editor += ' %s' % fname
+
+    print 'Invoking the editor: "%s"...' % editor,
+    sys.stdout.flush()
+    print 'done (exit code: %d)' % os.system(editor)
+
+    # read the message back
+    f = file(fname)
+    msg = f.read()
+    f.close()
+
+    return msg
+
 def __build_cover(tmpl, total_nr, msg_id, options):
     """Build the cover message (series description) to be sent via SMTP
     """

@@ -326,30 +357,7 @@ def __build_cover(tmpl, total_nr, msg_id
               'supported in the patch template'
 
     if options.edit:
-        fname = '.stgitmail.txt'
-
-        # create the initial file
-        f = file(fname, 'w+')
-        f.write(msg)
-        f.close()
-
-        # the editor
-        if config.has_option('stgit', 'editor'):
-            editor = config.get('stgit', 'editor')
-        elif 'EDITOR' in os.environ:
-            editor = os.environ['EDITOR']
-        else:
-            editor = 'vi'
-        editor += ' %s' % fname
-
-        print 'Invoking the editor: "%s"...' % editor,
-        sys.stdout.flush()
-        print 'done (exit code: %d)' % os.system(editor)
-
-        # read the message back
-        f = file(fname)
-        msg = f.read()
-        f.close()
+        msg = edit_message(msg)
 
     return msg.strip('\n')
 

@@ -431,6 +439,9 @@ def __build_message(tmpl, patch, patch_n
         raise CmdException, 'Only "%(name)s" variables are ' \
               'supported in the patch template'
 
+    if options.edit_patches:
+        msg = edit_message(msg)
+
     return msg.strip('\n')
 

Applied it but with some conflicts since you have some other patches which I haven't merged yet (like QP encoding). I'll try to have a look at this QP encoding this week. The main problem is that Gnus shows plenty of "=20" instead of new lines and 'stg import' cannot cope with them.

Thanks.

That Gnus shows =20 suggests that I haven't got the QP encoding 100% right. I haven't had time to look at that yet, and it looks like you may beat me to it.

Karl Hasselström, [email protected]

Well, it might be just the way I read the list (via the gmane news gateway) but I have to run some tests. However, I'd like the message body to still be 8bit and the QP encoding optional (either command line option or [stgit] config variable).

I believe all the mails I send with mutt are QP-encoded, and they don't look funny, do they? (You'd see the separator just above my signature as "--=20" if that was the case; it has a trailing whitespace.)

Are you sure? Wouldn't it be better to teach StGIT to eat QP-encoded mails, and use 7bit encoding when the patch is all ascii? (And still have an option to force 8bit instead of QP if desired, of course.)

I'm really not comfortable with assuming all mail servers are 8-bit clean, when they're evidently not. :-)

Karl Hasselström, [email protected]

I just checked, and that one certainly was.

Karl Hasselström, [email protected]

Dear diary, on Fri, Nov 03, 2006 at 11:01:42AM CET, I got a letter where Karl Hasselström <[email protected]> said that...

Are you sure? As far as I can see, it's 8bit.

				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
#!/bin/perl -sp0777i<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<j]dsj
$/=unpack('H*',$_);$_=`echo 16dio\U$k"SK$/SM$n\EsN0p[lN*1

8bit here too.

Andreas Ericsson                   [email protected]
OP5 AB                             www.op5.se

Spooky. When I get copies of my own mails via the git list, they are QP-encoded; but when I bcc myself directly, I get them in 8bit. But you all say you get them still 8bit-encoded. I'm not sure I even _want_ to know what is happening there.

One thing is certain, though: When I sent 8bit-encoded patches from StGIT to this list, the vger mail server added headers saying that it didn't like 8bit, and had re-encoded the mail (to what, I don't recall just now). It warned that in doing so, it had to make assumptions about the charset used. The assumption it had made was that the text was latin1, which is not so good when it is in fact utf8.

This all just strengthens my belief that StGIT should go to great legths to avoid stepping on mail servers' toes.

Karl Hasselström, [email protected]

I wonder if this can be solved by simply reusing the machinery format-patch already has. If calling it as a standalone script does more unnecessary things than what StGIT wants, we should certainly be able to separate the only necessary part out to suit StGIT's needs.

I would like to keep the e-mail templates feature which is not available via format-patch. The reason is because I also send patches to the ARM Linux maintainer via a patch tracking system which expects the messages formatted in a strict certain way
(http://www.arm.linux.org.uk/developer/patches/info.php). It would probably be easier to fix StGIT than modifying format-patch.

Yes, I agree. Mail code tends to not be so beautiful that you want to maintain more copies of it than absolutely necessary.

Of course, the same reasoning applies when importing mails into stgit.

Karl Hasselström, [email protected]

fredag 03 november 2006 11:21 skrev Petr Baudis:

The patch mail was QP-encoded, not the other messasges in this thread.

Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: QUOTED-PRINTABLE

It reached me as quoted printable, with =20 on your signature intro.

-apw

The copy of your mail cc'ed to me reached me as 8bit, but the copy I got through the list reached me as QP.

As I said, I don't really want to know why. I just want my patches to pass through smtp hell unmangled. :-)

Karl Hasselström, [email protected]

Heh, well the copy thunderbird wrote directly to my mailbox is also 8bit. And thinking about it I've seen talk of MTA's not supporting 8bit xfers between themselves, so thats probabally when it gets switched.

If your ISP mailer doesn't answer with "8BITMIME" to your EHLO, your mail client is supposed to downgrade to a 7-bit format (or the same thing can happen anywhere in between on one of the other hops). Of course, that would be a really old and broken mailer, and if you find one of those at the ISP you use, you should probably switch ISPs.

Some other mailers have other issues, and qmail for example is apparently totally broken (it accepts 8-bit input, but cannot forward it properly to somebody who wants it converted to 7-bit).

And some of us use "fetchmail" and ask it do do mimedecode for us, so that even if it arrived as 7-bit crud, we don't have to see it. So the fact that some people see it as 8-bit can be because the hops to them didn't involve a broken remailer, but it could also be because something in between (like fetchmail) tries to fix it up after the fact.

That said: it tends to be unusual to see true 7-bit mailer setups these days.

So the most _likely_ cause of unnecessary 7-bit QP crud is not in fact a 7-bit mailserver, but usually just the mail client that is just lazy and just sends out everything as 7-bit because it's too bothersome to even look at whether the mail server supports 8-bit data. Check your Thunderbird settings - it's quite possible that there's a flag there somewhere to turn on 8-bit mail transfers.

So the right thing to do would be to teach StGIT to generate 8bit-encoded output, and trust the SMTP transfer path do mangle it correctly? (Hmm. No, since StGIT talks directly with the first SMTP server in the chain, it needs to be able to QP-encode the mail itself if necessary -- but it should seldom be necessary, then.)

In that case, the problem with the current implementation (without my patch applied) is likely to be that it fails to provide the headers needed for the SMTP path to be able to transform it losslessly.

[ ... digs through old mail ... ]

This is the interesting part of such a mail that was sent from my computer, passed through the list, and ended up back at my computer again:

Received: ([email protected]) by vger.kernel.org via listexpand

          id S1750700AbWJVMCV (ORCPT <rfc822;[email protected]>);
          Sun, 22 Oct 2006 08:02:21 -0400
  X-Warning: Original message contained 8-bit characters, however during
             the SMTP transport session the receiving system did not announce
             capability of receiving 8-bit SMTP (RFC 1651-1653), and as this
             message does not have MIME headers (RFC 2045-2049) to enable
             encoding change, we had very little choice.
  X-Warning: We ASSUME it is less harmful to add the MIME headers, and
             convert the text to Quoted-Printable, than not to do so,
             and to strip the message to 7-bits.. (RFC 1428 Appendix A)
  X-Warning: We don't know what character set the user used, thus we had to
             write these MIME-headers with our local system default value.
  MIME-Version: 1.0
  Content-Transfer-Encoding: QUOTED-PRINTABLE
  Content-Type: TEXT/PLAIN; charset=ISO-8859-1
  Received: ([email protected]) by vger.kernel.org id S1751780AbWJVMCV
          (ORCPT <rfc822;git-outgoing>); Sun, 22 Oct 2006 08:02:21 -0400

The mail server (vger talking to itself, if the Received: headers were added in order) complained that there were no MIME headers, so it had to guess the charset. As it happened, it guessed wrong, so the non-ascii characters of the mail body was garbled.

My stab at fixing this was to make StGIT QP-encode the mail from the outset, but if I've not misunderstood things yet again, the actual problem was that StGIT did not produce the right MIME headers.

Karl Hasselström, [email protected]

Right. You could even just consider it an error if the mailserver doesn't reply to EHLO with 8BITMIME, I really think it's that rare.

I _think_ it should be sufficient to just set the Content-Type and Content-Transfer-Encoding to say something like "text/plain; charset=UTF8" and "8bit" respectively. But somebody who know the SMTP rules better should check.

HOWEVER:

This does seem to say that somebody didn't even announce 8-bit capability in the first place. That's a zmailer error message, and it does imply that somebody was running a bad server.

That said, _if_ your message had had the proper mime-type specifiers, then zmailer would happily have QP-converted the message for you, so everything would have been fine.

vger itself? Strange.

Linus

Makes sense (unless the Python SMTP library can do this for us -- it would be impolite to refuse then).

These are the headers that StGIT uses without my patch -- that is, the headers used to get that error message:

Content-Type: text/plain; charset=utf-8; format=fixed
Content-Transfer-Encoding: 8bit

Which is obviously not good enough for some picky part of the SMTP chain.

E-mail seems to be like driving. You can't just follow the rules, you also have to be careful to make allowances for those who don't. :-(

Well, it added these headers:

MIME-Version: 1.0
Content-Transfer-Encoding: QUOTED-PRINTABLE
Content-Type: TEXT/PLAIN; charset=ISO-8859-1

Maybe MIME-Version was the only thing missing? I'll have to try.

It looks that way, but I don't know enough about mail servers to be very certain. The complete header of the message (in my git list mailbox) is this:

From [email protected] Sun Oct 22 14:02:35 2006
Received: from vger.kernel.org ([209.132.176.167])

	by diana.vm.bytemark.co.uk with esmtp (Exim 3.36 #1 (Debian))
	id 1Gbc2E-0005lw-00
	for <[email protected]>; Sun, 22 Oct 2006 13:02:35 +0100
Received: ([email protected]) by vger.kernel.org via listexpand
	id S1750700AbWJVMCV (ORCPT <rfc822;[email protected]>);
	Sun, 22 Oct 2006 08:02:21 -0400

X-Warning: Original message contained 8-bit characters, however during
the SMTP transport session the receiving system did not announce capability of receiving 8-bit SMTP (RFC 1651-1653), and as this message does not have MIME headers (RFC 2045-2049) to enable encoding change, we had very little choice.
X-Warning: We ASSUME it is less harmful to add the MIME headers, and
convert the text to Quoted-Printable, than not to do so, and to strip the message to 7-bits.. (RFC 1428 Appendix A)
X-Warning: We don't know what character set the user used, thus we had to
write these MIME-headers with our local system default value.
MIME-Version: 1.0
Content-Transfer-Encoding: QUOTED-PRINTABLE
Content-Type: TEXT/PLAIN; charset=ISO-8859-1
Received: ([email protected]) by vger.kernel.org id S1751780AbWJVMCV

	(ORCPT <rfc822;git-outgoing>); Sun, 22 Oct 2006 08:02:21 -0400
Received: from mxfep01.bredband.com ([195.54.107.70]:50054 "EHLO
	mxfep01.bredband.com") by vger.kernel.org with ESMTP
	id S1750700AbWJVMCU (ORCPT <rfc822;[email protected]>);
	Sun, 22 Oct 2006 08:02:20 -0400
Received: from ironport2.bredband.com ([195.54.107.84] [195.54.107.84])
          by mxfep01.bredband.com with ESMTP
          id <20061022120218.PHVT953.mxfep01.bredband.com@ironport2.bredband.com>
          for <[email protected]>; Sun, 22 Oct 2006 14:02:18 +0200
Received: from ua-83-227-180-148.cust.bredbandsbolaget.se (HELO yoghurt.hemma.treskal.com) ([83.227.180.148])
  by ironport2.bredband.com with ESMTP; 22 Oct 2006 14:02:18 +0200
Received: from [127.0.0.1] (localhost [127.0.0.1])
	by yoghurt.hemma.treskal.com (Postfix) with ESMTP id 1DFDF4C010;
	Sun, 22 Oct 2006 14:02:18 +0200 (CEST)

From: Karl =?utf-8?q?Hasselstr=C3=B6m?= <[email protected]>
Subject: [PATCH] RFC2047-encode email headers
Date: Sun, 22 Oct 2006 14:02:17 +0200
To: Catalin Marinas <[email protected]>
Cc: [email protected]
Message-Id: <20061022120217.7650.23715.stgit@localhost>
User-Agent: StGIT/0.11
Sender: [email protected]
Precedence: bulk
X-Mailing-List: [email protected]
X-Envelope-Username: kha-list-git
X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=0.94.4
X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on
diana.vm.bytemark.co.uk
X-Spam-Level:
X-Spam-Status: No, score=-0.3 required=5.0 tests=AWL,FORGED_RCVD_HELO
autolearn=disabled version=3.0.3

Status: RO
X-Status: A
Content-Length: 4148

Karl Hasselström, [email protected]

I think you are missing MIME-Version: header.

[...]

MIME-Version _is_ important. I remember that Outlook Express sent news messages without this header, and those posts were either mangled by news server, or news readers didn't parse MIME headers without MIME-Version:

Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

From: Karl Hasselström <[email protected]>

This is required by some mail servers that want to change the transfer encoding of the mail.

---

OK, this is a test to see if adding MIME-Version makes the problem go away. I don't have any other way to test this than sending a patch to the list. :-)

 stgit/commands/mail.py |   12 +++++-------
 1 files changed, 5 insertions(+), 7 deletions(-)

index 45f7438..c428d67 100644
@@ -271,13 +271,11 @@ def __get_signers_list(msg):
     return addr_list
 
 def __build_extra_headers():
-    """Build extra headers like content-type etc.
-    """
-    headers  = 'Content-Type: text/plain; charset=utf-8; format=fixed\n'
-    headers += 'Content-Transfer-Encoding: 8bit\n'
-    headers += 'User-Agent: StGIT/%s\n' % version.version
-
-    return headers
+    """Build extra headers like content-type etc."""
+    return ('MIME-Version: 1.0\n'
+            'Content-Type: text/plain; charset=utf-8; format=fixed\n'
+            'Content-Transfer-Encoding: 8bit\n'
+            'User-Agent: StGIT/%s\n' % version.version)
 
 def edit_message(msg):

Karl,

Thanks for digging into this problem and thanks to everyone else (especially Linus) for the detailed explanations.

I'll include these patches (maybe modified) and I'll try to fix "import" as well.

I re-implemented parts of the mail and import commands (inspiring from your patches). They now use the email Python package. The mail command encodes the body to 7 or 8bit depending on non-ascii characters. The headers are QP-encoded. The import command can decode messages properly and can also handle multipart e-mails.

I still have to write a test script but my simple tests showed that it works. Please let me know if there is anything wrong (especially with the QP-encoding of the mail headers).

Thanks.

Even though it turned out to be possible to use 8bit always?

Strictly speaking, it's rfc2047-encoded. I think the options are QP and base64.

Goodie.

I won't have time to check this out today (in fact, it may have to wait until the weekend). But when I do check it out, I might have time to write that test, so please holler if you start working on that.

Karl Hasselström, [email protected]

I just let the Python email package to the encoding and it first tries pure ascii and switches to 8bit if this fails. This package automatically adds the proper e-mail headers based on the encoding type. It can also do QP or base64 encodings (if someone complains of an old SMTP server, maybe I'll add a config option to force QP body).

It would be great if you can run a test script. I've been thinking more of an "import" test that also checks both the "export" and "mail" commands. BTW, I'll try to add support for mbox importing which is pretty easy with the email package.

Ah, OK, I misread you. This is precisely what I wanted.

Yes, but let's hope we don't have to. :-) And if we do have to, we should probably autodetect it from the SMTP dialog as Linus explained (or rather, we should hope that the library can do this for us).

I was thinking along these lines too; first make sure the importer works, by feeding it a number of hard-coded patches in different formats, and then make sure that the importer can read what the exporter produces. (With "exporter" meaning both "stg export" and "stg mail".)

I didn't originally plan to thest the actual e-mail sending, but now that I think of it, a simple script listening on a random port, pretending to be an SMTP daemon, should be quite simple to set up.

Sounds good. If you do it in time, I'll write a test for that too. ;-)

Karl Hasselström, [email protected]

I'm not sure it's worth the hassle. You could just generate an mbox file with "mail" and import it with "import --mbox" (it will hopefully be ready this week). Problems with SMTP would be noticed quickly anyway.

Tests for "stg import" and "stg mail". The later test is not very ambitious, but it can always be extended later.

Karl Hasselström, [email protected]

From: Karl Hasselström <[email protected]>

---

 t/t1800-import.sh         |   85 ++++++++++++++++++++++++++++++++++++++
 t/t1800-import/email-8bit |   37 ++++++++++++++++
 t/t1800-import/email-mbox |  102 +++++++++++++++++++++++++++++++++++++++++++++
 t/t1800-import/email-qp   |   37 ++++++++++++++++
 t/t1800-import/git-diff   |   20 +++++++++
 t/t1800-import/gnu-diff   |   19 ++++++++
 t/t1800-import/stg-export |   26 +++++++++++
 7 files changed, 326 insertions(+), 0 deletions(-)

new file mode 100755
index 0000000..0c1baa3
@@ -0,0 +1,85 @@
+#!/bin/sh
+# Copyright (c) 2006 Karl Hasselström
+test_description='Test the import command'
+. ./test-lib.sh
+
+test_expect_success \
+    'Initialize the StGIT repository' \
+    '
+    for x in {do,di,da}{be,bi,bo}{dam,dim,dum}; do
+      echo $x
+    done > foo.txt &&
+    git add foo.txt &&
+    git commit -a -m "initial version" &&
+    stg init
+    '
+
+test_expect_success \
+    'Apply a patch created with "git diff"' \
+    '
+    stg import ../t1800-import/git-diff &&
+    [ $(git cat-file -p $(stg id) \
+        | grep -c "tree e96b1fba2160890ff600b675d7140d46b022b155") == 1 ] &&
+    stg delete ..
+    '
+
+test_expect_success \
+    'Apply a patch created with GNU diff' \
+    '
+    stg import ../t1800-import/gnu-diff &&
+    [ $(git cat-file -p $(stg id) \
+        | grep -c "tree e96b1fba2160890ff600b675d7140d46b022b155") == 1 ] &&
+    stg delete ..
+    '
+
+test_expect_success \
+    'Apply a patch created with "stg export"' \
+    '
+    stg import ../t1800-import/stg-export &&
+    [ $(git cat-file -p $(stg id) \
+        | grep -c "tree e96b1fba2160890ff600b675d7140d46b022b155") == 1 ] &&
+    stg delete ..
+    '
+
+test_expect_success \
+    'Apply a patch from an 8bit-encoded e-mail' \
+    '
+    stg import -m ../t1800-import/email-8bit &&
+    [ $(git cat-file -p $(stg id) \
+        | grep -c "tree 030be42660323ff2a1958f9ee79589a4f3fbee2f") == 1 ] &&
+    [ $(git cat-file -p $(stg id) \
+        | grep -c "author Inge Ström <[email protected]>") == 1 ] &&
+    stg delete ..
+    '
+
+test_expect_success \
+    'Apply a patch from a QP-encoded e-mail' \
+    '
+    stg import -m ../t1800-import/email-qp &&
+    [ $(git cat-file -p $(stg id) \
+        | grep -c "tree 030be42660323ff2a1958f9ee79589a4f3fbee2f") == 1 ] &&
+    [ $(git cat-file -p $(stg id) \
+        | grep -c "author Inge Ström <[email protected]>") == 1 ] &&
+    stg delete ..
+    '
+
+test_expect_success \
+    'Apply several patches from an mbox file' \
+    '
+    stg import -M ../t1800-import/email-mbox &&
+    [ $(git cat-file -p $(stg id change-1) \
+        | grep -c "tree 401bef82cd9fb403aba18f480a63844416a2e023") == 1 ] &&
+    [ $(git cat-file -p $(stg id change-1) \
+        | grep -c "author Inge Ström <[email protected]>") == 1 ] &&
+    [ $(git cat-file -p $(stg id change-2) \
+        | grep -c "tree e49dbce010ec7f441015a8c64bce0b99108af4cc") == 1 ] &&
+    [ $(git cat-file -p $(stg id change-2) \
+        | grep -c "author Inge Ström <[email protected]>") == 1 ] &&
+    [ $(git cat-file -p $(stg id change-3) \
+        | grep -c "tree 166bbaf27a44aee21ba78c98822a741e6f7d78f5") == 1 ] &&
+    [ $(git cat-file -p $(stg id change-3) \
+        | grep -c "author Inge Ström <[email protected]>") == 1 ] &&
+    stg delete ..
+    '
+
+test_done

new file mode 100644
index 0000000..316fe27
@@ -0,0 +1,37 @@
+From: Inge =?utf-8?q?Str=C3=B6m?= <[email protected]>
+Subject: [PATCH] test patch
+To: Upstream <[email protected]>
+Date: Sat, 11 Nov 2006 11:58:14 +0100
+Message-ID: <20061111105814.23209.46952.stgit@localhost>
+User-Agent: StGIT/0.11
+MIME-Version: 1.0
+Content-Type: text/plain; charset="utf-8"
+Content-Transfer-Encoding: 8bit
+
+Signed-off-by: Inge Ström <[email protected]>
+---
+
+ foo.txt |    2 ++
+ 1 files changed, 2 insertions(+), 0 deletions(-)
+
+diff --git a/foo.txt b/foo.txt
+index ad01662..d3cd5b6 100644
+--- a/foo.txt
++++ b/foo.txt
+@@ -3,6 +3,7 @@ dobedim
+ dobedum
+ dobidam
+ dobidim
++pum-pöddelipåm
+ dobidum
+ dobodam
+ dobodim
+@@ -20,6 +21,7 @@ dabedam
+ dabedim
+ dabedum
+ dabidam
++pum-däddelidum
+ dabidim
+ dabidum
+ dabodam
+

new file mode 100644
index 0000000..1a77c57
@@ -0,0 +1,37 @@
+From: Inge =?utf-8?q?Str=C3=B6m?= <[email protected]>
+Subject: [PATCH] test patch
+To: Upstream <[email protected]>
+Date: Sat, 11 Nov 2006 11:58:14 +0100
+Message-ID: <20061111105814.23209.46952.stgit@localhost>
+User-Agent: StGIT/0.11
+MIME-Version: 1.0
+Content-Type: text/plain; charset="utf-8"
+Content-Transfer-Encoding: quoted-printable
+
+Signed-off-by: Inge Str=C3=B6m <[email protected]>
+---
+
+ foo.txt |    2 ++
+ 1 files changed, 2 insertions(+), 0 deletions(-)
+
+diff --git a/foo.txt b/foo.txt
+index ad01662..d3cd5b6 100644
+--- a/foo.txt
++++ b/foo.txt
+@@ -3,6 +3,7 @@ dobedim
+ dobedum
+ dobidam
+ dobidim
++pum-p=C3=B6ddelip=C3=A5m
+ dobidum
+ dobodam
+ dobodim
+@@ -20,6 +21,7 @@ dabedam
+ dabedim
+ dabedum
+ dabidam
++pum-d=C3=A4ddelidum
+ dabidim
+ dabidum
+ dabodam
+

new file mode 100644
index 0000000..642e11d
@@ -0,0 +1,20 @@
+diff --git a/foo.txt b/foo.txt
+index ad01662..d3cd5b6 100644
+--- a/foo.txt
++++ b/foo.txt
+@@ -3,6 +3,7 @@ dobedim
+ dobedum
+ dobidam
+ dobidim
++dabadadash
+ dobidum
+ dobodam
+ dobodim
+@@ -20,6 +21,7 @@ dabedam
+ dabedim
+ dabedum
+ dabidam
++dadadadash
+ dabidim
+ dabidum
+ dabodam

new file mode 100644
index 0000000..46b338a
@@ -0,0 +1,19 @@
+diff -Naur old/foo.txt new/foo.txt
+--- old/foo.txt	2006-11-11 11:26:18.000000000 +0100
++++ new/foo.txt	2006-11-11 11:25:18.000000000 +0100
+@@ -3,6 +3,7 @@
+ dobedum
+ dobidam
+ dobidim
++dabadadash
+ dobidum
+ dobodam
+ dobodim
+@@ -20,6 +21,7 @@
+ dabedim
+ dabedum
+ dabidam
++dadadadash
+ dabidim
+ dabidum
+ dabodam

new file mode 100644
index 0000000..ddcfc3e
@@ -0,0 +1,26 @@
+test patch
+---
+
+ foo.txt |    2 ++
+ 1 files changed, 2 insertions(+), 0 deletions(-)
+
+diff --git a/foo.txt b/foo.txt
+index ad01662..d3cd5b6 100644
+--- a/foo.txt
++++ b/foo.txt
+@@ -3,6 +3,7 @@ dobedim
+ dobedum
+ dobidam
+ dobidim
++dabadadash
+ dobidum
+ dobodam
+ dobodim
+@@ -20,6 +21,7 @@ dabedam
+ dabedim
+ dabedum
+ dabidam
++dadadadash
+ dabidim
+ dabidum

From: Karl Hasselström <[email protected]>

---

 t/t1900-mail.sh |   34 ++++++++++++++++++++++++++++++++++
 1 files changed, 34 insertions(+), 0 deletions(-)

new file mode 100755
index 0000000..6a950b4
@@ -0,0 +1,34 @@
+#!/bin/sh
+# Copyright (c) 2006 Karl Hasselström
+test_description='Test the mail command'
+. ./test-lib.sh
+
+test_expect_success \
+    'Initialize the StGIT repository' \
+    '
+    for i in 1 2 3 4 5; do
+      touch foo.txt &&
+      echo "line $i" >> foo.txt &&
+      git add foo.txt &&
+      git commit -a -m "Patch $i"
+    done &&
+    stg init &&
+    stg uncommit -n 5 foo
+    '
+
+test_expect_success \
+    'Put all the patches in an mbox' \
+    'stg mail --to="Inge Ström <[email protected]>" -a -m \
+       -t ../../templates/patchmail.tmpl > mbox0'
+
+test_expect_success \
+    'Import the mbox and compare' \
+    '
+    t1=$(git cat-file -p $(stg id) | grep ^tree)
+    stg pop -a &&
+    stg import -M mbox0 &&
+    t2=$(git cat-file -p $(stg id) | grep ^tree) &&
+    [ "$t1" == "$t2" ]
+    '
+

One potentially hazardous thing: you encode the mail before letting the user edit it (with the -e and -E switches). This means that the user can insert non-ascii characters in the body after you've already decided it's safe to use 7bit encoding. It also means that the user must be careful to rfc2047-encode any changes to the Subject: and
From: headers.

Karl Hasselström, [email protected]

I changed this to edit the mail before encoding it. You can no longer see the full headers (those added by StGIT) but even in a normal e-mail client you can only see the whole headers after sending it (and if you receive it back).

All right, it seems to have worked! When I send patches to myself, they arrive still 8bit-encoded, and when I get them through the list, they have been properly QP-encoded by whatever part of the chain didn't like 8bit.

Catalin, this should replace the QP-encoding patch. (But not the RFC2047-encoding patch -- the headers are required to always be ascii, if I've not misunderstood something.)

Thanks everyone!

Karl Hasselström, [email protected]

This e-mail, in gmail:

Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit

In Gnus (via gmane, no "=20" displayed though Gnus can handle this encoding correctly):

Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: QUOTED-PRINTABLE

It looks like the mail server re-encodes the message.

Well, I don't have any strong opinion here, as long as QP is more widely available. My impression with Gnus was that it cannot handle but I think there was an encoding problem.

The initial mail command implementation was inspired from sendpatchset but we should probably now use the email Python package and let it handle the encoding and decoding (for import).

Most probably, yes. Not supporting QP would count as a serious deficiency in an e-mail client.

Or try to use the git plumbing directly, if reasonable. Mail encoding and decoding isn't my idea of fun -- I just want it to not break. And I don't see much point in StGIT replicating core GIT functionality when there's no reason to.

Karl Hasselström, [email protected]

I understand this and I think we should get this fixed. In the meantime, you can always run git-format-patch/git-send-email on the StGIT patches (the id would be "patches/<branch>/<patch>"). On my side I just run git-am and "stg uncommit" or "assimilate".

Dear diary, on Fri, Nov 03, 2006 at 02:25:36PM CET, I got a letter where Catalin Marinas <[email protected]> said that...

git-am calls git-mailsplit and git-mailinfo, and those two tools will do most of the work (basically everything except the actual committing) for you, I use them in cg-patch -m and it should be fairly trivial to make a stg command reusing those too.

				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
#!/bin/perl -sp0777i<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<j]dsj
$/=unpack('H*',$_);$_=`echo 16dio\U$k"SK$/SM$n\EsN0p[lN*1

Yes, indeed, these are the commands I would use. I wouldn't go further in calling git-am or git-applymbox directly as I try not to use too many of the porcelain'ish git scripts.

Does that mean that "stg mail" with my QP patch generates output that even git-am can't read? I had guessed from what you said earlier that git-am _could_ read them.

Yes, aren't they handy sometimes? :-)

Karl Hasselström, [email protected]

(Yes, I'm way too lazy. I tested the "stg mail" output by mailing patches to myself and making sure that they looked good in my mail reader. And by looking at the raw output quite a bit.)

Karl Hasselström, [email protected]

Yes, git-am was able to apply them but after using Gnus to write them to an mbox file.

I changed your patch slightly to add "Content-Disposition: inline" and QUOTED-PRINTABLE instead of quoted-printable. It seems that Gnus can show it properly now (but import still fails, of course).

As I said in my reply to Junio, I really like to keep the e-mail templates functionality of StGIT. This is not available in git-format-patch.

Fair enough. How about e-mail importing -- is there a good reason to not use the git plumbing there?

Karl Hasselström, [email protected]