git-send-email: Send with mutt(1)

I'm trying to use mutt(1) with git-send-email(1). Is that possible? I tried --sendmail-cmd=mutt, but am getting strange errors. The reason I want to send with mutt(1) is because it can encrypt and sign mail (with some tweaks), which git-send-mail(1) doesn't.

I think there's a lot of overlap between what git-send-email does and what mutt does, to the point that you probably don't need to use send-email at all.

I assume what you want out of send-email here is the actual generation of patch emails. But under the hood that is all done by git-format-patch anyway. So for example if you do:

  git format-patch --stdout origin..HEAD >patches
  mutt -f patches

And then you can use mutt's "resend-message" function to send each one. I use config like this:

macro index,pager b ":set edit_headers=yes<enter><resend-message>:set edit_headers=no<enter>"

If you're sending a long series, it's helpful to pre-populate various headers in the format-patch command with "--to", etc. I usually do so by sending the cover letter directly via mutt, and then using some perl hackery to convert those headers into format-patch args. The script I

use is below (it will also, when run without a terminal, generate the patch summary for the cover letter; I use it with "r!my-script" while writing the cover letter in vim).

(This script is what I use every day, so it should be fairly robust. But it is also over 15 years old, so I don't promise there isn't a simpler way to do some of what it does ;) ).

-- >8 --
#!/bin/sh
upstream_branch() {
current=git symbolic-ref HEAD
upstream=git for-each-ref --format='%(upstream)' "$current" if test -n "$upstream"; then
echo $upstream
else
echo origin
fi
}

get_reply_headers() {
perl -ne '

    if (defined $opt) {
      if (/^\s+(.*)/) {
        $val .= " $1";
        next;
      }
      print "--$opt=", quotemeta($val), " ";
      $opt = $val = undef;
    }
    if (/^(cc|to):\s*(.*)/i) {
      $opt = lc($1);
      $val = $2;
    }
    elsif (/^message-id:\s*(.*)/i) {
      $opt = "in-reply-to";
      $val = $1;
    }
    elsif (/^subject:\s*\[PATCH v(\d+)/i) {
      print "-v$1 ";
    }
    elsif (/^$/) {
      last;
    }

'
}

format_patch() {
git format-patch -s --stdout --from "$@"
}

has_nonoption=
for i in "$@"; do
case "$i" in

    -[0-9]*) has_nonoption=yes ;;
    -*) ;;
     *) has_nonoption=yes

esac
done

: ${REPLY:=$HOME/patch}
test -e "$REPLY" && eval "set -- get_reply_headers <\"$REPLY\" \"\$@\"" test "$has_nonoption" = "yes" || set -- "$@" upstream_branch

if test -t 1; then
format_patch "$@" >.mbox
mutt -e 'set sort=mailbox-order' -f .mbox
rm -f .mbox
else
format_patch "$@" |
perl -lne '

    if (/^Subject: (.*)/) {
      $subject = $1;
    }
    elsif ($subject && /^\s+(.*)/) {
      $subject .= " $1";
    }
    elsif ($subject) {
      print $subject;
      $subject = undef;
    }
  ' |
  sed -e 's/\[PATCH /[/' \
      -e 's/]/]:/' \
      -e 's/^/  /'

echo
format_patch --cover-letter "$@" |
sed -ne '/|/,/^$/p; /^-- /q'
fi

Quoted 3 lines

I assume what you want out of send-email here is the actual generation of patch emails. But under the hood that is all done by git-format-patch anyway. So for example if you do:

Yeah, most of it is done by format-patch. There are few things I actually need from send-email. One of them is generating the Cc from the sign-offs and other tags found in the patch.

I had been thinking these days that it would be useful to have format-patch generate those. How about adding a --signed-off-by-cc to format-patch?

Quoted 4 lines

If you're sending a long series, it's helpful to pre-populate various headers in the format-patch command with "--to", etc. I usually do so by sending the cover letter directly via mutt, and then using some perl hackery to convert those headers into format-patch args. The script I

Indeed, that hackery is what send-email already does, so how about moving those features a bit upstream so that format-patch can do them too?

Although then, maybe it's simpler to teach send-email to learn to use mutt(1) under the hood for the actual send.

Quoted 3 lines

I had been thinking these days that it would be useful to have format-patch generate those. How about adding a --signed-off-by-cc to format-patch?

That seems like a reasonable feature. Probably it should be --cc-from-trailer=signed-off-by, and then you could do the same with other trailers.

It feels like you could _almost_ do it with the existing --format='%(trailers)' functionality, but there's no way to say "do the regular --format=email output, but also stick this extra format in the headers section". Plus there are probably some niceties you'd get from Git knowing that you're adding headers (like de-duping addresses).

That feature might end up somewhat hairy, though, as then you get into questions of parsing address lists, etc. We do all that now in perl with send-email, where we can lean on some parsing libraries. So I dunno.

Quoted 9 lines

Quoted 5 lines

If you're sending a long series, it's helpful to pre-populate various headers in the format-patch command with "--to", etc. I usually do so by sending the cover letter directly via mutt, and then using some perl hackery to convert those headers into format-patch args. The script I

Indeed, that hackery is what send-email already does, so how about moving those features a bit upstream so that format-patch can do them too?

Yeah, if they existed in format-patch I might be able to reuse them. I am hesitant, though, just because handling all the corner cases on parsing is going to be a bit of new C code.

Quoted 2 lines

Although then, maybe it's simpler to teach send-email to learn to use mutt(1) under the hood for the actual send.

I think you will find some corner cases in trying to make mutt act just like an mta accepting delivery. Two I can think of:

1. It will take a body on stdin, but not a whole message. We can hack around that with some postponed-folder magic, though.

2. Bcc headers are stripped before sendmail sees the message (but those addresses appear on the command-line). Converting that back to bcc so that mutt can then re-strip them would be annoying but possible. If you don't use bcc, it probably makes sense to just punt on this.

So maybe a script like this:

-- >8 --
#!/bin/sh

# ignore arguments; mutt will parse them itself
# from to/cc headers. Note that we'll miss bcc this
# way, but handling that would probably be kind of
# tricky; we'd need to re-add those recipients as actual
# bcc headers so that mutt knows how to handle them.

# spool the message to a fake mbox; we need to add
# a "From" line to make it look legit
trap 'rm -f to-send' 0 &&
{
  echo "From whatever Mon Sep 17 00:00:00 2001" &&
  cat
} >to-send &&

# and then have mutt "resume" it. We have to redirect
# stdin back from the terminal, since ours is a pipe
# with the message contents.
mutt -p \
  -e 'set postponed=to-send' \
  -e 'set edit_headers=yes' \
  </dev/tty
-- 8< --

and then in your git config:

[sendemail]
sendmailcmd = /path/to/mutt-as-mta.sh

There are mutt-specific bits there that I don't think send-email should have to know about. Perhaps there are generic options that send-email could learn, but it really feels like you'd do better teaching mutt to be more ready to handle this (like taking a whole message on stdin, headers and all, rather than just a body).

-Peff

Quoted 3 lines

That seems like a reasonable feature. Probably it should be --cc-from-trailer=signed-off-by, and then you could do the same with other trailers.

That would work for me. I only suggested the other one because it's aleady in send-email in that form. But yeah, it might be useful to have finer control. In my case, something that CCs every mail in the trailer would work (Reviewed-by, Suggested-by, ... I want them all, always).

Quoted 7 lines

# spool the message to a fake mbox; we need to add
# a "From" line to make it look legit
trap 'rm -f to-send' 0 &&
{
  echo "From whatever Mon Sep 17 00:00:00 2001" &&
  cat
} >to-send &&

Would a named pipe work? Or maybe we could use $(mktemp)?

Quoted 2 lines

mutt -p \
-e 'set postponed=to-send' \

Huh, this is magic sauce! Works perfect for what I need. This would need to be packaged to the masses. :-)

I found a minor problem: If I ctrl+C within mutt(1), I expect it to cancel the last action, but this script intercepts the signal and exits. We would probably need to ignore SIGINT from mutt-as-mta.

Would you mind adding this as part of git? Or should we suggest the mutt project adding this script?

Quoted 9 lines

Quoted 7 lines

# spool the message to a fake mbox; we need to add
# a "From" line to make it look legit
trap 'rm -f to-send' 0 &&
{
  echo "From whatever Mon Sep 17 00:00:00 2001" &&
  cat
} >to-send &&

Would a named pipe work? Or maybe we could use $(mktemp)?

I suspect mutt wants it to be a real file. But yeah, mktemp would definitely work. I actually started to write it that way but switched to a static name for simplicity in demonstrating the idea. :)

One note, though. Later we need to pass this filename to mutt config:

Quoted 2 lines

Quoted 2 lines

mutt -p \
-e 'set postponed=to-send' \

so it's a potential worry if "mktemp" might use a path with spaces or funny characters (e.g., from $TMPDIR). Probably not much of a problem in practice, though.

Quoted 6 lines

Huh, this is magic sauce! Works perfect for what I need. This would need to be packaged to the masses. :-)

I found a minor problem: If I ctrl+C within mutt(1), I expect it to cancel the last action, but this script intercepts the signal and exits. We would probably need to ignore SIGINT from mutt-as-mta.

Yeah, that might make sense, and can be done with trap.

Quoted 3 lines

Would you mind adding this as part of git? Or should we suggest the mutt project adding this script?

IMHO it is a little too weird and user-specific to really make sense in either project. It's really glue-ing together two systems. And as it's not something I use myself, I don't plan it moving it further along. But you are welcome to take what I wrote and do what you will with it, including submitting it to mutt.

I used it for sending a couple of patches to linux-man@, and it seems to work. I don't have much experience with mutt, so maybe I'm missing some corner cases. Do you expect it to not work for some case? Otherwise, we might have a winner. :)

Quoted 8 lines

Quoted 2 lines

Would you mind adding this as part of git? Or should we suggest the mutt project adding this script?

IMHO it is a little too weird and user-specific to really make sense in either project. It's really glue-ing together two systems. And as it's not something I use myself, I don't plan it moving it further along. But you are welcome to take what I wrote and do what you will with it, including submitting it to mutt.

I'll start by creating a git repository in my own server, and will write something about it to let the public know about it. I'll also start requiring contributors to linux-man@ to sign their patches, and recommend them using this if they use mutt(1).

Quoted 4 lines · Alejandro Colomar

I used it for sending a couple of patches to linux-man@, and it seems to work. I don't have much experience with mutt, so maybe I'm missing some corner cases. Do you expect it to not work for some case? Otherwise, we might have a winner. :)

Since it's a Linux project, I suggest also checking out b4, which will do the mail sending for you as part of the contributor-oriented features:

https://b4.docs.kernel.org/en/latest/contributor/overview.html

We also provide a web relay for people who can't configure or use SMTP due to their company policies.

Quoted 9 lines

Quoted 8 lines

Quoted 2 lines

Would you mind adding this as part of git? Or should we suggest the mutt project adding this script?

IMHO it is a little too weird and user-specific to really make sense in either project. It's really glue-ing together two systems. And as it's not something I use myself, I don't plan it moving it further along. But you are welcome to take what I wrote and do what you will with it, including submitting it to mutt.

Quoted 4 lines

I'll start by creating a git repository in my own server, and will write something about it to let the public know about it. I'll also start requiring contributors to linux-man@ to sign their patches, and recommend them using this if they use mutt(1).

B4 will also sign your patches for you. ;)

-K

I haven't yet tried b4(1), and considered trying it some time ago, but really didn't find git-send-email(1) and mutt(1) so difficult to use that b4(1) would simplify much.

But I have tried patatt(1) before, which is what I think b4(1) uses for signing. Here are some of my concerns about patatt(4):

It doesn't sign the mail, but just the patch.

There's not much
difference, if any, in authenticability terms, but there's a big difference in usability terms:

To authenticate a given patch submitted to a mailing list, the receiver needs to also have patatt(1) configured. Otherwise, it looks like a random message.

MUAs normally don't show random headers (patatt(1)
signs by adding the signature header), so unless one is searching for that header, it will be ignored. This means, if I contribute to other projects, I need to tell them my patch is signed via patatt(1) in order for them to verify. If instead, I sign the email as usual with my MUA, every MUA will recognize the signature by default and show it to the reader.

It also doesn't allow encrypting mail, so let's say I send some patch fixing a security vulnerability, I'll need a custom tool to send it. If instead, I use mutt(1) to send it signed+encrypted to a mailing list that provides a PGP public key, I can reuse my usual tools.

Also, and I don't know if b4(1) handles this automagically, but AFAIR, patatt(1) didn't: fo signing a patch, I had to configure per-directory with patatt install-hook. I have more than a hundred git worktrees (think of dozens of git repositories, and half a dozen worktrees --see git-worktree(1)-- per repository). If I need to configure every one of those worktrees to sign all of my patches, that's going to be cumbersome. Also, I scrape and re-create worktrees for new branches all the time, so I'd need to be installing hooks for patatt(1) all the time. Compare that to having mutt(1) configured once. It doesn't scale that well.

Quoted 3 lines · Alejandro Colomar

I haven't yet tried b4(1), and considered trying it some time ago, but really didn't find git-send-email(1) and mutt(1) so difficult to use that b4(1) would simplify much.

Well, sending is only a small part of what b4 will do for you -- the core benefits are really cover letter management, automatic versioning and simplified list trailer collection. It's all tailored to kernel needs, but it will work for any project that depends on mailed patches.

Quoted 4 lines

But I have tried patatt(1) before, which is what I think b4(1) uses for signing. Here are some of my concerns about patatt(4):

It doesn't sign the mail, but just the patch.

Well, no, it signs the entire thing, not just the patch, but it's true that it's specifically targeted at patches (hence the name).

Quoted 7 lines

There's not much
difference, if any, in authenticability terms, but there's a big difference in usability terms:

To authenticate a given patch submitted to a mailing list, the receiver needs to also have patatt(1) configured. Otherwise, it looks like a random message.

Yes, but that's a feature.

Quoted 7 lines

MUAs normally don't show random headers (patatt(1)
signs by adding the signature header), so unless one is searching for that header, it will be ignored. This means, if I contribute to other projects, I need to tell them my patch is signed via patatt(1) in order for them to verify. If instead, I sign the email as usual with my MUA, every MUA will recognize the signature by default and show it to the reader.

I go into this in the FAQ for patatt:
https://github.com/mricon/patatt#why-not-simply-pgp-sign-all-patches

Basically, developers really hated getting patches signed with PGP, either inline or MIME, which is why it never took off. Putting it into the header where it's not seen except by specialized tooling was a design choice.

Quoted 5 lines

It also doesn't allow encrypting mail, so let's say I send some patch fixing a security vulnerability, I'll need a custom tool to send it. If instead, I use mutt(1) to send it signed+encrypted to a mailing list that provides a PGP public key, I can reuse my usual tools.

Right, the goal was really *just* attestation. For encrypted patch exchange we have remail (https://korg.docs.kernel.org/remail.html), which worked significantly better than any other alternative we've considered.

Quoted 10 lines

Also, and I don't know if b4(1) handles this automagically, but AFAIR, patatt(1) didn't: fo signing a patch, I had to configure per-directory with patatt install-hook. I have more than a hundred git worktrees (think of dozens of git repositories, and half a dozen worktrees --see git-worktree(1)-- per repository). If I need to configure every one of those worktrees to sign all of my patches, that's going to be cumbersome. Also, I scrape and re-create worktrees for new branches all the time, so I'd need to be installing hooks for patatt(1) all the time. Compare that to having mutt(1) configured once. It doesn't scale that well.

Also true -- patatt was really envisioned as a library for b4, where you can configure patch signing in your ~/.gitconfig for all projects.

-K

I've tried something even simpler:

---8<---
#!/bin/sh

mutt -H -;
--->8---

I used it for sending a couple of patches to linux-man@, and it seems to work. I don't have much experience with mutt, so maybe I'm missing some corner cases. Do you expect it to not work for some case? Otherwise, we might have a winner. :)

Quoted 5 lines · Alejandro Colomar

I've tried something even simpler:

---8<---
#!/bin/sh

Quoted 7 lines

mutt -H -;
--->8---

I used it for sending a couple of patches to linux-man@, and it seems to work. I don't have much experience with mutt, so maybe I'm missing some corner cases. Do you expect it to not work for some case? Otherwise, we might have a winner. :)

Wow, I don't know how I missed that when I read the manual. That was exactly the feature I was thinking that mutt would need. ;)

So yeah, that is obviously better than the "postponed" hackery I showed. I notice that "-H" even causes mutt to ignore "-i" (a sendmail flag that Git adds to sendemail.sendmailcmd). So you can just invoke it directly from your config like:

git config sendemail.sendmailcmd "mutt -H -"

Annoyingly, "-E" doesn't work when reading over stdin (I guess mutt isn't willing to re-open the tty itself). But if you're happy with not editing as they go through, then "-H" is then that's enough (in my workflow, I do the final proofread via mutt).

-Peff

Quoted 5 lines · Alejandro Colomar

I've tried something even simpler:

---8<---
#!/bin/sh

Quoted 7 lines

mutt -H -;
--->8---

I used it for sending a couple of patches to linux-man@, and it seems to work. I don't have much experience with mutt, so maybe I'm missing some corner cases. Do you expect it to not work for some case? Otherwise, we might have a winner. :)

Wow, I don't know how I missed that when I read the manual. That was exactly the feature I was thinking that mutt would need. ;)

So yeah, that is obviously better than the "postponed" hackery I showed. I notice that "-H" even causes mutt to ignore "-i" (a sendmail flag that Git adds to sendemail.sendmailcmd). So you can just invoke it directly from your config like:

git config sendemail.sendmailcmd "mutt -H -"

Annoyingly, "-E" doesn't work when reading over stdin (I guess mutt isn't willing to re-open the tty itself). But if you're happy with not editing as they go through, then "-H" is then that's enough (in my workflow, I do the final proofread via mutt).

-Peff

Hi Jeff,

Quoted 18 lines · Jeff King

Quoted 5 lines · Alejandro Colomar

I've tried something even simpler:

---8<---
#!/bin/sh

Quoted 7 lines

mutt -H -;
--->8---

I used it for sending a couple of patches to linux-man@, and it seems to work. I don't have much experience with mutt, so maybe I'm missing some corner cases. Do you expect it to not work for some case? Otherwise, we might have a winner. :)

Wow, I don't know how I missed that when I read the manual. That was exactly the feature I was thinking that mutt would need. ;)

Quoted 6 lines

So yeah, that is obviously better than the "postponed" hackery I showed. I notice that "-H" even causes mutt to ignore "-i" (a sendmail flag that Git adds to sendemail.sendmailcmd). So you can just invoke it directly from your config like:

git config sendemail.sendmailcmd "mutt -H -"

Having it directly in sendmailcmd causes some glitch: It repeats all CCs in TO. See a log:

Send this email? ([y]es|[n]o|[e]dit|[q]uit|[a]ll): y
OK. Log says:
Sendmail: mutt -H - -i [email protected] [email protected] [email protected] [email protected] [email protected]
From: Alejandro Colomar <[email protected]>
To: Kevin McCarthy <[email protected]>,
[email protected]
Cc: Alejandro Colomar <[email protected]>,
Jenya Sovetkin <[email protected]>,
[email protected]
Subject: [PATCH] send.c: Allow crypto operations in batch and mailx modes.
Date: Fri, 10 Nov 2023 01:41:24 +0100
Message-ID: <[email protected]>
X-Mailer: git-send-email 2.42.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Result: OK

The sent mail ended up being

From: Alejandro Colomar <[email protected]>
To: Kevin McCarthy <[email protected]>, [email protected], [email protected],
[email protected], [email protected]
Cc: Alejandro Colomar <[email protected]>,
Jenya Sovetkin <[email protected]>, [email protected]

So maybe we need the wrapper script to ignore the arguments.

Cheers,
Alex

Quoted 7 lines

Annoyingly, "-E" doesn't work when reading over stdin (I guess mutt isn't willing to re-open the tty itself). But if you're happy with not editing as they go through, then "-H" is then that's enough (in my workflow, I do the final proofread via mutt).

-Peff

Signature · 1 line

<https://www.alejandro-colomar.es/>

Quoted 2 lines

Having it directly in sendmailcmd causes some glitch: It repeats all CCs in TO. See a log:

Quoted 1 line

So maybe we need the wrapper script to ignore the arguments.

Heh! The following trick works as well, without needing a script:

[sendemail]

	sendmailcmd = mutt -H - && true

It probably relies too much on git-send-email(1)'s current implementation, but it works. :)

[sendemail]

	sendmailcmd = mutt -H - && true