Skip to contentSkip to conversation

Reorder headers for `git imap-send` compatibility

PATCH In review Philippe Blain 2 participants 3 messages Aug 11, 2021 – Aug 26, 2021

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Reorder headers for `git imap-send` compatibility

When downloading messages from a public-inbox instance, reorder the "From", "Date" and "Subject" headers in that specific order. This allows the output from 'b4 mbox -o- $url' to be piped to 'git imap-send', which expects these headers in that specific order.

An easy place to implement that is in 'mailsplit_bytes', which is called by 'get_pi_thread_by_url' and already loops over the messages in the 'git mailsplit'-ed mbox file to populate the message list.

Note that 'mailsplit_bytes' is also called by 'get_msgs' directly when piping in a local mailbox, i.e. when '-' is used as the value of '--use-local-mbox', but it does no harm to also reorder the headers in that case.

---

Notes:
Hi, I suggested that feature in [1] last December and just recently took the time to implement it.

Konstantin suggested in his response to [1] to try using isync/mbsync instead, but I would prefer not having to change my workflow. I've been using 'git imap-send' for a while with my custom script [2] when hitting emails that were not created with 'git send-email'/'git format-patch' and I did not hit any corner case where 'git imap-send' was not able to send the email.

[1] https://lore.kernel.org/tools/[email protected]/
[2] https://gist.github.com/phil-blain/d350e91959efa6e7afce60e74bf7e4a8

 b4/__init__.py | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
b4/__init__.py +8 -1
index b45e26f..baea003 100644
@@ -2162,6 +2162,13 @@ def get_strict_thread(msgs, msgid):
 
     return strict
 
+def reorder_headers(msg):
+    """ Reorder "From", "Date" and "Subject" headers in the order `git imap-send` expects"""
+    for header in ['From', 'Date', 'Subject']:
+        header_content = msg[header]
+        del msg[header]
+        msg[header] = header_content
+    return msg
 
 def mailsplit_bytes(bmbox: bytes, outdir: str) -> list:
     logger.debug('Mailsplitting the mbox into %s', outdir)
@@ -2174,7 +2181,7 @@ def mailsplit_bytes(bmbox: bytes, outdir: str) -> list:
     # Read in the files
     for msg in os.listdir(outdir):
         with open(os.path.join(outdir, msg), 'rb') as fh:
-            msgs.append(email.message_from_binary_file(fh))
+            msgs.append(reorder_headers(email.message_from_binary_file(fh)))
     return msgs
 
 

base-commit: 45ef591d2fb245a06600514d8df57ab64d519fd8

Signature · 1 line
2.29.2
#1 lore raw
Quoted 5 lines · Philippe Blain

When downloading messages from a public-inbox instance, reorder the "From", "Date" and "Subject" headers in that specific order. This allows the output from 'b4 mbox -o- $url' to be piped to 'git imap-send', which expects these headers in that specific order.

Quoted 8 lines

An easy place to implement that is in 'mailsplit_bytes', which is called by 'get_pi_thread_by_url' and already loops over the messages in the 'git mailsplit'-ed mbox file to populate the message list.

Note that 'mailsplit_bytes' is also called by 'get_msgs' directly when piping in a local mailbox, i.e. when '-' is used as the value of '--use-local-mbox', but it does no harm to also reorder the headers in that case.

I've decided not to take this patch because it will rearrange headers on all processed messages just to work around a deficiency in a single tool (git-imap-send). The order of headers in an RFC-2822 message should not matter, so any tool that expects them in a specific arrangement is what needs fixing.

Furthermore, reordering headers may actually cause DKIM validation errors in the case where there are multiple From, Date or Subject headers. I know this sounds contrived, but I've learned not to consider anything too bizarre of a corner case when it comes to email.

For your particular case, I suggest you simply make a tiny python command that takes an mbox on stdin and spits it out on stdout with headers in the desired order, which should be about 20 lines of Python at most. This way you can do b4 mbox -o- $url | reorder.py | git imap-send.

-K

#2 lore raw

Hi Konstantin,

Le 2021-08-26 à 15:40, Konstantin Ryabitsev a écrit :

Quoted 21 lines · Philippe Blain
Quoted 5 lines

When downloading messages from a public-inbox instance, reorder the "From", "Date" and "Subject" headers in that specific order. This allows the output from 'b4 mbox -o- $url' to be piped to 'git imap-send', which expects these headers in that specific order.

Quoted 8 lines

An easy place to implement that is in 'mailsplit_bytes', which is called by 'get_pi_thread_by_url' and already loops over the messages in the 'git mailsplit'-ed mbox file to populate the message list.

Note that 'mailsplit_bytes' is also called by 'get_msgs' directly when piping in a local mailbox, i.e. when '-' is used as the value of '--use-local-mbox', but it does no harm to also reorder the headers in that case.

I've decided not to take this patch because it will rearrange headers on all processed messages just to work around a deficiency in a single tool (git-imap-send). The order of headers in an RFC-2822 message should not matter, so any tool that expects them in a specific arrangement is what needs fixing.

Quoted 4 lines

Furthermore, reordering headers may actually cause DKIM validation errors in the case where there are multiple From, Date or Subject headers. I know this sounds contrived, but I've learned not to consider anything too bizarre of a corner case when it comes to email.

Fair enough :)

Quoted 4 lines

For your particular case, I suggest you simply make a tiny python command that takes an mbox on stdin and spits it out on stdout with headers in the desired order, which should be about 20 lines of Python at most. This way you can do b4 mbox -o- $url | reorder.py | git imap-send.

That is what I'll do!

Thanks, Philippe.