Skip to contentSkip to conversation

footer isn't appended to multipart messages

DISCUSSION Casey Allen Shobe 7 participants 35 messages Jul 5, 2005 – Oct 11, 2005

lore
Share & tools
lore
#0 lore raw
Quoted 5 lines · Patrick Bennett

Ahah! You're right, the footer *is* there on the "Rich Text" formatted raw source, but is not appearing in the email viewer.
I assume this is because the footer is being appended in an
inappropriate place/way for an HTML mail, so the email viewer is not interpretting the footer properly. Will this be fixed?

Well, HTML mail is generally strongly discouraged on mailing lists. But it's not exactly "HTML mail", but any sort of MIME-compliant multipart message. In multipart messages, the mail contains more than one part. Typically that's one HTML version, and one text version of the message; but inconsiderate people send HTML-only messages, and multipart serves many other purposes.

Quoted 4 lines

It isn't acceptable, IMHO, for a list recipient/subscriber to get no ongoing notice of how to unsubscribe from the list... is there, perhaps, a subscription "reminder" function in mlmmj as there are in some other mailing list softwares?

There is only one thing more annoying on mailing lists than appending stupid user text to every single message, and that is the monthly password reminders that mailman sends out. I get spammed with about 50 of them at about the same time every month, so I wrote a filter to automatically delete them to retain sanity.

(Exception: your mailing lists are used for outgoing marketing purposes only, in which case you can add unsubscription instructions yourself since you're the only one sending mail.)

That being said, List-Subscribe, List-Unsubcribe, etc. headers should be present, and they don't appear to be on this list. Does mlmmj support them yet?

(this is not to say that your feature request/bug report is invalid
- it is in fact useful in some cases and should work as you expect, but the example you provided should be discouraged and it's also not the easiest thing to implement due to the varying nature of what the multipart messages might contain, how the HTML is structured, etc.)

Cheers,

Signature · 4 lines
Casey Allen Shobe | http://casey.shobe.info
[email protected] | cell 425-443-4653
AIM & Yahoo:  SomeLinuxGuy | ICQ:  1494523
SeattleServer.com, Inc. | http://www.seattleserver.com
#1 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Casey Allen Shobe wrote:

| There is only one thing more annoying on mailing lists than
| appending stupid user text to every single message, and that is the
| monthly password reminders that mailman sends out.  I get spammed
| with about 50 of them at about the same time every month, so I
| wrote a filter to automatically delete them to retain sanity.

Full ACK. I got flooded by them, too. Never mind, everything can be filtered ;)

| That being said, List-Subscribe, List-Unsubcribe, etc. headers
| should be present, and they don't appear to be on this list.  Does
| mlmmj support them yet?

Well, yes and no. Yes: you can add them to Custom headers file,
No: they're not added automaticly (maybe this could be implemented but i think it's ok if you run a lot of lists to generate the customheaders file by a script or so ;)

| (this is not to say that your feature request/bug report is invalid
| - it is in fact useful in some cases and should work as you expect,
| but the example you provided should be discouraged and it's also
| not the easiest thing to implement due to the varying nature of
| what the multipart messages might contain, how the HTML is
| structured, etc.)

correctly. I think the 'append a footer' feature is ok, too. Just disable html mail and stuff and it's nearly fine. The List- Headers should be always present (not hard to implement ;) Implementing appending of stuff for multipartmessages just suck cause you have so many ways clients do 'their' encoding and maybe some gateways who add shitty 'this is confidential mail' footers, too, so you may end up in a 4 or more part mail which only contains a one liner...

Regards,
Sven
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)

iD8DBQFCywUDQoCguWUBzBwRAtdcAJ9o5ozqcY/H69Sn0kJhvDekr/TcEwCeJESi bM+5Uw0+deHZHAnN30gddRw=EHej
-----END PGP SIGNATURE-----

#2 lore raw

Re: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe

----- Original Message -----
From: "Sven 'Darkman' Michels" <[email protected]>
To: <[email protected]>
Sent: Tuesday, July 05, 2005 3:09 PM
Subject: Re: footer isn't appended to multipart messages

Quoted 4 lines

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Casey Allen Shobe wrote:

Quoted 8 lines
| There is only one thing more annoying on mailing lists than
| appending stupid user text to every single message, and that is the
| monthly password reminders that mailman sends out.  I get spammed
| with about 50 of them at about the same time every month, so I
| wrote a filter to automatically delete them to retain sanity.

Full ACK. I got flooded by them, too. Never mind, everything can be filtered ;)

This is absolutely true- you (that is to say, *we*, that is to say techie/programmer types) find this stuff very annoying. However, mailing lists are used by knitting clubs too, and I, for one, DO NOT want the nightmare of administrating a knitting club mailing list that has absolutely no dependable notification of "how to do it yourself".

Quoted 12 lines

| That being said, List-Subscribe, List-Unsubcribe, etc. headers
| should be present, and they don't appear to be on this list.  Does
| mlmmj support them yet?

Well, yes and no. Yes: you can add them to Custom headers file,
No: they're not added automaticly (maybe this could be implemented but i think it's ok if you run a lot of lists to generate the customheaders file by a script or so ;)

Quoted 7 lines
| (this is not to say that your feature request/bug report is invalid
| - it is in fact useful in some cases and should work as you expect,
| but the example you provided should be discouraged and it's also
| not the easiest thing to implement due to the varying nature of
| what the multipart messages might contain, how the HTML is
| structured, etc.)

not sure which "example" you mean, but if it's monthly nags (a.k.a. mailing list subscribe/unsubscribe howto's), then, yes, it should be discouraged *for appropriate lists and when there is no reliable way to append footer text*. For other lists, on the contrary, it should be *encouraged* (see above)!

Quoted 9 lines

correctly. I think the 'append a footer' feature is ok, too. Just disable html mail and stuff and it's nearly fine. The List- Headers should be always present (not hard to implement ;) Implementing appending of stuff for multipartmessages just suck cause you have so many ways clients do 'their' encoding and maybe some gateways who add shitty 'this is confidential mail' footers, too, so you may end up in a 4 or more part mail which only contains a one liner...

1) I haven't found the documentation on how to enforce text-only mailing lists (it's one of the first bits of info i looked for when i figured out what the problem was; 2) not sure what happens on a text-only mlmmj list when a multi-part msg is sent, but whatever it is, for the "knitting club", it *can't* be a reject msg- a mailing list that fits the "knitting club's" needs (and I realize this may never be what mlmmj is, though I wish it could be, because I've very much liked the software otherwise) has to handle the default settings of widespread (though not necessarily popular with us programmer/techies) email clients (eg. outlook express).

Regards,

Quoted 2 lines

Sven

-Patrick

#3 lore raw

Re: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 6 lines · you

This is absolutely true- you (that is to say, *we*, that is to say techie/programmer types) find this stuff very annoying. However, mailing lists are used by knitting clubs too, and I, for one, DO NOT want the nightmare of administrating a knitting club mailing list that has absolutely no dependable notification of "how to do it yourself".

Reality check indeed: knitting club members aren't going to have a clue wtf the reminder mail or appended text means. They don't understand what "unsubscribing from a mailing list" is, they just use a particular address to talk to other club members. If they want off the list, no matter how much you spam them with instructions, they will ask the head of the knitting club to do it. I'm on some lists which append unsubscription messages to every mail, and guess what I see posted there all the time? empty messages with "unsubscribe" subjects (often with unsubscribe misspelled, not that it matters since that's not how the lists work).

Reality is, nontechy users have no interest in doing it themselves, and techy users know right well how to find out. Most anyone in between will look up the website where they signed up to try to figure it out.

Also, even though Outhouse Express is not yet one of them, many mail clients have built-in support for List-Unsubscribe header which enables a button for unsubscribing from the list.

Quoted 1 line

not sure which "example" you mean

The example you asked about: appending footers with subscribe/unsubscribe information.

Quoted 2 lines

For other lists, on the contrary, it should be *encouraged* (see above)!

I could not disagree with this statement more. You really think that when I (the idiot user in this example) feel like unsubscribing that I'm going to wait around for the next monthly mail or go digging around for an old one? No, I'm going to whine to the list owner or whomever I think is in charge if I can't figure things out, and probably just the whole list to make sure somebody hears me.

Quoted 2 lines

1) I haven't found the documentation on how to enforce text-only mailing lists

You could filter incoming messages with something like maildrop or your own script. Not sure how you'd go about mingling this with mlmmj, perhaps somebody else can answer that.

Quoted 3 lines

2) not sure what happens on a text-only mlmmj list when a multi-part msg is sent, but [it] has to handle the default settings of widespread [...] email clients [...].

Short answer, if you want HTML mail to reach the list, then don't block it. If you care about people seeing signatures on every single mail (even the HTML ones), then don't allow HTML. You can't (easily) have your cake and eat it too.

Long answer, you *can* have your cake and eat it too, but it will require substantial work on your part to reduce the messages (which I don't think you're actually willing to do or you might have just implemented multipart footer appending yourself):

* Outlook Express sends HTML and text multipart messages by default I believe. You could reduce this to plain text with some parsing, removing the HTML part and leaving the plain text.

* At least Thunderbird sends HTML-only mail by default. It's not that hard to go through and strip out HTML tags, since 99.9% of the time, end users never use any custom HTML anyways, and if they do, it's just a font or color and doesn't affect the legibility of the message (except for the worse).

* You will run into messages with lots of parts, where people think it's a good idea to tack on a VCF, HTML message, text message, separate parts for their spam filtering mechanism's signature, one for saying the mail passed their lame virus scan, and an image of the company's logo. Good luck with those.

Realistic answer, chances are you will never make your knitting club members happy, because the majority of them cannot and will not understand what a mailing list even is. You might come closest with the least effort by just going and signing up for yahoogroups, which even gives you the option to convert all incoming messages to HTML (*shudder*) and appends the dummy footer to every message, and gives them an ad-filled web-interface that they can use instead of email if they want to boot.

Another answer, if you just want a monthly/weekly/whatever reminder, just mail the list from a cron job.

Cheers,

Signature · 4 lines
Casey Allen Shobe | http://casey.shobe.info
[email protected] | cell 425-443-4653
AIM & Yahoo:  SomeLinuxGuy | ICQ:  1494523
SeattleServer.com, Inc. | http://www.seattleserver.com
#4 lore raw

Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe

oops, forgot to cc the list. apologies.

-Patrick

----- Original Message -----
From: "Patrick Bennett" <[email protected]>
To: "Casey Allen Shobe" <[email protected]>
Sent: Tuesday, July 05, 2005 11:14 PM
Subject: Re: footer isn't appended to multipart messages (part II: reality check!)

Quoted 148 lines

Alright, first, not intending to have started a flame war whatsoever, but.... well, anyway, hoping I'm being read in a tone that takes it down a notch and injects a bit of humor....

I think we (ie. "free/open source" software developers {you, I presume}, administrators {me} and users {lots of oft strident and fanatic folk out there}), often miss the boat by insisting our preferences must be foisted as "the one true way" on all the other innocent users out there (and by "users" I mean not just end-users, but purchasing decision makers, pointy haired bosses, systems administrators, etc.). This, imho, is one large factor that hinders open source software adoption by "the stupid users", the business world, pointy haired bosses, ad infinitum. Don't we come off as a bit kooky with such unbending sloganeering? We do propose to create in the bazaar, rather than the cathedral, no? I thought the point was to create high quality, highly flexible software and share it with the world for everyone's mutual benefit..... or maybe I got that wrong and it is instead to create a "free as in beer" fourth reich.

----- Original Message -----
From: "Casey Allen Shobe" <[email protected]>
To: <[email protected]>
Sent: Tuesday, July 05, 2005 7:38 PM
Subject: Re: footer isn't appended to multipart messages (part II: reality check!)

Quoted 8 lines · you
Quoted 6 lines

This is absolutely true- you (that is to say, *we*, that is to say techie/programmer types) find this stuff very annoying. However, mailing lists are used by knitting clubs too, and I, for one, DO NOT want the nightmare of administrating a knitting club mailing list that has absolutely no dependable notification of "how to do it yourself".

Quoted 12 lines

Reality check indeed: knitting club members aren't going to have a clue wtf the reminder mail or appended text means. They don't understand what "unsubscribing from a mailing list" is, they just use a particular address to talk to other club members. If they want off the list, no matter how much you spam them with instructions, they will ask the head of the knitting club to do it. I'm on some lists which append unsubscription messages to every mail, and guess what I see posted there all the time? empty messages with "unsubscribe" subjects (often with unsubscribe misspelled, not that it matters since that's not how the lists work).

You're right, my hypothetical "knitting club" was probably too far on the other end of the spectrum and thus a poor example, mea maxima culpa. My target users are actually in between- comfortable with technology, but not professionals nor geeks. They desire obviousness and convenience and more or less easily understand and are eager to follow a terse "unsubscribe footer" or regular reminder; but are frustrated by enforced ignorance. The damn shame of wasting a few hundred bits of bandwidth a day on a helpful (to them, but obviously not to you) footer, or a few hundred bits of bandwidth a month on a helpful (to them, but obviously not to you) how-to-be-a-good-mailing-listizen email is utterly lost on them. Some of them, only a few mind you, might even actually have a background jpg embedded into their email. {Gasp!}

Quoted 4 lines

Also, even though Outhouse Express is not yet one of them, many mail clients have built-in support for List-Unsubscribe header which enables a button for unsubscribing from the list.

I wasn't aware of this feature. Cool. I'll see if I can get it to work. Any further info on how to implement this?

Quoted 16 lines
Quoted 1 line

not sure which "example" you mean

The example you asked about: appending footers with subscribe/unsubscribe information.

Quoted 2 lines

For other lists, on the contrary, it should be *encouraged* (see above)!

I could not disagree with this statement more. You really think that when I (the idiot user in this example) feel like unsubscribing that I'm going to wait around for the next monthly mail or go digging around for an old one? No, I'm going to whine to the list owner or whomever I think is in charge if I can't figure things out, and probably just the whole list to make sure somebody hears me.

Quoted 7 lines
Quoted 2 lines

1) I haven't found the documentation on how to enforce text-only mailing lists

You could filter incoming messages with something like maildrop or your own script. Not sure how you'd go about mingling this with mlmmj, perhaps somebody else can answer that.

I had inferred (apparently incorrectly; a thousand apologies) from something another respondent had said that there was a mlmmj built-in method for enforcing text-only posting. Thus, my comment about not being able to find any documentation on it.

Quoted 9 lines
Quoted 3 lines

2) not sure what happens on a text-only mlmmj list when a multi-part msg is sent, but [it] has to handle the default settings of widespread [...] email clients [...].

Short answer, if you want HTML mail to reach the list, then don't block it. If you care about people seeing signatures on every single mail (even the HTML ones), then don't allow HTML. You can't (easily) have your cake and eat it too.

Quoted 5 lines

Long answer, you *can* have your cake and eat it too, but it will require substantial work on your part to reduce the messages (which I don't think you're actually willing to do or you might have just implemented multipart footer appending yourself):

If I understand correctly, and I may well be mistaken, other mailing list softwares out there in the open source world (gnu mailman comes to mind as an example), are able to handle appending footer messages to a wide variety of incoming email formats. No suggestion that, coding wise, this is accomplished *easily*, but just that it would be cool (and, in particular, useful to *me*) if mlmmj could do it as well. Besides, I doubt a corrupted final email (that is to say, a multi-part mime mail with a plain-text footer appended to it inappropriately) is the goal of the mlmmj crafters. If I were both 1) an experienced programmer, and 2) an expert in email protocol, formats, etc., you're right, I could implement this my own damned self, couldn't I?!? As it is, I've just wasted every one's time with my unworthyness and rubish opinions.

Quoted 3 lines

Another answer, if you just want a monthly/weekly/whatever reminder, just mail the list from a cron job.

Now *there* is a useful suggestion that I might well steal. Even though, again, I couldn't find any documentation of it, I had just hoped and half-way suspected (since it seems like a common feature, even if you personally are offended by it, in many mailing list managers) that mlmmj had something built-in to handle this already.

Quoted 8 lines

Cheers,

Signature · 4 lines
Casey Allen Shobe | http://casey.shobe.info
[email protected] | cell 425-443-4653
AIM & Yahoo:  SomeLinuxGuy | ICQ:  1494523
SeattleServer.com, Inc. | http://www.seattleserver.com

Eeek! Is that a foote... nay, inline-spam!!!?!??!??

-Patrick

#5 lore raw
In reply to Casey Allen Shobe
Quoted 8 lines · Sven 'Darkman' Michels
| That being said, List-Subscribe, List-Unsubcribe, etc. headers
| should be present, and they don't appear to be on this list.  Does
| mlmmj support them yet?

Well, yes and no. Yes: you can add them to Custom headers file,
No: they're not added automaticly (maybe this could be implemented but i think it's ok if you run a lot of lists to generate the customheaders file by a script or so ;)

It's meant to be added to customheaders. Plenty of people are using this for newsletters, etc. where they want control over stuff. If we add such stuff automatically people cannot take it away, while if we don't it's easy to add.

Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.
#6 lore raw

Re: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 8 lines · Patrick Bennett
Quoted 2 lines

Full ACK. I got flooded by them, too. Never mind, everything can be filtered ;)

This is absolutely true- you (that is to say, *we*, that is to say techie/programmer types) find this stuff very annoying. However, mailing lists are used by knitting clubs too, and I, for one, DO NOT want the nightmare of administrating a knitting club mailing list that has absolutely no dependable notification of "how to do it yourself".

If your users are that clueless I'm not sure mlmmj is the right list to use.

Quoted 17 lines
Quoted 10 lines
| That being said, List-Subscribe, List-Unsubcribe, etc. headers
| should be present, and they don't appear to be on this list.  Does
| mlmmj support them yet?

Well, yes and no. Yes: you can add them to Custom headers file,
No: they're not added automaticly (maybe this could be implemented but i think it's ok if you run a lot of lists to generate the customheaders file by a script or so ;)

Quoted 6 lines
| (this is not to say that your feature request/bug report is invalid
| - it is in fact useful in some cases and should work as you expect,
| but the example you provided should be discouraged and it's also
| not the easiest thing to implement due to the varying nature of
| what the multipart messages might contain, how the HTML is
| structured, etc.)
Quoted 5 lines

not sure which "example" you mean, but if it's monthly nags (a.k.a. mailing list subscribe/unsubscribe howto's), then, yes, it should be discouraged *for appropriate lists and when there is no reliable way to append footer text*. For other lists, on the contrary, it should be *encouraged* (see above)!

If you want monthly notification mails, please consider using mailman. mlmmj is not going to have it.

Quoted 9 lines
Quoted 8 lines

correctly. I think the 'append a footer' feature is ok, too. Just disable html mail and stuff and it's nearly fine. The List- Headers should be always present (not hard to implement ;) Implementing appending of stuff for multipartmessages just suck cause you have so many ways clients do 'their' encoding and maybe some gateways who add shitty 'this is confidential mail' footers, too, so you may end up in a 4 or more part mail which only contains a one liner...

Quoted 9 lines

1) I haven't found the documentation on how to enforce text-only mailing lists (it's one of the first bits of info i looked for when i figured out what the problem was; 2) not sure what happens on a text-only mlmmj list when a multi-part msg is sent, but whatever it is, for the "knitting club", it *can't* be a reject msg- a mailing list that fits the "knitting club's" needs (and I realize this may never be what mlmmj is, though I wish it could be, because I've very much liked the software otherwise) has to handle the default settings of widespread (though not necessarily popular with us programmer/techies) email clients (eg. outlook express).

One of the very first examples in README.access:

deny !^Content-Type: text/plain
allow

Put that in DIR/control/access, and a reject mail will be sent for everyone not sending a text/plain message. Now whether or not this reject mails fit the knitting club is up to you. Every list has it's own text dir, so you can simply modify DIR/text/access to fit what you need. Might even carry intructions on how to turn off html mail in outlook if you want. Or a link to a webpage explainging it.

mlmmj allows for heavy customization.

Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.
#7 lore raw

Re: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 6 lines · Casey Allen Shobe
Quoted 2 lines

1) I haven't found the documentation on how to enforce text-only mailing lists

You could filter incoming messages with something like maildrop or your own script. Not sure how you'd go about mingling this with mlmmj, perhaps somebody else can answer that.

http://mlmmj.mmj.dk/files/README.access has it, as mentioned in another reply.

Quoted 2 lines

Another answer, if you just want a monthly/weekly/whatever reminder, just mail the list from a cron job.

Yeah, couln't agree more.

Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.
#8 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 2 lines · Patrick Bennett

I thought the point was to create high quality, highly flexible software and share it with the world for everyone's mutual benefit.....

No no no, that's a byproduct. The reason for free software is to give other people the OPTION to make the software fit their particular purpose. And when you have done that (made the software fit your purpose), please send us the patch. We would appreciate that.

Morten

Signature · 2 lines
Morten K. Poulsen <[email protected]>
http://www.afdelingp.dk/
#9 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality

In reply to Casey Allen Shobe
Quoted 1 line · Patrick Bennett

oops, forgot to cc the list. apologies.

you could have sent the same mail again to the list, so threading and quoting wouldn't have been broken.

Quoted 6 lines
Quoted 6 lines

If I understand correctly, and I may well be mistaken, other mailing list softwares out there in the open source world (gnu mailman comes to mind as an example), are able to handle appending footer messages to a wide variety of incoming email formats. No suggestion that, coding wise, this is accomplished *easily*, but just that it would be cool (and, in particular, useful to *me*) if mlmmj could do it as

After playing around a little with mailman, I think it's not too hard to do it, you just have to handle the possible cases correctly.

Mads, would you accept a patch?

#10 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 11 lines · Jakob Hirsch
Quoted 6 lines
Quoted 6 lines

If I understand correctly, and I may well be mistaken, other mailing list softwares out there in the open source world (gnu mailman comes to mind as an example), are able to handle appending footer messages to a wide variety of incoming email formats. No suggestion that, coding wise, this is accomplished *easily*, but just that it would be cool (and, in particular, useful to *me*) if mlmmj could do it as

After playing around a little with mailman, I think it's not too hard to do it, you just have to handle the possible cases correctly.

Mads, would you accept a patch?

Of course :-)

Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.
#11 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality

In reply to Casey Allen Shobe
Quoted 2 lines · Mads Martin Joergensen
Quoted 1 line

Mads, would you accept a patch?

Of course :-)

Hah, I was not so sure, the patch is somewhat invasive and I'm not totally happy with all the things, but it works and I think it takes measures not to break things. Corrections and suggestions welcome.

src/do_all_the_voodo_here.c +143 -11
@@ -82,6 +82,9 @@
 	char *hdrline, *subject, *unqp;
 	int hdrsadded = 0;
 	int subject_present = 0;
+	int mime = 0;
+	char *content_orig = NULL;  /* original content-type header */
+	char *boundary = NULL;
 
 	allhdrs->count = 0;
 	allhdrs->strs = NULL;
@@ -160,6 +163,51 @@
 			}
 		}
 		
+		/* Content-Type */
+		if(footfd>=0 && !strncasecmp(hdrline, "Content-Type:", 13)) {
+			char* p;
+			int len;
+			/* search first non-LWSP */
+			for (p = hdrline+13; *p==' ' || *p=='\t'; p++);
+
+			if (!strncasecmp(p, "multipart/mixed", 15)) {
+				mime = 1;
+                
+				/* get boundary string */
+				p = strstr(p, "boundary");
+				p = index(p, '=');
+				p = index(p, '"') + 1;
+				if (p == NULL) {
+	                log_error(LOG_ARGS, "MIME multipart but no boundary");
+	                mime = 0;
+				} else {
+					/* find closing quote */
+					/* for(len=0; p[len]!='"' && p[len]!=0; ++len); */
+					len = index(p, '"') - p;
+					if(p[len] == 0) {
+	                    log_error(LOG_ARGS, "no closing quote in boundary");
+	                    mime = 0;
+					} else {
+						boundary = mymalloc(len + 1);
+						strncpy(boundary, p, len);
+						boundary[len] = 0;
+					}
+				}
+			} else if (!strncasecmp(p, "multipart/alternative", 21)) {
+				mime = 2;
+				/* create new boundary for MIME-encapsulation */
+				boundary = random_str();
+				
+				/* save for later use */
+				content_orig = hdrline; 
+				/* and create new content-type header */
+				hdrline = concatstr(3,
+					"Content-Type: multipart/mixed; boundary=\"", 
+				    boundary, "\"\n");
+
+			}
+		}
+
 		/* Should it be stripped? */
 		if(delhdrs) {
 			if(!findit(hdrline, delhdrs))
@@ -171,18 +219,102 @@
 		myfree(hdrline);
 	}
 
-	/* Just print the rest of the mail */
-	if(dumpfd2fd(infd, outfd) < 0) {
-		log_error(LOG_ARGS, "Error when dumping rest of mail");
-		return -1;
-	}
+    /* MIME-Handling */
+	switch (mime) {
+		char* s;
+		unsigned int i;
+		
+		case 0 : /* no MIME, plain message */
+            log_error(LOG_ARGS, "no MIME, plain message"); /* debug */
+			/* Just print the rest of the mail */
+			if(dumpfd2fd(infd, outfd) < 0) {
+				log_error(LOG_ARGS, "Error when dumping rest of mail");
+				return -1;
+			}
 
-	/* No more, lets add the footer if one */
-	if(footfd >= 0)
-		if(dumpfd2fd(footfd, outfd) < 0) {
-			log_error(LOG_ARGS, "Error when adding footer");
-			return -1;
-		}
+			/* No more, lets add the footer if one */
+			if(footfd >= 0)
+				if(dumpfd2fd(footfd, outfd) < 0) {
+					log_error(LOG_ARGS, "Error when adding footer");
+					return -1;
+				}
+				
+			break;
+			
+		case 1 : /* multipart/mixed */
+
+			if(dumpfd2fd(infd, outfd) < 0) {
+				log_error(LOG_ARGS, "Error when dumping rest of mail");
+				return -1;
+			}
+
+			/* seek back to the end of the close-delimiter to kill
+			  the last two hyphens. hopefully nobody puts too
+			  much garbage in the epilogue or else this will fail... */
+			i = 2 * strlen(boundary) - 1;
+			lseek(outfd, -i, SEEK_END);
+			s = mymalloc(i+1);
+			read(outfd, s, i);
+			s[i] = 0;
+			i -= strstr(s, boundary) - s + strlen(boundary);
+			myfree(s);
+			if (i>2*strlen(boundary)) {
+				log_error(LOG_ARGS, "Error seeking MIME close-delimiter");
+			} else {
+				lseek(outfd, -i, SEEK_END);
+
+				s = concatstr(2, "\nContent-Type: text/plain\n",
+					"Content-Disposition: inline\n\n");
+				writen(outfd, s, strlen(s));
+				myfree(s);
+
+				if(dumpfd2fd(footfd, outfd) < 0) {
+					log_error(LOG_ARGS, "Error when adding footer");
+					myfree(boundary);
+					return -1;
+				}
+
+				s = concatstr(3, "\n--", boundary, "--\n");
+				writen(outfd, s, strlen(s));
+				myfree(s);
+			}
+			myfree(boundary);
+
+			break;
+
+		case 2 : /* multipart/alternative */
+			s = concatstr(5,
+				"This is a multi-part message in MIME format.\n\n--",
+				boundary, "\n",
+				content_orig, "\n");
+			myfree(content_orig);
+			writen(outfd, s, strlen(s));
+			myfree(s);
+
+			if(dumpfd2fd(infd, outfd) < 0) {
+				log_error(LOG_ARGS, "Error when dumping rest of mail");
+				return -1;
+			}
+
+		    s = concatstr(4,
+		        "\n--", boundary,
+		        "\nContent-Type: text/plain\n",
+		        "Content-Disposition: inline\n\n");
+			writen(outfd, s, strlen(s));
+			myfree(s);
+			
+			if(dumpfd2fd(footfd, outfd) < 0) {
+				log_error(LOG_ARGS, "Error when adding footer");
+				return -1;
+			}
+
+			s = concatstr(3, "\n--", boundary, "--\n");
+			myfree(boundary);
+			writen(outfd, s, strlen(s));
+			myfree(s);
+
+			break;
+	}
 
 	fsync(outfd);
 
#12 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 8 lines · Jakob Hirsch
Quoted 2 lines · Mads Martin Joergensen
Quoted 1 line

Mads, would you accept a patch?

Of course :-)

Hah, I was not so sure, the patch is somewhat invasive and I'm not totally happy with all the things, but it works and I think it takes measures not to break things. Corrections and suggestions welcome.

Thanks for the patch!

I'll have a look next week, when I'm back from vacation.

Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.
#13 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality

In reply to Casey Allen Shobe
Quoted 1 line · Mads Martin Joergensen

Thanks for the patch!

de nada.
But better take the attached one. :)

Quoted 1 line

I'll have a look next week, when I'm back from vacation.

Have a nice holiday!

src/do_all_the_voodo_here.c +146 -11
@@ -82,6 +82,9 @@
 	char *hdrline, *subject, *unqp;
 	int hdrsadded = 0;
 	int subject_present = 0;
+	int mime = 0;
+	char *content_orig = NULL;  /* original content-type header */
+	char *boundary = NULL;
 
 	allhdrs->count = 0;
 	allhdrs->strs = NULL;
@@ -160,6 +163,55 @@
 			}
 		}
 		
+		/* Content-Type */
+		if(footfd>=0 && !mime && !strncasecmp(hdrline, "Content-Type:", 13)) {
+			char* p;
+			int len;
+			/* search first non-LWSP */
+			for (p = hdrline+13; *p==' ' || *p=='\t'; p++);
+
+			if (!strncasecmp(p, "multipart/mixed", 15)) {
+				mime = 1;
+                
+				/* get boundary string */
+				p = index(p, ';');
+				if (p != NULL)
+					p = strstr(p, "boundary");
+				if (p != NULL)
+					p = index(p, '=');
+				if (p != NULL)
+					p = index(p, '"') + 1;
+				if (p == NULL) {
+	                log_error(LOG_ARGS, "MIME multipart but no boundary");
+	                mime = 0;
+				} else {
+					/* find closing quote */
+					for(len=0; p[len]!='"' && p[len]!=0; ++len);
+					if(p[len] == 0) {
+	                    log_error(LOG_ARGS, "no closing quote in boundary");
+	                    mime = 0;
+					} else {
+						boundary = mymalloc(len + 1);
+						strncpy(boundary, p, len);
+						boundary[len] = 0;
+					}
+				}
+			} else if (!strncasecmp(p, "multipart/alternative", 21)) {
+				mime = 2;
+
+				/* create new boundary for MIME-encapsulation */
+				boundary = random_str();
+				
+				/* save for later use */
+				content_orig = hdrline; 
+				/* and create new content-type header */
+				hdrline = concatstr(3,
+					"Content-Type: multipart/mixed; boundary=\"", 
+				    boundary, "\"\n");
+
+			}
+		}
+
 		/* Should it be stripped? */
 		if(delhdrs) {
 			if(!findit(hdrline, delhdrs))
@@ -171,18 +223,101 @@
 		myfree(hdrline);
 	}
 
-	/* Just print the rest of the mail */
-	if(dumpfd2fd(infd, outfd) < 0) {
-		log_error(LOG_ARGS, "Error when dumping rest of mail");
-		return -1;
-	}
+    /* MIME-Handling */
+	switch (mime) {
+		char* s;
+		unsigned int i;
+		
+		case 0 : /* no MIME, plain message */
+			/* Just print the rest of the mail */
+			if(dumpfd2fd(infd, outfd) < 0) {
+				log_error(LOG_ARGS, "Error when dumping rest of mail");
+				return -1;
+			}
 
-	/* No more, lets add the footer if one */
-	if(footfd >= 0)
-		if(dumpfd2fd(footfd, outfd) < 0) {
-			log_error(LOG_ARGS, "Error when adding footer");
-			return -1;
-		}
+			/* No more, lets add the footer if one */
+			if(footfd >= 0)
+				if(dumpfd2fd(footfd, outfd) < 0) {
+					log_error(LOG_ARGS, "Error when adding footer");
+					return -1;
+				}
+				
+			break;
+			
+		case 1 : /* multipart/mixed */
+
+			if(dumpfd2fd(infd, outfd) < 0) {
+				log_error(LOG_ARGS, "Error when dumping rest of mail");
+				return -1;
+			}
+
+			/* seek back to the end of the close-delimiter to kill
+			  the last two hyphens. hopefully nobody puts too
+			  much garbage in the epilogue or else this will fail... */
+			i = 2 * strlen(boundary) - 1;
+			lseek(outfd, -i, SEEK_END);
+			s = mymalloc(i+1);
+			read(outfd, s, i);
+			s[i] = 0;
+			i -= strstr(s, boundary) - s + strlen(boundary);
+			myfree(s);
+			if (i>2*strlen(boundary)) {
+				log_error(LOG_ARGS, "Error seeking MIME close-delimiter");
+			} else {
+				lseek(outfd, -i, SEEK_END);
+
+				s = concatstr(2, "\nContent-Type: text/plain\n",
+					"Content-Disposition: inline\n\n");
+				writen(outfd, s, strlen(s));
+				myfree(s);
+
+				if(dumpfd2fd(footfd, outfd) < 0) {
+					log_error(LOG_ARGS, "Error when adding footer");
+					myfree(boundary);
+					return -1;
+				}
+
+				s = concatstr(3, "\n--", boundary, "--\n");
+				writen(outfd, s, strlen(s));
+				myfree(s);
+			}
+			myfree(boundary);
+
+			break;
+
+		case 2 : /* multipart/alternative */
+			s = concatstr(5,
+				"This is a multi-part message in MIME format.\n\n--",
+				boundary, "\n",
+				content_orig, "\n");
+			myfree(content_orig);
+			writen(outfd, s, strlen(s));
+			myfree(s);
+
+			if(dumpfd2fd(infd, outfd) < 0) {
+				log_error(LOG_ARGS, "Error when dumping rest of mail");
+				return -1;
+			}
+
+		    s = concatstr(4,
+		        "\n--", boundary,
+		        "\nContent-Type: text/plain\n",
+		        "Content-Disposition: inline\n\n");
+			writen(outfd, s, strlen(s));
+			myfree(s);
+			
+			if(dumpfd2fd(footfd, outfd) < 0) {
+				log_error(LOG_ARGS, "Error when adding footer");
+				return -1;
+			}
+
+			s = concatstr(3, "\n--", boundary, "--\n");
+			myfree(boundary);
+			writen(outfd, s, strlen(s));
+			myfree(s);
+
+			break;
+	}
 
 	fsync(outfd);
 
#14 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 10 lines · Jakob Hirsch
Quoted 1 line · Mads Martin Joergensen

Thanks for the patch!

de nada.
But better take the attached one. :)

Quoted 1 line

I'll have a look next week, when I'm back from vacation.

Have a nice holiday!

Thanks :-)

src/do_all_the_voodo_here.c line 87

Quoted 9 lines
src/do_all_the_voodo_here.c +3 -0
@@ -82,6 +82,9 @@
 	char *hdrline, *subject, *unqp;
 	int hdrsadded = 0;
 	int subject_present = 0;
+	int mime = 0;
+	char *content_orig = NULL;  /* original content-type header */
+	char *boundary = NULL;

Couldn't you read it from mlmmj-process.c ?

        struct mailhdr readhdrs[] = {
                { "From:", 0, NULL },
                { "To:", 0, NULL },
                { "Cc:", 0, NULL },
                { "Return-Path:", 0, NULL },
                { "Delivered-To:", 0, NULL },
                { NULL, 0, NULL }
        };

Just add it here, and you can access it directly. These headers are collected anyway.

Have a look at main() in mlmmj-process.c.

Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.
#15 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality

In reply to Casey Allen Shobe
Quoted 5 lines · Mads Martin Joergensen

Review

Quoted 4 lines

--- mlmmj-1.2.8/src/do_all_the_voodo_here.c 2005-05-09 14:50:15.000000000 +0200

(quoted diff) +3 -0
+	int mime = 0;
+	char *content_orig = NULL;  /* original content-type header */
+	char *boundary = NULL;

Couldn't you read it from mlmmj-process.c ?

I'm not sure if I understand what you mean.

I have to modify the Content-Type header and parts of the message, which can only be done in do_all_the_voodo_here(), AFAICT. The Content-Type is not needed outside of do_all_the_voodo_here(), so I did not mess with it in mlmmj-process.c.

Or did I miss something here?

#16 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 6 lines · Jakob Hirsch
Quoted 5 lines

Review

Quoted 4 lines

--- mlmmj-1.2.8/src/do_all_the_voodo_here.c 2005-05-09 14:50:15.000000000 +0200

(quoted diff) +3 -0
+	int mime = 0;
+	char *content_orig = NULL;  /* original content-type header */
+	char *boundary = NULL;

Couldn't you read it from mlmmj-process.c ?

Quoted 8 lines

I'm not sure if I understand what you mean.

I have to modify the Content-Type header and parts of the message, which can only be done in do_all_the_voodo_here(), AFAICT. The Content-Type is not needed outside of do_all_the_voodo_here(), so I did not mess with it in mlmmj-process.c.

Or did I miss something here?

You're right. I was thinking somewhere else in the code--but indeed; in do_all_the_voodo_here() is when that stuff is read.

Is it just me, or is the second patch much better than the first one?

How much testing have this seen?

Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.
#17 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality

In reply to Casey Allen Shobe
Quoted 1 line · Mads Martin Joergensen

Is it just me, or is the second patch much better than the first one?

Of course it's better, else I wouldn't have posted it. :) But the difference is not that big. I took out one forgotten debug message and made it a little more reliable. (Seems that the local Samba Festival and the Tour de France passing through was a little too much distraction :)

Quoted 1 line

How much testing have this seen?

I tested the most common cases: no content-type, text/plain, multipart/mixed (i.e. attachment) and multipart/alternative (i.e. html and text, with or without attachment). Everybody feel free to send something else to [email protected].

Possible problems I see:

- as I did no full parsing of the Content-Type header, the recognition of the boundary is not 100% reliable, but it's very unlikely that ;boundary=" is something else.

- I only handle multipart/mixed and multipart/alternative, everything else is assumed to be plain text, which is probably not true but it's also the former behaviour. Maybe it's better to re-encapsulate everything that is not already multipart/mixed. What do you think?

- Finding the close delimiter of multipart/mixed messages will not work if there is (too much) text after the closing delimiter. This should normally not happen, as this text will not be displayed by mail clients (which was the why this came up at all). I think this is a minor issue.

- memory leaks if something goes wrong (i.e., at every return -1). Also not important IMHO, as the mlmmj-process will immediatly exit then, but I can fix this if you want.

- "voodo" is spelled "voodoo", btw. :)

#18 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 7 lines · Jakob Hirsch
Quoted 1 line

Is it just me, or is the second patch much better than the first one?

Of course it's better, else I wouldn't have posted it. :) But the difference is not that big. I took out one forgotten debug message and made it a little more reliable. (Seems that the local Samba Festival and the Tour de France passing through was a little too much distraction :)

Hehe :)

Quoted 6 lines
Quoted 1 line

How much testing have this seen?

I tested the most common cases: no content-type, text/plain, multipart/mixed (i.e. attachment) and multipart/alternative (i.e. html and text, with or without attachment). Everybody feel free to send something else to [email protected].

People, please send all kinds of potential mails breaking this to this list. I would really like to have this tested.

Quoted 5 lines

Possible problems I see:

- as I did no full parsing of the Content-Type header, the recognition of the boundary is not 100% reliable, but it's very unlikely that ;boundary=" is something else.

How hard would it be to parse it completely?

Quoted 4 lines

- I only handle multipart/mixed and multipart/alternative, everything else is assumed to be plain text, which is probably not true but it's also the former behaviour. Maybe it's better to re-encapsulate everything that is not already multipart/mixed. What do you think?

But if it's assumed to be text/plain, then the behaviour is as before, no more, no less?

Quoted 4 lines

- Finding the close delimiter of multipart/mixed messages will not work if there is (too much) text after the closing delimiter. This should normally not happen, as this text will not be displayed by mail clients (which was the why this came up at all). I think this is a minor issue.

What does "too much text" mean in this case?

Quoted 3 lines

- memory leaks if something goes wrong (i.e., at every return -1). Also not important IMHO, as the mlmmj-process will immediatly exit then, but I can fix this if you want.

Don't.

Quoted 1 line

- "voodo" is spelled "voodoo", btw. :)

Yeah--mlmmj have a history of misspelled words. Look at mlmmj-recieve :-)

Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.
#19 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality

In reply to Casey Allen Shobe
Quoted 4 lines · Mads Martin Joergensen
Quoted 2 lines

and text, with or without attachment). Everybody feel free to send something else to [email protected].

People, please send all kinds of potential mails breaking this to this list. I would really like to have this tested.

Found 1 non-working case already: Pine uses an all upper-case Content-Type line, strstr("boundary") fails. There is a strchrstr() on my system, but it seems to be a GNUism (even without having a man-page), so I have to invent my own.

Quoted 6 lines
Quoted 5 lines

Possible problems I see:

- as I did no full parsing of the Content-Type header, the recognition of the boundary is not 100% reliable, but it's very unlikely that ;boundary=" is something else.

How hard would it be to parse it completely?

Depends on what can possibly appear in the Content-Type line. Until now, all I ever saw there was 'multipart/something; boundary="...."' at most. I have to investigate.

Quoted 6 lines
Quoted 4 lines

- I only handle multipart/mixed and multipart/alternative, everything else is assumed to be plain text, which is probably not true but it's also the former behaviour. Maybe it's better to re-encapsulate everything that is not already multipart/mixed. What do you think?

But if it's assumed to be text/plain, then the behaviour is as before, no more, no less?

Yes. In most cases you'll get the old behaviour if something goes wrong. Worst case is no footer at all, but then the user won't have seen it anyway.

Quoted 5 lines
Quoted 4 lines

- Finding the close delimiter of multipart/mixed messages will not work if there is (too much) text after the closing delimiter. This should normally not happen, as this text will not be displayed by mail clients (which was the why this came up at all). I think this is a minor issue.

What does "too much text" mean in this case?

The code goes back 2*strlen(boundary)-1, so the garbage has to be less than strlen(boundary), which is about 10..30 chars. I could read a bigger buffer or even consecutively read back overlapping blocks. Hmm, not too hard to do, so maybe I should stick to reliability.

Quoted 2 lines
Quoted 1 line

- "voodo" is spelled "voodoo", btw. :)

Yeah--mlmmj have a history of misspelled words. Look at mlmmj-recieve

Stumbled across that already :)

#20 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 9 lines · Jakob Hirsch
Quoted 4 lines
Quoted 2 lines

and text, with or without attachment). Everybody feel free to send something else to [email protected].

People, please send all kinds of potential mails breaking this to this list. I would really like to have this tested.

Found 1 non-working case already: Pine uses an all upper-case Content-Type line, strstr("boundary") fails. There is a strchrstr() on my system, but it seems to be a GNUism (even without having a man-page), so I have to invent my own.

Look at findit(). It uses strncasecmp().

Quoted 10 lines
Quoted 6 lines
Quoted 5 lines

Possible problems I see:

- as I did no full parsing of the Content-Type header, the recognition of the boundary is not 100% reliable, but it's very unlikely that ;boundary=" is something else.

How hard would it be to parse it completely?

Depends on what can possibly appear in the Content-Type line. Until now, all I ever saw there was 'multipart/something; boundary="...."' at most. I have to investigate.

It doesn't really matter what one have ever seen, it only matters what is possible according to standards.

Quoted 9 lines
Quoted 6 lines
Quoted 4 lines

- I only handle multipart/mixed and multipart/alternative, everything else is assumed to be plain text, which is probably not true but it's also the former behaviour. Maybe it's better to re-encapsulate everything that is not already multipart/mixed. What do you think?

But if it's assumed to be text/plain, then the behaviour is as before, no more, no less?

Yes. In most cases you'll get the old behaviour if something goes wrong. Worst case is no footer at all, but then the user won't have seen it anyway.

Fine. If only it's not a regression.

Quoted 10 lines
Quoted 5 lines
Quoted 4 lines

- Finding the close delimiter of multipart/mixed messages will not work if there is (too much) text after the closing delimiter. This should normally not happen, as this text will not be displayed by mail clients (which was the why this came up at all). I think this is a minor issue.

What does "too much text" mean in this case?

The code goes back 2*strlen(boundary)-1, so the garbage has to be less than strlen(boundary), which is about 10..30 chars. I could read a bigger buffer or even consecutively read back overlapping blocks. Hmm, not too hard to do, so maybe I should stick to reliability.

mlmmj is all about reliability :-)

Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.
#21 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality

In reply to Casey Allen Shobe
Quoted 5 lines · Mads Martin Joergensen
Quoted 3 lines

Depends on what can possibly appear in the Content-Type line. Until now, all I ever saw there was 'multipart/something; boundary="...."' at most. I have to investigate.

It doesn't really matter what one have ever seen, it only matters what is possible according to standards.

That's what I meant. According to RFC1341, multipart/mixed has only the boundary parameter. But this is not an issue anymore (see below).

Quoted 5 lines
Quoted 4 lines

The code goes back 2*strlen(boundary)-1, so the garbage has to be less than strlen(boundary), which is about 10..30 chars. I could read a bigger buffer or even consecutively read back overlapping blocks. Hmm, not too hard to do, so maybe I should stick to reliability.

mlmmj is all about reliability :-)

Sure, that's one the reasons we use it. :)

To make it really reliable, I will take the easy way: Create a new MIME envelope with multipart/mixed for every Content-Type (not only for multipart/alternative, as now). It's not even the easiest way to do it, it may also be the best one. Should have done it this way right from the start.
We can even think about doing this with all messages, so the behaviour and appearance is consistent. But that should probably be controlled by a switch in the control directory (like "footer_always_mime").

btw, what happens with log_error messages? I thought they appear in mlmmj.operation.log...

#22 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality

In reply to Casey Allen Shobe
Quoted 2 lines · Jakob Hirsch

To make it really reliable, I will take the easy way: Create a new MIME envelope with multipart/mixed for every Content-Type (not only for

done. Blatantly simple, compared with the original approach.

Quoted 3 lines

We can even think about doing this with all messages, so the behaviour and appearance is consistent. But that should probably be controlled by a switch in the control directory (like "footer_always_mime").

ok, maybe later... :)

It could also be used for specifying a charset, but then an encoding would also be needed (or we assume something, 8bit or qp).

src/do_all_the_voodo_here.c +90 -10
@@ -79,9 +79,12 @@
 		 const char **delhdrs, struct mailhdr *readhdrs,
 		 struct strlist *allhdrs, const char *prefix)
 {
-	char *hdrline, *subject, *unqp;
+	char *hdrline, *subject, *unqp, *buf;
 	int hdrsadded = 0;
 	int subject_present = 0;
+	char *content_type_orig = NULL;	/* original Content-Type header */
+	char *content_te_orig = NULL;   /* original Content-Transfer-Encoding */
+	char *boundary = NULL;
 
 	allhdrs->count = 0;
 	allhdrs->strs = NULL;
@@ -116,7 +119,22 @@
 					myfree(subject);
 					subject_present = 1;
 				}
-
+				
+				/* time to add Content-Type */
+				if (content_type_orig || content_te_orig) {
+					/* create new boundary for MIME-encapsulation */
+					buf = random_str();
+					boundary = concatstr(3, "=_=", buf, "=_=");
+					myfree(buf);
+
+					/* and create new content-type header */
+					buf = concatstr(3,
+						"Content-Type: multipart/mixed; boundary=\"",
+						boundary, "\"\n");
+					writen(outfd, buf, strlen(buf));
+					myfree(buf);
+				}
+				
 				if(writen(outfd, hdrline, strlen(hdrline))
 						< 0) {
 					myfree(hdrline);
@@ -159,7 +177,25 @@
 				myfree(unqp);
 			}
 		}
-		
+
+		if (footfd>=0) {
+			/* save MIME header for later use */
+			if (!strncasecmp(hdrline, "Content-Type:", 13)) {
+				if (!content_type_orig)
+					content_type_orig = hdrline;
+				else
+					myfree(hdrline); /* drop surplus */
+				continue;
+			} else if (!strncasecmp(hdrline, 
+					"Content-Transfer-Encoding:", 26)) {
+				if (!content_te_orig)
+				    content_te_orig = hdrline;
+				else
+				    myfree(hdrline); /* drop surplus */
+				continue;
+			}
+		}
+
 		/* Should it be stripped? */
 		if(delhdrs) {
 			if(!findit(hdrline, delhdrs))
@@ -170,20 +206,64 @@
 
 		myfree(hdrline);
 	}
+	
+	if (content_type_orig || content_te_orig) {
 
-	/* Just print the rest of the mail */
-	if(dumpfd2fd(infd, outfd) < 0) {
-		log_error(LOG_ARGS, "Error when dumping rest of mail");
-		return -1;
-	}
+		buf = concatstr(3,
+			"This is a multi-part message in MIME format.\n\n--",
+			boundary, "\n");
+		writen(outfd, buf, strlen(buf));
+		myfree(buf);
+
+		if (content_type_orig) {
+			writen(outfd, content_type_orig, strlen(content_type_orig));
+			myfree(content_type_orig);
+		}
+        
+		if (content_te_orig) {
+			writen(outfd, content_te_orig, strlen(content_te_orig));
+			myfree(content_te_orig);
+		}
+		
+		writen(outfd, "\n", 1);
 
-	/* No more, lets add the footer if one */
-	if(footfd >= 0)
+		/* Just print the rest of the mail */
+		if(dumpfd2fd(infd, outfd) < 0) {
+			log_error(LOG_ARGS, "Error when dumping rest of mail");
+			return -1;
+		}
+
+		buf = concatstr(3, "\n--", boundary,
+			"\nContent-Type: text/plain\nContent-Disposition: inline\n\n");
+		writen(outfd, buf, strlen(buf));
+ 		myfree(buf);
+
+		/* No more, lets add the footer if one */
 		if(dumpfd2fd(footfd, outfd) < 0) {
 			log_error(LOG_ARGS, "Error when adding footer");
 			return -1;
 		}
 
+		buf = concatstr(3, "\n--", boundary, "--\n");
+		myfree(boundary);
+		writen(outfd, buf, strlen(buf));
+		myfree(buf);
+		
+	} else {
+		/* Just print the rest of the mail */
+		if(dumpfd2fd(infd, outfd) < 0) {
+			log_error(LOG_ARGS, "Error when dumping rest of mail");
+			return -1;
+		}
+
+		/* No more, lets add the footer if one */
+		if(footfd >= 0)
+			if(dumpfd2fd(footfd, outfd) < 0) {
+				log_error(LOG_ARGS, "Error when adding footer");
+				return -1;
+			}
+	}
+
 	fsync(outfd);
 
 	return 0;
#23 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe

* Jakob Hirsch <[email protected]> [Jul 19. 2005 20:44]:

[...]

Quoted 2 lines

btw, what happens with log_error messages? I thought they appear in mlmmj.operation.log...

No, that ends up in mail syslog, /var/log/maillog on a FreeBSD system, /var/log/mail on SUSE LINUX.

I know--mlmmj logging is horrible. Some people have promised in the past to clean it all up, so we had nice proper logging using syslog, but so far it failed.

mlmmj.operations.log is log_oper().

Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.
#24 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 5 lines · Jakob Hirsch
Quoted 2 lines

Also those who are running patched versions, please please download this tar-ball and make sure your patches are folded in.

Um, you didn't like my footer-mime patch? I didn't get any comments, so I wonder what's wrong about it.

Well, if someone wants to add a footer, then every single mail will be "massaged" even in the case of it not being needed. I really would like it to happen only when the mail is not a text mail.

I also would like to see some checking of the writen() return values. If any one these writes fail, it just continues on.

Quoted 139 lines · Jakob Hirsch
src/do_all_the_voodo_here.c +90 -10
@@ -79,9 +79,12 @@
 		 const char **delhdrs, struct mailhdr *readhdrs,
 		 struct strlist *allhdrs, const char *prefix)
 {
-	char *hdrline, *subject, *unqp;
+	char *hdrline, *subject, *unqp, *buf;
 	int hdrsadded = 0;
 	int subject_present = 0;
+	char *content_type_orig = NULL;	/* original Content-Type header */
+	char *content_te_orig = NULL;   /* original Content-Transfer-Encoding */
+	char *boundary = NULL;
 
 	allhdrs->count = 0;
 	allhdrs->strs = NULL;
@@ -116,7 +119,22 @@
 					myfree(subject);
 					subject_present = 1;
 				}
-
+				
+				/* time to add Content-Type */
+				if (content_type_orig || content_te_orig) {
+					/* create new boundary for MIME-encapsulation */
+					buf = random_str();
+					boundary = concatstr(3, "=_=", buf, "=_=");
+					myfree(buf);
+
+					/* and create new content-type header */
+					buf = concatstr(3,
+						"Content-Type: multipart/mixed; boundary=\"",
+						boundary, "\"\n");
+					writen(outfd, buf, strlen(buf));
+					myfree(buf);
+				}
+				
 				if(writen(outfd, hdrline, strlen(hdrline))
 						< 0) {
 					myfree(hdrline);
@@ -159,7 +177,25 @@
 				myfree(unqp);
 			}
 		}
-		
+
+		if (footfd>=0) {
+			/* save MIME header for later use */
+			if (!strncasecmp(hdrline, "Content-Type:", 13)) {
+				if (!content_type_orig)
+					content_type_orig = hdrline;
+				else
+					myfree(hdrline); /* drop surplus */
+				continue;
+			} else if (!strncasecmp(hdrline, 
+					"Content-Transfer-Encoding:", 26)) {
+				if (!content_te_orig)
+				    content_te_orig = hdrline;
+				else
+				    myfree(hdrline); /* drop surplus */
+				continue;
+			}
+		}
+
 		/* Should it be stripped? */
 		if(delhdrs) {
 			if(!findit(hdrline, delhdrs))
@@ -170,20 +206,64 @@
 
 		myfree(hdrline);
 	}
+	
+	if (content_type_orig || content_te_orig) {
 
-	/* Just print the rest of the mail */
-	if(dumpfd2fd(infd, outfd) < 0) {
-		log_error(LOG_ARGS, "Error when dumping rest of mail");
-		return -1;
-	}
+		buf = concatstr(3,
+			"This is a multi-part message in MIME format.\n\n--",
+			boundary, "\n");
+		writen(outfd, buf, strlen(buf));
+		myfree(buf);
+
+		if (content_type_orig) {
+			writen(outfd, content_type_orig, strlen(content_type_orig));
+			myfree(content_type_orig);
+		}
+        
+		if (content_te_orig) {
+			writen(outfd, content_te_orig, strlen(content_te_orig));
+			myfree(content_te_orig);
+		}
+		
+		writen(outfd, "\n", 1);
 
-	/* No more, lets add the footer if one */
-	if(footfd >= 0)
+		/* Just print the rest of the mail */
+		if(dumpfd2fd(infd, outfd) < 0) {
+			log_error(LOG_ARGS, "Error when dumping rest of mail");
+			return -1;
+		}
+
+		buf = concatstr(3, "\n--", boundary,
+			"\nContent-Type: text/plain\nContent-Disposition: inline\n\n");
+		writen(outfd, buf, strlen(buf));
+ 		myfree(buf);
+
+		/* No more, lets add the footer if one */
 		if(dumpfd2fd(footfd, outfd) < 0) {
 			log_error(LOG_ARGS, "Error when adding footer");
 			return -1;
 		}
 
+		buf = concatstr(3, "\n--", boundary, "--\n");
+		myfree(boundary);
+		writen(outfd, buf, strlen(buf));
+		myfree(buf);
+		
+	} else {
+		/* Just print the rest of the mail */
+		if(dumpfd2fd(infd, outfd) < 0) {
+			log_error(LOG_ARGS, "Error when dumping rest of mail");
+			return -1;
+		}
+
+		/* No more, lets add the footer if one */
+		if(footfd >= 0)
+			if(dumpfd2fd(footfd, outfd) < 0) {
+				log_error(LOG_ARGS, "Error when adding footer");
+				return -1;
+			}
+	}
+
 	fsync(outfd);
 
 	return 0;
Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                 -- A. P. J.
#25 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality

In reply to Casey Allen Shobe
Quoted 4 lines · Mads Martin Joergensen
Quoted 1 line

so I wonder what's wrong about it.

Well, if someone wants to add a footer, then every single mail will be "massaged" even in the case of it not being needed. I really would like it to happen only when the mail is not a text mail.

I considered this, but thought it would be an advantage to have constant behaviour. There's no real downside in always having a MIME mail, or is it? Mailing list software always modify messages and all current MUAs are MIME-capable. Checking for text/plain adds only a little complexity, so if you insist on it, I'll do it. It's probably a good idea to make this configurable (footer_force_mime or something).

Quoted 2 lines

I also would like to see some checking of the writen() return values. If any one these writes fail, it just continues on.

Should we abort if a critical writen() fails? There are already some writen() without checking...

#26 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 2 lines · you

There's no real downside in always having a MIME mail, or is it?

I would consider this behaviour undesirable. YMMV. If this does get added I hope it's optional.

#27 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality

In reply to Casey Allen Shobe
Quoted 2 lines · Neale Pickett
Quoted 1 line

There's no real downside in always having a MIME mail, or is it?

I would consider this behaviour undesirable. YMMV.

I don't want to raise another "MIME - bad or not?" discussion, but why? MIME was considered a bad back in the old days, when bandwidth was expensive and many MUAs couldn't handle it. Things have changed a lot since then.

Quoted 1 line

If this does get added I hope it's optional.

So there should be an option footer_never_mime? Don't think that makes much sense. If the mail has a multipart content-type, the footer will be lost. If it's already MIME, we only meet the standards.

#28 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 5 lines · Jakob Hirsch
Quoted 1 line

If this does get added I hope it's optional.

So there should be an option footer_never_mime? Don't think that makes much sense. If the mail has a multipart content-type, the footer will be lost. If it's already MIME, we only meet the standards.

So how about meeting the standars when and only when it's already MIME?

And BTW--if you don't configure mlmmj to do it, you can have a mailing list manager that doesn't touch the mails whatsoever.

Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                 -- A. P. J.
#29 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 5 lines · Mads Martin Joergensen
Quoted 1 line · Jakob Hirsch

If it's already MIME, we only meet the standards.

So how about meeting the standars when and only when it's already MIME?

+1 for this. There's no need to make a message multipart if there's only one part in the first place (unless that one part isn't text/plain, I guess).

#30 lore raw

Re: Fw: footer isn't appended to multipart messages (part II:

In reply to Casey Allen Shobe
Quoted 2 lines · Neale Pickett
Quoted 2 lines
Quoted 1 line

If it's already MIME, we only meet the standards.

So how about meeting the standars when and only when it's already MIME?

Content-Type and Content-Transfer-Encoding _are_ MIME headers, so messages containing these headers are MIME messages and should be treated as such.

Quoted 3 lines

+1 for this. There's no need to make a message multipart if there's only one part in the first place (unless that one part isn't text/plain, I guess).

This is only valid if this one part's transfer-encoding is 7/8bit or QP. Even then it's closer to the "MIME spirit" to make it multipart and add things in it's own part. This also gives you the possibility to use your own charset etc. (though this is not supported ATM).

If somebody cares about message modification (and does MTA debugging, like I also do) he shouldn't use a footer in the first place, so there is no modification at all.
But if you have to, MIME gives you the opportunity to reconstruct a message as it was, since the original body is preserved in a MIME envelope. Simply appending a footer is more tampering.

But ok, I yield to the overwhelming majority. :)
I'll add a switch so people can disable MIME handling. Or should the default be "off"?

#31 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 3 lines · Jakob Hirsch

But ok, I yield to the overwhelming majority. :)
I'll add a switch so people can disable MIME handling. Or should the default be "off"?

I think the default should be off, so we're keeping compatibility, and old installations will never all of a sudden change behaviour.

DIR/control/mimeencapsulate seems like the TUNABLE for it :-)

Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                 -- A. P. J.
#32 lore raw

Re: Fw: footer isn't appended to multipart messages (part II:

In reply to Casey Allen Shobe
Quoted 5 lines · Mads Martin Joergensen
Quoted 3 lines

But ok, I yield to the overwhelming majority. :)
I'll add a switch so people can disable MIME handling. Or should the default be "off"?

I think the default should be off, so we're keeping compatibility, and old installations will never all of a sudden change behaviour.

sounds sensible.

Quoted 1 line

DIR/control/mimeencapsulate seems like the TUNABLE for it :-)

something like footer_use_mime sounded logical for me, as it's only used if there's a footer, but I don't really care about that.

About the writen()s you mentioned: abort if they fail?

#33 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 6 lines · Jakob Hirsch
Quoted 5 lines
Quoted 3 lines

But ok, I yield to the overwhelming majority. :)
I'll add a switch so people can disable MIME handling. Or should the default be "off"?

I think the default should be off, so we're keeping compatibility, and old installations will never all of a sudden change behaviour.

Quoted 6 lines

sounds sensible.

Quoted 1 line

DIR/control/mimeencapsulate seems like the TUNABLE for it :-)

something like footer_use_mime sounded logical for me, as it's only used if there's a footer, but I don't really care about that.

/me neither. You're making the patch, call it what you want.

Quoted 1 line

About the writen()s you mentioned: abort if they fail?

Like this first writen in mlmmj-process.c:

if(writen(queuefd, tmp, strlen(tmp)) < 0)
log_error(LOG_ARGS, "Could not write line for "
"moderatemail");

You mention you've found other writen() calls that there's no check for. It might very well be the case, but it's a bug then. Albeit a minor one, since writen() catches EINTR and ensures that all bytes one wants to write are written by calling write() subsequently.

Signature · 4 lines
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                 -- A. P. J.
#34 lore raw

Re: Fw: footer isn't appended to multipart messages (part II: reality check!)

In reply to Casey Allen Shobe
Quoted 4 lines · Jakob Hirsch
Quoted 1 line

DIR/control/mimeencapsulate seems like the TUNABLE for it :-)

something like footer_use_mime sounded logical for me, as it's only used if there's a footer, but I don't really care about that.

What if you call it control/mime_footer, which would supercede control/footer and contain a full MIME part? For instance:

Content-Type: text/plain

This is the snot mail list. To unsubscribe, send a message with the subject "UNSUBSCIBE" to every member of the list, individually.

The advantage here would be that you could make the footer anything you like:
it could be HTML, or a PNG, or even a multipart/alternative with all three.

If you wanted to be super spiff about it, you could check to make sure there's a header block in the file and if not prepend the Content-Type: text/plain part.

Call me crazy,

Neale