Skip to contentSkip to conversation

recipient delimiter bug with RC1

PATCH Jeremy Hinegardner 3 participants 4 messages Nov 13, 2005 – Nov 14, 2005

lore
Share & tools
lore
#0 lore raw

Hi all,

I was looking forward recipient delimiter to be configurable. But it just so happens that on postfix with virtual domains and postfix's recipient delimiter set to '-' it doesn't quite work. The bug manifests itself if you have a mailing list that has a name containing the recipient delimiter.

In other words, if both postfix and mlmmj have their recipient delimiters set to '-' and you create a mailing list named 'mlmmj-test' then your subscribes, unsubscribes are mishandled by 'mlmmj' when it attempts to parse the email address for control commands.

For example:
postfix recipient delimiter = -
mlmmj recipient delimiter = -
list name = mlmmj-test

When [email protected] gets to mlmmj it parse the command as being 'test-subscribe' which doesn't match so it exits. Even if it was just a regular post to [email protected] then mlmmj assumes that it has a command since the recipient delimiter is present.

I'm not sure if I explained the problem very well, but the attached patch fixes it for me.

enjoy,

-jeremy

--
========================================================================
Jeremy Hinegardner [email protected]

mlmmj-1.2.9-RC1/include/listcontrol.h renamed from mlmmj-1.2.9-RC1-orig/include/listcontrol.h +1 -1
diff -r -u ./mlmmj-1.2.9-RC1-orig/include/listcontrol.h ./mlmmj-1.2.9-RC1/include/listcontrol.h
@@ -25,7 +25,7 @@
 #define LISTCONTROL_H
 
 #include "find_email_adr.h"
-
+char* getdelimlocation(const char* emailaddress, const char *listdelim);
 int listcontrol(struct email_container *fromemails, const char *listdir,
 		const char *controladdr, const char *mlmmjsub,
 		const char *mlmmjunsub, const char *mlmmjsend,
mlmmj-1.2.9-RC1/src/listcontrol.c renamed from mlmmj-1.2.9-RC1-orig/src/listcontrol.c +113 -47

201 lines not on this page · raw message

Only in ./mlmmj-1.2.9-RC1-orig/src: mlmmj-make-ml.sh

mlmmj-1.2.9-RC1/src/mlmmj-process.c renamed from mlmmj-1.2.9-RC1-orig/src/mlmmj-process.c +2 -2
diff -r -u ./mlmmj-1.2.9-RC1-orig/src/mlmmj-process.c ./mlmmj-1.2.9-RC1/src/mlmmj-process.c
@@ -517,7 +517,7 @@
 
 	listdelim = getlistdelim(listdir);
 	if(whichto && whichto->emaillist && whichto->emaillist[0]){
-		recipextra = strstr(whichto->emaillist[0], listdelim);
+		recipextra = getdelimlocation(whichto->emaillist[0], listdelim);
 		if (recipextra)
 			recipextra += strlen(listdelim);
 	} else
@@ -571,7 +571,7 @@
 			exit(EXIT_FAILURE);
 		}
 #if 0
-		log_error(LOG_ARGS, "listcontrol(from, %s, %s, %s, %s, %s, %s)\n", listdir, toemails.emaillist[0], mlmmjsub, mlmmjunsub, mlmmjsend, mlmmjbounce);
+		log_error(LOG_ARGS, "listcontrol(from, %s, %s, %s, %s, %s, %s)\n", listdir, whichto->emaillist[0], mlmmjsub, mlmmjunsub, mlmmjsend, mlmmjbounce);
 #endif
 		unlink(mailfile);
 		listcontrol(&fromemails, listdir, whichto->emaillist[0],
#1 lore raw
Quoted 18 lines · Jeremy Hinegardner

Hi all,

I was looking forward recipient delimiter to be configurable. But it just so happens that on postfix with virtual domains and postfix's recipient delimiter set to '-' it doesn't quite work. The bug manifests itself if you have a mailing list that has a name containing the recipient delimiter.

In other words, if both postfix and mlmmj have their recipient delimiters set to '-' and you create a mailing list named 'mlmmj-test' then your subscribes, unsubscribes are mishandled by 'mlmmj' when it attempts to parse the email address for control commands.

For example:
postfix recipient delimiter = -
mlmmj recipient delimiter = -
list name = mlmmj-test

Quoted 7 lines

When [email protected] gets to mlmmj it parse the command as being 'test-subscribe' which doesn't match so it exits. Even if it was just a regular post to [email protected] then mlmmj assumes that it has a command since the recipient delimiter is present.

I'm not sure if I explained the problem very well, but the attached patch fixes it for me.

I believe I documented this issue with the original patches; I did consider it, but (as your patch shows) the fix for it was highly invasive, and involved some significant logic restructuring. I chose not to do that, first pass, because one of my goals was "minimize changes" and another was "keep the patch as human-readable as possible". Now that it's had some more significant 'field testing' of the core, it certainly seems like it might be worth addressing the weirder quirks where this sort of thing happens (there are also several memory leaks which I left alone, entirely because they were items that would not make any significant difference in the typical mlmmj mode of 'run, then end', and fixing them was going to introduce either logic changes or additional patch dependancies).

As a sidenote, however, I use Exim with a list delimiter of "-" (as in, for example, [email protected]), and it seemed to be functioning basically correctly under those circumstances, during my tests with the original patch. It is possible that this is due to some of the logic handled by Exim, before it ever hands things to mlmmj (it autodetects valid lists, matching [email protected] based on the same rules implemented by qmail/ezmlm; a-b-c-d, then a-b-c, then a-b, then a...), so it may be doing something that shortcuts the entire situation.

I honestly think that there might be some call for an attempt to restructure some things for 1.3; there are several points where the list delimiter is re-read by different levels of the program, because there was no obviously sane way to pass it through the intervening functions; again, I chose the way I did based on minimal change in a patch being higher priority than maximum efficiency of the final result (note 'maximum'; *useable* efficiency was a base requirement, of course).

I still think it's worth looking into allowing the MTA to provide environment variables that will be treated as the canonical "list" and "extra" portions, if they're set, and bypass all of this. Some MTAs do it automagically, and many (even most) can do it if configured to do so. Generally, they have a far more flexible configuration language, can do things like "talk to LDAP" or "make DNS queries" when relevant, and if the administrator has configured them to be able to decide which parts are core delivery and which are permitted extras, mlmmj shouldn't try to second-guess them (though keeping the logic available, for use when the MTA *doesn't* provide such details, or when there is a need to break down the 'extra' section further than the MTA does, is still worthwhile).

Signature · 1 line
Joel Aelwyn <[email protected]>
#2 lore raw
In reply to Jeremy Hinegardner
Quoted 5 lines · Jeremy Hinegardner

I was looking forward recipient delimiter to be configurable. But it just so happens that on postfix with virtual domains and postfix's recipient delimiter set to '-' it doesn't quite work. The bug manifests itself if you have a mailing list that has a name containing the recipient delimiter.

I noticed the same thing. My patch is attached, it does a bit less work than Jeremy's.

My patch also fixes what I suspect (but have not verified) to be another bug:
if the list is not the first address in the header (ie. Cc: [email protected], [email protected]), it doesn't see itself as being in the tocc. I fixed this by iterating over every address in the header, and not just looking at the first.

I've been running this on my lists for a while and it seems to be working fine.

Neale

PS: Jeremy, how did you set up your postfix's aliases table to deal with the list name with the - in it? I had to create two entries: "pajarito" and "pajarito-riders" that both pipe to mlmmj-recieve with the same directory.

Mon Nov 7 23:44:04 MST 2005 Neale Pickett <[email protected]>
* Handle when recip_delim occurs >1 time
Sun Oct 30 20:00:38 MST 2005 Neale Pickett <[email protected]>
* strgen malloc fix

include/strrstr.h +6 -0
diff -rN -u old-mlmmj-darcs/include/strrstr.h new-mlmmj-darcs/include/strrstr.h
@@ -0,0 +1,6 @@
+#ifndef MLMMJ_STRRSTR__H
+#define MLMMJ_STRRSTR__H
+
+char *strrstr(const char *haystack, const char *needle);
+
+#endif /* MLMMJ_STRRSTR__H */
src/listcontrol.c +2 -1
diff -rN -u old-mlmmj-darcs/src/listcontrol.c new-mlmmj-darcs/src/listcontrol.c
@@ -42,6 +42,7 @@
 #include "statctrl.h"
 #include "mygetline.h"
 #include "chomp.h"
+#include "strrstr.h"
 #include "memory.h"
 #include "log_oper.h"
 #include "ctrlvalues.h"
@@ -123,7 +124,7 @@
 		subswitch = "-C";
 	
 	listdelim = getlistdelim(listdir);
-	recipdelimsign = strstr(controladdr, listdelim);
+	recipdelimsign = strrstr(controladdr, listdelim);
 	MY_ASSERT(recipdelimsign);
 	atsign = index(controladdr, '@');
 	MY_ASSERT(atsign);
src/Makefile.am +6 -6
diff -rN -u old-mlmmj-darcs/src/Makefile.am new-mlmmj-darcs/src/Makefile.am
@@ -12,7 +12,7 @@
 
 EXTRA_DIST = mlmmj-make-ml.sh
 
-mlmmj_send_SOURCES = mlmmj-send.c writen.c mail-functions.c itoa.c chomp.c \
+mlmmj_send_SOURCES = mlmmj-send.c writen.c mail-functions.c itoa.c chomp.c strrstr.c \
                      incindexfile.c checkwait_smtpreply.c getlistaddr.c \
 		     mylocking.c init_sockfd.c strgen.c random-int.c \
 		     print-version.c log_error.c mygetline.c memory.c \
@@ -24,7 +24,7 @@
 			log_oper.c mylocking.c readn.c
 
 mlmmj_process_SOURCES = mlmmj-process.c writen.c find_email_adr.c \
-			incindexfile.c itoa.c getlistaddr.c chomp.c \
+			incindexfile.c itoa.c getlistaddr.c chomp.c strrstr.c \
 			mylocking.c listcontrol.c random-int.c strgen.c \
 			print-version.c send_help.c prepstdreply.c \
 			do_all_the_voodo_here.c mygetline.c gethdrline.c \
@@ -33,25 +33,25 @@
 			send_list.c readn.c getlistdelim.c
 
 mlmmj_sub_SOURCES = mlmmj-sub.c writen.c mylocking.c \
-			getlistaddr.c chomp.c random-int.c strgen.c \
+			getlistaddr.c chomp.c strrstr.c random-int.c strgen.c \
 			subscriberfuncs.c print-version.c \
 			log_error.c mygetline.c prepstdreply.c memory.c \
 			statctrl.c readn.c getlistdelim.c
 
 mlmmj_unsub_SOURCES = mlmmj-unsub.c writen.c mylocking.c \
-			getlistaddr.c chomp.c subscriberfuncs.c random-int.c \
+			getlistaddr.c chomp.c strrstr.c subscriberfuncs.c random-int.c \
 			strgen.c print-version.c log_error.c mygetline.c \
 			prepstdreply.c memory.c statctrl.c readn.c \
 			getlistdelim.c
 			
 mlmmj_bounce_SOURCES = mlmmj-bounce.c print-version.c log_error.c \
 		       subscriberfuncs.c strgen.c random-int.c writen.c \
-		       prepstdreply.c mygetline.c chomp.c getlistaddr.c \
+		       prepstdreply.c mygetline.c chomp.c strrstr.c getlistaddr.c \
 		       memory.c find_email_adr.c gethdrline.c readn.c \
 		       getlistdelim.c
 
 mlmmj_maintd_SOURCES = mlmmj-maintd.c print-version.c log_error.c mygetline.c \
-		       strgen.c random-int.c chomp.c writen.c memory.c \
+		       strgen.c random-int.c chomp.c strrstr.c writen.c memory.c \
 		       ctrlvalue.c send_digest.c getlistaddr.c dumpfd2fd.c \
 		       mylocking.c log_oper.c readn.c getlistdelim.c
 
src/Makefile.in +29 -25

128 lines not on this page · raw message

src/mlmmj-process.c +27 -9
diff -rN -u old-mlmmj-darcs/src/mlmmj-process.c new-mlmmj-darcs/src/mlmmj-process.c
@@ -52,6 +52,7 @@
 #include "memory.h"
 #include "log_oper.h"
 #include "chomp.h"
+#include "strrstr.h"
 
 enum action {
 	ALLOW,
@@ -515,13 +516,33 @@
 	else
 		whichto = NULL;
 
+	listaddr = getlistaddr(listdir);
+	alternates = ctrlvalues(listdir, "listaddress");
+
 	listdelim = getlistdelim(listdir);
-	if(whichto && whichto->emaillist && whichto->emaillist[0]){
-		recipextra = strstr(whichto->emaillist[0], listdelim);
-		if (recipextra)
-			recipextra += strlen(listdelim);
-	} else
-		recipextra = NULL;
+        recipextra = NULL;
+	if(whichto){
+		for(i = 0; i < alternates->count; i += 1){
+			int locallen;
+			char *at;
+
+			at = strchr(alternates->strs[i], '@');
+			if (NULL == at) {
+				continue;
+			}
+			locallen = at - alternates->strs[i];
+
+			for(j = 0; j < whichto->emailcount; j += 1){
+				if((0 == strncmp(alternates->strs[i],
+                                                 whichto->emaillist[j],
+                                                 locallen)) &&
+                                   ('@' != whichto->emaillist[j][locallen])){
+                                        recipextra = whichto->emaillist[j] + locallen;
+                                }
+                        }
+
+		}
+        }
 	myfree(listdelim);
 
 	if(recipextra) {
@@ -611,9 +632,6 @@
 
 	unlink(mailfile);
 
-	listaddr = getlistaddr(listdir);
-	alternates = ctrlvalues(listdir, "listaddress");
-
 	addrtocc = !(statctrl(listdir, "tocc"));
 	if(addrtocc) {
 		for(i = 0; i < toemails.emailcount; i++) {
src/mlmmj-send.c +2 -1
diff -rN -u old-mlmmj-darcs/src/mlmmj-send.c new-mlmmj-darcs/src/mlmmj-send.c
@@ -49,6 +49,7 @@
 #include "itoa.h"
 #include "incindexfile.h"
 #include "chomp.h"
+#include "strrstr.h"
 #include "checkwait_smtpreply.h"
 #include "getlistaddr.h"
 #include "getlistdelim.h"
@@ -163,7 +164,7 @@
 	*c = '\0';
 	num = strrchr(myfrom, '-');
 	num++;
-	c = strstr(myfrom, listdelim);
+	c = strrstr(myfrom, listdelim);
 	myfrom = strchr(c, '-');
 	myfrom++;
 	len = num - myfrom - 1;
src/strgen.c +1 -1
diff -rN -u old-mlmmj-darcs/src/strgen.c new-mlmmj-darcs/src/strgen.c
@@ -245,7 +245,7 @@
 	const char *weekday = NULL, *month = NULL;
 
 	/* 6 + 26 + ' ' + timezone which is 5 + '\n\0' == 40 */
-	timestr = (char *)malloc(40);
+	timestr = (char *)mymalloc(40);
 	t = time(NULL);
 
 	localtime_r(&t, &lttm);
src/strrstr.c +17 -0
diff -rN -u old-mlmmj-darcs/src/strrstr.c new-mlmmj-darcs/src/strrstr.c
@@ -0,0 +1,17 @@
+#include <string.h>
+#include "strrstr.h"
+
+char *strrstr(const char *haystack, const char *needle)
+{
+	char *result;
+	char *p;
+
+	result = NULL;
+	p = strstr(haystack, needle);
+	while (NULL != p) {
+		result = p;
+		p = strstr(result + strlen(needle), needle);
+	}
+
+	return result;
+}
#3 lore raw
In reply to Jeremy Hinegardner
Quoted 3 lines · Neale Pickett

PS: Jeremy, how did you set up your postfix's aliases table to deal with the list name with the - in it? I had to create two entries: "pajarito" and "pajarito-riders" that both pipe to mlmmj-recieve with the same directory.

Yes, this was an issue. My testing of the mlmmj-test list with recipient delimiter of '-' is what brought this whole issue about. And it seems to be an issue with Postfix and the local(8) delivery agent.

Using virtual as:

    /^(mlmmj-test.*)@example.com$/      $1

And aliases as:
mlmmj-test: "| /usr/local/mlmmj-recieve -L ...."

For some reason local(8) wouldn't do the right thing. It would be looking for a user 'mlmmj-test-subscribe' or 'mlmmj' so the alias wouldn't resolve. And a status=bounced (unknown user:
"mlmmj-test-subscribe") would result.

So next I tried virtual as:

    /^(mlmmj-test).*@example.com$/      $1

And in this case, it would deliver to mlmmj-recieve but mlmmj-recieve wouldn't know what the command was since trivial-rewrite hand changed the delivery address.

Joel pointed out that this is handled correctly in Exim, so it apparently does a more exhaustive search of the possibilities to find out if it can find something to deliver too. Maybe this is a cause to submit a bug report to Postfix?

In any case, I wanted to have a single 'mailing list user' manage all the mailing lists for my domain, which ended up being at odds with local(8)'s delivery. As a result I'm using a postfix transport map to delivery mail to the 'mlmmj' user. I wrote up a README me on it and I've attached it.

enjoy,

-jeremy

--
========================================================================
Jeremy Hinegardner [email protected]

README.postfix Nov 12th 2005

POSTFIX ISSUES

The main issue with Postfix and mlmmj is the mlmmj requirement that the mlmmj executables must be executed by root or the owner of the list directory.

This is at odds with Postfix. The standard local delivery mechanism for Postfix is local(8) that ships with Postfix. According to local(8) delivery to external programs is done on behalf of the receiving user. But when delivering to a program without using a .forward file there is no user context. And using an alias file does not provide user context.

The man page also explains that in the absence of user context the local(8) daemon will use the owner of the :include: file from the aliases file. But this is a problem too. By default :include:
files are disabled as a security precaution in aliases files for delivering to external programs.

This leaves us with a conundrum on how to execute the mlmmj executables as an 'mlmmj' user without using alias files. One answer is to use a postfix transport.

First we'll get the 'mlmmj' user setup and then move onto the postfix configuration:

MLMMJ SETUP

Create a 'mlmmj' user that will own all the lists. Use whatever user creation app/script is provided by your system. Generally 'useradd'.

Create the spool directory that is owned by the 'mlmmj' user. This is typically /var/spool/mlmmj but can be any directory so long as it is owned by 'mlmmj'. It can even be the home directory of the 'mlmmj' user. If the spool directory is not /var/spool/mlmmj then everywhere in this file replace /var/spool/mlmmj with your spool directory.

Create a mailing list using mlmmj-make-ml.sh. Make sure to use the -s flag to set the spool directory if it isn't /var/spool/mlmmj

POSTFIX SETUP

First thing is to make sure that the postfix server accepts mail for the mailing lists. For a server that handles mail for multiple domains, this is done with a 'virtual_alias_map'. This is how I'll demonstrate.

Add a virtual_alias_map file to main.cf configuration. We'll use a regular expression map since we need to be able to match all the various mjmml delimiter addresses (list-subscribe, list-unsubscribe, etc)

main.cf:
virtual_alias_maps = hash:/etc/postfix/virtual,
regexp:/var/spool/mlmmj/virtual.regexp

/var/spool/mlmmj/virtual.regexp:
/^(mlmmj-test.*)@example.com$/ $1
/^(another-list.*)@sample.com$/ $1

One line needs to be in the virtual map for each list the 'mlmmj' id is to handle. The regex formula is:

/^(list-name.*)@domain$/ $1

Next we make sure that postfix can invoke the mlmmj executables as the 'mlmmj' user. This is where the transport map comes in. So we add a transport map and a configuration option that instructs the transport to only deliver one file at a time. See transport(5) for more information on transports.

main.cf:
transport_maps = regexp:/var/spool/mlmmj/transport mlmmj_destination_recipient_limit = 1

/var/spool/mlmmj/transport:
/^(mlmmj-test).*$/ mlmmj:$1
/^(another-list).*$/ mlmmj:$1

What this transport file says, is that any message destined for an email address that matches the regexp on the left, deliver it using the transport 'mlmmj' and setting 'nexthop' to the value in $1. Which in this case is the mailing list name. 'nexthop' is special variable for transports.

Now we setup the 'mlmmj' transport. The 'mlmmj' in mlmmj:$1 above indicates a transport listed in the postfix master.cf file. We are just going to create a transport called 'mlmmj' but it is nothing more than a pipe(8) to the mlmmj-recieve program that is invoked as the 'mlmmj' user.

master.cf:
# mlmmj mailing lists
mlmmj unix - n n - - pipe
flags=DORhu user=mlmmj argv=/usr/local/bin/mlmmj-recieve -F -L /var/spool/mlmmj/$nexthop/

This takes the pipe(8) postfix delivery agent and tells it to invoke '/usr/local/bin/mlmmj-recieve' as the 'mlmmj' user and pipe the email to it on stdin. This mode of transportation is given the name 'mlmmj'.

The 'flags' parameter to pipe(8) is pretty critical here. In particular if the 'R' option is not used mlmmj-recieve fails to recieve the mail correctly. The options mean:

D - Prepend a 'Delivered-To: recipient' header O - Prepend an 'X-Original-To: recipient' header R - Prepend a 'Return-Path:'. header
h - fold $nexthop to lowercase
u - fold $recipient to lowercase

$nexthop gets set to what was on the right had side of the ':' in the transport file. The way we have that configured is that $nexthop will get set to the name of the mailing list.

Restart postfix and enjoy your new lists.