convert udevsend/udevd to DGRAM and single-threaded

Kay, you said "unless we can get rid of _all_ the threads or at least getting faster, I don't want to change it."

Well how about we get rid of all the threads, *and* we get faster?

This patch applies to current bk trees, and does the following:

1) Switch to DGRAM sockets rather than STREAM. This simplifies things as mentioned in the previous message.

2) Invalid sequence numbers are mapped to -1 rather than zero, since zero is a valid sequence number (I think). Also, this allows for real speed tests using scripts starting at a zero sequence number, since that is what the initial expected sequence number is.

3) Get rid of all threading. This is the biggie. Some highlights:
a) timeout using setitimer() and SIGALRM
b) async child death notification via SIGCHLD
c) these two signal handlers do nothing but raise volatile flags, all the
work is done in the main loop
d) locking no longer required

I tested it using the following script

for i in seq 0 999;
do

	SEQNUM=$i ACTION=add DEVPATH=$i ./udevsend x;

done

Timing was done using the tsc, each number measures the total time taken to receive and process a hundred messages, from just after the first recv() call to just after the last child was reaped.

multi-threaded (secs):
0.433183
0.435601
0.289772
0.290449
0.298509
0.604979
0.998002
0.298365
0.827817
1.361001

single-threaded (secs):
0.205998
0.210246
0.219476
0.209425
0.203916
0.219981
0.215359
0.227679
0.218097
0.203311

Interestingly, not only is the single-threaded version faster, its also more deterministic.

I also tested it from 1 to 1000, causing all events to be delayed until the timeout period passed:

multithreaded:
3.213272
3.316999
3.446929
3.583432
3.711062
3.845539
3.987120
4.126300
4.244764
4.364072

single-threaded:
3.798218
3.799521
3.801352
3.803176
3.805020
3.806643
3.808498
3.810318
3.812169
3.814010

I do want to acknowledge the caveat that the threaded version may perform better on NPTL, but I think this is decent enough behaviour to warrent serious consideration.

Comments?

Chris

diff -Nur --exclude=RCS --exclude=CVS --exclude=SCCS --exclude=BitKeeper --exclude=ChangeSet udev/Makefile myudev/Makefile
@@ -262,7 +262,7 @@
 	$(STRIPCMD) $@
 
 $(DAEMON): udevd.h $(GEN_HEADERS) udevd.o
-	$(LD) $(LDFLAGS) -lpthread -o $@ $(CRT0) udevd.o $(LIB_OBJS) $(ARCH_LIB_OBJS)
+	$(LD) $(LDFLAGS) -o $@ $(CRT0) udevd.o $(LIB_OBJS) $(ARCH_LIB_OBJS)
 	$(STRIPCMD) $@
 
 $(SENDER): udevd.h $(GEN_HEADERS) udevsend.o

diff -Nur --exclude=RCS --exclude=CVS --exclude=SCCS --exclude=BitKeeper --exclude=ChangeSet udev/udevd.h myudev/udevd.h
@@ -24,6 +24,7 @@
 
 #include "list.h"
 
+#define UDEV_SOCK_NAME "udevd"
 #define UDEV_MAGIC			"udevd_" UDEV_VERSION
 #define EVENT_TIMEOUT_SEC		5
 #define UDEVSEND_CONNECT_RETRY		20 /* x 100 millisec */

Yes, we are twice as fast now on my box :)

I cleaned up the rest of the comments, the whitespace and a few names to match the whole thing. Please recheck it. Test script is switched to work on subsystem 'test' to let udev ignore it.

Chris, very nice work. Sweet to get rid of the threads. Greg, if it works for you too, I'm fine to switch over.

thanks,
Kay

===== Makefile 1.104 vs edited =====

@@ -262,7 +262,7 @@
 	$(STRIPCMD) $@
 
 $(DAEMON): udevd.h $(GEN_HEADERS) udevd.o
-	$(LD) $(LDFLAGS) -lpthread -o $@ $(CRT0) udevd.o $(LIB_OBJS) $(ARCH_LIB_OBJS)
+	$(LD) $(LDFLAGS) -o $@ $(CRT0) udevd.o $(LIB_OBJS) $(ARCH_LIB_OBJS)
 	$(STRIPCMD) $@
 
 $(SENDER): udevd.h $(GEN_HEADERS) udevsend.o

===== udevd.c 1.14 vs edited =====

===== udevsend.c 1.17 vs edited =====

===== test/udevd_test.sh 1.5 vs edited =====

*clap* *clap* *clap*

Thanks, Chris. Oh so sweet.

Robert Love

-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Very nice, thanks a lot Chris.

I've applied this patch.

greg k-h

-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

If udevd is not present, then the udevsend creates it. Otherwise, it blocks until the message is placed in udevd's buffer. No problem there.

If udevd crashes after the message was placed in its rx buffer but before handling it, then you have a problem, but this problem exists for any protocol and to solve it you need to have an ack message sent back to udevsend *after* the udev has run and returned for that message.

Chris

-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

I mean, SOCK_DGRAM is an unreliable transport, so what happens if the transport drops packets? It might be unlikely, especially for the AF_LOCAL, but it is possible.

I don't see any ack or retransmit code in udevsend.

-- Patrick Mansfield

-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

SOCK_DGRAM is a reliable transport under AF_UNIX / AF_LOCAL. From unix(7):

Valid types are SOCK_STREAM for a stream oriented socket and
SOCK_DGRAM
for a datagram oriented socket that preserves message
boundaries. Unix
sockets are always reliable and don't reorder datagrams.

Mike Waychison
Sun Microsystems, Inc.
1 (650) 352-5299 voice
1 (416) 202-8336 voice
mailto: [email protected]
http://www.sun.com

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE:  The opinions expressed in this email are held by me, 
and may not represent the views of Sun Microsystems, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 



-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Unix sockets are basically reliable. Barring something really wierd going on, if the server is present, a blocking sendto() (with valid parameters, of course) will succeed eventually, which guarantees that the message was queued up in the server's receive buffer. If the server is not present, then you get an errno of ECONNREFUSED, and you know to start the server.

From the man page: "Unix sockets are always reliable and don't reorder datagrams."

It would be easy to make it totally reliable by doing periodic retries in udevsend and have udevd send back an ack when cleaning up the message after udev completes. I could send this in if people are interested.

Chris

-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

OK, thanks for the explanations, I did not know about the "man unix" ;-)

-- Patrick Mansfield

-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

I tried to compile udevd with klibc. Here are the neccessary changes to compile it, but I doesn't work well now. There seems to be a issue with the signal handling in klibc.
Any idea?

Kay

===== udevd.c 1.15 vs edited =====

@@ -279,6 +279,7 @@
 	struct sockaddr_un saddr;
 	socklen_t addrlen;
 	int retval;
+	struct sigaction act;
 
 	init_logging("udevd");
 

@@ -288,8 +289,12 @@
 	signal(SIGCHLD, sig_handler);
 
 	/* we want these two to interrupt system calls */
-	siginterrupt(SIGALRM, 1);
-	siginterrupt(SIGCHLD, 1);
+	sigaction(SIGALRM, NULL, &act);
+	act.sa_flags &= ~SA_RESTART;
+	sigaction(SIGALRM, &act, NULL);
+	sigaction(SIGCHLD, NULL, &act);
+	act.sa_flags &= ~SA_RESTART;
+	sigaction(SIGCHLD, &act, NULL);
 
 	memset(&saddr, 0x00, sizeof(saddr));
 	saddr.sun_family = AF_LOCAL;

@@ -304,7 +309,7 @@
 	}
 
 	/* the bind takes care of ensuring only one copy running */
-	retval = bind(ssock, &saddr, addrlen);
+	retval = bind(ssock, (struct sockaddr *) &saddr, addrlen);
 	if (retval < 0) {
 		dbg("bind failed\n");
 		goto exit;

===== udevsend.c 1.18 vs edited =====

@@ -174,7 +174,7 @@
 	/* If we can't send, try to start daemon and resend message */
 	loop = UDEVSEND_CONNECT_RETRY;
 	while (loop--) {
-		retval = sendto(sock, &message, size, 0, (struct sockaddr*)&saddr, addrlen);
+		retval = sendto(sock, &message, size, 0, (struct sockaddr *)&saddr, addrlen);
 		if (retval != -1) {
 			retval = 0;
 			goto close_and_exit;