Skip to contentSkip to conversation

Add test of message queues to rt-tests

PATCH New Carsten Emde 3 participants 4 messages Mar 29, 2010 – Apr 1, 2010 Kbuild

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Add test of message queues to rt-tests

This patch adds the program pmqtest to the rt-tests suite. The test mechanism is the same as in ptsematest, svsematest and friends, but it uses message queues to synchronize the test threads. To test the - now hopefully fixed - kernel problem that occurred when a timeout was specified, the -T option is available.

On an 8-way machine, the test result may look like:

# pmqtest -Sp99 -i100 -d0
#0: ID19903, P99, CPU0, I100; #1: ID19904, P99, CPU0, Cycles 870686
#2: ID19905, P99, CPU1, I100; #3: ID19906, P99, CPU1, Cycles 872555
#4: ID19907, P99, CPU2, I100; #5: ID19908, P99, CPU2, Cycles 894958
#6: ID19909, P99, CPU3, I100; #7: ID19910, P99, CPU3, Cycles 897776
#8: ID19911, P99, CPU4, I100; #9: ID19912, P99, CPU4, Cycles 870067
#10: ID19913, P99, CPU5, I100; #11: ID19914, P99, CPU5, Cycles 872351
#12: ID19915, P99, CPU6, I100; #13: ID19916, P99, CPU6, Cycles 894460
#14: ID19917, P99, CPU7, I100; #15: ID19918, P99, CPU7, Cycles 897480
#1 -> #0, Min    1, Cur    4, Avg    5, Max   19
#3 -> #2, Min    1, Cur    7, Avg    5, Max   18
#5 -> #4, Min    1, Cur    5, Avg    5, Max   19
#7 -> #6, Min    1, Cur    5, Avg    4, Max   23
#9 -> #8, Min    1, Cur    4, Avg    6, Max   20
#11 -> #10, Min    1, Cur    8, Avg    5, Max   18
#13 -> #12, Min    1, Cur    5, Avg    5, Max   33
#15 -> #14, Min    1, Cur    5, Avg    4, Max   18
Makefile +7 -2
index d6ab07a..f98111b 100644
@@ -1,8 +1,8 @@
 VERSION_STRING = 0.68
 
 sources = cyclictest.c signaltest.c pi_stress.c rt-migrate-test.c	\
-	  ptsematest.c sigwaittest.c svsematest.c sendme.c pip_stress.c \
-	  hackbench.c
+	  ptsematest.c sigwaittest.c svsematest.c pmqtest.c sendme.c 	\
+	  pip_stress.c hackbench.c
 
 TARGETS = $(sources:.c=)
 
@@ -36,6 +36,7 @@ VPATH	+= src/rt-migrate-test:
 VPATH	+= src/ptsematest:
 VPATH	+= src/sigwaittest:
 VPATH	+= src/svsematest:
+VPATH	+= src/pmqtest:
 VPATH	+= src/backfire:
 VPATH	+= src/lib
 VPATH	+= src/hackbench
@@ -78,6 +79,9 @@ sigwaittest: sigwaittest.o rt-utils.o rt-get_cpu.o
 svsematest: svsematest.o rt-utils.o rt-get_cpu.o
 	$(CC) $(CFLAGS) -o $@ $^ $(LIBS) $(EXTRA_LIBS)
 
+pmqtest: pmqtest.o rt-utils.o rt-get_cpu.o
+	$(CC) $(CFLAGS) -o $@ $^ $(LIBS) $(EXTRA_LIBS)
+
 sendme: sendme.o rt-utils.o rt-get_cpu.o
 	$(CC) $(CFLAGS) -o $@ $^ $(LIBS) $(EXTRA_LIBS)
 
@@ -120,6 +124,7 @@ install: all
 	gzip src/ptsematest/ptsematest.8 -c >"$(DESTDIR)$(mandir)/man8/ptsematest.8.gz"
 	gzip src/sigwaittest/sigwaittest.8 -c >"$(DESTDIR)$(mandir)/man8/sigwaittest.8.gz"
 	gzip src/svsematest/svsematest.8 -c >"$(DESTDIR)$(mandir)/man8/svsematest.8.gz"
+	gzip src/pmqtest/pmqtest.8 -c >"$(DESTDIR)$(mandir)/man8/pmqtest.8.gz"
 	gzip src/backfire/sendme.8 -c >"$(DESTDIR)$(mandir)/man8/sendme.8.gz"
 	gzip src/hackbench/hackbench.8 -c >"$(DESTDIR)$(mandir)/man8/hackbench.8.gz"
 
rt-tests.spec-in +2 -0
index 6f45db4..7279813 100644
@@ -45,6 +45,7 @@ rm -rf $RPM_BUILD_ROOT
 /usr/bin/sendme
 /usr/bin/sigwaittest
 /usr/bin/svsematest
+/usr/bin/pmqtest
 /usr/bin/hackbench
 /usr/src/backfire/backfire.c
 %doc
@@ -56,6 +57,7 @@ rm -rf $RPM_BUILD_ROOT
 /usr/share/man/man8/sendme.8.gz
 /usr/share/man/man8/sigwaittest.8.gz
 /usr/share/man/man8/svsematest.8.gz
+/usr/share/man/man8/pmqtest.8.gz
 /usr/share/man/man8/hackbench.8.gz
 
 %changelog
src/pmqtest/Makefile new file +16 -0
new file mode 100644
index 0000000..0902c9b
@@ -0,0 +1,16 @@
+CFLAGS += -Wall -O2
+LDFLAGS += -lpthread
+
+all:	pmqtest
+	@echo Done
+
+pmqtest.o: pmqtest.c
+
+pmqtest:
+
+clean:
+	@rm -f *.o
+
+tar:	clean
+	@rm -f pmqtest
+	$(shell bn=`basename $$PWD`; cd ..; tar -zcf $$bn.tgz $$bn)
src/pmqtest/pmqtest.8 new file +68 -0

68 lines not on this page · raw message

src/pmqtest/pmqtest.c new file +510 -0

510 lines not on this page · raw message

#1 lore raw

PATCH v2 1/1 rt-tests: Add option to force timeout to pmqtest

Quoted 6 lines · Carsten Emde

This patch adds the program pmqtest to the rt-tests suite. The test mechanism is the same as in ptsematest, svsematest and friends, but it uses message queues to synchronize the test threads. To test the - now hopefully fixed - kernel problem that occurred when a timeout was specified, the -T option is available.

This patch adds the option -f to force a timeout condition of mq_timedreceive() to pmqtest - only meaningful along with -T.

The call

# pmqtest -Sp99 -i100 -d0 -T1 -f2

will display

#0: ID17791, P99, CPU0, I100; #1: ID17792, P99, CPU0, Cycles 8 #2: ID17793, P99, CPU1, I100; #3: ID17794, P99, CPU1, Cycles 8 #4: ID17795, P99, CPU2, I100; #5: ID17796, P99, CPU2, Cycles 8 #6: ID17797, P99, CPU3, I100; #7: ID17798, P99, CPU3, Cycles 8 #8: ID17799, P99, CPU4, I100; #9: ID17800, P99, CPU4, Cycles 8 #10: ID17801, P99, CPU5, I100; #11: ID17802, P99, CPU5, Cycles 8 #12: ID17803, P99, CPU6, I100; #13: ID17804, P99, CPU6, Cycles 8 #14: ID17805, P99, CPU7, I100; #15: ID17806, P99, CPU7, Cycles 8

#1 -> #0, Min   16, Cur   22, Avg   18, Max   24
#3 -> #2, Min   13, Cur   14, Avg   14, Max   16
#5 -> #4, Min   14, Cur   20, Avg   16, Max   20
#7 -> #6, Min   13, Cur   14, Avg   14, Max   15
#9 -> #8, Min   14, Cur   14, Avg   15, Max   19
#11 -> #10, Min   13, Cur   13, Avg   16, Max   20
#13 -> #12, Min   13, Cur   14, Avg   15, Max   19
#15 -> #14, Min   13, Cur   14, Avg   15, Max   19

etc.

but the next cycle will only continue when the timeout of 2 seconds is elapsed.

src/pmqtest/pmqtest.8 +3 -0
index 7c886c9..e2ffe77 100644
@@ -21,6 +21,9 @@ It is useful to track down unexpected large latencies of a system.
 .B \-d, \-\-distance=DIST
 Set the distance of thread intervals in microseconds (default is 500 us). When  cylictest is called with the -t option and more than one thread is created, then this distance value is added to the interval of the threads: Interval(thread N) = Interval(thread N-1) + DIST
 .TP
+.B \-f, \-\-forcetimeout=TO
+Set an artificial delay of the send function to force timeout of the receiver, requires the -T option
+.TP
 .B \-i, \-\-interval=INTV
 Set the base interval of the thread(s) in microseconds (default is 1000 us). This sets the interval of the first thread. See also -d.
 .TP
src/pmqtest/pmqtest.c +30 -5
index f464e20..4e20682 100644
@@ -77,6 +77,7 @@ struct params {
 	struct timeval sent, received, diff;
 	pthread_t threadid;
 	int timeout;
+	int forcetimeout;
 	mqd_t syncmq, testmq;
 	char recvsyncmsg[MSG_SIZE];
 	char recvtestmsg[MSG_SIZE];
@@ -110,6 +111,16 @@ void *pmqthread(void *param)
 
 	while (!par->shutdown) {
 		if (par->sender) {
+
+			/* Optionally force receiver timeout */
+			if (par->forcetimeout) {
+				struct timespec senddelay;
+				
+				senddelay.tv_sec = par->forcetimeout;
+				senddelay.tv_nsec = 0;
+				nanosleep(&senddelay, NULL);
+			}
+
 			/* Send message: Start of latency measurement ... */
 			gettimeofday(&par->sent, NULL);
 			if (mq_send(par->testmq, testmsg, strlen(testmsg), 1) != 0) {
@@ -148,11 +159,17 @@ void *pmqthread(void *param)
 			if (par->timeout) {
 				clock_gettime(CLOCK_REALTIME, &ts);
 				ts.tv_sec += par->timeout;
-				if (mq_timedreceive(par->testmq, par->recvtestmsg, MSG_SIZE, NULL, &ts) !=
-				    strlen(testmsg)) {
-					perror("could not receive test message");
-					par->shutdown = 1;
+				do {
+					if (mq_timedreceive(par->testmq, par->recvtestmsg,
+					    MSG_SIZE, NULL, &ts) != strlen(testmsg)) {
+						if (!par->forcetimeout || errno != ETIMEDOUT) {
+							perror("could not receive test message");
+							par->shutdown = 1;
+						}
+					} else
+						break;
 				}
+				while (1);
 			} else {
 				if (mq_receive(par->testmq, par->recvtestmsg, MSG_SIZE, NULL) !=
 				    strlen(testmsg)) {
@@ -224,6 +241,7 @@ static void display_help(void)
 	"                           with NUM pin all threads to the processor NUM\n"
 	"-b USEC  --breaktrace=USEC send break trace command when latency > USEC\n"
 	"-d DIST  --distance=DIST   distance of thread intervals in us default=500\n"
+	"-f TO    --forcetimeout=TO force timeout of mq_timedreceive(), requires -T\n"
 	"-i INTV  --interval=INTV   base interval of thread in us default=1000\n"
 	"-l LOOPS --loops=LOOPS     number of loops: default=0(endless)\n"
 	"-p PRIO  --prio=PRIO       priority\n"
@@ -250,6 +268,7 @@ static int distance = 500;
 static int smp;
 static int sameprio;
 static int timeout;
+static int forcetimeout;
 
 static void process_options (int argc, char *argv[])
 {
@@ -263,6 +282,7 @@ static void process_options (int argc, char *argv[])
 			{"affinity", optional_argument, NULL, 'a'},
 			{"breaktrace", required_argument, NULL, 'b'},
 			{"distance", required_argument, NULL, 'd'},
+			{"forcetimeout", required_argument, NULL, 'f'},
 			{"interval", required_argument, NULL, 'i'},
 			{"loops", required_argument, NULL, 'l'},
 			{"priority", required_argument, NULL, 'p'},
@@ -272,7 +292,7 @@ static void process_options (int argc, char *argv[])
 			{"help", no_argument, NULL, '?'},
 			{NULL, 0, NULL, 0}
 		};
-		int c = getopt_long (argc, argv, "a::b:d:i:l:p:St::T:",
+		int c = getopt_long (argc, argv, "a::b:d:f:i:l:p:St::T:",
 			long_options, &option_index);
 		if (c == -1)
 			break;
@@ -294,6 +314,7 @@ static void process_options (int argc, char *argv[])
 			break;
 		case 'b': tracelimit = atoi(optarg); break;
 		case 'd': distance = atoi(optarg); break;
+		case 'f': forcetimeout = atoi(optarg); break;
 		case 'i': interval = atoi(optarg); break;
 		case 'l': max_cycles = atoi(optarg); break;
 		case 'p': priority = atoi(optarg); break;
@@ -338,6 +359,9 @@ static void process_options (int argc, char *argv[])
 	if (num_threads < 1)
 		error = 1;
 
+	if (forcetimeout && !timeout)
+		error = 1;
+ 
 	if (priority && smp)
 		sameprio = 1;
 
@@ -426,6 +450,7 @@ int main(int argc, char *argv[])
 		receiver[i].sender = 0;
 		receiver[i].neighbor = &sender[i];
 		receiver[i].timeout = timeout;
+		receiver[i].forcetimeout = forcetimeout;
 		pthread_create(&receiver[i].threadid, NULL, pmqthread, &receiver[i]);
 		memcpy(&sender[i], &receiver[i], sizeof(receiver[0]));
 		sender[i].sender = 1;
#2 lore raw

Re: [PATCH] rt-tests: Add timeout count to pmqtest's output

Quoted 9 lines · Carsten Emde
Quoted 6 lines · Carsten Emde

This patch adds the program pmqtest to the rt-tests suite. The test mechanism is the same as in ptsematest, svsematest and friends, but it uses message queues to synchronize the test threads. To test the - now hopefully fixed - kernel problem that occurred when a timeout was specified, the -T option is available.

This patch adds the option -f to force a timeout condition of mq_timedreceive() to pmqtest - only meaningful along with -T.

This patch adds the timeout count (TO) to the output to prove that the timeout of the mq_timedreceive() call, in fact, works as expected.

The additional options -T1 -f2 will cause, for example, the display to be updated once per second with the lines:

#0: ID24739, P99, CPU0, I100; #1: ID24740, P99, CPU0, TO 0, Cycles 2 #0: ID24739, P99, CPU0, I100; #1: ID24740, P99, CPU0, TO 1, Cycles 2 #0: ID24739, P99, CPU0, I100; #1: ID24740, P99, CPU0, TO 0, Cycles 3 #0: ID24739, P99, CPU0, I100; #1: ID24740, P99, CPU0, TO 1, Cycles 3 #0: ID24739, P99, CPU0, I100; #1: ID24740, P99, CPU0, TO 1, Cycles 4 #0: ID24739, P99, CPU0, I100; #1: ID24740, P99, CPU0, TO 0, Cycles 5 #0: ID24739, P99, CPU0, I100; #1: ID24740, P99, CPU0, TO 1, Cycles 5

The related kernel patches to make the mq_* functions RT compliant are on their way.

src/pmqtest/pmqtest.c +65 -37
index 5c7a8da..42715b6 100644
@@ -78,6 +78,7 @@ struct params {
 	pthread_t threadid;
 	int timeout;
 	int forcetimeout;
+	int timeoutcount;
 	mqd_t syncmq, testmq;
 	char recvsyncmsg[MSG_SIZE];
 	char recvtestmsg[MSG_SIZE];
@@ -133,21 +134,10 @@ void *pmqthread(void *param)
 			if (mustgetcpu)
 				par->cpu = get_cpu();
 			/* Wait until receiver ready */
-			if (par->timeout) {
-				clock_gettime(CLOCK_REALTIME, &ts);
-				ts.tv_sec += par->timeout;
-
-				if (mq_timedreceive(par->syncmq, par->recvsyncmsg, MSG_SIZE, NULL, &ts)
-				    != strlen(syncmsg)) {
-					fprintf(stderr, "could not receive sync message\n");
-					par->shutdown = 1;
-				}
-			} else {
-				if (mq_receive(par->syncmq, par->recvsyncmsg, MSG_SIZE, NULL) !=
-				    strlen(syncmsg)) {
-					perror("could not receive sync message");
-					par->shutdown = 1;
-				}
+			if (mq_receive(par->syncmq, par->recvsyncmsg, MSG_SIZE, NULL) !=
+			    strlen(syncmsg)) {
+				perror("could not receive sync message");
+				par->shutdown = 1;
 			}
 			if (!par->shutdown && strcmp(syncmsg, par->recvsyncmsg)) {
 				fprintf(stderr, "ERROR: Sync message mismatch detected\n");
@@ -158,6 +148,7 @@ void *pmqthread(void *param)
 			/* Receiver */
 			if (par->timeout) {
 				clock_gettime(CLOCK_REALTIME, &ts);
+				par->timeoutcount = 0;
 				ts.tv_sec += par->timeout;
 				do {
 					if (mq_timedreceive(par->testmq, par->recvtestmsg,
@@ -165,6 +156,12 @@ void *pmqthread(void *param)
 						if (!par->forcetimeout || errno != ETIMEDOUT) {
 							perror("could not receive test message");
 							par->shutdown = 1;
+							break;
+						}
+						if (errno == ETIMEDOUT) {
+							par->timeoutcount++;
+							clock_gettime(CLOCK_REALTIME, &ts);
+							ts.tv_sec += par->timeout;
 						}
 					} else
 						break;
@@ -381,10 +378,13 @@ int main(int argc, char *argv[])
 {
 	int i;
 	int max_cpus = sysconf(_SC_NPROCESSORS_CONF);
-	int oldsamples = 1;
 	struct params *receiver = NULL;
 	struct params *sender = NULL;
 	sigset_t sigset;
+	int oldsamples = INT_MAX;
+	int oldtimeoutcount = INT_MAX;
+	int first = 1;
+	int errorlines = 0;
 	struct timespec maindelay;
 	int oflag = O_CREAT|O_RDWR;
 	struct mq_attr mqstat;
@@ -404,6 +404,11 @@ int main(int argc, char *argv[])
 		return 1;
 	}
 
+	sigemptyset(&sigset);
+	sigaddset(&sigset, SIGTERM);
+	sigaddset(&sigset, SIGINT);
+	pthread_sigmask(SIG_SETMASK, &sigset, NULL);
+
 	signal(SIGINT, sighand);
 	signal(SIGTERM, sighand);
 
@@ -461,20 +466,33 @@ int main(int argc, char *argv[])
 	maindelay.tv_sec = 0;
 	maindelay.tv_nsec = 50000000; /* 50 ms */
 
-	while (!shutdown) {
-		int printed;
-		int errorlines = 0;
+	sigemptyset(&sigset);
+	pthread_sigmask(SIG_SETMASK, &sigset, NULL);
 
-		for (i = 0; i < num_threads; i++)
-			shutdown |= receiver[i].shutdown | sender[i].shutdown;
+	do {
+		int newsamples = 0, newtimeoutcount = 0;
+		int minsamples = INT_MAX;
+
+		for (i = 0; i < num_threads; i++) {
+			newsamples += receiver[i].samples;
+			newtimeoutcount += receiver[i].timeoutcount;
+			if (receiver[i].samples < minsamples)
+				minsamples = receiver[i].samples;
+		}
+
+		if (minsamples > 1 && (shutdown || newsamples > oldsamples ||
+			newtimeoutcount > oldtimeoutcount)) {
+
+			if (!first)
+				printf("\033[%dA", num_threads*2 + errorlines);
+			first = 0;
 
-		if (receiver[0].samples > oldsamples || shutdown) {
 			for (i = 0; i < num_threads; i++) {
-				printf("#%1d: ID%d, P%d, CPU%d, I%ld; #%1d: ID%d, P%d, CPU%d, Cycles %d\n",
+				printf("#%1d: ID%d, P%d, CPU%d, I%ld; #%1d: ID%d, P%d, CPU%d, TO %d, Cycles %d   \n",
 				    i*2, receiver[i].tid, receiver[i].priority, receiver[i].cpu,
 				    receiver[i].delay.tv_nsec / 1000,
 				    i*2+1, sender[i].tid, sender[i].priority, sender[i].cpu,
-				    sender[i].samples);
+				    receiver[i].timeoutcount, sender[i].samples);
 			}
 			for (i = 0; i < num_threads; i++) {
 				printf("#%d -> #%d, Min %4d, Cur %4d, Avg %4d, Max %4d\n",
@@ -493,23 +511,26 @@ int main(int argc, char *argv[])
 					receiver[i].error[0] = '\0';
 				}
 			}
-			printed = 1;
-		} else
-			printed = 0;
-
-		sigemptyset(&sigset);
-		sigaddset(&sigset, SIGTERM);
-		sigaddset(&sigset, SIGINT);
-		pthread_sigmask(SIG_SETMASK, &sigset, NULL);
+		} else {
+			if (minsamples < 1)
+				printf("Collecting ...\n\033[1A");
+		}
+		
+		fflush(NULL);
+
+		oldsamples = 0;
+		oldtimeoutcount = 0;
+		for (i = 0; i < num_threads; i++) {
+			oldsamples += receiver[i].samples;
+			oldtimeoutcount += receiver[i].timeoutcount;
+		}
 
 		nanosleep(&maindelay, NULL);
 
-		sigemptyset(&sigset);
-		pthread_sigmask(SIG_SETMASK, &sigset, NULL);
+		for (i = 0; i < num_threads; i++)
+			shutdown |= receiver[i].shutdown | sender[i].shutdown;
 
-		if (printed && !shutdown)
-			printf("\033[%dA", num_threads*2 + errorlines);
-	}
+	} while (!shutdown);
 
 	for (i = 0; i < num_threads; i++) {
 		receiver[i].shutdown = 1;
@@ -524,12 +545,18 @@ int main(int argc, char *argv[])
 	}
 	nanosleep(&maindelay, NULL);
 	for (i = 0; i < num_threads; i++) {
+		char mqname[16];
+
 		mq_close(receiver[i].syncmq);
+		sprintf(mqname, SYNCMQ_NAME, i);
+		mq_unlink(mqname);
+
 		mq_close(receiver[i].testmq);
+		sprintf(mqname, TESTMQ_NAME, i);
+		mq_unlink(mqname);
 	}
 
 	nomem:
 
 	return 0;
 }
#3 lore raw

Re: [PATCH v2] rt-tests: Add option to force timeout to pmqtest

In reply to Carsten Emde
Quoted 31 lines · On Tue, 30 Mar 2010 16:25:49 +0200 Carsten Emde <[email protected]> wrote:
Quoted 6 lines · Carsten Emde

This patch adds the program pmqtest to the rt-tests suite. The test mechanism is the same as in ptsematest, svsematest and friends, but it uses message queues to synchronize the test threads. To test the - now hopefully fixed - kernel problem that occurred when a timeout was specified, the -T option is available.

This patch adds the option -f to force a timeout condition of mq_timedreceive() to pmqtest - only meaningful along with -T.

The call

# pmqtest -Sp99 -i100 -d0 -T1 -f2

will display

#0: ID17791, P99, CPU0, I100; #1: ID17792, P99, CPU0, Cycles 8 #2: ID17793, P99, CPU1, I100; #3: ID17794, P99, CPU1, Cycles 8 #4: ID17795, P99, CPU2, I100; #5: ID17796, P99, CPU2, Cycles 8 #6: ID17797, P99, CPU3, I100; #7: ID17798, P99, CPU3, Cycles 8 #8: ID17799, P99, CPU4, I100; #9: ID17800, P99, CPU4, Cycles 8 #10: ID17801, P99, CPU5, I100; #11: ID17802, P99, CPU5, Cycles 8 #12: ID17803, P99, CPU6, I100; #13: ID17804, P99, CPU6, Cycles 8 #14: ID17805, P99, CPU7, I100; #15: ID17806, P99, CPU7, Cycles 8

#1 -> #0, Min   16, Cur   22, Avg   18, Max   24
#3 -> #2, Min   13, Cur   14, Avg   14, Max   16
#5 -> #4, Min   14, Cur   20, Avg   16, Max   20
#7 -> #6, Min   13, Cur   14, Avg   14, Max   15
#9 -> #8, Min   14, Cur   14, Avg   15, Max   19
#11 -> #10, Min   13, Cur   13, Avg   16, Max   20
#13 -> #12, Min   13, Cur   14, Avg   15, Max   19
#15 -> #14, Min   13, Cur   14, Avg   15, Max   19
Quoted 6 lines

etc.

but the next cycle will only continue when the timeout of 2 seconds is elapsed.

Carsten,

Not ignoring you, just took a while for your task to get scheduled (I'm an old uni-processor :)).

I'll pull these in and spin a new rt-tests tonight.

Clark