Skip to contentSkip to conversation

System V msg queue bugs in latest kernels

DISCUSSION Mark Swanson 2 participants 5 messages Feb 17, 2001

lore
Share & tools
lore
#0 lore raw

Hello,

ipcs (msg) gives incorrect results if used-bytes is above 65536. It stays at 65536 even though messages are being read and removed from the msg queue.

The sysv msg queue either ignores the /proc/sys/kernel/msgmnb value if it is above 65536 or simply gets it wrong. Proof: I can place more than msgmnb bytes in a queue. The behavior is not consistent, but 100% reproducable. It's not consistent because if I use messages of about 1000-2000 bytes the msgmnb never gets bigger than 65536 (even if /proc/sys/kernel/msgmnb is set to 1048576 - bug). However, if I use small messages like 13 bytes I can get bizarre (wrong) ipcs results like this:

used-bytes messages
65536 65536

Why does Linux ignore the /proc/sys/kernel/msgmnb value - or seem to partly ignore it if it is above 65536? I *really* need this to be around a MB. Is there an undocumented limit here or is this a bug?

Thanks.

=====
A camel is ugly but useful; it may stink, and it may spit, but it'll get you where you're going. - Larry Wall -

__________________________________________________
Do You Yahoo!?
Get personalized email addresses from Yahoo! Mail - only $35 a year! http://personal.mail.yahoo.com/

#1 lore raw
Quoted 7 lines · Mark Swanson

Hello,

ipcs (msg) gives incorrect results if used-bytes is above 65536. It stays at 65536 even though messages are being read and removed from the msg queue.

I'm testing it.

Could you check /proc/sysvipc/msg?

I know that several API's have 16-bit numbers, perhaps wrong values are returned to user space.

--
Manfred

#2 lore raw

You are right.
/proc/sysvipc/msg is correct. It shows:
cbytes: 1048575
qnum: 95325

ipcs shows:
used-bytes: 65535
messages: 65535

It's a 16-bit number issue.

Quoted 11 lines · --- Manfred Spraul
Quoted 5 lines · Mark Swanson

Hello,

ipcs (msg) gives incorrect results if used-bytes is above 65536. It stays at 65536 even though messages are being read and removed from

the

Quoted 2 lines

msg queue.

I'm testing it.

Quoted 8 lines

Could you check /proc/sysvipc/msg?

I know that several API's have 16-bit numbers, perhaps wrong values are
returned to user space.

--
Manfred

=====
A camel is ugly but useful; it may stink, and it may spit, but it'll get you where you're going. - Larry Wall -

__________________________________________________
Do You Yahoo!?
Get personalized email addresses from Yahoo! Mail - only $35 a year! http://personal.mail.yahoo.com/

#3 lore raw
In reply to Manfred Spraul

The exact error is in /usr/include/linux/msg.h

The three unsigned shorts should be unsigned int instead. Would too many things break if this was changed?
Should user-space tools like ipcs be rewritten to use /proc/sysvipc instead? (I notice that my old 2.2.14 kernel doesn't have /proc/sysvipc...)

Thanks.

/* one msqid structure for each queue on the system */ struct msqid_ds {
struct ipc_perm msg_perm;
struct msg *msg_first; /* first message on queue */
struct msg *msg_last; /* last message in queue */
__kernel_time_t msg_stime; /* last msgsnd time */ __kernel_time_t msg_rtime; /* last msgrcv time */ __kernel_time_t msg_ctime; /* last change time */ struct wait_queue *wwait;
struct wait_queue *rwait;
unsigned short msg_cbytes; /* current number of bytes on queue */
unsigned short msg_qnum; /* number of messages in queue */
unsigned short msg_qbytes; /* max number of bytes on queue */
__kernel_ipc_pid_t msg_lspid; /* pid of last msgsnd */
__kernel_ipc_pid_t msg_lrpid; /* last receive pid */
};

=====
A camel is ugly but useful; it may stink, and it may spit, but it'll get you where you're going. - Larry Wall -

__________________________________________________
Do You Yahoo!?
Get personalized email addresses from Yahoo! Mail - only $35 a year! http://personal.mail.yahoo.com/

#4 lore raw
In reply to Mark Swanson
Quoted 9 lines · Manfred Spraul
Quoted 7 lines · Mark Swanson

Hello,

ipcs (msg) gives incorrect results if used-bytes is above 65536. It stays at 65536 even though messages are being read and removed from the msg queue.

Ok, does the value stay at 65536 or 65535?
It should stay at 65535 if you use a too old version of util-linux.

Please upgrade (see linux/Documentation/Changes)

The proc interface at /proc/sysvipc/msg should report the correct numbers.

If you want to access values > 65535 from your app you have 2 options:

1) use the new msqid64_ds structure. You must pass IPC_64 to the msgctl call. This is the only option if you need correct 32-bit uids. Check the util-linux source, I don't have sample code. msqid64_ds is only supported by the 2.4 kernel.

2) the old msqid_ds structure also support 32-bit queue length, an unused field was reused. No support for 32-bit uids.

#define msg_lqbytes __rwait;

I've attached my old sample code.
--
Manfred

/*
* This code is public domain sample code.
* Written by Manfred Spraul, 1999
*
* The application must be started by root or
* setuid(root).
*
* $Header: /pub/cvs/ms/ipcmsg/longqueue.c,v 1.2 1999/10/09 23:27:54 manfreds Exp $ */

#include <sys/msg.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

/* result codes:
* 0: success
* 1: partial success, queue len now USHORT_MAX
* 100: invalid parameters.
* 101: other error
* 256: fatal error, please delete the queue: queue len now 0. */

#define USHORT_MAX 0xFFff

struct queuelen {
	int llen;
	unsigned short slen;
};

struct msqid_ds g_q;

#define msg_lqbytes __rwait
void failure(char* msg)

{
	printf(" unexpected error in %s.\n",msg);
	exit(101);
}

int init_ipc(int id)

{
	int res;
	
	res = msgget(id,0);
	if(res == -1)
		failure("findkey()");
	id = res;
	res = msgctl(id,IPC_STAT,&g_q);
	if(res == -1)
		failure("init_ipc()");
	return id;
}

void get_queuelen(int id,
struct queuelen *out)

{
	int res;
	struct msqid_ds q;

	res = msgctl(id,IPC_STAT,&q);
	if(res == -1)
		failure("get_queuelen()");
	out->llen = q.msg_lqbytes;
	out->slen = q.msg_qbytes;
}

int set_queuelen(int id, int len)

{
	struct msqid_ds q;

	memcpy(&q,&g_q,sizeof(q));
	if(len > USHORT_MAX) {
		q.msg_qbytes = 0;
		q.msg_lqbytes = len;
	} else
	{
		q.msg_qbytes = len;
	}
	return msgctl(id,IPC_SET,&q);
}

int main(int argc,char** argv)

{
	int id;
	int len;
	struct queuelen prev;
	struct queuelen new;

	printf("longqueue <id> <len>\n");
	if(argc != 3) {
		printf("Invalid parameters.\n");
		return 100;
	}
	id = atoi(argv[1]);
	len = atoi(argv[2]);
	if(len <= 0) {
		printf("Invalid parameters.\n");
		return 100;
	}
	id = init_ipc(id);
	get_queuelen(id,&prev);
	if(set_queuelen(id,len) == -1)
		failure("set_queuelen()");
	if(len <= USHORT_MAX) {
out_success:
		get_queuelen(id,&prev);
		printf(" new queuelen: (%d,%d).\n",prev.slen,prev.llen);
		return 0;
	}
	/* the old Linux ipcmsg code doesn't support
	 * long queues. It interprets this as "queue len 0".
	 * Check for this, and try USHORT_MAX, then the original
	 * value.
	 */
	get_queuelen(id,&new);
	if(new.slen != 0)
		goto out_success;

	if(set_queuelen(id,USHORT_MAX) == -1) {
		if(set_queuelen(id,prev.slen) == -1) {
			printf(" fatal error. queue len now 0.\n");
			return 256;
		};
		failure("set_queuelen()");
	}
	get_queuelen(id,&new);
	printf(" new queuelen: (%d,%d).\n",prev.slen,prev.llen);
	return 1;
}