POSIX message queues should not allocate memory on send

There is a problem with the current POSIX message queue implementation. mq_send()/mq_timedsend() may not return ENOMEM and this means memory for mq_maxmsg*mq_msgsize will have to be allocated on queue creation. I think POSIX MSG message passing being part of the REALTIME extensions this makes sense. I've already mentioned this once to the implementors of the current implementation, but they did not agree with my reading of the standard.

The default mq_msgsize also seems a little large to me, but I don't see why defaults are needed; if I understand the standard correctly then creating a new message queue without mq_attr should create an empty queue, which thus cannot be used to pass messages.

--ms

No idea where you found this.

"If attr is NULL, the message queue shall be created with implementation-defined default message queue attributes."

Empty queue means a message queue which has no messages in it, not that mq_msgsize and/or mq_maxmsg is 0.
And mq_open with mq_msgsize 0 and/or mq_maxmsg 0 must fail (with EINVAL), so the implementation-defined defaults IMHO must be > 0 for both limits.

" The mq_open() function shall fail if:"
...
" [EINVAL]
O_CREAT was specified in oflag, the value of attr is not NULL, and either mq_maxmsg or mq_msgsize was less than or equal to zero."

Jakub

NULL,

You are correct; defaults are indeed needed. The current default value for mq_msgsize seems rather large considering that mq_msgsize*mq_maxmsg bytes will have to be allocated on queue creation. If variable sized large payload messages are needed one might consider using shared memory in combination with a message queue.

My main point was that mq_send()/mq_timedsend() may not return ENOMEM and I am positive I did not misread the standard on that.

--ms

Even that is not clear.
http://www.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_03.html#tag_02_03
" Implementations may support additional errors not included in this list,
may generate errors included in this list under circumstances other than those described here, or may contain extensions or limitations that prevent some errors from occurring. The ERRORS section on each reference page specifies whether an error shall be returned, or whether it may be returned. Implementations shall not generate a different error number from the ones described here for error conditions described in this volume of IEEE Std 1003.1-2001, but may generate additional errors unless explicitly disallowed for a particular function."

Explicitely disallowed in the general section is only EINTR for the THR option functions unless explitely listed for that function and nothing else.

I don't see ENOMEM explicitly forbidden for mq_send/mq_timedsend nor any wording in mq_open description which would require the buffers to be preallocated.

Jakub

large

http://www.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_03.html#tag_02_03

list,

than

whether

error

errors

else.

Indeed maybe returning ENOMEM from mq_send() is conforming to the standard. Perhaps returning ENOMEM instead of ENOSPC from mq_open() to indicate that there is unsufficient space for the creation of a new queue isn't; I'm no longer sure.

What _is_ clear from the standard is that a queue is supposed to be allocated on creation, even if it may not be required. Why else would ENOSPC be explicitely listed for mq_open() when creating a new queue, but not for mq_send()/mq_timedsend()?

What use is a message queue for a realtime application when on mq_send() it may have to wait for memory allocation that may even fail?

Even if POSIX allows errors to be returned that are not in the function's ERRORS list, I don't think one should do this if it can be avoided.

--ms

Indeed, it is.

Nothing is clear when it comes to the behavior. There are certainly some aspects of a possible implementation which the designers of the APIs had in mind, but this is absolutely no requirement.

That's a completely different issue. You want everybody to suffer because of the requirements of have. This is the same as if you would demand the overcommit is disable.

Yes, there are situations where the allocation-on-demand is not desirable and the POSIX API description is indeed hinting at such uses.
But the APIs are useful even without having such hard requirements and
so far the people who developed the code have not seen this hard realtime requirement.

If you have such requirements I suggest that you sit done and write some extensions. I.e., do not replace the current default behavior, but instead make it possible to force pre-allocation. I'd say the natural way is to define a O_* flag which only mq_open() and mq_setattr() understands. The former will get it in the second parameter, the latter via the mq_flags field in the incoming struct mq_attr structure. Something like O_PREALLOCATE. Again, the flag would only be needed for message queues, so a bit other than the ones used by mq_open() can be reused.

Do your part to fulfill your niche-requirements.

➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖

From: "Jakub Jelinek" <[email protected]>

You are correct, but from the mq_send/mq_timedsend DESCRIPTION:

``If the specified message queue is not full, mq_send() shall behave as if the message is inserted into the message queue at the position indicated by the msg_prio argument.''

and from ERRORS:

``The mq_send() and mq_timedsend() functions shall fail if:
[EAGAIN] The O_NONBLOCK flag is set in the message queue description associated with mqdes, and the specified message queue is full.''

Thus, if full could be interpreted as to also mean that there is no memory to store the message, returning ENOMEM is not allowed and EAGAIN would have to be used in stead since an implementation isn't allowed to use a different error number for the same error condition.

If full is taken to mean that the queue contains its maximum number of messages, which I think is the correct meaning, then returning ENOMEM is not allowed, i.e. the queue has to be preallocated, I think, since the description states that a message shall be added if the queue is not full.

--ms

The defaults have been reduced in -mm tree which should make it to mainline in relative near future.

#define DFLT_MSGMAX 10 /* max number of messages in each queue */
#define DFLT_MSGSIZEMAX 8192 /* max message size */

I'm not sure it's that clear, however, we somewhat adhered to this principle when adding rlimits to mqueues, so rlimit checks are enforced on mq_open() as if whole thing was pre-allocated.

thanks,
-chris

Linux Security Modules     http://lsm.immunix.org     http://lsm.bkbits.net