sigopen() vs. /dev/sigtimedwait

So I've been thinking about the sigopen() system call I proposed. (To recap: sigopen() would let you use read() instead of sigwaitinfo()
to retrieve lots of realtime signals at one go, AND would protect your signal from being swiped by hostile code elsewhere in the application, a la Sun's JDK.)

Upon further consideration, maybe I should model it after /dev/epoll. That would get rid of nagging questions like
"but read() can't leave holes like sigtimedwait could",
and would be even higher performance than read()
(see graphs at http://www.xmailserver.org/linux-patches/nio-improve.html )

So I'm proposing the following user story:

  // open a fd linked to signal mysignum
  int fd = open("/dev/sigtimedwait", O_RDWR);
  int sigs[1]; sigs[0] = mysignum;
  write(fd, sigs, sizeof(sigs[0]));

// memory map a result buffer
struct siginfo_t *map = mmap(NULL, mapsize, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);

  for (;;) {
      // grab recent siginfo_t's
      struct devsiginfo dsi;
      dsi.dsi_nsis = 1000;
      dsi.dsi_sis = NULL;      // NULL means "use map instead of buffer"
      dsi.dsi_timeout = 1;
      int nsis = ioctl(fd, DS_SIGTIMEDWAIT, &dvp);   

      // use 'em.  Some might be completion notifications; some might be readiness notifications.
      for (i=0; i<nsis; i++)
          handle_siginfo(map+i);

}

Sure, the interface is crap, but it's fast, and at least it doesn't add any syscalls (the sigopen() proposal required two new syscalls: sigopen() and timedread()).

Comments?

BTW I'm halfway thru "Understanding the Linux Kernel" and it's a very good read (modulo some strange lingo, e.g. "cycle" for "loop" and "table" for "record" or "struct").
So since I only halfway understand the linux kernel, the above proposal may be half baked.
- Dan

"I have seen the future, and it licks itself clean." -- Bucky Katt

And the advantage of this over /dev/epoll would be that you don't have to explicitly add/remove fd's?

I ask because yesterday I used /dev/epoll in a project and it behaves *very* well, so I'm wondering what advantages your interface would bring.

How do you handle signal queue overflow? signal-per-fd helps, but you still have to have the queue as big as the maximum number of fds is...

Petru

The advantage is that it can be used to collect completion notifications for aio. (It can also be
used to collect readiness notification via either linux's traditional rtsig stuff, or the signal-per-fd stuff, so this unifies readiness notification and completion notification, in case you happen to want to use both in the same thread.)

I am a huge fan of /dev/epoll and would like to see it integrated into the ac series. /dev/epoll doesn't address the needs of those who are doing aio, though.

I am not addressing that issue. However, when doing aio, the application can simply avoid issuing more than N I/O operations, where N is comfortably lower than the current size of the signal queue.

When I get around to reading the kernel source finally, maybe I'll have a look at what the costs of large signal queues are.

- Dan

"I have seen the future, and it licks itself clean." -- Bucky Katt

On the other hand, if /dev/epoll were flexible enough that it could deliver AIO completion notifications, then /dev/sigtimedwait would not be needed. For instance:

// extend bits/poll.h
#define POLLAIO 0x800 // aio completion event; pollfd.fd contains aiocb *

      // open /dev/epoll and set up map as usual
      kdpfd = open("/dev/epoll", O_RDWR);
      char *map = mmap(NULL, mapsize, PROT_READ | PROT_WRITE, MAP_PRIVATE, kdpfd, 0);

      // tell our /dev/epoll fd that we're interested in events on fd diskfd
      struct pollfd pfd;
      pfd.fd = diskfd;
      pfd.events = AIOEVENT;
      pfd.revents = 0;
      write(kdpfd, &pfd, sizeof(pfd));

      // set up an asynchronous read from 'diskfd'
      struct aiocb *r = malloc(sizeof(*r));
      r->aio_filedes = diskfd;
      r->aio_... = ...
      // when read is finished, have it notify the /dev/epoll device 
      // interested in diskfd rather than sending a signal
      r->aio_sigevent.sigev_notify = SIGEV_NONE;
      aio_read(r);
      ...

      // Pick up events
      for (;;) {
          struct devpoll dvp;
          dvp.dp_nfds = 1000;
          dvp.dp_fds = NULL;      // NULL means "use map instead of buffer"
          dvp.dp_timeout = 1;
          int nevents = ioctl(kdpfd, DS_SIGTIMEDWAIT, &dvp);
          struct pollfd *result = map + dvp.result_offset;

          // use 'em.  Some might be aio completion notifications; 
          // some might be traditional poll notifications
          // (and if this is AIX, some might be sysv message queue notifications!)
          for (i=0; i<nevents; i++)
             if (result[i].revents & POLLAIO)
                  handle_aio_completion((struct aiocb *)result[i].fd);
             else 
                  handle_readiness(&result[i]);
      }

Davide, is that along the lines of what you were thinking of
for /dev/epoll and disk files? (Plain old polling of disk
files doesn't make much sense unless you're just interested in them growing, I suppose; aio completion notification is what you really want.)

- Dan

"I have seen the future, and it licks itself clean." -- Bucky Katt

Dan, sorry for the response delay but I was in Vacation Mode ( new CPU execution mode that will be included in the next x86 generation ).
I trashed my original idea to extend /dev/epoll to other other files coz this will make the patch way more intrusive. The only easy extension that comes in my mind is pipes. As soon as I'll finish to read the remaining 4231 messages in my mbox I'll fix /dev/epoll to get rid of stale events that I ( and Erich Nahum ) noticed in the current implementation.

- Davide

As far as I know, Ben LaHaise ([email protected]) already has a fine method conceived for receiving batches of async completion, including an "async poll". It should give the sort of behaviour you want and is also useful for other AIO things, obviously :)

You should really chat with him.

- z

I suppose it shouldn't suprise me that real kernel hackers like Ben just work on cool things quietly, and bufoons like me get excited at the merest thought, and have to broadcast them on l-k. Sigh.

A little digging finds a few references to Ben's aio work:

http://uwsg.iu.edu/hypermail/linux/kernel/0102.0/0384.html
http://www.kvack.org/~blah/aio/v2.4.5-ac9-bcrl4-aio.diff

His patch creates /dev/aio, among other things, and includes the wonderful excerpt

diff -urN /md0/kernels/2.4/v2.4.5-ac9/include/asm-i386/errno.h aio-2.4.5-ac9/include/asm-i386/errno.h
@@ -128,5 +128,6 @@
 
 #define        ENOMEDIUM       123     /* No medium found */
 #define        EMEDIUMTYPE     124     /* Wrong medium type */
+#define        ENOCLUE         125     /* userland programmer induced race condition */
 

:-)

Given how occupied Ben is with urgent matters like tracking down the VM suckage, I don't expect he has much time for chatting about this stuff.

OK, guess I'll content myself with working through Ben's (and Davide's) code and understanding it. Sorry for all the line noise, folks.
- Dan

"I have seen the future, and it licks itself clean." -- Bucky Katt