async network I/O, event channels, etc

I was very much surprised by the reactions I got after my OLS talk. Lots of people declared interest and even agreed with the approach and asked me to do further ahead with all this. For those who missed it, the paper and the slides are available on my home page:

http://people.redhat.com/drepper/

As for the next steps I see a number of possible ways. The discussions can be held on the usual mailing lists (i.e., lkml and netdev) but due to the raw nature of the current proposal I would imagine that would be mainly perceived as noise.

So the second proposal is the I can create an appropriate mailing list here. Alternatively possibly two lists, one for the event channel stuff and one for the DMA/AIO part since the two parts are very much unrelated.

I'd appreciate input from those who feel responsible for these lists. I don't mind either way. If there is strong interest right now (let me know, don't pollute the list) I can create mailing lists now and then if necessary close them down and redirect traffic to lkml/netdev.

Waiting for input...

I would prefer directing the API discussion to [email protected]

I was very much surprised by the reactions I got after my OLS talk. Lots of people declared interest and even agreed with the approach and asked me to do further ahead with all this. For those who missed it, the paper and the slides are available on my home page:

http://people.redhat.com/drepper/

As for the next steps I see a number of possible ways. The discussions can be held on the usual mailing lists (i.e., lkml and netdev) but due to the raw nature of the current proposal I would imagine that would be mainly perceived as noise.

From: Ulrich Drepper <[email protected]>
Date: Tue, 25 Jul 2006 12:23:53 -0700

Quoted 11 lines

I was very much surprised by the reactions I got after my OLS talk. Lots of people declared interest and even agreed with the approach and asked me to do further ahead with all this. For those who missed it, the paper and the slides are available on my home page:

http://people.redhat.com/drepper/

As for the next steps I see a number of possible ways. The discussions can be held on the usual mailing lists (i.e., lkml and netdev) but due to the raw nature of the current proposal I would imagine that would be mainly perceived as noise.

Since I gave a big thumbs up for Evgivny's kevent work yesterday on linux-kernel, you might want to start by comparing your work to his. Because his has the advantage that 1) we have code now and 2) he has written many test applications and performed many benchmarks against his code which has flushed out most of the major implementation issues.

I think most of the people who have encouraged your work are unaware of Evgivny's kevent stuff, which is extremely unfortunate, the two works are more similar than they are different.

I do not think discussing all of this on netdev would be perceived as noise. :)

From: Ulrich Drepper <[email protected]>
Date: Tue, 25 Jul 2006 12:23:53 -0700

Quoted 11 lines

I was very much surprised by the reactions I got after my OLS talk. Lots of people declared interest and even agreed with the approach and asked me to do further ahead with all this. For those who missed it, the paper and the slides are available on my home page:

http://people.redhat.com/drepper/

As for the next steps I see a number of possible ways. The discussions can be held on the usual mailing lists (i.e., lkml and netdev) but due to the raw nature of the current proposal I would imagine that would be mainly perceived as noise.

Since I gave a big thumbs up for Evgivny's kevent work yesterday on linux-kernel, you might want to start by comparing your work to his. Because his has the advantage that 1) we have code now and 2) he has written many test applications and performed many benchmarks against his code which has flushed out most of the major implementation issues.

I think most of the people who have encouraged your work are unaware of Evgivny's kevent stuff, which is extremely unfortunate, the two works are more similar than they are different.

I do not think discussing all of this on netdev would be perceived as noise. :)

This patch includes asynchronous propagation of file's data into VFS cache and aio_sendfile() implementation.
Network aio_sendfile() works lazily - it asynchronously populates pages into the VFS cache (which can be used for various tricks with adaptive readahead) and then uses usual ->sendfile() callback.

Signed-off-by Evgeniy Polyakov <[email protected]>

fs/bio.c +1 -1

index 6a0b9ad..a3ee530 100644
@@ -119,7 +119,7 @@ void bio_free(struct bio *bio, struct bi
 /*
  * default destructor for a bio allocated with bio_alloc_bioset()
  */
-static void bio_fs_destructor(struct bio *bio)
+void bio_fs_destructor(struct bio *bio)
 {
 	bio_free(bio, fs_bio_set);
 }

fs/ext2/inode.c +0 -0

index 04af9c4..295fce9 100644

fs/ext2/inode.c ext2_writepages(struct address_space *ma · line 688

Quoted 2 lines

struct address_space_operations ext2_aops = {

fs/ext2/inode.c +1 -0

+	.get_block		= ext2_get_block,

No way in hell. For whatever you do please provide a interface at the readpage/writepage/sendfile/etc abstraction layer. get_block is nothing that can be exposed to the common code.

Quoted 6 lines · Christoph Hellwig

fs/ext2/inode.c ext2_writepages(struct address_space *ma · line 688

Quoted 2 lines

struct address_space_operations ext2_aops = {

fs/ext2/inode.c +1 -0

+	.get_block		= ext2_get_block,

No way in hell. For whatever you do please provide a interface at the readpage/writepage/sendfile/etc abstraction layer. get_block is nothing that can be exposed to the common code.

Compare this with sync read methods - all they do is exactly the same operations with low-level blocks, which are combined into nice exported function, so there is _no_ readpage layer - it calls only one function which works with blocks.

So it is not a technical problem, but political one.

Quoted 8 lines · Evgeniy Polyakov

Quoted 6 lines · Christoph Hellwig

fs/ext2/inode.c ext2_writepages(struct address_space *ma · line 688

Quoted 2 lines

struct address_space_operations ext2_aops = {

fs/ext2/inode.c +1 -0

+	.get_block		= ext2_get_block,

No way in hell. For whatever you do please provide a interface at the readpage/writepage/sendfile/etc abstraction layer. get_block is nothing that can be exposed to the common code.

Quoted 4 lines

Compare this with sync read methods - all they do is exactly the same operations with low-level blocks, which are combined into nice exported function, so there is _no_ readpage layer - it calls only one function which works with blocks.

No. The abtraction layer there is ->readpage(s). _A_ common implementation works with a get_block callback from the filesystem, but there are various others. We've been there before, up to mid-2.3.x we had a get_block inode operation and we got rid of it because it is the wrong abstraction.

Quoted 1 line

So it is not a technical problem, but political one.

It's a technical problem, and it's called get you abstractions right. And ontop of that a political one and that's called get your abstraction coherent. If you managed to argue all of us into accept that get_block is the right abstraction (and as I mentioned above that's technically not true) you'd still have the burden to update everything to use the same abstraction.

This patch includes asynchronous propagation of file's data into VFS cache and aio_sendfile() implementation.
Network aio_sendfile() works lazily - it asynchronously populates pages into the VFS cache (which can be used for various tricks with adaptive readahead) and then uses usual ->sendfile() callback.

And to be honest, I don't think adding all this code is acceptable if it can't replace the existing aio code while keeping the interface. So while you interface looks pretty sane the implementation needs a lot of work still :)

From: Christoph Hellwig <[email protected]>
Date: Wed, 26 Jul 2006 11:04:31 +0100

Quoted 4 lines

And to be honest, I don't think adding all this code is acceptable if it can't replace the existing aio code while keeping the interface. So while you interface looks pretty sane the implementation needs a lot of work still :)

Networking and disk AIO have significantly different needs.

Therefore, I really don't see it as reasonable to expect a merge of these two things. It doesn't make any sense.

Quoted 4 lines

Networking and disk AIO have significantly different needs.

Therefore, I really don't see it as reasonable to expect a merge of these two things. It doesn't make any sense.

I'm not sure about that. The current aio interface isn't exactly nice for disk I/O either. I'm more than happy to have a discussion about that aspect.

From: Christoph Hellwig <[email protected]>
Date: Wed, 26 Jul 2006 11:04:31 +0100

Quoted 4 lines

And to be honest, I don't think adding all this code is acceptable if it can't replace the existing aio code while keeping the interface. So while you interface looks pretty sane the implementation needs a lot of work still :)

Networking and disk AIO have significantly different needs.

Quoted 6 lines · Evgeniy Polyakov

This patch includes asynchronous propagation of file's data into VFS cache and aio_sendfile() implementation.
Network aio_sendfile() works lazily - it asynchronously populates pages into the VFS cache (which can be used for various tricks with adaptive readahead) and then uses usual ->sendfile() callback.

And please don't base this on sendfile. Please make the splice infrastructure aynschronous without duplicating all the code but rather make the existing code aynch and the existing synchronous call wait on them to finish, similar to how we handle async/sync direct I/O. And to be honest, I don't think adding all this code is acceptable if it can't replace the existing aio code while keeping the interface. So while you interface looks pretty sane the implementation needs a lot of work still :)

I stopped to work on AIO, since neither existing, nor mine implementation were able to outperform sync speeds (one of the major problems in my implementation is get_user_pages() overhead, which can be completely eliminated with physical memory allocation being done in advance in userspace, like Ulrich described).
My personal opinion on existing AIO is that it is not the right design. Benjamin LaHaise agree with me (if I understood him right),

but he
failed to move AIO outside repeated-call model (2.4 had state machine based one, and out-of-the tree 2.6 patches have that design too). In theory existing AIO (with all posix userspace API) can be replaced with kevent (it will even take less space), but I would present it as a TODO item, since kevent itself has nothing to do with AIO.

Quoted 2 lines

My personal opinion on existing AIO is that it is not the right design. Benjamin LaHaise agree with me (if I understood him right),

I completely agree with that aswell.

Quoted 5 lines · Christoph Hellwig

Quoted 3 lines

My personal opinion on existing AIO is that it is not the right design. Benjamin LaHaise agree with me (if I understood him right),

I completely agree with that aswell.

I agree, too, but the current code is not the last of the line. Suparna has a st of patches which make the current kernel aio code work much better and especially make it really usable to implement POSIX AIO.

In Ottawa we were talking about submitting it and Suparna will. We just thought about a little longer timeframe. I guess it could be accelerated since he mostly has the patch done. But I don't know her schedule.

Important here is, don't base any decision on the current aio implementation.

Quoted 12 lines · Ulrich Drepper

Quoted 5 lines · Christoph Hellwig

Quoted 3 lines

My personal opinion on existing AIO is that it is not the right design. Benjamin LaHaise agree with me (if I understood him right),

I completely agree with that aswell.

I agree, too, but the current code is not the last of the line. Suparna has a st of patches which make the current kernel aio code work much better and especially make it really usable to implement POSIX AIO.

Quoted 8 lines

In Ottawa we were talking about submitting it and Suparna will. We just thought about a little longer timeframe. I guess it could be accelerated since he mostly has the patch done. But I don't know her schedule.

Important here is, don't base any decision on the current aio implementation.

Ulrich,

Suparna mentioned your interest in making POSIX glibc aio work with kernel-aio at OLS.
We thought taking a re-look at the (kernel side) work BULL did, would be a nice starting
point. I re-based those patches to 2.6.18-rc2 and sent it to Zach Brown for review before
sending them out to list.

These patches does NOT make AIO any cleaner. All they do is add functionality to support
POSIX AIO easier. These are

[ PATCH 1/3 ] Adding signal notification for event completion

[ PATCH 2/3 ] lio (listio) completion semantics

[ PATCH 3/3 ] cancel_fd support

Suparna explained these in the following article:

http://lwn.net/Articles/148755/

If you think, this is a reasonable direction/approach for the kernel and you would take care
of glibc side of things - I can spend time on these patches, getting them to reasonable shape
and push for inclusion.

Quoted 32 lines · Badari Pulavarty

Quoted 14 lines · Ulrich Drepper

Quoted 7 lines · Christoph Hellwig

Quoted 4 lines

My personal opinion on existing AIO is that it is not the right design. Benjamin LaHaise agree with me (if I understood him right),

I completely agree with that aswell.

I agree, too, but the current code is not the last of the line. Suparna has a st of patches which make the current kernel aio code work much better and especially make it really usable to implement POSIX AIO.

Quoted 9 lines

In Ottawa we were talking about submitting it and Suparna will. We just thought about a little longer timeframe. I guess it could be accelerated since he mostly has the patch done. But I don't know her schedule.

Important here is, don't base any decision on the current aio implementation.

Ulrich,

Suparna mentioned your interest in making POSIX glibc aio work with kernel-aio at OLS. We thought taking a re-look at the (kernel side) work BULL did, would be a nice starting point. I re-based those patches to 2.6.18-rc2 and sent it to Zach Brown for review before sending them out to list.

Quoted 9 lines

These patches does NOT make AIO any cleaner. All they do is add functionality to support POSIX AIO easier. These are

[ PATCH 1/3 ] Adding signal notification for event completion

[ PATCH 2/3 ] lio (listio) completion semantics

[ PATCH 3/3 ] cancel_fd support

Badari,

Thanks for refreshing those patches, they have been sitting here
for quite some time now and collected dust.

I also think Suparna's patchset for doing buffered AIO would be
a real plus here.

Quoted 8 lines

Suparna explained these in the following article:

http://lwn.net/Articles/148755/

If you think, this is a reasonable direction/approach for the kernel and you would take care of glibc side of things - I can spend time on these patches, getting them to reasonable shape and push for inclusion.

Ulrich, I you want to have a look at how those patches are put to
use in libposix-aio, have a look at http://sourceforge.net/projects/paiol.

It could be a starting point for glibc.

Thanks,

Sébastien.

Suparna mentioned at Ulrich wants us to concentrate on kernel-side support, so that he can look at glibc side of things (along with other work he is already doing). So, if we can get an agreement on what kind of kernel support is needed - we can focus our efforts on kernel side first and leave glibc enablement to capable hands of Uli :)

Quoted 6 lines

Suparna mentioned at Ulrich wants us to concentrate on kernel-side support, so that he can look at glibc side of things (along with other work he is already doing). So, if we can get an agreement on what kind of kernel support is needed - we can focus our efforts on kernel side first and leave glibc enablement to capable hands of Uli :)

Yeah, and the existing patches still need some cleanup. Badari, did you still want me to look into that?

We need someone to claim ultimate responsibility for getting these patches suitable for merging :). I'm happy to do that if Suparna isn't already on it.

Before we spend too much time cleaning up and merging into mainline - I would like an agreement that what we add is good enough for glibc POSIX AIO.

Quoted 3 lines · Badari Pulavarty

Before we spend too much time cleaning up and merging into mainline - I would like an agreement that what we add is good enough for glibc POSIX AIO.

I haven't seen a description of the interface so far. Would be good if it existed. But I briefly mentioned one quirk in the interface about which Suparna wasn't sure whether it's implemented/implementable in the current interface.

If a lio_listio call is made the individual requests are handle just as if they'd be issue separately. I.e., the notification specified in the individual aiocb is performed when the specific request is done. Then, once all requests are done, another notification is made, this time controlled by the sigevent parameter if lio_listio.

Another feature which I always wanted: the current lio_listio call returns in blocking mode only if all requests are done. In non-blocking mode it returns immediately and the program needs to poll the aiocbs. What is needed is something in the middle. For instance, if multiple read requests are issued the program might be able to start working as soon as one request is satisfied. I.e., a call similar to lio_listio would be nice which also takes another parameter specifying how many of the NENT aiocbs have to finish before the call returns.

Quoted 5 lines · Ulrich Drepper

Quoted 3 lines · Badari Pulavarty

Before we spend too much time cleaning up and merging into mainline - I would like an agreement that what we add is good enough for glibc POSIX AIO.

Quoted 4 lines

I haven't seen a description of the interface so far. Would be good if it existed. But I briefly mentioned one quirk in the interface about which Suparna wasn't sure whether it's implemented/implementable in the current interface.

Sebastien, could you provide a description of interfaces you are adding ? Since you did all the work, it would be appropriate for you to do it :)

Quoted 5 lines · Ulrich Drepper

Quoted 3 lines · Badari Pulavarty

Before we spend too much time cleaning up and merging into mainline - I would like an agreement that what we add is good enough for glibc POSIX AIO.

Quoted 4 lines

I haven't seen a description of the interface so far. Would be good if it existed. But I briefly mentioned one quirk in the interface about which Suparna wasn't sure whether it's implemented/implementable in the current interface.

Sebastien, could you provide a description of interfaces you are adding ? Since you did all the work, it would be appropriate for you to do it :)

aio completion notification

Details:
-------

A struct sigevent *aio_sigeventp is added to struct iocb in
include/linux/aio_abi.h

An enum {IO_NOTIFY_SIGNAL = 0, IO_NOTIFY_THREAD_ID = 1} is added in
include/linux/aio.h:

- IO_NOTIFY_SIGNAL means that the signal is to be sent to the requesting thread

- IO_NOTIFY_THREAD_ID means that the signal is to be sent to a specifi thread.

The following fields are added to struct kiocb in include/linux/aio.h:

- pid_t ki_pid: target of the signal

- __u16 ki_signo: signal number

- __u16 ki_notify: kind of notification, IO_NOTIFY_SIGNAL or IO_NOTIFY_THREAD_ID

- uid_t ki_uid, ki_euid: filled with the submitter credentials

- check whether the submitting thread wants to be notified directly (sigevent->sigev_notify_thread_id is 0) or wants the signal to be sent to another thread.
In the latter case a check is made to assert that the target thread is in the same thread group

listio support

Details:
-------

An IOCB_CMD_GROUP is added to the IOCB_CMD enum in include/linux/aio_abi.h

A struct lio_event is added in include/linux/aio.h

A struct lio_event *ki_lio is added to struct iocb in include/linux/aio.h

Quoted 1 line · Sébastien Dugué

aio completion notification

I looked over this now but I don't think I understand everything. Or I don't see how it all is integrated. And no, I'm not looking at the proposed glibc code since would mean being tainted.

Quoted 14 lines

Details:
-------

A struct sigevent *aio_sigeventp is added to struct iocb in
include/linux/aio_abi.h

An enum {IO_NOTIFY_SIGNAL = 0, IO_NOTIFY_THREAD_ID = 1} is added in
include/linux/aio.h:

- IO_NOTIFY_SIGNAL means that the signal is to be sent to the requesting thread

Quoted 2 lines

- IO_NOTIFY_THREAD_ID means that the signal is to be sent to a specifi thread.

This has been proved to be sufficient in the timer code which basically has the same problem. But why do you need separate constants? We have the various SIGEV_* constants, among them SIGEV_THREAD_ID. Just use these constants for the values of ki_notify.

Quoted 7 lines

The following fields are added to struct kiocb in include/linux/aio.h:

- pid_t ki_pid: target of the signal

- __u16 ki_signo: signal number

Quoted 4 lines

- __u16 ki_notify: kind of notification, IO_NOTIFY_SIGNAL or IO_NOTIFY_THREAD_ID

- uid_t ki_uid, ki_euid: filled with the submitter credentials

These two fields aren't needed for the POSIX interfaces. Where does the requirement come from? I don't say they should be removed, they might be useful, but if the costs are non-negligible then they could go away.

Quoted 7 lines

- check whether the submitting thread wants to be notified directly (sigevent->sigev_notify_thread_id is 0) or wants the signal to be sent to another thread.
In the latter case a check is made to assert that the target thread is in the same thread group

Is this really how it's implemented? This is not how it should be. Either a signal is sent to a specific thread in the same process (this is what SIGEV_THREAD_ID is for) or the signal is sent to a calling process. Sending a signal to the process means that from the kernel's POV any thread which doesn't have the signal blocked can receive it. The final decision is made by the kernel. There is no mechanism to send the signal to another process.

So, for the purpose of the POSIX AIO code the ki_pid value is only needed when the SIGEV_THREAD_ID bit is set.

It could be an extension and I don't mind it being introduced. But again, it's not necessary and if it adds costs then it could be left out. It is something which could easily be introduced later if the need arises.

Quoted 2 lines

listio support

I really don't understand the kernel interface for this feature.

Quoted 10 lines

Details:
-------

An IOCB_CMD_GROUP is added to the IOCB_CMD enum in include/linux/aio_abi.h

A struct lio_event is added in include/linux/aio.h

A struct lio_event *ki_lio is added to struct iocb in include/linux/aio.h

So you have a pointer in the structure for the individual requests. I assume you use the atomic counter to trigger the final delivery. I further assume that if lio_wait is set the calling thread is suspended until all requests are handled and that the final notification in this case means that thread gets woken.

This is all fine.

But how do you pass the requests to the kernel? If you have a new lio_listio-like syscall it'll be easy. But I haven't seen anything like this mentioned.

The alternative is to pass the requests one-by-one in which case I don't see how you create the reference to the lio_listio control block. This approach seems to be slower.

If all requests are passed at once, do you have the equivalent of LIO_NOP entries?

How can we support the extension where we wait for a number of requests which need not be all of them. I.e., I submit N requests and want to be notified when at least M (M <= N) notified. I am not yet clear about the actual semantics we should implement (e.g., do we send another notification after the first one?) but it's something which IMO should be taken into account in the design.

Finally, and this is very important, does you code send out the individual requests notification and then in the end the lio_listio completion? I think Suparna wrote this is the case but I want to make sure.

Overall, this looks much better than the old code. If the answers to my questions show that the behavior is compatible with the POSIX AIO code I'm certainly very much in favor of adding the kernel code.

Signature · 2 lines

-- 
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖

I am wondering about that too. IIRC, the IO_NOTIFY_* constants are not part of the ABI, but only internal to the kernel implementation. I think Zach had suggested inferring THREAD_ID notification if the pid specified is not zero. But, I don't see why ->sigev_notify couldn't used directly (just like the POSIX timers code does) thus doing away with the new constants altogether. Sebestian/Laurent, do you recall?

I'm guessing they are being used for validation of permissions at the time of sending the signal, but maybe saving the task pointer in the iocb instead of the pid would suffice ?

So I think the
intended behaviour is as you describe it should be

The way it works (and better ideas are welcome) is that, since the io_submit() syscall already accepts an array of iocbs[], no new syscall was introduced. To implement lio_listio, one has to set up such an array, with the first iocb in the array having the special (new) grouping opcode of IOCB_CMD_GROUP which specifies the sigev notification to be associated with group completion (a NULL value of the sigev notification pointer would imply equivalent of LIO_WAIT).

My thought here was that it should be possible to include M as a parameter to the IOCB_CMD_GROUP opcode iocb, and thus incorporated in the lio control block ... then whatever semantics are agreed upon can be implemented.

Let us know what you think about the listio interface ... hopefully the other issues are mostly simple to resolve.

Quoted 6 lines

I am wondering about that too. IIRC, the IO_NOTIFY_* constants are not part of the ABI, but only internal to the kernel implementation. I think Zach had suggested inferring THREAD_ID notification if the pid specified is not zero. But, I don't see why ->sigev_notify couldn't used directly (just like the POSIX timers code does) thus doing away with the new constants altogether. Sebestian/Laurent, do you recall?

I suggest to model the implementation after the timer code which does exactly what we need.

Quoted 7 lines · Ulrich Drepper

Quoted 6 lines

I am wondering about that too. IIRC, the IO_NOTIFY_* constants are not part of the ABI, but only internal to the kernel implementation. I think Zach had suggested inferring THREAD_ID notification if the pid specified is not zero. But, I don't see why ->sigev_notify couldn't used directly (just like the POSIX timers code does) thus doing away with the new constants altogether. Sebestian/Laurent, do you recall?

Quoted 2 lines

I suggest to model the implementation after the timer code which does exactly what we need.

Yeah, and if at all possible we want to use just one helper thread for SIGEV_THREAD notification of timers/aio/etc., so it really should behave the same as timer thread notification.

Quoted 6 lines · Suparna Bhattacharya

I am wondering about that too. IIRC, the IO_NOTIFY_* constants are not part of the ABI, but only internal to the kernel implementation. I think Zach had suggested inferring THREAD_ID notification if the pid specified is not zero. But, I don't see why ->sigev_notify couldn't used directly (just like the POSIX timers code does) thus doing away with the new constants altogether. Sebestian/Laurent, do you recall?

I suggest to model the implementation after the timer code which does exactly what we need.

Quoted 5 lines

I'm guessing they are being used for validation of permissions at the time of sending the signal, but maybe saving the task pointer in the iocb instead of the pid would suffice ?

Why should any verification be necessary? The requests are generated in the same process which will receive the notification. Even if the POSIX process (aka, kernel process group) changes the IDs the notifications should be set. The key is that notifications cannot be sent to another POSIX process.

Adding this as a feature just makes things so much more complicated.

Quoted 2 lines

So I think the
intended behaviour is as you describe it should be

Then the documentation needs to be adjusted.

Quoted 9 lines

The way it works (and better ideas are welcome) is that, since the io_submit() syscall already accepts an array of iocbs[], no new syscall was introduced. To implement lio_listio, one has to set up such an array, with the first iocb in the array having the special (new) grouping opcode of IOCB_CMD_GROUP which specifies the sigev notification to be associated with group completion (a NULL value of the sigev notification pointer would imply equivalent of LIO_WAIT).

OK, this seems OK. We have to construct the iocb arrays dynamically anyway.

Quoted 3 lines

My thought here was that it should be possible to include M as a parameter to the IOCB_CMD_GROUP opcode iocb, and thus incorporated in the lio control block ... then whatever semantics are agreed upon can be implemented.

If you have room for the parameter this is fine. For the beginning we can enforce the number to be the same as the total number of requests.

Quoted 3 lines

Let us know what you think about the listio interface ... hopefully the other issues are mostly simple to resolve.

It should be fine and I would support adding all this assuming the normal file support (as opposed to direct I/O only) is added, too.

But I have one last question: sockets, pipes and the like are already supported, right? If this is not the case we have a problem with the currently proposed lio_listio interface.

Signature · 2 lines

-- 
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖

Is there a (remote) possibility that the thread could have died and its pid got reused by a new thread in another process ? Or is there a mechanism that prevents such a possibility from arising (not just in NPTL library, but at the kernel level) ?

AIO for pipes should not be a problem - Chris Mason had a patch, so we can just bring it up to the current levels, possibly with some additional improvements.

I'm not sure what would be the right thing to do for the sockets case. While we could put together a patch for basic aio_read/write (based on the same model used for files), given the whole ongoing kevent effort, its not yet clear to me what would make the most sense ...

Ben had a patch to do a fallback to kernel threads for AIO operations that are not yet supported natively. I had some concerns about the approach, but I guess he had intended it as an interim path for cases like this.

Quoted 4 lines · Suparna Bhattacharya

Is there a (remote) possibility that the thread could have died and its pid got reused by a new thread in another process ? Or is there a mechanism that prevents such a possibility from arising (not just in NPTL library, but at the kernel level) ?

The UID/GID won't help you with dying processes. What if the same user creates a process with the same PID? That process will not expect the notification and mustn't receive it. If you cannot detect whether the issuing process died you have problems which cannot be solved with a uid/gid pair.

Quoted 6 lines · Suparna Bhattacharya

I am wondering about that too. IIRC, the IO_NOTIFY_* constants are not part of the ABI, but only internal to the kernel implementation. I think Zach had suggested inferring THREAD_ID notification if the pid specified is not zero. But, I don't see why ->sigev_notify couldn't used directly (just like the POSIX timers code does) thus doing away with the new constants altogether. Sebestian/Laurent, do you recall?

I suggest to model the implementation after the timer code which does exactly what we need.

Quoted 4 lines

I'm guessing they are being used for validation of permissions at the time of sending the signal, but maybe saving the task pointer in the iocb instead of the pid would suffice ?

Why should any verification be necessary? The requests are generated in the same process which will receive the notification. Even if the POSIX process (aka, kernel process group) changes the IDs the notifications should be set. The key is that notifications cannot be sent to another POSIX process.

Adding this as a feature just makes things so much more complicated.

BTW, if anyone would like to be dropped off this growing cc list, please let us know.

Quoted 6 lines · Ulrich Drepper

Quoted 1 line · Sébastien Dugué

aio completion notification

I looked over this now but I don't think I understand everything. Or I don't see how it all is integrated. And no, I'm not looking at the proposed glibc code since would mean being tainted.

Oh, I didn't realise that.
I'll make an attempt to clarify parts that I understand based on what I have gleaned from my reading of the code and intent, but hopefully Sebastien, Ben, Zach et al will be able to pitch in for a more accurate and complete picture.

Quoted 14 lines

Quoted 14 lines

Details:
-------

A struct sigevent *aio_sigeventp is added to struct iocb in
include/linux/aio_abi.h

An enum {IO_NOTIFY_SIGNAL = 0, IO_NOTIFY_THREAD_ID = 1} is added in
include/linux/aio.h:

- IO_NOTIFY_SIGNAL means that the signal is to be sent to the requesting thread

Quoted 8 lines

Quoted 2 lines

- IO_NOTIFY_THREAD_ID means that the signal is to be sent to a specifi thread.

This has been proved to be sufficient in the timer code which basically has the same problem. But why do you need separate constants? We have the various SIGEV_* constants, among them SIGEV_THREAD_ID. Just use these constants for the values of ki_notify.

I am wondering about that too. IIRC, the IO_NOTIFY_* constants are not part of the ABI, but only internal to the kernel implementation. I think Zach had suggested inferring THREAD_ID notification if the pid specified is not zero. But, I don't see why ->sigev_notify couldn't used directly (just like the POSIX timers code does) thus doing away with the new constants altogether. Sebestian/Laurent, do you recall?

Quoted 13 lines

Quoted 12 lines

The following fields are added to struct kiocb in include/linux/aio.h:

- pid_t ki_pid: target of the signal

- __u16 ki_signo: signal number

- __u16 ki_notify: kind of notification, IO_NOTIFY_SIGNAL or IO_NOTIFY_THREAD_ID

- uid_t ki_uid, ki_euid: filled with the submitter credentials

Quoted 3 lines

These two fields aren't needed for the POSIX interfaces. Where does the requirement come from? I don't say they should be removed, they might be useful, but if the costs are non-negligible then they could go away.

I'm guessing they are being used for validation of permissions at the time of sending the signal, but maybe saving the task pointer in the iocb instead of the pid would suffice ?

Quoted 9 lines

Quoted 8 lines

- check whether the submitting thread wants to be notified directly (sigevent->sigev_notify_thread_id is 0) or wants the signal to be sent to another thread.
In the latter case a check is made to assert that the target thread is in the same thread group

Quoted 7 lines

Is this really how it's implemented? This is not how it should be. Either a signal is sent to a specific thread in the same process (this is what SIGEV_THREAD_ID is for) or the signal is sent to a calling process. Sending a signal to the process means that from the kernel's POV any thread which doesn't have the signal blocked can receive it. The final decision is made by the kernel. There is no mechanism to send the signal to another process.

The code seems to be set up to call specific_send_sig_info() in the case of *_THREAD_ID , and __group_send_sig_info() otherwise. So I think the intended behaviour is as you describe it should be (__group_send_sig_info does the equivalent of sending a signal to the process and so any thread which doesn't have signals blocked can receive it, while specific_send_sig_info sends it to a particular thread).

But, I should really leave it to Sebestian to confirm that.

Quoted 4 lines

Quoted 2 lines

listio support

I really don't understand the kernel interface for this feature.

I'm sorry this is confusing. This probably means that we need to separate the external interface description more clearly and completely from the internals.

Quoted 19 lines

Quoted 10 lines

Details:
-------

An IOCB_CMD_GROUP is added to the IOCB_CMD enum in include/linux/aio_abi.h

A struct lio_event is added in include/linux/aio.h

A struct lio_event *ki_lio is added to struct iocb in include/linux/aio.h

So you have a pointer in the structure for the individual requests. I assume you use the atomic counter to trigger the final delivery. I further assume that if lio_wait is set the calling thread is suspended until all requests are handled and that the final notification in this case means that thread gets woken.

This is all fine.

Quoted 7 lines

But how do you pass the requests to the kernel? If you have a new lio_listio-like syscall it'll be easy. But I haven't seen anything like this mentioned.

The alternative is to pass the requests one-by-one in which case I don't see how you create the reference to the lio_listio control block. This approach seems to be slower.

The way it works (and better ideas are welcome) is that, since the io_submit() syscall already accepts an array of iocbs[], no new syscall was introduced. To implement lio_listio, one has to set up such an array, with the first iocb in the array having the special (new) grouping opcode of IOCB_CMD_GROUP which specifies the sigev notification to be associated with group completion (a NULL value of the sigev notification pointer would imply equivalent of LIO_WAIT). The following iocbs in the array should correspond to the set of listio aiocbs. Whenever it encounters an IOCB_CMD_GROUP iocb opcode, the kernel would interpret all subsequent iocbs[] submitted in the same io_submit() call to be associated with the same lio control block.

Does that clarify ?

Would an example help ?

Quoted 3 lines

If all requests are passed at once, do you have the equivalent of LIO_NOP entries?

Quoted 1 line

Good question - we do have an IOCB_CMD_NOOP defined, and I seem to even recall a patch that implemented it, but am wondering if it ever got merged. Ben/Zach ?

Quoted 8 lines

How can we support the extension where we wait for a number of requests which need not be all of them. I.e., I submit N requests and want to be notified when at least M (M <= N) notified. I am not yet clear about the actual semantics we should implement (e.g., do we send another notification after the first one?) but it's something which IMO should be taken into account in the design.

My thought here was that it should be possible to include M as a parameter to the IOCB_CMD_GROUP opcode iocb, and thus incorporated in the lio control block ... then whatever semantics are agreed upon can be implemented.

Quoted 1 line

Quoted 3 lines

Finally, and this is very important, does you code send out the individual requests notification and then in the end the lio_listio completion? I think Suparna wrote this is the case but I want to make sure.

Sebestian, could you confirm ?

Quoted 3 lines · Badari Pulavarty

Before we spend too much time cleaning up and merging into mainline - I would like an agreement that what we add is good enough for glibc POSIX AIO.

I haven't seen a description of the interface so far. Would be good if it existed. But I briefly mentioned one quirk in the interface about which Suparna wasn't sure whether it's implemented/implementable in the current interface.

If a lio_listio call is made the individual requests are handle just as if they'd be issue separately. I.e., the notification specified in the individual aiocb is performed when the specific request is done. Then, once all requests are done, another notification is made, this time controlled by the sigevent parameter if lio_listio.

Another feature which I always wanted: the current lio_listio call returns in blocking mode only if all requests are done. In non-blocking mode it returns immediately and the program needs to poll the aiocbs. What is needed is something in the middle. For instance, if multiple read requests are issued the program might be able to start working as soon as one request is satisfied. I.e., a call similar to lio_listio would be nice which also takes another parameter specifying how many of the NENT aiocbs have to finish before the call returns.

Quoted 3 lines · Badari Pulavarty

Before we spend too much time cleaning up and merging into mainline - I would like an agreement that what we add is good enough for glibc POSIX AIO.

I haven't seen a description of the interface so far. Would be good if

it existed. But I briefly mentioned one quirk in the interface about which Suparna wasn't sure whether it's implemented/implementable in the current interface.

If a lio_listio call is made the individual requests are handle just as if they'd be issue separately. I.e., the notification specified in the individual aiocb is performed when the specific request is done. Then, once all requests are done, another notification is made, this time controlled by the sigevent parameter if lio_listio.

Another feature which I always wanted: the current lio_listio call returns in blocking mode only if all requests are done. In non-blocking mode it returns immediately and the program needs to poll the aiocbs. What is needed is something in the middle. For instance, if multiple read requests are issued the program might be able to start working as soon as one request is satisfied. I.e., a call similar to lio_listio would be nice which also takes another parameter specifying how many of the NENT aiocbs have to finish before the call returns.

Quoted 68 lines · Sébastien Dugué

Quoted 39 lines · Badari Pulavarty

Quoted 14 lines · Ulrich Drepper

Quoted 7 lines · Christoph Hellwig

Quoted 4 lines

My personal opinion on existing AIO is that it is not the right design. Benjamin LaHaise agree with me (if I understood him right),

I completely agree with that aswell.

I agree, too, but the current code is not the last of the line. Suparna has a st of patches which make the current kernel aio code work much better and especially make it really usable to implement POSIX AIO.

Quoted 9 lines

In Ottawa we were talking about submitting it and Suparna will. We just thought about a little longer timeframe. I guess it could be accelerated since he mostly has the patch done. But I don't know her schedule.

Important here is, don't base any decision on the current aio implementation.

Ulrich,

Suparna mentioned your interest in making POSIX glibc aio work with kernel-aio at OLS.
We thought taking a re-look at the (kernel side) work BULL did, would be a nice starting
point. I re-based those patches to 2.6.18-rc2 and sent it to Zach Brown for review before
sending them out to list.

These patches does NOT make AIO any cleaner. All they do is add functionality to support
POSIX AIO easier. These are

Quoted 6 lines

[ PATCH 1/3 ] Adding signal notification for event completion

[ PATCH 2/3 ] lio (listio) completion semantics

[ PATCH 3/3 ] cancel_fd support

Badari,

Thanks for refreshing those patches, they have been sitting here
for quite some time now and collected dust.

I also think Suparna's patchset for doing buffered AIO would be
a real plus here.

Quoted 10 lines

Suparna explained these in the following article:

http://lwn.net/Articles/148755/

If you think, this is a reasonable direction/approach for the kernel and you would take care
of glibc side of things - I can spend time on these patches, getting them to reasonable shape
and push for inclusion.

Quoted 10 lines

Ulrich, I you want to have a look at how those patches are put to
use in libposix-aio, have a look at http://sourceforge.net/projects/paiol.

It could be a starting point for glibc.

Thanks,

Sébastien.

Sebastien,

Suparna mentioned at Ulrich wants us to concentrate on kernel-side support, so that he
can look at glibc side of things (along with other work he is already doing). So, if we
can get an agreement on what kind of kernel support is needed - we can focus our
efforts on kernel side first and leave glibc enablement to capable hands of Uli :)

kernel/kevent/kevent_user.c new file +24 -0

+static int kevent_ctl_process(struct file *file, 
+		struct kevent_user_control *ctl, void __user *arg)
+{
+	int err;
+	struct kevent_user *u = file->private_data;
+
+	if (!u)
+		return -EINVAL;
+
+	switch (ctl->cmd) {
+		case KEVENT_CTL_ADD:
+			err = kevent_user_ctl_add(u, ctl, 
+					arg+sizeof(struct kevent_user_control));
+			break;
+		case KEVENT_CTL_REMOVE:
+			err = kevent_user_ctl_remove(u, ctl, 
+					arg+sizeof(struct kevent_user_control));
+			break;
+		case KEVENT_CTL_MODIFY:
+			err = kevent_user_ctl_modify(u, ctl, 
+					arg+sizeof(struct kevent_user_control));
+			break;
+		case KEVENT_CTL_WAIT:
+			err = kevent_user_wait(file, u, ctl, arg);

kernel/kevent/kevent_user.c new file +11 -0

+asmlinkage long sys_kevent_ctl(int fd, void __user *arg)
+{
+	int err, fput_needed;
+	struct kevent_user_control ctl;
+	struct file *file;
+
+	if (copy_from_user(&ctl, arg, sizeof(struct kevent_user_control)))
+		return -EINVAL;
+
+	if (ctl.cmd == KEVENT_CTL_INIT)
+		return kevent_ctl_init();

Please indent the body of the switch one tabstop to the left.

If the user passes this an fd which was obtained via means other than kevent_ctl_init(), the kernel will explode. Do

	if (file->f_fop != &kevent_user_fops)
		return -EINVAL;

I have not created additional DMA memory allocation methods, like Ulrich described in his article, so I handle it inside NAIO which has some overhead (I posted get_user_pages() sclability graph some time ago).

From: Evgeniy Polyakov <[email protected]>
Date: Wed, 26 Jul 2006 10:28:17 +0400

Quoted 4 lines

I have not created additional DMA memory allocation methods, like Ulrich described in his article, so I handle it inside NAIO which has some overhead (I posted get_user_pages() sclability graph some time ago).

I've been thinking about this aspect, and I think it's very interesting. Let's be clear what the ramifications of this are first.

Using the terminology of Network Algorithmics, this is an instance of Principle 2, "Shift computation in time".

Instead of using get_user_pages() at AIO setup, we instead map the thing to userspace later when the user wants it. Pinning pages is a pain because both user and kernel refer to the buffer at the same time. We get more flexibility when the user has to map the thing explicitly.

I want us to think about how a user might want to use this. What I anticipate is that users will want to organize a pool of AIO buffers for themselves using this DMA interface. So the events they are truly interested in are of a finer granularity than you might expect. They want to know when pieces of a buffer are available for reuse.

I.e. map skb's data to userspace? Not a good idea especially with it's tricky lifetime and unability for userspace to inform kernel when it finished and skb can be freed (without additional syscall).

I did it with af_tlb zero-copy sniffer (but I substitute mapped pages with physical skb->data pages), and it was not very good.

Well, I think preallocate some buffers and use that in AIO setup is a plus, since in that case user does not care about when it is possible to reuse the same buffer - when appropriate kevent is completed, that means that provided buffer is no longer in use by kernel, and user can reuse it.

From: Evgeniy Polyakov <[email protected]>
Date: Thu, 27 Jul 2006 11:49:02 +0400

Quoted 3 lines

I.e. map skb's data to userspace? Not a good idea especially with it's tricky lifetime and unability for userspace to inform kernel when it finished and skb can be freed (without additional syscall).

Hmmm...

If it is paged based, I do not see the problem. Events and calls to AIO I/O routines make transfer of buffer ownership. The fact that while kernel (and thus networking stack) "owns" the buffer for an AIO call, the user can have a valid mapping to it is a unimportant detail.

Quoted 9 lines · David Miller

From: Evgeniy Polyakov <[email protected]>
Date: Thu, 27 Jul 2006 11:49:02 +0400

Quoted 3 lines

I.e. map skb's data to userspace? Not a good idea especially with it's tricky lifetime and unability for userspace to inform kernel when it finished and skb can be freed (without additional syscall).

Hmmm...

Quoted 4 lines

If it is paged based, I do not see the problem. Events and calls to AIO I/O routines make transfer of buffer ownership. The fact that while kernel (and thus networking stack) "owns" the buffer for an AIO call, the user can have a valid mapping to it is a unimportant detail.

Ownership may be clear, but "when can I reuse" is tricky. The same issue comes up for vmsplice -> splice to socket.

Ownership transition from user -> kernel that is, what I'm trying to say that returning ownership to the user again is the tricky part.

From: Jens Axboe <[email protected]>
Date: Thu, 27 Jul 2006 10:11:15 +0200

Quoted 2 lines

Ownership transition from user -> kernel that is, what I'm trying to say that returning ownership to the user again is the tricky part.

Yes, it is important that for TCP, for example, we don't give the user the event until the data is acknowledged and the skb's referencing that data are fully freed.

This is further complicated by the fact that packetization boundaries are going to be different from AIO buffer boundaries.

I think this is what you are alluding to.

Precisely. And this is the bit that is currently still broken for splice-to-socket, since it gives that ack right after ->sendpage() has been called. But that's a known deficiency right now, I think Alexey is currently looking at that (as well as receive side support).

From: Jens Axboe <[email protected]>
Date: Thu, 27 Jul 2006 10:29:24 +0200

Quoted 4 lines

Precisely. And this is the bit that is currently still broken for splice-to-socket, since it gives that ack right after ->sendpage() has been called. But that's a known deficiency right now, I think Alexey is currently looking at that (as well as receive side support).

That's right, I was discussing this with him just a few days ago.

It's good to hear that he's looking at those patches you were working on several months ago.

From: Evgeniy Polyakov <[email protected]>
Date: Thu, 27 Jul 2006 11:49:02 +0400

Quoted 3 lines

I.e. map skb's data to userspace? Not a good idea especially with it's tricky lifetime and unability for userspace to inform kernel when it finished and skb can be freed (without additional syscall).

Hmmm...

If it is paged based, I do not see the problem. Events and calls to AIO I/O routines make transfer of buffer ownership. The fact that while kernel (and thus networking stack) "owns" the buffer for an AIO call, the user can have a valid mapping to it is a unimportant detail.

If the user will scramble a piece of data that is in flight to or from the network card, it is his problem.

If we are using a primitive network card that does not support scatter-gather I/O and thus not page based SKBs, we will make copies. But this is transparent to the user.

The idea is that DMA mappings have page granularity.

At least on transmit it should work well. Receive side is more difficult and initial implementation will need to copy.

Quoted 2 lines

I did it with af_tlb zero-copy sniffer (but I substitute mapped pages with physical skb->data pages), and it was not very good.

Trying to be too clever with skb->data has always been catastrophic. :)

Quoted 5 lines

Well, I think preallocate some buffers and use that in AIO setup is a plus, since in that case user does not care about when it is possible to reuse the same buffer - when appropriate kevent is completed, that means that provided buffer is no longer in use by kernel, and user can reuse it.

We now enter the most interesting topic of AIO buffer pool management and where it belongs. :-) We are assuming up to this point that the user manages this stuff with explicit DMA calls for allocation, then passes the key based references to those buffers as arguments to the AIO I/O calls.

But I want to suggest another possibility. What if the kernel managed the AIO buffer pool for a task? It could grow this dynamically based upon need. The only implementation road block is how large to we allow this to grow, but I think normal VM mechanisms can take care of it.

On transmit this is not straightforward, but for receive it has really nice possibilities. :)

Btw, according to DMA allocations - there are some problems here too. Some pieces of the world can not dma behind 16mb, and someone can do it over 4gb.

Those physical pages can be managed within kernel and userspace can map them. But there is another possibility - replace slab allocation for network devices with allocation from premapped pool.
That naturally allows to manage that pool for AIO needs and have zero-copy sending and receiving support. That is what I talked in netchannel topic when question about allocation/freeing cost in atomic context arised. I work on that solution, which can be used both for netchannels (and full userspace processing) and usual networking code.

From: Evgeniy Polyakov <[email protected]>
Date: Thu, 27 Jul 2006 12:58:13 +0400

Quoted 3 lines

Btw, according to DMA allocations - there are some problems here too. Some pieces of the world can not dma behind 16mb, and someone can do it over 4gb.

I think people take this "DMA" in Ulrich's interface names too literally. It is logically something different, although it could be used directly for this purpose.

View it rather as memory you have by some key based ID, but need to explicitly map to access directly.

Quoted 8 lines

Those physical pages can be managed within kernel and userspace can map them. But there is another possibility - replace slab allocation for network devices with allocation from premapped pool.
That naturally allows to manage that pool for AIO needs and have zero-copy sending and receiving support. That is what I talked in netchannel topic when question about allocation/freeing cost in atomic context arised. I work on that solution, which can be used both for netchannels (and full userspace processing) and usual networking code.

Interesting idea, and yes I have been watching you stress test your AVL tree code :))