Kevent subsystem.
Subsystems (8)
PATCH 1/1 Kevent subsystem.
Hello.
Kevent subsystem incorporates several AIO/kqueue design notes and ideas. Kevent can be used both for edge and level notifications. It supports socket notifications, network AIO (aio_send(), aio_recv() and aio_sendfile()), inode notifications (create/remove), generic poll()/select() notifications and timer notifications.
It was tested against FreeBSD kqueue and Linux epoll and showed noticeble performance win.
Network asynchronous IO operations were tested against Linux synchronous socket code and showed noticeble performance win.
Patch against linux-2.6.17-git tree attached (gzipped). I would like to hear some comments about the overall design, implementation and plans about it's usefullness for generic kernel.
Design notes, patches, userspace application and perfomance tests can be found at project's homepages.
1. Kevent subsystem.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=kevent
2. Network AIO.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=naio
3. LWN article about kevent.
http://lwn.net/Articles/172844/
Thank you.
Signature · 1 line
Evgeniy Polyakov
Quoted 3 lines · Evgeniy Polyakov
Patch against linux-2.6.17-git tree attached (gzipped). I would like to hear some comments about the overall design, implementation and plans about it's usefullness for generic kernel.
Please send patches as in-line ascii text, along with documentation.
If they're too big, split them up logically into smaller pieces.
- James
Signature · 2 lines
James Morris <[email protected]>
Quoted 10 lines · James Morris
Quoted 3 lines · Evgeniy Polyakov
Patch against linux-2.6.17-git tree attached (gzipped). I would like to hear some comments about the overall design, implementation and plans about it's usefullness for generic kernel.
Please send patches as in-line ascii text, along with documentation.
If they're too big, split them up logically into smaller pieces.
Hmm... It is too big (>100kb) to send as plain text.
I will try to split it to several pieces and resend in a couple of moments, although they are quite strongly logically connected.
Quoted 4 lines
- James
Signature · 2 lines
James Morris <[email protected]>
Signature · 1 line
Evgeniy Polyakov
Evgeniy,
Quoted 5 lines · Evgeniy Polyakov
Kevent subsystem incorporates several AIO/kqueue design notes and ideas. Kevent can be used both for edge and level notifications. It supports socket notifications, network AIO (
aio_send(),aio_recv()andaio_sendfile()), inode notifications (create/remove), generic poll()/select() notifications and timer notifications.
Great job!
Smooth integration with userland asynch POSIX frameworks (e.g. ACE POSIX_Proactor) may require syscalls (or their emulation) with POSIX interface:
* POSIX_API
* aio_read
* aio_write
* aio_suspend
* aio_error
* aio_return
* aio_cancel
where aio_suspend is very important.
Signature · 5 lines
Sincerely, ------------------------------------------------------------------ Robert Iakobashvili, coroberti at gmail dot com Navigare necesse est, vivere non est necesse. ------------------------------------------------------------------
Quoted 23 lines · Robert Iakobashvili
Evgeniy,
Quoted 5 lines · Evgeniy Polyakov
Kevent subsystem incorporates several AIO/kqueue design notes and ideas. Kevent can be used both for edge and level notifications. It supports socket notifications, network AIO (
aio_send(),aio_recv()andaio_sendfile()), inode notifications (create/remove), generic poll()/select() notifications and timer notifications.Great job!
Smooth integration with userland asynch POSIX frameworks (e.g. ACEPOSIX_Proactor) may require syscalls (or their emulation) with POSIX interface:*
POSIX_API
*aio_read
*aio_write
*aio_suspend
*aio_error
*aio_return
*aio_cancel
Quoted 1 line
where
aio_suspendis very important.
I've designed network AIO differently as how posix declared it's functions. POSIX AIO is like rocket science - while one tries to implement them all there is no time to do the real work. Although all of them including suspend can be emulated on top of existing calls to kevent.
Quoted 6 lines
Signature · 5 lines
Sincerely, ------------------------------------------------------------------ Robert Iakobashvili, coroberti at gmail dot com Navigare necesse est, vivere non est necesse. ------------------------------------------------------------------
Signature · 1 line
Evgeniy Polyakov
In article <[email protected]> (at Thu, 22 Jun 2006 21:14:37 +0400), Evgeniy Polyakov <[email protected]> says:
Quoted 6 lines
Patch against linux-2.6.17-git tree attached (gzipped). I would like to hear some comments about the overall design, implementation and plans about it's usefullness for generic kernel.
Design notes, patches, userspace application and perfomance tests can be found at project's homepages.
| diff --git a/arch/i386/kernel/syscall_table.S b/arch/i386/kernel/syscall_table.S | index af56987..93e23ff 100644 | --- a/arch/i386/kernel/syscall_table.S | +++ b/arch/i386/kernel/syscall_table.S | @@ -316,3 +316,7 @@ ENTRY(sys_call_table) | .long sys_sync_file_range | .long sys_tee /* 315 */ | .long sys_vmsplice | + .long sys_aio_recv | + .long sys_aio_send | + .long sys_aio_sendfile | + .long sys_kevent_ctl
:
We do not waste syscall entries; we can probably use socketcall for sys_aio_recv, sys_aio_send (and maybe, sys_aio_sendfile).
| diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c | index 25ecc6e..05d7086 100644 | --- a/net/ipv4/tcp_ipv4.c | +++ b/net/ipv4/tcp_ipv4.c
:
Please do for tcp_ipv6.c as well. Thank you.
Signature · 2 lines
YOSHIFUJI Hideaki @ USAGI Project <[email protected]> GPG-FP : 9022 65EB 1ECF 3AD1 0BDF 80D8 4807 F894 E062 0EEA
David Miller
maintainerFrom: YOSHIFUJI Hideaki <[email protected]>
Date: Fri, 23 Jun 2006 15:12:44 +0900 (JST)
Quoted 2 lines
We do not waste syscall entries; we can probably use socketcall for
sys_aio_recv,sys_aio_send(and maybe,sys_aio_sendfile).
socketcall is deprecated and some architectures do not even have a slot for it in their system call tables, one example is x86_64
In article <[email protected]> (at Thu, 22 Jun 2006 23:14:07 -0700 (PDT)), David Miller <[email protected]> says:
Quoted 9 lines
From: YOSHIFUJI Hideaki <[email protected]>
Date: Fri, 23 Jun 2006 15:12:44 +0900 (JST)Quoted 2 lines
We do not waste syscall entries; we can probably use socketcall for
sys_aio_recv,sys_aio_send(and maybe,sys_aio_sendfile).socketcall is deprecated and some architectures do not even have a slot for it in their system call tables, one example is
x86_64
Ah, okay. Thanks.
--yoshfuji
PATCH 1/4 kevent: core files.
In reply to Evgeniy PolyakovThis patch includes core kevent files:
- userspace controlling
- kernelspace interfaces
- initialisation
- notification state machines
It might also inlclude parts from other subsystem (like network related syscalls so it is possible that it will not compile without other patches applied).
arch/i386/kernel/syscall_table.S +4 -0
@@ -316,3 +316,7 @@ ENTRY(sys_call_table) .long sys_sync_file_range .long sys_tee /* 315 */ .long sys_vmsplice + .long sys_aio_recv + .long sys_aio_send + .long sys_aio_sendfile + .long sys_kevent_ctl
arch/x86_64/ia32/ia32entry.S +4 -0
@@ -696,4 +696,8 @@ #endif .quad sys_sync_file_range .quad sys_tee .quad compat_sys_vmsplice + .quad sys_aio_recv + .quad sys_aio_send + .quad sys_aio_sendfile + .quad sys_kevent_ctl ia32_syscall_end:
include/asm-i386/unistd.h +5 -1
@@ -322,10 +322,14 @@ #define __NR_splice 313#define __NR_sync_file_range 314#define __NR_tee 315#define __NR_vmsplice 316 +#define __NR_aio_recv 317 +#define __NR_aio_send 318 +#define __NR_aio_sendfile 319 +#define __NR_kevent_ctl 320#ifdef __KERNEL__ -#define NR_syscalls 317 +#define NR_syscalls 321/* * user-visible error numbers are in the range -1 - -128: see
include/asm-x86_64/socket.h +2 -0
@@ -49,4 +49,6 @@ #define SO_ACCEPTCONN 30#define SO_PEERSEC 31 +#define SO_ASYNC_SOCK 34 +#endif /* _ASM_SOCKET_H */
include/asm-x86_64/unistd.h +9 -2
@@ -617,11 +617,18 @@ #define __NR_sync_file_range 277 __SYSCALL(__NR_sync_file_range, sys_sync_file_range)#define __NR_vmsplice 278 __SYSCALL(__NR_vmsplice, sys_vmsplice) +#define __NR_aio_recv 279 +__SYSCALL(__NR_aio_recv, sys_aio_recv) +#define __NR_aio_send 280 +__SYSCALL(__NR_aio_send, sys_aio_send) +#define __NR_aio_sendfile 281 +__SYSCALL(__NR_aio_sendfile, sys_aio_sendfile) +#define __NR_kevent_ctl 282 +__SYSCALL(__NR_kevent_ctl, sys_kevent_ctl)#ifdef __KERNEL__ -#define __NR_syscall_max __NR_vmsplice - +#define __NR_syscall_max __NR_kevent_ctl#ifndef __NO_STUBS/* user-visible error numbers are in the range -1 - -4095 */
include/linux/kevent.h new file +263 -0
263 lines not on this page · raw message
include/linux/kevent_storage.h new file +12 -0
@@ -0,0 +1,12 @@ +#ifndef __KEVENT_STORAGE_H +#define __KEVENT_STORAGE_H + +struct kevent_storage +{ +void *origin;/* Originator's pointer, e.g. struct sock or struct file. Can be NULL. */ +struct list_head list;/* List of queued kevents. */ +unsigned int qlen;/* Number of queued kevents. */ +spinlock_t lock;/* Protects users queue. */ +}; + +#endif /* __KEVENT_STORAGE_H */
include/linux/syscalls.h +4 -0
@@ -587,4 +587,8 @@ asmlinkage long sys_get_robust_list(intasmlinkage long sys_set_robust_list (struct robust_list_head __user *head,size_t len); +asmlinkage long sys_aio_recv (int ctl_fd,int s,void __user *buf,size_t size,unsigned flags); +asmlinkage long sys_aio_send (int ctl_fd,int s,void __user *buf,size_t size,unsigned flags); +asmlinkage long sys_aio_sendfile (int ctl_fd,int fd,int s,size_t size,unsigned flags); +asmlinkage long sys_kevent_ctl (int ctl_fd,void __user *buf);#endif
init/Kconfig +2 -0
@@ -185,6 +185,8 @@ config AUDITSYSCALL such as SELinux. To use audit's filesystem watch feature, please ensure that INOTIFY is configured. +source "kernel/kevent/Kconfig" + config IKCONFIG bool "Kernel .config support" ---help---
kernel/Makefile +1 -0
@@ -36,6 +36,7 @@ obj-$(CONFIG_DETECT_SOFTLOCKUP) += softl obj-$(CONFIG_GENERIC_HARDIRQS) += irq/ obj-$(CONFIG_SECCOMP) += seccomp.o obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o +obj-$(CONFIG_KEVENT) += kevent/ obj-$(CONFIG_RELAY) += relay.o ifneq ($(CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER),y)
kernel/kevent/Kconfig new file +57 -0
57 lines not on this page · raw message
kernel/kevent/Makefile new file +7 -0
@@ -0,0 +1,7 @@ +obj-y := kevent.o kevent_user.o kevent_init.o +obj-$(CONFIG_KEVENT_SOCKET) += kevent_socket.o +obj-$(CONFIG_KEVENT_INODE) += kevent_inode.o +obj-$(CONFIG_KEVENT_TIMER) += kevent_timer.o +obj-$(CONFIG_KEVENT_POLL) += kevent_poll.o +obj-$(CONFIG_KEVENT_NAIO) += kevent_naio.o +obj-$(CONFIG_KEVENT_AIO) += kevent_aio.o
kernel/kevent/kevent.c new file +260 -0
260 lines not on this page · raw message
kernel/kevent/kevent_init.c new file +85 -0
85 lines not on this page · raw message
kernel/kevent/kevent_user.c new file +728 -0
728 lines not on this page · raw message
kernel/sys_ni.c +5 -0
@@ -121,6 +121,11 @@ cond_syscall(ppc_rtas);cond_syscall (sys_spu_run);cond_syscall (sys_spu_create); +cond_syscall (sys_aio_recv); +cond_syscall (sys_aio_send); +cond_syscall (sys_aio_sendfile); +cond_syscall (sys_kevent_ctl); +/* mmu depending weak syscall entries */ cond_syscall (sys_mprotect);cond_syscall (sys_msync);
Signature · 1 line
Evgeniy Polyakov
Re: [1/4] kevent: core files.
Quoted 5 lines · Evgeniy Polyakov
This patch includes core kevent files:
- userspace controlling
- kernelspace interfaces
- initialisation
- notification state machines
We don't need yet another event mechanism in the kernel, so I don't see why the new syscalls should be added when they don't interoperate with existing solutions. If your results are enough to sway akpm that it is worth taking the patches, then it would make sense to merge the code with the already in-tree APIs.
-ben
Re: [1/4] kevent: core files.
Quoted 7 lines · Benjamin LaHaise
Quoted 5 lines · Evgeniy Polyakov
This patch includes core kevent files:
- userspace controlling
- kernelspace interfaces
- initialisation
- notification state machines
Quoted 5 lines
We don't need yet another event mechanism in the kernel, so I don't see why the new syscalls should be added when they don't interoperate with existing solutions. If your results are enough to sway akpm that it is worth taking the patches, then it would make sense to merge the code with the already in-tree APIs.
What API are you talking about?
There is only epoll(), which is 40% slower than kevent, and AIO, which works not as state machine, but as repeated call for the same work. There is also inotify, which allocates new message each time event occurs, which is not a good solution for every situation.
Linux just does not have unified event processing mechanism, which was pointed to many times in AIO mail list and when epoll() was only introduced. I would even say, that Linux does not have such mechanism at all, since every potential user implements it's own, which can not be used with others.
Kevent fixes that. Although implementation itself can be suboptimal for some cases or even unacceptible at all, but it is really needed functionality.
Every existing notification can be built on top of kevent. One can find how easy it was to implement generic poll/select notifications (what epoll() does) or socket notifications (which are similar to epoll(), but are called from inside socket state machine, thus improving processing performance).
Quoted 1 line
-ben
Signature · 1 line
Evgeniy Polyakov
Re: [1/4] kevent: core files.
Quoted 5 lines · Evgeniy Polyakov
What API are you talking about?
There is onlyepoll(), which is 40% slower than kevent, and AIO, which works not as state machine, but as repeated call for the same work. There is also inotify, which allocates new message each time event occurs, which is not a good solution for every situation.
AIO can be implemented as a state machine. Nothing in the API stops you from doing that, and in fact there was code which was implemented as a state machine used on 2.4 kernels.
Quoted 5 lines
Linux just does not have unified event processing mechanism, which was pointed to many times in AIO mail list and when
epoll()was only introduced. I would even say, that Linux does not have such mechanism at all, since every potential user implements it's own, which can not be used with others.
The epoll event API doesn't have space in the event fields for result codes as needed for AIO. The AIO API does -- how is it lacking in this regard?
Quoted 3 lines
Kevent fixes that. Although implementation itself can be suboptimal for some cases or even unacceptible at all, but it is really needed functionality.
At the expense of adding another API? How is this a good thing? Why not spit out events in the existing format?
Quoted 5 lines
Every existing notification can be built on top of kevent. One can find how easy it was to implement generic poll/select notifications (what
epoll()does) or socket notifications (which are similar toepoll(), but are called from inside socket state machine, thus improving processing performance).
So far your code is adding a lot without unifying anything.
-ben
Signature · 2 lines
"Time is of no importance, Mr. President, only life is important." Don't Email: <[email protected]>.
Re: [1/4] kevent: core files.
Quoted 10 lines · Benjamin LaHaise
Quoted 5 lines · Evgeniy Polyakov
What API are you talking about?
There is onlyepoll(), which is 40% slower than kevent, and AIO, which works not as state machine, but as repeated call for the same work. There is also inotify, which allocates new message each time event occurs, which is not a good solution for every situation.AIO can be implemented as a state machine. Nothing in the API stops you from doing that, and in fact there was code which was implemented as a state machine used on 2.4 kernels.
But now it is implemented as repeated call for the same work, which does not look like it can be used for any other types of work. And repeated work introduce latencies.
As far as I recall, it is you who wanted to remove thread based approach from AIO subsystem.
Quoted 8 lines
Quoted 5 lines
Linux just does not have unified event processing mechanism, which was pointed to many times in AIO mail list and when
epoll()was only introduced. I would even say, that Linux does not have such mechanism at all, since every potential user implements it's own, which can not be used with others.The epoll event API doesn't have space in the event fields for result codes as needed for AIO. The AIO API does -- how is it lacking in this regard?
AIO completion approach was designed to be used with process context VFS update. read/write approach can not cover other types of notifications, like inode updates or timers.
Quoted 6 lines
Quoted 3 lines
Kevent fixes that. Although implementation itself can be suboptimal for some cases or even unacceptible at all, but it is really needed functionality.
At the expense of adding another API? How is this a good thing? Why not spit out events in the existing format?
Format of the structure transferred between the objects does not matter at all. We can create a wrapper on kevent structures or kevent can transform data from AIO objects.
The main design goal of kevent is to provide easy connected hooks into any state machine, which might be used by kernelspace to notify about any kind of events without any knowledge of it's background nature. Kevent can be used for example as notification blocks for address changes or it can replace netlink completely (it can even emulate event multicasting).
Kevent is queue of events, which can be transferred from any object to any destination.
Quoted 7 lines
Quoted 5 lines
Every existing notification can be built on top of kevent. One can find how easy it was to implement generic poll/select notifications (what
epoll()does) or socket notifications (which are similar toepoll(), but are called from inside socket state machine, thus improving processing performance).So far your code is adding a lot without unifying anything.
Not at all!
Kevent is a mechanism, which allows to impleement AIO, network AIO, poll and select, timer control, adaptive readhead (as example of AIO VFS update). All the code I present shows how to use kevent, it is not part of the kevent. One can find Makefile in kevent dir to check what is the core of the subsystem, which allows to be used as a transport for events.
AIO, NAIO, poll/select, socket and timer notifications are just users. One can add it's own usage as easy as to call kevent_storage initialization function and event generation function. All other pieces are hidded in the implementation.
Quoted 4 lines
-ben
Signature · 2 lines
"Time is of no importance, Mr. President, only life is important." Don't Email: <[email protected]>.
Signature · 1 line
Evgeniy Polyakov
Re: [1/4] kevent: core files.
Quoted 2 lines · Evgeniy Polyakov
But now it is implemented as repeated call for the same work, which does not look like it can be used for any other types of work.
Given an iocb, you do not have to return -EIOCBRETRY, instead return -EIOCBQUEUED and then from whatever context do an aio_complete() with the result for that iocb.
Quoted 3 lines
And repeated work introduce latencies.
As far as I recall, it is you who wanted to remove thread based approach from AIO subsystem.
I have essentially given up on trying to get the filesystem AIO patches in given that the concerns against them are "woe complexity" with no real recourse for inclusion being open. If David is open to changes in the networking area, I'd love to see it built on top of your code.
Quoted 3 lines
AIO completion approach was designed to be used with process context VFS update. read/write approach can not cover other types of notifications, like inode updates or timers.
The completion event is 100% generic and does not need to come from process context. Calling aio_complete() from irq context is entirely valid.
Quoted 11 lines
Format of the structure transferred between the objects does not matter at all. We can create a wrapper on kevent structures or kevent can transform data from AIO objects.
The main design goal of kevent is to provide easy connected hooks into any state machine, which might be used by kernelspace to notify about any kind of events without any knowledge of it's background nature. Kevent can be used for example as notification blocks for address changes or it can replace netlink completely (it can even emulate event multicasting).
Quoted 2 lines
Kevent is queue of events, which can be transferred from any object to any destination.
And io_getevents() reads a queue of events, so I'm not sure why you need a new syscall.
Quoted 8 lines
Not at all!
Kevent is a mechanism, which allows to impleement AIO, network AIO, poll and select, timer control, adaptive readhead (as example of AIO VFS update). All the code I present shows how to use kevent, it is not part of the kevent. One can findMakefilein kevent dir to check what is the core of the subsystem, which allows to be used as a transport for events.
Quoted 4 lines
AIO, NAIO, poll/select, socket and timer notifications are just users. One can add it's own usage as easy as to call
kevent_storageinitialization function and event generation function. All other pieces are hidded in the implementation.
I'll look at adapting your code to use the existing syscalls. Maybe code will be better at expressing my concerns.
-ben
Signature · 2 lines
"Time is of no importance, Mr. President, only life is important." Don't Email: <[email protected]>.
Re: [1/4] kevent: core files.
Quoted 6 lines · Benjamin LaHaise
Quoted 3 lines
AIO completion approach was designed to be used with process context VFS update. read/write approach can not cover other types of notifications, like inode updates or timers.
The completion event is 100% generic and does not need to come from process context. Calling
aio_complete()from irq context is entirely valid.
put_ioctx() can sleep.
And the whole approach is different: AIO just wakes up requesting thread, so user must provide a lot to be able to work with AIO. It perfectly fits VFS design, but it is not acceptible for generic event notifications.
Quoted 11 lines
Quoted 11 lines
Format of the structure transferred between the objects does not matter at all. We can create a wrapper on kevent structures or kevent can transform data from AIO objects.
The main design goal of kevent is to provide easy connected hooks into any state machine, which might be used by kernelspace to notify about any kind of events without any knowledge of it's background nature. Kevent can be used for example as notification blocks for address changes or it can replace netlink completely (it can even emulate event multicasting).
Quoted 5 lines
Quoted 2 lines
Kevent is queue of events, which can be transferred from any object to any destination.
And
io_getevents()reads a queue of events, so I'm not sure why you need a new syscall.
It is not syscall, but overall design should be analyzed. It is possible to use existing ssycalls, kevent design does not care about how it's data structures are delivered to the internal "processor".
Quoted 8 lines
Quoted 8 lines
Not at all!
Kevent is a mechanism, which allows to impleement AIO, network AIO, poll and select, timer control, adaptive readhead (as example of AIO VFS update). All the code I present shows how to use kevent, it is not part of the kevent. One can findMakefilein kevent dir to check what is the core of the subsystem, which allows to be used as a transport for events.
Quoted 7 lines
Quoted 4 lines
AIO, NAIO, poll/select, socket and timer notifications are just users. One can add it's own usage as easy as to call
kevent_storageinitialization function and event generation function. All other pieces are hidded in the implementation.I'll look at adapting your code to use the existing syscalls. Maybe code will be better at expressing my concerns.
That would be great.
Quoted 1 line
-ben
Signature · 1 line
Evgeniy Polyakov
Re: [1/4] kevent: core files.
Quoted 9 lines · Evgeniy Polyakov
Quoted 6 lines · Benjamin LaHaise
Quoted 3 lines
AIO completion approach was designed to be used with process context VFS update. read/write approach can not cover other types of notifications, like inode updates or timers.
The completion event is 100% generic and does not need to come from process context. Calling
aio_complete()from irq context is entirely valid.
put_ioctx()can sleep.
Err, no, that should definately not be the case. If it can, someone has completely broken aio.
Quoted 4 lines
It is not syscall, but overall design should be analyzed. It is possible to use existing ssycalls, kevent design does not care about how it's data structures are delivered to the internal "processor".
Okay, that's good to hear. =-)
-ben
Signature · 2 lines
"Time is of no importance, Mr. President, only life is important." Don't Email: <[email protected]>.
Re: [1/4] kevent: core files.
Quoted 6 lines · Benjamin LaHaise
Quoted 5 lines · Evgeniy Polyakov
Quoted 4 lines · Benjamin LaHaise
Quoted 3 lines
AIO completion approach was designed to be used with process context VFS update. read/write approach can not cover other types of notifications, like inode updates or timers.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
The completion event is 100% generic and does not need to come from process context. Calling
aio_complete()from irq context is entirely valid.
put_ioctx()can sleep.Err, no, that should definately not be the case. If it can, someone has completely broken aio.
When reference counter hits zero it flushes aio workqueue, which can sleep.put_ioctx() -> __put_ioctx() -> cancel_delayed_work()/flush_workqueue().
It is there at least from 2.6.15 days (it is the oldest tree I can access using my extremely slow GPRS link).
Hang the looter!
Quoted 1 line
-ben
Signature · 1 line
Evgeniy Polyakov
David Miller
maintainerRe: [1/4] kevent: core files.
In reply to Evgeniy PolyakovFrom: Evgeniy Polyakov <[email protected]>
Date: Fri, 23 Jun 2006 23:24:29 +0400
Quoted 9 lines
Linux just does not have unified event processing mechanism, which was pointed to many times in AIO mail list and when
epoll()was only introduced. I would even say, that Linux does not have such mechanism at all, since every potential user implements it's own, which can not be used with others.Kevent fixes that. Although implementation itself can be suboptimal for some cases or even unacceptible at all, but it is really needed functionality.
I completely agree with Evgeniy here.
There is nothing in the kernel today that provides integrated event handling. Nothing. So when someone says to use the "existing" stuff, they need to have their head examined.
The existing AIO stuff stinks as a set of interfaces. It was designed by a standards committee, not by people truly interested in a good performing event processing design. It is especially poorly suited for networking, and any networking developer understands this.
It is pretty much a foregone conclusion that we will need new APIs to get good networking performance. Every existing interface has one limitation or another.
So we should be happy people like Evgeniy try to work on this stuff, instead of discouraging them.
Re: [1/4] kevent: core files.
Quoted 5 lines · David Miller
I completely agree with Evgeniy here.
There is nothing in the kernel today that provides integrated event handling. Nothing. So when someone says to use the "existing" stuff, they need to have their head examined.
The existing AIO events are *events*, with the syscalls providing the reading of events.
Quoted 4 lines
The existing AIO stuff stinks as a set of interfaces. It was designed by a standards committee, not by people truly interested in a good performing event processing design. It is especially poorly suited for networking, and any networking developer understands this.
I disagree. Stuffing an event that a read or write is complete/ready is a good way of handling things, even more so with hardware that will perform the memory copies to/from user buffers.
Quoted 3 lines
It is pretty much a foregone conclusion that we will need new APIs to get good networking performance. Every existing interface has one limitation or another.
Eh? Nobody has posted any numbers comparing the approaches yet, so this is pure handwaving, unless you have real concrete results?
Quoted 2 lines
So we should be happy people like Evgeniy try to work on this stuff, instead of discouraging them.
I would like to encourage him, but at the same time I don't want to see creating APIs that essentially duplicate existing work and needlessly break compatibility. I completely agree that the in-kernel APIs are not as encompassing as they should be, and within the kernel Evgeniy's work may well be the way to go. What I do not agree is that we need new syscalls at this point. I'm perfectly willing to accept proof that change is needed if we do a proper comparision between any new syscall API and the use of the existing syscall API, but the pain of introducing a new API is sufficiently large that I think it is worth looking at the numbers.
-ben
Signature · 2 lines
"Time is of no importance, Mr. President, only life is important." Don't Email: <[email protected]>.
Re: [1/4] kevent: core files.
Quoted 5 lines · Benjamin LaHaise
may well be the way to go. What I do not agree is that we need new syscalls at this point. I'm perfectly willing to accept proof that change is needed if we do a proper comparision between any new syscall API and the use of the existing syscall API, but the pain of introducing a new API is sufficiently large that I think it is worth looking at the numbers.
New syscall is just an interface. Originally kevent (and it still can) use char device and it's ioctl method.
It is perfectly possible to create wrappers for posix aio_* calls, although I do not see why it is needed.
No need to concentrate on end-users interface at this point - it can be changed at any time since design allows it, we should think about overall design and if it is ok, move forward in implementation.
Btw, new API adds only one syscall for userspace kevent processing (and three for send/recv/sendfile for network AIO).
According to numbers: kevent compared to epoll resulted in the folllowing numbers:
kevent: more than 2600 requests per second (trivial web server)
epoll: about 1600-1800 requests.
Number of errors for 3k bursts of connections with 30K connections total in 10seconds:
kevent: about 2k errors.
epoll: upto 15k errors.
More detailed results can be found on project's homepage at:
tservice.net.ru/~s0mbre/old/?section=projects&item=kevent
Quoted 1 line
-ben
Signature · 1 line
Evgeniy Polyakov
Re: [1/4] kevent: core files.
Hi,
Quoted 8 lines
According to numbers: kevent compared to epoll resulted in the folllowing numbers:
kevent: more than 2600 requests per second (trivial web server)
epoll: about 1600-1800 requests.
Number of errors for 3k bursts of connections with 30K connections total in 10seconds:
kevent: about 2k errors.
epoll: upto 15k errors.
If it beats the great epoll, it means a real business case for kevent.
All previous attempts in kernel as well as by glibc and other userland emulations to provide some real AIO infrastructure and API for server applications with performance benefits, were not too much successful. Heavy load networking servers are normally not using AIOs on linux due to low performance.
Quoted 1 line
From another side Windows have a very strong I/O completion ports API,
which are widely used for the most heavy load applications.
Kevent may take linux servers productivity forward in general as well as encourage moving aio-applications from windows to linux.
Signature · 5 lines
Sincerely, ------------------------------------------------------------------ Robert Iakobashvili, coroberti at gmail dot com Navigare necesse est, vivere non est necesse. ------------------------------------------------------------------
David Miller
maintainerRe: [1/4] kevent: core files.
In reply to Benjamin LaHaiseFrom: Benjamin LaHaise <[email protected]>
Date: Fri, 23 Jun 2006 16:31:14 -0400
Quoted 2 lines
Eh? Nobody has posted any numbers comparing the approaches yet, so this is pure handwaving, unless you have real concrete results?
Evgeniy posts numbers and performance graphs on his kevent work all the time.
Van Jacobson did in his LCA2006 net channel slides too, perhaps you missed that.
Re: [1/4] kevent: core files.
Quoted 8 lines · David Miller
From: Benjamin LaHaise <[email protected]>
Date: Fri, 23 Jun 2006 16:31:14 -0400Quoted 2 lines
Eh? Nobody has posted any numbers comparing the approaches yet, so this is pure handwaving, unless you have real concrete results?
Evgeniy posts numbers and performance graphs on his kevent work all the time.
But you're argueing that the performance of something that hasn't been tested is worse simply by nature of it not having been tested. That's a fallacy of omission, iiuc.
Quoted 2 lines
Van Jacobson did in his LCA2006 net channel slides too, perhaps you missed that.
I have yet to be convinced that the layering violation known as net channels is the right way to go, mostly because it breaks horribly in a few cases -- think what happens during periods of CPU overcommit, in which case doing too much in interrupt context will kill a system (which is why softirqs are needed). The effect of doing all processing in user context creates issues with delayed acks (due to context switching to other tasks in the system), which will cause excess retransmits. The hard problems associated with packet filtering and security are also still unresolved, which is okay for a paper, but a concern in real life.
There are also a number of performance flaws in the current stack that show up under profiling, some of which I posted fixes for, some of which have yet to be fixed. The pushf/popf pipeline stall was one of the bigger instances of CPU wastage that Van Jacobson noticed (it shows up as bottom halves using lots of CPU). Iirc, Ingo's real time patches may avoid that by way of reworking the irq disable/enable mechanism, which would mean the results need retesting. Using the cr8 register to enable/disable interrupts on x86-64 might also improve things, as that would eliminate the flags dependancy of cli/sti...
In short, there's a lot of work that still has to be done.
-ben
Signature · 2 lines
"Time is of no importance, Mr. President, only life is important." Don't Email: <[email protected]>.
David Miller
maintainerRe: [1/4] kevent: core files.
From: Benjamin LaHaise <[email protected]>
Date: Fri, 23 Jun 2006 17:53:14 -0400
Quoted 2 lines
The effect of doing all processing in user context creates issues with delayed acks (due to context switching to other tasks in the system),
The Linux TCP stack does this today.
Full TCP input protocol processing is done in the user process context.
What you are not understanding is that process scheduling helps TCP, it does not hinder it. If the system is loaded, we want the senders to pace themselves to the rate at which the kernel can schedule the abundance of receiver work it has. And this happens naturally when the TCP protocol input processing operates in process context.
Your fear of cpu overcommit in interrupt handlers is also heavily flawed. Net channels do a socket demux, and a queue entail plus ring a doorbell if necessary, nothing more.
PATCH 2/4 kevent: network notifications.
In reply to Evgeniy PolyakovThis patchset includes socket notifications and network asynchronous IO.
include/asm-i386/socket.h +2 -0
@@ -49,4 +49,6 @@ #define SO_ACCEPTCONN 30#define SO_PEERSEC 31 +#define SO_ASYNC_SOCK 34 +#endif /* _ASM_SOCKET_H */
include/linux/skbuff.h +2 -0
@@ -1269,6 +1269,8 @@ extern struct sk_buff *skb_recv_datagram int noblock, int *err);extern unsigned int datagram_poll (struct file *file,struct socket *sock,struct poll_table_struct *wait); +extern int skb_copy_datagram (const struct sk_buff *from, +int offset,void *dst,int size);extern int skb_copy_datagram_iovec (const struct sk_buff *from,int offset,struct iovec *to,int size);
include/net/sock.h +25 -15
@@ -47,6 +47,7 @@ #include <linux/module.h>#include <linux/netdevice.h> #include <linux/skbuff.h> /* struct sk_buff */ #include <linux/security.h> +#include <linux/kevent.h> #include <linux/filter.h>
@@ -386,6 +387,8 @@ enum sock_flags {SOCK_NO_LARGESEND ,/* whether to sent large segments or not */ SOCK_LOCALROUTE ,/* route locally only, %SO_DONTROUTE setting */ SOCK_QUEUE_SHRUNK ,/* write queue has been shrunk recently */ +SOCK_ASYNC , + SOCK_ASYNC_INUSE, };static inline void sock_copy_flags (struct sock *nsk,struct sock *osk)
@@ -445,6 +448,21 @@ static inline int sk_stream_memory_free(extern void sk_stream_rfree (struct sk_buff *skb); +struct socket_alloc { +struct socket socket; +struct inode vfs_inode; +}; + +static inline struct socket *SOCKET_I (struct inode *inode) +{ + return &container_of(inode, struct socket_alloc, vfs_inode)->socket; +} + +static inline struct inode *SOCK_INODE (struct socket *socket) +{ + return &container_of(socket, struct socket_alloc, socket)->vfs_inode; +} +static inline void sk_stream_set_owner_r (struct sk_buff *skb,struct sock *sk) { skb->sk = sk;
@@ -472,6 +490,7 @@ static inline void sk_add_backlog(struct sk->sk_backlog.tail = skb; } skb->next =NULL ; +kevent_socket_notify (sk,KEVENT_SOCKET_RECV ); }#define sk_wait_event (__sk, __timeo, __condition) \
@@ -543,6 +562,12 @@ struct proto {int (*backlog_rcv) (struct sock *sk,struct sk_buff *skb); + +int (*async_recv) (struct sock *sk, +void *dst,size_t size); +int (*async_send) (struct sock *sk, +struct page **pages,unsigned int poffset, +size_t size);/* Keeping track of sk's, looking them up, and port selection methods. */ void (*hash)(struct sock *sk);
@@ -674,21 +699,6 @@ static inline struct kiocb *siocb_to_kioreturn si->kiocb; } -struct socket_alloc { -struct socket socket; -struct inode vfs_inode; -}; - -static inline struct socket *SOCKET_I (struct inode *inode) -{ - return &container_of(inode, struct socket_alloc, vfs_inode)->socket; -} - -static inline struct inode *SOCK_INODE (struct socket *socket) -{ - return &container_of(socket, struct socket_alloc, socket)->vfs_inode; -} -extern void __sk_stream_mem_reclaim (struct sock *sk);extern int sk_stream_mem_schedule (struct sock *sk,int size,int kind);
include/net/tcp.h +3 -0
@@ -364,6 +364,8 @@ extern int compat_tcp_setsockopt(struc int level, int optname, char __user *optval, int optlen);extern void tcp_set_keepalive (struct sock *sk,int val); +extern int tcp_async_recv (struct sock *sk,void *dst,size_t size); +extern int tcp_async_send (struct sock *sk,struct page **pages,unsigned int poffset,size_t size); extern int tcp_recvmsg(struct kiocb *iocb, struct sock *sk, struct msghdr *msg, size_t len, int nonblock,
@@ -857,6 +859,7 @@ static inline int tcp_prequeue(struct so tp->ucopy.memory =0 ; }else if (skb_queue_len (&tp->ucopy.prequeue) ==1 ) {wake_up_interruptible (sk->sk_sleep); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND );if (!inet_csk_ack_scheduled (sk)) inet_csk_reset_xmit_timer(sk, ICSK_TIME_DACK, (3 * TCP_RTO_MIN) / 4,
kernel/kevent/kevent_naio.c new file +239 -0
239 lines not on this page · raw message
kernel/kevent/kevent_socket.c new file +125 -0
125 lines not on this page · raw message
net/core/datagram.c +55 -0
67 lines not on this page · raw message
net/core/sock.c +17 -1
@@ -564,6 +564,16 @@ #endifspin_unlock_bh (&sk->sk_lock.slock); ret = -ENONET ;break ; +#ifdef CONFIG_KEVENT_SOCKET +case SO_ASYNC_SOCK : +spin_lock_bh (&sk->sk_lock.slock); +if (valbool) +sock_set_flag (sk,SOCK_ASYNC ); +else +sock_reset_flag (sk,SOCK_ASYNC ); +spin_unlock_bh (&sk->sk_lock.slock); +break ; +#endif /* We implement the SO_SNDLOWAT etc to not be settable (1003.1g 5.3) */
@@ -1313,6 +1323,7 @@ static void sock_def_wakeup(struct sockif (sk->sk_sleep &&waitqueue_active (sk->sk_sleep))wake_up_interruptible_all (sk->sk_sleep);read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_error_report (struct sock *sk)
@@ -1322,6 +1333,7 @@ static void sock_def_error_report(structwake_up_interruptible (sk->sk_sleep);sk_wake_async (sk,0 ,POLL_ERR );read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_readable (struct sock *sk,int len)
@@ -1331,6 +1343,7 @@ static void sock_def_readable(struct socwake_up_interruptible (sk->sk_sleep);sk_wake_async (sk,1 ,POLL_IN );read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_write_space (struct sock *sk)
@@ -1350,6 +1363,7 @@ static void sock_def_write_space(struct }read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_SEND |KEVENT_SOCKET_RECV ); }static void sock_def_destruct (struct sock *sk)
@@ -1454,8 +1468,10 @@ void fastcall release_sock(struct sock *if (sk->sk_backlog.tail)__release_sock (sk); sk->sk_lock.owner =NULL ; -if (waitqueue_active (&(sk->sk_lock.wq))) +if (waitqueue_active (&(sk->sk_lock.wq))) {wake_up (&(sk->sk_lock.wq)); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); + }spin_unlock_bh (&(sk->sk_lock.slock)); }EXPORT_SYMBOL (release_sock);
net/core/stream.c +1 -0
@@ -36,6 +36,7 @@ void sk_stream_write_space(struct sock *wake_up_interruptible (sk->sk_sleep);if (sock->fasync_list && !(sk->sk_shutdown &SEND_SHUTDOWN ))sock_wake_async (sock,2 ,POLL_OUT ); +kevent_socket_notify (sk,KEVENT_SOCKET_SEND |KEVENT_SOCKET_RECV ); } }
net/ipv4/tcp.c +272 -0
290 lines not on this page · raw message
net/ipv4/tcp_input.c +3 -1
@@ -3113,6 +3113,7 @@ static void tcp_ofo_queue(struct sock *s__skb_unlink (skb, &tp->out_of_order_queue);__skb_queue_tail (&sk->sk_receive_queue, skb); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV ); tp->rcv_nxt =TCP_SKB_CB (skb)->end_seq;if (skb->h.th->fin)tcp_fin (skb, sk, skb->h.th);
@@ -3956,7 +3957,8 @@ int tcp_rcv_established(struct sock *sk,int copied_early =0 ;if (tp->copied_seq == tp->rcv_nxt && - len - tcp_header_len <= tp->ucopy.len) { + len - tcp_header_len <= tp->ucopy.len && + !sock_async (sk)) {#ifdef CONFIG_NET_DMA if (tcp_dma_try_early_copy (sk, skb, tcp_header_len)) { copied_early =1 ;
net/ipv4/tcp_ipv4.c +25 -15
@@ -62,6 +62,7 @@ #include <linux/cache.h>#include <linux/jhash.h> #include <linux/init.h> #include <linux/times.h> +#include <linux/kevent.h> #include <net/icmp.h> #include <net/inet_hashtables.h>
@@ -850,6 +851,7 @@ #endifreqsk_free (req); }else {inet_csk_reqsk_queue_hash_add (sk, req,TCP_TIMEOUT_INIT ); +kevent_socket_notify (sk,KEVENT_SOCKET_ACCEPT ); }return 0 ;
@@ -1089,24 +1091,30 @@ process: skb->dev =NULL ; -bh_lock_sock (sk); ret =0 ; -if (!sock_owned_by_user (sk)) { +if (sock_async (sk)) { +spin_lock_bh (&sk->sk_lock.slock); + ret =tcp_v4_do_rcv (sk, skb); +spin_unlock_bh (&sk->sk_lock.slock); + }else { +bh_lock_sock (sk); +if (!sock_owned_by_user (sk)) {#ifdef CONFIG_NET_DMA -struct tcp_sock *tp =tcp_sk (sk); -if (!tp->ucopy.dma_chan && tp->ucopy.pinned_list) - tp->ucopy.dma_chan =get_softnet_dma (); -if (tp->ucopy.dma_chan) - ret =tcp_v4_do_rcv (sk, skb); - else +struct tcp_sock *tp =tcp_sk (sk); +if (!tp->ucopy.dma_chan && tp->ucopy.pinned_list) + tp->ucopy.dma_chan =get_softnet_dma (); +if (tp->ucopy.dma_chan) + ret =tcp_v4_do_rcv (sk, skb); + else#endif - { -if (!tcp_prequeue (sk, skb)) - ret =tcp_v4_do_rcv (sk, skb); - } - }else -sk_add_backlog (sk, skb); -bh_unlock_sock (sk); + { +if (!tcp_prequeue (sk, skb)) + ret =tcp_v4_do_rcv (sk, skb); + } + }else +sk_add_backlog (sk, skb); +bh_unlock_sock (sk); + }sock_put (sk);
@@ -1830,6 +1838,8 @@ struct proto tcp_prot = { .getsockopt = tcp_getsockopt, .sendmsg = tcp_sendmsg, .recvmsg = tcp_recvmsg, + .async_recv = tcp_async_recv, + .async_send = tcp_async_send, .backlog_rcv = tcp_v4_do_rcv, .hash = tcp_v4_hash, .unhash = tcp_unhash,
Signature · 1 line
Evgeniy Polyakov
PATCH 3/4 kevent: fs/aio notifications.
In reply to Evgeniy PolyakovThis patch includes asynchronous propagation of file's data into VFS cache and aio_sendfile() implementation.
fs/bio.c +1 -1
@@ -119,7 +119,7 @@ void bio_free(struct bio *bio, struct bi/* * default destructor for a bio allocated with bio_alloc_bioset() */ -static void bio_fs_destructor (struct bio *bio) +void bio_fs_destructor (struct bio *bio) {bio_free (bio, fs_bio_set); }
fs/ext2/inode.c +1 -0
@@ -685,6 +685,7 @@ ext2_writepages(struct address_space *ma }struct address_space_operations ext2_aops = { + .get_block = ext2_get_block, .readpage = ext2_readpage, .readpages = ext2_readpages, .writepage = ext2_writepage,
fs/ext3/inode.c +1 -0
@@ -1700,6 +1700,7 @@ static int ext3_journalled_set_page_dirt }static struct address_space_operations ext3_ordered_aops = { + .get_block = ext3_get_block, .readpage = ext3_readpage, .readpages = ext3_readpages, .writepage = ext3_ordered_writepage,
fs/file_table.c +6 -0
@@ -113,6 +113,9 @@ struct file *get_empty_filp(void)if (security_file_alloc (f)) goto fail_sec; +#ifdef CONFIG_KEVENT_POLL +kevent_storage_init (f, &f->st); +#endif tsk = current;INIT_LIST_HEAD (&f->f_u.fu_list);atomic_set (&f->f_count,1 );
@@ -160,6 +163,9 @@ void fastcall __fput(struct file *file)might_sleep ();fsnotify_close (file); +#ifdef CONFIG_KEVENT_POLL +kevent_storage_fini (&file->st); +#endif /* * The function eventpoll_release() should be the first called * in the file cleanup chain.
fs/inode.c +7 -0
@@ -22,6 +22,7 @@ #include <linux/pagemap.h>#include <linux/cdev.h> #include <linux/bootmem.h> #include <linux/inotify.h> +#include <linux/kevent.h> #include <linux/mount.h> /*
@@ -166,12 +167,18 @@ #endif }memset (&inode->u,0 ,sizeof (inode->u)); inode->i_mapping = mapping; +#if definedCONFIG_KEVENT +kevent_storage_init (inode, &inode->st); +#endif }return inode; }void destroy_inode (struct inode *inode) { +#if definedCONFIG_KEVENT_INODE || definedCONFIG_KEVENT_SOCKET +kevent_storage_fini (&inode->st); +#endif BUG_ON (inode_has_buffers (inode));security_inode_free (inode);if (inode->i_sb->s_op->destroy_inode)
fs/reiserfs/inode.c +1 -0
@@ -2997,6 +2997,7 @@ int reiserfs_setattr(struct dentry *dent }struct address_space_operations reiserfs_address_space_operations = { + .get_block = reiserfs_get_block, .writepage = reiserfs_writepage, .readpage = reiserfs_readpage, .readpages = reiserfs_readpages,
include/linux/fs.h +12 -0
@@ -236,6 +236,9 @@ #include <linux/mutex.h>#include <asm/atomic.h> #include <asm/semaphore.h> #include <asm/byteorder.h> +#ifdef CONFIG_KEVENT +#include <linux/kevent_storage.h> +#endif struct hd_geometry ;struct iovec ;
@@ -348,6 +351,8 @@ struct address_space;struct writeback_control ;struct address_space_operations { +int (*get_block)(struct inode *inode,sector_t iblock, +struct buffer_head *bh_result,int create);int (*writepage)(struct page *page,struct writeback_control *wbc);int (*readpage)(struct file *,struct page *);void (*sync_page)(struct page *);
@@ -526,6 +531,10 @@ #ifdef CONFIG_INOTIFYstruct mutex inotify_mutex;/* protects the watches list */ #endif +#ifdef CONFIG_KEVENT_INODE +struct kevent_storage st; +#endif +unsigned long i_state;unsigned long dirtied_when;/* jiffies of first dirtying */
@@ -659,6 +668,9 @@ #ifdef CONFIG_EPOLLstruct list_head f_ep_links;spinlock_t f_ep_lock;#endif /* #ifdef CONFIG_EPOLL */ +#ifdef CONFIG_KEVENT_POLL +struct kevent_storage st; +#endif struct address_space *f_mapping; };extern spinlock_t files_lock;
include/linux/fsnotify.h +5 -0
@@ -15,6 +15,7 @@ #ifdef __KERNEL__#include <linux/dnotify.h> #include <linux/inotify.h> +#include <linux/kevent.h> #include <linux/audit.h> /*
@@ -79,6 +80,7 @@ static inline void fsnotify_nameremove(s isdir =IN_ISDIR ;dnotify_parent (dentry,DN_DELETE );inotify_dentry_parent_queue_event (dentry,IN_DELETE |isdir,0 , dentry->d_name.name); +kevent_inode_notify_parent (dentry,KEVENT_INODE_REMOVE ); }/*
@@ -88,6 +90,7 @@ static inline void fsnotify_inoderemove( {inotify_inode_queue_event (inode,IN_DELETE_SELF ,0 ,NULL ,NULL );inotify_inode_is_dead (inode); +kevent_inode_remove (inode); }/*
@@ -96,6 +99,7 @@ static inline void fsnotify_inoderemove(static inline void fsnotify_create (struct inode *inode,struct dentry *dentry) {inode_dir_notify (inode,DN_CREATE ); +kevent_inode_notify (inode,KEVENT_INODE_CREATE );inotify_inode_queue_event (inode,IN_CREATE ,0 , dentry->d_name.name, dentry->d_inode);audit_inode_child (dentry->d_name.name, dentry->d_inode, inode->i_ino);
@@ -107,6 +111,7 @@ static inline void fsnotify_create(strucstatic inline void fsnotify_mkdir (struct inode *inode,struct dentry *dentry) {inode_dir_notify (inode,DN_CREATE ); +kevent_inode_notify (inode,KEVENT_INODE_CREATE );inotify_inode_queue_event (inode,IN_CREATE |IN_ISDIR ,0 , dentry->d_name.name, dentry->d_inode);audit_inode_child (dentry->d_name.name, dentry->d_inode, inode->i_ino);
kernel/kevent/kevent_aio.c new file +580 -0
580 lines not on this page · raw message
kernel/kevent/kevent_inode.c new file +110 -0
110 lines not on this page · raw message
Signature · 1 line
Evgeniy Polyakov
PATCH 4/4 kevent: generic poll and timer notifications.
In reply to Evgeniy PolyakovThis patch includes generic poll/select and timer notifications.
kernel/kevent/kevent_poll.c new file +223 -0
@@ -0,0 +1,223 @@ +/* +* kevent_poll.c +* +* 2006 Copyright (c) Evgeniy Polyakov <[email protected]> +* All rights reserved. +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +*/ + +#include <linux/kernel.h> +#include <linux/types.h> +#include <linux/list.h> +#include <linux/slab.h> +#include <linux/spinlock.h> +#include <linux/timer.h> +#include <linux/file.h> +#include <linux/kevent.h> +#include <linux/poll.h> +#include <linux/fs.h> + +static kmem_cache_t *kevent_poll_container_cache; +static kmem_cache_t *kevent_poll_priv_cache; + +struct kevent_poll_ctl +{ +struct poll_table_struct pt; +struct kevent *k; +}; + +struct kevent_poll_wait_container +{ +struct list_head container_entry; +wait_queue_head_t *whead; +wait_queue_t wait; +struct kevent *k; +}; + +struct kevent_poll_private +{ +struct list_head container_list; +spinlock_t container_lock; +}; + +static int kevent_poll_enqueue (struct kevent *k); +static int kevent_poll_dequeue (struct kevent *k); +static int kevent_poll_callback (struct kevent *k); + +static int kevent_poll_wait_callback (wait_queue_t *wait, +unsigned mode,int sync,void *key) +{ +struct kevent_poll_wait_container *cont = + container_of(wait, struct kevent_poll_wait_container, wait); +struct kevent *k = cont->k; +struct file *file = k->st->origin; +unsigned long flags; +u32 revents, event; + + revents = file->f_op->poll (file,NULL ); +spin_lock_irqsave (&k->lock, flags); + event = k->event.event; +spin_unlock_irqrestore (&k->lock, flags); + +kevent_storage_ready (k->st,NULL , revents); + +return 0 ; +} + +static void kevent_poll_qproc (struct file *file,wait_queue_head_t *whead, +struct poll_table_struct *poll_table) +{ +struct kevent *k = + container_of(poll_table, struct kevent_poll_ctl, pt)->k; +struct kevent_poll_private *priv = k->priv; +struct kevent_poll_wait_container *cont; +unsigned long flags; + + cont =kmem_cache_alloc (kevent_poll_container_cache,SLAB_KERNEL ); +if (!cont) { +kevent_break (k); +return ; + } + + cont->k = k; +init_waitqueue_func_entry (&cont->wait, kevent_poll_wait_callback); + cont->whead = whead; + +spin_lock_irqsave (&priv->container_lock, flags); +list_add_tail (&cont->container_entry, &priv->container_list); +spin_unlock_irqrestore (&priv->container_lock, flags); + +add_wait_queue (whead, &cont->wait); +} + +static int kevent_poll_enqueue (struct kevent *k) +{ +struct file *file; +int err, ready =0 ; +unsigned int revents; +struct kevent_poll_ctl ctl; +struct kevent_poll_private *priv; + + file =fget (k->event.id.raw[0 ]); +if (!file) +return -ENODEV ; + + err = -EINVAL ; +if (!file->f_op || !file->f_op->poll) + goto err_out_fput; + + err = -ENOMEM ; + priv =kmem_cache_alloc (kevent_poll_priv_cache,SLAB_KERNEL ); +if (!priv) + goto err_out_fput; + +spin_lock_init (&priv->container_lock); +INIT_LIST_HEAD (&priv->container_list); + + k->priv = priv; + + ctl.k = k; +init_poll_funcptr (&ctl.pt, &kevent_poll_qproc); + + err =kevent_storage_enqueue (&file->st, k); +if (err) + goto err_out_free; + + revents = file->f_op->poll (file, &ctl.pt); +if (revents & k->event.event) { + ready =1 ; +kevent_poll_dequeue (k); + } + +return ready; + +err_out_free: +kmem_cache_free (kevent_poll_priv_cache, priv); +err_out_fput: +fput (file); +return err; +} + +static int kevent_poll_dequeue (struct kevent *k) +{ +struct file *file = k->st->origin; +struct kevent_poll_private *priv = k->priv; +struct kevent_poll_wait_container *w, *n; +unsigned long flags; + +kevent_storage_dequeue (k->st, k); + +spin_lock_irqsave (&priv->container_lock, flags); +list_for_each_entry_safe (w, n, &priv->container_list, container_entry) { +list_del (&w->container_entry); +remove_wait_queue (w->whead, &w->wait); +kmem_cache_free (kevent_poll_container_cache, w); + } +spin_unlock_irqrestore (&priv->container_lock, flags); + +kmem_cache_free (kevent_poll_priv_cache, priv); + k->priv =NULL ; + +fput (file); + +return 0 ; +} + +static int kevent_poll_callback (struct kevent *k) +{ +struct file *file = k->st->origin; +unsigned int revents = file->f_op->poll (file,NULL ); +return (revents & k->event.event); +} + +int kevent_init_poll (struct kevent *k) +{ +if (!kevent_poll_container_cache || !kevent_poll_priv_cache) +return -ENOMEM ; + + k->enqueue = &kevent_poll_enqueue; + k->dequeue = &kevent_poll_dequeue; + k->callback = &kevent_poll_callback; +return 0 ; +} + + +static int __init kevent_poll_sys_init (void ) +{ + kevent_poll_container_cache =kmem_cache_create ("kevent_poll_container_cache" , +sizeof (struct kevent_poll_wait_container ),0 ,0 ,NULL ,NULL ); +if (!kevent_poll_container_cache) { +printk (KERN_ERR "Failed to create kevent poll container cache.\n" ); +return -ENOMEM ; + } + + kevent_poll_priv_cache =kmem_cache_create ("kevent_poll_priv_cache" , +sizeof (struct kevent_poll_private ),0 ,0 ,NULL ,NULL ); +if (!kevent_poll_priv_cache) { +printk (KERN_ERR "Failed to create kevent poll private data cache.\n" ); +kmem_cache_destroy (kevent_poll_container_cache); + kevent_poll_container_cache =NULL ; +return -ENOMEM ; + } + +printk (KERN_INFO "Kevent poll()/select() subsystem has been initialized.\n" ); +return 0 ; +} + +static void __exit kevent_poll_sys_fini(void) +{ +kmem_cache_destroy (kevent_poll_priv_cache); +kmem_cache_destroy (kevent_poll_container_cache); +} + +module_init (kevent_poll_sys_init); +module_exit (kevent_poll_sys_fini);
kernel/kevent/kevent_timer.c new file +112 -0
112 lines not on this page · raw message
Signature · 1 line
Evgeniy Polyakov