Skip to contentSkip to conversation

Kevent subsystem.

PATCH In review Evgeniy Polyakov 6 participants 27 messages Jun 22, 2006 – Jun 24, 2006

Subsystems (8)
Networking Network drivers VFS TCP +4 more subsystems
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Kevent subsystem.

Hello.

Kevent subsystem incorporates several AIO/kqueue design notes and ideas. Kevent can be used both for edge and level notifications. It supports socket notifications, network AIO (aio_send(), aio_recv() and aio_sendfile()), inode notifications (create/remove), generic poll()/select() notifications and timer notifications.

It was tested against FreeBSD kqueue and Linux epoll and showed noticeble performance win.

Network asynchronous IO operations were tested against Linux synchronous socket code and showed noticeble performance win.

Patch against linux-2.6.17-git tree attached (gzipped). I would like to hear some comments about the overall design, implementation and plans about it's usefullness for generic kernel.

Design notes, patches, userspace application and perfomance tests can be found at project's homepages.

1. Kevent subsystem.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=kevent

2. Network AIO.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=naio

3. LWN article about kevent.
http://lwn.net/Articles/172844/

Thank you.

Signature · 1 line
	Evgeniy Polyakov
#1 lore raw
Quoted 3 lines · Evgeniy Polyakov

Patch against linux-2.6.17-git tree attached (gzipped). I would like to hear some comments about the overall design, implementation and plans about it's usefullness for generic kernel.

Please send patches as in-line ascii text, along with documentation.

If they're too big, split them up logically into smaller pieces.

- James

Signature · 2 lines
James Morris
<[email protected]>
#2 lore raw
Quoted 10 lines · James Morris
Quoted 3 lines · Evgeniy Polyakov

Patch against linux-2.6.17-git tree attached (gzipped). I would like to hear some comments about the overall design, implementation and plans about it's usefullness for generic kernel.

Please send patches as in-line ascii text, along with documentation.

If they're too big, split them up logically into smaller pieces.

Hmm... It is too big (>100kb) to send as plain text.
I will try to split it to several pieces and resend in a couple of moments, although they are quite strongly logically connected.

Quoted 4 lines

- James

Signature · 2 lines
James Morris
<[email protected]>
Signature · 1 line
	Evgeniy Polyakov
#3 lore raw
In reply to Evgeniy Polyakov

Evgeniy,

Quoted 5 lines · Evgeniy Polyakov

Kevent subsystem incorporates several AIO/kqueue design notes and ideas. Kevent can be used both for edge and level notifications. It supports socket notifications, network AIO (aio_send(), aio_recv() and aio_sendfile()), inode notifications (create/remove), generic poll()/select() notifications and timer notifications.

Great job!
Smooth integration with userland asynch POSIX frameworks (e.g. ACE POSIX_Proactor) may require syscalls (or their emulation) with POSIX interface:

* POSIX_API
* aio_read
* aio_write
* aio_suspend
* aio_error
* aio_return
* aio_cancel

where aio_suspend is very important.

Signature · 5 lines
Sincerely,
------------------------------------------------------------------
Robert Iakobashvili, coroberti at gmail dot com
Navigare necesse est, vivere non est necesse.
------------------------------------------------------------------
#4 lore raw
Quoted 23 lines · Robert Iakobashvili

Evgeniy,

Quoted 5 lines · Evgeniy Polyakov

Kevent subsystem incorporates several AIO/kqueue design notes and ideas. Kevent can be used both for edge and level notifications. It supports socket notifications, network AIO (aio_send(), aio_recv() and aio_sendfile()), inode notifications (create/remove), generic poll()/select() notifications and timer notifications.

Great job!
Smooth integration with userland asynch POSIX frameworks (e.g. ACE POSIX_Proactor) may require syscalls (or their emulation) with POSIX interface:

* POSIX_API
* aio_read
* aio_write
* aio_suspend
* aio_error
* aio_return
* aio_cancel

Quoted 1 line

where aio_suspend is very important.

I've designed network AIO differently as how posix declared it's functions. POSIX AIO is like rocket science - while one tries to implement them all there is no time to do the real work. Although all of them including suspend can be emulated on top of existing calls to kevent.

Quoted 6 lines
Signature · 5 lines
Sincerely,
------------------------------------------------------------------
Robert Iakobashvili, coroberti at gmail dot com
Navigare necesse est, vivere non est necesse.
------------------------------------------------------------------
Signature · 1 line
	Evgeniy Polyakov
#5 lore raw
In reply to Evgeniy Polyakov

In article <[email protected]> (at Thu, 22 Jun 2006 21:14:37 +0400), Evgeniy Polyakov <[email protected]> says:

Quoted 6 lines

Patch against linux-2.6.17-git tree attached (gzipped). I would like to hear some comments about the overall design, implementation and plans about it's usefullness for generic kernel.

Design notes, patches, userspace application and perfomance tests can be found at project's homepages.

| diff --git a/arch/i386/kernel/syscall_table.S b/arch/i386/kernel/syscall_table.S
| index af56987..93e23ff 100644
| --- a/arch/i386/kernel/syscall_table.S
| +++ b/arch/i386/kernel/syscall_table.S
| @@ -316,3 +316,7 @@ ENTRY(sys_call_table)
|  	.long sys_sync_file_range
|  	.long sys_tee			/* 315 */
|  	.long sys_vmsplice
| +	.long sys_aio_recv
| +	.long sys_aio_send
| +	.long sys_aio_sendfile
| +	.long sys_kevent_ctl

:

We do not waste syscall entries; we can probably use socketcall for sys_aio_recv, sys_aio_send (and maybe, sys_aio_sendfile).

| diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
| index 25ecc6e..05d7086 100644
| --- a/net/ipv4/tcp_ipv4.c
| +++ b/net/ipv4/tcp_ipv4.c

:

Please do for tcp_ipv6.c as well. Thank you.

Signature · 2 lines
YOSHIFUJI Hideaki @ USAGI Project  <[email protected]>
GPG-FP  : 9022 65EB 1ECF 3AD1 0BDF  80D8 4807 F894 E062 0EEA
maintainer
#6 lore raw

From: YOSHIFUJI Hideaki <[email protected]>
Date: Fri, 23 Jun 2006 15:12:44 +0900 (JST)

Quoted 2 lines

We do not waste syscall entries; we can probably use socketcall for sys_aio_recv, sys_aio_send (and maybe, sys_aio_sendfile).

socketcall is deprecated and some architectures do not even have a slot for it in their system call tables, one example is x86_64

#7 lore raw

In article <[email protected]> (at Thu, 22 Jun 2006 23:14:07 -0700 (PDT)), David Miller <[email protected]> says:

Quoted 9 lines

From: YOSHIFUJI Hideaki <[email protected]>
Date: Fri, 23 Jun 2006 15:12:44 +0900 (JST)

Quoted 2 lines

We do not waste syscall entries; we can probably use socketcall for sys_aio_recv, sys_aio_send (and maybe, sys_aio_sendfile).

socketcall is deprecated and some architectures do not even have a slot for it in their system call tables, one example is x86_64

Ah, okay. Thanks.

--yoshfuji

#8 lore raw

PATCH 1/4 kevent: core files.

In reply to Evgeniy Polyakov

This patch includes core kevent files:
- userspace controlling
- kernelspace interfaces
- initialisation
- notification state machines

It might also inlclude parts from other subsystem (like network related syscalls so it is possible that it will not compile without other patches applied).

arch/i386/kernel/syscall_table.S +4 -0
index af56987..93e23ff 100644
@@ -316,3 +316,7 @@ ENTRY(sys_call_table)
 	.long sys_sync_file_range
 	.long sys_tee			/* 315 */
 	.long sys_vmsplice
+	.long sys_aio_recv
+	.long sys_aio_send
+	.long sys_aio_sendfile
+	.long sys_kevent_ctl
arch/x86_64/ia32/ia32entry.S +4 -0
index 5a92fed..534d516 100644
@@ -696,4 +696,8 @@ #endif
 	.quad sys_sync_file_range
 	.quad sys_tee
 	.quad compat_sys_vmsplice
+	.quad sys_aio_recv
+	.quad sys_aio_send
+	.quad sys_aio_sendfile
+	.quad sys_kevent_ctl
 ia32_syscall_end:		
include/asm-i386/unistd.h +5 -1
index de2ccc1..52f8642 100644
@@ -322,10 +322,14 @@ #define __NR_splice		313
 #define __NR_sync_file_range	314
 #define __NR_tee		315
 #define __NR_vmsplice		316
+#define __NR_aio_recv		317
+#define __NR_aio_send		318
+#define __NR_aio_sendfile	319
+#define __NR_kevent_ctl		320
 
 #ifdef __KERNEL__
 
-#define NR_syscalls 317
+#define NR_syscalls 321
 
 /*
  * user-visible error numbers are in the range -1 - -128: see
include/asm-x86_64/socket.h +2 -0
index f2cdbea..1f31f86 100644
@@ -49,4 +49,6 @@ #define SO_ACCEPTCONN		30
 
 #define SO_PEERSEC             31
 
+#define SO_ASYNC_SOCK		34
+
 #endif /* _ASM_SOCKET_H */
include/asm-x86_64/unistd.h +9 -2
index 0aff22b..352c34b 100644
@@ -617,11 +617,18 @@ #define __NR_sync_file_range	277
 __SYSCALL(__NR_sync_file_range, sys_sync_file_range)
 #define __NR_vmsplice		278
 __SYSCALL(__NR_vmsplice, sys_vmsplice)
+#define __NR_aio_recv		279
+__SYSCALL(__NR_aio_recv, sys_aio_recv)
+#define __NR_aio_send		280
+__SYSCALL(__NR_aio_send, sys_aio_send)
+#define __NR_aio_sendfile	281
+__SYSCALL(__NR_aio_sendfile, sys_aio_sendfile)
+#define __NR_kevent_ctl		282
+__SYSCALL(__NR_kevent_ctl, sys_kevent_ctl)
 
 #ifdef __KERNEL__
 
-#define __NR_syscall_max __NR_vmsplice
-
+#define __NR_syscall_max __NR_kevent_ctl
 #ifndef __NO_STUBS
 
 /* user-visible error numbers are in the range -1 - -4095 */
include/linux/kevent.h new file +263 -0

263 lines not on this page · raw message

include/linux/kevent_storage.h new file +12 -0
new file mode 100644
index 0000000..bd891f0
@@ -0,0 +1,12 @@
+#ifndef __KEVENT_STORAGE_H
+#define __KEVENT_STORAGE_H
+
+struct kevent_storage
+{
+	void			*origin;		/* Originator's pointer, e.g. struct sock or struct file. Can be NULL. */
+	struct list_head	list;			/* List of queued kevents. */
+	unsigned int		qlen;			/* Number of queued kevents. */
+	spinlock_t		lock;			/* Protects users queue. */
+};
+
+#endif /* __KEVENT_STORAGE_H */
include/linux/syscalls.h +4 -0
index bd67a44..33d436e 100644
@@ -587,4 +587,8 @@ asmlinkage long sys_get_robust_list(int 
 asmlinkage long sys_set_robust_list(struct robust_list_head __user *head,
 				    size_t len);
 
+asmlinkage long sys_aio_recv(int ctl_fd, int s, void __user *buf, size_t size, unsigned flags);
+asmlinkage long sys_aio_send(int ctl_fd, int s, void __user *buf, size_t size, unsigned flags);
+asmlinkage long sys_aio_sendfile(int ctl_fd, int fd, int s, size_t size, unsigned flags);
+asmlinkage long sys_kevent_ctl(int ctl_fd, void __user *buf);
 #endif
init/Kconfig +2 -0
index df864a3..6135afc 100644
@@ -185,6 +185,8 @@ config AUDITSYSCALL
 	  such as SELinux.  To use audit's filesystem watch feature, please
 	  ensure that INOTIFY is configured.
 
+source "kernel/kevent/Kconfig"
+
 config IKCONFIG
 	bool "Kernel .config support"
 	---help---
kernel/Makefile +1 -0
index f6ef00f..eb057ea 100644
@@ -36,6 +36,7 @@ obj-$(CONFIG_DETECT_SOFTLOCKUP) += softl
 obj-$(CONFIG_GENERIC_HARDIRQS) += irq/
 obj-$(CONFIG_SECCOMP) += seccomp.o
 obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o
+obj-$(CONFIG_KEVENT) += kevent/
 obj-$(CONFIG_RELAY) += relay.o
 
 ifneq ($(CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER),y)
kernel/kevent/Kconfig new file +57 -0

57 lines not on this page · raw message

kernel/kevent/Makefile new file +7 -0
new file mode 100644
index 0000000..7dcd651
@@ -0,0 +1,7 @@
+obj-y := kevent.o kevent_user.o kevent_init.o
+obj-$(CONFIG_KEVENT_SOCKET) += kevent_socket.o
+obj-$(CONFIG_KEVENT_INODE) += kevent_inode.o
+obj-$(CONFIG_KEVENT_TIMER) += kevent_timer.o
+obj-$(CONFIG_KEVENT_POLL) += kevent_poll.o
+obj-$(CONFIG_KEVENT_NAIO) += kevent_naio.o
+obj-$(CONFIG_KEVENT_AIO) += kevent_aio.o
kernel/kevent/kevent.c new file +260 -0

260 lines not on this page · raw message

kernel/kevent/kevent_init.c new file +85 -0

85 lines not on this page · raw message

kernel/kevent/kevent_user.c new file +728 -0

728 lines not on this page · raw message

kernel/sys_ni.c +5 -0
index 5433195..dcbacf5 100644
@@ -121,6 +121,11 @@ cond_syscall(ppc_rtas);
 cond_syscall(sys_spu_run);
 cond_syscall(sys_spu_create);
 
+cond_syscall(sys_aio_recv);
+cond_syscall(sys_aio_send);
+cond_syscall(sys_aio_sendfile);
+cond_syscall(sys_kevent_ctl);
+
 /* mmu depending weak syscall entries */
 cond_syscall(sys_mprotect);
 cond_syscall(sys_msync);
Signature · 1 line
	Evgeniy Polyakov
#9 lore raw

Re: [1/4] kevent: core files.

Quoted 5 lines · Evgeniy Polyakov

This patch includes core kevent files:
- userspace controlling
- kernelspace interfaces
- initialisation
- notification state machines

We don't need yet another event mechanism in the kernel, so I don't see why the new syscalls should be added when they don't interoperate with existing solutions. If your results are enough to sway akpm that it is worth taking the patches, then it would make sense to merge the code with the already in-tree APIs.

-ben

#10 lore raw

Re: [1/4] kevent: core files.

Quoted 7 lines · Benjamin LaHaise
Quoted 5 lines · Evgeniy Polyakov

This patch includes core kevent files:
- userspace controlling
- kernelspace interfaces
- initialisation
- notification state machines

Quoted 5 lines

We don't need yet another event mechanism in the kernel, so I don't see why the new syscalls should be added when they don't interoperate with existing solutions. If your results are enough to sway akpm that it is worth taking the patches, then it would make sense to merge the code with the already in-tree APIs.

What API are you talking about?
There is only epoll(), which is 40% slower than kevent, and AIO, which works not as state machine, but as repeated call for the same work. There is also inotify, which allocates new message each time event occurs, which is not a good solution for every situation.

Linux just does not have unified event processing mechanism, which was pointed to many times in AIO mail list and when epoll() was only introduced. I would even say, that Linux does not have such mechanism at all, since every potential user implements it's own, which can not be used with others.

Kevent fixes that. Although implementation itself can be suboptimal for some cases or even unacceptible at all, but it is really needed functionality.

Every existing notification can be built on top of kevent. One can find how easy it was to implement generic poll/select notifications (what epoll() does) or socket notifications (which are similar to epoll(), but are called from inside socket state machine, thus improving processing performance).

Quoted 1 line

-ben

Signature · 1 line
	Evgeniy Polyakov
#11 lore raw

Re: [1/4] kevent: core files.

Quoted 5 lines · Evgeniy Polyakov

What API are you talking about?
There is only epoll(), which is 40% slower than kevent, and AIO, which works not as state machine, but as repeated call for the same work. There is also inotify, which allocates new message each time event occurs, which is not a good solution for every situation.

AIO can be implemented as a state machine. Nothing in the API stops you from doing that, and in fact there was code which was implemented as a state machine used on 2.4 kernels.

Quoted 5 lines

Linux just does not have unified event processing mechanism, which was pointed to many times in AIO mail list and when epoll() was only introduced. I would even say, that Linux does not have such mechanism at all, since every potential user implements it's own, which can not be used with others.

The epoll event API doesn't have space in the event fields for result codes as needed for AIO. The AIO API does -- how is it lacking in this regard?

Quoted 3 lines

Kevent fixes that. Although implementation itself can be suboptimal for some cases or even unacceptible at all, but it is really needed functionality.

At the expense of adding another API? How is this a good thing? Why not spit out events in the existing format?

Quoted 5 lines

Every existing notification can be built on top of kevent. One can find how easy it was to implement generic poll/select notifications (what epoll() does) or socket notifications (which are similar to epoll(), but are called from inside socket state machine, thus improving processing performance).

So far your code is adding a lot without unifying anything.

-ben

Signature · 2 lines
"Time is of no importance, Mr. President, only life is important."
Don't Email: <[email protected]>.
#12 lore raw

Re: [1/4] kevent: core files.

Quoted 10 lines · Benjamin LaHaise
Quoted 5 lines · Evgeniy Polyakov

What API are you talking about?
There is only epoll(), which is 40% slower than kevent, and AIO, which works not as state machine, but as repeated call for the same work. There is also inotify, which allocates new message each time event occurs, which is not a good solution for every situation.

AIO can be implemented as a state machine. Nothing in the API stops you from doing that, and in fact there was code which was implemented as a state machine used on 2.4 kernels.

But now it is implemented as repeated call for the same work, which does not look like it can be used for any other types of work. And repeated work introduce latencies.
As far as I recall, it is you who wanted to remove thread based approach from AIO subsystem.

Quoted 8 lines
Quoted 5 lines

Linux just does not have unified event processing mechanism, which was pointed to many times in AIO mail list and when epoll() was only introduced. I would even say, that Linux does not have such mechanism at all, since every potential user implements it's own, which can not be used with others.

The epoll event API doesn't have space in the event fields for result codes as needed for AIO. The AIO API does -- how is it lacking in this regard?

AIO completion approach was designed to be used with process context VFS update. read/write approach can not cover other types of notifications, like inode updates or timers.

Quoted 6 lines
Quoted 3 lines

Kevent fixes that. Although implementation itself can be suboptimal for some cases or even unacceptible at all, but it is really needed functionality.

At the expense of adding another API? How is this a good thing? Why not spit out events in the existing format?

Format of the structure transferred between the objects does not matter at all. We can create a wrapper on kevent structures or kevent can transform data from AIO objects.
The main design goal of kevent is to provide easy connected hooks into any state machine, which might be used by kernelspace to notify about any kind of events without any knowledge of it's background nature. Kevent can be used for example as notification blocks for address changes or it can replace netlink completely (it can even emulate event multicasting).

Kevent is queue of events, which can be transferred from any object to any destination.

Quoted 7 lines
Quoted 5 lines

Every existing notification can be built on top of kevent. One can find how easy it was to implement generic poll/select notifications (what epoll() does) or socket notifications (which are similar to epoll(), but are called from inside socket state machine, thus improving processing performance).

So far your code is adding a lot without unifying anything.

Not at all!
Kevent is a mechanism, which allows to impleement AIO, network AIO, poll and select, timer control, adaptive readhead (as example of AIO VFS update). All the code I present shows how to use kevent, it is not part of the kevent. One can find Makefile in kevent dir to check what is the core of the subsystem, which allows to be used as a transport for events.

AIO, NAIO, poll/select, socket and timer notifications are just users. One can add it's own usage as easy as to call kevent_storage initialization function and event generation function. All other pieces are hidded in the implementation.

Quoted 4 lines

-ben

Signature · 2 lines
"Time is of no importance, Mr. President, only life is important."
Don't Email: <[email protected]>.
Signature · 1 line
	Evgeniy Polyakov
#13 lore raw

Re: [1/4] kevent: core files.

Quoted 2 lines · Evgeniy Polyakov

But now it is implemented as repeated call for the same work, which does not look like it can be used for any other types of work.

Given an iocb, you do not have to return -EIOCBRETRY, instead return -EIOCBQUEUED and then from whatever context do an aio_complete() with the result for that iocb.

Quoted 3 lines

And repeated work introduce latencies.
As far as I recall, it is you who wanted to remove thread based approach from AIO subsystem.

I have essentially given up on trying to get the filesystem AIO patches in given that the concerns against them are "woe complexity" with no real recourse for inclusion being open. If David is open to changes in the networking area, I'd love to see it built on top of your code.

Quoted 3 lines

AIO completion approach was designed to be used with process context VFS update. read/write approach can not cover other types of notifications, like inode updates or timers.

The completion event is 100% generic and does not need to come from process context. Calling aio_complete() from irq context is entirely valid.

Quoted 11 lines

Format of the structure transferred between the objects does not matter at all. We can create a wrapper on kevent structures or kevent can transform data from AIO objects.

The main design goal of kevent is to provide easy connected hooks into any state machine, which might be used by kernelspace to notify about any kind of events without any knowledge of it's background nature. Kevent can be used for example as notification blocks for address changes or it can replace netlink completely (it can even emulate event multicasting).

Quoted 2 lines

Kevent is queue of events, which can be transferred from any object to any destination.

And io_getevents() reads a queue of events, so I'm not sure why you need a new syscall.

Quoted 8 lines

Not at all!
Kevent is a mechanism, which allows to impleement AIO, network AIO, poll and select, timer control, adaptive readhead (as example of AIO VFS update). All the code I present shows how to use kevent, it is not part of the kevent. One can find Makefile in kevent dir to check what is the core of the subsystem, which allows to be used as a transport for events.

Quoted 4 lines

AIO, NAIO, poll/select, socket and timer notifications are just users. One can add it's own usage as easy as to call kevent_storage initialization function and event generation function. All other pieces are hidded in the implementation.

I'll look at adapting your code to use the existing syscalls. Maybe code will be better at expressing my concerns.

-ben

Signature · 2 lines
"Time is of no importance, Mr. President, only life is important."
Don't Email: <[email protected]>.
#14 lore raw

Re: [1/4] kevent: core files.

Quoted 6 lines · Benjamin LaHaise
Quoted 3 lines

AIO completion approach was designed to be used with process context VFS update. read/write approach can not cover other types of notifications, like inode updates or timers.

The completion event is 100% generic and does not need to come from process context. Calling aio_complete() from irq context is entirely valid.

put_ioctx() can sleep.
And the whole approach is different: AIO just wakes up requesting thread, so user must provide a lot to be able to work with AIO. It perfectly fits VFS design, but it is not acceptible for generic event notifications.

Quoted 11 lines
Quoted 11 lines

Format of the structure transferred between the objects does not matter at all. We can create a wrapper on kevent structures or kevent can transform data from AIO objects.

The main design goal of kevent is to provide easy connected hooks into any state machine, which might be used by kernelspace to notify about any kind of events without any knowledge of it's background nature. Kevent can be used for example as notification blocks for address changes or it can replace netlink completely (it can even emulate event multicasting).

Quoted 5 lines
Quoted 2 lines

Kevent is queue of events, which can be transferred from any object to any destination.

And io_getevents() reads a queue of events, so I'm not sure why you need a new syscall.

It is not syscall, but overall design should be analyzed. It is possible to use existing ssycalls, kevent design does not care about how it's data structures are delivered to the internal "processor".

Quoted 8 lines
Quoted 8 lines

Not at all!
Kevent is a mechanism, which allows to impleement AIO, network AIO, poll and select, timer control, adaptive readhead (as example of AIO VFS update). All the code I present shows how to use kevent, it is not part of the kevent. One can find Makefile in kevent dir to check what is the core of the subsystem, which allows to be used as a transport for events.

Quoted 7 lines
Quoted 4 lines

AIO, NAIO, poll/select, socket and timer notifications are just users. One can add it's own usage as easy as to call kevent_storage initialization function and event generation function. All other pieces are hidded in the implementation.

I'll look at adapting your code to use the existing syscalls. Maybe code will be better at expressing my concerns.

That would be great.

Quoted 1 line

-ben

Signature · 1 line
	Evgeniy Polyakov
#15 lore raw

Re: [1/4] kevent: core files.

Quoted 9 lines · Evgeniy Polyakov
Quoted 6 lines · Benjamin LaHaise
Quoted 3 lines

AIO completion approach was designed to be used with process context VFS update. read/write approach can not cover other types of notifications, like inode updates or timers.

The completion event is 100% generic and does not need to come from process context. Calling aio_complete() from irq context is entirely valid.

put_ioctx() can sleep.

Err, no, that should definately not be the case. If it can, someone has completely broken aio.

Quoted 4 lines

It is not syscall, but overall design should be analyzed. It is possible to use existing ssycalls, kevent design does not care about how it's data structures are delivered to the internal "processor".

Okay, that's good to hear. =-)

-ben

Signature · 2 lines
"Time is of no importance, Mr. President, only life is important."
Don't Email: <[email protected]>.
#16 lore raw

Re: [1/4] kevent: core files.

Quoted 6 lines · Benjamin LaHaise
Quoted 5 lines · Evgeniy Polyakov
Quoted 4 lines · Benjamin LaHaise
Quoted 3 lines

AIO completion approach was designed to be used with process context VFS update. read/write approach can not cover other types of notifications, like inode updates or timers.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

The completion event is 100% generic and does not need to come from process context. Calling aio_complete() from irq context is entirely valid.

put_ioctx() can sleep.

Err, no, that should definately not be the case. If it can, someone has completely broken aio.

When reference counter hits zero it flushes aio workqueue, which can sleep.
put_ioctx() -> __put_ioctx() -> cancel_delayed_work()/flush_workqueue().

It is there at least from 2.6.15 days (it is the oldest tree I can access using my extremely slow GPRS link).

Hang the looter!

Quoted 1 line

-ben

Signature · 1 line
	Evgeniy Polyakov
maintainer
#17 lore raw

Re: [1/4] kevent: core files.

In reply to Evgeniy Polyakov

From: Evgeniy Polyakov <[email protected]>
Date: Fri, 23 Jun 2006 23:24:29 +0400

Quoted 9 lines

Linux just does not have unified event processing mechanism, which was pointed to many times in AIO mail list and when epoll() was only introduced. I would even say, that Linux does not have such mechanism at all, since every potential user implements it's own, which can not be used with others.

Kevent fixes that. Although implementation itself can be suboptimal for some cases or even unacceptible at all, but it is really needed functionality.

I completely agree with Evgeniy here.

There is nothing in the kernel today that provides integrated event handling. Nothing. So when someone says to use the "existing" stuff, they need to have their head examined.

The existing AIO stuff stinks as a set of interfaces. It was designed by a standards committee, not by people truly interested in a good performing event processing design. It is especially poorly suited for networking, and any networking developer understands this.

It is pretty much a foregone conclusion that we will need new APIs to get good networking performance. Every existing interface has one limitation or another.

So we should be happy people like Evgeniy try to work on this stuff, instead of discouraging them.

#18 lore raw

Re: [1/4] kevent: core files.

Quoted 5 lines · David Miller

I completely agree with Evgeniy here.

There is nothing in the kernel today that provides integrated event handling. Nothing. So when someone says to use the "existing" stuff, they need to have their head examined.

The existing AIO events are *events*, with the syscalls providing the reading of events.

Quoted 4 lines

The existing AIO stuff stinks as a set of interfaces. It was designed by a standards committee, not by people truly interested in a good performing event processing design. It is especially poorly suited for networking, and any networking developer understands this.

I disagree. Stuffing an event that a read or write is complete/ready is a good way of handling things, even more so with hardware that will perform the memory copies to/from user buffers.

Quoted 3 lines

It is pretty much a foregone conclusion that we will need new APIs to get good networking performance. Every existing interface has one limitation or another.

Eh? Nobody has posted any numbers comparing the approaches yet, so this is pure handwaving, unless you have real concrete results?

Quoted 2 lines

So we should be happy people like Evgeniy try to work on this stuff, instead of discouraging them.

I would like to encourage him, but at the same time I don't want to see creating APIs that essentially duplicate existing work and needlessly break compatibility. I completely agree that the in-kernel APIs are not as encompassing as they should be, and within the kernel Evgeniy's work may well be the way to go. What I do not agree is that we need new syscalls at this point. I'm perfectly willing to accept proof that change is needed if we do a proper comparision between any new syscall API and the use of the existing syscall API, but the pain of introducing a new API is sufficiently large that I think it is worth looking at the numbers.

-ben

Signature · 2 lines
"Time is of no importance, Mr. President, only life is important."
Don't Email: <[email protected]>.
#19 lore raw

Re: [1/4] kevent: core files.

Quoted 5 lines · Benjamin LaHaise

may well be the way to go. What I do not agree is that we need new syscalls at this point. I'm perfectly willing to accept proof that change is needed if we do a proper comparision between any new syscall API and the use of the existing syscall API, but the pain of introducing a new API is sufficiently large that I think it is worth looking at the numbers.

New syscall is just an interface. Originally kevent (and it still can) use char device and it's ioctl method.
It is perfectly possible to create wrappers for posix aio_* calls, although I do not see why it is needed.
No need to concentrate on end-users interface at this point - it can be changed at any time since design allows it, we should think about overall design and if it is ok, move forward in implementation.

Btw, new API adds only one syscall for userspace kevent processing (and three for send/recv/sendfile for network AIO).

According to numbers: kevent compared to epoll resulted in the folllowing numbers:
kevent: more than 2600 requests per second (trivial web server)
epoll: about 1600-1800 requests.
Number of errors for 3k bursts of connections with 30K connections total in 10seconds:
kevent: about 2k errors.
epoll: upto 15k errors.

More detailed results can be found on project's homepage at:
tservice.net.ru/~s0mbre/old/?section=projects&item=kevent

Quoted 1 line

-ben

Signature · 1 line
	Evgeniy Polyakov
#20 lore raw

Re: [1/4] kevent: core files.

Hi,

Quoted 8 lines

According to numbers: kevent compared to epoll resulted in the folllowing numbers:
kevent: more than 2600 requests per second (trivial web server)
epoll: about 1600-1800 requests.
Number of errors for 3k bursts of connections with 30K connections total in 10seconds:
kevent: about 2k errors.
epoll: upto 15k errors.

If it beats the great epoll, it means a real business case for kevent.

All previous attempts in kernel as well as by glibc and other userland emulations to provide some real AIO infrastructure and API for server applications with performance benefits, were not too much successful. Heavy load networking servers are normally not using AIOs on linux due to low performance.

Quoted 1 line

From another side Windows have a very strong I/O completion ports API,

which are widely used for the most heavy load applications.

Kevent may take linux servers productivity forward in general as well as encourage moving aio-applications from windows to linux.

Signature · 5 lines
Sincerely,
------------------------------------------------------------------
Robert Iakobashvili, coroberti at gmail dot com
Navigare necesse est, vivere non est necesse.
------------------------------------------------------------------
maintainer
#21 lore raw

Re: [1/4] kevent: core files.

In reply to Benjamin LaHaise

From: Benjamin LaHaise <[email protected]>
Date: Fri, 23 Jun 2006 16:31:14 -0400

Quoted 2 lines

Eh? Nobody has posted any numbers comparing the approaches yet, so this is pure handwaving, unless you have real concrete results?

Evgeniy posts numbers and performance graphs on his kevent work all the time.

Van Jacobson did in his LCA2006 net channel slides too, perhaps you missed that.

#22 lore raw

Re: [1/4] kevent: core files.

Quoted 8 lines · David Miller

From: Benjamin LaHaise <[email protected]>
Date: Fri, 23 Jun 2006 16:31:14 -0400

Quoted 2 lines

Eh? Nobody has posted any numbers comparing the approaches yet, so this is pure handwaving, unless you have real concrete results?

Evgeniy posts numbers and performance graphs on his kevent work all the time.

But you're argueing that the performance of something that hasn't been tested is worse simply by nature of it not having been tested. That's a fallacy of omission, iiuc.

Quoted 2 lines

Van Jacobson did in his LCA2006 net channel slides too, perhaps you missed that.

I have yet to be convinced that the layering violation known as net channels is the right way to go, mostly because it breaks horribly in a few cases -- think what happens during periods of CPU overcommit, in which case doing too much in interrupt context will kill a system (which is why softirqs are needed). The effect of doing all processing in user context creates issues with delayed acks (due to context switching to other tasks in the system), which will cause excess retransmits. The hard problems associated with packet filtering and security are also still unresolved, which is okay for a paper, but a concern in real life.

There are also a number of performance flaws in the current stack that show up under profiling, some of which I posted fixes for, some of which have yet to be fixed. The pushf/popf pipeline stall was one of the bigger instances of CPU wastage that Van Jacobson noticed (it shows up as bottom halves using lots of CPU). Iirc, Ingo's real time patches may avoid that by way of reworking the irq disable/enable mechanism, which would mean the results need retesting. Using the cr8 register to enable/disable interrupts on x86-64 might also improve things, as that would eliminate the flags dependancy of cli/sti...

In short, there's a lot of work that still has to be done.

-ben

Signature · 2 lines
"Time is of no importance, Mr. President, only life is important."
Don't Email: <[email protected]>.
maintainer
#23 lore raw

Re: [1/4] kevent: core files.

From: Benjamin LaHaise <[email protected]>
Date: Fri, 23 Jun 2006 17:53:14 -0400

Quoted 2 lines

The effect of doing all processing in user context creates issues with delayed acks (due to context switching to other tasks in the system),

The Linux TCP stack does this today.

Full TCP input protocol processing is done in the user process context.

What you are not understanding is that process scheduling helps TCP, it does not hinder it. If the system is loaded, we want the senders to pace themselves to the rate at which the kernel can schedule the abundance of receiver work it has. And this happens naturally when the TCP protocol input processing operates in process context.

Your fear of cpu overcommit in interrupt handlers is also heavily flawed. Net channels do a socket demux, and a queue entail plus ring a doorbell if necessary, nothing more.

#24 lore raw

PATCH 2/4 kevent: network notifications.

In reply to Evgeniy Polyakov

This patchset includes socket notifications and network asynchronous IO.

include/asm-i386/socket.h +2 -0
index 802ae76..3473f5c 100644
@@ -49,4 +49,6 @@ #define SO_ACCEPTCONN		30
 
 #define SO_PEERSEC		31
 
+#define SO_ASYNC_SOCK		34
+
 #endif /* _ASM_SOCKET_H */
include/linux/skbuff.h +2 -0
index 66f8819..ea914c3 100644
@@ -1269,6 +1269,8 @@ extern struct sk_buff *skb_recv_datagram
 					 int noblock, int *err);
 extern unsigned int    datagram_poll(struct file *file, struct socket *sock,
 				     struct poll_table_struct *wait);
+extern int	       skb_copy_datagram(const struct sk_buff *from, 
+					 int offset, void *dst, int size);
 extern int	       skb_copy_datagram_iovec(const struct sk_buff *from,
 					       int offset, struct iovec *to,
 					       int size);
include/net/sock.h +25 -15
index d10dfec..7a2bee3 100644
@@ -47,6 +47,7 @@ #include <linux/module.h>
 #include <linux/netdevice.h>
 #include <linux/skbuff.h>	/* struct sk_buff */
 #include <linux/security.h>
+#include <linux/kevent.h>
 
 #include <linux/filter.h>
 
@@ -386,6 +387,8 @@ enum sock_flags {
 	SOCK_NO_LARGESEND, /* whether to sent large segments or not */
 	SOCK_LOCALROUTE, /* route locally only, %SO_DONTROUTE setting */
 	SOCK_QUEUE_SHRUNK, /* write queue has been shrunk recently */
+	SOCK_ASYNC,
+	SOCK_ASYNC_INUSE,
 };
 
 static inline void sock_copy_flags(struct sock *nsk, struct sock *osk)
@@ -445,6 +448,21 @@ static inline int sk_stream_memory_free(
 
 extern void sk_stream_rfree(struct sk_buff *skb);
 
+struct socket_alloc {
+	struct socket socket;
+	struct inode vfs_inode;
+};
+
+static inline struct socket *SOCKET_I(struct inode *inode)
+{
+	return &container_of(inode, struct socket_alloc, vfs_inode)->socket;
+}
+
+static inline struct inode *SOCK_INODE(struct socket *socket)
+{
+	return &container_of(socket, struct socket_alloc, socket)->vfs_inode;
+}
+
 static inline void sk_stream_set_owner_r(struct sk_buff *skb, struct sock *sk)
 {
 	skb->sk = sk;
@@ -472,6 +490,7 @@ static inline void sk_add_backlog(struct
 		sk->sk_backlog.tail = skb;
 	}
 	skb->next = NULL;
+	kevent_socket_notify(sk, KEVENT_SOCKET_RECV);
 }
 
 #define sk_wait_event(__sk, __timeo, __condition)		\
@@ -543,6 +562,12 @@ struct proto {
 
 	int			(*backlog_rcv) (struct sock *sk, 
 						struct sk_buff *skb);
+	
+	int			(*async_recv) (struct sock *sk, 
+						void *dst, size_t size);
+	int			(*async_send) (struct sock *sk, 
+						struct page **pages, unsigned int poffset, 
+						size_t size);
 
 	/* Keeping track of sk's, looking them up, and port selection methods. */
 	void			(*hash)(struct sock *sk);
@@ -674,21 +699,6 @@ static inline struct kiocb *siocb_to_kio
 	return si->kiocb;
 }
 
-struct socket_alloc {
-	struct socket socket;
-	struct inode vfs_inode;
-};
-
-static inline struct socket *SOCKET_I(struct inode *inode)
-{
-	return &container_of(inode, struct socket_alloc, vfs_inode)->socket;
-}
-
-static inline struct inode *SOCK_INODE(struct socket *socket)
-{
-	return &container_of(socket, struct socket_alloc, socket)->vfs_inode;
-}
-
 extern void __sk_stream_mem_reclaim(struct sock *sk);
 extern int sk_stream_mem_schedule(struct sock *sk, int size, int kind);
 
include/net/tcp.h +3 -0
index 5f4eb5c..820cd5a 100644
@@ -364,6 +364,8 @@ extern int			compat_tcp_setsockopt(struc
 					int level, int optname,
 					char __user *optval, int optlen);
 extern void			tcp_set_keepalive(struct sock *sk, int val);
+extern int			tcp_async_recv(struct sock *sk, void *dst, size_t size);
+extern int			tcp_async_send(struct sock *sk, struct page **pages, unsigned int poffset, size_t size);
 extern int			tcp_recvmsg(struct kiocb *iocb, struct sock *sk,
 					    struct msghdr *msg,
 					    size_t len, int nonblock, 
@@ -857,6 +859,7 @@ static inline int tcp_prequeue(struct so
 			tp->ucopy.memory = 0;
 		} else if (skb_queue_len(&tp->ucopy.prequeue) == 1) {
 			wake_up_interruptible(sk->sk_sleep);
+			kevent_socket_notify(sk, KEVENT_SOCKET_RECV|KEVENT_SOCKET_SEND);
 			if (!inet_csk_ack_scheduled(sk))
 				inet_csk_reset_xmit_timer(sk, ICSK_TIME_DACK,
 						          (3 * TCP_RTO_MIN) / 4,
kernel/kevent/kevent_naio.c new file +239 -0

239 lines not on this page · raw message

kernel/kevent/kevent_socket.c new file +125 -0

125 lines not on this page · raw message

net/core/datagram.c +55 -0

67 lines not on this page · raw message

net/core/sock.c +17 -1
index 5d820c3..3345048 100644
@@ -564,6 +564,16 @@ #endif
 			spin_unlock_bh(&sk->sk_lock.slock);
 			ret = -ENONET;
 			break;
+#ifdef CONFIG_KEVENT_SOCKET
+		case SO_ASYNC_SOCK:
+			spin_lock_bh(&sk->sk_lock.slock);
+			if (valbool)
+				sock_set_flag(sk, SOCK_ASYNC);
+			else
+				sock_reset_flag(sk, SOCK_ASYNC);
+			spin_unlock_bh(&sk->sk_lock.slock);
+			break;
+#endif
 
 		/* We implement the SO_SNDLOWAT etc to
 		   not be settable (1003.1g 5.3) */
@@ -1313,6 +1323,7 @@ static void sock_def_wakeup(struct sock 
 	if (sk->sk_sleep && waitqueue_active(sk->sk_sleep))
 		wake_up_interruptible_all(sk->sk_sleep);
 	read_unlock(&sk->sk_callback_lock);
+	kevent_socket_notify(sk, KEVENT_SOCKET_RECV|KEVENT_SOCKET_SEND);
 }
 
 static void sock_def_error_report(struct sock *sk)
@@ -1322,6 +1333,7 @@ static void sock_def_error_report(struct
 		wake_up_interruptible(sk->sk_sleep);
 	sk_wake_async(sk,0,POLL_ERR); 
 	read_unlock(&sk->sk_callback_lock);
+	kevent_socket_notify(sk, KEVENT_SOCKET_RECV|KEVENT_SOCKET_SEND);
 }
 
 static void sock_def_readable(struct sock *sk, int len)
@@ -1331,6 +1343,7 @@ static void sock_def_readable(struct soc
 		wake_up_interruptible(sk->sk_sleep);
 	sk_wake_async(sk,1,POLL_IN);
 	read_unlock(&sk->sk_callback_lock);
+	kevent_socket_notify(sk, KEVENT_SOCKET_RECV|KEVENT_SOCKET_SEND);
 }
 
 static void sock_def_write_space(struct sock *sk)
@@ -1350,6 +1363,7 @@ static void sock_def_write_space(struct 
 	}
 
 	read_unlock(&sk->sk_callback_lock);
+	kevent_socket_notify(sk, KEVENT_SOCKET_SEND|KEVENT_SOCKET_RECV);
 }
 
 static void sock_def_destruct(struct sock *sk)
@@ -1454,8 +1468,10 @@ void fastcall release_sock(struct sock *
 	if (sk->sk_backlog.tail)
 		__release_sock(sk);
 	sk->sk_lock.owner = NULL;
-        if (waitqueue_active(&(sk->sk_lock.wq)))
+        if (waitqueue_active(&(sk->sk_lock.wq))) {
 		wake_up(&(sk->sk_lock.wq));
+		kevent_socket_notify(sk, KEVENT_SOCKET_RECV|KEVENT_SOCKET_SEND);
+	}
 	spin_unlock_bh(&(sk->sk_lock.slock));
 }
 EXPORT_SYMBOL(release_sock);
net/core/stream.c +1 -0
index e948969..91e2e07 100644
@@ -36,6 +36,7 @@ void sk_stream_write_space(struct sock *
 			wake_up_interruptible(sk->sk_sleep);
 		if (sock->fasync_list && !(sk->sk_shutdown & SEND_SHUTDOWN))
 			sock_wake_async(sock, 2, POLL_OUT);
+		kevent_socket_notify(sk, KEVENT_SOCKET_SEND|KEVENT_SOCKET_RECV);
 	}
 }
 
net/ipv4/tcp.c +272 -0

290 lines not on this page · raw message

net/ipv4/tcp_input.c +3 -1
index e08245b..5655b1e 100644
@@ -3113,6 +3113,7 @@ static void tcp_ofo_queue(struct sock *s
 
 		__skb_unlink(skb, &tp->out_of_order_queue);
 		__skb_queue_tail(&sk->sk_receive_queue, skb);
+		kevent_socket_notify(sk, KEVENT_SOCKET_RECV);
 		tp->rcv_nxt = TCP_SKB_CB(skb)->end_seq;
 		if(skb->h.th->fin)
 			tcp_fin(skb, sk, skb->h.th);
@@ -3956,7 +3957,8 @@ int tcp_rcv_established(struct sock *sk,
 			int copied_early = 0;
 
 			if (tp->copied_seq == tp->rcv_nxt &&
-			    len - tcp_header_len <= tp->ucopy.len) {
+			    len - tcp_header_len <= tp->ucopy.len &&
+			    !sock_async(sk)) {
 #ifdef CONFIG_NET_DMA
 				if (tcp_dma_try_early_copy(sk, skb, tcp_header_len)) {
 					copied_early = 1;
net/ipv4/tcp_ipv4.c +25 -15
index 25ecc6e..05d7086 100644
@@ -62,6 +62,7 @@ #include <linux/cache.h>
 #include <linux/jhash.h>
 #include <linux/init.h>
 #include <linux/times.h>
+#include <linux/kevent.h>
 
 #include <net/icmp.h>
 #include <net/inet_hashtables.h>
@@ -850,6 +851,7 @@ #endif
 	   	reqsk_free(req);
 	} else {
 		inet_csk_reqsk_queue_hash_add(sk, req, TCP_TIMEOUT_INIT);
+		kevent_socket_notify(sk, KEVENT_SOCKET_ACCEPT);
 	}
 	return 0;
 
@@ -1089,24 +1091,30 @@ process:
 
 	skb->dev = NULL;
 
-	bh_lock_sock(sk);
 	ret = 0;
-	if (!sock_owned_by_user(sk)) {
+	if (sock_async(sk)) {
+		spin_lock_bh(&sk->sk_lock.slock);
+		ret = tcp_v4_do_rcv(sk, skb);
+		spin_unlock_bh(&sk->sk_lock.slock);
+	} else {
+		bh_lock_sock(sk);
+		if (!sock_owned_by_user(sk)) {
 #ifdef CONFIG_NET_DMA
-		struct tcp_sock *tp = tcp_sk(sk);
-		if (!tp->ucopy.dma_chan && tp->ucopy.pinned_list)
-			tp->ucopy.dma_chan = get_softnet_dma();
-		if (tp->ucopy.dma_chan)
-			ret = tcp_v4_do_rcv(sk, skb);
-		else
+			struct tcp_sock *tp = tcp_sk(sk);
+			if (!tp->ucopy.dma_chan && tp->ucopy.pinned_list)
+				tp->ucopy.dma_chan = get_softnet_dma();
+			if (tp->ucopy.dma_chan)
+				ret = tcp_v4_do_rcv(sk, skb);
+			else
 #endif
-		{
-			if (!tcp_prequeue(sk, skb))
-			ret = tcp_v4_do_rcv(sk, skb);
-		}
-	} else
-		sk_add_backlog(sk, skb);
-	bh_unlock_sock(sk);
+			{
+				if (!tcp_prequeue(sk, skb))
+				ret = tcp_v4_do_rcv(sk, skb);
+			}
+		} else
+			sk_add_backlog(sk, skb);
+		bh_unlock_sock(sk);
+	}
 
 	sock_put(sk);
 
@@ -1830,6 +1838,8 @@ struct proto tcp_prot = {
 	.getsockopt		= tcp_getsockopt,
 	.sendmsg		= tcp_sendmsg,
 	.recvmsg		= tcp_recvmsg,
+	.async_recv		= tcp_async_recv,
+	.async_send		= tcp_async_send,
 	.backlog_rcv		= tcp_v4_do_rcv,
 	.hash			= tcp_v4_hash,
 	.unhash			= tcp_unhash,
Signature · 1 line
	Evgeniy Polyakov
#25 lore raw

PATCH 3/4 kevent: fs/aio notifications.

In reply to Evgeniy Polyakov

This patch includes asynchronous propagation of file's data into VFS cache and aio_sendfile() implementation.

fs/bio.c +1 -1
index 6a0b9ad..a3ee530 100644
@@ -119,7 +119,7 @@ void bio_free(struct bio *bio, struct bi
 /*
  * default destructor for a bio allocated with bio_alloc_bioset()
  */
-static void bio_fs_destructor(struct bio *bio)
+void bio_fs_destructor(struct bio *bio)
 {
 	bio_free(bio, fs_bio_set);
 }
fs/ext2/inode.c +1 -0
index 04af9c4..295fce9 100644
@@ -685,6 +685,7 @@ ext2_writepages(struct address_space *ma
 }
 
 struct address_space_operations ext2_aops = {
+	.get_block		= ext2_get_block,
 	.readpage		= ext2_readpage,
 	.readpages		= ext2_readpages,
 	.writepage		= ext2_writepage,
fs/ext3/inode.c +1 -0
index 2edd7ee..e44f5ad 100644
@@ -1700,6 +1700,7 @@ static int ext3_journalled_set_page_dirt
 }
 
 static struct address_space_operations ext3_ordered_aops = {
+	.get_block	= ext3_get_block,
 	.readpage	= ext3_readpage,
 	.readpages	= ext3_readpages,
 	.writepage	= ext3_ordered_writepage,
fs/file_table.c +6 -0
index bcea199..8759479 100644
@@ -113,6 +113,9 @@ struct file *get_empty_filp(void)
 	if (security_file_alloc(f))
 		goto fail_sec;
 
+#ifdef CONFIG_KEVENT_POLL
+	kevent_storage_init(f, &f->st);
+#endif
 	tsk = current;
 	INIT_LIST_HEAD(&f->f_u.fu_list);
 	atomic_set(&f->f_count, 1);
@@ -160,6 +163,9 @@ void fastcall __fput(struct file *file)
 	might_sleep();
 
 	fsnotify_close(file);
+#ifdef CONFIG_KEVENT_POLL
+	kevent_storage_fini(&file->st);
+#endif
 	/*
 	 * The function eventpoll_release() should be the first called
 	 * in the file cleanup chain.
fs/inode.c +7 -0
index 3a2446a..0493935 100644
@@ -22,6 +22,7 @@ #include <linux/pagemap.h>
 #include <linux/cdev.h>
 #include <linux/bootmem.h>
 #include <linux/inotify.h>
+#include <linux/kevent.h>
 #include <linux/mount.h>
 
 /*
@@ -166,12 +167,18 @@ #endif
 		}
 		memset(&inode->u, 0, sizeof(inode->u));
 		inode->i_mapping = mapping;
+#if defined CONFIG_KEVENT
+		kevent_storage_init(inode, &inode->st);
+#endif
 	}
 	return inode;
 }
 
 void destroy_inode(struct inode *inode) 
 {
+#if defined CONFIG_KEVENT_INODE || defined CONFIG_KEVENT_SOCKET
+	kevent_storage_fini(&inode->st);
+#endif
 	BUG_ON(inode_has_buffers(inode));
 	security_inode_free(inode);
 	if (inode->i_sb->s_op->destroy_inode)
fs/reiserfs/inode.c +1 -0
index 9857e50..bbbb578 100644
@@ -2997,6 +2997,7 @@ int reiserfs_setattr(struct dentry *dent
 }
 
 struct address_space_operations reiserfs_address_space_operations = {
+	.get_block = reiserfs_get_block,
 	.writepage = reiserfs_writepage,
 	.readpage = reiserfs_readpage,
 	.readpages = reiserfs_readpages,
include/linux/fs.h +12 -0
index ecc8c2c..248f6a1 100644
@@ -236,6 +236,9 @@ #include <linux/mutex.h>
 #include <asm/atomic.h>
 #include <asm/semaphore.h>
 #include <asm/byteorder.h>
+#ifdef CONFIG_KEVENT
+#include <linux/kevent_storage.h>
+#endif
 
 struct hd_geometry;
 struct iovec;
@@ -348,6 +351,8 @@ struct address_space;
 struct writeback_control;
 
 struct address_space_operations {
+	int  (*get_block)(struct inode *inode, sector_t iblock,
+			struct buffer_head *bh_result, int create);
 	int (*writepage)(struct page *page, struct writeback_control *wbc);
 	int (*readpage)(struct file *, struct page *);
 	void (*sync_page)(struct page *);
@@ -526,6 +531,10 @@ #ifdef CONFIG_INOTIFY
 	struct mutex		inotify_mutex;	/* protects the watches list */
 #endif
 
+#ifdef CONFIG_KEVENT_INODE
+	struct kevent_storage	st;
+#endif
+
 	unsigned long		i_state;
 	unsigned long		dirtied_when;	/* jiffies of first dirtying */
 
@@ -659,6 +668,9 @@ #ifdef CONFIG_EPOLL
 	struct list_head	f_ep_links;
 	spinlock_t		f_ep_lock;
 #endif /* #ifdef CONFIG_EPOLL */
+#ifdef CONFIG_KEVENT_POLL
+	struct kevent_storage	st;
+#endif
 	struct address_space	*f_mapping;
 };
 extern spinlock_t files_lock;
include/linux/fsnotify.h +5 -0
index cc5dec7..0acc8db 100644
@@ -15,6 +15,7 @@ #ifdef __KERNEL__
 
 #include <linux/dnotify.h>
 #include <linux/inotify.h>
+#include <linux/kevent.h>
 #include <linux/audit.h>
 
 /*
@@ -79,6 +80,7 @@ static inline void fsnotify_nameremove(s
 		isdir = IN_ISDIR;
 	dnotify_parent(dentry, DN_DELETE);
 	inotify_dentry_parent_queue_event(dentry, IN_DELETE|isdir, 0, dentry->d_name.name);
+	kevent_inode_notify_parent(dentry, KEVENT_INODE_REMOVE);
 }
 
 /*
@@ -88,6 +90,7 @@ static inline void fsnotify_inoderemove(
 {
 	inotify_inode_queue_event(inode, IN_DELETE_SELF, 0, NULL, NULL);
 	inotify_inode_is_dead(inode);
+	kevent_inode_remove(inode);
 }
 
 /*
@@ -96,6 +99,7 @@ static inline void fsnotify_inoderemove(
 static inline void fsnotify_create(struct inode *inode, struct dentry *dentry)
 {
 	inode_dir_notify(inode, DN_CREATE);
+	kevent_inode_notify(inode, KEVENT_INODE_CREATE);
 	inotify_inode_queue_event(inode, IN_CREATE, 0, dentry->d_name.name,
 				  dentry->d_inode);
 	audit_inode_child(dentry->d_name.name, dentry->d_inode, inode->i_ino);
@@ -107,6 +111,7 @@ static inline void fsnotify_create(struc
 static inline void fsnotify_mkdir(struct inode *inode, struct dentry *dentry)
 {
 	inode_dir_notify(inode, DN_CREATE);
+	kevent_inode_notify(inode, KEVENT_INODE_CREATE);
 	inotify_inode_queue_event(inode, IN_CREATE | IN_ISDIR, 0, 
 				  dentry->d_name.name, dentry->d_inode);
 	audit_inode_child(dentry->d_name.name, dentry->d_inode, inode->i_ino);
kernel/kevent/kevent_aio.c new file +580 -0

580 lines not on this page · raw message

kernel/kevent/kevent_inode.c new file +110 -0

110 lines not on this page · raw message

Signature · 1 line
	Evgeniy Polyakov
#26 lore raw

PATCH 4/4 kevent: generic poll and timer notifications.

In reply to Evgeniy Polyakov

This patch includes generic poll/select and timer notifications.

kernel/kevent/kevent_poll.c new file +223 -0
new file mode 100644
index 0000000..4950e7c
@@ -0,0 +1,223 @@
+/*
+ * 	kevent_poll.c
+ * 
+ * 2006 Copyright (c) Evgeniy Polyakov <[email protected]>
+ * All rights reserved.
+ * 
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/timer.h>
+#include <linux/file.h>
+#include <linux/kevent.h>
+#include <linux/poll.h>
+#include <linux/fs.h>
+
+static kmem_cache_t *kevent_poll_container_cache;
+static kmem_cache_t *kevent_poll_priv_cache;
+
+struct kevent_poll_ctl
+{
+	struct poll_table_struct 	pt;
+	struct kevent			*k;
+};
+
+struct kevent_poll_wait_container
+{
+	struct list_head		container_entry;
+	wait_queue_head_t		*whead;
+	wait_queue_t			wait;
+	struct kevent			*k;
+};
+
+struct kevent_poll_private
+{
+	struct list_head		container_list;
+	spinlock_t			container_lock;
+};
+
+static int kevent_poll_enqueue(struct kevent *k);
+static int kevent_poll_dequeue(struct kevent *k);
+static int kevent_poll_callback(struct kevent *k);
+
+static int kevent_poll_wait_callback(wait_queue_t *wait, 
+		unsigned mode, int sync, void *key)
+{
+	struct kevent_poll_wait_container *cont = 
+		container_of(wait, struct kevent_poll_wait_container, wait);
+	struct kevent *k = cont->k;
+	struct file *file = k->st->origin;
+	unsigned long flags;
+	u32 revents, event;
+
+	revents = file->f_op->poll(file, NULL);
+	spin_lock_irqsave(&k->lock, flags);
+	event = k->event.event;
+	spin_unlock_irqrestore(&k->lock, flags);
+
+	kevent_storage_ready(k->st, NULL, revents);
+
+	return 0;
+}
+
+static void kevent_poll_qproc(struct file *file, wait_queue_head_t *whead, 
+		struct poll_table_struct *poll_table)
+{
+	struct kevent *k = 
+		container_of(poll_table, struct kevent_poll_ctl, pt)->k;
+	struct kevent_poll_private *priv = k->priv;
+	struct kevent_poll_wait_container *cont;
+	unsigned long flags;
+
+	cont = kmem_cache_alloc(kevent_poll_container_cache, SLAB_KERNEL);
+	if (!cont) {
+		kevent_break(k);
+		return;
+	}
+		
+	cont->k = k;
+	init_waitqueue_func_entry(&cont->wait, kevent_poll_wait_callback);
+	cont->whead = whead;
+
+	spin_lock_irqsave(&priv->container_lock, flags);
+	list_add_tail(&cont->container_entry, &priv->container_list);
+	spin_unlock_irqrestore(&priv->container_lock, flags);
+
+	add_wait_queue(whead, &cont->wait);
+}
+
+static int kevent_poll_enqueue(struct kevent *k)
+{
+	struct file *file;
+	int err, ready = 0;
+	unsigned int revents;
+	struct kevent_poll_ctl ctl;
+	struct kevent_poll_private *priv;
+
+	file = fget(k->event.id.raw[0]);
+	if (!file)
+		return -ENODEV;
+
+	err = -EINVAL;
+	if (!file->f_op || !file->f_op->poll)
+		goto err_out_fput;
+
+	err = -ENOMEM;
+	priv = kmem_cache_alloc(kevent_poll_priv_cache, SLAB_KERNEL);
+	if (!priv)
+		goto err_out_fput;
+
+	spin_lock_init(&priv->container_lock);
+	INIT_LIST_HEAD(&priv->container_list);
+
+	k->priv = priv;
+
+	ctl.k = k;
+	init_poll_funcptr(&ctl.pt, &kevent_poll_qproc);
+
+	err = kevent_storage_enqueue(&file->st, k);
+	if (err)
+		goto err_out_free;
+
+	revents = file->f_op->poll(file, &ctl.pt);
+	if (revents & k->event.event) {
+		ready = 1;
+		kevent_poll_dequeue(k);
+	}
+	
+	return ready;
+
+err_out_free:
+	kmem_cache_free(kevent_poll_priv_cache, priv);
+err_out_fput:
+	fput(file);
+	return err;
+}
+
+static int kevent_poll_dequeue(struct kevent *k)
+{
+	struct file *file = k->st->origin;
+	struct kevent_poll_private *priv = k->priv;
+	struct kevent_poll_wait_container *w, *n;
+	unsigned long flags;
+
+	kevent_storage_dequeue(k->st, k);
+
+	spin_lock_irqsave(&priv->container_lock, flags);
+	list_for_each_entry_safe(w, n, &priv->container_list, container_entry) {
+		list_del(&w->container_entry);
+		remove_wait_queue(w->whead, &w->wait);
+		kmem_cache_free(kevent_poll_container_cache, w);
+	}
+	spin_unlock_irqrestore(&priv->container_lock, flags);
+	
+	kmem_cache_free(kevent_poll_priv_cache, priv);
+	k->priv = NULL;
+	
+	fput(file);
+
+	return 0;
+}
+
+static int kevent_poll_callback(struct kevent *k)
+{
+	struct file *file = k->st->origin;
+	unsigned int revents = file->f_op->poll(file, NULL);
+	return (revents & k->event.event);
+}
+
+int kevent_init_poll(struct kevent *k)
+{
+	if (!kevent_poll_container_cache || !kevent_poll_priv_cache)
+		return -ENOMEM;
+
+	k->enqueue = &kevent_poll_enqueue;
+	k->dequeue = &kevent_poll_dequeue;
+	k->callback = &kevent_poll_callback;
+	return 0;
+}
+
+
+static int __init kevent_poll_sys_init(void)
+{
+	kevent_poll_container_cache = kmem_cache_create("kevent_poll_container_cache", 
+			sizeof(struct kevent_poll_wait_container), 0, 0, NULL, NULL);
+	if (!kevent_poll_container_cache) {
+		printk(KERN_ERR "Failed to create kevent poll container cache.\n");
+		return -ENOMEM;
+	}
+	
+	kevent_poll_priv_cache = kmem_cache_create("kevent_poll_priv_cache", 
+			sizeof(struct kevent_poll_private), 0, 0, NULL, NULL);
+	if (!kevent_poll_priv_cache) {
+		printk(KERN_ERR "Failed to create kevent poll private data cache.\n");
+		kmem_cache_destroy(kevent_poll_container_cache);
+		kevent_poll_container_cache = NULL;
+		return -ENOMEM;
+	}
+
+	printk(KERN_INFO "Kevent poll()/select() subsystem has been initialized.\n");
+	return 0;
+}
+
+static void __exit kevent_poll_sys_fini(void)
+{
+	kmem_cache_destroy(kevent_poll_priv_cache);
+	kmem_cache_destroy(kevent_poll_container_cache);
+}
+
+module_init(kevent_poll_sys_init);
+module_exit(kevent_poll_sys_fini);
kernel/kevent/kevent_timer.c new file +112 -0

112 lines not on this page · raw message

Signature · 1 line
	Evgeniy Polyakov