kevent: Core files.
Subsystems (11)
A message that is not in the archive (sent off-list or not yet received): [email protected]
cover kevent: Generic event handling mechanism.
Generic event handling mechanism.
Consider for inclusion.
Changes from 'take20' patchset:
* new ring buffer implementation
* removed artificial limit on possible number of kevents
With this release and fixed userspace web server it was possible to achive 3960+ req/s with client connection rate of 4000 con/s over 100 Mbit lan, data IO over network was about 10582.7 KB/s, which is too close to wire speed if we get into account headers and the like.
Changes from 'take19' patchset:
* use __init instead of __devinit
* removed 'default N' from config for user statistic
* removed kevent_user_fini() since kevent can not be unloaded
* use KERN_INFO for statistic output
Changes from 'take18' patchset:
* use __init instead of __devinit
* removed 'default N' from config for user statistic
* removed kevent_user_fini() since kevent can not be unloaded
* use KERN_INFO for statistic output
Changes from 'take17' patchset:
* Use RB tree instead of hash table.
At least for a web sever, frequency of addition/deletion of new kevent is comparable with number of search access, i.e. most of the time events are added, accesed only couple of times and then removed, so it justifies RB tree usage over AVL tree, since the latter does have much slower deletion time (max O(log(N)) compared to 3 ops),
although faster search time (1.44*O(log(N)) vs. 2*O(log(N))).
So for kevents I use RB tree for now and later, when my AVL tree implementation is ready, it will be possible to compare them.
* Changed readiness check for socket notifications.
With both above changes it is possible to achieve more than 3380 req/second compared to 2200, sometimes 2500 req/second for epoll() for trivial web-server and httperf client on the same hardware.
It is possible that above kevent limit is due to maximum allowed kevents in a time limit, which is 4096 events.
Changes from 'take16' patchset:
* misc cleanups (__read_mostly, const ...)
* created special macro which is used for mmap size (number of pages) calculation
* export kevent_socket_notify(), since it is used in network protocols which can be built as modules (IPv6 for example)
Changes from 'take15' patchset:
* converted kevent_timer to high-resolution timers, this forces timer API update at
http://linux-net.osdl.org/index.php/Kevent
* use struct ukevent* instead of void * in syscalls (documentation has been updated)
* added warning in kevent_add_ukevent() if ring has broken index (for testing)
Changes from 'take14' patchset:
* added kevent_wait()
This syscall waits until either timeout expires or at least one event becomes ready. It also commits that @num events from @start are processed by userspace and thus can be be removed or rearmed (depending on it's flags). It can be used for commit events read by userspace through mmap interface. Example userspace code (evtest.c) can be found on project's homepage.
* added socket notifications (send/recv/accept)
Changes from 'take13' patchset:
* do not get lock aroung user data check in __kevent_search()
* fail early if there were no registered callbacks for given type of kevent
* trailing whitespace cleanup
Changes from 'take12' patchset:
* remove non-chardev interface for initialization
* use pointer to kevent_mring instead of unsigned longs
* use aligned 64bit type in raw user data (can be used by high-res timer if needed)
* simplified enqueue/dequeue callbacks and kevent initialization
* use nanoseconds for timeout
* put number of milliseconds into timer's return data
* move some definitions into user-visible header
* removed filenames from comments
Changes from 'take11' patchset:
* include missing headers into patchset
* some trivial code cleanups (use goto instead of if/else games and so on)
* some whitespace cleanups
* check for ready_callback() callback before main loop which should save us some ticks
Changes from 'take10' patchset:
* removed non-existent prototypes
* added helper function for kevent_registered_callbacks
* fixed 80 lines comments issues
* added shared between userspace and kernelspace header instead of embedd them in one
* core restructuring to remove forward declarations
* s o m e w h i t e s p a c e c o d y n g s t y l e c l e a n u p
* use vm_insert_page() instead of remap_pfn_range()
Changes from 'take9' patchset:
* fixed ->nopage method
Changes from 'take8' patchset:
* fixed mmap release bug
* use module_init() instead of late_initcall()
* use better structures for timer notifications
Changes from 'take7' patchset:
* new mmap interface (not tested, waiting for other changes to be acked)
- use nopage() method to dynamically substitue pages
- allocate new page for events only when new added kevent requres it
- do not use ugly index dereferencing, use structure instead
- reduced amount of data in the ring (id and flags), maximum 12 pages on x86 per kevent fd
Changes from 'take6' patchset:
* a lot of comments!
* do not use list poisoning for detection of the fact, that entry is in the list
* return number of ready kevents even if copy*user() fails
* strict check for number of kevents in syscall
* use ARRAY_SIZE for array size calculation
* changed superblock magic number
* use SLAB_PANIC instead of direct panic() call
* changed -E* return values
* a lot of small cleanups and indent fixes
Changes from 'take5' patchset:
* removed compilation warnings about unused wariables when lockdep is not turned on
* do not use internal socket structures, use appropriate (exported) wrappers instead
* removed default 1 second timeout
* removed AIO stuff from patchset
Changes from 'take4' patchset:
* use miscdevice instead of chardevice
* comments fixes
Changes from 'take3' patchset:
* removed serializing mutex from kevent_user_wait()
* moved storage list processing to RCU
* removed lockdep screaming - all storage locks are initialized in the same function, so it was
learned
to differentiate between various cases
* remove kevent from storage if is marked as broken after callback
* fixed a typo in mmaped buffer implementation which would end up in wrong index calcualtion
Changes from 'take2' patchset:
* split kevent_finish_user() to locked and unlocked variants
* do not use KEVENT_STAT ifdefs, use inline functions instead
* use array of callbacks of each type instead of each kevent callback initialization
* changed name of ukevent guarding lock
* use only one kevent lock in kevent_user for all hash buckets instead of per-bucket locks
* do not use kevent_user_ctl structure instead provide needed arguments as syscall parameters
* various indent cleanups
* added optimisation, which is aimed to help when a lot of kevents are being copied from
userspace
* mapped buffer (initial) implementation (no userspace yet)
Changes from 'take1' patchset:
- rebased against 2.6.18-git tree
- removed ioctl controlling
- added new syscall kevent_get_events(int fd, unsigned int min_nr, unsigned int max_nr, unsigned int timeout, void __user *buf, unsigned flags)
- use old syscall kevent_ctl for creation/removing, modification and initial kevent initialization
- use mutuxes instead of semaphores
- added file descriptor check and return error if provided descriptor does not match kevent file operations
- various indent fixes
- removed aio_sendfile() declarations.
Thank you.
PATCH 1/4 kevent: Core files.
Core files.
This patch includes core kevent files:
* userspace controlling
* kernelspace interfaces
* initialization
* notification state machines
Some bits of documentation can be found on project's homepage (and links from there):
http://tservice.net.ru/~s0mbre/old/?section=projects&item=kevent
arch/i386/kernel/syscall_table.S +3 -0
@@ -318,3 +318,6 @@ ENTRY(sys_call_table) .long sys_vmsplice .long sys_move_pages .long sys_getcpu + .long sys_kevent_get_events + .long sys_kevent_ctl /* 320 */ + .long sys_kevent_wait
arch/x86_64/ia32/ia32entry.S +4 -1
@@ -714,8 +714,11 @@ #endif .quad compat_sys_get_robust_list .quad sys_splice .quad sys_sync_file_range - .quad sys_tee + .quad sys_tee /* 315 */ .quad compat_sys_vmsplice .quad compat_sys_move_pages .quad sys_getcpu + .quad sys_kevent_get_events + .quad sys_kevent_ctl /* 320 */ + .quad sys_kevent_wait ia32_syscall_end:
include/asm-i386/unistd.h +4 -1
@@ -324,10 +324,13 @@ #define __NR_tee 315#define __NR_vmsplice 316#define __NR_move_pages 317#define __NR_getcpu 318 +#define __NR_kevent_get_events 319 +#define __NR_kevent_ctl 320 +#define __NR_kevent_wait 321#ifdef __KERNEL__ -#define NR_syscalls 319 +#define NR_syscalls 322#include <linux/err.h> /*
include/asm-x86_64/unistd.h +7 -1
@@ -619,10 +619,16 @@ #define __NR_vmsplice 278 __SYSCALL(__NR_vmsplice, sys_vmsplice)#define __NR_move_pages 279 __SYSCALL(__NR_move_pages, sys_move_pages) +#define __NR_kevent_get_events 280 +__SYSCALL(__NR_kevent_get_events, sys_kevent_get_events) +#define __NR_kevent_ctl 281 +__SYSCALL(__NR_kevent_ctl, sys_kevent_ctl) +#define __NR_kevent_wait 282 +__SYSCALL(__NR_kevent_wait, sys_kevent_wait)#ifdef __KERNEL__ -#define __NR_syscall_max __NR_move_pages +#define __NR_syscall_max __NR_kevent_wait#include <linux/err.h> #ifndef __NO_STUBS
include/linux/kevent.h new file +205 -0
205 lines not on this page · raw message
include/linux/kevent_storage.h new file +11 -0
@@ -0,0 +1,11 @@ +#ifndef __KEVENT_STORAGE_H +#define __KEVENT_STORAGE_H + +struct kevent_storage +{ +void *origin;/* Originator's pointer, e.g. struct sock or struct file. Can be NULL. */ +struct list_head list;/* List of queued kevents. */ +spinlock_t lock;/* Protects users queue. */ +}; + +#endif /* __KEVENT_STORAGE_H */
include/linux/syscalls.h +5 -0
@@ -54,6 +54,7 @@ struct compat_stat;struct compat_timeval ;struct robust_list_head ;struct getcpu_cache ; +struct ukevent ;#include <linux/types.h> #include <linux/aio_abi.h>
@@ -599,4 +600,8 @@ asmlinkage long sys_set_robust_list(stru size_t len);asmlinkage long sys_getcpu (unsigned __user *cpu,unsigned __user *node,struct getcpu_cache __user *cache); +asmlinkage long sys_kevent_get_events (int ctl_fd,unsigned int min,unsigned int max, +__u64 timeout,struct ukevent __user *buf,unsigned flags); +asmlinkage long sys_kevent_ctl (int ctl_fd,unsigned int cmd,unsigned int num,struct ukevent __user *buf); +asmlinkage long sys_kevent_wait (int ctl_fd,unsigned int start,unsigned int num,__u64 timeout);#endif
include/linux/ukevent.h new file +163 -0
163 lines not on this page · raw message
init/Kconfig +2 -0
@@ -201,6 +201,8 @@ config AUDITSYSCALL such as SELinux. To use audit's filesystem watch feature, please ensure that INOTIFY is configured. +source "kernel/kevent/Kconfig" + config IKCONFIG bool "Kernel .config support" ---help---
kernel/Makefile +1 -0
@@ -47,6 +47,7 @@ obj-$(CONFIG_DETECT_SOFTLOCKUP) += softl obj-$(CONFIG_GENERIC_HARDIRQS) += irq/ obj-$(CONFIG_SECCOMP) += seccomp.o obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o +obj-$(CONFIG_KEVENT) += kevent/ obj-$(CONFIG_RELAY) += relay.o obj-$(CONFIG_TASK_DELAY_ACCT) += delayacct.o obj-$(CONFIG_TASKSTATS) += taskstats.o
kernel/kevent/Kconfig new file +39 -0
@@ -0,0 +1,39 @@ +config KEVENT + bool "Kernel event notification mechanism" + help + This option enables event queue mechanism. + It can be used as replacement for poll()/select(), AIO callback + invocations, advanced timer notifications and other kernel + object status changes. + +config KEVENT_USER_STAT + bool "Kevent user statistic" + depends on KEVENT + help + This option will turn kevent_user statistic collection on. + Statistic data includes total number of kevent, number of kevents + which are ready immediately at insertion time and number of kevents + which were removed through readiness completion. + It will be printed each time control kevent descriptor is closed. + +config KEVENT_TIMER + bool "Kernel event notifications for timers" + depends on KEVENT + help + This option allows to use timers through KEVENT subsystem. + +config KEVENT_POLL + bool "Kernel event notifications for poll()/select()" + depends on KEVENT + help + This option allows to use kevent subsystem for poll()/select() + notifications. + +config KEVENT_SOCKET + bool "Kernel event notifications for sockets" + depends on NET && KEVENT + help + This option enables notifications through KEVENT subsystem of + sockets operations, like new packet receiving conditions, + ready for accept conditions and so on. +
kernel/kevent/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +obj-y := kevent.o kevent_user.o +obj-$(CONFIG_KEVENT_TIMER) += kevent_timer.o +obj-$(CONFIG_KEVENT_POLL) += kevent_poll.o +obj-$(CONFIG_KEVENT_SOCKET) += kevent_socket.o
kernel/kevent/kevent.c new file +227 -0
227 lines not on this page · raw message
kernel/kevent/kevent_user.c new file +1000 -0
1,000 lines not on this page · raw message
kernel/sys_ni.c +4 -0
@@ -122,6 +122,10 @@ cond_syscall(ppc_rtas);cond_syscall (sys_spu_run);cond_syscall (sys_spu_create); +cond_syscall (sys_kevent_get_events); +cond_syscall (sys_kevent_wait); +cond_syscall (sys_kevent_ctl); +/* mmu depending weak syscall entries */ cond_syscall (sys_mprotect);cond_syscall (sys_msync);
PATCH 2/4 kevent: poll/select() notifications.
poll/select() notifications.
This patch includes generic poll/select notifications. kevent_poll works simialr to epoll and has the same issues (callback is invoked not from internal state machine of the caller, but through process awake, a lot of allocations and so on).
include/linux/fs.h +8 -0
@@ -276,6 +276,7 @@ #include <linux/prio_tree.h>#include <linux/init.h> #include <linux/sched.h> #include <linux/mutex.h> +#include <linux/kevent.h> #include <asm/atomic.h> #include <asm/semaphore.h>
@@ -586,6 +587,10 @@ #ifdef CONFIG_INOTIFYstruct mutex inotify_mutex;/* protects the watches list */ #endif +#ifdef CONFIG_KEVENT_SOCKET +struct kevent_storage st; +#endif +unsigned long i_state;unsigned long dirtied_when;/* jiffies of first dirtying */
@@ -739,6 +744,9 @@ #ifdef CONFIG_EPOLLstruct list_head f_ep_links;spinlock_t f_ep_lock;#endif /* #ifdef CONFIG_EPOLL */ +#ifdef CONFIG_KEVENT_POLL +struct kevent_storage st; +#endif struct address_space *f_mapping; };extern spinlock_t files_lock;
kernel/kevent/kevent_poll.c new file +222 -0
222 lines not on this page · raw message
PATCH 3/4 kevent: Socket notifications.
Socket notifications.
This patch includes socket send/recv/accept notifications. Using trivial web server based on kevent and this features instead of epoll it's performance increased more than noticebly. More details about various benchmarks and server itself (evserver_kevent.c) can be found on project's homepage.
fs/inode.c +7 -0
@@ -21,6 +21,7 @@ #include <linux/pagemap.h>#include <linux/cdev.h> #include <linux/bootmem.h> #include <linux/inotify.h> +#include <linux/kevent.h> #include <linux/mount.h> /*
@@ -164,12 +165,18 @@ #endif } inode->i_private =0 ; inode->i_mapping = mapping; +#if definedCONFIG_KEVENT_SOCKET +kevent_storage_init (inode, &inode->st); +#endif }return inode; }void destroy_inode (struct inode *inode) { +#if definedCONFIG_KEVENT_SOCKET +kevent_storage_fini (&inode->st); +#endif BUG_ON (inode_has_buffers (inode));security_inode_free (inode);if (inode->i_sb->s_op->destroy_inode)
include/net/sock.h +17 -15
@@ -48,6 +48,7 @@ #include <linux/lockdep.h>#include <linux/netdevice.h> #include <linux/skbuff.h> /* struct sk_buff */ #include <linux/security.h> +#include <linux/kevent.h> #include <linux/filter.h>
@@ -450,6 +451,21 @@ static inline int sk_stream_memory_free(extern void sk_stream_rfree (struct sk_buff *skb); +struct socket_alloc { +struct socket socket; +struct inode vfs_inode; +}; + +static inline struct socket *SOCKET_I (struct inode *inode) +{ + return &container_of(inode, struct socket_alloc, vfs_inode)->socket; +} + +static inline struct inode *SOCK_INODE (struct socket *socket) +{ + return &container_of(socket, struct socket_alloc, socket)->vfs_inode; +} +static inline void sk_stream_set_owner_r (struct sk_buff *skb,struct sock *sk) { skb->sk = sk;
@@ -477,6 +493,7 @@ static inline void sk_add_backlog(struct sk->sk_backlog.tail = skb; } skb->next =NULL ; +kevent_socket_notify (sk,KEVENT_SOCKET_RECV ); }#define sk_wait_event (__sk, __timeo, __condition) \
@@ -679,21 +696,6 @@ static inline struct kiocb *siocb_to_kioreturn si->kiocb; } -struct socket_alloc { -struct socket socket; -struct inode vfs_inode; -}; - -static inline struct socket *SOCKET_I (struct inode *inode) -{ - return &container_of(inode, struct socket_alloc, vfs_inode)->socket; -} - -static inline struct inode *SOCK_INODE (struct socket *socket) -{ - return &container_of(socket, struct socket_alloc, socket)->vfs_inode; -} -extern void __sk_stream_mem_reclaim (struct sock *sk);extern int sk_stream_mem_schedule (struct sock *sk,int size,int kind);
include/net/tcp.h +1 -0
@@ -857,6 +857,7 @@ static inline int tcp_prequeue(struct so tp->ucopy.memory =0 ; }else if (skb_queue_len (&tp->ucopy.prequeue) ==1 ) {wake_up_interruptible (sk->sk_sleep); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND );if (!inet_csk_ack_scheduled (sk)) inet_csk_reset_xmit_timer(sk, ICSK_TIME_DACK, (3 * TCP_RTO_MIN) / 4,
kernel/kevent/kevent_socket.c new file +129 -0
129 lines not on this page · raw message
net/core/sock.c +9 -1
@@ -1402,6 +1402,7 @@ static void sock_def_wakeup(struct sockif (sk->sk_sleep &&waitqueue_active (sk->sk_sleep))wake_up_interruptible_all (sk->sk_sleep);read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_error_report (struct sock *sk)
@@ -1411,6 +1412,7 @@ static void sock_def_error_report(structwake_up_interruptible (sk->sk_sleep);sk_wake_async (sk,0 ,POLL_ERR );read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_readable (struct sock *sk,int len)
@@ -1420,6 +1422,7 @@ static void sock_def_readable(struct socwake_up_interruptible (sk->sk_sleep);sk_wake_async (sk,1 ,POLL_IN );read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_write_space (struct sock *sk)
@@ -1439,6 +1442,7 @@ static void sock_def_write_space(struct }read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_SEND |KEVENT_SOCKET_RECV ); }static void sock_def_destruct (struct sock *sk)
@@ -1489,6 +1493,8 @@ #endif sk->sk_state =TCP_CLOSE ; sk->sk_socket = sock; +kevent_sk_reinit (sk); +sock_set_flag (sk,SOCK_ZAPPED );if (sock)
@@ -1555,8 +1561,10 @@ void fastcall release_sock(struct sock *if (sk->sk_backlog.tail)__release_sock (sk); sk->sk_lock.owner =NULL ; -if (waitqueue_active (&sk->sk_lock.wq)) +if (waitqueue_active (&sk->sk_lock.wq)) {wake_up (&sk->sk_lock.wq); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); + }spin_unlock_bh (&sk->sk_lock.slock); }EXPORT_SYMBOL (release_sock);
net/core/stream.c +1 -0
@@ -36,6 +36,7 @@ void sk_stream_write_space(struct sock *wake_up_interruptible (sk->sk_sleep);if (sock->fasync_list && !(sk->sk_shutdown &SEND_SHUTDOWN ))sock_wake_async (sock,2 ,POLL_OUT ); +kevent_socket_notify (sk,KEVENT_SOCKET_SEND |KEVENT_SOCKET_RECV ); } }
net/ipv4/tcp_input.c +1 -0
@@ -3119,6 +3119,7 @@ static void tcp_ofo_queue(struct sock *s__skb_unlink (skb, &tp->out_of_order_queue);__skb_queue_tail (&sk->sk_receive_queue, skb); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV ); tp->rcv_nxt =TCP_SKB_CB (skb)->end_seq;if (skb->h.th->fin)tcp_fin (skb, sk, skb->h.th);
net/ipv4/tcp_ipv4.c +2 -0
@@ -61,6 +61,7 @@ #include <linux/cache.h>#include <linux/jhash.h> #include <linux/init.h> #include <linux/times.h> +#include <linux/kevent.h> #include <net/icmp.h> #include <net/inet_hashtables.h>
@@ -870,6 +871,7 @@ #endifreqsk_free (req); }else {inet_csk_reqsk_queue_hash_add (sk, req,TCP_TIMEOUT_INIT ); +kevent_socket_notify (sk,KEVENT_SOCKET_ACCEPT ); }return 0 ;
net/socket.c +3 -0
@@ -85,6 +85,7 @@ #include <linux/compat.h>#include <linux/kmod.h> #include <linux/audit.h> #include <linux/wireless.h> +#include <linux/kevent.h> #include <asm/uaccess.h> #include <asm/unistd.h>
@@ -490,6 +491,8 @@ static struct socket *sock_alloc(void) inode->i_uid = current->fsuid; inode->i_gid = current->fsgid; +kevent_socket_reinit (sock); +get_cpu_var (sockets_in_use)++;put_cpu_var (sockets_in_use);return sock;
PATCH 4/4 kevent: Timer notifications.
Timer notifications.
Timer notifications can be used for fine grained per-process time management, since interval timers are very inconvenient to use, and they are limited.
This subsystem uses high-resolution timers.
id.raw[0] is used as number of seconds
id.raw[1] is used as number of nanoseconds
kernel/kevent/kevent_timer.c new file +113 -0
@@ -0,0 +1,113 @@ +/* +* 2006 Copyright (c) Evgeniy Polyakov <[email protected]> +* All rights reserved. +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +*/ + +#include <linux/kernel.h> +#include <linux/types.h> +#include <linux/list.h> +#include <linux/slab.h> +#include <linux/spinlock.h> +#include <linux/hrtimer.h> +#include <linux/jiffies.h> +#include <linux/kevent.h> + +struct kevent_timer +{ +struct hrtimer ktimer; +struct kevent_storage ktimer_storage; +struct kevent *ktimer_event; +}; + +static int kevent_timer_func (struct hrtimer *timer) +{ + struct kevent_timer *t = container_of(timer, struct kevent_timer, ktimer); +struct kevent *k = t->ktimer_event; + +kevent_storage_ready (&t->ktimer_storage,NULL ,KEVENT_MASK_ALL ); +hrtimer_forward (timer, timer->base->softirq_time, +ktime_set (k->event.id.raw[0 ], k->event.id.raw[1 ])); +return HRTIMER_RESTART ; +} + +static struct lock_class_key kevent_timer_key; + +static int kevent_timer_enqueue (struct kevent *k) +{ +int err; +struct kevent_timer *t; + + t =kmalloc (sizeof (struct kevent_timer ),GFP_KERNEL ); +if (!t) +return -ENOMEM ; + +hrtimer_init (&t->ktimer,CLOCK_MONOTONIC ,HRTIMER_REL ); + t->ktimer.expires =ktime_set (k->event.id.raw[0 ], k->event.id.raw[1 ]); + t->ktimer.function = kevent_timer_func; + t->ktimer_event = k; + + err =kevent_storage_init (&t->ktimer, &t->ktimer_storage); +if (err) + goto err_out_free; +lockdep_set_class (&t->ktimer_storage.lock, &kevent_timer_key); + + err =kevent_storage_enqueue (&t->ktimer_storage, k); +if (err) + goto err_out_st_fini; + +printk ("%s: jiffies: %lu, timer: %p.\n" , __func__, jiffies, &t->ktimer); +hrtimer_start (&t->ktimer, t->ktimer.expires,HRTIMER_REL ); + +return 0 ; + +err_out_st_fini: +kevent_storage_fini (&t->ktimer_storage); +err_out_free: +kfree (t); + +return err; +} + +static int kevent_timer_dequeue (struct kevent *k) +{ +struct kevent_storage *st = k->st; + struct kevent_timer *t = container_of(st, struct kevent_timer, ktimer_storage); + +hrtimer_cancel (&t->ktimer); +kevent_storage_dequeue (st, k); +kfree (t); + +return 0 ; +} + +static int kevent_timer_callback (struct kevent *k) +{ + k->event.ret_data[0 ] =jiffies_to_msecs (jiffies); +return 1 ; +} + +static int __init kevent_init_timer (void ) +{ +struct kevent_callbacks tc = { + .callback = &kevent_timer_callback, + .enqueue = &kevent_timer_enqueue, + .dequeue = &kevent_timer_dequeue}; + +return kevent_add_callbacks (&tc,KEVENT_TIMER ); +} +module_init (kevent_init_timer); +
Re: [take21 2/4] kevent: poll/select() notifications.
In reply to Evgeniy Polyakovkernel/kevent/kevent_poll.c line 106
Quoted 3 lines · Evgeniy Polyakov
kernel/kevent/kevent_poll.c new file +3
-0+ file =fget (k->event.id.raw[0 ]); +if (!file) +return -ENODEV ;
Please, do us a favor, and use EBADF instead of ENODEV.
EBADF : /* Bad file number */
ENODEV : /* No such device */
You have many ENODEV uses in your patches and that really hurts.
Eric
Re: [take21 2/4] kevent: poll/select() notifications.
Quoted 8 lines · Eric Dumazet
kernel/kevent/kevent_poll.c line 106
Quoted 3 lines · Evgeniy Polyakov
kernel/kevent/kevent_poll.c new file +3
-0+ file =fget (k->event.id.raw[0 ]); +if (!file) +return -ENODEV ;Please, do us a favor, and use EBADF instead of ENODEV.
Quoted 5 lines
EBADF : /* Bad file number */
ENODEV : /* No such device */
You have many ENODEV uses in your patches and that really hurts.
Ok :)
Quoted 1 line
Eric
Signature · 1 line
Evgeniy Polyakov
+/*
+ * Called under kevent_user->ready_lock, so updates are always protected.
+ */
+int kevent_user_ring_add_event(struct kevent *k)
+{
+ unsigned int pidx, off;
+ struct kevent_mring *ring, *copy_ring;
+
+ ring = k->user->pring[0];
+
+ if ((ring->kidx + 1 == ring->uidx) ||
+ ((ring->kidx + 1 == KEVENT_MAX_EVENTS) && ring->uidx == 0)) {
+ if (k->user->overflow_kevent == NULL)
+ k->user->overflow_kevent = k;
+ return -EAGAIN;
+ }
+
I really dont understand how you manage to queue multiple kevents in the 'overflow list'. You just queue one kevent at most. What am I missing ?
kernel/kevent/kevent_user.c line 115
Quoted 9 lines
kernel/kevent/kevent_user.c new file +9
-0+ +for (i=0 ; i<KEVENT_MAX_PAGES ; ++i) { + u->pring[i] = (struct kevent_mring *)__get_free_page (GFP_KERNEL ); +if (!u->pring[i]) +break ; + } + +if (i !=KEVENT_MAX_PAGES ) + goto err_out_free;
Why dont you use goto directly ?
if (!u->pring[i]) goto err_out_free;
Quoted 32 lines
kernel/kevent/kevent_user.c new file +28
-0+ + u->pring[0 ]->uidx = u->pring[0 ]->kidx =0 ; + +return 0 ; + +err_out_free: +for (i=0 ; i<KEVENT_MAX_PAGES ; ++i) { +if (!u->pring[i]) +break ; + +free_page ((unsigned long )u->pring[i]); + } +return k; +} + +static int kevent_user_ctl_add (struct kevent_user *u,unsigned int num,void __user *arg) +{ +int err, cerr =0 , knum =0 , rnum =0 , i; +void __user *orig = arg; +struct ukevent uk; + +mutex_lock (&u->ctl_mutex); + + err = -EINVAL ; +if (num >KEVENT_MIN_BUFFS_ALLOC ) { +struct ukevent *ukev; + + ukev =kevent_get_user (num, arg);
kernel/kevent/kevent_user.c line 685
Quoted 10 lines
kernel/kevent/kevent_user.c new file +10
-0+if (ukev) { +for (i =0 ; i < num; ++i) { + err =kevent_user_add_ukevent (&ukev[i], u); +if (err) { +kevent_stat_im (u); +if (i != rnum) +memcpy (&ukev[rnum], &ukev[i],sizeof (struct ukevent )); + rnum++; + }else + knum++;
Why are you using/counting knum ?
Quoted 40 lines
kernel/kevent/kevent_user.c new file +40
-0+ } +if (copy_to_user (orig, ukev, rnum*sizeof (struct ukevent ))) + cerr = -EFAULT ; +kfree (ukev); + goto out_setup; + } + } + +for (i =0 ; i < num; ++i) { +if (copy_from_user (&uk, arg,sizeof (struct ukevent ))) { + cerr = -EFAULT ; +break ; + } + arg +=sizeof (struct ukevent ); + + err =kevent_user_add_ukevent (&uk, u); +if (err) { +kevent_stat_im (u); +if (copy_to_user (orig, &uk,sizeof (struct ukevent ))) { + cerr = -EFAULT ; +break ; + } + orig +=sizeof (struct ukevent ); + rnum++; + }else + knum++; + } + +out_setup: +if (cerr <0 ) { + err = cerr; + goto out_remove; + } + + err = rnum; +out_remove: +mutex_unlock (&u->ctl_mutex); + +return err; +}
Quoted 20 lines · Eric Dumazet
kernel/kevent/kevent_user.c new file +12
-0+/* +* Called under kevent_user->ready_lock, so updates are always protected. +*/ +int kevent_user_ring_add_event (struct kevent *k) +{ +unsigned int pidx, off; +struct kevent_mring *ring, *copy_ring; + + ring = k->user->pring[0 ]; + + if ((ring->kidx + 1 == ring->uidx) || + ((ring->kidx + 1 == KEVENT_MAX_EVENTS) && ring->uidx== 0)) {
kernel/kevent/kevent_user.c new file +5
-0+if (k->user->overflow_kevent ==NULL ) + k->user->overflow_kevent = k; +return -EAGAIN ; + } +
Quoted 2 lines
I really dont understand how you manage to queue multiple kevents in the 'overflow list'. You just queue one kevent at most. What am I missing ?
There is no overflow list - it is a pointer to the first kevent in the ready queue, which was not put into ring buffer. It is an optimisation, which allows to not search for that position each time new event should be placed into the buffer, when it starts to have an empty slot.
Quoted 14 lines
kernel/kevent/kevent_user.c line 115
Quoted 11 lines
kernel/kevent/kevent_user.c new file +3
-0+ +for (i=0 ; i<KEVENT_MAX_PAGES ; ++i) { + u->pring[i] = (struct kevent_mring*)__get_free_page(GFP_KERNEL);
kernel/kevent/kevent_user.c new file +6
-0+if (!u->pring[i]) +break ; + } + +if (i !=KEVENT_MAX_PAGES ) + goto err_out_free;Why dont you use goto directly ?
Quoted 3 lines
if (!u->pring[i]) goto err_out_free;
I used a fallback mode here which allowed to use smaller number of pages for kevent ring buffer, but then decided to drop it.
So it is possible to use goto directly.
Quoted 39 lines
Quoted 39 lines
kernel/kevent/kevent_user.c new file +16
-0+ + u->pring[0 ]->uidx = u->pring[0 ]->kidx =0 ; + +return 0 ; + +err_out_free: +for (i=0 ; i<KEVENT_MAX_PAGES ; ++i) { +if (!u->pring[i]) +break ; + +free_page ((unsigned long )u->pring[i]); + } +return k; +} + +static int kevent_user_ctl_add(struct kevent_user *u, unsigned int num,void
__user*arg)kernel/kevent/kevent_user.c new file +18
-0+{ +int err, cerr =0 , knum =0 , rnum =0 , i; +void __user *orig = arg; +struct ukevent uk; + +mutex_lock (&u->ctl_mutex); + + err = -EINVAL ; +if (num >KEVENT_MIN_BUFFS_ALLOC ) { +struct ukevent *ukev; + + ukev =kevent_get_user (num, arg); +if (ukev) { +for (i =0 ; i < num; ++i) { + err =kevent_user_add_ukevent (&ukev[i], u); +if (err) { +kevent_stat_im (u); +if (i != rnum)
Quoted 8 lines
kernel/kevent/kevent_user.c line 685
Quoted 5 lines
(quoted diff) +1
-0+ memcpy(&ukev[rnum],&ukev[i], sizeof(struct ukevent));
kernel/kevent/kevent_user.c new file +3
-0+ rnum++; + }else + knum++;Why are you using/counting knum ?
It should go avay.
Quoted 46 lines
Quoted 32 lines
(quoted diff) +2
-0+ } + if (copy_to_user(orig, ukev, rnum*sizeof(structukevent)))
kernel/kevent/kevent_user.c new file +17
-0+ cerr = -EFAULT ; +kfree (ukev); + goto out_setup; + } + } + +for (i =0 ; i < num; ++i) { +if (copy_from_user (&uk, arg,sizeof (struct ukevent ))) { + cerr = -EFAULT ; +break ; + } + arg +=sizeof (struct ukevent ); + + err =kevent_user_add_ukevent (&uk, u); +if (err) { +kevent_stat_im (u); +if (copy_to_user (orig, &uk,sizeof (struct ukevent ))){
kernel/kevent/kevent_user.c new file +11
-0+ cerr = -EFAULT ; +break ; + } + orig +=sizeof (struct ukevent ); + rnum++; + }else + knum++; + } + +out_setup: +if (cerr <0 ) {kernel/kevent/kevent_user.c line 725
Quoted 10 lines
kernel/kevent/kevent_user.c new file +10
-0+ err = cerr; + goto out_remove; + } + + err = rnum; +out_remove: +mutex_unlock (&u->ctl_mutex); + +return err; +}-
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 1 line
Evgeniy Polyakov
Evgeniy Polyakov a e'crit :
Quoted 6 lines · Eric Dumazet
Quoted 2 lines
I really dont understand how you manage to queue multiple kevents in the 'overflow list'. You just queue one kevent at most. What am I missing ?
There is no overflow list - it is a pointer to the first kevent in the
ready queue, which was not put into ring buffer. It is an optimisation, which allows to not search for that position each time new event should be placed into the buffer, when it starts to have an empty slot.
This overflow list (you may call it differently, but still it IS a list), is not complete. I feel you add it just to make me happy, but I am not (yet :) )
For example, you make no test at kevent_finish_user_complete() time.
Obviously, you can have a dangling pointer, and crash your box in certain conditions.
static void kevent_finish_user_complete(struct kevent *k, int deq)
{
struct kevent_user *u = k->user;
unsigned long flags;
if (deq)
kevent_dequeue(k);
spin_lock_irqsave(&u->ready_lock, flags);
if (k->flags & KEVENT_READY) {
+ if (u->overflow_event == k) {
+ /* MUST do something to change u->overflow_kevent */
+ }
list_del(&k->ready_entry);
k->flags &= ~KEVENT_READY;
u->ready_num--;
}
spin_unlock_irqrestore(&u->ready_lock, flags);
kevent_user_put(u);
call_rcu(&k->rcu_head, kevent_free_rcu);
}Eric
Quoted 7 lines · Eric Dumazet
Evgeniy Polyakov a e'crit :
Quoted 6 lines
Quoted 3 lines · Eric Dumazet
I really dont understand how you manage to queue multiple kevents in the 'overflow list'. You just queue one kevent at most. What am I missing ?
Quoted 8 lines
Quoted 4 lines
There is no overflow list - it is a pointer to the first kevent in the ready queue, which was not put into ring buffer. It is an optimisation, which allows to not search for that position each time new event should be placed into the buffer, when it starts to have an empty slot.
This overflow list (you may call it differently, but still it IS a list), is not complete. I feel you add it just to make me happy, but I am not (yet :) )
There is no overflow list.
There is ready queue, part of which (first several entries) is copied into the ring buffer, overflow_kevent is a pointer to the first kevent which was not copied.
Quoted 4 lines
For example, you make no test at
kevent_finish_user_complete()time.Obviously, you can have a dangling pointer, and crash your box in certain conditions.
You are right, I did not put overflow_kevent check into all places which can remove kevent.
Here is a patch I am about to commit into the kevent tree:
kernel/kevent/kevent_user.c +33 -24
@@ -235,6 +235,36 @@ static void kevent_free_rcu(struct rcu_h }/* +* Must be called under u->ready_lock. +* This function removes kevent from ready queue and +* tries to add new kevent into ring buffer. +*/ +static void kevent_remove_ready (struct kevent *k) +{ +struct kevent_user *u = k->user; + +list_del (&k->ready_entry); + k->flags &= ~KEVENT_READY ; + u->ready_num--; +if (++u->pring[0 ]->uidx ==KEVENT_MAX_EVENTS ) + u->pring[0 ]->uidx =0 ; + +if (u->overflow_kevent) { +int err; + + err =kevent_user_ring_add_event (u->overflow_kevent); +if (!err || u->overflow_kevent == k) { +if (u->overflow_kevent->ready_entry.next == &u->ready_list) + u->overflow_kevent =NULL ; +else + u->overflow_kevent = +list_entry (u->overflow_kevent->ready_entry.next, + struct kevent, ready_entry); + } + } +} + +/* * Complete kevent removing - it dequeues kevent from storage list * if it is requested, removes kevent from ready list, drops userspace * control block reference counter and schedules kevent freeing through RCU.
@@ -248,11 +278,8 @@ static void kevent_finish_user_complete(kevent_dequeue (k);spin_lock_irqsave (&u->ready_lock, flags); -if (k->flags &KEVENT_READY ) { -list_del (&k->ready_entry); - k->flags &= ~KEVENT_READY ; - u->ready_num--; - } +if (k->flags &KEVENT_READY ) +kevent_remove_ready (k);spin_unlock_irqrestore (&u->ready_lock, flags);kevent_user_put (u);
@@ -303,25 +330,7 @@ static struct kevent *kqueue_dequeue_reaspin_lock_irqsave (&u->ready_lock, flags);if (u->ready_num && !list_empty (&u->ready_list)) { k = list_entry(u->ready_list.next, struct kevent, ready_entry); -list_del (&k->ready_entry); - k->flags &= ~KEVENT_READY ; - u->ready_num--; -if (++u->pring[0 ]->uidx ==KEVENT_MAX_EVENTS ) - u->pring[0 ]->uidx =0 ; - -if (u->overflow_kevent) { -int err; - - err =kevent_user_ring_add_event (u->overflow_kevent); -if (!err) { -if (u->overflow_kevent->ready_entry.next == &u->ready_list) - u->overflow_kevent =NULL ; -else - u->overflow_kevent = -list_entry (u->overflow_kevent->ready_entry.next, - struct kevent, ready_entry); - } - } +kevent_remove_ready (k); }spin_unlock_irqrestore (&u->ready_lock, flags);
It tries to put next kevent into the ring and thus update overflow_kevent if new kevent has been put into the
buffer or kevent being removed is overflow kevent.
Patch depends on committed changes of returned error numbers and unused variables cleanup, it will be included into next patchset if there are no problems with it.
Signature · 1 line
Evgeniy Polyakov
Evgeniy Polyakov a e'crit :
Quoted 29 lines · Eric Dumazet
Quoted 6 lines
Evgeniy Polyakov a e'crit :
Quoted 4 lines · Eric Dumazet
Quoted 1 line
I really dont understand how you manage to queue multiple kevents in the 'overflow list'. You just queue one kevent at most. What am I missing ?
There is no overflow list - it is a pointer to the first kevent in the
ready queue, which was not put into ring buffer. It is an optimisation, which allows to not search for that position each time new event should be placed into the buffer, when it starts to have an empty slot.This overflow list (you may call it differently, but still it IS a list), is not complete. I feel you add it just to make me happy, but I am not (yet :) )
There is no overflow list.
There is ready queue, part of which (first several entries) is copied into the ring buffer,overflow_keventis a pointer to the first kevent which was not copied.Quoted 3 lines
For example, you make no test at
kevent_finish_user_complete()time.Obviously, you can have a dangling pointer, and crash your box in certain conditions.
You are right, I did not put
overflow_keventcheck into all places which can remove kevent.Here is a patch I am about to commit into the kevent tree:
kernel/kevent/kevent_user.c +0
-0@@ -235,6 +235,36 @@ static void kevent_free_rcu(struct rcu_h }
kernel/kevent/kevent_user.c static void kevent_free_rcu(struct rcu_h · line 246
Quoted 10 lines
kernel/kevent/kevent_user.c +9
-0/* +* Must be called under u->ready_lock. +* This function removes kevent from ready queue and +* tries to add new kevent into ring buffer. +*/ +static void kevent_remove_ready (struct kevent *k) +{ +struct kevent_user *u = k->user; + +list_del (&k->ready_entry);
Arg... no
You cannot call list_del() , then check overflow_kevent.
I you call list_del on what happens to be the kevent pointed by overflow_kevent, you loose...
Quoted 65 lines
kernel/kevent/kevent_user.c +24
-24+ k->flags &= ~KEVENT_READY ; + u->ready_num--; +if (++u->pring[0 ]->uidx ==KEVENT_MAX_EVENTS ) + u->pring[0 ]->uidx =0 ; + +if (u->overflow_kevent) { +int err; + + err =kevent_user_ring_add_event (u->overflow_kevent); +if (!err || u->overflow_kevent == k) { +if (u->overflow_kevent->ready_entry.next == &u->ready_list) + u->overflow_kevent =NULL ; +else + u->overflow_kevent = +list_entry (u->overflow_kevent->ready_entry.next, + struct kevent, ready_entry); + } + } +} + +/* * Complete kevent removing - it dequeues kevent from storage list * if it is requested, removes kevent from ready list, drops userspace * control block reference counter and schedules kevent freeing through RCU. @@ -248,11 +278,8 @@ static void kevent_finish_user_complete(kevent_dequeue (k);spin_lock_irqsave (&u->ready_lock, flags); -if (k->flags &KEVENT_READY ) { -list_del (&k->ready_entry); - k->flags &= ~KEVENT_READY ; - u->ready_num--; - } +if (k->flags &KEVENT_READY ) +kevent_remove_ready (k);spin_unlock_irqrestore (&u->ready_lock, flags);kevent_user_put (u);@@ -303,25 +330,7 @@ static struct kevent *kqueue_dequeue_reaspin_lock_irqsave (&u->ready_lock, flags);if (u->ready_num && !list_empty (&u->ready_list)) { k = list_entry(u->ready_list.next, struct kevent, ready_entry); -list_del (&k->ready_entry); - k->flags &= ~KEVENT_READY ; - u->ready_num--; -if (++u->pring[0 ]->uidx ==KEVENT_MAX_EVENTS ) - u->pring[0 ]->uidx =0 ; - -if (u->overflow_kevent) { -int err; - - err =kevent_user_ring_add_event (u->overflow_kevent); -if (!err) { -if (u->overflow_kevent->ready_entry.next == &u->ready_list) - u->overflow_kevent =NULL ; -else - u->overflow_kevent = -list_entry (u->overflow_kevent->ready_entry.next, - struct kevent, ready_entry); - } - } +kevent_remove_ready (k); }spin_unlock_irqrestore (&u->ready_lock, flags);
Quoted 20 lines · Eric Dumazet
Quoted 7 lines
kernel/kevent/kevent_user.c +0
-0@@ -235,6 +235,36 @@ static void kevent_free_rcu(struct rcu_h }kernel/kevent/kevent_user.c static void kevent_free_rcu(struct rcu_h · line 246
Quoted 10 lines
kernel/kevent/kevent_user.c +9
-0/* +* Must be called under u->ready_lock. +* This function removes kevent from ready queue and +* tries to add new kevent into ring buffer. +*/ +static void kevent_remove_ready (struct kevent *k) +{ +struct kevent_user *u = k->user; + +list_del (&k->ready_entry);Arg... no
Quoted 4 lines
You cannot call
list_del(), then checkoverflow_kevent.I you call
list_delon what happens to be the kevent pointed byoverflow_kevent, you loose...
This function is always called from appropriate context, where it is guaranteed that it is safe to call list_del:
1. when kevent is removed. It is called after check, that given kevent is in the ready queue.
2. when dequeued from ready queue, which means that it can be removed from that queue.
Signature · 1 line
Evgeniy Polyakov
Evgeniy Polyakov a e'crit :
Quoted 22 lines · Eric Dumazet
Quoted 21 lines
Quoted 7 lines
kernel/kevent/kevent_user.c +0
-0@@ -235,6 +235,36 @@ static void kevent_free_rcu(struct rcu_h }kernel/kevent/kevent_user.c static void kevent_free_rcu(struct rcu_h · line 246
Quoted 10 lines
kernel/kevent/kevent_user.c +9
-0/* +* Must be called under u->ready_lock. +* This function removes kevent from ready queue and +* tries to add new kevent into ring buffer. +*/ +static void kevent_remove_ready (struct kevent *k) +{ +struct kevent_user *u = k->user; + +list_del (&k->ready_entry);Arg... no
You cannot call
list_del(), then checkoverflow_kevent.
Quoted 8 lines
Quoted 1 line
I you call
list_delon what happens to be the kevent pointed byoverflow_kevent, you loose...This function is always called from appropriate context, where it is guaranteed that it is safe to call
list_del:
1. when kevent is removed. It is called after check, that given kevent is in the ready queue.
2. when dequeued from ready queue, which means that it can be removed
from that queue.
Could you please check the list_del() function ?
file include/linux/list.h
static inline void list_del(struct list_head *entry) {
__list_del(entry->prev, entry->next); entry->next = LIST_POISON1;
entry->prev = LIST_POISON2;
}
So, after calling list_del(&k->read_entry);
next and prev are basically destroyed.
So when you write later :
+ if (!err || u->overflow_kevent == k) {
+ if (u->overflow_kevent->ready_entry.next == &u->ready_list)
+ u->overflow_kevent = NULL;
+ else
+ u->overflow_kevent = + list_entry(u->overflow_kevent->ready_entry.next, + struct kevent, ready_entry);
+ }
then you have a problem, since
list_entry(k->ready_entry.next, struct kevent, ready_entry);
will give you garbage.
Eric
Quoted 42 lines · Eric Dumazet
Quoted 8 lines
Quoted 7 lines
Review
Quoted 1 line
(quoted diff) +1
-0+ list_del(&k->ready_entry);Arg... no
You cannot call
list_del(), then checkoverflow_kevent.I you call
list_delon what happens to be the kevent pointed byoverflow_kevent, you loose...Quoted 7 lines
This function is always called from appropriate context, where it is guaranteed that it is safe to call
list_del:
1. when kevent is removed. It is called after check, that given kevent is in the ready queue.
2. when dequeued from ready queue, which means that it can be removed from that queue.Could you please check the
list_del()function ?file
include/linux/list.hstatic inline void
list_del(struct list_head *entry){
__list_del(entry->prev, entry->next);entry->next=LIST_POISON1;
entry->prev=LIST_POISON2;
}So, after calling
list_del(&k->read_entry);
next and prev are basically destroyed.So when you write later :
(quoted diff) +5
-0+ if (!err || u->overflow_kevent == k) { + if (u->overflow_kevent->ready_entry.next == &u->ready_list) + u->overflow_kevent = NULL; + else + u->overflow_kevent = +
list_entry(u->overflow_kevent->ready_entry.next, + struct kevent, ready_entry);
+ }
Quoted 5 lines
then you have a problem, since
list_entry(k->ready_entry.next, struct kevent, ready_entry);will give you garbage.
Ok, I understand you now.
To remove this issue we can delete entry from the list after all checks with overflow_kevent pointer are completed, i.e. have something like this:
kernel/kevent/kevent_user.c +33 -24
@@ -235,6 +235,36 @@ static void kevent_free_rcu(struct rcu_h }/* +* Must be called under u->ready_lock. +* This function removes kevent from ready queue and +* tries to add new kevent into ring buffer. +*/ +static void kevent_remove_ready (struct kevent *k) +{ +struct kevent_user *u = k->user; + +if (++u->pring[0 ]->uidx ==KEVENT_MAX_EVENTS ) + u->pring[0 ]->uidx =0 ; + +if (u->overflow_kevent) { +int err; + + err =kevent_user_ring_add_event (u->overflow_kevent); +if (!err || u->overflow_kevent == k) { +if (u->overflow_kevent->ready_entry.next == &u->ready_list) + u->overflow_kevent =NULL ; +else + u->overflow_kevent = +list_entry (u->overflow_kevent->ready_entry.next, + struct kevent, ready_entry); + } + } +list_del (&k->ready_entry); + k->flags &= ~KEVENT_READY ; + u->ready_num--; +} + +/* * Complete kevent removing - it dequeues kevent from storage list * if it is requested, removes kevent from ready list, drops userspace * control block reference counter and schedules kevent freeing through RCU.
@@ -248,11 +278,8 @@ static void kevent_finish_user_complete(kevent_dequeue (k);spin_lock_irqsave (&u->ready_lock, flags); -if (k->flags &KEVENT_READY ) { -list_del (&k->ready_entry); - k->flags &= ~KEVENT_READY ; - u->ready_num--; - } +if (k->flags &KEVENT_READY ) +kevent_remove_ready (k);spin_unlock_irqrestore (&u->ready_lock, flags);kevent_user_put (u);
@@ -303,25 +330,7 @@ static struct kevent *kqueue_dequeue_reaspin_lock_irqsave (&u->ready_lock, flags);if (u->ready_num && !list_empty (&u->ready_list)) { k = list_entry(u->ready_list.next, struct kevent, ready_entry); -list_del (&k->ready_entry); - k->flags &= ~KEVENT_READY ; - u->ready_num--; -if (++u->pring[0 ]->uidx ==KEVENT_MAX_EVENTS ) - u->pring[0 ]->uidx =0 ; - -if (u->overflow_kevent) { -int err; - - err =kevent_user_ring_add_event (u->overflow_kevent); -if (!err) { -if (u->overflow_kevent->ready_entry.next == &u->ready_list) - u->overflow_kevent =NULL ; -else - u->overflow_kevent = -list_entry (u->overflow_kevent->ready_entry.next, - struct kevent, ready_entry); - } - } +kevent_remove_ready (k); }spin_unlock_irqrestore (&u->ready_lock, flags);
Thanks.
Quoted 1 line
Eric
Signature · 1 line
Evgeniy Polyakov
Re: [take21 0/4] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovQuoted 7 lines · Evgeniy Polyakov
Generic event handling mechanism.
Consider for inclusion.
Changes from 'take20' patchset:
* new ring buffer implementation
Test userspace application can be found in archive on project's homepage. It is also attached to this mail.
Short design notes about ring buffer implementation.
Ring buffer is designed in a way that first ready kevent will be at ring->uidx position, and all other ready events will be in FIFO order after it. So when we need to commit num events, it means we should just remove first num kevents from ready queue and commit them. We do not use any special locking to protect this function against simultaneous running - kevent dequeueing is atomic, and we do not care about order in which events were committed.
An example: thread 1 and thread 2 simultaneously call kevent_wait() to commit 2 and 3 events. It is possible that first thread will commit events 0 and 2 while second thread will commit events 1, 3 and 4. If there were only 3 ready events, then one of the calls will return lesser number of committed events than it was requested.ring->uidx update is atomic, since it is protected by u->ready_lock, which removes race with kevent_user_ring_add_event().
If user asks to commit events which have beed removed by kevent_get_events() recently (for example when one thread looked into ring indexes and started to commit evets, which were simultaneously committed by other thread through kevent_get_events(), kevent_wait() will not commit unprocessed events, but will return number of actually committed events instead.
It is forbidden to try to commit events not from the start of the buffer, but from some 'futher' event.
An example: if ready events use positions 2-5, it is permitted to start to commit 3 events from position 0, in this case 0 and 1 positions will be ommited and only event in position 2 will be committed and kevent_wait() will return 1, since only one event was actually committed.
It is forbidden to try to commit from position 4, 0 will be returned. This means that if some events were committed using kevent_get_events(), they will not be counted, instead userspace should check ring index and try to commit again.
Signature · 1 line
Evgeniy Polyakov
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <sys/time.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <time.h>
#include <unistd.h>
#include <linux/unistd.h>
#include <linux/types.h>
#define PAGE_SIZE 4096
#include <linux/ukevent.h>
#define _syscall3(type,name,type1,arg1,type2,arg2,type3,arg3) \
type name (type1 arg1, type2 arg2, type3 arg3) \
{\
return syscall(__NR_##name, arg1, arg2, arg3);\
}#define _syscall4(type,name,type1,arg1,type2,arg2,type3,arg3,type4,arg4) \
type name (type1 arg1, type2 arg2, type3 arg3, type4 arg4) \
{\
return syscall(__NR_##name, arg1, arg2, arg3, arg4);\
}#define _syscall5(type,name,type1,arg1,type2,arg2,type3,arg3,type4,arg4, \
type5,arg5) \
type name (type1 arg1,type2 arg2,type3 arg3,type4 arg4,type5 arg5) \
{\
return syscall(__NR_##name, arg1, arg2, arg3, arg4, arg5);\
}#define _syscall6(type,name,type1,arg1,type2,arg2,type3,arg3,type4,arg4, \
type5,arg5,type6,arg6) \
type name (type1 arg1,type2 arg2,type3 arg3,type4 arg4,type5 arg5, type6 arg6) \
{\
return syscall(__NR_##name, arg1, arg2, arg3, arg4, arg5, arg6);\
}_syscall4(int, kevent_ctl, int, arg1, unsigned int, argv2, unsigned int, argv3, void *, argv4);
_syscall6(int, kevent_get_events, int, arg1, unsigned int, argv2, unsigned int, argv3, __u64, argv4, void *, argv5, unsigned, arg6); _syscall4(int, kevent_wait, int, arg1, unsigned int, arg2, unsigned int, argv3, __u64, argv4);
#define ulog(f, a...) fprintf(stderr, "%8u: "f, time(NULL), ##a) #define ulog_err(f, a...) ulog(f ": %s [%d].\n", ##a, strerror(errno), errno)
static void usage(char *p)
{
ulog("Usage: %s -t type -e event -o oneshot -p path -n wait_num -f kevent_file -r ready_num -h\n", p);
}static int evtest_mmap(int fd, struct kevent_mring **ring, int number)
{
int i;
off_t o = 0;
for (i=0; i<number; ++i) {
ring[i] = mmap(NULL, PAGE_SIZE, PROT_READ, MAP_SHARED, fd, o);
if (ring[i] == MAP_FAILED) {
ulog_err("Failed to mmap: i: %d, number: %u, offset: %lu", i, number, o);
return -ENOMEM;
}
printf("mmap: %d: number: %u, offset: %lu.\n", i, number, o);
o += PAGE_SIZE;
}
return 0;
}int main(int argc, char *argv[])
{
int ch, fd, err, oneshot, wait_num;
unsigned int i, ready_num, old_idx, new_idx, tm_sec, tm_nsec;
char *file;
char buf[4096];
struct ukevent *uk;
struct mukevent *m;
struct kevent_mring *ring[KEVENT_MAX_PAGES];
off_t offset;
oneshot = 0;
wait_num = 10;
offset = 0;
old_idx = 0;
file = "/dev/kevent";
tm_sec = 2;
tm_nsec = 0;
ready_num = 1;
while ((ch = getopt(argc, argv, "r:f:t:T:o:n:h")) > 0) {
switch (ch) {
case 'f':
file = optarg;
break;
case 'r':
ready_num = atoi(optarg);
break;
case 'n':
wait_num = atoi(optarg);
break;
case 't':
tm_sec = atoi(optarg);
break;
case 'T':
tm_nsec = atoi(optarg);
break;
case 'o':
oneshot = atoi(optarg);
break;
default:
usage(argv[0]);
return -1;
}
}
fd = open(file, O_RDWR);
if (fd == -1) {
ulog_err("Failed create kevent control block using file %s", file);
return -1;
}
err = evtest_mmap(fd, ring, KEVENT_MAX_PAGES);
if (err)
return err;
memset(buf, 0, sizeof(buf));
for (i=0; i<ready_num; ++i) {
uk = (struct ukevent *)buf;
uk->event = KEVENT_TIMER_FIRED;
uk->type = KEVENT_TIMER;
if (oneshot)
uk->req_flags |= KEVENT_REQ_ONESHOT;
uk->user[0] = i;
uk->id.raw[0] = tm_sec;
uk->id.raw[1] = tm_nsec+i;
err = kevent_ctl(fd, KEVENT_CTL_ADD, 1, uk);
if (err < 0) {
ulog_err("Failed to perform control operation: oneshot: %d, sec: %u, nsec: %u",
oneshot, tm_sec, tm_nsec);
close(fd);
return err;
}
if (err) {
ulog("%d: %016llx: ret_flags: 0x%x, ret_data: %u %d.\n",
i, uk->id.raw_u64,
uk->ret_flags, uk->ret_data[0], (int)uk->ret_data[1]);
}
}
old_idx = ready_num = 0;
while (1) {
new_idx = ring[0]->kidx;
old_idx = ring[0]->uidx;
if (new_idx != old_idx) {
ready_num = (old_idx > new_idx)?(KEVENT_MAX_EVENTS - (old_idx - new_idx)):(new_idx - old_idx);
ulog("mmap: new: %u, old: %u, ready: %u.\n", new_idx, old_idx, ready_num);
for (i=0; i<ready_num; ++i) {
int ridx = old_idx / KEVENTS_ON_PAGE;
int idx = old_idx % KEVENTS_ON_PAGE;
m = &ring[ridx]->event[idx % KEVENTS_ON_PAGE];
ulog("%08x: %08x.%08x - %08x\n",
i, m->id.raw[0], m->id.raw[1], m->ret_flags);
}
}
ulog("going to wait: old: %u, new: %u, ready_num: %u, uidx: %u, kidx: %u.\n",
old_idx, new_idx, ready_num, ring[0]->uidx, ring[0]->kidx);
err = kevent_wait(fd, old_idx, ready_num, 10000000000ULL);
if (err < 0) {
if (errno != EAGAIN) {
ulog_err("Failed to perform control operation: oneshot: %d, sec: %u, nsec: %u",
oneshot, tm_sec, tm_nsec);
close(fd);
return err;
}
old_idx = (old_idx + ready_num) % KEVENT_MAX_EVENTS;
ready_num = 0;
}
ulog("wait: old: %u, ready: %u, ret: %d.\n", old_idx, ready_num, err);
}
close(fd);
return 0;
}Re: [take21 0/4] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovQuoted 10 lines · Evgeniy Polyakov
Generic event handling mechanism.
Consider for inclusion.
Changes from 'take20' patchset:
* new ring buffer implementation
* removed artificial limit on possible number of kevents
With this release and fixed userspace web server it was possible to achive 3960+ req/s with client connection rate of 4000 con/s over 100 Mbit lan, data IO over network was about 10582.7 KB/s, which
is too close to wire speed if we get into account headers and the like.
OK, now that ring buffer is here, I definitely like the direction this code is taking. I just committed the patches to a local repo for a good in-depth review.
Could you write up a simple text file, documenting (a) your proposed syscalls and (b) your ring buffer design?
Overall I have a Linux "design wish", that I hope kevent can fulfill:
To develop completely async applications (generally network servers, in Linux-land) and increase the chance of zero-copy I/O, network and file I/O submission and completion should be as async as possible.
As such, syscalls themselves have come a serializing bottleneck that isn't strictly necessary. A fully-async application should be able to submit file read, file write, and network write requests asynchronously... in batches. Network reads, and file I/O completions should be received asynchronously, potentially in batches.
Even with epoll and AIO syscalls, Linux isn't quite up to the task.
So to me, the design of the userspace interface that solves this problem is a fundamental issue.
My best guess at a solution would be two classes of mmap'd ring buffers, request and response. Let the app allocate one or more. Then have two hooks, (a) kick the kernel to read the request ring, and (b) kick the app when one or more events have arrived on a ring.
But that's just thinking out loud. I welcome any solution that gives userspace a fully-async submission/completion interface for both network and file I/O.
Setting the standard for a good interface here means Linux will kick ass for decades more to come ;-) This is IMO a Big Deal(tm).
Jeff
Re: [take21 0/4] kevent: Generic event handling mechanism.
At an aside... This may be useful. Or not.
Al Viro had an interesting idea about kernel<->userspace data passing interfaces. He had suggested creating a task-specific filesystem derived from ramfs. Through the normal VFS/VM codepaths, the user can easily create [subject to resource/priv checks] a buffer that is locked into the pagecache. Using mmap, read, write, whatever they prefer. Derive from tmpfs, and the buffers are swappable.
Then it would be a simple matter to associate a file stored in "keventfs" with a ring buffer guaranteed to be pagecache-friendly.
Heck, that might make zero-copy easier in some cases, too. And using a filesystem would mean that you could do all this without adding syscalls, by using special (poll-able!) files in the filesystem for control and notification purposes.
Jeff
Re: [take21 0/4] kevent: Generic event handling mechanism.
Quoted 8 lines · Jeff Garzik
At an aside... This may be useful. Or not.
Al Viro had an interesting idea about kernel<->userspace data passing interfaces. He had suggested creating a task-specific filesystem derived from ramfs. Through the normal VFS/VM codepaths, the user can easily create [subject to resource/priv checks] a buffer that is locked into the pagecache. Using mmap, read, write, whatever they prefer. Derive from tmpfs, and the buffers are swappable.
It looks like Al likes filesystems more than any other part of kernel tree...
Existing ring buffer is created in process' memory, so it is swappable too (which is probably the most significant part of this ring buffer version), but in theory kevent file descriptor can be obtained not from the char device, but from special filesystem (well, it was done in that way in first releases but then I was asked to remove such functionality).
Quoted 7 lines
Then it would be a simple matter to associate a file stored in "keventfs" with a ring buffer guaranteed to be pagecache-friendly.
Heck, that might make zero-copy easier in some cases, too. And using a filesystem would mean that you could do all this without adding syscalls, by using special (poll-able!) files in the filesystem for control and notification purposes.
There are too many ideas about networking zero-copy both sending and receiving, and some of them are even implemented on different layers (starting from special allocator down to splice() with additional single allocation/copy).
Quoted 1 line
Jeff
Signature · 1 line
Evgeniy Polyakov
Re: [take21 0/4] kevent: Generic event handling mechanism.
In reply to Jeff GarzikQuoted 13 lines · Jeff Garzik
Quoted 11 lines · Evgeniy Polyakov
Generic event handling mechanism.
Consider for inclusion.
Changes from 'take20' patchset:
* new ring buffer implementation
* removed artificial limit on possible number of kevents With this release and fixed userspace web server it was possible to achive 3960+ req/s with client connection rate of 4000 con/s over 100 Mbit lan, data IO over network was about 10582.7 KB/s, which is too close to wire speed if we get into account headers and the like.
Quoted 3 lines
OK, now that ring buffer is here, I definitely like the direction this code is taking. I just committed the patches to a local repo for a good in-depth review.
It is third ring buffer, the fourth one will be in the next release, which should satisfy everyone.
Quoted 2 lines
Could you write up a simple text file, documenting (a) your proposed syscalls and (b) your ring buffer design?
Initial draft about supported syscalls can be found at documentation page at
http://linux-net.osdl.org/index.php/Kevent
Ring buffer background bits pasted below (quotations from blog, do not pay too much attention if sometimes something is not in sync).
New ring buffer is implemented fully in userspace in process' memory, which means that there are no memory pinned, its size can have almost any length, several threads and processes can access it simultaneously. There is new system call
int kevent_ring_init(int ctl_fd, struct ring_buffer *ring, unsigned int num);
which initializes kevent's ring buffer (int ctl_fd is a kevent file descriptor, struct ring_buffer *ring is a userspace allocated ring buffer, and unsigned int num is maximum number of events (struct ukevent) which can be placed into that buffer).
Ring buffer is described with following structure:
struct kevent_ring
{
unsigned int ring_kidx, ring_uidx;
struct ukevent event[0];
};where unsigned int ring_kidx, ring_uidx are last kernel's position (i.e. position which points to the first place after the last kevent put by kernel into the ring buffer) and last userspace commit (i.e. position where first unread kevent lives) positions appropriately.
I will release appropriate userspace test application when tests are completed.
When kevent is removed (not dequeued when it is ready, but just removed), even if it was ready, it is not copied into ring buffer, since if it is removed, no one cares about it (otherwise user would wait until it becomes ready and got it through usual way using kevent_get_events() or kevent_wait()) and thus no need to copy it to the ring buffer. Dequeueing of the kevent (calling kevent_get_events()) means that user has processed previously dequeued kevent and is ready to process new one, which means that position in the ring buffer previously ocupied but that event can be reused by currently dequeued event. In the world where only one type of syscalls to get events is used (either usual way and kevent_get_events() or ring buffer and kevent_wait()) it should not be a problem, since kevent_wait() only allows to mark number of events as processed by userspace starting from the beginning (i.e. from the last processed event), but if several threads will use different models, that can rise some questions, for example one thread can start to read events from ring buffer, and in that time other thread will call kevent_get_events(), which can rewrite that events. Actually other thread can call kevent_wait() to commit that events (i.e. mark them as processed by userspace so kernel could free them or requeue), so appropriate locking is required in userspace in any way.
So I want to repeat, that it is possible with userspace ring buffer, that events in the ring buffer can be replaced without knowledge for the thread currently reading them (when other thread calls kevent_get_events() or kevent_wait()), so appropriate locking between threads or processes, which can simultaneously access the same ring buffer, is required.
Having userspace ring buffer allows to make all kevent syscalls as so called 'cancellation points' by glibc, i.e. when thread has been cancelled in kevent syscall, thread can be safely removed and no events will be lost, since each syscall will copy event into special ring buffer, accessible from other threads or even processes (if shared memory is used).
Quoted 15 lines
Overall I have a Linux "design wish", that I hope kevent can fulfill:
To develop completely async applications (generally network servers, in Linux-land) and increase the chance of zero-copy I/O, network and file I/O submission and completion should be as async as possible.
As such, syscalls themselves have come a serializing bottleneck that isn't strictly necessary. A fully-async application should be able to submit file read, file write, and network write requests asynchronously... in batches. Network reads, and file I/O completions should be received asynchronously, potentially in batches.
Even with epoll and AIO syscalls, Linux isn't quite up to the task.
Quoted 7 lines
So to me, the design of the userspace interface that solves this problem is a fundamental issue.
My best guess at a solution would be two classes of mmap'd ring buffers, request and response. Let the app allocate one or more. Then have two hooks, (a) kick the kernel to read the request ring, and (b) kick the app when one or more events have arrived on a ring.
Mmap ring buffer implementation was stopped by Andrew Morton and Ulrich Drepper, process' memory is used instead. copy_to_user() is slower (and some times noticebly), but there are major advantages of such approach.
Quoted 3 lines
But that's just thinking out loud. I welcome any solution that gives userspace a fully-async submission/completion interface for both network and file I/O.
Well, kevent network and FS AIO are suspended for now (although first patches included them all).
Quoted 5 lines
Setting the standard for a good interface here means Linux will kick ass for decades more to come ;-) This is IMO a Big Deal(tm).
Jeff
Signature · 1 line
Evgeniy Polyakov
Re: [take21 0/4] kevent: Generic event handling mechanism.
Quoted 1 line · Evgeniy Polyakov
Well, kevent network and FS AIO are suspended for now (although first
Why?
IMO, getting async event submission right is important. It should be designed in parallel with async event reception.
Jeff
Re: [take21 0/4] kevent: Generic event handling mechanism.
Quoted 7 lines · Jeff Garzik
Quoted 1 line · Evgeniy Polyakov
Well, kevent network and FS AIO are suspended for now (although first
Why?
IMO, getting async event submission right is important. It should be designed in parallel with async event reception.
It was not only designed but also implemented, but...
FS AIO was confirmed to have correct design, but there were minor (from my point of view) layering design problems
(I was almost suggested to make myself a lobotomy after I put get_block() callback into address_space_operations, there were also some code duplication of mpage_readpages() in async way in kevent/kevent_aio.c - I made it to separate kevent as much as possible, both changes can live in fs/ with appropriate callback export).
Network AIO I postponed for a while, since looking how hard core changed are processed, it looks like a better decision...
Using Ulrich's DMA allocation API (if it would exist not only as proposal) it would be possible to speed up NAIO yet a bit too.
Kevent based FS AIO patch can be found for example here (it contains full kevent subsystem with network aio and fs aio):
http://tservice.net.ru/~s0mbre/archive/kevent/kevent_full.diff.3
Network aio homepage:
http://tservice.net.ru/~s0mbre/old/?section=projects&item=naio
Quoted 2 lines
Jeff
Signature · 1 line
Evgeniy Polyakov
Re: [take21 0/4] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovQuoted 3 lines · Evgeniy Polyakov
Mmap ring buffer implementation was stopped by Andrew Morton and Ulrich Drepper, process' memory is used instead.
copy_to_user()is slower (and some times noticebly), but there are major advantages of such approach.
hmmmm. I say there are advantages to both.
Perhaps create a "kevent_direct_limit" resource limit for each thread. By default, each thread could mmap $n pinned pagecache pages. Sysadmin can tune certain app resource limits to permit more.
I would think that retaining the option to avoid copy_to_user() -somehow- in -some- cases would be wise.
Jeff
Re: [take21 0/4] kevent: Generic event handling mechanism.
Quoted 7 lines · On Tue, 07 Nov 2006 07:32:20 -0500 Jeff Garzik <[email protected]> wrote:
Quoted 3 lines · Evgeniy Polyakov
Mmap ring buffer implementation was stopped by Andrew Morton and Ulrich Drepper, process' memory is used instead.
copy_to_user()is slower (and some times noticebly), but there are major advantages of such approach.hmmmm. I say there are advantages to both.
My problem with the old mmapped ringbuffer was that it permitted each user to pin (typically) 48MB of unswappable memory. Plus this pinned-memory problem would put upper bounds on the ring size.
Quoted 6 lines
Perhaps create a "
kevent_direct_limit" resource limit for each thread. By default, each thread could mmap $n pinned pagecache pages. Sysadmin can tune certain app resource limits to permit more.I would think that retaining the option to avoid
copy_to_user()-somehow- in -some- cases would be wise.
What Evgeniy means here is that copy_to_user() is slower than memcpy() (on his machine, with his kernel config, at least).
Which is kinda weird and unexpected and is something which we should investigate independently from this project. (Rather than simply going and bypassing it!)
David Miller
maintainerRe: [take21 0/4] kevent: Generic event handling mechanism.
From: Andrew Morton <[email protected]>
Date: Tue, 7 Nov 2006 11:34:00 -0800
Quoted 6 lines
What Evgeniy means here is that
copy_to_user()is slower thanmemcpy()(on his machine, with his kernel config, at least).Which is kinda weird and unexpected and is something which we should investigate independently from this project. (Rather than simply going and bypassing it!)
It's straightforward to me. :-)
If the kerne memcpy()'s, it uses those nice 4MB PTE mappings to the kernel pages. With copy_to_user() you run through tiny 4K or 8K PTE mappings which thrash the TLB.
The TLB is therefore able to hold more of the accessed state at a time if you touch the pages on the kernel side.
Re: [take21 0/4] kevent: Generic event handling mechanism.
Quoted 10 lines · On Tue, 07 Nov 2006 12:52:41 -0800 (PST) David Miller <[email protected]>…
From: Andrew Morton <[email protected]>
Date: Tue, 7 Nov 2006 11:34:00 -0800Quoted 6 lines
What Evgeniy means here is that
copy_to_user()is slower thanmemcpy()(on his machine, with his kernel config, at least).Which is kinda weird and unexpected and is something which we should investigate independently from this project. (Rather than simply going and bypassing it!)
Quoted 8 lines
It's straightforward to me. :-)
If the kerne memcpy()'s, it uses those nice 4MB PTE mappings to the kernel pages. With
copy_to_user()you run through tiny 4K or 8K PTE mappings which thrash the TLB.The TLB is therefore able to hold more of the accessed state at a time if you touch the pages on the kernel side.
Maybe. Evgeniy tends to favour teeny microbenchmarks. I'd also be suspecting the considerable setup code in the x86 uaccess funtions. That would show up in a tight loop doing large numbers of small copies.
cover kevent: Generic event handling mechanism.
In reply to a message not in the archiveGeneric event handling mechanism.
Consider for inclusion.
Changes from 'take21' patchset:
* minor cleanups (different return values, removed unneded variables, whitespaces and so on)
* fixed bug in kevent removal in case when kevent being removed
is the same as overflow_kevent (spotted by Eric Dumazet)
Changes from 'take20' patchset:
* new ring buffer implementation
* removed artificial limit on possible number of kevents
With this release and fixed userspace web server it was possible to achive 3960+ req/s with client connection rate of 4000 con/s over 100 Mbit lan, data IO over network was about 10582.7 KB/s, which is too close to wire speed if we get into account headers and the like.
Changes from 'take19' patchset:
* use __init instead of __devinit
* removed 'default N' from config for user statistic
* removed kevent_user_fini() since kevent can not be unloaded
* use KERN_INFO for statistic output
Changes from 'take18' patchset:
* use __init instead of __devinit
* removed 'default N' from config for user statistic
* removed kevent_user_fini() since kevent can not be unloaded
* use KERN_INFO for statistic output
Changes from 'take17' patchset:
* Use RB tree instead of hash table.
At least for a web sever, frequency of addition/deletion of new kevent is comparable with number of search access, i.e. most of the time events are added, accesed only couple of times and then removed, so it justifies RB tree usage over AVL tree, since the latter does have much slower deletion time (max O(log(N)) compared to 3 ops),
although faster search time (1.44*O(log(N)) vs. 2*O(log(N))).
So for kevents I use RB tree for now and later, when my AVL tree implementation is ready, it will be possible to compare them.
* Changed readiness check for socket notifications.
With both above changes it is possible to achieve more than 3380 req/second compared to 2200, sometimes 2500 req/second for epoll() for trivial web-server and httperf client on the same hardware.
It is possible that above kevent limit is due to maximum allowed kevents in a time limit, which is 4096 events.
Changes from 'take16' patchset:
* misc cleanups (__read_mostly, const ...)
* created special macro which is used for mmap size (number of pages) calculation
* export kevent_socket_notify(), since it is used in network protocols which can be built as modules (IPv6 for example)
Changes from 'take15' patchset:
* converted kevent_timer to high-resolution timers, this forces timer API update at
http://linux-net.osdl.org/index.php/Kevent
* use struct ukevent* instead of void * in syscalls (documentation has been updated)
* added warning in kevent_add_ukevent() if ring has broken index (for testing)
Changes from 'take14' patchset:
* added kevent_wait()
This syscall waits until either timeout expires or at least one event becomes ready. It also commits that @num events from @start are processed by userspace and thus can be be removed or rearmed (depending on it's flags). It can be used for commit events read by userspace through mmap interface. Example userspace code (evtest.c) can be found on project's homepage.
* added socket notifications (send/recv/accept)
Changes from 'take13' patchset:
* do not get lock aroung user data check in __kevent_search()
* fail early if there were no registered callbacks for given type of kevent
* trailing whitespace cleanup
Changes from 'take12' patchset:
* remove non-chardev interface for initialization
* use pointer to kevent_mring instead of unsigned longs
* use aligned 64bit type in raw user data (can be used by high-res timer if needed)
* simplified enqueue/dequeue callbacks and kevent initialization
* use nanoseconds for timeout
* put number of milliseconds into timer's return data
* move some definitions into user-visible header
* removed filenames from comments
Changes from 'take11' patchset:
* include missing headers into patchset
* some trivial code cleanups (use goto instead of if/else games and so on)
* some whitespace cleanups
* check for ready_callback() callback before main loop which should save us some ticks
Changes from 'take10' patchset:
* removed non-existent prototypes
* added helper function for kevent_registered_callbacks
* fixed 80 lines comments issues
* added shared between userspace and kernelspace header instead of embedd them in one
* core restructuring to remove forward declarations
* s o m e w h i t e s p a c e c o d y n g s t y l e c l e a n u p
* use vm_insert_page() instead of remap_pfn_range()
Changes from 'take9' patchset:
* fixed ->nopage method
Changes from 'take8' patchset:
* fixed mmap release bug
* use module_init() instead of late_initcall()
* use better structures for timer notifications
Changes from 'take7' patchset:
* new mmap interface (not tested, waiting for other changes to be acked)
- use nopage() method to dynamically substitue pages
- allocate new page for events only when new added kevent requres it
- do not use ugly index dereferencing, use structure instead
- reduced amount of data in the ring (id and flags), maximum 12 pages on x86 per kevent fd
Changes from 'take6' patchset:
* a lot of comments!
* do not use list poisoning for detection of the fact, that entry is in the list
* return number of ready kevents even if copy*user() fails
* strict check for number of kevents in syscall
* use ARRAY_SIZE for array size calculation
* changed superblock magic number
* use SLAB_PANIC instead of direct panic() call
* changed -E* return values
* a lot of small cleanups and indent fixes
Changes from 'take5' patchset:
* removed compilation warnings about unused wariables when lockdep is not turned on
* do not use internal socket structures, use appropriate (exported) wrappers instead
* removed default 1 second timeout
* removed AIO stuff from patchset
Changes from 'take4' patchset:
* use miscdevice instead of chardevice
* comments fixes
Changes from 'take3' patchset:
* removed serializing mutex from kevent_user_wait()
* moved storage list processing to RCU
* removed lockdep screaming - all storage locks are initialized in the same function, so it was
learned
to differentiate between various cases
* remove kevent from storage if is marked as broken after callback
* fixed a typo in mmaped buffer implementation which would end up in wrong index calcualtion
Changes from 'take2' patchset:
* split kevent_finish_user() to locked and unlocked variants
* do not use KEVENT_STAT ifdefs, use inline functions instead
* use array of callbacks of each type instead of each kevent callback initialization
* changed name of ukevent guarding lock
* use only one kevent lock in kevent_user for all hash buckets instead of per-bucket locks
* do not use kevent_user_ctl structure instead provide needed arguments as syscall parameters
* various indent cleanups
* added optimisation, which is aimed to help when a lot of kevents are being copied from
userspace
* mapped buffer (initial) implementation (no userspace yet)
Changes from 'take1' patchset:
- rebased against 2.6.18-git tree
- removed ioctl controlling
- added new syscall kevent_get_events(int fd, unsigned int min_nr, unsigned int max_nr, unsigned int timeout, void __user *buf, unsigned flags)
- use old syscall kevent_ctl for creation/removing, modification and initial kevent initialization
- use mutuxes instead of semaphores
- added file descriptor check and return error if provided descriptor does not match kevent file operations
- various indent fixes
- removed aio_sendfile() declarations.
Thank you.
PATCH 1/4 kevent: Core files.
Core files.
This patch includes core kevent files:
* userspace controlling
* kernelspace interfaces
* initialization
* notification state machines
Some bits of documentation can be found on project's homepage (and links from there):
http://tservice.net.ru/~s0mbre/old/?section=projects&item=kevent
arch/i386/kernel/syscall_table.S +3 -0
@@ -318,3 +318,6 @@ ENTRY(sys_call_table) .long sys_vmsplice .long sys_move_pages .long sys_getcpu + .long sys_kevent_get_events + .long sys_kevent_ctl /* 320 */ + .long sys_kevent_wait
arch/x86_64/ia32/ia32entry.S +4 -1
@@ -714,8 +714,11 @@ #endif .quad compat_sys_get_robust_list .quad sys_splice .quad sys_sync_file_range - .quad sys_tee + .quad sys_tee /* 315 */ .quad compat_sys_vmsplice .quad compat_sys_move_pages .quad sys_getcpu + .quad sys_kevent_get_events + .quad sys_kevent_ctl /* 320 */ + .quad sys_kevent_wait ia32_syscall_end:
include/asm-i386/unistd.h +4 -1
@@ -324,10 +324,13 @@ #define __NR_tee 315#define __NR_vmsplice 316#define __NR_move_pages 317#define __NR_getcpu 318 +#define __NR_kevent_get_events 319 +#define __NR_kevent_ctl 320 +#define __NR_kevent_wait 321#ifdef __KERNEL__ -#define NR_syscalls 319 +#define NR_syscalls 322#include <linux/err.h> /*
include/asm-x86_64/unistd.h +7 -1
@@ -619,10 +619,16 @@ #define __NR_vmsplice 278 __SYSCALL(__NR_vmsplice, sys_vmsplice)#define __NR_move_pages 279 __SYSCALL(__NR_move_pages, sys_move_pages) +#define __NR_kevent_get_events 280 +__SYSCALL(__NR_kevent_get_events, sys_kevent_get_events) +#define __NR_kevent_ctl 281 +__SYSCALL(__NR_kevent_ctl, sys_kevent_ctl) +#define __NR_kevent_wait 282 +__SYSCALL(__NR_kevent_wait, sys_kevent_wait)#ifdef __KERNEL__ -#define __NR_syscall_max __NR_move_pages +#define __NR_syscall_max __NR_kevent_wait#include <linux/err.h> #ifndef __NO_STUBS
include/linux/kevent.h new file +205 -0
205 lines not on this page · raw message
include/linux/kevent_storage.h new file +11 -0
@@ -0,0 +1,11 @@ +#ifndef __KEVENT_STORAGE_H +#define __KEVENT_STORAGE_H + +struct kevent_storage +{ +void *origin;/* Originator's pointer, e.g. struct sock or struct file. Can be NULL. */ +struct list_head list;/* List of queued kevents. */ +spinlock_t lock;/* Protects users queue. */ +}; + +#endif /* __KEVENT_STORAGE_H */
include/linux/syscalls.h +5 -0
@@ -54,6 +54,7 @@ struct compat_stat;struct compat_timeval ;struct robust_list_head ;struct getcpu_cache ; +struct ukevent ;#include <linux/types.h> #include <linux/aio_abi.h>
@@ -599,4 +600,8 @@ asmlinkage long sys_set_robust_list(stru size_t len);asmlinkage long sys_getcpu (unsigned __user *cpu,unsigned __user *node,struct getcpu_cache __user *cache); +asmlinkage long sys_kevent_get_events (int ctl_fd,unsigned int min,unsigned int max, +__u64 timeout,struct ukevent __user *buf,unsigned flags); +asmlinkage long sys_kevent_ctl (int ctl_fd,unsigned int cmd,unsigned int num,struct ukevent __user *buf); +asmlinkage long sys_kevent_wait (int ctl_fd,unsigned int start,unsigned int num,__u64 timeout);#endif
include/linux/ukevent.h new file +163 -0
163 lines not on this page · raw message
init/Kconfig +2 -0
@@ -201,6 +201,8 @@ config AUDITSYSCALL such as SELinux. To use audit's filesystem watch feature, please ensure that INOTIFY is configured. +source "kernel/kevent/Kconfig" + config IKCONFIG bool "Kernel .config support" ---help---
kernel/Makefile +1 -0
@@ -47,6 +47,7 @@ obj-$(CONFIG_DETECT_SOFTLOCKUP) += softl obj-$(CONFIG_GENERIC_HARDIRQS) += irq/ obj-$(CONFIG_SECCOMP) += seccomp.o obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o +obj-$(CONFIG_KEVENT) += kevent/ obj-$(CONFIG_RELAY) += relay.o obj-$(CONFIG_TASK_DELAY_ACCT) += delayacct.o obj-$(CONFIG_TASKSTATS) += taskstats.o
kernel/kevent/Kconfig new file +39 -0
@@ -0,0 +1,39 @@ +config KEVENT + bool "Kernel event notification mechanism" + help + This option enables event queue mechanism. + It can be used as replacement for poll()/select(), AIO callback + invocations, advanced timer notifications and other kernel + object status changes. + +config KEVENT_USER_STAT + bool "Kevent user statistic" + depends on KEVENT + help + This option will turn kevent_user statistic collection on. + Statistic data includes total number of kevent, number of kevents + which are ready immediately at insertion time and number of kevents + which were removed through readiness completion. + It will be printed each time control kevent descriptor is closed. + +config KEVENT_TIMER + bool "Kernel event notifications for timers" + depends on KEVENT + help + This option allows to use timers through KEVENT subsystem. + +config KEVENT_POLL + bool "Kernel event notifications for poll()/select()" + depends on KEVENT + help + This option allows to use kevent subsystem for poll()/select() + notifications. + +config KEVENT_SOCKET + bool "Kernel event notifications for sockets" + depends on NET && KEVENT + help + This option enables notifications through KEVENT subsystem of + sockets operations, like new packet receiving conditions, + ready for accept conditions and so on. +
kernel/kevent/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +obj-y := kevent.o kevent_user.o +obj-$(CONFIG_KEVENT_TIMER) += kevent_timer.o +obj-$(CONFIG_KEVENT_POLL) += kevent_poll.o +obj-$(CONFIG_KEVENT_SOCKET) += kevent_socket.o
kernel/kevent/kevent.c new file +227 -0
227 lines not on this page · raw message
kernel/kevent/kevent_user.c new file +1004 -0
1,004 lines not on this page · raw message
kernel/sys_ni.c +4 -0
@@ -122,6 +122,10 @@ cond_syscall(ppc_rtas);cond_syscall (sys_spu_run);cond_syscall (sys_spu_create); +cond_syscall (sys_kevent_get_events); +cond_syscall (sys_kevent_wait); +cond_syscall (sys_kevent_ctl); +/* mmu depending weak syscall entries */ cond_syscall (sys_mprotect);cond_syscall (sys_msync);
PATCH 2/4 kevent: poll/select() notifications.
poll/select() notifications.
This patch includes generic poll/select notifications. kevent_poll works simialr to epoll and has the same issues (callback is invoked not from internal state machine of the caller, but through process awake, a lot of allocations and so on).
include/linux/fs.h +8 -0
@@ -276,6 +276,7 @@ #include <linux/prio_tree.h>#include <linux/init.h> #include <linux/sched.h> #include <linux/mutex.h> +#include <linux/kevent.h> #include <asm/atomic.h> #include <asm/semaphore.h>
@@ -586,6 +587,10 @@ #ifdef CONFIG_INOTIFYstruct mutex inotify_mutex;/* protects the watches list */ #endif +#ifdef CONFIG_KEVENT_SOCKET +struct kevent_storage st; +#endif +unsigned long i_state;unsigned long dirtied_when;/* jiffies of first dirtying */
@@ -739,6 +744,9 @@ #ifdef CONFIG_EPOLLstruct list_head f_ep_links;spinlock_t f_ep_lock;#endif /* #ifdef CONFIG_EPOLL */ +#ifdef CONFIG_KEVENT_POLL +struct kevent_storage st; +#endif struct address_space *f_mapping; };extern spinlock_t files_lock;
kernel/kevent/kevent_poll.c new file +222 -0
222 lines not on this page · raw message
PATCH 3/4 kevent: Socket notifications.
Socket notifications.
This patch includes socket send/recv/accept notifications. Using trivial web server based on kevent and this features instead of epoll it's performance increased more than noticebly. More details about various benchmarks and server itself (evserver_kevent.c) can be found on project's homepage.
fs/inode.c +7 -0
@@ -21,6 +21,7 @@ #include <linux/pagemap.h>#include <linux/cdev.h> #include <linux/bootmem.h> #include <linux/inotify.h> +#include <linux/kevent.h> #include <linux/mount.h> /*
@@ -164,12 +165,18 @@ #endif } inode->i_private =0 ; inode->i_mapping = mapping; +#if definedCONFIG_KEVENT_SOCKET +kevent_storage_init (inode, &inode->st); +#endif }return inode; }void destroy_inode (struct inode *inode) { +#if definedCONFIG_KEVENT_SOCKET +kevent_storage_fini (&inode->st); +#endif BUG_ON (inode_has_buffers (inode));security_inode_free (inode);if (inode->i_sb->s_op->destroy_inode)
include/net/sock.h +17 -15
@@ -48,6 +48,7 @@ #include <linux/lockdep.h>#include <linux/netdevice.h> #include <linux/skbuff.h> /* struct sk_buff */ #include <linux/security.h> +#include <linux/kevent.h> #include <linux/filter.h>
@@ -450,6 +451,21 @@ static inline int sk_stream_memory_free(extern void sk_stream_rfree (struct sk_buff *skb); +struct socket_alloc { +struct socket socket; +struct inode vfs_inode; +}; + +static inline struct socket *SOCKET_I (struct inode *inode) +{ + return &container_of(inode, struct socket_alloc, vfs_inode)->socket; +} + +static inline struct inode *SOCK_INODE (struct socket *socket) +{ + return &container_of(socket, struct socket_alloc, socket)->vfs_inode; +} +static inline void sk_stream_set_owner_r (struct sk_buff *skb,struct sock *sk) { skb->sk = sk;
@@ -477,6 +493,7 @@ static inline void sk_add_backlog(struct sk->sk_backlog.tail = skb; } skb->next =NULL ; +kevent_socket_notify (sk,KEVENT_SOCKET_RECV ); }#define sk_wait_event (__sk, __timeo, __condition) \
@@ -679,21 +696,6 @@ static inline struct kiocb *siocb_to_kioreturn si->kiocb; } -struct socket_alloc { -struct socket socket; -struct inode vfs_inode; -}; - -static inline struct socket *SOCKET_I (struct inode *inode) -{ - return &container_of(inode, struct socket_alloc, vfs_inode)->socket; -} - -static inline struct inode *SOCK_INODE (struct socket *socket) -{ - return &container_of(socket, struct socket_alloc, socket)->vfs_inode; -} -extern void __sk_stream_mem_reclaim (struct sock *sk);extern int sk_stream_mem_schedule (struct sock *sk,int size,int kind);
include/net/tcp.h +1 -0
@@ -857,6 +857,7 @@ static inline int tcp_prequeue(struct so tp->ucopy.memory =0 ; }else if (skb_queue_len (&tp->ucopy.prequeue) ==1 ) {wake_up_interruptible (sk->sk_sleep); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND );if (!inet_csk_ack_scheduled (sk)) inet_csk_reset_xmit_timer(sk, ICSK_TIME_DACK, (3 * TCP_RTO_MIN) / 4,
kernel/kevent/kevent_socket.c new file +129 -0
129 lines not on this page · raw message
net/core/sock.c +9 -1
@@ -1402,6 +1402,7 @@ static void sock_def_wakeup(struct sockif (sk->sk_sleep &&waitqueue_active (sk->sk_sleep))wake_up_interruptible_all (sk->sk_sleep);read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_error_report (struct sock *sk)
@@ -1411,6 +1412,7 @@ static void sock_def_error_report(structwake_up_interruptible (sk->sk_sleep);sk_wake_async (sk,0 ,POLL_ERR );read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_readable (struct sock *sk,int len)
@@ -1420,6 +1422,7 @@ static void sock_def_readable(struct socwake_up_interruptible (sk->sk_sleep);sk_wake_async (sk,1 ,POLL_IN );read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_write_space (struct sock *sk)
@@ -1439,6 +1442,7 @@ static void sock_def_write_space(struct }read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_SEND |KEVENT_SOCKET_RECV ); }static void sock_def_destruct (struct sock *sk)
@@ -1489,6 +1493,8 @@ #endif sk->sk_state =TCP_CLOSE ; sk->sk_socket = sock; +kevent_sk_reinit (sk); +sock_set_flag (sk,SOCK_ZAPPED );if (sock)
@@ -1555,8 +1561,10 @@ void fastcall release_sock(struct sock *if (sk->sk_backlog.tail)__release_sock (sk); sk->sk_lock.owner =NULL ; -if (waitqueue_active (&sk->sk_lock.wq)) +if (waitqueue_active (&sk->sk_lock.wq)) {wake_up (&sk->sk_lock.wq); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); + }spin_unlock_bh (&sk->sk_lock.slock); }EXPORT_SYMBOL (release_sock);
net/core/stream.c +1 -0
@@ -36,6 +36,7 @@ void sk_stream_write_space(struct sock *wake_up_interruptible (sk->sk_sleep);if (sock->fasync_list && !(sk->sk_shutdown &SEND_SHUTDOWN ))sock_wake_async (sock,2 ,POLL_OUT ); +kevent_socket_notify (sk,KEVENT_SOCKET_SEND |KEVENT_SOCKET_RECV ); } }
net/ipv4/tcp_input.c +1 -0
@@ -3119,6 +3119,7 @@ static void tcp_ofo_queue(struct sock *s__skb_unlink (skb, &tp->out_of_order_queue);__skb_queue_tail (&sk->sk_receive_queue, skb); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV ); tp->rcv_nxt =TCP_SKB_CB (skb)->end_seq;if (skb->h.th->fin)tcp_fin (skb, sk, skb->h.th);
net/ipv4/tcp_ipv4.c +2 -0
@@ -61,6 +61,7 @@ #include <linux/cache.h>#include <linux/jhash.h> #include <linux/init.h> #include <linux/times.h> +#include <linux/kevent.h> #include <net/icmp.h> #include <net/inet_hashtables.h>
@@ -870,6 +871,7 @@ #endifreqsk_free (req); }else {inet_csk_reqsk_queue_hash_add (sk, req,TCP_TIMEOUT_INIT ); +kevent_socket_notify (sk,KEVENT_SOCKET_ACCEPT ); }return 0 ;
net/socket.c +3 -0
@@ -85,6 +85,7 @@ #include <linux/compat.h>#include <linux/kmod.h> #include <linux/audit.h> #include <linux/wireless.h> +#include <linux/kevent.h> #include <asm/uaccess.h> #include <asm/unistd.h>
@@ -490,6 +491,8 @@ static struct socket *sock_alloc(void) inode->i_uid = current->fsuid; inode->i_gid = current->fsgid; +kevent_socket_reinit (sock); +get_cpu_var (sockets_in_use)++;put_cpu_var (sockets_in_use);return sock;
PATCH 4/4 kevent: Timer notifications.
Timer notifications.
Timer notifications can be used for fine grained per-process time management, since interval timers are very inconvenient to use, and they are limited.
This subsystem uses high-resolution timers.
id.raw[0] is used as number of seconds
id.raw[1] is used as number of nanoseconds
kernel/kevent/kevent_timer.c new file +113 -0
@@ -0,0 +1,113 @@ +/* +* 2006 Copyright (c) Evgeniy Polyakov <[email protected]> +* All rights reserved. +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +*/ + +#include <linux/kernel.h> +#include <linux/types.h> +#include <linux/list.h> +#include <linux/slab.h> +#include <linux/spinlock.h> +#include <linux/hrtimer.h> +#include <linux/jiffies.h> +#include <linux/kevent.h> + +struct kevent_timer +{ +struct hrtimer ktimer; +struct kevent_storage ktimer_storage; +struct kevent *ktimer_event; +}; + +static int kevent_timer_func (struct hrtimer *timer) +{ + struct kevent_timer *t = container_of(timer, struct kevent_timer, ktimer); +struct kevent *k = t->ktimer_event; + +kevent_storage_ready (&t->ktimer_storage,NULL ,KEVENT_MASK_ALL ); +hrtimer_forward (timer, timer->base->softirq_time, +ktime_set (k->event.id.raw[0 ], k->event.id.raw[1 ])); +return HRTIMER_RESTART ; +} + +static struct lock_class_key kevent_timer_key; + +static int kevent_timer_enqueue (struct kevent *k) +{ +int err; +struct kevent_timer *t; + + t =kmalloc (sizeof (struct kevent_timer ),GFP_KERNEL ); +if (!t) +return -ENOMEM ; + +hrtimer_init (&t->ktimer,CLOCK_MONOTONIC ,HRTIMER_REL ); + t->ktimer.expires =ktime_set (k->event.id.raw[0 ], k->event.id.raw[1 ]); + t->ktimer.function = kevent_timer_func; + t->ktimer_event = k; + + err =kevent_storage_init (&t->ktimer, &t->ktimer_storage); +if (err) + goto err_out_free; +lockdep_set_class (&t->ktimer_storage.lock, &kevent_timer_key); + + err =kevent_storage_enqueue (&t->ktimer_storage, k); +if (err) + goto err_out_st_fini; + +printk ("%s: jiffies: %lu, timer: %p.\n" , __func__, jiffies, &t->ktimer); +hrtimer_start (&t->ktimer, t->ktimer.expires,HRTIMER_REL ); + +return 0 ; + +err_out_st_fini: +kevent_storage_fini (&t->ktimer_storage); +err_out_free: +kfree (t); + +return err; +} + +static int kevent_timer_dequeue (struct kevent *k) +{ +struct kevent_storage *st = k->st; + struct kevent_timer *t = container_of(st, struct kevent_timer, ktimer_storage); + +hrtimer_cancel (&t->ktimer); +kevent_storage_dequeue (st, k); +kfree (t); + +return 0 ; +} + +static int kevent_timer_callback (struct kevent *k) +{ + k->event.ret_data[0 ] =jiffies_to_msecs (jiffies); +return 1 ; +} + +static int __init kevent_init_timer (void ) +{ +struct kevent_callbacks tc = { + .callback = &kevent_timer_callback, + .enqueue = &kevent_timer_enqueue, + .dequeue = &kevent_timer_dequeue}; + +return kevent_add_callbacks (&tc,KEVENT_TIMER ); +} +module_init (kevent_init_timer); +
Re: [take22 0/4] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovHi!
Quoted 5 lines
Generic event handling mechanism.
Consider for inclusion.
Changes from 'take21' patchset:
We are not interrested in how many times you spammed us, nor we want to know what was wrong in previous versions. It would be nice to have short summary of what this is good for, instead.
Pavel
Signature · 1 line
Thanks, Sharp!
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 11 lines · Pavel Machek
Hi!
Quoted 5 lines
Generic event handling mechanism.
Consider for inclusion.
Changes from 'take21' patchset:
We are not interrested in how many times you spammed us, nor we want to know what was wrong in previous versions. It would be nice to have short summary of what this is good for, instead.
Let me guess, short explaination in subsequent emails is not enough... If changelog will be removed, then how people will detect what happend after previous release?
Kevent is a generic subsytem which allows to handle event notifications. It supports both level and edge triggered events. It is similar to poll/epoll in some cases, but it is more scalable, it is faster and allows to work with essentially eny kind of events.
Events are provided into kernel through control syscall and can be read back through mmaped ring or syscall.
Kevent update (i.e. readiness switching) happens directly from internals of the appropriate state machine of the underlying subsytem (like network, filesystem, timer or any other).
I will put that text into introduction message.
Quoted 3 lines
Pavel
Signature · 1 line
Thanks, Sharp!
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
Hi!
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Generic event handling mechanism.
Consider for inclusion.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Changes from 'take21' patchset:
We are not interrested in how many times you spammed us, nor we want to know what was wrong in previous versions. It would be nice to have short summary of what this is good for, instead.
Let me guess, short explaination in subsequent emails is not enough...
Yes.
Quoted 4 lines
Kevent is a generic subsytem which allows to handle event notifications. It supports both level and edge triggered events. It is similar to poll/epoll in some cases, but it is more scalable, it is faster and allows to work with essentially eny kind of events.
Quantifying "how much more scalable" would be nice, as would be some example where it is useful. ("It makes my webserver twice as fast on monster 64-cpu box").
Quoted 7 lines
Events are provided into kernel through control syscall and can be read back through mmaped ring or syscall.
Kevent update (i.e. readiness switching) happens directly from internals of the appropriate state machine of the underlying subsytem (like network, filesystem, timer or any other).I will put that text into introduction message.
Thanks.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 1 line · Pavel Machek
Hi!
Hi Pavel.
Quoted 8 lines
Quoted 4 lines
Kevent is a generic subsytem which allows to handle event notifications. It supports both level and edge triggered events. It is similar to poll/epoll in some cases, but it is more scalable, it is faster and allows to work with essentially eny kind of events.
Quantifying "how much more scalable" would be nice, as would be some example where it is useful. ("It makes my webserver twice as fast on monster 64-cpu box").
Trivial kevent web-server can handle 3960+ req/sec on Xeon 2.4Ghz with 1Gb RAM, epoll based - 2200-2500 req/sec.
100 Mbit wire is filled almost 100% (10582.7 KB/s of data without TCP and below headers).
More benchmarks created by me and Johann Borck can be found on project's homepage as long as all my sources used in tests.
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
Hallo, Evgeniy Polyakov.
On 2006-11-01, you wrote:
[]
Quoted 5 lines
Quoted 3 lines
Quantifying "how much more scalable" would be nice, as would be some example where it is useful. ("It makes my webserver twice as fast on monster 64-cpu box").
Trivial kevent web-server can handle 3960+ req/sec on Xeon 2.4Ghz with
[...]
Seriously. I'm seeing that patches also. New, shiny, always ready "for inclusion". But considering kernel (linux in this case) as not thing for itself, i want to ask following question.
Where's real-life application to do configure && make && make install?
There were some comments about laking much of such programs, answers were "was in prev. e-mail", "need to update them", something like that. "Trivial web server" sources url, mentioned in benchmark isn't pointed in patch advertisement. If it was, should i actually try that new *trivial* wheel?
Saying that, i want to give you some short examples, i know. *Linux kernel <-> userspace*:
o Alexey Kuznetsov networking <-> (excellent) iproute set of utilities;
o Maxim Krasnyansky tun net driver <-> vtun daemon application;
*Glibc with mister Drepper* has huge set of tests, please search for `tst*' files in the sources.
To make a little hint to you, Evgeniy, why don't you find a little animal in the open source zoo to implement little interface to proposed kernel subsystem and then show it to The Big Jury (not me), we have here? And i can not see, how you've managed to implement something like that having almost nothing on the test basket. Very *suspicious* ch.
One, that comes in mind is lighthttpd <http://www.lighttpd.net/>.
It had sub-interface for event systems like select,poll,epoll, when i checked its sources last time. And it is mature, btw.
Cheers.
[ -*- OT -*- ] [ I wouldn't write all this, unless saw your opinion about the ] [ reportbug (part of the Debian Bug Tracking System) this week. ] [ While i'm nobody here, imho, the first thing about good programmer ] [ must be, that he is excellent user. ]
____
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 2 lines · Oleg Verych
Hallo, Evgeniy Polyakov.
Hello, Oleg.
Quoted 6 lines · you
[]
Quoted 4 lines
Quoted 3 lines
Quantifying "how much more scalable" would be nice, as would be some example where it is useful. ("It makes my webserver twice as fast on monster 64-cpu box").
Quoted 8 lines
Quoted 1 line
Trivial kevent web-server can handle 3960+ req/sec on Xeon 2.4Ghz with
[...]
Seriously. I'm seeing that patches also. New, shiny, always ready "for inclusion". But considering kernel (linux in this case) as not thing for itself, i want to ask following question.
Where's real-life application to do configure && make && make install?
Your real life or mine as developer?
I fortunately do not know anything about your real life, but my real life applications can be found on project's homepage.
There is a link to archive there, where you can find plenty of sources. You likely do not know, but it is a bit risky business to patch all existing applications to show that approach is correct, if implementation is not completed.
You likely do not know, but after I first time announced kevents in February I changed interfaces 4 times - and it is just interfaces, not including numerous features added/removed by developer's requests.
Quoted 5 lines
There were some comments about laking much of such programs, answers were "was in prev. e-mail", "need to update them", something like that. "Trivial web server" sources url, mentioned in benchmark isn't pointed in patch advertisement. If it was, should i actually try that new *trivial* wheel?
Answer is trivial - there is archive where one can find a source code (filenames are posted regulary). Should I create a rpm? For what glibc version?
Quoted 3 lines
Saying that, i want to give you some short examples, i know. *Linux kernel <-> userspace*:
o Alexey Kuznetsov networking <-> (excellent) iproute set of utilities;
iproute documentation was way too bad when Alexey presented it first time :)
Quoted 4 lines
o Maxim Krasnyansky tun net driver <-> vtun daemon application;
*Glibc with mister Drepper* has huge set of tests, please search for `tst*' files in the sources.
Btw, show me splice() 'shiny' application? Does lighttpd use it? Or move_pages().
Quoted 6 lines
To make a little hint to you, Evgeniy, why don't you find a little animal in the open source zoo to implement little interface to proposed kernel subsystem and then show it to The Big Jury (not me), we have here? And i can not see, how you've managed to implement something like that having almost nothing on the test basket. Very *suspicious* ch.
There are always people who do not like something, what can I do with it? I present the code, we discuss it, I ask for inclusion (since it is the only way to get feedback), something requires changes, it is changed and so on - it is development process.
I created 'little animal in the open source zoo' by myself to show how simple kevents are.
Quoted 3 lines
One, that comes in mind is lighthttpd <http://www.lighttpd.net/>.
It had sub-interface for event systems like select,poll,epoll, when i checked its sources last time. And it is mature, btw.
As I already told several times, I changed only interfaces 4 times already, since no one seems to know what we really want and how interface should look like. You suggest to patch lighttpd? Well, it is doable, but then I will be asked to change apache and nginx. And then someone will suggest to change order of parameters. Will you help me rewrite userspace? No, you will not. You asks for something without providing anything back (not getting into account code, but discussion, ideas, testing time, nothing), and you do it in ultimate manner. Btw, kevent also support AIO notifications - do you suggest to patch reactor/proactor for tests?
It supports network AIO - do you suggest to write support for that into apache?
What about timers? It is possible to rewrite all POSIX timers users to usem instead.
There is feature request for userspace events and singal delivery - what to do with that?
I created trivial web servers, which send single static page and use various event handling schemes, and I test new subsystem with new tools, when tests are completed and all requested features are implemented it is time to work on different more complex users.
So let's at least complete what we have right now, so no developer's efforts could be wasted writing empty chars in various places.
Quoted 8 lines
Cheers.
[ -*- OT -*- ] [ I wouldn't write all this, unless saw your opinion about the ] [ reportbug (part of the Debian Bug Tracking System) this week. ] [ While i'm nobody here, imho, the first thing about good programmer ] [ must be, that he is excellent user. ]____
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 71 lines · Evgeniy Polyakov
Quoted 2 lines · Oleg Verych
Hallo, Evgeniy Polyakov.
Hello, Oleg.
Quoted 6 lines · you
[]
Quoted 4 lines
Quoted 3 lines
Quantifying "how much more scalable" would be nice, as would be some example where it is useful. ("It makes my webserver twice as fast on monster 64-cpu box").
Quoted 8 lines
Quoted 1 line
Trivial kevent web-server can handle 3960+ req/sec on Xeon 2.4Ghz with
[...]
Seriously. I'm seeing that patches also. New, shiny, always ready "for inclusion". But considering kernel (linux in this case) as not thing for itself, i want to ask following question.
Where's real-life application to do configure && make && make install?
Your real life or mine as developer?
I fortunately do not know anything about your real life, but my real life applications can be found on project's homepage.
There is a link to archive there, where you can find plenty of sources. You likely do not know, but it is a bit risky business to patch all existing applications to show that approach is correct, if implementation is not completed.
You likely do not know, but after I first time announced kevents in February I changed interfaces 4 times - and it is just interfaces, not including numerous features added/removed by developer's requests.Quoted 5 lines
There were some comments about laking much of such programs, answers were "was in prev. e-mail", "need to update them", something like that. "Trivial web server" sources url, mentioned in benchmark isn't pointed in patch advertisement. If it was, should i actually try that new *trivial* wheel?
Answer is trivial - there is archive where one can find a source code (filenames are posted regulary). Should I create a rpm? For what glibc version?
Quoted 3 lines
Saying that, i want to give you some short examples, i know. *Linux kernel <-> userspace*:
o Alexey Kuznetsov networking <-> (excellent) iproute set of utilities;iproute documentation was way too bad when Alexey presented it first time :)
Quoted 4 lines
o Maxim Krasnyansky tun net driver <-> vtun daemon application;
*Glibc with mister Drepper* has huge set of tests, please search for `tst*' files in the sources.
Btw, show me
splice()'shiny' application? Does lighttpd use it? Ormove_pages().Quoted 6 lines
To make a little hint to you, Evgeniy, why don't you find a little animal in the open source zoo to implement little interface to proposed kernel subsystem and then show it to The Big Jury (not me), we have here? And i can not see, how you've managed to implement something like that having almost nothing on the test basket. Very *suspicious* ch.
There are always people who do not like something, what can I do with it? I present the code, we discuss it, I ask for inclusion (since it is the only way to get feedback), something requires changes, it is changed and so on - it is development process.
I created 'little animal in the open source zoo' by myself to show how simple kevents are.
Quoted 7 lines
Quoted 3 lines
One, that comes in mind is lighthttpd <http://www.lighttpd.net/>.
It had sub-interface for event systems like select,poll,epoll, when i checked its sources last time. And it is mature, btw.As I already told several times, I changed only interfaces 4 times already, since no one seems to know what we really want and how interface should look like.
Indesiciveness has certainly been an issue here, but I remember akpm and Ulrich both giving concrete suggestions. I was particularly interested in Andrew's request to explain and justify the differences between kevent and BSD's kqueue interface. Was there a discussion that I missed? I am very interested to see your work on this mechanism merged, because you've clearly emphasized performance and shown impressive results. But it seems like we lose out on a lot by throwing out all the applications that already use kqueue.
NATE
A message that is not in the archive (sent off-list or not yet received): [email protected]
re: [take22 0/4] kevent: Generic event handling mechanism.
performance is great, and we are exciting at the result.
I want to know why there can be so much improvement, can we improve epoll too ?
for Kevent, I am more interesting at a universal event machanism. In one interface, we can wait for timer event, socket event, and disk AIO event, this can make the userland application easier to handle multiple event.
2006/11/2, Nate Diller <[email protected]>:
Quoted 96 lines · Evgeniy Polyakov
Quoted 71 lines · Oleg Verych
Quoted 2 lines
Hallo, Evgeniy Polyakov.
Hello, Oleg.
Quoted 6 lines · you
[]
Quoted 4 lines
Quoted 3 lines
Quantifying "how much more scalable" would be nice, as would be some example where it is useful. ("It makes my webserver twice as fast on monster 64-cpu box").
Quoted 8 lines
Quoted 1 line
Trivial kevent web-server can handle 3960+ req/sec on Xeon 2.4Ghz with
[...]
Seriously. I'm seeing that patches also. New, shiny, always ready "for inclusion". But considering kernel (linux in this case) as not thing for itself, i want to ask following question.
Where's real-life application to do configure && make && make install?
Your real life or mine as developer?
I fortunately do not know anything about your real life, but my real life applications can be found on project's homepage.
There is a link to archive there, where you can find plenty of sources. You likely do not know, but it is a bit risky business to patch all existing applications to show that approach is correct, if implementation is not completed.
You likely do not know, but after I first time announced kevents in February I changed interfaces 4 times - and it is just interfaces, not including numerous features added/removed by developer's requests.Quoted 5 lines
There were some comments about laking much of such programs, answers were "was in prev. e-mail", "need to update them", something like that. "Trivial web server" sources url, mentioned in benchmark isn't pointed in patch advertisement. If it was, should i actually try that new *trivial* wheel?
Answer is trivial - there is archive where one can find a source code (filenames are posted regulary). Should I create a rpm? For what glibc version?
Quoted 3 lines
Saying that, i want to give you some short examples, i know. *Linux kernel <-> userspace*:
o Alexey Kuznetsov networking <-> (excellent) iproute set of utilities;iproute documentation was way too bad when Alexey presented it first time :)
Quoted 4 lines
o Maxim Krasnyansky tun net driver <-> vtun daemon application;
*Glibc with mister Drepper* has huge set of tests, please search for `tst*' files in the sources.
Btw, show me
splice()'shiny' application? Does lighttpd use it? Ormove_pages().Quoted 6 lines
To make a little hint to you, Evgeniy, why don't you find a little animal in the open source zoo to implement little interface to proposed kernel subsystem and then show it to The Big Jury (not me), we have here? And i can not see, how you've managed to implement something like that having almost nothing on the test basket. Very *suspicious* ch.
There are always people who do not like something, what can I do with it? I present the code, we discuss it, I ask for inclusion (since it is the only way to get feedback), something requires changes, it is changed and so on - it is development process.
I created 'little animal in the open source zoo' by myself to show how simple kevents are.Quoted 7 lines
Quoted 3 lines
One, that comes in mind is lighthttpd <http://www.lighttpd.net/>.
It had sub-interface for event systems like select,poll,epoll, when i checked its sources last time. And it is mature, btw.As I already told several times, I changed only interfaces 4 times already, since no one seems to know what we really want and how interface should look like.
Indesiciveness has certainly been an issue here, but I remember akpm and Ulrich both giving concrete suggestions. I was particularly interested in Andrew's request to explain and justify the differences between kevent and BSD's kqueue interface. Was there a discussion that I missed? I am very interested to see your work on this mechanism merged, because you've clearly emphasized performance and shown impressive results. But it seems like we lose out on a lot by throwing out all the applications that already use kqueue.
NATE
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 3 lines · zhou drangon
performance is great, and we are exciting at the result.
I want to know why there can be so much improvement, can we improve epoll too ?
Why did you remove most of CC addresses but lkml ? Dont do that please...
Good question :)
Hum, I think I can look into epoll and see how it can be improved (if necessary)
This is not to say we dont need kevent ! Please Evgeniy continue your work !
Just to remind you that according to http://www.xmailserver.org/linux-patches/nio-improve.html David Libenzi had to wait 18 months before epoll being officialy added into kernel.
At that time, many applications were using epoll, and we were patching our kernels for that.
I cooked a very simple program (attached in this mail), using pipes and epoll, and got 250.000 events received per second on an otherwise lightly loaded machine (dual opteron 246 , 2GHz, 1MB cache per cpu) with 10.000 pipes (20.000 handles)
It could be nice to add support for other event providers in this program (AF_INET & AF_UNIX sockets for example), and also add support for kevent, so that we really can compare epoll/kevent without a complex setup. I should extend the program to also add/remove sources during lifetime, not only insert at setup time.
# gcc -O2 -o epoll_pipe_bench epoll_pipe_bench.c -lpthread # ulimit -n 1000000 # epoll_pipe_bench -n 10000 ^C after a while...
oprofile results say that ep_poll_callback() and sys_epoll_wait() use 20% of cpu time. Even if we gain a two factor in cpu time or cache usage, we wont eliminate other costs...
oprofile results gave :
Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 50000
samples % symbol name 2015420 11.1309 ep_poll_callback 1867431 10.3136 pipe_writev 1791872 9.8963 sys_epoll_wait 1357297 7.4962 fget_light 1277515 7.0556 pipe_readv 998447 5.5143 current_fs_time 801597 4.4271 __mark_inode_dirty 755268 4.1713 __wake_up 587065 3.2423 __write_lock_failed 582931 3.2195 system_call 297132 1.6410 iov_fault_in_pages_read 296136 1.6355 sys_write 290106 1.6022 __wake_up_common 270692 1.4950 bad_pipe_w 261516 1.4443 do_pipe 257208 1.4205 tg3_start_xmit_dma_bug 254917 1.4079 pipe_poll 252925 1.3969 copy_user_generic_c 234212 1.2935 generic_pipe_buf_map 228659 1.2629 ret_from_sys_call 212541 1.1738 sysret_check 166529 0.9197 sys_read 160038 0.8839 vfs_write 151091 0.8345 pipe_ioctl 136301 0.7528 file_update_time 107173 0.5919 tg3_poll 77846 0.4299 ipt_do_table 75081 0.4147 schedule 73059 0.4035 vfs_read 69787 0.3854 get_task_comm 63923 0.3530 memcpy 60019 0.3315 touch_atime 57490 0.3175 eventpoll_release_file 56152 0.3101 tg3_write_flush_reg32 54468 0.3008 rw_verify_area 47833 0.2642 generic_pipe_buf_unmap 47777 0.2639 __switch_to 44106 0.2436 bad_pipe_r 41824 0.2310 proc_nr_files 41319 0.2282 pipe_iov_copy_from_user
Eric
/*
* How to stress epoll
*
* This program uses many pipes and two threads.
* First we open as many pipes we can. (see ulimit -n)
* Then we create a worker thread.
* The worker thread will send bytes to random pipes.
* The main thread uses epoll to collect ready pipes and read them.
* Each second, a number of collected bytes is printed on stderr *
* Usage : epoll_bench [-n X]
*/
#include <pthread.h>
#include <stdlib.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <sys/epoll.h>
#include <signal.h>
#include <unistd.h>
#include <sys/time.h>
int nbpipes = 1024;
struct pipefd {
int fd[2];
} *tab;int epoll_fd;
static int alloc_pipes()
{
int i;
epoll_fd = epoll_create(nbpipes);
if (epoll_fd == -1) {
perror("epoll_create");
return -1;
}
tab = malloc(sizeof(struct pipefd) * nbpipes);
if (tab ==NULL) {
perror("malloc");
return -1;
}
for (i = 0 ; i < nbpipes ; i++) {
struct epoll_event ev;
if (pipe(tab[i].fd) == -1)
break;
ev.events = EPOLLIN | EPOLLOUT | EPOLLHUP | EPOLLPRI | EPOLLET;
ev.data.u64 = (uint64_t)i;
epoll_ctl(epoll_fd, EPOLL_CTL_ADD, tab[i].fd[0], &ev);
}
nbpipes = i;
printf("%d pipes setup\n", nbpipes);
return 0;
}unsigned long nbhandled;
static void timer_func()
{
char buffer[32];
size_t len;
static unsigned long old;
unsigned long delta = nbhandled - old;
old = nbhandled;
len = sprintf(buffer, "%lu\n", delta);
write(2, buffer, len);
}static void timer_setup()
{
struct itimerval it;
struct sigaction sg;
memset(&sg, 0, sizeof(sg));
sg.sa_handler = timer_func;
sigaction(SIGALRM, &sg, 0);
it.it_interval.tv_sec = 1;
it.it_interval.tv_usec = 0;
it.it_value.tv_sec = 1;
it.it_value.tv_usec = 0;
if (setitimer(ITIMER_REAL, &it, 0))
perror("setitimer");
}static void * worker_thread_func(void *arg)
{
int fd;
char c = 1;
for (;;) {
fd = rand() % nbpipes;
write(tab[fd].fd[1], &c, 1);
}
}int main(int argc, char *argv[])
{
char buff[1024];
pthread_t tid;
int c;
while ((c = getopt(argc, argv, "n:")) != EOF) {
if (c == 'n') nbpipes = atoi(optarg);
}
alloc_pipes();
pthread_create(&tid, NULL, worker_thread_func, (void *)0);
timer_setup();
for (;;) {
struct epoll_event events[128];
int nb = epoll_wait(epoll_fd, events, 128, 10000);
int i, fd;
for (i = 0 ; i < nb ; i++) {
fd = tab[events[i].data.u64].fd[0];
if (read(fd, buff, 1024) > 0)
nbhandled++;
}
}
}Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 8 lines · Eric Dumazet
Quoted 4 lines · zhou drangon
performance is great, and we are exciting at the result.
I want to know why there can be so much improvement, can we improve epoll too ?
Why did you remove most of CC addresses but lkml ? Dont do that please...
Sure, since for example I'm not subscribed (fortunately) to lkml, and I think you want me to answer the question too...
Quoted 4 lines
Good question :)
Hum, I think I can look into epoll and see how it can be improved (if necessary)
epoll can not be improved, since the whole polling is designed to have several layers of dereferencing, kevent simplifies that chain noticebly.
Quoted 1 line
This is not to say we dont need kevent ! Please Evgeniy continue your work !
I will :)
Quoted 3 lines
Just to remind you that according to
http://www.xmailserver.org/linux-patches/nio-improve.html David Libenzi had to wait 18 months before epoll being officialy added into kernel.
kevent exists for about 10 month. We have plenty of time :)
Quoted 8 lines
At that time, many applications were using epoll, and we were patching our kernels for that.
I cooked a very simple program (attached in this mail), using pipes and epoll, and got 250.000 events received per second on an otherwise lightly loaded machine (dual opteron 246 , 2GHz, 1MB cache per cpu) with 10.000 pipes (20.000 handles)
pipes will work with kevent's poll mechanisms only, so there will not be any performance gain at all since it is essentially the same as epoll design with waiting and rescheduling (all my measurements with epoll vs. kevent_poll always showed the same rates), pipes require the same notifications as sockets for maximum perfomance.
I've put it into todo list.
Quoted 5 lines
It could be nice to add support for other event providers in this program (
AF_INET&AF_UNIXsockets for example), and also add support for kevent, so that we really can compare epoll/kevent without a complex setup. I should extend the program to also add/remove sources during lifetime, not only insert at setup time.
If there would exist sockets support, then I could patch it to work with kevents.
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 5 lines · Evgeniy Polyakov
pipes will work with kevent's poll mechanisms only, so there will not be
any performance gain at all since it is essentially the same as epoll
design with waiting and rescheduling (all my measurements with epoll vs.kevent_pollalways showed the same rates), pipes require the same notifications as sockets for maximum perfomance.
I've put it into todo list.
Evgeniy I think this part is *important*. I think most readers of lkml are not aware of exact mechanisms used in epoll, kevent poll, and 'kevent'
I dont understand why epoll is bad for you, since for me, ep_poll_callback() is fast enough, even if we can make it touch less cache lines if reoredering 'struct epitem' correctly. My epoll_pipe_bench doesnt change the rescheduling rate of the test machine.
Could you in your home page add some doc that clearly show the path taken for those 3 mechanisms and different events sources (At least sockets)
Eric
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Evgeniy Polyakov
pipes will work with kevent's poll mechanisms only, so there will not be any performance gain at all since it is essentially the same as epoll design with waiting and rescheduling (all my measurements with epoll vs.
kevent_pollalways showed the same rates), pipes require the same notifications as sockets for maximum perfomance.
I've put it into todo list.Evgeniy I think this part is *important*. I think most readers of lkml are not aware of exact mechanisms used in epoll, kevent poll, and 'kevent'
Quoted 7 lines
I dont understand why epoll is bad for you, since for me,
ep_poll_callback()is fast enough, even if we can make it touch less cache lines if reoredering 'struct epitem' correctly. Myepoll_pipe_benchdoesnt change the rescheduling rate of the test machine.Could you in your home page add some doc that clearly show the path taken for those 3 mechanisms and different events sources (At least sockets)
It is.
"It [kevent] supports socket notifications (accept, sending and receiving), network AIO (aio_send(), aio_recv() and aio_sendfile()), inode notifications (create/remove), generic poll()/select() notifications and timer notifications."
In each patch I give a short description and socket notification patch
By poll design we have to setup following data:poll_table_struct, which contains a callback
that callback will be called in each sys_poll()->drivers_poll()->poll_wait(),
callback will allocate new private structure, which must have wait_queue_t (it's callback will be invoked each time wake_up() is called for given wait_queue_head), which should be linked to the given wait_queue_head.
Kevent has different approach: so called origins (files, inodes, sockets and so on) have a queues of userspace requests, for example socket origin can only have a queue which will contain one of the following events ($type.$event): socket.send, socket.recv, socket.accept. So when new data has arrived, appropriate event is marked as ready and moved into ready queue (very short operations) and requested thread is awakened, which can then get ready events and requeue them back (or remove, depending on flags). There are no allocations in kevent_get_events() (epoll_wait() does not have it too), no potentially long lists of wait_queue linked to the same wait_queue_head_t, which is traversed each time we call wake_up(), it has much smaller memory footprint compared to epoll (there is only one kevent compared to epitem and eppoll_entry).
Quoted 1 line
Eric
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 8 lines · Evgeniy Polyakov
By poll design we have to setup following data:
poll_table_struct, which contains a callback
that callback will be called in each sys_poll()->drivers_poll()->poll_wait(),
callback will allocate new private structure, which must havewait_queue_t(it's callback will be invoked each timewake_up()is called for givenwait_queue_head), which should be linked to the givenwait_queue_head.
In epoll case, the setup is done at epoll_ctl() time, not for each event received by the consumer like poll()/select()
As for wake_up() overhead, I feel it's necessary if we want to wakeup a consumer (tell him some events are availlable)
I suspect benchmark results might depend for a large part on some 'features' of the scheduler, or the number of events provider and CPU cache size, not on inherent limitations of epoll.
I changed a litle bit epoll_pipe_bench and we can see effects of scheduling on the overall rate of events per second.
I can now receive 350.000 events per second, instead of 230.000
Tests done on my laptop : a UP machine (Intel(R) Pentium(R) M processor 1.60GHz) (Dell latitude D610)
# ./epoll_pipe_bench -n 1000 -l 1 1000 pipes setup 223065 evts/sec 1.00563 samples per call 228508 evts/sec 1.00277 samples per call 229076 evts/sec 1.00184 samples per call 227860 evts/sec 1.00191 samples per call 229498 evts/sec 1.00153 samples per call 228027 evts/sec 1.00136 samples per call 229465 evts/sec 1.00122 samples per call 227845 evts/sec 1.00132 samples per call 227456 evts/sec 1.00118 samples per call 228355 evts/sec 1.00113 samples per call
# ./epoll_pipe_bench -n 1000 -l 10 1000 pipes setup 328599 evts/sec 314.75 samples per call 344947 evts/sec 314.447 samples per call 342844 evts/sec 314.185 samples per call 345486 evts/sec 314.013 samples per call 345144 evts/sec 314.079 samples per call 344270 evts/sec 313.989 samples per call 344249 evts/sec 314.004 samples per call 320577 evts/sec 326.967 samples per call 313990 evts/sec 343.926 samples per call 313578 evts/sec 359.034 samples per call
In oprofile, It's not obvious epoll is responsible for cpu costs...
CPU: PIII, speed 1600 MHz (estimated)
Counted CPU_CLK_UNHALTED events (clocks processor is not halted) with a unit mask of 0x00 (No unit mask) count 50000
samples % symbol name 206966 7.9385 sysenter_past_esp 158337 6.0733 _spin_lock_irqsave 143339 5.4980 pipe_writev 137434 5.2715 fget_light 132293 5.0743 _spin_unlock_irqrestore 109593 4.2036 pipe_readv 102669 3.9380 ep_poll_callback 99801 3.8280 dnotify_parent 99388 3.8122 sys_epoll_wait 98710 3.7862 vfs_write 94560 3.6270 _write_lock_irqsave 91774 3.5201 current_fs_time 90541 3.4728 pipe_poll 89153 3.4196 _spin_lock 85150 3.2661 __wake_up 83213 3.1918 __wake_up_common 72913 2.7967 try_to_wake_up 67996 2.6081 rw_verify_area 64243 2.4641 file_update_time 54211 2.0793 vfs_read
But but but ! If I add more pipes, results are reversed, because CPU cache is not large enough : It's better to deliver events as fast as possible to consumer to keep hot caches (but only on UP machine, and because my test prog is threaded. If it was using two process, context switch would be more expensive)
# ./epoll_pipe_bench -n 10000 -l 1 10000 pipes setup 171444 evts/sec 1 samples per call 174556 evts/sec 1 samples per call 173976 evts/sec 1 samples per call 174715 evts/sec 1.00003 samples per call 173215 evts/sec 1.00478 samples per call 174930 evts/sec 1.00397 samples per call
# ./epoll_pipe_bench -n 10000 -l 10 10000 pipes setup 149701 evts/sec 759.904 samples per call 153476 evts/sec 767.537 samples per call 149217 evts/sec 767.585 samples per call 152396 evts/sec 763.624 samples per call 153517 evts/sec 762.215 samples per call 148026 evts/sec 763.434 samples per call
CPU: PIII, speed 1600 MHz (estimated)
Counted CPU_CLK_UNHALTED events (clocks processor is not halted) with a unit mask of 0x00 (No unit mask) count 50000
samples % symbol name 404924 12.4647 pipe_poll 263399 8.1081 pipe_writev 259230 7.9798 rw_verify_area 246368 7.5839 fget_light 242360 7.4605 sys_epoll_wait 174913 5.3843 kmap_atomic 168623 5.1907 __wake_up_common 147661 4.5454 ep_poll_callback 137003 4.2173 pipe_readv 128799 3.9648 _spin_lock_irqsave 100015 3.0787 sysenter_past_esp 74031 2.2789 __copy_to_user_ll 72857 2.2427 file_update_time 69523 2.1401 vfs_write 69444 2.1377 mutex_lock
I also added a -f flag, to bypass epoll completely and measure the pipe/read/write overhead/performance.
# ./epoll_pipe_bench -n 10000 -f 10000 pipes setup 300230 evts/sec 309770 evts/sec 264426 evts/sec 265842 evts/sec 265551 evts/sec 266814 evts/sec 266551 evts/sec 264415 evts/sec
Re: [take22 0/4] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovQuoted 4 lines · Evgeniy Polyakov
If there would exist sockets support, then I could patch it to work with kevents.
OK I post here my last version of epoll_bench.
It works with pipes (default),
or AF_UNIX socketpair() (option -u),
or AF_INET sockets (on loopback device), (option -i)
Only one machine involved (so no real ethernet trafic, and a limit on max number of AF_INET sockets since I use one listener 'only')
Option -f ask to bypass epoll.
On a dual opteron 246 machine (2GHZ cpu, 1MB of cache on each cpu, but somewhat busy 2.6.18 )
Perf for 2000 concurrent streams is :
259643 evts/sec for pipes
170188 evts/sec for AF_UNIX sockets (-u)
58771 evts/sec for AF_INET sockets (-i)
69475 evts/sec for AF_INET and no epoll gathering at all. (-i -f)
I believe difference between AF_INET sockets and other streams come from synchronous/asynchronous wakeups :
I added counters of context switches per second and also number of events handled per epoll_wait() call, and we can see that in AF_INET case, the consumer is awaken more often. That means lower latency, but less bandwidth alas.
Detailed Results :
pipe
# ./epoll_bench -n 2000 2000 handles setup 255320 evts/sec 362.074 samples per call 254054 evts/sec 10473 ctxt/sec 381.569 samples per call 249868 evts/sec 9155 ctxt/sec 407.461 samples per call 181010 evts/sec 22656 ctxt/sec 420.36 samples per call 233368 evts/sec 8565 ctxt/sec 348.773 samples per call 284682 evts/sec 11114 ctxt/sec 299.987 samples per call 292485 evts/sec 10235 ctxt/sec 279.042 samples per call 279194 evts/sec 10760 ctxt/sec 267.694 samples per call 267917 evts/sec 12035 ctxt/sec 264.106 samples per call 291450 evts/sec 11024 ctxt/sec 247.028 samples per call 266837 evts/sec 11732 ctxt/sec 241.915 samples per call 272762 evts/sec 11492 ctxt/sec 247.629 samples per call 253756 evts/sec 11011 ctxt/sec 253.395 samples per call 251250 evts/sec 9912 ctxt/sec 259.88 samples per call 260706 evts/sec 10754 ctxt/sec 265.079 samples per call Avg: 259643 evts/sec
AF_UNIX
# ./epoll_bench -n 2000 -u 2000 handles setup 264827 evts/sec 6.01538 samples per call 259241 evts/sec 15682 ctxt/sec 5.70332 samples per call 262266 evts/sec 17072 ctxt/sec 5.64829 samples per call 262730 evts/sec 16744 ctxt/sec 5.43087 samples per call 253212 evts/sec 17343 ctxt/sec 5.14736 samples per call 255219 evts/sec 17579 ctxt/sec 5.0197 samples per call 166655 evts/sec 13090 ctxt/sec 5.27575 samples per call 111348 evts/sec 10127 ctxt/sec 5.61362 samples per call 104812 evts/sec 9476 ctxt/sec 5.93361 samples per call 95897 evts/sec 8876 ctxt/sec 6.22481 samples per call 97096 evts/sec 9372 ctxt/sec 6.51874 samples per call 113808 evts/sec 11142 ctxt/sec 6.86422 samples per call 102509 evts/sec 10035 ctxt/sec 7.17618 samples per call 100318 evts/sec 9731 ctxt/sec 7.47926 samples per call 102893 evts/sec 9458 ctxt/sec 7.78841 samples per call Avg: 170188 evts/sec
AF_INET
# ./epoll_bench -n 2000 -i 2000 handles setup 69210 evts/sec 2.97224 samples per call 59436 evts/sec 12876 ctxt/sec 5.48675 samples per call 60722 evts/sec 12093 ctxt/sec 8.03185 samples per call 60583 evts/sec 14582 ctxt/sec 10.5644 samples per call 58192 evts/sec 12066 ctxt/sec 12.999 samples per call 54291 evts/sec 10613 ctxt/sec 15.2398 samples per call 47978 evts/sec 10942 ctxt/sec 17.2222 samples per call 59009 evts/sec 13692 ctxt/sec 19.6426 samples per call 58248 evts/sec 15099 ctxt/sec 22.0306 samples per call 58708 evts/sec 15118 ctxt/sec 24.4497 samples per call 58613 evts/sec 14608 ctxt/sec 26.816 samples per call 58490 evts/sec 13593 ctxt/sec 29.1708 samples per call 59108 evts/sec 15078 ctxt/sec 31.5557 samples per call 59636 evts/sec 15053 ctxt/sec 33.9292 samples per call 59355 evts/sec 15531 ctxt/sec 36.2914 samples per call Avg: 58771 evts/sec
The last sample shows that epoll overhead is very small indeed, since disabling it doesnt boost AF_INET perf at all. AF_INET + no epoll
# ./epoll_bench -n 2000 -i -f 2000 handles setup 79939 evts/sec 78468 evts/sec 9989 ctxt/sec 73153 evts/sec 10207 ctxt/sec 73668 evts/sec 10163 ctxt/sec 73667 evts/sec 20084 ctxt/sec 74106 evts/sec 10068 ctxt/sec 73442 evts/sec 10119 ctxt/sec 74220 evts/sec 10122 ctxt/sec 74367 evts/sec 10097 ctxt/sec 64402 evts/sec 47873 ctxt/sec 53555 evts/sec 58733 ctxt/sec 46000 evts/sec 48984 ctxt/sec 67052 evts/sec 21006 ctxt/sec 68460 evts/sec 12344 ctxt/sec 67629 evts/sec 10655 ctxt/sec Avg: 69475 evts/sec
I add here oprofile results for the AF_INET (with epoll) test
CPU: AMD64 processors, speed 1992.3 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 50000
samples % symbol name 1127210 9.1969 tcp_sendmsg 692516 5.6502 fget_light 598653 4.8844 lock_sock 575396 4.6946 __tcp_push_pending_frames 364699 2.9756 tcp_ack 364352 2.9727 tcp_v4_rcv 356383 2.9077 ipt_do_table 324388 2.6467 do_sync_write 257869 2.1039 wait_on_retry_sync_kiocb 255977 2.0885 inet_sk_rebuild_header 255171 2.0819 tcp_recvmsg 249554 2.0361 copy_user_generic_c 232551 1.8974 tcp_transmit_skb 215471 1.7580 release_sock 208563 1.7017 tcp_window_allows 194983 1.5909 kfree 186842 1.5244 system_call 180074 1.4692 kmem_cache_free 160799 1.3120 ep_poll_callback 159235 1.2992 update_send_head 134291 1.0957 sys_epoll_wait 133670 1.0906 ip_queue_xmit 132829 1.0837 ret_from_sys_call 129348 1.0553 __mod_timer 129258 1.0546 sys_write 117884 0.9618 tcp_rcv_established 115181 0.9398 tcp_poll 102805 0.8388 memcpy 99017 0.8079 skb_clone 91125 0.7435 vfs_write 87087 0.7105 __kfree_skb 75387 0.6151 tcp_mss_to_mtu 72483 0.5914 init_or_fini 72207 0.5891 do_sync_read 72054 0.5879 tcp_ioctl 70555 0.5757 local_bh_enable_ip 70001 0.5711 tg3_start_xmit_dma_bug 69914 0.5704 ip_local_deliver 69002 0.5630 tcp_v4_do_rcv 68681 0.5604 dev_queue_xmit 68411 0.5582 do_ip_getsockopt 68235 0.5567 skb_copy_datagram_iovec 66489 0.5425 local_bh_enable
oprofile results for the pipe case :
(where epoll is not noise)
CPU: AMD64 processors, speed 1992.3 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 50000
samples % symbol name 1346203 12.2441 ep_poll_callback 1220770 11.1033 pipe_writev 1020377 9.2806 sys_epoll_wait 991913 9.0218 pipe_readv 779611 7.0908 fget_light 638929 5.8113 __wake_up 625332 5.6876 current_fs_time 486427 4.4242 __mark_inode_dirty 385763 3.5086 __write_lock_failed 217402 1.9773 system_call 175292 1.5943 sys_write 153698 1.3979 __wake_up_common 153242 1.3938 bad_pipe_w 143597 1.3061 generic_pipe_buf_map 140814 1.2807 pipe_poll 130028 1.1826 ret_from_sys_call 122930 1.1181 do_pipe 122359 1.1129 copy_user_generic_c 107443 0.9772 file_update_time 106037 0.9644 sysret_check 101256 0.9210 sys_read 99176 0.9020 iov_fault_in_pages_read 96823 0.8806 generic_pipe_buf_unmap 96675 0.8793 vfs_write 64635 0.5879 rw_verify_area 62997 0.5730 pipe_ioctl 60983 0.5547 tg3_start_xmit_dma_bug 59624 0.5423 get_task_comm 49573 0.4509 tg3_poll 46041 0.4188 schedule 44321 0.4031 vfs_read 35962 0.3271 eventpoll_release_file 30267 0.2753 tg3_write_flush_reg32 29395 0.2674 ipt_do_table 27683 0.2518 page_to_pfn 27492 0.2500 touch_atime 24921 0.2267 memcpy
Eric
/*
* How to stress epoll
*
* This program uses many pipes|sockets and two threads.
* First we open as many pipes|sockets we can. (see ulimit -n)
* Then we create a worker thread.
* The worker thread will send bytes to random streams.
* The main thread uses epoll to collect ready events and clear them, reading streams.
* Each second, a number of collected events is printed on stderr
* After one minute, program prints an average value and stops.
*
* Usage : epoll_bench [-f] [-{u|i}] [-n X]
* -f : No epoll loop, just feed streams in a cyclic manner * -u : Use AF_UNIX sockets (instead of pipes) * -i : Use AF_INET sockets
*/
#include <pthread.h>
#include <stdlib.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <sys/epoll.h>
#include <signal.h>
#include <unistd.h>
#include <sys/time.h>
#include <sys/types.h>
#include <sys/socket.h>
# include <netinet/in.h> #include <fcntl.h> #include <sys/ioctl.h>
int nbhandles = 1024;
int time_test = 15;
unsigned long nbhandled;
unsigned long epw_samples;
unsigned long epw_samples_cnt;
struct pipefd {
int fd[2];
} *tab;int epoll_fd;
int fflag;
int afunix;
int afinet;
static int alloc_streams()
{
int i;
int listen_sock;
struct sockaddr_in me, to;
socklen_t namelen;
int on = 1;
int off = 0;
if (!fflag) {
epoll_fd = epoll_create(nbhandles);
if (epoll_fd == -1) {
perror("epoll_create");
return -1;
}
}
tab = malloc(sizeof(struct pipefd) * nbhandles);
if (tab == NULL) {
perror("malloc");
return -1;
}
if (afinet) {
listen_sock = socket(AF_INET, SOCK_STREAM, 0);
if (listen_sock == -1) {
perror("socket");
return -1;
}
if (listen(listen_sock, 256) == -1) {
perror("listen");
return -1;
}
namelen = sizeof(me);
getsockname(listen_sock, (struct sockaddr *)&me, &namelen);
}
for (i = 0 ; i < nbhandles ; i++) {
if (afinet) {
tab[i].fd[0] = socket(AF_INET, SOCK_STREAM, 0);
if (tab[i].fd[0] == -1)
break;
to = me;
ioctl(tab[i].fd[0], FIONBIO, &on);
if (connect(tab[i].fd[0], (struct sockaddr *)&to, sizeof(to)) != -1 || errno != EINPROGRESS)
break;
tab[i].fd[1] = accept(listen_sock, (struct sockaddr *)&to, &namelen);
if (tab[i].fd[1] == -1)
break;
ioctl(tab[i].fd[0], FIONBIO, &off);
}
else if (afunix) {
if (socketpair(AF_UNIX, SOCK_STREAM, 0, tab[i].fd) == -1)
break;
} else {
if (pipe(tab[i].fd) == -1)
break;
}
if (!fflag) {
struct epoll_event ev;
ev.events = EPOLLIN | EPOLLET;
ev.data.u64 = (uint64_t)i;
epoll_ctl(epoll_fd, EPOLL_CTL_ADD, tab[i].fd[0], &ev);
}
}
nbhandles = i;
printf("%d handles setup\n", nbhandles);
return 0;
}int sample_proc_stat(long *ctxt)
{
int fd = open("/proc/stat", 0);
char buffer[4096+1], *p;
int lu;
*ctxt = 0;
if (fd == -1) {
perror("/proc/stat");
return -1;
}
lu = read(fd, buffer, sizeof(buffer));
close(fd);
if (lu < 10)
return -1;
buffer[lu] = 0;
p = strstr(buffer, "ctxt");
if (p)
*ctxt = atol(p + 4);
return 0;
}static void timer_func()
{
char buffer[128];
size_t len;
static unsigned long old;
static unsigned long oldctxt=0;
unsigned long ctxt;
unsigned long delta = nbhandled - old;
static int alarm_events = 0;
old = nbhandled;
len = sprintf(buffer, "%lu evts/sec", delta);
sample_proc_stat(&ctxt);
delta = ctxt - oldctxt;
if (delta && oldctxt)
len += sprintf(buffer + len, " %lu ctxt/sec", delta);
oldctxt = ctxt;
if (epw_samples)
len += sprintf(buffer + len, " %g samples per call", (double)epw_samples_cnt/(double)epw_samples);
buffer[len++] = '\n';
write(2, buffer, len);
if (++alarm_events >= time_test) {
delta = nbhandled/alarm_events;
len = sprintf(buffer, "Avg: %lu evts/sec\n", delta);
write(2, buffer, len);
exit(0);
}
}static void timer_setup()
{
struct itimerval it;
struct sigaction sg;
memset(&sg, 0, sizeof(sg));
sg.sa_handler = timer_func;
sigaction(SIGALRM, &sg, 0);
it.it_interval.tv_sec = 1;
it.it_interval.tv_usec = 0;
it.it_value.tv_sec = 1;
it.it_value.tv_usec = 0;
if (setitimer(ITIMER_REAL, &it, 0))
perror("setitimer");
}static void * worker_thread_func(void *arg)
{
int fd = -1;
char c = 1;
int cnt = 0;
nice(10);
for (;;) {
if (fflag)
fd = (fd + 1) % nbhandles;
else
fd = rand() % nbhandles;
write(tab[fd].fd[1], &c, 1);
if (++cnt >= nbhandles) {
cnt = 0 ;
pthread_yield(); /* relax :) */
}
}
}void usage(int code)
{
fprintf(stderr, "Usage : epoll_bench [-n num] [-{u|i}] [-f] [-t duration] [-l limit] [-e maxepoll]\n");
exit(code);
}int main(int argc, char *argv[])
{
char buff[1024];
pthread_t tid;
int c, fd;
int limit = 1000;
int max_epoll = 1024;
while ((c = getopt(argc, argv, "fuin:l:e:t:")) != EOF) {
if (c == 'n') nbhandles = atoi(optarg);
else if (c == 'f') fflag++;
else if (c == 'l') limit = atoi(optarg);
else if (c == 'e') max_epoll = atoi(optarg);
else if (c == 't') time_test = atoi(optarg);
else if (c == 'u') afunix++;
else if (c == 'i') afinet++;
else usage(1);
}
alloc_streams();
pthread_create(&tid, NULL, worker_thread_func, (void *)0);
timer_setup();
if (fflag) {
for (fd = 0;;fd = (fd + 1) % nbhandles) {
if (read(tab[fd].fd[0], buff, 1024) > 0)
nbhandled++;
}
}
else {
struct epoll_event *events;
events = malloc(sizeof(struct epoll_event) * max_epoll) ;
for (;;) {
int nb = epoll_wait(epoll_fd, events, max_epoll, -1);
int i;
epw_samples++;
epw_samples_cnt += nb;
for (i = 0 ; i < nb ; i++) {
fd = tab[events[i].data.u64].fd[0];
if (read(fd, buff, 1024) > 0)
nbhandled++;
}
if (nb < limit)
pthread_yield();
}
}
}Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 33 lines · Eric Dumazet
AF_INET# ./epoll_bench -n 2000 -i 2000 handles setup 69210 evts/sec 2.97224 samples per call 59436 evts/sec 12876 ctxt/sec 5.48675 samples per call 60722 evts/sec 12093 ctxt/sec 8.03185 samples per call 60583 evts/sec 14582 ctxt/sec 10.5644 samples per call 58192 evts/sec 12066 ctxt/sec 12.999 samples per call 54291 evts/sec 10613 ctxt/sec 15.2398 samples per call 47978 evts/sec 10942 ctxt/sec 17.2222 samples per call 59009 evts/sec 13692 ctxt/sec 19.6426 samples per call 58248 evts/sec 15099 ctxt/sec 22.0306 samples per call 58708 evts/sec 15118 ctxt/sec 24.4497 samples per call 58613 evts/sec 14608 ctxt/sec 26.816 samples per call 58490 evts/sec 13593 ctxt/sec 29.1708 samples per call 59108 evts/sec 15078 ctxt/sec 31.5557 samples per call 59636 evts/sec 15053 ctxt/sec 33.9292 samples per call 59355 evts/sec 15531 ctxt/sec 36.2914 samples per call Avg: 58771 evts/secThe last sample shows that epoll overhead is very small indeed, since disabling it doesnt boost
AF_INETperf at all.AF_INET+ no epoll# ./epoll_bench -n 2000 -i -f 2000 handles setup 79939 evts/sec 78468 evts/sec 9989 ctxt/sec 73153 evts/sec 10207 ctxt/sec 73668 evts/sec 10163 ctxt/sec 73667 evts/sec 20084 ctxt/sec 74106 evts/sec 10068 ctxt/sec 73442 evts/sec 10119 ctxt/sec 74220 evts/sec 10122 ctxt/sec
Quoted 8 lines
74367 evts/sec 10097 ctxt/sec 64402 evts/sec 47873 ctxt/sec 53555 evts/sec 58733 ctxt/sec 46000 evts/sec 48984 ctxt/sec 67052 evts/sec 21006 ctxt/sec 68460 evts/sec 12344 ctxt/sec 67629 evts/sec 10655 ctxt/sec Avg: 69475 evts/sec
Without epoll number of events/sec is about 18% more - 58k vs 69k.
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
In reply to Eric DumazetQuoted 7 lines · Eric Dumazet
Quoted 4 lines · Evgeniy Polyakov
If there would exist sockets support, then I could patch it to work with kevents.
OK I post here my last version of
epoll_bench.
My results with AF_INET are inlined.
Hardware 2.4 Ghz Xeon w(1 CPU, HT enabled) with 1 GB of RAM.
[root@pcix event]# ./epoll_bench -n 2000 -i 2000 handles setup
49758 evts/sec 1.56177 samples per call 38999 evts/sec 95 ctxt/sec 2.77247 samples per call 54042 evts/sec 130 ctxt/sec 4.19909 samples per call 60155 evts/sec 188 ctxt/sec 5.38024 samples per call 59588 evts/sec 178 ctxt/sec 6.38112 samples per call 60023 evts/sec 188 ctxt/sec 7.19564 samples per call 59694 evts/sec 186 ctxt/sec 7.93067 samples per call 60182 evts/sec 190 ctxt/sec 8.52397 samples per call 59750 evts/sec 182 ctxt/sec 9.08015 samples per call 60158 evts/sec 192 ctxt/sec 9.53548 samples per call 59739 evts/sec 188 ctxt/sec 9.97013 samples per call 60054 evts/sec 216 ctxt/sec 10.32 samples per call 59820 evts/sec 206 ctxt/sec 10.6641 samples per call 60095 evts/sec 218 ctxt/sec 10.9289 samples per call 59376 evts/sec 158 ctxt/sec 11.3231 samples per call
Avg: 57428 evts/sec
[root@pcix event]# ./kevent_bench -n2000 -i 2000 handles setup
57960 evts/sec 0.276702 samples per call 59802 evts/sec 75 ctxt/sec 0.462737 samples per call 59864 evts/sec 71 ctxt/sec 0.623457 samples per call 59651 evts/sec 72 ctxt/sec 0.721579 samples per call 59504 evts/sec 84 ctxt/sec 0.804311 samples per call 61019 evts/sec 72 ctxt/sec 0.904817 samples per call 59846 evts/sec 72 ctxt/sec 0.949439 samples per call 60550 evts/sec 74 ctxt/sec 1.00416 samples per call 59421 evts/sec 66 ctxt/sec 1.04133 samples per call 60334 evts/sec 75 ctxt/sec 1.06845 samples per call 60000 evts/sec 67 ctxt/sec 1.09594 samples per call 59429 evts/sec 74 ctxt/sec 1.11404 samples per call 60508 evts/sec 77 ctxt/sec 1.14482 samples per call 59530 evts/sec 66 ctxt/sec 1.15454 samples per call 59506 evts/sec 73 ctxt/sec 1.17937 samples per call
Avg: 59794 evts/sec
[root@pcix event]# ./kevent_bench -n2000 -i -f 2000 handles setup
82893 evts/sec
88624 evts/sec 390 ctxt/sec
88751 evts/sec 475 ctxt/sec
88784 evts/sec 488 ctxt/sec
88918 evts/sec 458 ctxt/sec
88866 evts/sec 504 ctxt/sec
88950 evts/sec 458 ctxt/sec
88883 evts/sec 472 ctxt/sec
88915 evts/sec 404 ctxt/sec
88836 evts/sec 368 ctxt/sec
89065 evts/sec 442 ctxt/sec
88859 evts/sec 398 ctxt/sec
89070 evts/sec 446 ctxt/sec
88809 evts/sec 428 ctxt/sec
89012 evts/sec 542 ctxt/sec
Avg: 88482 evts/sec
epoll: 57428
kevent: 59794
max: 88482
BUT!
Kevent does not support analogue for EPOLLET, i.e. the case when the same event is used, instead kevent must modify existing one (i.e. behave exactly like epoll without EPOLLET), so modified epoll_bench to work without EPOLLET like kevent.
epoll with EPOLLET shows upto 71k events/sec.
Lack of such feature is a minus for kevent indeed.
I will add it into todo list behind (implemented) new ring buffer implementation and (implemented) wake-up-one-thread flag implementation. Hopefully I will include it into next kevent release soon, but do not expect it today/tomorrow, there some unrelated to hacking problems.
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 78 lines · Evgeniy Polyakov
Quoted 7 lines · Eric Dumazet
Quoted 4 lines · Evgeniy Polyakov
If there would exist sockets support, then I could patch it to work with kevents.
OK I post here my last version of
epoll_bench.My results with
AF_INETare inlined.
Hardware 2.4 Ghz Xeonw(1 CPU, HT enabled)with 1 GB of RAM.[root@pcix event]# ./epoll_bench -n 2000 -i 2000 handles setup
49758 evts/sec 1.56177 samples per call 38999 evts/sec 95 ctxt/sec 2.77247 samples per call 54042 evts/sec 130 ctxt/sec 4.19909 samples per call 60155 evts/sec 188 ctxt/sec 5.38024 samples per call 59588 evts/sec 178 ctxt/sec 6.38112 samples per call 60023 evts/sec 188 ctxt/sec 7.19564 samples per call 59694 evts/sec 186 ctxt/sec 7.93067 samples per call 60182 evts/sec 190 ctxt/sec 8.52397 samples per call 59750 evts/sec 182 ctxt/sec 9.08015 samples per call 60158 evts/sec 192 ctxt/sec 9.53548 samples per call 59739 evts/sec 188 ctxt/sec 9.97013 samples per call 60054 evts/sec 216 ctxt/sec 10.32 samples per call 59820 evts/sec 206 ctxt/sec 10.6641 samples per call 60095 evts/sec 218 ctxt/sec 10.9289 samples per call 59376 evts/sec 158 ctxt/sec 11.3231 samples per call
Avg: 57428 evts/sec
[root@pcix event]# ./kevent_bench -n2000 -i 2000 handles setup
57960 evts/sec 0.276702 samples per call 59802 evts/sec 75 ctxt/sec 0.462737 samples per call 59864 evts/sec 71 ctxt/sec 0.623457 samples per call 59651 evts/sec 72 ctxt/sec 0.721579 samples per call 59504 evts/sec 84 ctxt/sec 0.804311 samples per call 61019 evts/sec 72 ctxt/sec 0.904817 samples per call 59846 evts/sec 72 ctxt/sec 0.949439 samples per call 60550 evts/sec 74 ctxt/sec 1.00416 samples per call 59421 evts/sec 66 ctxt/sec 1.04133 samples per call 60334 evts/sec 75 ctxt/sec 1.06845 samples per call 60000 evts/sec 67 ctxt/sec 1.09594 samples per call 59429 evts/sec 74 ctxt/sec 1.11404 samples per call 60508 evts/sec 77 ctxt/sec 1.14482 samples per call 59530 evts/sec 66 ctxt/sec 1.15454 samples per call 59506 evts/sec 73 ctxt/sec 1.17937 samples per call
Avg: 59794 evts/sec
[root@pcix event]# ./kevent_bench -n2000 -i -f 2000 handles setup
82893 evts/sec
88624 evts/sec 390 ctxt/sec
88751 evts/sec 475 ctxt/sec
88784 evts/sec 488 ctxt/sec
88918 evts/sec 458 ctxt/sec
88866 evts/sec 504 ctxt/sec
88950 evts/sec 458 ctxt/sec
88883 evts/sec 472 ctxt/sec
88915 evts/sec 404 ctxt/sec
88836 evts/sec 368 ctxt/sec
89065 evts/sec 442 ctxt/sec
88859 evts/sec 398 ctxt/sec
89070 evts/sec 446 ctxt/sec
88809 evts/sec 428 ctxt/sec
89012 evts/sec 542 ctxt/sec
Avg: 88482 evts/secepoll: 57428
kevent: 59794
max: 88482BUT!
Kevent does not support analogue for EPOLLET, i.e. the case when the same event is used, instead kevent must modify existing one (i.e. behave exactly like epoll without EPOLLET), so modifiedepoll_benchto work without EPOLLET like kevent.
epoll with EPOLLET shows upto 71k events/sec.
Quoted 5 lines
Lack of such feature is a minus for kevent indeed.
I will add it into todo list behind (implemented) new ring buffer implementation and (implemented) wake-up-one-thread flag implementation. Hopefully I will include it into next kevent release soon, but do not expect it today/tomorrow, there some unrelated to hacking problems.
Here is edge-triggered behavior of kevent:
[root@pcix event]# ./kevent_bench -n2000 -i 2000 handles setup
67057 evts/sec 1.18746 samples per call 79239 evts/sec 68 ctxt/sec 1.30531 samples per call 78877 evts/sec 140 ctxt/sec 1.34172 samples per call 79017 evts/sec 82 ctxt/sec 1.35835 samples per call 78957 evts/sec 115 ctxt/sec 1.36885 samples per call 79084 evts/sec 70 ctxt/sec 1.37419 samples per call 79083 evts/sec 98 ctxt/sec 1.38 samples per call 79083 evts/sec 72 ctxt/sec 1.38194 samples per call 79025 evts/sec 111 ctxt/sec 1.38426 samples per call 79139 evts/sec 78 ctxt/sec 1.38554 samples per call 79055 evts/sec 112 ctxt/sec 1.38701 samples per call 79118 evts/sec 72 ctxt/sec 1.39 samples per call 79040 evts/sec 94 ctxt/sec 1.39108 samples per call 79098 evts/sec 81 ctxt/sec 1.39136 samples per call 79104 evts/sec 90 ctxt/sec 1.39269 samples per call
Avg: 78265 evts/sec
So, kevent is faster than epoll.
It was proven using three independent benchmarks (mine evserver_kevent.c, Johann Borck's own web server and Eric's epoll_bench).
I plan to release new version with all additional goodies today.
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
Kevent pipe benchmark kevent_pipe kernel kevent part:
epoll (edge-triggered): 248408 events/sec kevent (edge-triggered): 269282 events/sec Busy reading loop: 269519 events/sec
Kevent is definitely a winner with extremely small overhead.
I will add kevent_pipe into next kevent release which will be available soon.
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
In reply to Eric Dumazet2006/11/2, Eric Dumazet <[email protected]>:
Quoted 8 lines
Quoted 4 lines · zhou drangon
performance is great, and we are exciting at the result.
I want to know why there can be so much improvement, can we improve epoll too ?
Why did you remove most of CC addresses but lkml ? Dont do that please...
I seldom reply to the mailing list, Sorry for this.
Quoted 5 lines
Good question :)
Hum, I think I can look into epoll and see how it can be improved (if necessary)
I have an other question.
As for the VFS system, when we introduce the AIO machinism, we add aio_read, aio_write, etc... to file ops, and then we make the read, write op to call aio_read,aio_write, so that we only remain one implement in kernel.
Can we do event machinism the same way?
when kevent is robust enough, can we implement epoll/select/io_submit etc... base on kevent ??
In this way, we can simplified the kernel, and epoll can gain improvement from kevent.
Quoted 1 line
This is not to say we dont need kevent ! Please Evgeniy continue your work !
Yes! We are expecting for you greate work.
I create an userland event-driven framework for my application. but I have to use multiple thread to receive event, epoll to wait most event, and io_getevent to wait disk AIO event, I hope we can get a universal event machinism
to make the code elegance.
Quoted 205 lines
Just to remind you that according to
http://www.xmailserver.org/linux-patches/nio-improve.html David Libenzi had to wait 18 months before epoll being officialy added into kernel.At that time, many applications were using epoll, and we were patching our kernels for that.
I cooked a very simple program (attached in this mail), using pipes and epoll, and got 250.000 events received per second on an otherwise lightly loaded machine (dual opteron 246 , 2GHz, 1MB cache per cpu) with 10.000 pipes (20.000 handles)
It could be nice to add support for other event providers in this program (
AF_INET&AF_UNIXsockets for example), and also add support for kevent, so that we really can compare epoll/kevent without a complex setup. I should extend the program to also add/remove sources during lifetime, not only insert at setup time.# gcc -O2 -o epoll_pipe_bench epoll_pipe_bench.c -lpthread # ulimit -n 1000000 # epoll_pipe_bench -n 10000 ^C after a while...oprofile results say that
ep_poll_callback()andsys_epoll_wait()use 20% of cpu time.
Even if we gain a two factor in cpu time or cache usage, we wont eliminate other costs...oprofile results gave :
Counted
CPU_CLK_UNHALTEDevents (Cycles outside of halt state) with a unit mask of0x00(No unit mask) count 50000samples % symbol name 2015420 11.1309 ep_poll_callback 1867431 10.3136 pipe_writev 1791872 9.8963 sys_epoll_wait 1357297 7.4962 fget_light 1277515 7.0556 pipe_readv 998447 5.5143 current_fs_time 801597 4.4271 __mark_inode_dirty 755268 4.1713 __wake_up 587065 3.2423 __write_lock_failed 582931 3.2195 system_call 297132 1.6410 iov_fault_in_pages_read 296136 1.6355 sys_write 290106 1.6022 __wake_up_common 270692 1.4950 bad_pipe_w 261516 1.4443 do_pipe 257208 1.4205 tg3_start_xmit_dma_bug 254917 1.4079 pipe_poll 252925 1.3969 copy_user_generic_c 234212 1.2935 generic_pipe_buf_map 228659 1.2629 ret_from_sys_call 212541 1.1738 sysret_check 166529 0.9197 sys_read 160038 0.8839 vfs_write 151091 0.8345 pipe_ioctl 136301 0.7528 file_update_time 107173 0.5919 tg3_poll 77846 0.4299 ipt_do_table 75081 0.4147 schedule 73059 0.4035 vfs_read 69787 0.3854 get_task_comm 63923 0.3530 memcpy 60019 0.3315 touch_atime 57490 0.3175 eventpoll_release_file 56152 0.3101 tg3_write_flush_reg32 54468 0.3008 rw_verify_area 47833 0.2642 generic_pipe_buf_unmap 47777 0.2639 __switch_to 44106 0.2436 bad_pipe_r 41824 0.2310 proc_nr_files 41319 0.2282 pipe_iov_copy_from_userEric
/*
* How to stress epoll
*
* This program uses many pipes and two threads.
* First we open as many pipes we can. (see ulimit -n)
* Then we create a worker thread.
* The worker thread will send bytes to random pipes.
* The main thread uses epoll to collect ready pipes and read them.
* Each second, a number of collected bytes is printed on stderr *
* Usage :epoll_bench[-n X]
*/
#include <pthread.h>
#include <stdlib.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <sys/epoll.h>
#include <signal.h>
#include <unistd.h>
#include <sys/time.h>int nbpipes = 1024;
struct pipefd { int fd[2]; } *tab;int
epoll_fd;static int
alloc_pipes(){ int i; epoll_fd = epoll_create(nbpipes); if (epoll_fd == -1) { perror("epoll_create"); return -1; } tab = malloc(sizeof(struct pipefd) * nbpipes); if (tab ==NULL) { perror("malloc"); return -1; } for (i = 0 ; i < nbpipes ; i++) { struct epoll_event ev; if (pipe(tab[i].fd) == -1) break; ev.events = EPOLLIN | EPOLLOUT | EPOLLHUP | EPOLLPRI | EPOLLET; ev.data.u64 = (uint64_t)i; epoll_ctl(epoll_fd, EPOLL_CTL_ADD, tab[i].fd[0], &ev); } nbpipes = i; printf("%d pipes setup\n", nbpipes); return 0; }unsigned long nbhandled;
static voidtimer_func(){ char buffer[32]; size_t len; static unsigned long old; unsigned long delta = nbhandled - old; old = nbhandled; len = sprintf(buffer, "%lu\n", delta); write(2, buffer, len); }static void
timer_setup(){ struct itimerval it; struct sigaction sg; memset(&sg, 0, sizeof(sg)); sg.sa_handler = timer_func; sigaction(SIGALRM, &sg, 0); it.it_interval.tv_sec = 1; it.it_interval.tv_usec = 0; it.it_value.tv_sec = 1; it.it_value.tv_usec = 0; if (setitimer(ITIMER_REAL, &it, 0)) perror("setitimer"); }static void *
worker_thread_func(void *arg){ int fd; char c = 1; for (;;) { fd = rand() % nbpipes; write(tab[fd].fd[1], &c, 1); } }int
main(int argc, char *argv[]){ char buff[1024]; pthread_t tid; int c; while ((c = getopt(argc, argv, "n:")) != EOF) { if (c == 'n') nbpipes = atoi(optarg); } alloc_pipes(); pthread_create(&tid, NULL, worker_thread_func, (void *)0); timer_setup(); for (;;) { struct epoll_event events[128]; int nb = epoll_wait(epoll_fd, events, 128, 10000); int i, fd; for (i = 0 ; i < nb ; i++) { fd = tab[events[i].data.u64].fd[0]; if (read(fd, buff, 1024) > 0) nbhandled++; } } }
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 9 lines · zhou drangon
As for the VFS system, when we introduce the AIO machinism, we add
aio_read,aio_write, etc... to file ops, and then we make the read, write op to callaio_read,aio_write, so that we only remain one implement in kernel.
Can we do event machinism the same way?
when kevent is robust enough, can we implement epoll/select/io_submit etc... base on kevent ??
In this way, we can simplified the kernel, and epoll can gain improvement from kevent.
There is AIO implementaion on top of kevent, although it was confirmed that it has a good design, except minor API layering changes, it was postponed for a while.
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
In reply to Nate DillerQuoted 8 lines · Nate Diller
Indesiciveness has certainly been an issue here, but I remember akpm and Ulrich both giving concrete suggestions. I was particularly interested in Andrew's request to explain and justify the differences between kevent and BSD's kqueue interface. Was there a discussion that I missed? I am very interested to see your work on this mechanism merged, because you've clearly emphasized performance and shown impressive results. But it seems like we lose out on a lot by throwing out all the applications that already use kqueue.
It looks you missed that discussion - freebsd kqueue has fields in the kevent structure which have diffent sizes in 32 and 64 bit environments.
Quoted 1 line
NATE
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 10 lines · Evgeniy Polyakov
Quoted 8 lines · Nate Diller
Indesiciveness has certainly been an issue here, but I remember akpm and Ulrich both giving concrete suggestions. I was particularly interested in Andrew's request to explain and justify the differences between kevent and BSD's kqueue interface. Was there a discussion that I missed? I am very interested to see your work on this mechanism merged, because you've clearly emphasized performance and shown impressive results. But it seems like we lose out on a lot by throwing out all the applications that already use kqueue.
Quoted 2 lines
It looks you missed that discussion - freebsd kqueue has fields in the kevent structure which have diffent sizes in 32 and 64 bit environments.
Are you saying that the *only* reason we choose not to be source-compatible with BSD is the 32 bit userland on 64 bit arch problem? I've followed every thread that gmail 'kqueue' search returns, which thread are you referring to? Nicholas Miell, in "The Proposed Linux kevent API" thread, seems to think that there are no advantages over kqueue to justify the incompatibility, an argument you made no effort to refute. I've also read the Kevent wiki at linux-net.osdl.org, but it too is lacking in any direct comparisons (even theoretical, let alone benchmarks) of the flexibility, performance, etc. between the two.
I'm not arguing that you've done a bad design, I'm asking you to brag about the things you improved on vs. kqueue. Your emphasis on unifying all the different event types into one interface is really cool, fill me in on why that can't be effectively done with the kqueue compatability and I also will advocate for kevent inclusion.
NATE
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 3 lines · Nate Diller
Are you saying that the *only* reason we choose not to be source-compatible with BSD is the 32 bit userland on 64 bit arch problem? I've followed every thread that gmail 'kqueue' search
I.e. do you want that generic event handling mechanism would not work on x86_64? I doubt you do.
Quoted 8 lines
returns, which thread are you referring to? Nicholas Miell, in "The Proposed Linux kevent API" thread, seems to think that there are no advantages over kqueue to justify the incompatibility, an argument you made no effort to refute. I've also read the Kevent wiki at linux-net.osdl.org, but it too is lacking in any direct comparisons (even theoretical, let alone benchmarks) of the flexibility, performance, etc. between the two.
Quoted 5 lines
I'm not arguing that you've done a bad design, I'm asking you to brag about the things you improved on vs. kqueue. Your emphasis on unifying all the different event types into one interface is really cool, fill me in on why that can't be effectively done with the kqueue compatability and I also will advocate for kevent inclusion.
kqueue just can not be used as is in Linux (_maybe_ *bsd has different types, not those which I found in /usr/include in my FC5 and Debian distro). It will not work on x86_64 for example. Some kind of a pointer or unsigned long in structures which are transferred between kernelspace and userspace is so much questionable, than it is much better even do not see there... (if I would not have so political correctness, I would describe it in a much different words actually).
So, kqueue API and structures can not be usd in Linux.
Quoted 1 line
NATE
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
Hi!
Quoted 14 lines
Quoted 8 lines
returns, which thread are you referring to? Nicholas Miell, in "The Proposed Linux kevent API" thread, seems to think that there are no advantages over kqueue to justify the incompatibility, an argument you made no effort to refute. I've also read the Kevent wiki at linux-net.osdl.org, but it too is lacking in any direct comparisons (even theoretical, let alone benchmarks) of the flexibility, performance, etc. between the two.
Quoted 5 lines
I'm not arguing that you've done a bad design, I'm asking you to brag about the things you improved on vs. kqueue. Your emphasis on unifying all the different event types into one interface is really cool, fill me in on why that can't be effectively done with the kqueue compatability and I also will advocate for kevent inclusion.
Quoted 8 lines
kqueue just can not be used as is in Linux (_maybe_ *bsd has different types, not those which I found in
/usr/includein my FC5 and Debian distro). It will not work onx86_64for example. Some kind of a pointer or unsigned long in structures which are transferred between kernelspace and userspace is so much questionable, than it is much better even do not see there... (if I would not have so political correctness, I would describe it in a much different words actually).
So, kqueue API and structures can not be usd in Linux.
Not sure what you are smoking, but "there's unsigned long in *bsd version, lets rewrite it from scratch" sounds like very bad idea. What about fixing that one bit you don't like?
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
David Miller
maintainerRe: [take22 0/4] kevent: Generic event handling mechanism.
From: Pavel Machek <[email protected]>
Date: Fri, 3 Nov 2006 09:57:12 +0100
Quoted 3 lines
Not sure what you are smoking, but "there's unsigned long in *bsd version, lets rewrite it from scratch" sounds like very bad idea. What about fixing that one bit you don't like?
I disagree, it's more like since we have to be structure incompatible anyways, let's design something superior if we can.
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 9 lines · David Miller
From: Pavel Machek <[email protected]>
Date: Fri, 3 Nov 2006 09:57:12 +0100Quoted 3 lines
Not sure what you are smoking, but "there's unsigned long in *bsd version, lets rewrite it from scratch" sounds like very bad idea. What about fixing that one bit you don't like?
I disagree, it's more like since we have to be structure incompatible anyways, let's design something superior if we can.
Definitely agreed.
Jeff
Re: [take22 0/4] kevent: Generic event handling mechanism.
In reply to Pavel MachekQuoted 5 lines · Pavel Machek
Quoted 1 line
So, kqueue API and structures can not be usd in Linux.
Not sure what you are smoking, but "there's unsigned long in *bsd version, lets rewrite it from scratch" sounds like very bad idea. What about fixing that one bit you don't like?
It is not about what I dislike, but about what is broken or not. Putting u64 instead of a long or some kind of that _is_ incompatible already, so why should we even use it?
And, btw, what we are talking about? Is it about the whole kevent compared to kqueue in kernelspace, or just about what structure is being transferred between kernelspace and userspace?
I'm sure, it was some kind of a joke to 'not rewrite *bsd from scratch and use kqueue in Linux kernel as is'.
Quoted 4 lines
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
Hi!
Quoted 10 lines · Evgeniy Polyakov
Quoted 5 lines · Pavel Machek
Quoted 1 line
So, kqueue API and structures can not be usd in Linux.
Not sure what you are smoking, but "there's unsigned long in *bsd version, lets rewrite it from scratch" sounds like very bad idea. What about fixing that one bit you don't like?
It is not about what I dislike, but about what is broken or not. Putting u64 instead of a long or some kind of that _is_ incompatible already, so why should we even use it?
Well.. u64 vs unsigned long *is* binary incompatible, but it is similar enough that it is going to be compatible at source level, or maybe userland app will need *minor* ifdefs... That's better than two completely different versions...
Quoted 5 lines
And, btw, what we are talking about? Is it about the whole kevent compared to kqueue in kernelspace, or just about what structure is being transferred between kernelspace and userspace?
I'm sure, it was some kind of a joke to 'not rewrite *bsd from scratch and use kqueue in Linux kernel as is'.
No, it is probably not possible to take code from BSD kernel and "just port it". But keeping same/similar userland interface would be nice.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 19 lines · Pavel Machek
Hi!
Quoted 10 lines · Evgeniy Polyakov
Quoted 5 lines · Pavel Machek
Quoted 1 line
So, kqueue API and structures can not be usd in Linux.
Not sure what you are smoking, but "there's unsigned long in *bsd version, lets rewrite it from scratch" sounds like very bad idea. What about fixing that one bit you don't like?
It is not about what I dislike, but about what is broken or not. Putting u64 instead of a long or some kind of that _is_ incompatible already, so why should we even use it?
Well.. u64 vs unsigned long *is* binary incompatible, but it is similar enough that it is going to be compatible at source level, or maybe userland app will need *minor* ifdefs... That's better than two completely different versions...
Quoted 8 lines
Quoted 5 lines
And, btw, what we are talking about? Is it about the whole kevent compared to kqueue in kernelspace, or just about what structure is being transferred between kernelspace and userspace?
I'm sure, it was some kind of a joke to 'not rewrite *bsd from scratch and use kqueue in Linux kernel as is'.No, it is probably not possible to take code from BSD kernel and "just port it". But keeping same/similar userland interface would be nice.
It is not only probably, but not even unlikely - it is impossible to get FreeBSD kqueue code and port it - that port will be completely different system.
It is impossible to have the same event structure, one should create #if defined kqueue
fill all members of the structure
#else if defined kevent
fill different members name, since Linux does not even have some types #endif
*BSD kevent (structure transferred between userspace and kernelspace) struct kevent {
uintptr_t ident; /* identifier for this event */
short filter; /* filter for event */
u_short flags; /* action flags for kqueue */
u_int fflags; /* filter flag value */
intptr_t data; /* filter data value */
void *udata; /* opaque user data identifier */
};
You must fill all fields differently due to above.
Just an example: Linux kevent has extended ID field which is grouped into type.event, kqueue has different pointer indent and short filter.
Linux kevent does not have filters, but instead it has generic storages of events which can be processed in any way origin of the storage wants (this for example allows to create aio_sendfile() (which is dropped from patchset currently) which no other system in the wild has).
There are too many differences. It is just different systems. If both can be described by sentence "system which handles events", it does not mean that they are the same and can use the structures or even have similar design.
Kevent is not kqueue in any way (although there are certain similarities), so they can not share anything.
Quoted 4 lines
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
In reply to Nate DillerQuoted 8 lines · Nate Diller
Indesiciveness has certainly been an issue here, but I remember akpm and Ulrich both giving concrete suggestions. I was particularly interested in Andrew's request to explain and justify the differences between kevent and BSD's kqueue interface. Was there a discussion that I missed? I am very interested to see your work on this mechanism merged, because you've clearly emphasized performance and shown impressive results. But it seems like we lose out on a lot by throwing out all the applications that already use kqueue.
kqueue looks pretty nice, the filter/note models in particular. I don't see anything about ring buffers though.
I also wonder about the asynchronous event side (send), not just the event reception side.
Jeff
Re: [take22 0/4] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovQuoted 1 line · Evgeniy Polyakov
On Wed, Nov 01, 2006 at 06:20:43PM +0000, Oleg Verych ([email protected]) wrote:
[]
Quoted 4 lines
Quoted 1 line
Where's real-life application to do configure && make && make install?
Your real life or mine as developer?
I fortunately do not know anything about your real life, but my real life
To do not further shift conversation in no technical way, think of my sentence as question *and* as definition.
Quoted 2 lines
applications can be found on project's homepage.
There is a link to archive there, where you can find plenty of sources.
But no single makefile. Or what CC and options do not mater really? You can easily find in your server's apache logs, my visit of that archive in the day of my message (today i just confirmed my assertions):
browser lynx, host flower.upol.cz.
Quoted 3 lines
You likely do not know, but it is a bit risky business to patch all existing applications to show that approach is correct, if implementation is not completed.
Fortunately to me, `lighthttpd' is real-life *and* in the benchmark area also. Just see that site how much there was measured: different OSes, special tunning. *That* is i'm talking about. Epoll _wrapper_ there, is 3461 byte long, your answer to _me_ 2580. People are bringing you a test bed, with all set up ready to use; need less code, go on, comment needless out!
Quoted 3 lines
You likely do not know, but after I first time announced kevents in February I changed interfaces 4 times - and it is just interfaces, not including numerous features added/removed by developer's requests.
I think that called open source, linux kernel case.
Quoted 9 lines
Quoted 5 lines
There were some comments about laking much of such programs, answers were "was in prev. e-mail", "need to update them", something like that. "Trivial web server" sources url, mentioned in benchmark isn't pointed in patch advertisement. If it was, should i actually try that new *trivial* wheel?
Answer is trivial - there is archive where one can find a source code (filenames are posted regulary). Should I create a rpm? For what glibc version?
Hmm. Let me answer on that "dup" with stuff from LKML archive. That will reveal, that my guesses were told by The Big Jury to you already:
[^0] Message-ID: [email protected] [^1] Message-ID: [email protected],
Message-ID: [email protected]
more than 10 takes ago.
Quoted 6 lines
Quoted 3 lines
Saying that, i want to give you some short examples, i know. *Linux kernel <-> userspace*:
o Alexey Kuznetsov networking <-> (excellent) iproute set of utilities;iproute documentation was way too bad when Alexey presented it first time :)
As example, after have read some books on TCP/IP and Ethernet, internal help of `ip' was all i needed to know.
Quoted 2 lines
Btw, show me
splice()'shiny' application? Does lighttpd use it? Ormove_pages().
You know who proposed that, and you know how many (few) releases ago.
Quoted 8 lines
Quoted 6 lines
To make a little hint to you, Evgeniy, why don't you find a little animal in the open source zoo to implement little interface to proposed kernel subsystem and then show it to The Big Jury (not me), we have here? And i can not see, how you've managed to implement something like that having almost nothing on the test basket. Very *suspicious* ch.
There are always people who do not like something, what can I do with
I didn't think, that my message was offensive. Also i didn't even say, that you have not bothered feed your code to "scripts/Lindent".
[]
Quoted 4 lines
I created trivial web servers, which send single static page and use various event handling schemes, and I test new subsystem with new tools, when tests are completed and all requested features are implemented it is time to work on different more complex users.
Please, see [^0],
Quoted 2 lines
So let's at least complete what we have right now, so no developer's efforts could be wasted writing empty chars in various places.
and [^1].
[ Please do not answer just to answer, cc list is big, no one from ]
[ The Big Jury seems to care. (well, Jonathan does, but he wasn't in cc) ]
Friendly, Oleg.
____
Re: [take22 0/4] kevent: Generic event handling mechanism.
Quoted 7 lines · Oleg Verych
Quoted 2 lines
applications can be found on project's homepage.
There is a link to archive there, where you can find plenty of sources.But no single makefile. Or what CC and options do not mater really? You can easily find in your server's apache logs, my visit of that archive in the day of my message (today i just confirmed my assertions):
browser lynx, host flower.upol.cz.
If you can not compile that sources, than you should not use kevent for a while. Definitely.
Options are pretty simple: -W -Wall -I$(path_to_kernel_tree)/include
Quoted 10 lines
Quoted 3 lines
You likely do not know, but it is a bit risky business to patch all existing applications to show that approach is correct, if implementation is not completed.
Fortunately to me, `lighthttpd' is real-life *and* in the benchmark area also. Just see that site how much there was measured: different OSes, special tunning. *That* is i'm talking about. Epoll _wrapper_ there, is 3461 byte long, your answer to _me_ 2580. People are bringing you a test bed, with all set up ready to use; need less code, go on, comment needless out!
So what?
People bring me tons of various stuff, and I prefer to use my own for tests. If _you_ need it, _you_ can always patch any sources you like.
Quoted 5 lines
Quoted 3 lines
You likely do not know, but after I first time announced kevents in February I changed interfaces 4 times - and it is just interfaces, not including numerous features added/removed by developer's requests.
I think that called open source, linux kernel case.
You missed the point - I'm not going to patch tons of existing applications when I'm asked to change an interface once per month.
When all requested features are implemented I definitely with patch some popular web-server to show how kevent is used.
Quoted 10 lines
Quoted 9 lines
Quoted 5 lines
There were some comments about laking much of such programs, answers were "was in prev. e-mail", "need to update them", something like that. "Trivial web server" sources url, mentioned in benchmark isn't pointed in patch advertisement. If it was, should i actually try that new *trivial* wheel?
Answer is trivial - there is archive where one can find a source code (filenames are posted regulary). Should I create a rpm? For what glibc version?
Quoted 8 lines
Hmm. Let me answer on that "dup" with stuff from LKML archive. That will reveal, that my guesses were told by The Big Jury to you already:
[^0] Message-ID: [email protected] [^1] Message-ID: [email protected],
Message-ID: [email protected]more than 10 takes ago.
And? Please provide a link to archive.
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
Saying that, i want to give you some short examples, i know. *Linux kernel <-> userspace*:
o Alexey Kuznetsov networking <-> (excellent) iproute set of utilities;iproute documentation was way too bad when Alexey presented it first time :)
As example, after have read some books on TCP/IP and Ethernet, internal help of `ip' was all i needed to know.
:)) i.e. it is ok for you to 'read some books on TCP/IP and Ethernet' to understand how utility works, and it is not ok to determine how to compile my sources? Do not compile my sources.
Quoted 4 lines
Quoted 2 lines
Btw, show me
splice()'shiny' application? Does lighttpd use it? Ormove_pages().You know who proposed that, and you know how many (few) releases ago.
And why lighttpd still do not use it?
You should start to blame authors of the splice() for that. You will not? Then I can not consider your words in my direction as serious.
Quoted 11 lines
Quoted 8 lines
Quoted 6 lines
To make a little hint to you, Evgeniy, why don't you find a little animal in the open source zoo to implement little interface to proposed kernel subsystem and then show it to The Big Jury (not me), we have here? And i can not see, how you've managed to implement something like that having almost nothing on the test basket. Very *suspicious* ch.
There are always people who do not like something, what can I do with
I didn't think, that my message was offensive. Also i didn't even say, that you have not bothered feed your code to "
scripts/Lindent".
You do not use kevent, why do you care about indent of the userspace tools?
Quoted 8 lines
[]
Quoted 4 lines
I created trivial web servers, which send single static page and use various event handling schemes, and I test new subsystem with new tools, when tests are completed and all requested features are implemented it is time to work on different more complex users.
Please, see [^0],
Quoted 7 lines
Quoted 2 lines
So let's at least complete what we have right now, so no developer's efforts could be wasted writing empty chars in various places.
and [^1].
[ Please do not answer just to answer, cc list is big, no one from ]
[ The Big Jury seems to care. (well, Jonathan does, but he wasn't in cc) ]
This thread is just to answer for the sake of answers - there is completely no sense in it.
You blame me that I did not create some benchmarks you like, but I do not care about it. I created usefull patch and test is in the way I like, because it is much more productive, than spending a lot of time detemining how different sources work with appropriate loads. When there will be strong requirement to perform additional tests, I will do them.
Quoted 2 lines
Friendly, Oleg.
____
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
In reply to Oleg VerychQuoted 4 lines · Oleg Verych
[ Please do not answer just to answer, cc list is big, no one from ]
[ The Big Jury seems to care. (well, Jonathan does, but he wasn't in cc) ]Friendly, Oleg.
Just in case some misunderstanding happend: I do not want to insult anyone who is against kevent, I just do not understand cases, when people require me to do something to convince them in rude manner.
Signature · 1 line
Evgeniy Polyakov
Re: [take22 0/4] kevent: Generic event handling mechanism.
In reply to Pavel MachekQuoted 11 lines · Pavel Machek
Hi!
Quoted 5 lines
Generic event handling mechanism.
Consider for inclusion.
Changes from 'take21' patchset:
We are not interrested in how many times you spammed us, nor we want to know what was wrong in previous versions. It would be nice to have short summary of what this is good for, instead.
I'm interested in knowing which version the patches belong to and what has changed (geez, it's rare enough that someone actually bothers to do this with an updated patchset, and to complain about it?)
- James
Signature · 2 lines
James Morris <[email protected]>
cover kevent: Generic event handling mechanism.
In reply to a message not in the archiveGeneric event handling mechanism.
Kevent is a generic subsytem which allows to handle event notifications. It supports both level and edge triggered events. It is similar to poll/epoll in some cases, but it is more scalable, it is faster and allows to work with essentially eny kind of events.
Events are provided into kernel through control syscall and can be read back through mmaped ring or syscall.
Kevent update (i.e. readiness switching) happens directly from internals of the appropriate state machine of the underlying subsytem (like network, filesystem, timer or any other).
Homepage:
http://tservice.net.ru/~s0mbre/old/?section=projects&item=kevent
Documentation page:
http://linux-net.osdl.org/index.php/Kevent
Consider for inclusion.
Changes from 'take22' patchset:
* new ring buffer implementation in process' memory
* wakeup-one-thread flag
* edge-triggered behaviour
With this release additional independent benchmark shows kevent speed compared to epoll:
Eric Dumazet created special benchmark which creates set of AF_INET sockets and two threads start to simultaneously read and write data from/into them.
Here is results:
epoll (no EPOLLET): 57428 events/sec
kevent (no ET): 59794 events/sec
epoll (with EPOLLET): 71000 events/sec
kevent (with ET): 78265 events/sec
Maximum (busy loop reading events): 88482 events/sec
Changes from 'take21' patchset:
* minor cleanups (different return values, removed unneded variables, whitespaces and so on)
* fixed bug in kevent removal in case when kevent being removed
is the same as overflow_kevent (spotted by Eric Dumazet)
Changes from 'take20' patchset:
* new ring buffer implementation
* removed artificial limit on possible number of kevents
With this release and fixed userspace web server it was possible to achive 3960+ req/s with client connection rate of 4000 con/s over 100 Mbit lan, data IO over network was about 10582.7 KB/s, which is too close to wire speed if we get into account headers and the like.
Changes from 'take19' patchset:
* use __init instead of __devinit
* removed 'default N' from config for user statistic
* removed kevent_user_fini() since kevent can not be unloaded
* use KERN_INFO for statistic output
Changes from 'take18' patchset:
* use __init instead of __devinit
* removed 'default N' from config for user statistic
* removed kevent_user_fini() since kevent can not be unloaded
* use KERN_INFO for statistic output
Changes from 'take17' patchset:
* Use RB tree instead of hash table.
At least for a web sever, frequency of addition/deletion of new kevent is comparable with number of search access, i.e. most of the time events are added, accesed only couple of times and then removed, so it justifies RB tree usage over AVL tree, since the latter does have much slower deletion time (max O(log(N)) compared to 3 ops),
although faster search time (1.44*O(log(N)) vs. 2*O(log(N))).
So for kevents I use RB tree for now and later, when my AVL tree implementation is ready, it will be possible to compare them.
* Changed readiness check for socket notifications.
With both above changes it is possible to achieve more than 3380 req/second compared to 2200, sometimes 2500 req/second for epoll() for trivial web-server and httperf client on the same hardware.
It is possible that above kevent limit is due to maximum allowed kevents in a time limit, which is 4096 events.
Changes from 'take16' patchset:
* misc cleanups (__read_mostly, const ...)
* created special macro which is used for mmap size (number of pages) calculation
* export kevent_socket_notify(), since it is used in network protocols which can be built as modules (IPv6 for example)
Changes from 'take15' patchset:
* converted kevent_timer to high-resolution timers, this forces timer API update at
http://linux-net.osdl.org/index.php/Kevent
* use struct ukevent* instead of void * in syscalls (documentation has been updated)
* added warning in kevent_add_ukevent() if ring has broken index (for testing)
Changes from 'take14' patchset:
* added kevent_wait()
This syscall waits until either timeout expires or at least one event becomes ready. It also commits that @num events from @start are processed by userspace and thus can be be removed or rearmed (depending on it's flags). It can be used for commit events read by userspace through mmap interface. Example userspace code (evtest.c) can be found on project's homepage.
* added socket notifications (send/recv/accept)
Changes from 'take13' patchset:
* do not get lock aroung user data check in __kevent_search()
* fail early if there were no registered callbacks for given type of kevent
* trailing whitespace cleanup
Changes from 'take12' patchset:
* remove non-chardev interface for initialization
* use pointer to kevent_mring instead of unsigned longs
* use aligned 64bit type in raw user data (can be used by high-res timer if needed)
* simplified enqueue/dequeue callbacks and kevent initialization
* use nanoseconds for timeout
* put number of milliseconds into timer's return data
* move some definitions into user-visible header
* removed filenames from comments
Changes from 'take11' patchset:
* include missing headers into patchset
* some trivial code cleanups (use goto instead of if/else games and so on)
* some whitespace cleanups
* check for ready_callback() callback before main loop which should save us some ticks
Changes from 'take10' patchset:
* removed non-existent prototypes
* added helper function for kevent_registered_callbacks
* fixed 80 lines comments issues
* added shared between userspace and kernelspace header instead of embedd them in one
* core restructuring to remove forward declarations
* s o m e w h i t e s p a c e c o d y n g s t y l e c l e a n u p
* use vm_insert_page() instead of remap_pfn_range()
Changes from 'take9' patchset:
* fixed ->nopage method
Changes from 'take8' patchset:
* fixed mmap release bug
* use module_init() instead of late_initcall()
* use better structures for timer notifications
Changes from 'take7' patchset:
* new mmap interface (not tested, waiting for other changes to be acked)
- use nopage() method to dynamically substitue pages
- allocate new page for events only when new added kevent requres it
- do not use ugly index dereferencing, use structure instead
- reduced amount of data in the ring (id and flags), maximum 12 pages on x86 per kevent fd
Changes from 'take6' patchset:
* a lot of comments!
* do not use list poisoning for detection of the fact, that entry is in the list
* return number of ready kevents even if copy*user() fails
* strict check for number of kevents in syscall
* use ARRAY_SIZE for array size calculation
* changed superblock magic number
* use SLAB_PANIC instead of direct panic() call
* changed -E* return values
* a lot of small cleanups and indent fixes
Changes from 'take5' patchset:
* removed compilation warnings about unused wariables when lockdep is not turned on
* do not use internal socket structures, use appropriate (exported) wrappers instead
* removed default 1 second timeout
* removed AIO stuff from patchset
Changes from 'take4' patchset:
* use miscdevice instead of chardevice
* comments fixes
Changes from 'take3' patchset:
* removed serializing mutex from kevent_user_wait()
* moved storage list processing to RCU
* removed lockdep screaming - all storage locks are initialized in the same function, so it was
learned
to differentiate between various cases
* remove kevent from storage if is marked as broken after callback
* fixed a typo in mmaped buffer implementation which would end up in wrong index calcualtion
Changes from 'take2' patchset:
* split kevent_finish_user() to locked and unlocked variants
* do not use KEVENT_STAT ifdefs, use inline functions instead
* use array of callbacks of each type instead of each kevent callback initialization
* changed name of ukevent guarding lock
* use only one kevent lock in kevent_user for all hash buckets instead of per-bucket locks
* do not use kevent_user_ctl structure instead provide needed arguments as syscall parameters
* various indent cleanups
* added optimisation, which is aimed to help when a lot of kevents are being copied from
userspace
* mapped buffer (initial) implementation (no userspace yet)
Changes from 'take1' patchset:
- rebased against 2.6.18-git tree
- removed ioctl controlling
- added new syscall kevent_get_events(int fd, unsigned int min_nr, unsigned int max_nr, unsigned int timeout, void __user *buf, unsigned flags)
- use old syscall kevent_ctl for creation/removing, modification and initial kevent initialization
- use mutuxes instead of semaphores
- added file descriptor check and return error if provided descriptor does not match kevent file operations
- various indent fixes
- removed aio_sendfile() declarations.
Thank you.
PATCH 1/5 kevent: Description.
Description.
int kevent_ctl(int fd, unsigned int cmd, unsigned int num, struct ukevent *arg);
fd - is the file descriptor referring to the kevent queue to manipulate. It is created by opening "/dev/kevent" char device, which is created with dynamic minor number and major number assigned for misc devices.
cmd - is the requested operation. It can be one of the following:
KEVENT_CTL_ADD - add event notification KEVENT_CTL_REMOVE - remove event notification KEVENT_CTL_MODIFY - modify existing notification
num - number of struct ukevent in the array pointed to by arg arg - array of struct ukevent
When called, kevent_ctl will carry out the operation specified in the cmd parameter.
-------------------------------------------------------------------------------------
int kevent_get_events(int ctl_fd, unsigned int min_nr, unsigned int max_nr, __u64 timeout, struct ukevent *buf, unsigned flags)
ctl_fd - file descriptor referring to the kevent queuemin_nr - minimum number of completed events that kevent_get_events will block waiting for max_nr - number of struct ukevent in buf
timeout - number of nanoseconds to wait before returning less than min_nr events.
If this is -1, then wait forever.
buf - pointer to an array of struct ukevent.
flags - unused
kevent_get_events will wait timeout milliseconds for at least min_nr completed events, copying completed struct ukevents to buf and deleting any KEVENT_REQ_ONESHOT event requests. In nonblocking mode it returns as many events as possible, but not more than max_nr. In blocking mode it waits until timeout or if at least min_nr events are ready.
-------------------------------------------------------------------------------------
int kevent_wait(int ctl_fd, unsigned int num, __u64 timeout)
ctl_fd - file descriptor referring to the kevent queue
num - number of processed kevents
timeout - this timeout specifies number of nanoseconds to wait until there is free space in kevent queue
This syscall waits until either timeout expires or at least one event becomes ready.
It also copies that num events into special ring buffer and requeues them (or removes depending on flags).
-------------------------------------------------------------------------------------
int kevent_ring_init(int ctl_fd, struct kevent_ring *ring, unsigned int num)
ctl_fd - file descriptor referring to the kevent queue num - size of the ring buffer in events
struct kevent_ring
{
unsigned int ring_kidx;
struct ukevent event[0];
}
ring_kidx - is an index in the ring buffer where kernel will put new events when
kevent_wait() or kevent_get_events() is called
Example userspace code (ring_buffer.c) can be found on project's homepage.
Each kevent syscall can be so called cancellation point in glibc, i.e. when thread has been cancelled in kevent syscall, thread can be safely removed and no events will be lost, since each syscall (kevent_wait() or kevent_get_events()) will copy event into special ring buffer, accessible from other threads or even processes (if shared memory is used).
When kevent is removed (not dequeued when it is ready, but just removed), even if it was ready, it is not copied into ring buffer, since if it is removed, no one cares about it (otherwise user would wait until it becomes ready and got it through usual way using kevent_get_events() or kevent_wait()) and thus no need to copy it to the ring buffer.
It is possible with userspace ring buffer, that events in the ring buffer can be replaced without knowledge for the thread currently reading them (when other thread calls kevent_get_events() or kevent_wait()), so appropriate locking between threads or processes, which can simultaneously access the same ring buffer, is required.
-------------------------------------------------------------------------------------
The bulk of the interface is entirely done through the ukevent struct. It is used to add event requests, modify existing event requests, specify which event requests to remove, and return completed events.
struct ukevent contains the following members:
struct kevent_id id
Id of this request, e.g. socket number, file descriptor and so on__u32 type
Event type, e.g. KEVENT_SOCK, KEVENT_INODE, KEVENT_TIMER and so on__u32 event
Event itself, e.g. SOCK_ACCEPT, INODE_CREATED, TIMER_FIRED__u32 req_flags
Per-event request flags,
KEVENT_REQ_ONESHOT
event will be removed when it is ready
KEVENT_REQ_WAKEUP_ONE
When several threads wait on the same kevent queue and requested the same event, for example 'wake me up when new client has connected, so I could call accept()', then all threads will be awakened when new client has connected, but only one of them can process the data. This problem is known as thundering nerd problem. Events which have this flag set will not be marked as ready (and appropriate threads will not be awakened) if at least one event has been already marked.
KEVENT_REQ_ET
Edge Triggered behaviour. It is an optimisation which allows to move ready and dequeued (i.e. copied to userspace) event to move into set of interest for given storage (socket, inode and so on) again. It is very usefull for cases when the same event should be used many times (like reading from pipe). It is similar to epoll()'s EPOLLET flag.
__u32 ret_flags
Per-event return flags
KEVENT_RET_BROKEN
Kevent is broken
KEVENT_RET_DONE
Kevent processing was finished successfully
KEVENT_RET_COPY_FAILED
Kevent was not copied into ring buffer due to some error conditions.
__u32 ret_data
Event return data. Event originator fills it with anything it likes (for example timer notifications put number of milliseconds when timer has fired
union { __u32 user[2]; void *ptr; }
User's data. It is not used, just copied to/from user. The whole structure is aligned to 8 bytes already, so the last union is aligned properly.
---------------------------------------------------------------------------------
Usage
For KEVENT_CTL_ADD, all fields relevant to the event type must be filled
(id, type, possibly event, req_flags). After kevent_ctl(..., KEVENT_CTL_ADD, ...) returns each struct's ret_flags should be checked to see if the event is already broken or done.
For KEVENT_CTL_MODIFY, the id, req_flags, and user and event fields must be set and an existing kevent request must have matching id and user fields. If a match is found, req_flags and event are replaced with the newly supplied values and requeueing is started, so modified kevent can be checked and probably marked as ready immediately. If a match can't be found, the passed in ukevent's ret_flags has KEVENT_RET_BROKEN set. KEVENT_RET_DONE is always set.
For KEVENT_CTL_REMOVE, the id and user fields must be set and an existing kevent request must have matching id and user fields. If a match is found, the kevent request is removed. If a match can't be found, the passed in ukevent's ret_flags has KEVENT_RET_BROKEN set. KEVENT_RET_DONE is always set.
For kevent_get_events, the entire structure is returned.
---------------------------------------------------------------------------------
Usage cases
kevent_timer
struct ukevent should contain following fields:
type - KEVENT_TIMER
event - KEVENT_TIMER_FIRED
req_flags - KEVENT_REQ_ONESHOT if you want to fire that timer only once id.raw[0] - number of seconds after commit when this timer shout expire id.raw[0] - additional to number of seconds number of nanoseconds
PATCH 2/5 kevent: Core files.
Core files.
This patch includes core kevent files:
* userspace controlling
* kernelspace interfaces
* initialization
* notification state machines
Some bits of documentation can be found on project's homepage (and links from there):
http://tservice.net.ru/~s0mbre/old/?section=projects&item=kevent
arch/i386/kernel/syscall_table.S +4 -0
@@ -318,3 +318,7 @@ ENTRY(sys_call_table) .long sys_vmsplice .long sys_move_pages .long sys_getcpu + .long sys_kevent_get_events + .long sys_kevent_ctl /* 320 */ + .long sys_kevent_wait + .long sys_kevent_ring_init
arch/x86_64/ia32/ia32entry.S +5 -1
@@ -714,8 +714,12 @@ #endif .quad compat_sys_get_robust_list .quad sys_splice .quad sys_sync_file_range - .quad sys_tee + .quad sys_tee /* 315 */ .quad compat_sys_vmsplice .quad compat_sys_move_pages .quad sys_getcpu + .quad sys_kevent_get_events + .quad sys_kevent_ctl /* 320 */ + .quad sys_kevent_wait + .quad sys_kevent_ring_init ia32_syscall_end:
include/asm-i386/unistd.h +5 -1
@@ -324,10 +324,14 @@ #define __NR_tee 315#define __NR_vmsplice 316#define __NR_move_pages 317#define __NR_getcpu 318 +#define __NR_kevent_get_events 319 +#define __NR_kevent_ctl 320 +#define __NR_kevent_wait 321 +#define __NR_kevent_ring_init 322#ifdef __KERNEL__ -#define NR_syscalls 319 +#define NR_syscalls 323#include <linux/err.h> /*
include/asm-x86_64/unistd.h +9 -1
@@ -619,10 +619,18 @@ #define __NR_vmsplice 278 __SYSCALL(__NR_vmsplice, sys_vmsplice)#define __NR_move_pages 279 __SYSCALL(__NR_move_pages, sys_move_pages) +#define __NR_kevent_get_events 280 +__SYSCALL(__NR_kevent_get_events, sys_kevent_get_events) +#define __NR_kevent_ctl 281 +__SYSCALL(__NR_kevent_ctl, sys_kevent_ctl) +#define __NR_kevent_wait 282 +__SYSCALL(__NR_kevent_wait, sys_kevent_wait) +#define __NR_kevent_ring_init 283 +__SYSCALL(__NR_kevent_ring_init, sys_kevent_ring_init)#ifdef __KERNEL__ -#define __NR_syscall_max __NR_move_pages +#define __NR_syscall_max __NR_kevent_ring_init#include <linux/err.h> #ifndef __NO_STUBS
include/linux/kevent.h new file +201 -0
201 lines not on this page · raw message
include/linux/kevent_storage.h new file +11 -0
@@ -0,0 +1,11 @@ +#ifndef __KEVENT_STORAGE_H +#define __KEVENT_STORAGE_H + +struct kevent_storage +{ +void *origin;/* Originator's pointer, e.g. struct sock or struct file. Can be NULL. */ +struct list_head list;/* List of queued kevents. */ +spinlock_t lock;/* Protects users queue. */ +}; + +#endif /* __KEVENT_STORAGE_H */
include/linux/syscalls.h +7 -0
@@ -54,6 +54,8 @@ struct compat_stat;struct compat_timeval ;struct robust_list_head ;struct getcpu_cache ; +struct ukevent ; +struct kevent_ring ;#include <linux/types.h> #include <linux/aio_abi.h>
@@ -599,4 +601,9 @@ asmlinkage long sys_set_robust_list(stru size_t len);asmlinkage long sys_getcpu (unsigned __user *cpu,unsigned __user *node,struct getcpu_cache __user *cache); +asmlinkage long sys_kevent_get_events (int ctl_fd,unsigned int min,unsigned int max, +__u64 timeout,struct ukevent __user *buf,unsigned flags); +asmlinkage long sys_kevent_ctl (int ctl_fd,unsigned int cmd,unsigned int num,struct ukevent __user *buf); +asmlinkage long sys_kevent_wait (int ctl_fd,unsigned int num,__u64 timeout); +asmlinkage long sys_kevent_ring_init (int ctl_fd,struct kevent_ring __user *ring,unsigned int num);#endif
include/linux/ukevent.h new file +153 -0
153 lines not on this page · raw message
init/Kconfig +2 -0
@@ -201,6 +201,8 @@ config AUDITSYSCALL such as SELinux. To use audit's filesystem watch feature, please ensure that INOTIFY is configured. +source "kernel/kevent/Kconfig" + config IKCONFIG bool "Kernel .config support" ---help---
kernel/Makefile +1 -0
@@ -47,6 +47,7 @@ obj-$(CONFIG_DETECT_SOFTLOCKUP) += softl obj-$(CONFIG_GENERIC_HARDIRQS) += irq/ obj-$(CONFIG_SECCOMP) += seccomp.o obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o +obj-$(CONFIG_KEVENT) += kevent/ obj-$(CONFIG_RELAY) += relay.o obj-$(CONFIG_TASK_DELAY_ACCT) += delayacct.o obj-$(CONFIG_TASKSTATS) += taskstats.o
kernel/kevent/Kconfig new file +39 -0
@@ -0,0 +1,39 @@ +config KEVENT + bool "Kernel event notification mechanism" + help + This option enables event queue mechanism. + It can be used as replacement for poll()/select(), AIO callback + invocations, advanced timer notifications and other kernel + object status changes. + +config KEVENT_USER_STAT + bool "Kevent user statistic" + depends on KEVENT + help + This option will turn kevent_user statistic collection on. + Statistic data includes total number of kevent, number of kevents + which are ready immediately at insertion time and number of kevents + which were removed through readiness completion. + It will be printed each time control kevent descriptor is closed. + +config KEVENT_TIMER + bool "Kernel event notifications for timers" + depends on KEVENT + help + This option allows to use timers through KEVENT subsystem. + +config KEVENT_POLL + bool "Kernel event notifications for poll()/select()" + depends on KEVENT + help + This option allows to use kevent subsystem for poll()/select() + notifications. + +config KEVENT_SOCKET + bool "Kernel event notifications for sockets" + depends on NET && KEVENT + help + This option enables notifications through KEVENT subsystem of + sockets operations, like new packet receiving conditions, + ready for accept conditions and so on. +
kernel/kevent/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +obj-y := kevent.o kevent_user.o +obj-$(CONFIG_KEVENT_TIMER) += kevent_timer.o +obj-$(CONFIG_KEVENT_POLL) += kevent_poll.o +obj-$(CONFIG_KEVENT_SOCKET) += kevent_socket.o
kernel/kevent/kevent.c new file +232 -0
232 lines not on this page · raw message
kernel/kevent/kevent_user.c new file +913 -0
913 lines not on this page · raw message
kernel/sys_ni.c +5 -0
@@ -122,6 +122,11 @@ cond_syscall(ppc_rtas);cond_syscall (sys_spu_run);cond_syscall (sys_spu_create); +cond_syscall (sys_kevent_get_events); +cond_syscall (sys_kevent_wait); +cond_syscall (sys_kevent_ctl); +cond_syscall (sys_kevent_ring_init); +/* mmu depending weak syscall entries */ cond_syscall (sys_mprotect);cond_syscall (sys_msync);
PATCH 3/5 kevent: poll/select() notifications.
poll/select() notifications.
This patch includes generic poll/select notifications. kevent_poll works simialr to epoll and has the same issues (callback is invoked not from internal state machine of the caller, but through process awake, a lot of allocations and so on).
include/linux/fs.h +8 -0
@@ -276,6 +276,7 @@ #include <linux/prio_tree.h>#include <linux/init.h> #include <linux/sched.h> #include <linux/mutex.h> +#include <linux/kevent.h> #include <asm/atomic.h> #include <asm/semaphore.h>
@@ -586,6 +587,10 @@ #ifdef CONFIG_INOTIFYstruct mutex inotify_mutex;/* protects the watches list */ #endif +#ifdef CONFIG_KEVENT_SOCKET +struct kevent_storage st; +#endif +unsigned long i_state;unsigned long dirtied_when;/* jiffies of first dirtying */
@@ -739,6 +744,9 @@ #ifdef CONFIG_EPOLLstruct list_head f_ep_links;spinlock_t f_ep_lock;#endif /* #ifdef CONFIG_EPOLL */ +#ifdef CONFIG_KEVENT_POLL +struct kevent_storage st; +#endif struct address_space *f_mapping; };extern spinlock_t files_lock;
kernel/kevent/kevent_poll.c new file +222 -0
222 lines not on this page · raw message
PATCH 4/5 kevent: Socket notifications.
Socket notifications.
This patch includes socket send/recv/accept notifications. Using trivial web server based on kevent and this features instead of epoll it's performance increased more than noticebly. More details about various benchmarks and server itself (evserver_kevent.c) can be found on project's homepage.
fs/inode.c +7 -0
@@ -21,6 +21,7 @@ #include <linux/pagemap.h>#include <linux/cdev.h> #include <linux/bootmem.h> #include <linux/inotify.h> +#include <linux/kevent.h> #include <linux/mount.h> /*
@@ -164,12 +165,18 @@ #endif } inode->i_private =0 ; inode->i_mapping = mapping; +#if definedCONFIG_KEVENT_SOCKET +kevent_storage_init (inode, &inode->st); +#endif }return inode; }void destroy_inode (struct inode *inode) { +#if definedCONFIG_KEVENT_SOCKET +kevent_storage_fini (&inode->st); +#endif BUG_ON (inode_has_buffers (inode));security_inode_free (inode);if (inode->i_sb->s_op->destroy_inode)
include/net/sock.h +17 -15
@@ -48,6 +48,7 @@ #include <linux/lockdep.h>#include <linux/netdevice.h> #include <linux/skbuff.h> /* struct sk_buff */ #include <linux/security.h> +#include <linux/kevent.h> #include <linux/filter.h>
@@ -450,6 +451,21 @@ static inline int sk_stream_memory_free(extern void sk_stream_rfree (struct sk_buff *skb); +struct socket_alloc { +struct socket socket; +struct inode vfs_inode; +}; + +static inline struct socket *SOCKET_I (struct inode *inode) +{ + return &container_of(inode, struct socket_alloc, vfs_inode)->socket; +} + +static inline struct inode *SOCK_INODE (struct socket *socket) +{ + return &container_of(socket, struct socket_alloc, socket)->vfs_inode; +} +static inline void sk_stream_set_owner_r (struct sk_buff *skb,struct sock *sk) { skb->sk = sk;
@@ -477,6 +493,7 @@ static inline void sk_add_backlog(struct sk->sk_backlog.tail = skb; } skb->next =NULL ; +kevent_socket_notify (sk,KEVENT_SOCKET_RECV ); }#define sk_wait_event (__sk, __timeo, __condition) \
@@ -679,21 +696,6 @@ static inline struct kiocb *siocb_to_kioreturn si->kiocb; } -struct socket_alloc { -struct socket socket; -struct inode vfs_inode; -}; - -static inline struct socket *SOCKET_I (struct inode *inode) -{ - return &container_of(inode, struct socket_alloc, vfs_inode)->socket; -} - -static inline struct inode *SOCK_INODE (struct socket *socket) -{ - return &container_of(socket, struct socket_alloc, socket)->vfs_inode; -} -extern void __sk_stream_mem_reclaim (struct sock *sk);extern int sk_stream_mem_schedule (struct sock *sk,int size,int kind);
include/net/tcp.h +1 -0
@@ -857,6 +857,7 @@ static inline int tcp_prequeue(struct so tp->ucopy.memory =0 ; }else if (skb_queue_len (&tp->ucopy.prequeue) ==1 ) {wake_up_interruptible (sk->sk_sleep); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND );if (!inet_csk_ack_scheduled (sk)) inet_csk_reset_xmit_timer(sk, ICSK_TIME_DACK, (3 * TCP_RTO_MIN) / 4,
kernel/kevent/kevent_socket.c new file +135 -0
135 lines not on this page · raw message
net/core/sock.c +9 -1
@@ -1402,6 +1402,7 @@ static void sock_def_wakeup(struct sockif (sk->sk_sleep &&waitqueue_active (sk->sk_sleep))wake_up_interruptible_all (sk->sk_sleep);read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_error_report (struct sock *sk)
@@ -1411,6 +1412,7 @@ static void sock_def_error_report(structwake_up_interruptible (sk->sk_sleep);sk_wake_async (sk,0 ,POLL_ERR );read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_readable (struct sock *sk,int len)
@@ -1420,6 +1422,7 @@ static void sock_def_readable(struct socwake_up_interruptible (sk->sk_sleep);sk_wake_async (sk,1 ,POLL_IN );read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_write_space (struct sock *sk)
@@ -1439,6 +1442,7 @@ static void sock_def_write_space(struct }read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_SEND |KEVENT_SOCKET_RECV ); }static void sock_def_destruct (struct sock *sk)
@@ -1489,6 +1493,8 @@ #endif sk->sk_state =TCP_CLOSE ; sk->sk_socket = sock; +kevent_sk_reinit (sk); +sock_set_flag (sk,SOCK_ZAPPED );if (sock)
@@ -1555,8 +1561,10 @@ void fastcall release_sock(struct sock *if (sk->sk_backlog.tail)__release_sock (sk); sk->sk_lock.owner =NULL ; -if (waitqueue_active (&sk->sk_lock.wq)) +if (waitqueue_active (&sk->sk_lock.wq)) {wake_up (&sk->sk_lock.wq); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); + }spin_unlock_bh (&sk->sk_lock.slock); }EXPORT_SYMBOL (release_sock);
net/core/stream.c +1 -0
@@ -36,6 +36,7 @@ void sk_stream_write_space(struct sock *wake_up_interruptible (sk->sk_sleep);if (sock->fasync_list && !(sk->sk_shutdown &SEND_SHUTDOWN ))sock_wake_async (sock,2 ,POLL_OUT ); +kevent_socket_notify (sk,KEVENT_SOCKET_SEND |KEVENT_SOCKET_RECV ); } }
net/ipv4/tcp_input.c +1 -0
@@ -3119,6 +3119,7 @@ static void tcp_ofo_queue(struct sock *s__skb_unlink (skb, &tp->out_of_order_queue);__skb_queue_tail (&sk->sk_receive_queue, skb); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV ); tp->rcv_nxt =TCP_SKB_CB (skb)->end_seq;if (skb->h.th->fin)tcp_fin (skb, sk, skb->h.th);
net/ipv4/tcp_ipv4.c +2 -0
@@ -61,6 +61,7 @@ #include <linux/cache.h>#include <linux/jhash.h> #include <linux/init.h> #include <linux/times.h> +#include <linux/kevent.h> #include <net/icmp.h> #include <net/inet_hashtables.h>
@@ -870,6 +871,7 @@ #endifreqsk_free (req); }else {inet_csk_reqsk_queue_hash_add (sk, req,TCP_TIMEOUT_INIT ); +kevent_socket_notify (sk,KEVENT_SOCKET_ACCEPT ); }return 0 ;
net/socket.c +3 -0
@@ -85,6 +85,7 @@ #include <linux/compat.h>#include <linux/kmod.h> #include <linux/audit.h> #include <linux/wireless.h> +#include <linux/kevent.h> #include <asm/uaccess.h> #include <asm/unistd.h>
@@ -490,6 +491,8 @@ static struct socket *sock_alloc(void) inode->i_uid = current->fsuid; inode->i_gid = current->fsgid; +kevent_socket_reinit (sock); +get_cpu_var (sockets_in_use)++;put_cpu_var (sockets_in_use);return sock;
PATCH 5/5 kevent: Timer notifications.
Timer notifications.
Timer notifications can be used for fine grained per-process time management, since interval timers are very inconvenient to use, and they are limited.
This subsystem uses high-resolution timers.
id.raw[0] is used as number of seconds
id.raw[1] is used as number of nanoseconds
kernel/kevent/kevent_timer.c new file +112 -0
@@ -0,0 +1,112 @@ +/* +* 2006 Copyright (c) Evgeniy Polyakov <[email protected]> +* All rights reserved. +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +*/ + +#include <linux/kernel.h> +#include <linux/types.h> +#include <linux/list.h> +#include <linux/slab.h> +#include <linux/spinlock.h> +#include <linux/hrtimer.h> +#include <linux/jiffies.h> +#include <linux/kevent.h> + +struct kevent_timer +{ +struct hrtimer ktimer; +struct kevent_storage ktimer_storage; +struct kevent *ktimer_event; +}; + +static int kevent_timer_func (struct hrtimer *timer) +{ + struct kevent_timer *t = container_of(timer, struct kevent_timer, ktimer); +struct kevent *k = t->ktimer_event; + +kevent_storage_ready (&t->ktimer_storage,NULL ,KEVENT_MASK_ALL ); +hrtimer_forward (timer, timer->base->softirq_time, +ktime_set (k->event.id.raw[0 ], k->event.id.raw[1 ])); +return HRTIMER_RESTART ; +} + +static struct lock_class_key kevent_timer_key; + +static int kevent_timer_enqueue (struct kevent *k) +{ +int err; +struct kevent_timer *t; + + t =kmalloc (sizeof (struct kevent_timer ),GFP_KERNEL ); +if (!t) +return -ENOMEM ; + +hrtimer_init (&t->ktimer,CLOCK_MONOTONIC ,HRTIMER_REL ); + t->ktimer.expires =ktime_set (k->event.id.raw[0 ], k->event.id.raw[1 ]); + t->ktimer.function = kevent_timer_func; + t->ktimer_event = k; + + err =kevent_storage_init (&t->ktimer, &t->ktimer_storage); +if (err) + goto err_out_free; +lockdep_set_class (&t->ktimer_storage.lock, &kevent_timer_key); + + err =kevent_storage_enqueue (&t->ktimer_storage, k); +if (err) + goto err_out_st_fini; + +hrtimer_start (&t->ktimer, t->ktimer.expires,HRTIMER_REL ); + +return 0 ; + +err_out_st_fini: +kevent_storage_fini (&t->ktimer_storage); +err_out_free: +kfree (t); + +return err; +} + +static int kevent_timer_dequeue (struct kevent *k) +{ +struct kevent_storage *st = k->st; + struct kevent_timer *t = container_of(st, struct kevent_timer, ktimer_storage); + +hrtimer_cancel (&t->ktimer); +kevent_storage_dequeue (st, k); +kfree (t); + +return 0 ; +} + +static int kevent_timer_callback (struct kevent *k) +{ + k->event.ret_data[0 ] =jiffies_to_msecs (jiffies); +return 1 ; +} + +static int __init kevent_init_timer (void ) +{ +struct kevent_callbacks tc = { + .callback = &kevent_timer_callback, + .enqueue = &kevent_timer_enqueue, + .dequeue = &kevent_timer_dequeue}; + +return kevent_add_callbacks (&tc,KEVENT_TIMER ); +} +module_init (kevent_init_timer); +
Re: [take23 3/5] kevent: poll/select() notifications.
In reply to Evgeniy PolyakovQuoted 5 lines · Evgeniy Polyakov
kernel/kevent/kevent_poll.c new file +5
-0+static int kevent_poll_wait_callback (wait_queue_t *wait, +unsigned mode,int sync,void *key) +{ +struct kevent_poll_wait_container *cont = + container_of(wait, struct kevent_poll_wait_container, wait);
kernel/kevent/kevent_poll.c line 68
Quoted 10 lines
kernel/kevent/kevent_poll.c new file +10
-0+struct kevent *k = cont->k; +struct file *file = k->st->origin; +u32 revents; + + revents = file->f_op->poll (file,NULL ); + +kevent_storage_ready (k->st,NULL , revents); + +return 0 ; +}
Are you sure you can safely call file->f_op->poll() from inside a callback based wakeup? The low level driver may be calling the wakeup with one of its locks held, and during the file->f_op->poll may be trying to acquire the same lock. I remember there was a discussion about this, and assuming the above not true, made epoll code more complex (and slower, since an extra O(R) loop was needed to fetch events).
- Davide
Re: [take23 3/5] kevent: poll/select() notifications.
Quoted 16 lines · Davide Libenzi
Quoted 14 lines · Evgeniy Polyakov
kernel/kevent/kevent_poll.c new file +14
-0+static int kevent_poll_wait_callback (wait_queue_t *wait, +unsigned mode,int sync,void *key) +{ +struct kevent_poll_wait_container *cont = + container_of(wait, struct kevent_poll_wait_container, wait); +struct kevent *k = cont->k; +struct file *file = k->st->origin; +u32 revents; + + revents = file->f_op->poll (file,NULL ); + +kevent_storage_ready (k->st,NULL , revents); + +return 0 ;
Quoted 8 lines
Quoted 1 line
+}
Are you sure you can safely call
file->f_op->poll()from inside a callback based wakeup? The low level driver may be calling the wakeup with one of its locks held, and during thefile->f_op->pollmay be trying to acquire the same lock. I remember there was a discussion about this, and assuming the above not true, made epoll code more complex (and slower, since an extra O(R) loop was needed to fetch events).
Indeed, I have not paid too much attention to poll/select notifications in kevent actually. As far as I recall it should be called on behalf of process doing kevent_get_event(). I will check and fix if that is not correct. Thanks Davide.
Quoted 2 lines
- Davide
Signature · 1 line
Evgeniy Polyakov
Re: [take23 3/5] kevent: poll/select() notifications.
Quoted 11 lines · Evgeniy Polyakov
Quoted 6 lines
Are you sure you can safely call
file->f_op->poll()from inside a callback based wakeup? The low level driver may be calling the wakeup with one of its locks held, and during thefile->f_op->pollmay be trying to acquire the same lock. I remember there was a discussion about this, and assuming the above not true, made epoll code more complex (and slower, since an extra O(R) loop was needed to fetch events).Indeed, I have not paid too much attention to poll/select notifications in kevent actually. As far as I recall it should be called on behalf of process doing
kevent_get_event(). I will check and fix if that is not correct. Thanks Davide.
Indeed there was a bug.
Actually poll/select patch was broken quite noticebly - patchset did not include major changes I made for it.
I will put them all into next release.
Thanks again Davide for pointing that out.
Signature · 1 line
Evgeniy Polyakov
Quoted 5 lines · On Tue, 7 Nov 2006 19:50:48 +0300 Evgeniy Polyakov <[email protected]> wrote:
This patch includes core kevent files:
* userspace controlling
* kernelspace interfaces
* initialization
* notification state machines
I fixed up all the rejects, but your syscall numbers changed. Please always raise patches against the latest kernel.
Quoted 11 lines · Andrew Morton
Quoted 5 lines · On Tue, 7 Nov 2006 19:50:48 +0300 Evgeniy Polyakov <[email protected]> wrote:
This patch includes core kevent files:
* userspace controlling
* kernelspace interfaces
* initialization
* notification state machinesI fixed up all the rejects, but your syscall numbers changed. Please always raise patches against the latest kernel.
Will do. NUmbers actually are the same, but added new syscall which was against old tree.
Thanks Andrew.
Signature · 1 line
Evgeniy Polyakov
Re: [take23 1/5] kevent: Description.
In reply to Evgeniy PolyakovQuoted 1 line · On Tue, 7 Nov 2006 19:50:48 +0300 Evgeniy Polyakov <[email protected]> wrote:
Description.
I converted this into Documentation/kevent.txt. It looks like crap in an 80-col xterm btw.
Re: [take23 1/5] kevent: Description.
Quoted 7 lines · Andrew Morton
Quoted 1 line · On Tue, 7 Nov 2006 19:50:48 +0300 Evgeniy Polyakov <[email protected]> wrote:
Description.
I converted this into
Documentation/kevent.txt. It looks like crap in an 80-col xterm btw.
Thanks.
It was copied as is from documentation page, so it does looks like crap in non-browser window. I'm quite sure there will be some questions about kevent, so I will update that file and fix indent.
Signature · 1 line
Evgeniy Polyakov
Re: [take23 0/5] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovQuoted 1 line · On Tue, 7 Nov 2006 19:50:48 +0300 Evgeniy Polyakov <[email protected]> wrote:
Generic event handling mechanism.
I updated the version in -mm to v23. So people can play with it and review it. It looks like a bit of work will be needed to get it to compile.
It seems that most of the fixes which were added to the previous version were merged or are now irrelevant, however you lost this change:
From: Andrew Morton <[email protected]>
If kevent_user_wait() gets -EFAULT on the attempt to copy the first event, it will return 0, which is indistinguishable from "no events pending".
It can and should return EFAULT in this case.
---
kernel/kevent/kevent_user.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletion(-)
kernel/kevent/kevent_user.c renamed from kernel/kevent/kevent_user.c~kevent_user_wait-retval-fix +4 -1
@@ -690,8 +690,11 @@ static int kevent_user_wait(struct filewhile (num < max_nr && ((k =kqueue_dequeue_ready (u)) !=NULL )) {if (copy_to_user (buf + num*sizeof (struct ukevent ), - &k->event,sizeof (struct ukevent ))) + &k->event,sizeof (struct ukevent ))) { +if (num ==0 ) + num = -EFAULT ;break ; + }kevent_complete_ready (k); ++num;kevent_stat_wait (u);
_
Re: [take23 0/5] kevent: Generic event handling mechanism.
Quoted 6 lines · Andrew Morton
From: Andrew Morton <[email protected]>
If
kevent_user_wait()gets-EFAULTon the attempt to copy the first event, it will return 0, which is indistinguishable from "no events pending".It can and should return EFAULT in this case.
Correct, I missed that.
Thanks Andrew, I will put into my tree, -mm seems to have it already.
Signature · 1 line
Evgeniy Polyakov
Re: [take23 0/5] kevent: Generic event handling mechanism.
Quoted 1 line · Evgeniy Polyakov
On Tue, Nov 07, 2006 at 02:17:18PM -0800, Andrew Morton ([email protected])
Quoted 10 lines · wrote:
Quoted 7 lines
From: Andrew Morton <[email protected]>
If
kevent_user_wait()gets-EFAULTon the attempt to copy the first event, it will return 0, which is indistinguishable from "no events pending".It can and should return EFAULT in this case.
Correct, I missed that.
Thanks Andrew, I will put into my tree, -mm seems to have it already.
I believe eventpoll has a similar problem. Not a big problem, but we can be cleaner. Normally, the access_ok() done in sys_epoll_wait() should catch non writeable user area, unless another thread play VM game (the thread in sys_epoll_wait() can sleep)
[PATCH] eventpoll : In case a fault occurs during copy_to_user(), we should report the count of events that were successfully copied into user space, instead of EFAULT. That would be consistent with behavior of read/write() syscalls for example.
fs/eventpoll.c +1 -1
@@ -1447,7 +1447,7 @@ &events[eventcnt].events) ||__put_user (epi->event.data, &events[eventcnt].data)) - return -EFAULT; + return eventcnt ? eventcnt : -EFAULT;if (epi->event.events &EPOLLONESHOT ) epi->event.events &=EP_PRIVATE_BITS ; eventcnt++;
Re: [take23 0/5] kevent: Generic event handling mechanism.
Quoted 14 lines · On Wed, 8 Nov 2006 15:51:13 +0100 Eric Dumazet <[email protected]> wrote:
[PATCH] eventpoll : In case a fault occurs during
copy_to_user(), we should report the count of events that were successfully copied into user space, instead of EFAULT. That would be consistent with behavior of read/write() syscalls for example.Signed-off-by Eric Dumazet <[email protected]>[eventpoll.patch text/plain (424B)]
fs/eventpoll.c line 1453
Quoted 8 lines
fs/eventpoll.c +1
-1@@ -1447,7 +1447,7 @@ &events[eventcnt].events) ||__put_user (epi->event.data, &events[eventcnt].data)) - return -EFAULT; + return eventcnt ? eventcnt : -EFAULT;if (epi->event.events &EPOLLONESHOT ) epi->event.events &=EP_PRIVATE_BITS ; eventcnt++;
Definitely a better interface, but I wonder if it's too late to change it.
An app which does
if (epoll_wait(...) == -1) barf(errno); else assume_all_events_were_received();
will now do the wrong thing.
otoh, such an applciation basically _has_ to use the epoll_wait() return value to work out how many events it received, so maybe it's OK...
Re: [take23 0/5] kevent: Generic event handling mechanism.
Quoted 35 lines · Andrew Morton
Quoted 14 lines · On Wed, 8 Nov 2006 15:51:13 +0100 Eric Dumazet <[email protected]> wrote:
[PATCH] eventpoll : In case a fault occurs during
copy_to_user(), we should report the count of events that were successfully copied into user space, instead of EFAULT. That would be consistent with behavior of read/write() syscalls for example.Signed-off-by Eric Dumazet <[email protected]>[eventpoll.patch text/plain (424B)]
fs/eventpoll.c line 1453
Quoted 8 lines
fs/eventpoll.c +1
-1@@ -1447,7 +1447,7 @@ &events[eventcnt].events) ||__put_user (epi->event.data, &events[eventcnt].data)) - return -EFAULT; + return eventcnt ? eventcnt : -EFAULT;if (epi->event.events &EPOLLONESHOT ) epi->event.events &=EP_PRIVATE_BITS ; eventcnt++;Definitely a better interface, but I wonder if it's too late to change it.
An app which does
if (epoll_wait(...) == -1) barf(errno); else assume_all_events_were_received();
Quoted 4 lines
will now do the wrong thing.
otoh, such an applciation basically _has_ to use the
epoll_wait()return value to work out how many events it received, so maybe it's OK...
I don't care about both ways, but sys_poll() does the same thing epoll does right now, so I would not change epoll behaviour.
- Davide
Re: [take23 0/5] kevent: Generic event handling mechanism.
Quoted 33 lines · Davide Libenzi
Quoted 31 lines · Andrew Morton
Quoted 11 lines · On Wed, 8 Nov 2006 15:51:13 +0100 Eric Dumazet <[email protected]> wrote:
[PATCH] eventpoll : In case a fault occurs during
copy_to_user(), we should report the count of events that were successfully copied into user space, instead of EFAULT. That would be consistent with behavior of read/write() syscalls for example.Signed-off-by Eric Dumazet <[email protected]>[eventpoll.patch text/plain (424B)]
fs/eventpoll.c line 1453
Quoted 8 lines
fs/eventpoll.c +1
-1@@ -1447,7 +1447,7 @@ &events[eventcnt].events) ||__put_user (epi->event.data, &events[eventcnt].data)) - return -EFAULT; + return eventcnt ? eventcnt : -EFAULT;if (epi->event.events &EPOLLONESHOT ) epi->event.events &=EP_PRIVATE_BITS ; eventcnt++;Definitely a better interface, but I wonder if it's too late to change it.
An app which does
if (epoll_wait(...) == -1) barf(errno); else assume_all_events_were_received();
Quoted 7 lines
Quoted 4 lines
will now do the wrong thing.
otoh, such an applciation basically _has_ to use the
epoll_wait()return value to work out how many events it received, so maybe it's OK...I don't care about both ways, but
sys_poll()does the same thing epoll does right now, so I would not change epoll behaviour.
Sure poll() cannot return a partial count, since its return value is :
On success, a positive number is returned, where the number returned is
the number of structures which have non-zero revents fields (in other words, those descriptors with events or errors reported).
poll() is non destructive (it doesnt change any state into kernel). Returning EFAULT in case of an error in the very last bit of user area is mandatory.
On the contrary :
epoll_wait() does return a count of transfered events, and update some state in kernel (it consume Edge Trigered events : They can be lost forever if not reported to user)
So epoll_wait() is much more like read(), that also updates file state in kernel (current file position)
Re: [take23 0/5] kevent: Generic event handling mechanism.
Quoted 15 lines · Eric Dumazet
Davide Libenzi a �crit :
Quoted 4 lines
I don't care about both ways, but
sys_poll()does the same thing epoll does right now, so I would not change epoll behaviour.Sure
poll()cannot return a partial count, since its return value is :On success, a positive number is returned, where the number returned is
the number of structures which have non-zero revents fields (in other words, those descriptors with events or errors reported).
poll()is non destructive (it doesnt change any state into kernel). Returning EFAULT in case of an error in the very last bit of user area is mandatory.
Quoted 8 lines
On the contrary :
epoll_wait()does return a count of transfered events, and update some state in kernel (it consume Edge Trigered events : They can be lost forever if not reported to user)So
epoll_wait()is much more likeread(), that also updates file state in kernel (current file position)
Lost forever means? If there are more processes watching some fd (external events), they all get their own copy of the events in their own private epoll fd. It's not that we "steal" things out of the kernel, is not a 1:1 producer/consumer thing (one producer, 1 queue). It's one producer, broadcast to all listeners (consumers) thing. The only case where it'd matter is in the case of multiple threads sharing the same epoll fd.
In general, I'd be more for having the userspace get his own SEGFAULT instead of letting it go with broken parameters. If I'm coding userspace, and I'm doing something wrong, I like the kernel to let me know, instead of trying to fix things for me.
Also, epoll can easily be fixed (add a param to ep_reinject_items() to re-inject items in case of error/EFAULT) to leave events in the ready-list and let the EFAULT emerge.
Anyone else has opinions about this?
PS: Next time it'd be great if you Cc: me when posting epoll patches, so
you avoid Andrew the job of doing it.
- Davide
Re: [take23 0/5] kevent: Generic event handling mechanism.
Quoted 17 lines · Davide Libenzi
Quoted 15 lines · Eric Dumazet
Davide Libenzi a ?crit :
Quoted 3 lines
I don't care about both ways, but
sys_poll()does the same thing epoll does right now, so I would not change epoll behaviour.Sure
poll()cannot return a partial count, since its return value is :On success, a positive number is returned, where the number returned is
the number of structures which have non-zero revents fields (in other words, those descriptors with events or errors reported).
poll()is non destructive (it doesnt change any state into kernel). Returning EFAULT in case of an error in the very last bit of user area is mandatory.On the contrary :
Quoted 8 lines
Quoted 6 lines
epoll_wait()does return a count of transfered events, and update some state in kernel (it consume Edge Trigered events : They can be lost forever if not reported to user)So
epoll_wait()is much more likeread(), that also updates file state in kernel (current file position)Lost forever means? If there are more processes watching some fd (external events), they all get their own copy of the events in their own private epoll fd. It's not that we "steal" things out of the kernel, is not a 1:1 producer/consumer thing (one producer, 1 queue). It's one producer, broadcast to all listeners (consumers) thing. The only case where it'd matter is in the case of multiple threads sharing the same epoll fd.
In my particular epoll application, the producer is tcp stack, and I have one consumer. If an network event is lost in the EFAULT handling, its lost forever. In any case, my application do provide a correct user area, so this problem is only theorical.
Quoted 2 lines
In general, I'd be more for having the userspace get his own SEGFAULT instead of letting it go with broken parameters. If I'm coding userspace, and I'm doing something wrong, I like the kernel to let me know, instead of trying to fix things for me. Also, epoll can easily be fixed (add a param to
ep_reinject_items()to re-inject items in case of error/EFAULT) to leave events in the ready-list and let the EFAULT emerge.
Please dont slow the hot path for a basically "User Error". It's already tested in the transfert function, with two conditional branches for each transfered event.
Quoted 7 lines
Anyone else has opinions about this?
PS: Next time it'd be great if you Cc: me when posting epoll patches, so
you avoid Andrew the job of doing it.
Yes, but this particular patch was a followup on own kevent Andrew patch.
I have a bunch of patches for epoll I will send to you :)
Eric
Re: [take23 0/5] kevent: Generic event handling mechanism.
Quoted 4 lines · Eric Dumazet
Quoted 1 line · Davide Libenzi
Lost forever means? If there are more processes watching some fd (external events), they all get their own copy of the events in their own private epoll fd. It's not that we "steal" things out of the kernel, is not a 1:1 producer/consumer thing (one producer, 1 queue). It's one producer, broadcast to all listeners (consumers) thing. The only case where it'd matter is in the case of multiple threads sharing the same epoll fd.
In my particular epoll application, the producer is tcp stack, and I have one consumer. If an network event is lost in the EFAULT handling, its lost forever. In any case, my application do provide a correct user area, so this problem is only theorical.
I realize I was not explicit, and dit not answer your question (Lost forever means ?)
if (epi->revents) {
if (__put_user(epi->revents,
&events[eventcnt].events) ||
__put_user(epi->event.data,
&events[eventcnt].data))
return -EFAULT;Quoted 2 lines
Quoted 2 lines
if (epi->event.events & EPOLLONESHOT) epi->event.events &= EP_PRIVATE_BITS;
eventcnt++;
}If one EPOLLONESHOT event is correctly copied to user space, its status is updated.
If other ready events in the same epoll_wait() call cannot be transferred because of an EFAULT (we reach the real end of user provided area), this EPOLLONESHOT event is lost forever, because it wont be requeued in ready list.
Eric
Re: [take23 0/5] kevent: Generic event handling mechanism.
Quoted 26 lines · Eric Dumazet
Quoted 11 lines
Quoted 6 lines
Lost forever means? If there are more processes watching some fd (external events), they all get their own copy of the events in their own private epoll fd. It's not that we "steal" things out of the kernel, is not a 1:1 producer/consumer thing (one producer, 1 queue). It's one producer, broadcast to all listeners (consumers) thing. The only case where it'd matter is in the case of multiple threads sharing the same epoll fd.
In my particular epoll application, the producer is tcp stack, and I have one consumer. If an network event is lost in the EFAULT handling, its lost forever. In any case, my application do provide a correct user area, so this problem is only theorical.
I realize I was not explicit, and dit not answer your question (Lost forever means ?)
if (epi->revents) { if (__put_user(epi->revents, &events[eventcnt].events) || __put_user(epi->event.data, &events[eventcnt].data)) return -EFAULT;Quoted 2 lines
Quoted 2 lines
if (epi->event.events & EPOLLONESHOT) epi->event.events &= EP_PRIVATE_BITS;eventcnt++; }
Quoted 6 lines
If one EPOLLONESHOT event is correctly copied to user space, its status is updated.
If other ready events in the same
epoll_wait()call cannot be transferred because of an EFAULT (we reach the real end of user provided area), this EPOLLONESHOT event is lost forever, because it wont be requeued in ready list.
Your application is feeding crap to the kernel, because of programming bugs. If that happens, I want an EFAULT and not a partially filled buffer. And which buffer then? This could have been scribbled in userspace memory (the pointer), and the try of the kernel to mask out bugs might create even more subtle problems. Such bug will *never* show up in the up in case the wrong buffer is partially valid (first part, that is the *only* case where your fix would make a difference compared to the status quo), since in case of no ready events we'll never hit it, and in case of some events we'll always return few of them and never EFAULT. No, the more I think about it, the more I personally disagree with the change.
Quoted 3 lines
Please dont slow the hot path for a basically "User Error". It's already tested in the transfert function, with two conditional branches for each transfered event.
Ohh, if you think you can measure them from userspace, those can be turned in 'err |= __put_user();' with err tested only out of the loop.
- Davide
cover kevent: Generic event handling mechanism.
In reply to a message not in the archiveGeneric event handling mechanism.
Kevent is a generic subsytem which allows to handle event notifications. It supports both level and edge triggered events. It is similar to poll/epoll in some cases, but it is more scalable, it is faster and allows to work with essentially eny kind of events.
Events are provided into kernel through control syscall and can be read back through ring buffer or using usual syscalls.
Kevent update (i.e. readiness switching) happens directly from internals of the appropriate state machine of the underlying subsytem (like network, filesystem, timer or any other).
Homepage:
http://tservice.net.ru/~s0mbre/old/?section=projects&item=kevent
Documentation page:
http://linux-net.osdl.org/index.php/Kevent
Consider for inclusion.
Changes from 'take23' patchset:
* kevent PIPE notifications
* KEVENT_REQ_LAST_CHECK flag, which allows to perform last check in dequeuing time
* fixed poll/select notifications (were broken due to tree manipulations)
* made Documentation/kevent.txt look nice in 80-col terminal
* fix for copy_to_user() failure report for the first kevent (Andrew Morton)
* minor fucntion renames
Here is pipe result with kevent_pipe kernel kevent part with 2000 pipes (Eric Dumazet's application):
epoll (edge-triggered): 248408 events/sec kevent (edge-triggered): 269282 events/sec Busy reading loop: 269519 events/sec
Changes from 'take22' patchset:
* new ring buffer implementation in process' memory
* wakeup-one-thread flag
* edge-triggered behaviour
With this release additional independent benchmark shows kevent speed compared to epoll:
Eric Dumazet created special benchmark which creates set of AF_INET sockets and two threads start to simultaneously read and write data from/into them.
Here is results:
epoll (no EPOLLET): 57428 events/sec
kevent (no ET): 59794 events/sec
epoll (with EPOLLET): 71000 events/sec
kevent (with ET): 78265 events/sec
Maximum (busy loop reading events): 88482 events/sec
Changes from 'take21' patchset:
* minor cleanups (different return values, removed unneded variables, whitespaces and so on)
* fixed bug in kevent removal in case when kevent being removed
is the same as overflow_kevent (spotted by Eric Dumazet)
Changes from 'take20' patchset:
* new ring buffer implementation
* removed artificial limit on possible number of kevents
With this release and fixed userspace web server it was possible to achive 3960+ req/s with client connection rate of 4000 con/s over 100 Mbit lan, data IO over network was about 10582.7 KB/s, which is too close to wire speed if we get into account headers and the like.
Changes from 'take19' patchset:
* use __init instead of __devinit
* removed 'default N' from config for user statistic
* removed kevent_user_fini() since kevent can not be unloaded
* use KERN_INFO for statistic output
Changes from 'take18' patchset:
* use __init instead of __devinit
* removed 'default N' from config for user statistic
* removed kevent_user_fini() since kevent can not be unloaded
* use KERN_INFO for statistic output
Changes from 'take17' patchset:
* Use RB tree instead of hash table.
At least for a web sever, frequency of addition/deletion of new kevent is comparable with number of search access, i.e. most of the time events are added, accesed only couple of times and then removed, so it justifies RB tree usage over AVL tree, since the latter does have much slower deletion time (max O(log(N)) compared to 3 ops),
although faster search time (1.44*O(log(N)) vs. 2*O(log(N))).
So for kevents I use RB tree for now and later, when my AVL tree implementation is ready, it will be possible to compare them.
* Changed readiness check for socket notifications.
With both above changes it is possible to achieve more than 3380 req/second compared to 2200, sometimes 2500 req/second for epoll() for trivial web-server and httperf client on the same hardware.
It is possible that above kevent limit is due to maximum allowed kevents in a time limit, which is 4096 events.
Changes from 'take16' patchset:
* misc cleanups (__read_mostly, const ...)
* created special macro which is used for mmap size (number of pages) calculation
* export kevent_socket_notify(), since it is used in network protocols which can be built as modules (IPv6 for example)
Changes from 'take15' patchset:
* converted kevent_timer to high-resolution timers, this forces timer API update at
http://linux-net.osdl.org/index.php/Kevent
* use struct ukevent* instead of void * in syscalls (documentation has been updated)
* added warning in kevent_add_ukevent() if ring has broken index (for testing)
Changes from 'take14' patchset:
* added kevent_wait()
This syscall waits until either timeout expires or at least one event becomes ready. It also commits that @num events from @start are processed by userspace and thus can be be removed or rearmed (depending on it's flags). It can be used for commit events read by userspace through mmap interface. Example userspace code (evtest.c) can be found on project's homepage.
* added socket notifications (send/recv/accept)
Changes from 'take13' patchset:
* do not get lock aroung user data check in __kevent_search()
* fail early if there were no registered callbacks for given type of kevent
* trailing whitespace cleanup
Changes from 'take12' patchset:
* remove non-chardev interface for initialization
* use pointer to kevent_mring instead of unsigned longs
* use aligned 64bit type in raw user data (can be used by high-res timer if needed)
* simplified enqueue/dequeue callbacks and kevent initialization
* use nanoseconds for timeout
* put number of milliseconds into timer's return data
* move some definitions into user-visible header
* removed filenames from comments
Changes from 'take11' patchset:
* include missing headers into patchset
* some trivial code cleanups (use goto instead of if/else games and so on)
* some whitespace cleanups
* check for ready_callback() callback before main loop which should save us some ticks
Changes from 'take10' patchset:
* removed non-existent prototypes
* added helper function for kevent_registered_callbacks
* fixed 80 lines comments issues
* added shared between userspace and kernelspace header instead of embedd them in one
* core restructuring to remove forward declarations
* s o m e w h i t e s p a c e c o d y n g s t y l e c l e a n u p
* use vm_insert_page() instead of remap_pfn_range()
Changes from 'take9' patchset:
* fixed ->nopage method
Changes from 'take8' patchset:
* fixed mmap release bug
* use module_init() instead of late_initcall()
* use better structures for timer notifications
Changes from 'take7' patchset:
* new mmap interface (not tested, waiting for other changes to be acked)
- use nopage() method to dynamically substitue pages
- allocate new page for events only when new added kevent requres it
- do not use ugly index dereferencing, use structure instead
- reduced amount of data in the ring (id and flags), maximum 12 pages on x86 per kevent fd
Changes from 'take6' patchset:
* a lot of comments!
* do not use list poisoning for detection of the fact, that entry is in the list
* return number of ready kevents even if copy*user() fails
* strict check for number of kevents in syscall
* use ARRAY_SIZE for array size calculation
* changed superblock magic number
* use SLAB_PANIC instead of direct panic() call
* changed -E* return values
* a lot of small cleanups and indent fixes
Changes from 'take5' patchset:
* removed compilation warnings about unused wariables when lockdep is not turned on
* do not use internal socket structures, use appropriate (exported) wrappers instead
* removed default 1 second timeout
* removed AIO stuff from patchset
Changes from 'take4' patchset:
* use miscdevice instead of chardevice
* comments fixes
Changes from 'take3' patchset:
* removed serializing mutex from kevent_user_wait()
* moved storage list processing to RCU
* removed lockdep screaming - all storage locks are initialized in the same function, so it was
learned
to differentiate between various cases
* remove kevent from storage if is marked as broken after callback
* fixed a typo in mmaped buffer implementation which would end up in wrong index calcualtion
Changes from 'take2' patchset:
* split kevent_finish_user() to locked and unlocked variants
* do not use KEVENT_STAT ifdefs, use inline functions instead
* use array of callbacks of each type instead of each kevent callback initialization
* changed name of ukevent guarding lock
* use only one kevent lock in kevent_user for all hash buckets instead of per-bucket locks
* do not use kevent_user_ctl structure instead provide needed arguments as syscall parameters
* various indent cleanups
* added optimisation, which is aimed to help when a lot of kevents are being copied from
userspace
* mapped buffer (initial) implementation (no userspace yet)
Changes from 'take1' patchset:
- rebased against 2.6.18-git tree
- removed ioctl controlling
- added new syscall kevent_get_events(int fd, unsigned int min_nr, unsigned int max_nr, unsigned int timeout, void __user *buf, unsigned flags)
- use old syscall kevent_ctl for creation/removing, modification and initial kevent initialization
- use mutuxes instead of semaphores
- added file descriptor check and return error if provided descriptor does not match kevent file operations
- various indent fixes
- removed aio_sendfile() declarations.
Thank you.
PATCH 1/6 kevent: Description.
Description.
Documentation/kevent.txt new file +186 -0
@@ -0,0 +1,186 @@ +Description. + +int kevent_ctl(int fd, unsigned int cmd, unsigned int num, struct ukevent *arg); + +fd - is the file descriptor referring to the kevent queue to manipulate. +It is created by opening "/dev/kevent" char device, which is created with +dynamic minor number and major number assigned for misc devices. + +cmd - is the requested operation. It can be one of the following: + KEVENT_CTL_ADD - add event notification + KEVENT_CTL_REMOVE - remove event notification + KEVENT_CTL_MODIFY - modify existing notification + +num - number of struct ukevent in the array pointed to by arg +arg - array of struct ukevent + +When called, kevent_ctl will carry out the operation specified in the +cmd parameter. +------------------------------------------------------------------------------- + + int kevent_get_events(int ctl_fd, unsigned int min_nr, unsigned int max_nr, + __u64 timeout, struct ukevent *buf, unsigned flags) + +ctl_fd - file descriptor referring to the kevent queue +min_nr - minimum number of completed events that kevent_get_events will block + waiting for +max_nr - number of struct ukevent in buf +timeout - number of nanoseconds to wait before returning less than min_nr + events. If this is -1, then wait forever. +buf - pointer to an array of struct ukevent. +flags - unused + +kevent_get_events will wait timeout milliseconds for at least min_nr completed +events, copying completed struct ukevents to buf and deleting any +KEVENT_REQ_ONESHOT event requests. In nonblocking mode it returns as many +events as possible, but not more than max_nr. In blocking mode it waits until +timeout or if at least min_nr events are ready. +------------------------------------------------------------------------------- + + int kevent_wait(int ctl_fd, unsigned int num, __u64 timeout) + +ctl_fd - file descriptor referring to the kevent queue +num - number of processed kevents +timeout - this timeout specifies number of nanoseconds to wait until there is + free space in kevent queue + +This syscall waits until either timeout expires or at least one event becomes +ready. It also copies that num events into special ring buffer and requeues +them (or removes depending on flags). +------------------------------------------------------------------------------- + + int kevent_ring_init(int ctl_fd, struct kevent_ring *ring, unsigned int num) + +ctl_fd - file descriptor referring to the kevent queue +num - size of the ring buffer in events + + struct kevent_ring + { + unsigned int ring_kidx; + struct ukevent event[0]; + } + +ring_kidx - is an index in the ring buffer where kernel will put new events + when kevent_wait() or kevent_get_events() is called + +Example userspace code (ring_buffer.c) can be found on project's homepage. + +Each kevent syscall can be so called cancellation point in glibc, i.e. when +thread has been cancelled in kevent syscall, thread can be safely removed +and no events will be lost, since each syscall (kevent_wait() or +kevent_get_events()) will copy event into special ring buffer, accessible +from other threads or even processes (if shared memory is used). + +When kevent is removed (not dequeued when it is ready, but just removed), +even if it was ready, it is not copied into ring buffer, since if it is +removed, no one cares about it (otherwise user would wait until it becomes +ready and got it through usual way using kevent_get_events() or kevent_wait()) +and thus no need to copy it to the ring buffer. + +It is possible with userspace ring buffer, that events in the ring buffer +can be replaced without knowledge for the thread currently reading them +(when other thread calls kevent_get_events() or kevent_wait()), so appropriate +locking between threads or processes, which can simultaneously access the same +ring buffer, is required. +------------------------------------------------------------------------------- + +The bulk of the interface is entirely done through the ukevent struct. +It is used to add event requests, modify existing event requests, +specify which event requests to remove, and return completed events. + +struct ukevent contains the following members: + +struct kevent_id id + Id of this request, e.g. socket number, file descriptor and so on +__u32 type + Event type, e.g. KEVENT_SOCK, KEVENT_INODE, KEVENT_TIMER and so on +__u32 event + Event itself, e.g. SOCK_ACCEPT, INODE_CREATED, TIMER_FIRED +__u32 req_flags + Per-event request flags, + + KEVENT_REQ_ONESHOT + event will be removed when it is ready + + KEVENT_REQ_WAKEUP_ONE + When several threads wait on the same kevent queue and requested the + same event, for example 'wake me up when new client has connected, + so I could call accept()', then all threads will be awakened when new + client has connected, but only one of them can process the data. This + problem is known as thundering nerd problem. Events which have this + flag set will not be marked as ready (and appropriate threads will + not be awakened) if at least one event has been already marked. + + KEVENT_REQ_ET + Edge Triggered behaviour. It is an optimisation which allows to move + ready and dequeued (i.e. copied to userspace) event to move into set + of interest for given storage (socket, inode and so on) again. It is + very usefull for cases when the same event should be used many times + (like reading from pipe). It is similar to epoll()'s EPOLLET flag. + + KEVENT_REQ_LAST_CHECK + if set allows to perform the last check on kevent (call appropriate + callback) when kevent is marked as ready and has been removed from + ready queue. If it will be confirmed that kevent is ready + (k->callbacks.callback(k) returns true) then kevent will be copied + to userspace, otherwise it will be requeued back to storage. + Second (checking) call is performed with this bit cleared, so callback + can detect when it was called from kevent_storage_ready() - bit is set, + or kevent_dequeue_ready() - bit is cleared. If kevent will be requeued, + bit will be set again. + +__u32 ret_flags + Per-event return flags + + KEVENT_RET_BROKEN + Kevent is broken + + KEVENT_RET_DONE + Kevent processing was finished successfully + + KEVENT_RET_COPY_FAILED + Kevent was not copied into ring buffer due to some error conditions. + +__u32 ret_data + Event return data. Event originator fills it with anything it likes + (for example timer notifications put number of milliseconds when timer + has fired +union { __u32 user[2]; void *ptr; } + User's data. It is not used, just copied to/from user. The whole structure + is aligned to 8 bytes already, so the last union is aligned properly. + +------------------------------------------------------------------------------- + +Usage + +For KEVENT_CTL_ADD, all fields relevant to the event type must be filled +(id, type, possibly event, req_flags). +After kevent_ctl(..., KEVENT_CTL_ADD, ...) returns each struct's ret_flags +should be checked to see if the event is already broken or done. + +For KEVENT_CTL_MODIFY, the id, req_flags, and user and event fields must be +set and an existing kevent request must have matching id and user fields. If +match is found, req_flags and event are replaced with the newly supplied +values and requeueing is started, so modified kevent can be checked and +probably marked as ready immediately. If a match can't be found, the +passed in ukevent's ret_flags has KEVENT_RET_BROKEN set. KEVENT_RET_DONE is +always set. + +For KEVENT_CTL_REMOVE, the id and user fields must be set and an existing +kevent request must have matching id and user fields. If a match is found, +the kevent request is removed. If a match can't be found, the passed in +ukevent's ret_flags has KEVENT_RET_BROKEN set. KEVENT_RET_DONE is always set. + +For kevent_get_events, the entire structure is returned. + +------------------------------------------------------------------------------- + +Usage cases + +kevent_timer +struct ukevent should contain following fields: + type - KEVENT_TIMER + event - KEVENT_TIMER_FIRED + req_flags - KEVENT_REQ_ONESHOT if you want to fire that timer only once + id.raw[0] - number of seconds after commit when this timer shout expire + id.raw[0] - additional to number of seconds number of nanoseconds
PATCH 2/6 kevent: Core files.
Core files.
This patch includes core kevent files:
* userspace controlling
* kernelspace interfaces
* initialization
* notification state machines
Some bits of documentation can be found on project's homepage (and links from there):
http://tservice.net.ru/~s0mbre/old/?section=projects&item=kevent
arch/i386/kernel/syscall_table.S +4 -0
@@ -318,3 +318,7 @@ ENTRY(sys_call_table) .long sys_vmsplice .long sys_move_pages .long sys_getcpu + .long sys_kevent_get_events + .long sys_kevent_ctl /* 320 */ + .long sys_kevent_wait + .long sys_kevent_ring_init
arch/x86_64/ia32/ia32entry.S +5 -1
@@ -714,8 +714,12 @@ #endif .quad compat_sys_get_robust_list .quad sys_splice .quad sys_sync_file_range - .quad sys_tee + .quad sys_tee /* 315 */ .quad compat_sys_vmsplice .quad compat_sys_move_pages .quad sys_getcpu + .quad sys_kevent_get_events + .quad sys_kevent_ctl /* 320 */ + .quad sys_kevent_wait + .quad sys_kevent_ring_init ia32_syscall_end:
include/asm-i386/unistd.h +5 -1
@@ -324,10 +324,14 @@ #define __NR_tee 315#define __NR_vmsplice 316#define __NR_move_pages 317#define __NR_getcpu 318 +#define __NR_kevent_get_events 319 +#define __NR_kevent_ctl 320 +#define __NR_kevent_wait 321 +#define __NR_kevent_ring_init 322#ifdef __KERNEL__ -#define NR_syscalls 319 +#define NR_syscalls 323#include <linux/err.h> /*
include/asm-x86_64/unistd.h +9 -1
@@ -619,10 +619,18 @@ #define __NR_vmsplice 278 __SYSCALL(__NR_vmsplice, sys_vmsplice)#define __NR_move_pages 279 __SYSCALL(__NR_move_pages, sys_move_pages) +#define __NR_kevent_get_events 280 +__SYSCALL(__NR_kevent_get_events, sys_kevent_get_events) +#define __NR_kevent_ctl 281 +__SYSCALL(__NR_kevent_ctl, sys_kevent_ctl) +#define __NR_kevent_wait 282 +__SYSCALL(__NR_kevent_wait, sys_kevent_wait) +#define __NR_kevent_ring_init 283 +__SYSCALL(__NR_kevent_ring_init, sys_kevent_ring_init)#ifdef __KERNEL__ -#define __NR_syscall_max __NR_move_pages +#define __NR_syscall_max __NR_kevent_ring_init#include <linux/err.h> #ifndef __NO_STUBS
include/linux/kevent.h new file +223 -0
223 lines not on this page · raw message
include/linux/kevent_storage.h new file +11 -0
@@ -0,0 +1,11 @@ +#ifndef __KEVENT_STORAGE_H +#define __KEVENT_STORAGE_H + +struct kevent_storage +{ +void *origin;/* Originator's pointer, e.g. struct sock or struct file. Can be NULL. */ +struct list_head list;/* List of queued kevents. */ +spinlock_t lock;/* Protects users queue. */ +}; + +#endif /* __KEVENT_STORAGE_H */
include/linux/syscalls.h +7 -0
@@ -54,6 +54,8 @@ struct compat_stat;struct compat_timeval ;struct robust_list_head ;struct getcpu_cache ; +struct ukevent ; +struct kevent_ring ;#include <linux/types.h> #include <linux/aio_abi.h>
@@ -599,4 +601,9 @@ asmlinkage long sys_set_robust_list(stru size_t len);asmlinkage long sys_getcpu (unsigned __user *cpu,unsigned __user *node,struct getcpu_cache __user *cache); +asmlinkage long sys_kevent_get_events (int ctl_fd,unsigned int min,unsigned int max, +__u64 timeout,struct ukevent __user *buf,unsigned flags); +asmlinkage long sys_kevent_ctl (int ctl_fd,unsigned int cmd,unsigned int num,struct ukevent __user *buf); +asmlinkage long sys_kevent_wait (int ctl_fd,unsigned int num,__u64 timeout); +asmlinkage long sys_kevent_ring_init (int ctl_fd,struct kevent_ring __user *ring,unsigned int num);#endif
include/linux/ukevent.h new file +165 -0
165 lines not on this page · raw message
init/Kconfig +2 -0
@@ -201,6 +201,8 @@ config AUDITSYSCALL such as SELinux. To use audit's filesystem watch feature, please ensure that INOTIFY is configured. +source "kernel/kevent/Kconfig" + config IKCONFIG bool "Kernel .config support" ---help---
kernel/Makefile +1 -0
@@ -47,6 +47,7 @@ obj-$(CONFIG_DETECT_SOFTLOCKUP) += softl obj-$(CONFIG_GENERIC_HARDIRQS) += irq/ obj-$(CONFIG_SECCOMP) += seccomp.o obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o +obj-$(CONFIG_KEVENT) += kevent/ obj-$(CONFIG_RELAY) += relay.o obj-$(CONFIG_TASK_DELAY_ACCT) += delayacct.o obj-$(CONFIG_TASKSTATS) += taskstats.o
kernel/kevent/Kconfig new file +45 -0
45 lines not on this page · raw message
kernel/kevent/Makefile new file +5 -0
@@ -0,0 +1,5 @@ +obj-y := kevent.o kevent_user.o +obj-$(CONFIG_KEVENT_TIMER) += kevent_timer.o +obj-$(CONFIG_KEVENT_POLL) += kevent_poll.o +obj-$(CONFIG_KEVENT_SOCKET) += kevent_socket.o +obj-$(CONFIG_KEVENT_PIPE) += kevent_pipe.o
kernel/kevent/kevent.c new file +232 -0
232 lines not on this page · raw message
kernel/kevent/kevent_user.c new file +936 -0
936 lines not on this page · raw message
kernel/sys_ni.c +5 -0
@@ -122,6 +122,11 @@ cond_syscall(ppc_rtas);cond_syscall (sys_spu_run);cond_syscall (sys_spu_create); +cond_syscall (sys_kevent_get_events); +cond_syscall (sys_kevent_wait); +cond_syscall (sys_kevent_ctl); +cond_syscall (sys_kevent_ring_init); +/* mmu depending weak syscall entries */ cond_syscall (sys_mprotect);cond_syscall (sys_msync);
PATCH 3/6 kevent: poll/select() notifications.
poll/select() notifications.
This patch includes generic poll/select notifications. kevent_poll works simialr to epoll and has the same issues (callback is invoked not from internal state machine of the caller, but through process awake, a lot of allocations and so on).
fs/file_table.c +3 -0
@@ -20,6 +20,7 @@ #include <linux/capability.h>#include <linux/cdev.h> #include <linux/fsnotify.h> #include <linux/sysctl.h> +#include <linux/kevent.h> #include <linux/percpu_counter.h> #include <asm/atomic.h>
@@ -119,6 +120,7 @@ struct file *get_empty_filp(void) f->f_uid = tsk->fsuid; f->f_gid = tsk->fsgid;eventpoll_init_file (f); +kevent_init_file (f);/* f->f_version: 0 */ return f;
@@ -164,6 +166,7 @@ void fastcall __fput(struct file *file)* in the file cleanup chain. */ eventpoll_release (file); +kevent_cleanup_file (file);locks_remove_flock (file);if (file->f_op && file->f_op->release)
fs/inode.c +7 -0
@@ -21,6 +21,7 @@ #include <linux/pagemap.h>#include <linux/cdev.h> #include <linux/bootmem.h> #include <linux/inotify.h> +#include <linux/kevent.h> #include <linux/mount.h> /*
@@ -164,12 +165,18 @@ #endif } inode->i_private =0 ; inode->i_mapping = mapping; +#if definedCONFIG_KEVENT_SOCKET || definedCONFIG_KEVENT_PIPE +kevent_storage_init (inode, &inode->st); +#endif }return inode; }void destroy_inode (struct inode *inode) { +#if definedCONFIG_KEVENT_SOCKET +kevent_storage_fini (&inode->st); +#endif BUG_ON (inode_has_buffers (inode));security_inode_free (inode);if (inode->i_sb->s_op->destroy_inode)
include/linux/fs.h +8 -0
@@ -276,6 +276,7 @@ #include <linux/prio_tree.h>#include <linux/init.h> #include <linux/sched.h> #include <linux/mutex.h> +#include <linux/kevent_storage.h> #include <asm/atomic.h> #include <asm/semaphore.h>
@@ -586,6 +587,10 @@ #ifdef CONFIG_INOTIFYstruct mutex inotify_mutex;/* protects the watches list */ #endif +#ifdef CONFIG_KEVENT_SOCKET +struct kevent_storage st; +#endif +unsigned long i_state;unsigned long dirtied_when;/* jiffies of first dirtying */
@@ -739,6 +744,9 @@ #ifdef CONFIG_EPOLLstruct list_head f_ep_links;spinlock_t f_ep_lock;#endif /* #ifdef CONFIG_EPOLL */ +#ifdef CONFIG_KEVENT_POLL +struct kevent_storage st; +#endif struct address_space *f_mapping; };extern spinlock_t files_lock;
kernel/kevent/kevent_poll.c new file +228 -0
228 lines not on this page · raw message
PATCH 4/6 kevent: Socket notifications.
Socket notifications.
This patch includes socket send/recv/accept notifications. Using trivial web server based on kevent and this features instead of epoll it's performance increased more than noticebly. More details about various benchmarks and server itself (evserver_kevent.c) can be found on project's homepage.
fs/inode.c +7 -0
@@ -21,6 +21,7 @@ #include <linux/pagemap.h>#include <linux/cdev.h> #include <linux/bootmem.h> #include <linux/inotify.h> +#include <linux/kevent.h> #include <linux/mount.h> /*
@@ -164,12 +165,18 @@ #endif } inode->i_private =0 ; inode->i_mapping = mapping; +#if definedCONFIG_KEVENT_SOCKET || definedCONFIG_KEVENT_PIPE +kevent_storage_init (inode, &inode->st); +#endif }return inode; }void destroy_inode (struct inode *inode) { +#if definedCONFIG_KEVENT_SOCKET +kevent_storage_fini (&inode->st); +#endif BUG_ON (inode_has_buffers (inode));security_inode_free (inode);if (inode->i_sb->s_op->destroy_inode)
include/net/sock.h +17 -15
@@ -48,6 +48,7 @@ #include <linux/lockdep.h>#include <linux/netdevice.h> #include <linux/skbuff.h> /* struct sk_buff */ #include <linux/security.h> +#include <linux/kevent.h> #include <linux/filter.h>
@@ -450,6 +451,21 @@ static inline int sk_stream_memory_free(extern void sk_stream_rfree (struct sk_buff *skb); +struct socket_alloc { +struct socket socket; +struct inode vfs_inode; +}; + +static inline struct socket *SOCKET_I (struct inode *inode) +{ + return &container_of(inode, struct socket_alloc, vfs_inode)->socket; +} + +static inline struct inode *SOCK_INODE (struct socket *socket) +{ + return &container_of(socket, struct socket_alloc, socket)->vfs_inode; +} +static inline void sk_stream_set_owner_r (struct sk_buff *skb,struct sock *sk) { skb->sk = sk;
@@ -477,6 +493,7 @@ static inline void sk_add_backlog(struct sk->sk_backlog.tail = skb; } skb->next =NULL ; +kevent_socket_notify (sk,KEVENT_SOCKET_RECV ); }#define sk_wait_event (__sk, __timeo, __condition) \
@@ -679,21 +696,6 @@ static inline struct kiocb *siocb_to_kioreturn si->kiocb; } -struct socket_alloc { -struct socket socket; -struct inode vfs_inode; -}; - -static inline struct socket *SOCKET_I (struct inode *inode) -{ - return &container_of(inode, struct socket_alloc, vfs_inode)->socket; -} - -static inline struct inode *SOCK_INODE (struct socket *socket) -{ - return &container_of(socket, struct socket_alloc, socket)->vfs_inode; -} -extern void __sk_stream_mem_reclaim (struct sock *sk);extern int sk_stream_mem_schedule (struct sock *sk,int size,int kind);
include/net/tcp.h +1 -0
@@ -857,6 +857,7 @@ static inline int tcp_prequeue(struct so tp->ucopy.memory =0 ; }else if (skb_queue_len (&tp->ucopy.prequeue) ==1 ) {wake_up_interruptible (sk->sk_sleep); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND );if (!inet_csk_ack_scheduled (sk)) inet_csk_reset_xmit_timer(sk, ICSK_TIME_DACK, (3 * TCP_RTO_MIN) / 4,
kernel/kevent/kevent_socket.c new file +135 -0
135 lines not on this page · raw message
net/core/sock.c +9 -1
@@ -1402,6 +1402,7 @@ static void sock_def_wakeup(struct sockif (sk->sk_sleep &&waitqueue_active (sk->sk_sleep))wake_up_interruptible_all (sk->sk_sleep);read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_error_report (struct sock *sk)
@@ -1411,6 +1412,7 @@ static void sock_def_error_report(structwake_up_interruptible (sk->sk_sleep);sk_wake_async (sk,0 ,POLL_ERR );read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_readable (struct sock *sk,int len)
@@ -1420,6 +1422,7 @@ static void sock_def_readable(struct socwake_up_interruptible (sk->sk_sleep);sk_wake_async (sk,1 ,POLL_IN );read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); }static void sock_def_write_space (struct sock *sk)
@@ -1439,6 +1442,7 @@ static void sock_def_write_space(struct }read_unlock (&sk->sk_callback_lock); +kevent_socket_notify (sk,KEVENT_SOCKET_SEND |KEVENT_SOCKET_RECV ); }static void sock_def_destruct (struct sock *sk)
@@ -1489,6 +1493,8 @@ #endif sk->sk_state =TCP_CLOSE ; sk->sk_socket = sock; +kevent_sk_reinit (sk); +sock_set_flag (sk,SOCK_ZAPPED );if (sock)
@@ -1555,8 +1561,10 @@ void fastcall release_sock(struct sock *if (sk->sk_backlog.tail)__release_sock (sk); sk->sk_lock.owner =NULL ; -if (waitqueue_active (&sk->sk_lock.wq)) +if (waitqueue_active (&sk->sk_lock.wq)) {wake_up (&sk->sk_lock.wq); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV |KEVENT_SOCKET_SEND ); + }spin_unlock_bh (&sk->sk_lock.slock); }EXPORT_SYMBOL (release_sock);
net/core/stream.c +1 -0
@@ -36,6 +36,7 @@ void sk_stream_write_space(struct sock *wake_up_interruptible (sk->sk_sleep);if (sock->fasync_list && !(sk->sk_shutdown &SEND_SHUTDOWN ))sock_wake_async (sock,2 ,POLL_OUT ); +kevent_socket_notify (sk,KEVENT_SOCKET_SEND |KEVENT_SOCKET_RECV ); } }
net/ipv4/tcp_input.c +1 -0
@@ -3119,6 +3119,7 @@ static void tcp_ofo_queue(struct sock *s__skb_unlink (skb, &tp->out_of_order_queue);__skb_queue_tail (&sk->sk_receive_queue, skb); +kevent_socket_notify (sk,KEVENT_SOCKET_RECV ); tp->rcv_nxt =TCP_SKB_CB (skb)->end_seq;if (skb->h.th->fin)tcp_fin (skb, sk, skb->h.th);
net/ipv4/tcp_ipv4.c +2 -0
@@ -61,6 +61,7 @@ #include <linux/cache.h>#include <linux/jhash.h> #include <linux/init.h> #include <linux/times.h> +#include <linux/kevent.h> #include <net/icmp.h> #include <net/inet_hashtables.h>
@@ -870,6 +871,7 @@ #endifreqsk_free (req); }else {inet_csk_reqsk_queue_hash_add (sk, req,TCP_TIMEOUT_INIT ); +kevent_socket_notify (sk,KEVENT_SOCKET_ACCEPT ); }return 0 ;
net/socket.c +3 -0
@@ -85,6 +85,7 @@ #include <linux/compat.h>#include <linux/kmod.h> #include <linux/audit.h> #include <linux/wireless.h> +#include <linux/kevent.h> #include <asm/uaccess.h> #include <asm/unistd.h>
@@ -490,6 +491,8 @@ static struct socket *sock_alloc(void) inode->i_uid = current->fsuid; inode->i_gid = current->fsgid; +kevent_socket_reinit (sock); +get_cpu_var (sockets_in_use)++;put_cpu_var (sockets_in_use);return sock;
PATCH 5/6 kevent: Timer notifications.
Timer notifications.
Timer notifications can be used for fine grained per-process time management, since interval timers are very inconvenient to use, and they are limited.
This subsystem uses high-resolution timers.
id.raw[0] is used as number of seconds
id.raw[1] is used as number of nanoseconds
kernel/kevent/kevent_timer.c new file +112 -0
@@ -0,0 +1,112 @@ +/* +* 2006 Copyright (c) Evgeniy Polyakov <[email protected]> +* All rights reserved. +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +*/ + +#include <linux/kernel.h> +#include <linux/types.h> +#include <linux/list.h> +#include <linux/slab.h> +#include <linux/spinlock.h> +#include <linux/hrtimer.h> +#include <linux/jiffies.h> +#include <linux/kevent.h> + +struct kevent_timer +{ +struct hrtimer ktimer; +struct kevent_storage ktimer_storage; +struct kevent *ktimer_event; +}; + +static int kevent_timer_func (struct hrtimer *timer) +{ + struct kevent_timer *t = container_of(timer, struct kevent_timer, ktimer); +struct kevent *k = t->ktimer_event; + +kevent_storage_ready (&t->ktimer_storage,NULL ,KEVENT_MASK_ALL ); +hrtimer_forward (timer, timer->base->softirq_time, +ktime_set (k->event.id.raw[0 ], k->event.id.raw[1 ])); +return HRTIMER_RESTART ; +} + +static struct lock_class_key kevent_timer_key; + +static int kevent_timer_enqueue (struct kevent *k) +{ +int err; +struct kevent_timer *t; + + t =kmalloc (sizeof (struct kevent_timer ),GFP_KERNEL ); +if (!t) +return -ENOMEM ; + +hrtimer_init (&t->ktimer,CLOCK_MONOTONIC ,HRTIMER_REL ); + t->ktimer.expires =ktime_set (k->event.id.raw[0 ], k->event.id.raw[1 ]); + t->ktimer.function = kevent_timer_func; + t->ktimer_event = k; + + err =kevent_storage_init (&t->ktimer, &t->ktimer_storage); +if (err) + goto err_out_free; +lockdep_set_class (&t->ktimer_storage.lock, &kevent_timer_key); + + err =kevent_storage_enqueue (&t->ktimer_storage, k); +if (err) + goto err_out_st_fini; + +hrtimer_start (&t->ktimer, t->ktimer.expires,HRTIMER_REL ); + +return 0 ; + +err_out_st_fini: +kevent_storage_fini (&t->ktimer_storage); +err_out_free: +kfree (t); + +return err; +} + +static int kevent_timer_dequeue (struct kevent *k) +{ +struct kevent_storage *st = k->st; + struct kevent_timer *t = container_of(st, struct kevent_timer, ktimer_storage); + +hrtimer_cancel (&t->ktimer); +kevent_storage_dequeue (st, k); +kfree (t); + +return 0 ; +} + +static int kevent_timer_callback (struct kevent *k) +{ + k->event.ret_data[0 ] =jiffies_to_msecs (jiffies); +return 1 ; +} + +static int __init kevent_init_timer (void ) +{ +struct kevent_callbacks tc = { + .callback = &kevent_timer_callback, + .enqueue = &kevent_timer_enqueue, + .dequeue = &kevent_timer_dequeue}; + +return kevent_add_callbacks (&tc,KEVENT_TIMER ); +} +module_init (kevent_init_timer); +
PATCH 6/6 kevent: Pipe notifications.
Pipe notifications.
fs/pipe.c +6 -0
@@ -16,6 +16,7 @@ #include <linux/pipe_fs_i.h>#include <linux/uio.h> #include <linux/highmem.h> #include <linux/pagemap.h> +#include <linux/kevent.h> #include <asm/uaccess.h> #include <asm/ioctls.h>
@@ -312,6 +313,7 @@ redo:break ; }if (do_wakeup) { +kevent_pipe_notify (inode,KEVENT_SOCKET_SEND );wake_up_interruptible_sync (&pipe->wait);kill_fasync (&pipe->fasync_writers,SIGIO ,POLL_OUT ); }
@@ -321,6 +323,7 @@ redo:/* Signal writers asynchronously that there is more room. */ if (do_wakeup) { +kevent_pipe_notify (inode,KEVENT_SOCKET_SEND );wake_up_interruptible (&pipe->wait);kill_fasync (&pipe->fasync_writers,SIGIO ,POLL_OUT ); }
@@ -490,6 +493,7 @@ redo2:break ; }if (do_wakeup) { +kevent_pipe_notify (inode,KEVENT_SOCKET_RECV );wake_up_interruptible_sync (&pipe->wait);kill_fasync (&pipe->fasync_readers,SIGIO ,POLL_IN ); do_wakeup =0 ;
@@ -501,6 +505,7 @@ redo2: out:mutex_unlock (&inode->i_mutex);if (do_wakeup) { +kevent_pipe_notify (inode,KEVENT_SOCKET_RECV );wake_up_interruptible (&pipe->wait);kill_fasync (&pipe->fasync_readers,SIGIO ,POLL_IN ); }
@@ -605,6 +610,7 @@ pipe_release(struct inode *inode, int defree_pipe_info (inode); }else {wake_up_interruptible (&pipe->wait); +kevent_pipe_notify (inode,KEVENT_SOCKET_SEND |KEVENT_SOCKET_RECV );kill_fasync (&pipe->fasync_readers,SIGIO ,POLL_IN );kill_fasync (&pipe->fasync_writers,SIGIO ,POLL_OUT ); }
kernel/kevent/kevent_pipe.c new file +112 -0
112 lines not on this page · raw message
Re: [take24 3/6] kevent: poll/select() notifications.
In reply to Evgeniy PolyakovQuoted 50 lines · Evgeniy Polyakov
poll/select() notifications.
This patch includes generic poll/select notifications.
kevent_pollworks simialr to epoll and has the same issues (callback is invoked not from internal state machine of the caller, but through process awake, a lot of allocations and so on).Signed-off-by Evgeniy Polyakov <[email protected]>fs/file_table.c +3
-0@@ -20,6 +20,7 @@ #include <linux/capability.h>#include <linux/cdev.h> #include <linux/fsnotify.h> #include <linux/sysctl.h> +#include <linux/kevent.h> #include <linux/percpu_counter.h> #include <asm/atomic.h> @@ -119,6 +120,7 @@ struct file *get_empty_filp(void) f->f_uid = tsk->fsuid; f->f_gid = tsk->fsgid;eventpoll_init_file (f); +kevent_init_file (f);/* f->f_version: 0 */ return f;@@ -164,6 +166,7 @@ void fastcall __fput(struct file *file)* in the file cleanup chain. */ eventpoll_release (file); +kevent_cleanup_file (file);locks_remove_flock (file);if (file->f_op && file->f_op->release)fs/inode.c +1
-0@@ -21,6 +21,7 @@ #include <linux/pagemap.h>#include <linux/cdev.h> #include <linux/bootmem.h> #include <linux/inotify.h> +#include <linux/kevent.h> #include <linux/mount.h> /*
fs/inode.c #endif · line 167
Quoted 3 lines
fs/inode.c +0
-0@@ -164,12 +165,18 @@ #endif } inode->i_private =0 ; inode->i_mapping = mapping;
Here you test both KEVENT_SOCKET and KEVENT_PIPE
fs/inode.c #endif · line 176
Quoted 9 lines
fs/inode.c +3
-0+#if definedCONFIG_KEVENT_SOCKET || definedCONFIG_KEVENT_PIPE +kevent_storage_init (inode, &inode->st); +#endif }return inode; }void destroy_inode (struct inode *inode) {
but here you test only KEVENT_SOCKET
Quoted 19 lines
fs/inode.c +3
-0+#if definedCONFIG_KEVENT_SOCKET +kevent_storage_fini (&inode->st); +#endif BUG_ON (inode_has_buffers (inode));security_inode_free (inode);if (inode->i_sb->s_op->destroy_inode)include/linux/fs.h +1
-0@@ -276,6 +276,7 @@ #include <linux/prio_tree.h>#include <linux/init.h> #include <linux/sched.h> #include <linux/mutex.h> +#include <linux/kevent_storage.h> #include <asm/atomic.h> #include <asm/semaphore.h>
include/linux/fs.h #ifdef CONFIG_INOTIFY · line 589
Quoted 3 lines
include/linux/fs.h +0
-0@@ -586,6 +587,10 @@ #ifdef CONFIG_INOTIFYstruct mutex inotify_mutex;/* protects the watches list */ #endif
Here you include a kevent_storage only if KEVENT_SOCKET
Quoted 17 lines
include/linux/fs.h +7
-0+#ifdef CONFIG_KEVENT_SOCKET +struct kevent_storage st; +#endif +unsigned long i_state;unsigned long dirtied_when;/* jiffies of first dirtying */ @@ -739,6 +744,9 @@ #ifdef CONFIG_EPOLLstruct list_head f_ep_links;spinlock_t f_ep_lock;#endif /* #ifdef CONFIG_EPOLL */ +#ifdef CONFIG_KEVENT_POLL +struct kevent_storage st; +#endif struct address_space *f_mapping; };extern spinlock_t files_lock;
Re: [take24 3/6] kevent: poll/select() notifications.
Quoted 9 lines · Eric Dumazet
Here you test both
KEVENT_SOCKETandKEVENT_PIPEQuoted 7 lines
fs/inode.c +3
-0+#if definedCONFIG_KEVENT_SOCKET || definedCONFIG_KEVENT_PIPE +kevent_storage_init (inode, &inode->st); +#endif }return inode; }
Quoted 8 lines
Quoted 2 lines
void
destroy_inode(struct inode *inode)
{but here you test only
KEVENT_SOCKETfs/inode.c #endif · line 179
Quoted 3 lines
fs/inode.c +3
-0+#if definedCONFIG_KEVENT_SOCKET +kevent_storage_fini (&inode->st); +#endif
Indeed, it must be
#if defined CONFIG_KEVENT_SOCKET || defined CONFIG_KEVENT_PIPE
Quoted 22 lines
Quoted 16 lines
fs/inode.c +0
-0BUG_ON (inode_has_buffers (inode));security_inode_free (inode);if (inode->i_sb->s_op->destroy_inode)include/linux/fs.h +1
-0@@ -276,6 +276,7 @@ #include <linux/prio_tree.h>#include <linux/init.h> #include <linux/sched.h> #include <linux/mutex.h> +#include <linux/kevent_storage.h> #include <asm/atomic.h> #include <asm/semaphore.h> include/linux/fs.h #ifdef CONFIG_INOTIFY · line 589
Quoted 3 lines
include/linux/fs.h +0
-0@@ -586,6 +587,10 @@ #ifdef CONFIG_INOTIFYstruct mutex inotify_mutex;/* protects the watches list */ #endif Here you include a
kevent_storageonly ifKEVENT_SOCKET
Quoted 4 lines
include/linux/fs.h #ifdef CONFIG_INOTIFY · line 593
Quoted 4 lines
include/linux/fs.h +4
-0+#ifdef CONFIG_KEVENT_SOCKET +struct kevent_storage st; +#endif +
It must be
#if defined CONFIG_KEVENT_SOCKET || defined CONFIG_KEVENT_PIPE
Signature · 1 line
Evgeniy Polyakov
Re: [take24 3/6] kevent: poll/select() notifications.
In reply to Evgeniy Polyakovkernel/kevent/kevent_poll.c line 183
Quoted 13 lines · Evgeniy Polyakov
kernel/kevent/kevent_poll.c new file +13
-0+static int kevent_poll_callback (struct kevent *k) +{ +if (k->event.req_flags &KEVENT_REQ_LAST_CHECK ) { +return 1 ; + }else { +struct file *file = k->st->origin; +unsigned int revents = file->f_op->poll (file,NULL ); + + k->event.ret_data[0 ] = revents & k->event.event; + +return (revents & k->event.event); + } +}
You need to be careful that file->f_op->poll is not called inside the spin_lock_irqsave/spin_lock_irqrestore pair, since (even this came up during epoll developemtn days) file->f_op->poll might do a simple spin_lock_irq/spin_unlock_irq. This unfortunate constrain forced epoll to have a suboptimal double O(R) loop to handle LT events.
- Davide
Re: [take24 3/6] kevent: poll/select() notifications.
Quoted 13 lines · Davide Libenzi
Quoted 11 lines · Evgeniy Polyakov
kernel/kevent/kevent_poll.c new file +11
-0+static int kevent_poll_callback (struct kevent *k) +{ +if (k->event.req_flags &KEVENT_REQ_LAST_CHECK ) { +return 1 ; + }else { +struct file *file = k->st->origin; +unsigned int revents = file->f_op->poll (file,NULL ); + + k->event.ret_data[0 ] = revents & k->event.event; + +return (revents & k->event.event);
Quoted 8 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ } +}You need to be careful that
file->f_op->pollis not called inside the spin_lock_irqsave/spin_lock_irqrestore pair, since (even this came up during epoll developemtn days)file->f_op->pollmight do a simple spin_lock_irq/spin_unlock_irq. This unfortunate constrain forced epoll to have a suboptimal double O(R) loop to handle LT events.
It is tricky - users call wake_up() from any context, which in turn ends up calling kevent_storage_ready(), which calls kevent_poll_callback() with KEVENT_REQ_LAST_CHECK bit set, which becomes almost empty call in fast path. Since callback returns 1, kevent will be queued into ready queue, which is processed on behalf of syscalls - in that case kevent will check the flag and since KEVENT_REQ_LAST_CHECK is set, will call callback again to check if kevent is correctly marked, but already without that flag (it happens in syscall context, i.e. process context without any locks held), so callback calls ->poll(), which can sleep, but it is safe. If ->poll() returns 'ready' value, kevent is transfers data into userspace, otherwise it is 'requeued' (just removed from ready queue).
Quoted 2 lines
- Davide
Signature · 1 line
Evgeniy Polyakov
Re: [take24 3/6] kevent: poll/select() notifications.
Quoted 23 lines · Evgeniy Polyakov
Quoted 13 lines · Davide Libenzi
Quoted 11 lines · Evgeniy Polyakov
kernel/kevent/kevent_poll.c new file +11
-0+static int kevent_poll_callback (struct kevent *k) +{ +if (k->event.req_flags &KEVENT_REQ_LAST_CHECK ) { +return 1 ; + }else { +struct file *file = k->st->origin; +unsigned int revents = file->f_op->poll (file,NULL ); + + k->event.ret_data[0 ] = revents & k->event.event; + +return (revents & k->event.event);Quoted 8 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ } +}You need to be careful that
file->f_op->pollis not called inside the spin_lock_irqsave/spin_lock_irqrestore pair, since (even this came up during epoll developemtn days)file->f_op->pollmight do a simple spin_lock_irq/spin_unlock_irq. This unfortunate constrain forced epoll to have a suboptimal double O(R) loop to handle LT events.
Quoted 12 lines
It is tricky - users call
wake_up()from any context, which in turn ends up callingkevent_storage_ready(), which callskevent_poll_callback()withKEVENT_REQ_LAST_CHECKbit set, which becomes almost empty call in fast path. Since callback returns 1, kevent will be queued into ready queue, which is processed on behalf of syscalls - in that case kevent will check the flag and sinceKEVENT_REQ_LAST_CHECKis set, will call callback again to check if kevent is correctly marked, but already without that flag (it happens in syscall context, i.e. process context without any locks held), so callback calls ->poll(), which can sleep, but it is safe. If ->poll() returns 'ready' value, kevent is transfers data into userspace, otherwise it is 'requeued' (just removed from ready queue).
Oh, mine was only a general warn. I hadn't looked at the generic code before. But now that I poke on it, I see:
void kevent_requeue(struct kevent *k)
{
unsigned long flags;
spin_lock_irqsave(&k->st->lock, flags);
__kevent_requeue(k, 0);
spin_unlock_irqrestore(&k->st->lock, flags);
}and then:
static int __kevent_requeue(struct kevent *k, u32 event)
{
int ret, rem;
unsigned long flags;
ret = k->callbacks.callback(k);Isn't the k->callbacks.callback() possibly end up calling f_op->poll?
- Davide
Re: [take24 3/6] kevent: poll/select() notifications.
Quoted 52 lines · Davide Libenzi
Quoted 23 lines · Evgeniy Polyakov
Quoted 13 lines · Davide Libenzi
Quoted 11 lines · Evgeniy Polyakov
kernel/kevent/kevent_poll.c new file +11
-0+static int kevent_poll_callback (struct kevent *k) +{ +if (k->event.req_flags &KEVENT_REQ_LAST_CHECK ) { +return 1 ; + }else { +struct file *file = k->st->origin; +unsigned int revents = file->f_op->poll (file,NULL ); + + k->event.ret_data[0 ] = revents & k->event.event; + +return (revents & k->event.event);Quoted 8 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ } +}You need to be careful that
file->f_op->pollis not called inside the spin_lock_irqsave/spin_lock_irqrestore pair, since (even this came up during epoll developemtn days)file->f_op->pollmight do a simple spin_lock_irq/spin_unlock_irq. This unfortunate constrain forced epoll to have a suboptimal double O(R) loop to handle LT events.Quoted 12 lines
It is tricky - users call
wake_up()from any context, which in turn ends up callingkevent_storage_ready(), which callskevent_poll_callback()withKEVENT_REQ_LAST_CHECKbit set, which becomes almost empty call in fast path. Since callback returns 1, kevent will be queued into ready queue, which is processed on behalf of syscalls - in that case kevent will check the flag and sinceKEVENT_REQ_LAST_CHECKis set, will call callback again to check if kevent is correctly marked, but already without that flag (it happens in syscall context, i.e. process context without any locks held), so callback calls ->poll(), which can sleep, but it is safe. If ->poll() returns 'ready' value, kevent is transfers data into userspace, otherwise it is 'requeued' (just removed from ready queue).Oh, mine was only a general warn. I hadn't looked at the generic code before. But now that I poke on it, I see:
void
kevent_requeue(struct kevent *k){ unsigned long flags; spin_lock_irqsave(&k->st->lock, flags); __kevent_requeue(k, 0); spin_unlock_irqrestore(&k->st->lock, flags); }and then:
Quoted 8 lines
static int
__kevent_requeue(struct kevent *k, u32 event){ int ret, rem; unsigned long flags; ret = k->callbacks.callback(k);Isn't the
k->callbacks.callback()possibly end up callingf_op->poll?
Ack, there the check for KEVENT_REQ_LAST_CHECK inside the callback. The problem with f_op->poll was not that it can sleep (not excluded though) but that some f_op->poll can do a simple spin_lock_irq/spin_unlock_irq. But for a quick peek your new code seems fine with that.
- Davide
[take24 7/6] kevent: signal notifications.
In reply to Evgeniy PolyakovSignals which were requested to be delivered through kevent subsystem must be registered through usual signal() and others syscalls, this option allows alternative delivery.
With KEVENT_SIGNAL_NOMASK flag being set in kevent for set of signals, they will not be delivered in a usual way. Kevents for appropriate signals are not copied when process forks, new process must add new kevents after fork(). Mask of signals is copied as before.
Test application which registers two signal callbacks for usr1 and usr2 signals and it's deivery through kevent (the former with both callback and kevent notifications, the latter only through kevent) is called signal.c and can be found in archive on project homepage
http://tservice.net.ru/~s0mbre/old/?section=projects&item=kevent
include/linux/kevent.h +7 -0
@@ -28,6 +28,7 @@ #include <linux/wait.h>#include <linux/net.h> #include <linux/rcupdate.h> #include <linux/fs.h> +#include <linux/sched.h> #include <linux/kevent_storage.h> #include <linux/ukevent.h>
@@ -220,4 +221,10 @@ #elsestatic inline void kevent_pipe_notify (struct inode *inode,u32 events) {}#endif +#ifdef CONFIG_KEVENT_SIGNAL +extern int kevent_signal_notify (struct task_struct *tsk,int sig); +#else +static inline int kevent_signal_notify (struct task_struct *tsk,int sig) {return 0 ;} +#endif +#endif /* __KEVENT_H */
include/linux/sched.h +5 -0
@@ -80,6 +80,7 @@ #include <linux/param.h>#include <linux/resource.h> #include <linux/timer.h> #include <linux/hrtimer.h> +#include <linux/kevent_storage.h> #include <asm/processor.h>
@@ -1013,6 +1014,10 @@ #endif#ifdef CONFIG_TASK_DELAY_ACCT struct task_delay_info *delays;#endif +#ifdef CONFIG_KEVENT_SIGNAL +struct kevent_storage st; +u32 kevent_signals; +#endif };static inline pid_t process_group (struct task_struct *tsk)
include/linux/ukevent.h +15 -4
@@ -68,7 +68,8 @@ #define KEVENT_POLL 3#define KEVENT_NAIO 4#define KEVENT_AIO 5#define KEVENT_PIPE 6 -#define KEVENT_MAX 7 +#define KEVENT_SIGNAL 7 +#define KEVENT_MAX 8/* * Per-type event sets.
@@ -81,7 +82,7 @@ #define KEVENT_MAX 7#define KEVENT_TIMER_FIRED 0x1/* -* Socket/network asynchronous IO events. +* Socket/network asynchronous IO and PIPE events. */ #define KEVENT_SOCKET_RECV 0x1#define KEVENT_SOCKET_ACCEPT 0x2
@@ -115,10 +116,20 @@ #define KEVENT_POLL_POLLREMOVE 0x1000*/ #define KEVENT_AIO_BIO 0x1 -#define KEVENT_MASK_ALL 0xffffffff +/* +* Signal events. +*/ +#define KEVENT_SIGNAL_DELIVERY 0x1 + +/* If set in raw64, then given signals will not be delivered +* in a usual way through sigmask update and signal callback +* invokation. */ +#define KEVENT_SIGNAL_NOMASK 0x8000000000000000ULL +/* Mask of all possible event values. */ -#define KEVENT_MASK_EMPTY 0x0 +#define KEVENT_MASK_ALL 0xffffffff/* Empty mask of ready events. */ +#define KEVENT_MASK_EMPTY 0x0struct kevent_id {
kernel/fork.c +8 -0
@@ -46,6 +46,7 @@ #include <linux/cn_proc.h>#include <linux/delayacct.h> #include <linux/taskstats_kern.h> #include <linux/random.h> +#include <linux/kevent.h> #include <asm/pgtable.h> #include <asm/pgalloc.h>
@@ -115,6 +116,9 @@ void __put_task_struct(struct task_strucWARN_ON (atomic_read (&tsk->usage));WARN_ON (tsk == current); +#ifdef CONFIG_KEVENT_SIGNAL +kevent_storage_fini (&tsk->st); +#endif security_task_free (tsk);free_uid (tsk->user);put_group_info (tsk->group_info);
@@ -1121,6 +1125,10 @@ #endifif (retval) goto bad_fork_cleanup_namespace; +#ifdef CONFIG_KEVENT_SIGNAL +kevent_storage_init (p, &p->st); +#endif + p->set_child_tid = (clone_flags &CLONE_CHILD_SETTID ) ? child_tidptr :NULL ;/* * Clear TID on mm_release()?
kernel/kevent/Kconfig +15 -0
@@ -43,3 +43,18 @@ config KEVENT_PIPE help This option enables notifications through KEVENT subsystem of pipe read/write operations. + +config KEVENT_SIGNAL + bool "Kernel event notifications for signals" + depends on KEVENT + help + This option enables signal delivery through KEVENT subsystem. + Signals which were requested to be delivered through kevent + subsystem must be registered through usual signal() and others + syscalls, this option allows alternative delivery. + With KEVENT_SIGNAL_NOMASK flag being set in kevent for set of + signals, they will not be delivered in a usual way. + Kevents for appropriate signals are not copied when process forks, + new process must add new kevents after fork(). Mask of signals + is copied as before. +
kernel/kevent/Makefile +1 -0
@@ -3,3 +3,4 @@ obj-$(CONFIG_KEVENT_TIMER) += kevent_tim obj-$(CONFIG_KEVENT_POLL) += kevent_poll.o obj-$(CONFIG_KEVENT_SOCKET) += kevent_socket.o obj-$(CONFIG_KEVENT_PIPE) += kevent_pipe.o +obj-$(CONFIG_KEVENT_SIGNAL) += kevent_signal.o
kernel/kevent/kevent_signal.c new file +87 -0
87 lines not on this page · raw message
kernel/signal.c +26 -0
@@ -23,6 +23,7 @@ #include <linux/syscalls.h>#include <linux/ptrace.h> #include <linux/signal.h> #include <linux/capability.h> +#include <linux/kevent.h> #include <asm/param.h> #include <asm/uaccess.h> #include <asm/unistd.h>
@@ -703,6 +704,9 @@ static int send_signal(int sig, struct s {struct sigqueue * q =NULL ;int ret =0 ; + +if (kevent_signal_notify (t, sig)) +return 1 ;/* * fast-pathed signals for kernel-internal things like SIGSTOP
@@ -782,6 +786,17 @@ specific_send_sig_info(int sig, struct s ret =send_signal (sig, info, t, &t->pending);if (!ret && !sigismember (&t->blocked, sig))signal_wake_up (t, sig ==SIGKILL ); +#ifdef CONFIG_KEVENT_SIGNAL +/* +* Kevent allows to deliver signals through kevent queue, +* it is possible to setup kevent to not deliver +* signal through the usual way, in that case send_signal() +* returns 1 and signal is delivered only through kevent queue. +* We simulate successfull delivery notification through this hack: +*/ +if (ret ==1 ) + ret =0 ; +#endif out:return ret; }
@@ -971,6 +986,17 @@ __group_send_sig_info(int sig, struct si* to avoid several races. */ ret =send_signal (sig, info, p, &p->signal->shared_pending); +#ifdef CONFIG_KEVENT_SIGNAL +/* +* Kevent allows to deliver signals through kevent queue, +* it is possible to setup kevent to not deliver +* signal through the usual way, in that case send_signal() +* returns 1 and signal is delivered only through kevent queue. +* We simulate successfull delivery notification through this hack: +*/ +if (ret ==1 ) + ret =0 ; +#endif if (unlikely (ret))return ret;
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovQuoted 2 lines · Evgeniy Polyakov
Generic event handling mechanism.
[...]
Sorry for the delay again. Kernel work is simply not my highest priority.
I've collected my comments on some parts of the patch. I haven't gone through every part of the patch yet. Sorry for the length.
===================
- basic ring buffer problem: the kevent_copy_ring_buffer function stores the event in the ring buffer without disregard of the current content.
+ if dequeued entries larger than number of ring buffer entries events immediately get overwritten without passing anything to userlevel
+ as with the old approach, the ring buffer is basically unusable with multiple threads/processes. A thread calling kevent_wait might cause entries another thread is still working on to be overwritten.
Possible solution:
a) it would be possible to have a "used" flag in each ring buffer entry. That's too expensive, I guess.
b) kevent_wait needs another parameter which specifies the which is the last (i.e., least recently added) entry in the ring buffer. Everything between this entry and the current head (in ->kidx) is occupied. If multiple threads arrive in kevent_wait the highest idx (with wrap around possibly lowest) is used.
kevent_wait will not try to move more entries into the ring buffer if ->kidx and the higest index passed in to any kevent_wait call is equal (i.e., the ring buffer is full).
There is one issue, though, and that is that a system call is needed to signal to the kernel that more entries in the ring buffer are processed and that they can be refilled. This goes against the kernel filling the ring buffer automatically (see below)
Threads should be able to (not necessarily forced to) use the interfaces like this:
- by default all threads are "parked" in the kevent_wait syscall.
- If an event occurs one thread might be woken (depending on the 'num' parameter)
- the woken thread(s) work on all the events in the ring buffer and then call kevent_wait() again.
This requires that the threads can independently call kevent_wait() and that they can independently retrieve events from the ring buffer without fear the entry gets overwritten before it is retrieved. Atomically retrieving entries from the ring buffer can be implemented at userlevel. Either the ring buffer is writable and a field in each ring buffer entry can be used as a 'handled' flag. Obviously this can be done with atomic compare-and-exchange. If the ring buffer is not writable then, as part of the userlevel wrapper around the event handling interfaces, another array is created which contains the use flags for each ring buffer entry. This is less elegant and probably slower.
===================
- implementing the kevent_wait syscall the proposed way means we are missing out on one possible optimization. The ring buffer is currently only filled on kevent_wait calls. I expect that in really high traffic situations requests are coming in at a higher rate than the can be processed. At least for periods of time. If such situations it would be nice to not have to call into the kernel at all. If the kernel would deliver into the ring buffer on its own this would be possible.
If the argument against this is that kevent_get_event should be possible the answer is...
===================
- the kevent_get_event syscall is not needed at all. All reporting should be done using a ring buffer. There really is not reason to keep two interfaces around which serve the same purpose. Making the argument the kevent_get_event is so much easier to use is not valid. The exposed interface to access the ring buffer will be easy, too. In the OLS paper I more or wait hinted at the interfaces. I think they should be like this (names are irrelevant):
ec_t ec_create(unsigned flags);
int ec_destroy(ec_t ec);
int ec_poll_event(ec_t ec, event_data_t *d);
int ec_wait_event(ec_t ec, event_data_t *d);
int ec_timedwait_event(ec_t ec, event_data_t *d, struct timespec *to);
The latter three interfaces are the interesting ones. We have to get the data out of the ring buffer as quickly as possible. So the interfaces require passing in a reference to an object which can hold the data. The 'poll' variant won't delay, the other two will.
We need separate create and destroy functions since there will always be a userlevel component of the data structures. The create variant can allocate the ring buffer and the other memory needed ('handled' flags, tail pointers, ...) and destroy free all resources.
These interfaces are fast and easy to use. At least as easy as the kevent_get_event syscall. And all transparently implemented on top of the ring buffer. So, please let's drop the unneeded syscall.
===================
- another optimization I am thinking about is optimizing the thread wakeup and ring buffer use for cache line use. I.e., if we know an event was queued on a specific CPU then the wakeup function should take this into account. I.e., if any of the threads waiting was/will be scheduled on the same CPU it should be preferred.
With the current simple form of a ring buffer this isn't sufficient, though. Reading all entries in the ring buffer until finding the one written by the CPU in question is not helpful. We'd need a mechanism to point the thread to the entry in question. One possibility to do this is to return the ring buffer entry as the return value of the kevent_wait() syscall. This works fine if the thread only works for one event (which I guess will be 99.999% of all uses). An extension could be to extend the ukevent structure to contain an index of the next entry written the same CPU.
Another problem this entails is false sharing of the ring buffer entries. This would probably require to pad the ukevent structure to 64 bytes. It's not that much more, 40 bytes so far, it's also more future-safe. The alternative is to allocate have per-CPU regions in the ring buffer. With hotplug CPUs this is just plain silly.
I think this optimization has the potential to help quite a bit, especially for large machines.
===================
- we absolutely need an interface to signal the kernel that a thread, just woken from kevent_wait, cannot handle the events. I.e., the events are in the ring buffer but all the other threads are in the kernel in their kevent_wait calls. The new syscall would wake up one or more threads to handle the events.
This syscall is for instance necessary if the thread calling kevent_wait is canceled. It might also be needed when a thread requested more than one event and realizes processing an entry takes a long time and that another thread might work on the other items in the meantime.
Al Viro pointed out another possible solution which also could solve the "handled" flag problem and concurrency in use of the ring buffer.
The idea is to require the kevent_wait() syscall to signal which entry in the ring buffer is handled or not handled. This means:
+ the kernel knows at any time which entries in the buffer are free and which are not
+ concurrent filling of the ring buffer is no problem anymore since entries are not discarded until told
+ by not waiting for event (num parameter == 0) the syscall can be used to discard entries to free up the ring buffer before continuing to work on more entries. And, as per the requirement above, it can be used to tell the kernel that certain entries are *NOT* handled and need to be sent to another thread. This would be useful in the thread cancellation case.
This seems like a nice approach.
===================
- why no syscall to create kevent queue? With dynamic /dev this might be a problem and it's really not much additional code. What about programs which want to use these interfaces before /dev is set up?
===================
- still: the syscall should use a struct timespec* timeout parameter and not nanosecs. There are at least three timeout modes which are wanted:
+ relative, unconditionally wait that long
+ relative, aborted in case of large enough settimeofday() or NTP adjustment
+ absolute timeout. Probably even with selecting which clock ot use. This mode requires a timespec value parameter
We have all this code already in the futex syscall. It just needs to be generalized or copied and adjusted.
===================
- still: no signal mask parameter in the kevent_wait (and get_event) syscall. Regardless of what one thinks about signals, they are used and integrating the kevent interface into existing code requires this functionality. And it's not only about receiving signals. The signal mask parameter can also be used to _prevent_ signals from being delivered in that time.
===================
- the KEVENT_REQ_WAKEUP_ONE functionality is good and needed. But I would reverse the default. I cannot see many places where you want all threads to be woken. Introduce KEVENT_REQ_WAKEUP_ALL instead.
===================
- there is really no reason to invent yet another timer implementation. We have the POSIX timers which are feature rich and nicely implemented. All that is needed is to implement SIGEV_KEVENT as a notification mechanism. The timer is registered as part of the timer_create() syscalls.
===================
I haven't yet looked at the other event sources. I think the above is enough for now.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 8 lines · Ulrich Drepper
Quoted 2 lines · Evgeniy Polyakov
Generic event handling mechanism.
[...]Sorry for the delay again. Kernel work is simply not my highest priority.
I've collected my comments on some parts of the patch. I haven't gone through every part of the patch yet. Sorry for the length.
No problem.
Quoted 24 lines
===================
- basic ring buffer problem: the
kevent_copy_ring_bufferfunction stores the event in the ring buffer without disregard of the current content.+ if dequeued entries larger than number of ring buffer entries events immediately get overwritten without passing anything to userlevel
+ as with the old approach, the ring buffer is basically unusable with multiple threads/processes. A thread calling
kevent_waitmight cause entries another thread is still working on to be overwritten.Possible solution:
a) it would be possible to have a "used" flag in each ring buffer entry. That's too expensive, I guess.
b)
kevent_waitneeds another parameter which specifies the which is the last (i.e., least recently added) entry in the ring buffer. Everything between this entry and the current head (in ->kidx) is occupied. If multiple threads arrive inkevent_waitthe highest idx (with wrap around possibly lowest) is used.
Quoted 8 lines
kevent_waitwill not try to move more entries into the ring buffer if ->kidx and the higest index passed in to anykevent_waitcall is equal (i.e., the ring buffer is full).There is one issue, though, and that is that a system call is needed to signal to the kernel that more entries in the ring buffer are processed and that they can be refilled. This goes against the kernel filling the ring buffer automatically (see below)
If thread calls kevent_wait() it means it has processed previous entries, one can call kevent_wait() with $num parameter as zero, which means that thread does not want any new events, so nothing will be copied.
Quoted 12 lines
Threads should be able to (not necessarily forced to) use the interfaces like this:
- by default all threads are "parked" in the
kevent_waitsyscall.- If an event occurs one thread might be woken (depending on the 'num' parameter)
- the woken thread(s) work on all the events in the ring buffer and then call
kevent_wait()again.
Quoted 11 lines
This requires that the threads can independently call
kevent_wait()and that they can independently retrieve events from the ring buffer without fear the entry gets overwritten before it is retrieved. Atomically retrieving entries from the ring buffer can be implemented at userlevel. Either the ring buffer is writable and a field in each ring buffer entry can be used as a 'handled' flag. Obviously this can be done with atomic compare-and-exchange. If the ring buffer is not writable then, as part of the userlevel wrapper around the event handling interfaces, another array is created which contains the use flags for each ring buffer entry. This is less elegant and probably slower.
Writable ring buffer does not sound too good to me - what if one thread will overwrite the whole ring buffer so kernel's indexes can be screwed?
Ring buffer processed not in FIFO order is wrong idea - ring buffer can be potentially very big and searching there for the entry, which was been marked as 'free' by userspace is not a solution at all - userspace in that case must provide ukevent so fast tree search would be used, (and although it is already possible) it requires userspace to make additional syscalls which is not what we want.
So kevent ring buffer is designed in the following way: all entries can be processed _only_ in fifo order, i.e. they can be read in any order threads want, but when one thread calls kevent_wait(num), $num requested from the begining can be overwritten - kernel does not know how many users reads those $num events from the begining, and even if they have some flag that 'do not touch me, someone reads me', how and when those entries will be reused? Kernel does not store bitmask or any other type of objects to show that holes in the ring buffer are free - it works in FIFO order since it the fastest mode.
As a solution I can create folowing scheme:
there are two syscalls (or one with a switch) which get events and commits them.
kevent_wait() becomes a syscall which waits until number of events or one of them becomes ready and just copies them into ring buffer and returns. kevent_wait() will fail with special error code when ring buffer is full.
kevent_commit() frees requested number of events _from the beginning_, i.e. from special index, visible from userspace. Userspace can create special counters for events (and even put them into read-only ring buffer overwriting some fields of kevent, especially if we will increase it's size) and only call kevent_commit() when all events have zero usage counter.
I disagree that having possibility to have holes in the ring buffer is a good idea at all - it requires much more complex protocol, which will fill and reuse that holes, and the main disavantge - it requires to transfer much more information from userspace to kernelspace to free the ring entry in the hole - in that case it is already possible just to call kevent_ctl(KEVENT_REMOVE) and do not wash the brain with new approach at all.
Quoted 10 lines
===================
- implementing the
kevent_waitsyscall the proposed way means we are missing out on one possible optimization. The ring buffer is currently only filled onkevent_waitcalls. I expect that in really high traffic situations requests are coming in at a higher rate than the can be processed. At least for periods of time. If such situations it would be nice to not have to call into the kernel at all. If the kernel would deliver into the ring buffer on its own this would be possible.
Well, it can be done on behalf of workqueue or dedicated thread which will bring up appropriate mm context, although it means that userspace can not handle the load it requested, which is a bad sign...
Quoted 5 lines
If the argument against this is that
kevent_get_eventshould be possible the answer is...===================
Quoted 7 lines
- the
kevent_get_eventsyscall is not needed at all. All reporting should be done using a ring buffer. There really is not reason to keep two interfaces around which serve the same purpose. Making the argument thekevent_get_eventis so much easier to use is not valid. The exposed interface to access the ring buffer will be easy, too. In the OLS paper I more or wait hinted at the interfaces. I think they should be like this (names are irrelevant):
Well, kevent_get_events() _is_ much easier to use. And actually having only that interface it is possible to implement ring buffer with any kind or protocol for its controlling - userspace can have a wrapper which will call kevent_get_events() with pointer which shows to the place in the shared ring buffer where to place new events, that wrapper can handle essentially any kind of flags/parameters which are suitable for that ring buffer implementation.
But since we started to implement ring buffer as a additional feature of kevent, let's find the way all people will be happy with before removing something which was proven to work correctly.
Quoted 10 lines
ec_tec_create(unsigned flags);
intec_destroy(ec_t ec);
intec_poll_event(ec_t ec, event_data_t *d);
intec_wait_event(ec_t ec, event_data_t *d);
intec_timedwait_event(ec_t ec, event_data_t *d, struct timespec *to);The latter three interfaces are the interesting ones. We have to get the data out of the ring buffer as quickly as possible. So the interfaces require passing in a reference to an object which can hold the data. The 'poll' variant won't delay, the other two will.
The last three are exactly kevent_get_events() with different set of parameters - it is possible to get events without sleeping, it is possible to wait until at least something is ready and it is possible to sleep for timeout.
Quoted 8 lines
We need separate create and destroy functions since there will always be a userlevel component of the data structures. The create variant can allocate the ring buffer and the other memory needed ('handled' flags, tail pointers, ...) and destroy free all resources.
These interfaces are fast and easy to use. At least as easy as the
kevent_get_eventsyscall. And all transparently implemented on top of the ring buffer. So, please let's drop the unneeded syscall.
They all already imeplemented. Just all above, and it was done several months ago already. No need to reinvent what is already there. Even if we will decide to remove kevent_get_events() in favour of ring buffer-only implementation, winting-for-event syscall will be essentially kevent_get_events() without pointer to the place where to put events.
And I will not repeat, that it is (and was from the beginning for about 10 months already) to implement ring buffer using kevent_get_events().
I agree that having special syscall to initialize kevent is a good idea, and initial kevent implementation had it, but it was removed due to API cleanup work by Cristoph Hellwing.
So I again see the same problem as several months ago when there are many people who have opposite views on API, and I as author do not know who is right...
Can we all agree that initialization syscall is a good idea?
Quoted 8 lines
===================
- another optimization I am thinking about is optimizing the thread wakeup and ring buffer use for cache line use. I.e., if we know an event was queued on a specific CPU then the wakeup function should take this into account. I.e., if any of the threads waiting was/will be scheduled on the same CPU it should be preferred.
Do you have _any_ kind of benchmarks with epoll() which would show that it is feasible? ukevent is one cache line (well, 2 cache lines on old CPUs), which can be setup way too far away from the time when it is ready, and CPU which origianlly set that up can be busy, so we will lose performance waiting until CPU becomes free instead of calling other thread on different CPU.
So I'm asking is there at least some data except theoretical thoughts?
Quoted 17 lines
With the current simple form of a ring buffer this isn't sufficient, though. Reading all entries in the ring buffer until finding the one written by the CPU in question is not helpful. We'd need a mechanism to point the thread to the entry in question. One possibility to do this is to return the ring buffer entry as the return value of the
kevent_wait()syscall. This works fine if the thread only works for one event (which I guess will be 99.999% of all uses). An extension could be to extend the ukevent structure to contain an index of the next entry written the same CPU.Another problem this entails is false sharing of the ring buffer entries. This would probably require to pad the ukevent structure to 64 bytes. It's not that much more, 40 bytes so far, it's also more future-safe. The alternative is to allocate have per-CPU regions in the ring buffer. With hotplug CPUs this is just plain silly.
Quoted 2 lines
I think this optimization has the potential to help quite a bit, especially for large machines.
I think again that complete removal of ring buffer and its implementation in userspace wrapper and kevent_get_events() is a good idea. But probably I'm alone thinking in that direction, so let's think about ring buffer in kernelspace.
It is possible to specify CPU id in kevent (not in ukevent, i.e. not in shared by userspace structure, but in it's kernel representation), and then check if currently active CPU is the same or not, but what if it is not the same CPU? Entry order is important, since application can take advantage of synchronization, so idea to skip some entries is bad.
Quoted 8 lines
===================
- we absolutely need an interface to signal the kernel that a thread, just woken from
kevent_wait, cannot handle the events. I.e., the events are in the ring buffer but all the other threads are in the kernel in theirkevent_waitcalls. The new syscall would wake up one or more threads to handle the events.
Quoted 5 lines
This syscall is for instance necessary if the thread calling
kevent_waitis canceled. It might also be needed when a thread requested more than one event and realizes processing an entry takes a long time and that another thread might work on the other items in the meantime.
Hmm, send a signal to other thread when glibc cancells given one... This problem points me to the idea of userspace thread implementation I have in mind, but it is another story.
It is management task - kernel should not even know about someone has died and can not process events it requested.
Userspace can open a control pipe (and setup a kevent handler for it) and glibc will write there a byte thus awakening some other thread. It can be done in userspace and should be done in userspace.
If you insist I will create userspace kevent handling - userspace will be able to request kevents and mark them as ready.
Quoted 12 lines
Al Viro pointed out another possible solution which also could solve the "handled" flag problem and concurrency in use of the ring buffer.
The idea is to require the
kevent_wait()syscall to signal which entry in the ring buffer is handled or not handled. This means:+ the kernel knows at any time which entries in the buffer are free and which are not
+ concurrent filling of the ring buffer is no problem anymore since entries are not discarded until told
Quoted 8 lines
+ by not waiting for event (num parameter == 0) the syscall can be used to discard entries to free up the ring buffer before continuing to work on more entries. And, as per the requirement above, it can be used to tell the kernel that certain entries are *NOT* handled and need to be sent to another thread. This would be useful in the thread cancellation case.
This seems like a nice approach.
But unfortunately theory and practice are different in a real world. Kernel has millions of entries in _linear_ ring buffer, how do you think they should be handled without complex protocol between userspace and kernelspace? In that protocol userspace is required to transfer some information to kernelspace so it could find the entry (i.e. per entry field ! ), and then it should have a tree or other mechanism to store free and used chunks of entries...
You probably did not see my network tree allocator patches I posted in lkml@, netdev@ and linux-mm@ lists - it is quite big chunk of code which handles exactly that, but you do not want to implement it in glibc I think...
So, do not overdesign.
And as a side note, btw - _all_ above can be implemented in userspace.
Quoted 5 lines
===================
- why no syscall to create kevent queue? With dynamic /dev this might be a problem and it's really not much additional code. What about programs which want to use these interfaces before /dev is set up?
It was there - Cristoph Hellwig removed it in his API cleanup patch, so far it was not needed at all (and is not needed for now). That application can create /dev file by itself if it wants... Just a though.
Quoted 11 lines
===================
- still: the syscall should use a struct timespec* timeout parameter and not nanosecs. There are at least three timeout modes which are wanted:
+ relative, unconditionally wait that long
+ relative, aborted in case of large enough
settimeofday()or NTP adjustment
Quoted 6 lines
+ absolute timeout. Probably even with selecting which clock ot use. This mode requires a timespec value parameter
We have all this code already in the futex syscall. It just needs to be generalized or copied and adjusted.
Will we discuss it for death?
Kevent does not need to have absolute timeout.
Because timeout specified there is always related to the start of syscall, since it is a timeout which specifies maximum time frame syscall can live.
All such timeouts _ARE_ relative and should be relative since it is correct.
Quoted 8 lines
===================
- still: no signal mask parameter in the
kevent_wait(andget_event) syscall. Regardless of what one thinks about signals, they are used and integrating the kevent interface into existing code requires this functionality. And it's not only about receiving signals. The signal mask parameter can also be used to _prevent_ signals from being delivered in that time.
I created kevent_signal notifications - it allows user to setup any set of interested signals before call to kevent_get_events() and friends.
No need to solve a problem with operation way when there is tactical and strategical ones - kevent signal is that way which allows not to use workarounds for interfaces which do not support handling of different types of events except file descriptors.
Quoted 5 lines
===================
- the
KEVENT_REQ_WAKEUP_ONEfunctionality is good and needed. But I would reverse the default. I cannot see many places where you want all threads to be woken. IntroduceKEVENT_REQ_WAKEUP_ALLinstead.
I.e. to wake up only first thread always and in addon those threads which have specified flag set? Ok, will put into todo foer the next release.
Quoted 7 lines
===================
- there is really no reason to invent yet another timer implementation. We have the POSIX timers which are feature rich and nicely implemented. All that is needed is to implement
SIGEV_KEVENTas a notification mechanism. The timer is registered as part of thetimer_create()syscalls.
Feel free to add any interface you like - it is as simple as call for kevent_user_add_ukevent() in userspace.
Quoted 5 lines
===================
I haven't yet looked at the other event sources. I think the above is enough for now.
It looks like you generate ideas (or move them into different implementation layer) faster than I implement them :)
And I almost silently stay behind with the fact that it is possbile to implement _all_ above ring buffer things in userspace with kevent_get_events() and this functionality is there for almost a year :)
Let's solve problem in order of theirs appearance - what do you think about above interface for ring buffer?
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 10 lines · Evgeniy Polyakov
Quoted 7 lines
===================
- there is really no reason to invent yet another timer implementation. We have the POSIX timers which are feature rich and nicely implemented. All that is needed is to implement
SIGEV_KEVENTas a notification mechanism. The timer is registered as part of thetimer_create()syscalls.Feel free to add any interface you like - it is as simple as call for
kevent_user_add_ukevent()in userspace.
... in kernelspace I mean.
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovQuoted 15 lines · Evgeniy Polyakov
Quoted 15 lines
Possible solution:
a) it would be possible to have a "used" flag in each ring buffer entry. That's too expensive, I guess.
b)
kevent_waitneeds another parameter which specifies the which is the last (i.e., least recently added) entry in the ring buffer. Everything between this entry and the current head (in ->kidx) is occupied. If multiple threads arrive inkevent_waitthe highest idx (with wrap around possibly lowest) is used.
kevent_waitwill not try to move more entries into the ring buffer if ->kidx and the higest index passed in to anykevent_waitcall is equal (i.e., the ring buffer is full).
Quoted 8 lines
Quoted 4 lines
There is one issue, though, and that is that a system call is needed to signal to the kernel that more entries in the ring buffer are processed and that they can be refilled. This goes against the kernel filling the ring buffer automatically (see below)
If thread calls
kevent_wait()it means it has processed previous entries, one can callkevent_wait()with $num parameter as zero, which means that thread does not want any new events, so nothing will be
copied.
This doesn't solve the problem. You could only request new events when all previously reported events are processed. Plus: how do you report events if the you don't allow get_event pass them on?
Quoted 2 lines
Writable ring buffer does not sound too good to me - what if one thread will overwrite the whole ring buffer so kernel's indexes can be screwed?
Agreed, there are problems. This is why I suggested the ring buffer can be a structured. Parts of it might be read-only, other parts read/write. I don't necessarily think the 'used' flag is the right way.
And front/tail pointer solution seems to be better.
Quoted 1 line
Ring buffer processed not in FIFO order is wrong idea
Not necessarily, see my comments about CPU affinity in the previous mail.
Quoted 6 lines
- ring buffer can
be potentially very big and searching there for the entry, which was been marked as 'free' by userspace is not a solution at all - userspace in that case must provide ukevent so fast tree search would be used, (and although it is already possible) it requires userspace to make additional syscalls which is not what we want.
It is not necessary. I've proposed to only have a fron and tail pointer. The tail pointer is maintained by the application and passed to the kernel explicitly or via shared memory. The kernel maintains the front pointer. No tree needed.
Quoted 9 lines
As a solution I can create folowing scheme:
there are two syscalls (or one with a switch) which get events and commits them.
kevent_wait()becomes a syscall which waits until number of events or one of them becomes ready and just copies them into ring buffer and returns.kevent_wait()will fail with special error code when ring buffer is full.
Quoted 5 lines
kevent_commit()frees requested number of events_fromthebeginning_,
i.e. from special index, visible from userspace. Userspace can create
special counters for events (and even put them into read-only ring buffer overwriting some fields of kevent, especially if we will increase it's size) and only callkevent_commit()when all events have zero usage
counter.
Right, that's basically the front/tail pointer implementation. That would work. You just have to make sure that the kevent_wait() call takes the current front pointer/index as a parameter. This way if the buffer gets filled between the thread checking the ring buffer (and finding it empty) and the syscall being handled the thread is not suspended.
Quoted 7 lines
I disagree that having possibility to have holes in the ring buffer is a good idea at all - it requires much more complex protocol, which will fill and reuse that holes, and the main disavantge - it requires to transfer much more information from userspace to kernelspace to free the ring entry in the hole - in that case it is already possible just to call
kevent_ctl(KEVENT_REMOVE)and do not wash the brain with new approach at all.
Well, it would require more data transport of we'd use writable shared memory. But I agree, it's far too complicated and might not scale with growing ring buffer sizes.
Quoted 11 lines
Quoted 8 lines
- implementing the
kevent_waitsyscall the proposed way means we are missing out on one possible optimization. The ring buffer is currently only filled onkevent_waitcalls. I expect that in really high traffic situations requests are coming in at a higher rate than the can be processed. At least for periods of time. If such situations it would be nice to not have to call into the kernel at all. If the kernel would deliver into the ring buffer on its own this would be possible.Well, it can be done on behalf of workqueue or dedicated thread which will bring up appropriate mm context,
I think it should be done. It's potentially a huge advantage.
Quoted 2 lines
although it means that userspace
can not handle the load it requested, which is a bad sign...
I don't understand. What is not supposed to work? There is nothing which cannot work with automatic posting since the get_event() call does nothing but copying the event data over and wake a thread.
Quoted 8 lines
Quoted 7 lines
- the
kevent_get_eventsyscall is not needed at all. All reporting should be done using a ring buffer. There really is not reason to keep two interfaces around which serve the same purpose. Making the argument thekevent_get_eventis so much easier to use is not valid. The exposed interface to access the ring buffer will be easy, too. In the OLS paper I more or wait hinted at the interfaces. I think they should be like this (names are irrelevant):
Quoted 7 lines
Well,
kevent_get_events()_is_ much easier to use. And actually having only that interface it is possible to implement ring buffer with any kind or protocol for its controlling - userspace can have a wrapper which will callkevent_get_events()with pointer which shows to the place in the shared ring buffer where to place new events, that wrapper can handle essentially any kind of flags/parameters which are suitable for that ring buffer implementation.
That's far too slow. The whole point behind the ring buffer is speed. And emulation would defeat the purpose.
Quoted 3 lines
But since we started to implement ring buffer as a additional feature of kevent, let's find the way all people will be happy with before removing something which was proven to work correctly.
The get_event interface is basically the userlevel interface the runtime (glibc probably) would provide. Programmers don't see the complexity.
I'm concerned about the get_event interface holding the kernel implementation back. For instance, automatic filling the ring buffer. This would not be possible if the program is free to mix kevent_get_event and kevent_wait calls freely. If you do away with the get_event syscall the automatic ring buffer filling is possible and a logical extension.
Quoted 5 lines
The last three are exactly
kevent_get_events()with different set of parameters - it is possible to get events without sleeping, it is possible to wait until at least something is ready and it is possible to sleep for timeout.
Exactly. But these interfaces should be implemented at userlevel, not at the syscall level. It's not necessary. The kernel interface should be kept as small as possible and the get_event syscall is pure duplication.
Quoted 6 lines
They all already imeplemented. Just all above, and it was done several months ago already. No need to reinvent what is already there. Even if we will decide to remove
kevent_get_events()in favour of ring buffer-only implementation, winting-for-event syscall will be essentiallykevent_get_events()without pointer to the place where to put events.
Right, but this limitation of the interface is important. It means the interface of the kernel is smaller: fewer possibilities for problems and fewer constraints if in future something should be changed (and smaller kernel).
Quoted 3 lines
I agree that having special syscall to initialize kevent is a good idea, and initial kevent implementation had it, but it was removed due to API cleanup work by Cristoph Hellwing.
Well, he is wrong. If, for instance, init or any of the programs which start first wants to use the syscall it couldn't because /dev isn't mounted. The program might use libraries and therefore not have any influence on whether the kevent stuff is used or not.
Yes, the /dev interface is useful for some/many other kernel interfaces.
But this is a core interface. For the same reason epoll_create is a
syscall.
Quoted 6 lines
Do you have _any_ kind of benchmarks with
epoll()which would show that it is feasible? ukevent is one cache line (well, 2 cache lines on old CPUs), which can be setup way too far away from the time when it is ready, and CPU which origianlly set that up can be busy, so we will lose performance waiting until CPU becomes free instead of calling other thread on different CPU.
If the period between the generation of the event (e.g., incoming network traffic or sent data) and the delivery of the event by waking a thread is too long, it makes not too much sense. But if the L2 cache hasn't hasn't been flushed it might be a big advantage.
I think it's reasonable to only have the last queued entry for a CPU handled special. And note, this is only ever a hint. If an event entry was created by the kernel in one CPU but none of the threads which wait to be waken is on that CPU, nothing has to be done.
No, I don't have a benchmark. But it is likely quite easily possible to
create a synthetic benachmark. Maybe with pipes.
Quoted 4 lines
It is possible to specify CPU id in kevent (not in ukevent, i.e. not in shared by userspace structure, but in it's kernel representation), and then check if currently active CPU is the same or not, but what if it is not the same CPU?
Nothing special. It's up to the userlevel wrapper code. The CPU number would only be a hint.
Quoted 2 lines
Entry order is important, since application can
take advantage of synchronization, so idea to skip some entries is bad.
That's something the application should be make a call about. It's not always (or even mostly) the case that the ordering of the notification is important. Furthermore, this would also require the kernel to enforce an ordering. This is expensive on SMP machines. A locally generated event (i.e., source and the thread reporting the event) can be delivered faster than an event created on another CPU.
Quoted 2 lines
It is management task - kernel should not even know about someone has died and can not process events it requested.
But the kernel has to be involed.
Quoted 2 lines
Userspace can open a control pipe (and setup a kevent handler for it) and glibc will write there a byte thus awakening some other thread. It can be done in userspace and should be done in userspace.
That's invasive. The problem is that no userlevel interface should have to implicitly keep file descriptors open. This would mean the application would be influenced since suddenly a file descriptor is not available anymore. Yes, applications shouldn't care but they unfortunately sometimes do.
Quoted 3 lines
Will we discuss it for death?
Kevent does not need to have absolute timeout.
Of course it does. Just because you don't see a need for it for your applications right now it doesn't mean it's not a valid use.
Quoted 3 lines
Because timeout specified there is always related to the start of syscall, since it is a timeout which specifies maximum time frame syscall can live.
That's your current implementation. There is absolutely no reason whatsoever why this couldn't be changed.
Quoted 5 lines
I created
kevent_signalnotifications - it allows user to setup any set of interested signals before call tokevent_get_events()and friends.No need to solve a problem with operation way when there is tactical and strategical ones
Of course there is a need and I explained it before. Getting signal notifications is in no way the same as changing the signal mask temporarily. You cannot correctly emulate the case where you want to block a signal while in the call as reenable it afterwards. Receiving the signal as an event and then artificially raising it is not the same.
Especially timing-wise, the signal kevent might not be seen long after
the syscall returns because other entries are worked on first.
The opposite case is equally impossible to emulate: unblocking a signal just for the duration of the syscall. These are all possible and used cases.
Quoted 7 lines
Quoted 3 lines
- the
KEVENT_REQ_WAKEUP_ONEfunctionality is good and needed. But I would reverse the default. I cannot see many places where you want all threads to be woken. IntroduceKEVENT_REQ_WAKEUP_ALLinstead.I.e. to wake up only first thread always and in addon those threads which have specified flag set? Ok, will put into todo foer the next release.
It's a flag for an event. So the threads won't have the flag set. If an event is delivered with the flag set, wake all threads. Otherwise just one.
Quoted 8 lines
Quoted 5 lines
- there is really no reason to invent yet another timer implementation. We have the POSIX timers which are feature rich and nicely implemented. All that is needed is to implement
SIGEV_KEVENTas a notification mechanism. The timer is registered as part of thetimer_create()syscalls.Feel free to add any interface you like - it is as simple as call for
kevent_user_add_ukevent()in userspace.
No, that's not what I mean. There is no need for the special timer-related part of your patch. Instead the existing POSIX timer syscalls should be modified to handle SIGEV_KEVENT notification. Again, keep the interface as small as possible. Plus, the POSIX timer interface is very flexible. You don't want to duplicate all that functionality.
Quoted 3 lines
And I almost silently stay behind with the fact that it is possbile to implement _all_ above ring buffer things in userspace with
kevent_get_events()and this functionality is there for almost a year :)
Again, this defeats the purpose completely. The ring buffer is the faster interface, especially when coupled with asynchronous filling of ring buffer (i.e., without a syscal).
Quoted 2 lines
Let's solve problem in order of theirs appearance - what do you think about above interface for ring buffer?
Looks better, yes.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 21 lines · Ulrich Drepper
Quoted 20 lines · Evgeniy Polyakov
Quoted 19 lines
Possible solution:
a) it would be possible to have a "used" flag in each ring buffer entry. That's too expensive, I guess.
b)
kevent_waitneeds another parameter which specifies the which is the last (i.e., least recently added) entry in the ring buffer. Everything between this entry and the current head (in ->kidx) is occupied. If multiple threads arrive inkevent_waitthe highest idx (with wrap around possibly lowest) is used.
kevent_waitwill not try to move more entries into the ring buffer if ->kidx and the higest index passed in to anykevent_waitcall is equal (i.e., the ring buffer is full).There is one issue, though, and that is that a system call is needed to signal to the kernel that more entries in the ring buffer are processed and that they can be refilled. This goes against the kernel filling the ring buffer automatically (see below)
Quoted 8 lines
Quoted 4 lines
If thread calls
kevent_wait()it means it has processed previous entries, one can callkevent_wait()with $num parameter as zero, which means that thread does not want any new events, so nothing will be copied.This doesn't solve the problem. You could only request new events when all previously reported events are processed. Plus: how do you report events if the you don't allow
get_eventpass them on?
Userspace should itself maintain order and possibility to get event in this implementation, kernel just returns events which were requested.
Quoted 21 lines
Quoted 2 lines
Writable ring buffer does not sound too good to me - what if one thread will overwrite the whole ring buffer so kernel's indexes can be screwed?
Agreed, there are problems. This is why I suggested the ring buffer can be a structured. Parts of it might be read-only, other parts read/write. I don't necessarily think the 'used' flag is the right way.
And front/tail pointer solution seems to be better.Quoted 1 line
Ring buffer processed not in FIFO order is wrong idea
Not necessarily, see my comments about CPU affinity in the previous mail.
Quoted 6 lines
- ring buffer can
be potentially very big and searching there for the entry, which was been marked as 'free' by userspace is not a solution at all - userspace in that case must provide ukevent so fast tree search would be used, (and although it is already possible) it requires userspace to make additional syscalls which is not what we want.
Quoted 4 lines
It is not necessary. I've proposed to only have a fron and tail pointer. The tail pointer is maintained by the application and passed to the kernel explicitly or via shared memory. The kernel maintains the front pointer. No tree needed.
There was such implementation (in previous patchset) - sine no one commented, I changed that.
Quoted 16 lines
Quoted 9 lines
As a solution I can create folowing scheme:
there are two syscalls (or one with a switch) which get events and commits them.
kevent_wait()becomes a syscall which waits until number of events or one of them becomes ready and just copies them into ring buffer and returns.kevent_wait()will fail with special error code when ring buffer is full.Quoted 6 lines
kevent_commit()frees requested number of events_fromthebeginning_, i.e. from special index, visible from userspace. Userspace can create special counters for events (and even put them into read-only ring buffer overwriting some fields of kevent, especially if we will increase it's size) and only callkevent_commit()when all events have zero usage counter.
Quoted 5 lines
Right, that's basically the front/tail pointer implementation. That would work. You just have to make sure that the
kevent_wait()call takes the current front pointer/index as a parameter. This way if the buffer gets filled between the thread checking the ring buffer (and finding it empty) and the syscall being handled the thread is not suspended.
It is exactly how previous ring buffer (in mapped area though) was implemented.
I think I need to quickly setup my slightly used (bought on ebay) but still working mind reader, I will try to tune it to work with your brain waves so next time I would not spent weeks changing something which could be reused, while others keep silent :)
Quoted 28 lines
Quoted 7 lines
I disagree that having possibility to have holes in the ring buffer is a good idea at all - it requires much more complex protocol, which will fill and reuse that holes, and the main disavantge - it requires to transfer much more information from userspace to kernelspace to free the ring entry in the hole - in that case it is already possible just to call
kevent_ctl(KEVENT_REMOVE)and do not wash the brain with new approach at all.Well, it would require more data transport of we'd use writable shared memory. But I agree, it's far too complicated and might not scale with growing ring buffer sizes.
Quoted 11 lines
Quoted 8 lines
- implementing the
kevent_waitsyscall the proposed way means we are missing out on one possible optimization. The ring buffer is currently only filled onkevent_waitcalls. I expect that in really high traffic situations requests are coming in at a higher rate than the can be processed. At least for periods of time. If such situations it would be nice to not have to call into the kernel at all. If the kernel would deliver into the ring buffer on its own this would be possible.Well, it can be done on behalf of workqueue or dedicated thread which will bring up appropriate mm context,
I think it should be done. It's potentially a huge advantage.
Quoted 6 lines
Quoted 2 lines
although it means that userspace
can not handle the load it requested, which is a bad sign...I don't understand. What is not supposed to work? There is nothing which cannot work with automatic posting since the
get_event()call does nothing but copying the event data over and wake a thread.
If userspace is too slow to get events, dedicated thread or workqueue will be busy unneded things, although they can allow to remove peaks in the load.
Quoted 16 lines
Quoted 8 lines
Quoted 7 lines
- the
kevent_get_eventsyscall is not needed at all. All reporting should be done using a ring buffer. There really is not reason to keep two interfaces around which serve the same purpose. Making the argument thekevent_get_eventis so much easier to use is not valid. The exposed interface to access the ring buffer will be easy, too. In the OLS paper I more or wait hinted at the interfaces. I think they should be like this (names are irrelevant):Quoted 7 lines
Well,
kevent_get_events()_is_ much easier to use. And actually having only that interface it is possible to implement ring buffer with any kind or protocol for its controlling - userspace can have a wrapper which will callkevent_get_events()with pointer which shows to the place in the shared ring buffer where to place new events, that wrapper can handle essentially any kind of flags/parameters which are suitable for that ring buffer implementation.
Quoted 2 lines
That's far too slow. The whole point behind the ring buffer is speed. And emulation would defeat the purpose.
It was an example, I do not say ring-buffer maintained in kernelspace is bad idea. Actually it is possible to create several threads which will only read events into the buffer, which will be processed by some pool of 'working' threads. There are a lot of possibilities to work with only one syscall and create scalable system.
Quoted 7 lines
Quoted 3 lines
But since we started to implement ring buffer as a additional feature of kevent, let's find the way all people will be happy with before removing something which was proven to work correctly.
The
get_eventinterface is basically the userlevel interface the runtime (glibc probably) would provide. Programmers don't see the complexity.
Quoted 6 lines
I'm concerned about the
get_eventinterface holding the kernel implementation back. For instance, automatic filling the ring buffer. This would not be possible if the program is free to mixkevent_get_eventandkevent_waitcalls freely. If you do away with theget_eventsyscall the automatic ring buffer filling is possible and a logical extension.
Yes, that is why only one should be used.
If there are several threads, then ring buffer implementation should be used otherwise just kevent_get_events().
In theory yes, access library like glibc can provide kevent_get_events() which will read event from ring buffer, but there is no such call right now, so kernel's kevent_get_events() looks reasonable.
Quoted 8 lines
Quoted 4 lines
The last three are exactly
kevent_get_events()with different set of parameters - it is possible to get events without sleeping, it is possible to wait until at least something is ready and it is possible to sleep for timeout.Exactly. But these interfaces should be implemented at userlevel, not at the syscall level. It's not necessary. The kernel interface should be kept as small as possible and the
get_eventsyscall is pure duplication.
I would say that ring-buffer mainpulating syscalls are duplicatino, but it is just matter of a view :)
Quoted 11 lines
Quoted 6 lines
They all already imeplemented. Just all above, and it was done several months ago already. No need to reinvent what is already there. Even if we will decide to remove
kevent_get_events()in favour of ring buffer-only implementation, winting-for-event syscall will be essentiallykevent_get_events()without pointer to the place where to put events.Right, but this limitation of the interface is important. It means the interface of the kernel is smaller: fewer possibilities for problems and fewer constraints if in future something should be changed (and smaller kernel).
Ok, lets see for ring buffer implementation right now, and then we will decide if we want to remove or to stay with kevent_get_events() syscall.
Quoted 4 lines
Quoted 3 lines
I agree that having special syscall to initialize kevent is a good idea, and initial kevent implementation had it, but it was removed due to API cleanup work by Cristoph Hellwing.
Quoted 8 lines
Well, he is wrong. If, for instance, init or any of the programs which start first wants to use the syscall it couldn't because /dev isn't mounted. The program might use libraries and therefore not have any influence on whether the kevent stuff is used or not.
Yes, the /dev interface is useful for some/many other kernel interfaces.
But this is a core interface. For the same reasonepoll_createis a
syscall.
Ok, I will create initialization syscall.
Quoted 33 lines
Quoted 6 lines
Do you have _any_ kind of benchmarks with
epoll()which would show that it is feasible? ukevent is one cache line (well, 2 cache lines on old CPUs), which can be setup way too far away from the time when it is ready, and CPU which origianlly set that up can be busy, so we will lose performance waiting until CPU becomes free instead of calling other thread on different CPU.If the period between the generation of the event (e.g., incoming network traffic or sent data) and the delivery of the event by waking a thread is too long, it makes not too much sense. But if the L2 cache hasn't hasn't been flushed it might be a big advantage.
I think it's reasonable to only have the last queued entry for a CPU handled special. And note, this is only ever a hint. If an event entry was created by the kernel in one CPU but none of the threads which wait to be waken is on that CPU, nothing has to be done.
No, I don't have a benchmark. But it is likely quite easily possible to
create a synthetic benachmark. Maybe with pipes.Quoted 4 lines
It is possible to specify CPU id in kevent (not in ukevent, i.e. not in shared by userspace structure, but in it's kernel representation), and then check if currently active CPU is the same or not, but what if it is not the same CPU?
Nothing special. It's up to the userlevel wrapper code. The CPU number would only be a hint.
Quoted 2 lines
Entry order is important, since application can
take advantage of synchronization, so idea to skip some entries is bad.
Quoted 6 lines
That's something the application should be make a call about. It's not always (or even mostly) the case that the ordering of the notification is important. Furthermore, this would also require the kernel to enforce an ordering. This is expensive on SMP machines. A locally generated event (i.e., source and the thread reporting the event) can be delivered faster than an event created on another CPU.
How come? If signal was delivered earlier than data arrived, userspace should get signal before data - that is the rule. Ordering is maintained not for event insertion, but for marking them ready - it is atomic, so who first starts to mark even ready, that event will be read first from the ready queue.
Quoted 10 lines
Quoted 2 lines
It is management task - kernel should not even know about someone has died and can not process events it requested.
But the kernel has to be involed.
Quoted 3 lines
Userspace can open a control pipe (and setup a kevent handler for it) and glibc will write there a byte thus awakening some other thread. It can be done in userspace and should be done in userspace.
Quoted 5 lines
That's invasive. The problem is that no userlevel interface should have to implicitly keep file descriptors open. This would mean the application would be influenced since suddenly a file descriptor is not available anymore. Yes, applications shouldn't care but they unfortunately sometimes do.
Then I propose userspace notifications - each new thread can register 'wake me up when userspace event 1 is ready' and 'event 1' will be marked as ready by glibc when it removes the thread.
Quoted 6 lines
Quoted 3 lines
Will we discuss it for death?
Kevent does not need to have absolute timeout.
Of course it does. Just because you don't see a need for it for your applications right now it doesn't mean it's not a valid use.
Please explain why glibc AIO uses relatinve timeouts then :)
Quoted 6 lines
Quoted 3 lines
Because timeout specified there is always related to the start of syscall, since it is a timeout which specifies maximum time frame syscall can live.
That's your current implementation. There is absolutely no reason whatsoever why this couldn't be changed.
It has nothing with implementation - it is logic. Something starts and it has its maximum lifetime, but not something starts and should be stopped Jan 1, 2008. In the latter case one can setup a timer, but it does not allow to specify maximum lifetime. If glibc posix sleeping functions converts relatinve AIO timeouts into absolute it does not mean all should do it. It is just not needed.
Quoted 14 lines
Quoted 5 lines
I created
kevent_signalnotifications - it allows user to setup any set of interested signals before call tokevent_get_events()and friends.No need to solve a problem with operation way when there is tactical and strategical ones
Of course there is a need and I explained it before. Getting signal notifications is in no way the same as changing the signal mask temporarily. You cannot correctly emulate the case where you want to block a signal while in the call as reenable it afterwards. Receiving the signal as an event and then artificially raising it is not the same.
Especially timing-wise, the signal kevent might not be seen long after
the syscall returns because other entries are worked on first.
Quoted 3 lines
The opposite case is equally impossible to emulate: unblocking a signal just for the duration of the syscall. These are all possible and used cases.
Add and remove appropriate kevent - it is as simple as call for one function.
Quoted 11 lines
Quoted 7 lines
Quoted 3 lines
- the
KEVENT_REQ_WAKEUP_ONEfunctionality is good and needed. But I would reverse the default. I cannot see many places where you want all threads to be woken. IntroduceKEVENT_REQ_WAKEUP_ALLinstead.I.e. to wake up only first thread always and in addon those threads which have specified flag set? Ok, will put into todo foer the next release.
It's a flag for an event. So the threads won't have the flag set. If an event is delivered with the flag set, wake all threads. Otherwise just one.
Ok.
Quoted 9 lines
Quoted 8 lines
Quoted 5 lines
- there is really no reason to invent yet another timer implementation. We have the POSIX timers which are feature rich and nicely implemented. All that is needed is to implement
SIGEV_KEVENTas a notification mechanism. The timer is registered as part of thetimer_create()syscalls.Feel free to add any interface you like - it is as simple as call for
kevent_user_add_ukevent()in userspace.
Quoted 6 lines
No, that's not what I mean. There is no need for the special timer-related part of your patch. Instead the existing POSIX timer syscalls should be modified to handle
SIGEV_KEVENTnotification. Again, keep the interface as small as possible. Plus, the POSIX timer interface is very flexible. You don't want to duplicate all that functionality.
Interface is already there with kevent_ctl(KEVENT_ADD), I just created additional entry, which describes timers enqueue/dequeue callbacks - I have not invented new interfaces, just reused existing generic kevent facilities. It is possible to add timer events from any other place.
Quoted 7 lines
Quoted 3 lines
And I almost silently stay behind with the fact that it is possbile to implement _all_ above ring buffer things in userspace with
kevent_get_events()and this functionality is there for almost a year :)Again, this defeats the purpose completely. The ring buffer is the faster interface, especially when coupled with asynchronous filling of ring buffer (i.e., without a syscal).
It is still possible to have very scalable system with it, for example with one thread dedicated for syscall reading (with big number of events transferred in one shot syscall overhead becomes negligible) and pool of working threads. It is not about 'let's remove kernelspace ring buffer management', but about possibilities and flexibility of the existing model.
Quoted 4 lines
Quoted 2 lines
Let's solve problem in order of theirs appearance - what do you think about above interface for ring buffer?
Looks better, yes.
Ok, I will implement this new (old) ring buffer in present it in the next release. I will also schedule there userspace notifications, 'wake-up-one-thread' flag changes and other small updates.
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 27 lines · On Mon, 20 Nov 2006 11:25:01 +0300 Evgeniy Polyakov <[email protected]> wrote:
Quoted 26 lines · Ulrich Drepper
Quoted 24 lines · Evgeniy Polyakov
Quoted 11 lines
Possible solution:
a) it would be possible to have a "used" flag in each ring buffer entry. That's too expensive, I guess.
b)
kevent_waitneeds another parameter which specifies the which is the last (i.e., least recently added) entry in the ring buffer. Everything between this entry and the current head (in ->kidx) is occupied. If multiple threads arrive inkevent_waitthe highest idx (with wrap around possibly lowest) is used.Quoted 8 lines
kevent_waitwill not try to move more entries into the ring buffer if ->kidx and the higest index passed in to anykevent_waitcall is equal (i.e., the ring buffer is full).There is one issue, though, and that is that a system call is needed to signal to the kernel that more entries in the ring buffer are processed and that they can be refilled. This goes against the kernel filling the ring buffer automatically (see below)
If thread calls
kevent_wait()it means it has processed previous entries, one can callkevent_wait()with $num parameter as zero, which means that thread does not want any new events, so nothing will be copied.
Quoted 6 lines
Quoted 3 lines
This doesn't solve the problem. You could only request new events when all previously reported events are processed. Plus: how do you report events if the you don't allow
get_eventpass them on?Userspace should itself maintain order and possibility to get event in this implementation, kernel just returns events which were requested.
That would mean that in a multithreaded application (or multi-processes sharing the same MAP_SHARED ringbuffer), all threads/processes will be slowed down to wait for the slowest one.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
They all already imeplemented. Just all above, and it was done several months ago already. No need to reinvent what is already there. Even if we will decide to remove
kevent_get_events()in favour of ring buffer-only implementation, winting-for-event syscall will be essentiallykevent_get_events()without pointer to the place where to put events.
Quoted 7 lines
Quoted 4 lines
Right, but this limitation of the interface is important. It means the interface of the kernel is smaller: fewer possibilities for problems and fewer constraints if in future something should be changed (and smaller kernel).
Ok, lets see for ring buffer implementation right now, and then we will decide if we want to remove or to stay with
kevent_get_events()syscall.
I agree that kevent_get_events() is duplicative and we shouldn't need it. Better to concentrate all our development effort on the single and most flexible means of delivery.
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 9 lines · Andrew Morton
Quoted 9 lines
Quoted 8 lines
Quoted 4 lines
If thread calls
kevent_wait()it means it has processed previous entries, one can callkevent_wait()with $num parameter as zero, which means that thread does not want any new events, so nothing will be copied.This doesn't solve the problem. You could only request new events when all previously reported events are processed. Plus: how do you report events if the you don't allow
get_eventpass them on?
Quoted 6 lines
Quoted 2 lines
Userspace should itself maintain order and possibility to get event in this implementation, kernel just returns events which were requested.
That would mean that in a multithreaded application (or multi-processes sharing the same
MAP_SHAREDringbuffer), all threads/processes will be slowed down to wait for the slowest one.
Not at all - all other threads can call kevent_get_events() with theirs own place in the ring buffer, so while one of them is processing an entry, others can fill next entries.
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 6 lines
They all already imeplemented. Just all above, and it was done several months ago already. No need to reinvent what is already there. Even if we will decide to remove
kevent_get_events()in favour of ring buffer-only implementation, winting-for-event syscall will be essentiallykevent_get_events()without pointer to the place where to put events.Right, but this limitation of the interface is important. It means the interface of the kernel is smaller: fewer possibilities for problems and fewer constraints if in future something should be changed (and smaller kernel).
Quoted 6 lines
Quoted 2 lines
Ok, lets see for ring buffer implementation right now, and then we will decide if we want to remove or to stay with
kevent_get_events()syscall.I agree that
kevent_get_events()is duplicative and we shouldn't need it. Better to concentrate all our development effort on the single and most flexible means of delivery.
Let's wait for ring buffer imeplementation first :)
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 13 lines · On Mon, 20 Nov 2006 11:51:59 +0300 Evgeniy Polyakov <[email protected]> wrote:
Quoted 12 lines · Andrew Morton
Quoted 11 lines
Quoted 8 lines
Quoted 4 lines
If thread calls
kevent_wait()it means it has processed previous entries, one can callkevent_wait()with $num parameter as zero, which means that thread does not want any new events, so nothing will be copied.This doesn't solve the problem. You could only request new events when all previously reported events are processed. Plus: how do you report events if the you don't allow
get_eventpass them on?Userspace should itself maintain order and possibility to get event in this implementation, kernel just returns events which were requested.
Quoted 7 lines
Quoted 3 lines
That would mean that in a multithreaded application (or multi-processes sharing the same
MAP_SHAREDringbuffer), all threads/processes will be slowed down to wait for the slowest one.Not at all - all other threads can call
kevent_get_events()with theirs own place in the ring buffer, so while one of them is processing an entry, others can fill next entries.
eh? That's not a ringbuffer, and it sounds awfully complex.
I don't know if this (new?) proposal resolves the events-gets-lost-due-to-thread-cancellation problem? Would need to see considerably more detail.
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 24 lines · Andrew Morton
Quoted 13 lines · On Mon, 20 Nov 2006 11:51:59 +0300 Evgeniy Polyakov <[email protected]> wrote:
Quoted 12 lines · Andrew Morton
Quoted 11 lines
Quoted 8 lines
Quoted 4 lines
If thread calls
kevent_wait()it means it has processed previous entries, one can callkevent_wait()with $num parameter as zero, which means that thread does not want any new events, so nothing will be copied.This doesn't solve the problem. You could only request new events when all previously reported events are processed. Plus: how do you report events if the you don't allow
get_eventpass them on?Userspace should itself maintain order and possibility to get event in this implementation, kernel just returns events which were requested.
Quoted 7 lines
Quoted 3 lines
That would mean that in a multithreaded application (or multi-processes sharing the same
MAP_SHAREDringbuffer), all threads/processes will be slowed down to wait for the slowest one.Not at all - all other threads can call
kevent_get_events()with theirs own place in the ring buffer, so while one of them is processing an entry, others can fill next entries.
Quoted 5 lines
eh? That's not a ringbuffer, and it sounds awfully complex.
I don't know if this (new?) proposal resolves the events-gets-lost-due-to-thread-cancellation problem? Would need to see considerably more detail.
It does - event is copied into shared buffer, but place (or index in the ring buffer) is selected by userspace (wrapper, glibc, anything). It is simple and (from my point of view) elegant, but it will not be used - I surrender and implement kenelspace ring buffer management right now, I just said that it is possible to implement any kind of ring buffer in userspace with old kevent_get_events() syscall only.
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovQuoted 2 lines · Evgeniy Polyakov
It is exactly how previous ring buffer (in mapped area though) was implemented.
Not any of those I saw. The one I looked at always started again at index 0 to fill the ring buffer. I'll wait for the next implementation.
Quoted 7 lines
Quoted 1 line
That's something the application should be make a call about. It's not always (or even mostly) the case that the ordering of the notification is important. Furthermore, this would also require the kernel to enforce an ordering. This is expensive on SMP machines. A locally generated event (i.e., source and the thread reporting the event) can be delivered faster than an event created on another CPU.
How come? If signal was delivered earlier than data arrived, userspace should get signal before data - that is the rule. Ordering is maintained not for event insertion, but for marking them ready - it is atomic, so who first starts to mark even ready, that event will be read first from the ready queue.
This is as far as the kernel is concerned. Queue them in the order they arrive.
I'm talking about the userlevel side. *If* (and it needs to be verified that this has an advantage) a CPU creates an event for, e.g., a read event and then a number of threads could be notified about the event. When the kernel has to wake up a thread it'll look whether any thread is scheduled on the same CPU which generated the event. Then the thread, upon waking up, can be told about the entry in the ring buffer which can be accessed first best (due to caching). This entry needs not be the first available in the ring buffer but that's a problem the userlevel code has to worry about.
Quoted 3 lines
Then I propose userspace notifications - each new thread can register 'wake me up when userspace event 1 is ready' and 'event 1' will be marked as ready by glibc when it removes the thread.
You don't want to have a channel like this. The userlevel code doesn't know which threads are waiting in the kernel on the event queue. And it seems to be much more complicated then simply have an kevent call which tells the kernel "wake up N or 1 more threads since I cannot handle it".
Basically a futex_wake()-like call.
Quoted 3 lines
Quoted 1 line
Of course it does. Just because you don't see a need for it for your applications right now it doesn't mean it's not a valid use.
Please explain why glibc AIO uses relatinve timeouts then :)
You are still completely focused on AIO. We are talking here about a new generic event handling. It is not tied to AIO. We will add all kinds of events, e.g., hopefully futex support and many others. And even for AIO it's relevant.
As I said, relative timeouts are unable to cope with settimeofday calls or ntp adjustments. AIO is certainly usable in situations where timeouts are related to wall clock time.
Quoted 3 lines
It has nothing with implementation - it is logic. Something starts and it has its maximum lifetime, but not something starts and should be stopped Jan 1, 2008.
It is an implementation detail. Look at the PI futex support. It has timeouts which can be cut short (or increased) due to wall clock changes.
Quoted 4 lines
Quoted 1 line
The opposite case is equally impossible to emulate: unblocking a signal just for the duration of the syscall. These are all possible and used cases.
Add and remove appropriate kevent - it is as simple as call for one function.
No, it's not. The kevent stuff handles only the kevent handler (i.e., the replacement for calling the signal handler). It cannot set signal masks. I am talking about signal masks here. And don't suggest "I can add another kevent feature where I can register signal masks". This would be ridiculous since it's not an event source. Just add the parameter and every base is covered and, at least equally important, we have symmetry between the event handling interfaces.
Quoted 4 lines
Quoted 1 line
No, that's not what I mean. There is no need for the special timer-related part of your patch. Instead the existing POSIX timer syscalls should be modified to handle
SIGEV_KEVENTnotification. Again, keep the interface as small as possible. Plus, the POSIX timer interface is very flexible. You don't want to duplicate all that functionality.Interface is already there with
kevent_ctl(KEVENT_ADD), I just created additional entry, which describes timers enqueue/dequeue callbacks
New multiplexers cases are additional syscalls. This is unnecessary code. Increased kernel interface and such. We have the POSIX timer interfaces which are feature-rich and standardized *and* can be triviall extended (at least from the userlevel interface POV) to use event queues. If you don't want to do this, fine, I'll try to get it made. But drop the timer part of your patches.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 5 lines · Ulrich Drepper
Quoted 2 lines · Evgeniy Polyakov
It is exactly how previous ring buffer (in mapped area though) was implemented.
Not any of those I saw. The one I looked at always started again at index 0 to fill the ring buffer. I'll wait for the next implementation.
I like the two-pointer ring buffer approach, one pointer for the consumer and one for the producer.
Quoted 1 line
You don't want to have a channel like this. The userlevel code doesn't know which threads are waiting in the kernel on the event queue. And it
Agreed.
Quoted 1 line
You are still completely focused on AIO. We are talking here about a new generic event handling. It is not tied to AIO. We will add all
Agreed.
Quoted 1 line
As I said, relative timeouts are unable to cope with settimeofday calls or ntp adjustments. AIO is certainly usable in situations where timeouts are related to wall clock time.
I think we have lived with relative timeouts for so long, it would be unusual to change now. select(2), poll(2), epoll_wait(2) all take relative timeouts.
Jeff
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 1 line · Jeff Garzik
I think we have lived with relative timeouts for so long, it would be unusual to change now. select(2), poll(2),
epoll_wait(2)all take relative timeouts.
I'm not talking about always using absolute timeouts.
I'm saying the timeout parameter should be a struct timespec* and then the flags word could have a flag meaning "this is an absolute timeout".
I.e., enable both uses,, even make relative timeouts the default.
This is what the modern POSIX interfaces do, too, see clock_nanosleep.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Hi,
Quoted 11 lines · Ulrich Drepper
Quoted 3 lines · Jeff Garzik
I think we have lived with relative timeouts for so long, it would be unusual to change now. select(2), poll(2),
epoll_wait(2)all take relative timeouts.I'm not talking about always using absolute timeouts.
I'm saying the timeout parameter should be a struct timespec* and then the flags word could have a flag meaning "this is an absolute timeout".
I.e., enable both uses,, even make relative timeouts the default.
This is what the modern POSIX interfaces do, too, seeclock_nanosleep.
I agree here. And while you are at it: Have it say "not before" vs. "not after".
<rant>
And if you call "absolute timeout" an "alarm" or "deadline" everyone will agree, that this is useful.
Timeout means "I ran OUT of TIME to do it" and this is by definition relative to a starting point. A "deadline" is an absolute point in (wall) time where sth. has to be ready and an "alarm" is an absolute point in (wall) time where sth. is triggered (e.g. a bell rings on your "ALARM clock").
I don't know which person established that non-sense nomenclature about relative and absolute timouts.
</rant>
Regards
Ingo Oeser
Re: [take24 0/6] kevent: Generic event handling mechanism.
In reply to Ulrich DrepperQuoted 11 lines · Ulrich Drepper
Quoted 3 lines · Jeff Garzik
I think we have lived with relative timeouts for so long, it would be unusual to change now. select(2), poll(2),
epoll_wait(2)all take relative timeouts.I'm not talking about always using absolute timeouts.
I'm saying the timeout parameter should be a struct timespec* and then the flags word could have a flag meaning "this is an absolute timeout".
I.e., enable both uses,, even make relative timeouts the default. This
is what the modern POSIX interfaces do, too, seeclock_nanosleep.
Can't the argument be something like u64 instead of struct timespec, regardless of this discussion (relative vs absolute)?
Compare:
void mysleep(int msec) {
struct timeval tv;
tv.tv_sec = msec/1000;
tv.tv_usec = msec%1000;
select(0,0,0,0,&tv);
}
with
void mysleep(int msec) {
poll(0, 0, msec*SOME_TIME_SCALE_VALUE);
}That to say: struct time{spec,val,whatever} is more difficult to use than plain numbers.
But yes... existing struct timespec has an advantage of being already existed. Oh well.
/mjt
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 8 lines · Michael Tokarev
Quoted 7 lines · Ulrich Drepper
Quoted 3 lines · Jeff Garzik
I think we have lived with relative timeouts for so long, it would be unusual to change now. select(2), poll(2),
epoll_wait(2)all take relative timeouts.I'm not talking about always using absolute timeouts.
Quoted 8 lines
Quoted 4 lines
I'm saying the timeout parameter should be a struct timespec* and then the flags word could have a flag meaning "this is an absolute timeout".
I.e., enable both uses,, even make relative timeouts the default. This
is what the modern POSIX interfaces do, too, seeclock_nanosleep.Can't the argument be something like u64 instead of struct timespec, regardless of this discussion (relative vs absolute)?
It is right now :)
Quoted 1 line
/mjt
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
In reply to Michael TokarevQuoted 2 lines · Michael Tokarev
Can't the argument be something like u64 instead of struct timespec, regardless of this discussion (relative vs absolute)?
Newer syscalls (ppoll, pselect) take struct timespec, which is a reasonable, modern form of the timeout argument...
Jeff
Re: [take24 0/6] kevent: Generic event handling mechanism.
In reply to Ulrich DrepperQuoted 6 lines · Ulrich Drepper
Quoted 2 lines · Evgeniy Polyakov
It is exactly how previous ring buffer (in mapped area though) was implemented.
Not any of those I saw. The one I looked at always started again at index 0 to fill the ring buffer. I'll wait for the next implementation.
That what I'm talking about - there are at least 4 (!) different ring buffer implementations, most of them were not even looked at. But new version is ready, I will complete testing stage and will relese 'take25' soon today.
For those who like 'real-world benchmark and so on' I created a patch for the latest stable lighttpd version and test it with kevent.
Quoted 16 lines
Quoted 7 lines
Quoted 6 lines
That's something the application should be make a call about. It's not always (or even mostly) the case that the ordering of the notification is important. Furthermore, this would also require the kernel to enforce an ordering. This is expensive on SMP machines. A locally generated event (i.e., source and the thread reporting the event) can be delivered faster than an event created on another CPU.
Quoted 5 lines
How come? If signal was delivered earlier than data arrived, userspace should get signal before data - that is the rule. Ordering is maintained not for event insertion, but for marking them ready - it is atomic, so who first starts to mark even ready, that event will be read first from the ready queue.
This is as far as the kernel is concerned. Queue them in the order they arrive.
Quoted 9 lines
I'm talking about the userlevel side. *If* (and it needs to be verified that this has an advantage) a CPU creates an event for, e.g., a read event and then a number of threads could be notified about the event. When the kernel has to wake up a thread it'll look whether any thread is scheduled on the same CPU which generated the event. Then the thread, upon waking up, can be told about the entry in the ring buffer which can be accessed first best (due to caching). This entry needs not be the first available in the ring buffer but that's a problem the userlevel code has to worry about.
Ok, I've understood.
Quoted 9 lines
Quoted 3 lines
Then I propose userspace notifications - each new thread can register 'wake me up when userspace event 1 is ready' and 'event 1' will be marked as ready by glibc when it removes the thread.
You don't want to have a channel like this. The userlevel code doesn't know which threads are waiting in the kernel on the event queue. And it seems to be much more complicated then simply have an kevent call which tells the kernel "wake up N or 1 more threads since I cannot handle it".
Basically a futex_wake()-like call.
Kernel does not know about any threads which waits for events, it only has queue of events, it can only wake those who was parked in kevent_get_events() or kevent_wait(), but syscall will return only when condition it waits on is true, i.e. when there is new event in the ready queue and/or ring buffer has empty slots, but kernel will wake them up in any case if those conditions are true.
How should it know which syscall should be interrupted when special syscall is called?
Quoted 5 lines
Quoted 4 lines
Quoted 2 lines
Of course it does. Just because you don't see a need for it for your applications right now it doesn't mean it's not a valid use.
Please explain why glibc AIO uses relatinve timeouts then :)
Quoted 8 lines
You are still completely focused on AIO. We are talking here about a new generic event handling. It is not tied to AIO. We will add all kinds of events, e.g., hopefully futex support and many others. And even for AIO it's relevant.
As I said, relative timeouts are unable to cope with settimeofday calls or ntp adjustments. AIO is certainly usable in situations where timeouts are related to wall clock time.
No AIO, but syscall.
Only syscall time matters.
Syscall starts, it sould be sometime stopped. When it should be stopped? It should be stopped after some time after it was started!
I still do not understand how will you use absolute timeout values there. Please exaplain.
Quoted 6 lines
Quoted 3 lines
It has nothing with implementation - it is logic. Something starts and it has its maximum lifetime, but not something starts and should be stopped Jan 1, 2008.
It is an implementation detail. Look at the PI futex support. It has timeouts which can be cut short (or increased) due to wall clock changes.
futex_wait() uses relative timeouts:
static int futex_wait(u32 __user *uaddr, u32 val, unsigned long time)
Kernel use relative timeouts.
Only special syscalls, which work with absolute time, have absolute timeouts (like settimeofday).
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
The opposite case is equally impossible to emulate: unblocking a signal just for the duration of the syscall. These are all possible and used cases.
Add and remove appropriate kevent - it is as simple as call for one function.
Quoted 7 lines
No, it's not. The kevent stuff handles only the kevent handler (i.e., the replacement for calling the signal handler). It cannot set signal masks. I am talking about signal masks here. And don't suggest "I can add another kevent feature where I can register signal masks". This would be ridiculous since it's not an event source. Just add the parameter and every base is covered and, at least equally important, we have symmetry between the event handling interfaces.
We have not have such symmetry.
Other event handling interfaces can not work with events, which do not have file descriptor behind them. Kevent can and works. Signals are just usual events.
You request to get events - and you get them.
You request to not get events during syscall - you remove events.
Btw, please point me to the discussion about real life usefullness of that parameter for epoll. I read thread where sys_pepoll() was intruduced, but except some theoretical handwaving about possible usefullness there are no real signs of that requirement.
What is the ground research or extended explaination about blocking/unblocking some signals during syscall execution?
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
No, that's not what I mean. There is no need for the special timer-related part of your patch. Instead the existing POSIX timer syscalls should be modified to handle
SIGEV_KEVENTnotification. Again, keep the interface as small as possible. Plus, the POSIX timer interface is very flexible. You don't want to duplicate all that functionality.Interface is already there with
kevent_ctl(KEVENT_ADD), I just created additional entry, which describes timers enqueue/dequeue callbacks
Quoted 6 lines
New multiplexers cases are additional syscalls. This is unnecessary code. Increased kernel interface and such. We have the POSIX timer interfaces which are feature-rich and standardized *and* can be triviall extended (at least from the userlevel interface POV) to use event queues. If you don't want to do this, fine, I'll try to get it made. But drop the timer part of your patches.
There are _no_ additional syscalls.
I just introduced new case for event type.
You _need_ it to be done, since any kernel kevent user must have enqueue/dequeue/callback callbacks. It is just an implementation of that callbacks.
I made the work, one can create any interfaces (additional syscalls or anything else) on top of that.
Due to the fact that kevent was designed as generic event handling mechanism it is possible to work will all types of events using the same interface, which was created 10 month ago: kevent add, remove and so on... There is nothing special for timers there - it is separate file which does _not_ have any interfaces accessible outside kevent core (i.e. syscalls or exported symbols).
Btw, how POSIX API should be extended to allow to queue events - queue is required (which is created when user calls kevent_init() or previoisly opened /dev/kevent), how should it be accessed, since it is just a file descriptor in process task_struct.
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 10 lines · Evgeniy Polyakov
Quoted 2 lines
You don't want to have a channel like this. The userlevel code doesn't know which threads are waiting in the kernel on the event queue. And it seems to be much more complicated then simply have an kevent call which tells the kernel "wake up N or 1 more threads since I cannot handle it".
Basically a futex_wake()-like call.Kernel does not know about any threads which waits for events, it only has queue of events, it can only wake those who was parked in
kevent_get_events()orkevent_wait(), but syscall will return only when condition it waits on is true, i.e. when there is new event in the ready queue and/or ring buffer has empty slots, but kernel will wake them up in any case if those conditions are true.
Quoted 2 lines
How should it know which syscall should be interrupted when special syscall is called?
It's not about interrupting any threads.
The issue is that the wakeup of a thread from the kevent_wait call constitutes an "event notification". If, as it should be, only one thread is woken than this information mustn't get lost. If the woken thread cannot work on the events it got notified for, then it must tell the kernel about it so that, *if* there are other threads waiting in kevent_wait, one of those other threads can be woken.
What is needed is a simple "wake another thread waiting on this event queue" syscall. Yes, in theory we could open an additional pipe with each event queue and use it for waking threads, but this is influencing the ABI through the use of a file descriptor. It's much better to have an explicit way to do this.
Quoted 7 lines
No AIO, but syscall.
Only syscall time matters.
Syscall starts, it sould be sometime stopped. When it should be stopped? It should be stopped after some time after it was started!I still do not understand how will you use absolute timeout values there. Please exaplain.
What is there to explain? If you are waiting for events which must coincide with real-world events you'll naturally will want to formulate something like "wait for X until 10:15h". You cannot formulate this correctly with relative timeouts since the realtime clock might be adjusted.
Quoted 4 lines
futex_wait()uses relative timeouts:
static intfutex_wait(u32 __user *uaddr, u32 val, unsigned long time)Kernel use relative timeouts.
Look again. This time at the implementation. For FUTEX_LOCK_PI the timeout is an absolute timeout.
Quoted 7 lines
We have not have such symmetry.
Other event handling interfaces can not work with events, which do not have file descriptor behind them. Kevent can and works. Signals are just usual events.You request to get events - and you get them.
You request to not get events during syscall - you remove events.
None of this matches what I'm talking about. If you want to block a signal for the duration of the kevent_wait call this is nothing you can do by registering an event.
Registering events has nothing to do with signal masks. They are not modified. It is the program's responsibility to set the mask up correctly. Just like sigwaitinfo() etc expect all signals which are waited on to be blocked.
The signal mask handling is orthogonal to all this and must be explicit.
In some cases explicit pthread_sigmask/sigprocmask calls. But this is
not atomic if a signal must be masked/unmasked for the *_wait call. This is why we have variants like pselect/ppoll/epoll_pwait which explicitly and *atomically* change the signal mask for the duration of the call.
Quoted 4 lines
Btw, please point me to the discussion about real life usefullness of that parameter for epoll. I read thread where
sys_pepoll()was intruduced, but except some theoretical handwaving about possible usefullness there are no real signs of that requirement.
Don't search for epoll_pwait, it's not widely used yet. Search for pselect, which is standardized. You'll find plenty of uses of that interface. The number is certainly depressed in the moment since until recently there was no correct implementation on Linux. And the interface is mostly used in real-time contexts where signals are more commonly used.
Quoted 2 lines
What is the ground research or extended explaination about blocking/unblocking some signals during syscall execution?
Why is this even a question? Have you done programming with signals? You hatred of signals makes me think this isn't the case.
You might want to unblock a signal on a *_wait call if it can be used to interrupt the wait but you don't want this to happen during when the thread is working on a request.
You might want to block a signal, for instance, around a sigwaitinfo call or, in this case, a kevent_wait call where the signal might be delivered to the queue.
There are countless possibilities. Signals are very flexible.
Quoted 2 lines
There are _no_ additional syscalls.
I just introduced new case for event type.
Which is a new syscall. All demultiplexer cases are no syscalls. Which, BTW, implies that unrecognized types should actually cause a ENOSYS return value (this affects kevent_break). We've been over this many times. If EINVAL is return this case cannot be distinguished from invalid parameters. This is crucial for future extensions where userland (esp glibc) needs to be able to determine whether a new feature is supported on the system.
Quoted 3 lines
You _need_ it to be done, since any kernel kevent user must have enqueue/dequeue/callback callbacks. It is just an implementation of that callbacks.
I don't question that. But there is no need to add the callback. It extends the kernel ABI/API. And for what? A vastly inferior timer implementation compared to the POSIX timers. And this while all that needs to be done is to extend the POSIX timer code slightly to handle SIGEV_KEVENT in addition to the other notification methods currently used. If you do it right then the code can be shared with the file AIO code which currently is circulated as well and which uses parts of the POSIX timer infrastructure.
Quoted 4 lines
Btw, how POSIX API should be extended to allow to queue events - queue is required (which is created when user calls
kevent_init()or previoisly opened/dev/kevent), how should it be accessed, since it is just a file descriptor in processtask_struct.
I've explained this multiple times. The struct sigevent structure needs to be extended to get a new part in the union. Something like
struct {
int kevent_fd;
void *data;
} _sigev_kevent;Then define SIGEV_KEVENT as a value distinct from the other SIGEV_ values. In the code which handles setup of timers (the timer_create syscall), recognize SIGEV_KEVENT and handle it appropriately. I.e., call into the code to register the event source, just like you'd do with the current interface. Then add the code to post an event to the event queue where currently signals would be sent et voilà.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 26 lines · Ulrich Drepper
Quoted 13 lines · Evgeniy Polyakov
Quoted 5 lines
You don't want to have a channel like this. The userlevel code doesn't know which threads are waiting in the kernel on the event queue. And it seems to be much more complicated then simply have an kevent call which tells the kernel "wake up N or 1 more threads since I cannot handle it". Basically a futex_wake()-like call.
Kernel does not know about any threads which waits for events, it only has queue of events, it can only wake those who was parked in
kevent_get_events()orkevent_wait(), but syscall will return only when condition it waits on is true, i.e. when there is new event in the ready queue and/or ring buffer has empty slots, but kernel will wake them up in any case if those conditions are true.Quoted 2 lines
How should it know which syscall should be interrupted when special syscall is called?
It's not about interrupting any threads.
The issue is that the wakeup of a thread from the
kevent_waitcall constitutes an "event notification". If, as it should be, only one thread is woken than this information mustn't get lost. If the woken thread cannot work on the events it got notified for, then it must tell the kernel about it so that, *if* there are other threads waiting inkevent_wait, one of those other threads can be woken.
Quoted 5 lines
What is needed is a simple "wake another thread waiting on this event queue" syscall. Yes, in theory we could open an additional pipe with each event queue and use it for waking threads, but this is influencing the ABI through the use of a file descriptor. It's much better to have an explicit way to do this.
Threads are parked in syscalls - which one should be interrupted? And what if there were no threads waiting in syscalls?
Quoted 5 lines
Quoted 5 lines
No AIO, but syscall.
Only syscall time matters.
Syscall starts, it sould be sometime stopped. When it should be stopped? It should be stopped after some time after it was started!
Quoted 7 lines
Quoted 2 lines
I still do not understand how will you use absolute timeout values there. Please exaplain.
What is there to explain? If you are waiting for events which must coincide with real-world events you'll naturally will want to formulate something like "wait for X until 10:15h". You cannot formulate this correctly with relative timeouts since the realtime clock might be adjusted.
It has completely nothing with syscall.
You register a timer to wait until 10:15 that is all.
You do not ask to sleep in read() until some time, because read() has nothing in common with that time and event.
But actually it becomes stupid discussion, don't you think? What do you think about putting there timespec and a small warning in dmesg about absolute timeout? When someone will report it I will publically say that you were right and it is correct to have possibility to have absolute timeouts for syscalls? :)
Quoted 7 lines
Quoted 4 lines
futex_wait()uses relative timeouts:
static intfutex_wait(u32 __user *uaddr, u32 val, unsigned long time)Kernel use relative timeouts.
Look again. This time at the implementation. For
FUTEX_LOCK_PIthe timeout is an absolute timeout.
How come? It just uses timespec.
Quoted 17 lines
Quoted 7 lines
We have not have such symmetry.
Other event handling interfaces can not work with events, which do not have file descriptor behind them. Kevent can and works. Signals are just usual events.You request to get events - and you get them.
You request to not get events during syscall - you remove events.None of this matches what I'm talking about. If you want to block a signal for the duration of the
kevent_waitcall this is nothing you can do by registering an event.Registering events has nothing to do with signal masks. They are not modified. It is the program's responsibility to set the mask up correctly. Just like
sigwaitinfo()etc expect all signals which are waited on to be blocked.
Quoted 6 lines
The signal mask handling is orthogonal to all this and must be explicit.
In some cases explicit pthread_sigmask/sigprocmask calls. But this is
not atomic if a signal must be masked/unmasked for the*_waitcall. This is why we have variants like pselect/ppoll/epoll_pwait which explicitly and *atomically* change the signal mask for the duration of the call.
You probably missed kevent signal patch - signal will not be delivered (in special cases) since it will not be copied into signal mask. System just will not know that it happend. Completely. Like putting it into blocked mask.
Quoted 11 lines
Quoted 4 lines
Btw, please point me to the discussion about real life usefullness of that parameter for epoll. I read thread where
sys_pepoll()was intruduced, but except some theoretical handwaving about possible usefullness there are no real signs of that requirement.Don't search for
epoll_pwait, it's not widely used yet. Search for pselect, which is standardized. You'll find plenty of uses of that interface. The number is certainly depressed in the moment since until recently there was no correct implementation on Linux. And the interface is mostly used in real-time contexts where signals are more commonly used.
I found this:
... document a pselect() call intended to
remove the race condition that is present when one wants to wait on either a signal or some file descriptor. (See also Stevens, Unix Network Programming, Volume 1, 2nd Ed., 1998, p. 168 and the pselect.2 man page released today.) Glibc 2.0 has a bad version (wrong number of parameters) and glibc 2.1 a better version, but the whole purpose of pselect is to avoid the race, and glibc cannot do that, one needs kernel support.
But it is completely irrelevant with kevent signals - there is no race for that case when signal is delivered through file descriptor.
Quoted 5 lines
Quoted 2 lines
What is the ground research or extended explaination about blocking/unblocking some signals during syscall execution?
Why is this even a question? Have you done programming with signals? You hatred of signals makes me think this isn't the case.
It is much better to not know how thing works, then to not be possible to understand how new things can work.
Quoted 3 lines
You might want to unblock a signal on a
*_waitcall if it can be used to interrupt the wait but you don't want this to happen during when the thread is working on a request.
Add kevent signal and do not process that event.
Quoted 3 lines
You might want to block a signal, for instance, around a sigwaitinfo call or, in this case, a
kevent_waitcall where the signal might be delivered to the queue.
Having special type of kevent signal is the same as putting signal into blocked mask, but signal event will be marked as ready - to indicate that condition was there.
There will not be any race in that case.
Quoted 1 line
There are countless possibilities. Signals are very flexible.
That is why we want to get them through synchronous queue? :)
Quoted 4 lines
Quoted 2 lines
There are _no_ additional syscalls.
I just introduced new case for event type.Which is a new syscall. All demultiplexer cases are no syscalls.
I think I am a bit blind, probably parts of Leonids are still getting into my brain, but there is one syscall called kevent_ctl() which adds different events, including timer, signal, socket and others.
Quoted 6 lines
Which, BTW, implies that unrecognized types should actually cause a ENOSYS return value (this affects
kevent_break). We've been over this many times. If EINVAL is return this case cannot be distinguished from invalid parameters. This is crucial for future extensions where userland (esp glibc) needs to be able to determine whether a new feature is supported on the system.
I can replace with -ENOSYS if you like.
Quoted 5 lines
Quoted 3 lines
You _need_ it to be done, since any kernel kevent user must have enqueue/dequeue/callback callbacks. It is just an implementation of that callbacks.
I don't question that. But there is no need to add the callback. It
No one asked and pain me to create kevent, but it is done. Probably no the way some people wanted, but it always happend, it is really not that bad.
Kevent subsystem operates with structures which can be added into completely different objects in the system - inodes, files - anything. And to say to that object about new events there are special callbacks - enqueue and dequeue. Callback which has extremely unusual name 'callback' is invoked when object, where event is linked, has something to report - new data, fired alarm or anything else, so it calls kevent's ->callback and if return value is positive, kevent is marked as ready. It allows to have event with different sets of interests for the same type of the main object - for example socket can have read and write callbacks.
So you must have them.
As you probably saw, kevent_timer_callback() just returns 1.
Quoted 7 lines
extends the kernel ABI/API. And for what? A vastly inferior timer implementation compared to the POSIX timers. And this while all that needs to be done is to extend the POSIX timer code slightly to handle
SIGEV_KEVENTin addition to the other notification methods currently used. If you do it right then the code can be shared with the file AIO code which currently is circulated as well and which uses parts of the POSIX timer infrastructure.
Ulrich, tell me the truth, will you kill me if I say that I have an entry in TODO to implement different AIO design (details for interested readers can be found in my blog), and then present it to community? :))
Quoted 13 lines
Quoted 4 lines
Btw, how POSIX API should be extended to allow to queue events - queue is required (which is created when user calls
kevent_init()or previoisly opened/dev/kevent), how should it be accessed, since it is just a file descriptor in processtask_struct.I've explained this multiple times. The struct sigevent structure needs to be extended to get a new part in the union. Something like
struct { int kevent_fd; void *data; } _sigev_kevent;
Quoted 6 lines
Then define
SIGEV_KEVENTas a value distinct from the otherSIGEV_values. In the code which handles setup of timers (thetimer_createsyscall), recognizeSIGEV_KEVENTand handle it appropriately. I.e., call into the code to register the event source, just like you'd do with the current interface. Then add the code to post an event to the event queue where currently signals would be sent et voilà.
Ok, I see.
It is doable and simple.
I will try to implement it tomorrow.
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 15 lines · Evgeniy Polyakov
Quoted 8 lines
I've explained this multiple times. The struct sigevent structure needs to be extended to get a new part in the union. Something like
struct { int kevent_fd; void *data; } _sigev_kevent;Quoted 6 lines
Then define
SIGEV_KEVENTas a value distinct from the otherSIGEV_values. In the code which handles setup of timers (thetimer_createsyscall), recognizeSIGEV_KEVENTand handle it appropriately. I.e., call into the code to register the event source, just like you'd do with the current interface. Then add the code to post an event to the event queue where currently signals would be sent et voilà.
Quoted 3 lines
Ok, I see.
It is doable and simple.
I will try to implement it tomorrow.
I've checked the code.
Since it will be a union, it is impossible to use _sigev_thread and it becomes just SIGEV_SIGNAL case with different delivery mechanism. Is it what you want?
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
nitpick: in ring_buffer.c (example app), I would use posix_memalign(3) rather than malloc(3)
Jeff
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 2 lines · Jeff Garzik
nitpick: in
ring_buffer.c(example app), I would useposix_memalign(3)rather than malloc(3)
Yes, it can be done.
Quoted 1 line
Jeff
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovAnother: pass a 'flags' argument to kevent_init(2). I guarantee you will need it eventually. It IMO would help with later binary compatibility, if nothing else. You wouldn't need a new syscall to introduce struct kevent_ring_v2.
Jeff
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 4 lines · Jeff Garzik
Another: pass a 'flags' argument to
kevent_init(2). I guarantee you will need it eventually. It IMO would help with later binary compatibility, if nothing else. You wouldn't need a new syscall to introduce structkevent_ring_v2.
Yep, I will add there 'flags' field.
Quoted 1 line
Jeff
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovQuoted 4 lines · Evgeniy Polyakov
I've checked the code.
Since it will be a union, it is impossible to use_sigev_threadand it becomes justSIGEV_SIGNALcase with different delivery mechanism. Is it what you want?
struct sigevent is defined like this:
typedef struct sigevent {
sigval_t sigev_value;
int sigev_signo;
int sigev_notify;
union {
int _pad[SIGEV_PAD_SIZE];
int _tid;
struct {
void (*_function)(sigval_t);
void *_attribute; /* really pthread_attr_t */
} _sigev_thread;
} _sigev_un;
} sigevent_t;For the SIGEV_KEVENT case:
sigev_notify is set to SIGEV_KEVENT (obviously)
sigev_value can be used for the void* data passed along with the signal, just like in the case of a signal delivery
Now you need a way to specify the kevent descriptor. Just add
int _kevent;
inside the union and if you want
#define sigev_kevent_descr _sigev_un._kevent
That should be all.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 33 lines · Ulrich Drepper
Quoted 4 lines · Evgeniy Polyakov
I've checked the code.
Since it will be a union, it is impossible to use_sigev_threadand it becomes justSIGEV_SIGNALcase with different delivery mechanism. Is it what you want?struct sigevent is defined like this:
typedef struct sigevent { sigval_t sigev_value; int sigev_signo; int sigev_notify; union { int _pad[SIGEV_PAD_SIZE]; int _tid; struct { void (*_function)(sigval_t); void *_attribute; /* really pthread_attr_t */ } _sigev_thread; } _sigev_un; } sigevent_t;For the
SIGEV_KEVENTcase:
sigev_notifyis set toSIGEV_KEVENT(obviously)
sigev_valuecan be used for the void* data passed along with the signal, just like in the case of a signal deliveryNow you need a way to specify the kevent descriptor. Just add
Quoted 7 lines
int
_kevent;inside the union and if you want
#define sigev_kevent_descr _sigev_un._keventThat should be all.
That what I implemented.
But in this case it will be impossible to have SIGEV_THREAD and SIGEV_KEVENT at the same time, it will be just the same as SIGEV_SIGNAL but with different delivery mechanism. Is is what you expect for that?
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 3 lines · Evgeniy Polyakov
But in this case it will be impossible to have
SIGEV_THREADandSIGEV_KEVENTat the same time, it will be just the same asSIGEV_SIGNALbut with different delivery mechanism. Is is what you expect for that?
Yes, that's expected. The event if for the queue, not directed to a specific thread.
If in future we want to think about preferably waking a specific thread we can then think about it. But I doubt that'll be beneficial. The thread specific part in the signal handling is only used to implement the SIGEV_THREAD notification.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 6 lines · Ulrich Drepper
Quoted 4 lines · Evgeniy Polyakov
But in this case it will be impossible to have
SIGEV_THREADandSIGEV_KEVENT
at the same time, it will be just the same asSIGEV_SIGNALbut with different delivery mechanism. Is is what you expect for that?
Quoted 7 lines
Yes, that's expected. The event if for the queue, not directed to a specific thread.
If in future we want to think about preferably waking a specific thread we can then think about it. But I doubt that'll be beneficial. The thread specific part in the signal handling is only used to implement the
SIGEV_THREADnotification.
Ok, so please review patch I sent, if it is ok from design point of view, I will run some tests here.
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Kevent POSIX timers support.
In reply to Evgeniy PolyakovQuoted 4 lines · Evgeniy Polyakov
That what I implemented.
But in this case it will be impossible to haveSIGEV_THREADandSIGEV_KEVENTat the same time, it will be just the same asSIGEV_SIGNALbut with different delivery mechanism. Is is what you expect for that?
Something like this morning hack (compile tested only).
If my thoughts are correct, I will create some simple application and test if it works.
include/linux/posix-timers.h +4 -0
@@ -4,6 +4,7 @@#include <linux/spinlock.h> #include <linux/list.h> #include <linux/sched.h> +#include <linux/kevent_storage.h> union cpu_time_count {cputime_t cpu;
@@ -49,6 +50,9 @@ struct k_itimer {sigval_t it_sigev_value;/* value word of sigevent struct */ struct task_struct *it_process;/* process to send signal to */ struct sigqueue *sigq;/* signal queue entry. */ +#ifdef CONFIG_KEVENT_TIMER +struct kevent_storage st; +#endif union { struct { struct hrtimer timer;
kernel/posix-timers.c +164 -48
257 lines not on this page · raw message
Signature · 1 line
Evgeniy Polyakov
Re: Kevent POSIX timers support.
kernel/posix-timers.c static int posix_ktime_get_ts(clockid_t · line 250
Quoted 6 lines · Evgeniy Polyakov
kernel/posix-timers.c +6
-0+static int posix_kevent_init (void ) +{ +struct kevent_callbacks tc = { + .callback = &posix_kevent_callback, + .enqueue = &posix_kevent_enqueue, + .dequeue = &posix_kevent_dequeue};
How do we prevent that somebody tries to register a POSIX timer event source with kevent_ctl(KEVENT_CTL_ADD)? This should only be possible from sys_timer_create and nowhere else.
Can you add a parameter to kevent_enqueue indicating this is a call from inside the kernel and then ignore certain enqueue callbacks?
kernel/posix-timers.c static int posix_timer_fn(struct hrtimer · line 445
Quoted 8 lines
kernel/posix-timers.c +4
-0@@ -343,23 +439,27 @@ static int posix_timer_fn(struct hrtimer timr = container_of(timer, struct k_itimer, it.real.timer);spin_lock_irqsave (&timr->it_lock, flags); + +if (timr->it_sigev_notify &SIGEV_KEVENT ) { +kevent_storage_ready (&timr->st,NULL ,KEVENT_MASK_ALL ); + }else {
We need to pass the data in the sigev_value meember of the struct sigevent structure passed to timer_create to the caller. I don't see it being done here nor when the timer is created. Do I miss something? The sigev_value value should be stored in the user/ptr member of struct ukevent.
kernel/posix-timers.c sys_timer_create(const clockid_t which_c · line 603
Quoted 1 line
kernel/posix-timers.c +1
-0+if (event.sigev_notify &SIGEV_KEVENT ) {
Don't use a bit. It makes no sense to combine SIGEV_SIGNAL with SIGEV_KEVENT etc. Only SIGEV_THREAD_ID is a special case.
Just define SIGEV_KEVENT to 3 and replace the tests like the one cited above with
if (timr->it_sigev_notify == SIGEV_KEVENT)
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: Kevent POSIX timers support.
Quoted 12 lines · Ulrich Drepper
kernel/posix-timers.c static int posix_ktime_get_ts(clockid_t · line 250
Quoted 6 lines · Evgeniy Polyakov
kernel/posix-timers.c +6
-0+static int posix_kevent_init (void ) +{ +struct kevent_callbacks tc = { + .callback = &posix_kevent_callback, + .enqueue = &posix_kevent_enqueue, + .dequeue = &posix_kevent_dequeue};How do we prevent that somebody tries to register a POSIX timer event source with
kevent_ctl(KEVENT_CTL_ADD)? This should only be possible fromsys_timer_createand nowhere else.
Quoted 2 lines
Can you add a parameter to
kevent_enqueueindicating this is a call from inside the kernel and then ignore certain enqueue callbacks?
I think we need some set of flags for callbacks - where they can be called, maybe even from which context and so on. So userspace will not be allowed to create such timers through kevent API.
Will do it for release.
Quoted 14 lines
kernel/posix-timers.c static int posix_timer_fn(struct hrtimer · line 445
Quoted 8 lines
kernel/posix-timers.c +4
-0@@ -343,23 +439,27 @@ static int posix_timer_fn(struct hrtimer timr = container_of(timer, struct k_itimer, it.real.timer);spin_lock_irqsave (&timr->it_lock, flags); + +if (timr->it_sigev_notify &SIGEV_KEVENT ) { +kevent_storage_ready (&timr->st,NULL ,KEVENT_MASK_ALL ); + }else {We need to pass the data in the
sigev_valuemeember of the struct sigevent structure passed totimer_createto the caller. I don't see it being done here nor when the timer is created. Do I miss something? Thesigev_valuevalue should be stored in the user/ptr member of struct ukevent.
sigev_value was stored in k_itimer structure, I just do not know where to put it in the ukevent provided to userspace - it can be placed in pointer value if you like.
Quoted 9 lines
kernel/posix-timers.c sys_timer_create(const clockid_t which_c · line 603
Quoted 1 line
kernel/posix-timers.c +1
-0+if (event.sigev_notify &SIGEV_KEVENT ) {Don't use a bit. It makes no sense to combine
SIGEV_SIGNALwithSIGEV_KEVENTetc. OnlySIGEV_THREAD_IDis a special case.Just define
SIGEV_KEVENTto 3 and replace the tests like the one cited above withif (
timr->it_sigev_notify==SIGEV_KEVENT)
Ok.
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Re: Kevent POSIX timers support.
Quoted 5 lines · Evgeniy Polyakov
Quoted 1 line
We need to pass the data in the
sigev_valuemeember of the struct sigevent structure passed totimer_createto the caller. I don't see it being done here nor when the timer is created. Do I miss something? Thesigev_valuevalue should be stored in the user/ptr member of struct ukevent.
sigev_valuewas stored ink_itimerstructure, I just do not know where to put it in the ukevent provided to userspace - it can be placed in pointer value if you like.
sigev_value is a union and the largest element is a pointer. So, transporting the pointer value is sufficient and it should be passed up to the user in the ptr member of struct ukevent.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
David Miller
maintainerRe: Kevent POSIX timers support.
From: Ulrich Drepper <[email protected]>
Date: Mon, 27 Nov 2006 10:20:50 -0800
Quoted 7 lines
Quoted 6 lines · Evgeniy Polyakov
Quoted 5 lines
We need to pass the data in the
sigev_valuemeember of the struct sigevent structure passed totimer_createto the caller. I don't see it being done here nor when the timer is created. Do I miss something? Thesigev_valuevalue should be stored in the user/ptr member of struct ukevent.
Quoted 7 lines
Quoted 3 lines
sigev_valuewas stored ink_itimerstructure, I just do not know where to put it in the ukevent provided to userspace - it can be placed in pointer value if you like.
sigev_valueis a union and the largest element is a pointer. So, transporting the pointer value is sufficient and it should be passed up to the user in the ptr member of struct ukevent.
Now we'll have to have a compat layer for 32-bit/64-bit environments thanks to POSIX timers, which is rediculious.
This is exactly the kind of thing I was hoping we could avoid when designing these data structures. No pointers, no non-fixed sized types, only types which are identically sized and aligned between 32-bit and 64-bit environments.
It's OK to have these problems for things designed a long time ago before 32-bit/64-bit compat issues existed, but for new stuff no way.
Re: Kevent POSIX timers support.
Quoted 2 lines · David Miller
Now we'll have to have a compat layer for 32-bit/64-bit environments thanks to POSIX timers, which is rediculious.
We already have compat_sys_timer_create. It should be sufficient just to add the conversion (if anything new is needed) there. The pointer value can be passed to userland in one or two int fields, I don't really care. When reporting the event to the user code we cannot just point into the ring buffer anyway. So while copying the data we can rewrite it if necessary. I see no need to complicate the code more than it already is.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
David Miller
maintainerRe: Kevent POSIX timers support.
From: Ulrich Drepper <[email protected]>
Date: Mon, 27 Nov 2006 10:36:06 -0800
Quoted 11 lines
Quoted 2 lines · David Miller
Now we'll have to have a compat layer for 32-bit/64-bit environments thanks to POSIX timers, which is rediculious.
We already have
compat_sys_timer_create. It should be sufficient just to add the conversion (if anything new is needed) there. The pointer value can be passed to userland in one or two int fields, I don't really care. When reporting the event to the user code we cannot just point into the ring buffer anyway. So while copying the data we can rewrite it if necessary. I see no need to complicate the code more than it already is.
Ok, as long as that thing doesn't end up in the ring buffer entry data structure, that's where the real troubles would be.
Re: Kevent POSIX timers support.
Quoted 15 lines · David Miller
From: Ulrich Drepper <[email protected]>
Date: Mon, 27 Nov 2006 10:36:06 -0800Quoted 11 lines
Quoted 2 lines · David Miller
Now we'll have to have a compat layer for 32-bit/64-bit environments thanks to POSIX timers, which is rediculious.
We already have
compat_sys_timer_create. It should be sufficient just to add the conversion (if anything new is needed) there. The pointer value can be passed to userland in one or two int fields, I don't really care. When reporting the event to the user code we cannot just point into the ring buffer anyway. So while copying the data we can rewrite it if necessary. I see no need to complicate the code more than it already is.
Quoted 2 lines
Ok, as long as that thing doesn't end up in the ring buffer entry data structure, that's where the real troubles would be.
Although ukevent has pointer embedded, it is unioned with u64, so there should be no problems until 128 bit arch appeared, which likely will not happen soon. There is also unused in kevent posix timers patch 'u32 ret_val[2]' field, which can store segval's value too.
But the fact that ukevent does not and will not in any way have variable size is absolutely.
Signature · 1 line
Evgeniy Polyakov
David Miller
maintainerRe: Kevent POSIX timers support.
From: Evgeniy Polyakov <[email protected]>
Date: Tue, 28 Nov 2006 12:16:02 +0300
Quoted 7 lines
Although ukevent has pointer embedded, it is unioned with u64, so there should be no problems until 128 bit arch appeared, which likely will not happen soon. There is also unused in kevent posix timers patch 'u32 ret_val[2]' field, which can store segval's value too.
But the fact that ukevent does not and will not in any way have variable size is absolutely.
I believe that in order to be %100 safe you will need to use the special aligned_u64 type, as that takes care of a crucial difference between x86 and x86_64 API, namely that u64 needs 8-byte alignment on x86_64 but not on x86.
You probably know this already :-)
Re: Kevent POSIX timers support.
Quoted 11 lines · David Miller
From: Evgeniy Polyakov <[email protected]>
Date: Tue, 28 Nov 2006 12:16:02 +0300Quoted 7 lines
Although ukevent has pointer embedded, it is unioned with u64, so there should be no problems until 128 bit arch appeared, which likely will not happen soon. There is also unused in kevent posix timers patch 'u32 ret_val[2]' field, which can store segval's value too.
But the fact that ukevent does not and will not in any way have variable size is absolutely.
Quoted 6 lines
I believe that in order to be %100 safe you will need to use the special
aligned_u64type, as that takes care of a crucial difference between x86 andx86_64API, namely that u64 needs 8-byte alignment onx86_64but not on x86.You probably know this already :-)
Yep :)
So I put it at the end, where structure is already correctly aligned, so there is no need for special alignment.
And, btw, last time I checked, aligned_u64 was not exported to userspace.
Signature · 1 line
Evgeniy Polyakov
David Miller
maintainerRe: Kevent POSIX timers support.
From: Evgeniy Polyakov <[email protected]>
Date: Tue, 28 Nov 2006 22:22:36 +0300
Quoted 2 lines
And, btw, last time I checked,
aligned_u64was not exported to userspace.
It is in linux/types.h and not protected by __KERNEL__ ifdefs. Perhaps you mean something else?
Re: Kevent POSIX timers support.
Quoted 8 lines · David Miller
From: Evgeniy Polyakov <[email protected]>
Date: Tue, 28 Nov 2006 22:22:36 +0300Quoted 2 lines
And, btw, last time I checked,
aligned_u64was not exported to userspace.It is in
linux/types.hand not protected by__KERNEL__ifdefs. Perhaps you mean something else?
It looks like I checked wrong #ifdef __KERNEL__/#endif pair. It is there indeed.
Signature · 1 line
Evgeniy Polyakov
Re: Kevent POSIX timers support.
In reply to Ulrich DrepperQuoted 3 lines · Ulrich Drepper
sigev_valueis a union and the largest element is a pointer. So, transporting the pointer value is sufficient and it should be passed up to the user in the ptr member of struct ukevent.
That is where I've put it in current version.
Signature · 1 line
Evgeniy Polyakov
Re: Kevent POSIX timers support.
In reply to Evgeniy PolyakovQuoted 20 lines · Evgeniy Polyakov
On Wed, Nov 22, 2006 at 01:44:16PM +0300, Evgeniy Polyakov ([email protected]) wrote:
kernel/posix-timers.c +19
-0+static int posix_kevent_init_timer (struct k_itimer *tmr,int fd) +{ +struct ukevent uk; +struct file *file; +struct kevent_user *u; +int err; + + file =fget (fd); +if (!file) { + err = -EBADF ; + goto err_out; + } + +if (file->f_op != &kevent_user_fops) { + err = -EINVAL ; + goto err_out_fput; + } + + u = file->private_data;
kernel/posix-timers.c static int posix_ktime_get_ts(clockid_t · line 285
Quoted 10 lines
kernel/posix-timers.c +10
-0+ +memset (&uk,0 ,sizeof (struct ukevent )); + + uk.type =KEVENT_POSIX_TIMER ; + uk.id.raw_u64 = (unsigned long )(tmr);/* Just cast to something unique */ + uk.ptr = tmr; + + tmr->it_sigev_value.sival_ptr = file; + + err =kevent_user_add_ukevent (&uk, u);
I think these four lines are not required. Irrespective of return value of kevent_user_add_ukevent(), we are going to release file, and return err.
Quoted 15 lines
kernel/posix-timers.c +13
-0+if (err) + goto err_out_fput; + +fput (file); + +return 0 ; + +err_out_fput: +fput (file); +err_out: +return err; +} +
Signature · 4 lines
Regards, Tushar -------------------- It's not a problem, it's an opportunity for improvement. Lets improve.
Re: Kevent POSIX timers support.
Hello.
Please _NEVER_ drop Cc: list, since not everyone can be subscribed to linux-kernel@, fortunately I'm not for example.
Quoted 12 lines · Tushar Adeshara
I think these four lines are not required. Irrespective of return value of
kevent_user_add_ukevent(), we are going to release file, and return err.Quoted 6 lines
kernel/posix-timers.c +6
-0+if (err) + goto err_out_fput; + +fput (file); + +return 0 ;
Quoted 7 lines
kernel/posix-timers.c static int posix_ktime_get_ts(clockid_t · line 298
Quoted 7 lines
kernel/posix-timers.c +7
-0+ +err_out_fput: +fput (file); +err_out: +return err; +} +
I put them to always know where error path is and where it is not, so it could be easier to put error statistic or debug.
It can be removed, but imho it reduces readability.
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovQuoted 1 line · Evgeniy Polyakov
Threads are parked in syscalls - which one should be interrupted?
It doesn't matter, use the same policy you use when waking a thread in case of an event. This is not about waking a specific thread, it's about not dropping the event notification.
Quoted 1 line
And what if there were no threads waiting in syscalls?
This is fine, do nothing. It means that the other threads are about to read the ring buffer and will pick up the event.
The case which must be avoided is that of all threads being in the kernel, one threads gets woken, and then is canceled. Without notifying the kernel about the cancellation and in the absence of further events notifications the process is deadlocked.
A second case which should be avoided is that there is a thread waiting when a thread gets canceled and there are one or more addition threads around, but not in the kernel. But those other threads might not get to the ring buffer anytime soon, so handling the event is unnecessarily delayed.
Quoted 2 lines
It has completely nothing with syscall.
You register a timer to wait until 10:15 that is all.
That's a nonsense argument. In this case you would not add any timeout parameter at all. Of course nobody would want that since it's simply too slow. Stop thinking about the absolute timeout as an exceptional case, it might very well not be for some problems.
Beside, I've already mentioned another case where a struct timespec* parameter is needed. There are even two different relative timeouts: using the monotonis clock or using the realtime clock. The latter is affected by gettimeofday and ntp.
Quoted 4 lines
Quoted 2 lines
Quoted 1 line
Kernel use relative timeouts.
Look again. This time at the implementation. For
FUTEX_LOCK_PIthe timeout is an absolute timeout.How come? It just uses timespec.
Correct, it's using the value passed in.
Quoted 7 lines
Quoted 3 lines
The signal mask handling is orthogonal to all this and must be explicit.
In some cases explicit pthread_sigmask/sigprocmask calls. But this is
not atomic if a signal must be masked/unmasked for the*_waitcall. This is why we have variants like pselect/ppoll/epoll_pwait which explicitly and *atomically* change the signal mask for the duration of the call.You probably missed kevent signal patch - signal will not be delivered
(in special cases) since it will not be copied into signal mask. System just will not know that it happend. Completely. Like putting it into blocked mask.
I don't really understand what you want to say here.
I looked over the patch and I don't think I miss anything. You just deliver the signal as an event. No signal mask handling at all. This is exactly the problem.
Quoted 2 lines
But it is completely irrelevant with kevent signals - there is no race for that case when signal is delivered through file descriptor.
Of course there is a race. You might not want the signal delivered. This is what the signal mask is for. Of the other way around, as I've said before.
Quoted 2 lines
It is much better to not know how thing works, then to not be possible to understand how new things can work.
Well, this explains why you don't understand signal masks at all.
Quoted 1 line
Add kevent signal and do not process that event.
That's not only a horrible hack, it does not work. If I want to ignore a signal for the duration of the call, while you have it occasionally blocked for the rest of the program you would have to register the kevent for the signal, unblock the signal, the kevent_wait call, reset the mask, remove the kevent for the signal.. Otherwise it would not be delivered to be ignored. And then you have a race, the same race pselect is designed to prevent. In fact, you have two races.
There are other scenarios like this. Fact is, signal mask handling is necessary and it cannot be folded into the event handling, it's orthogonal.
Quoted 4 lines
Having special type of kevent signal is the same as putting signal into blocked mask, but signal event will be marked as ready - to indicate that condition was there.
There will not be any race in that case.
Nonsense on all counts.
Quoted 3 lines
I think I am a bit blind, probably parts of Leonids are still getting into my brain, but there is one syscall called
kevent_ctl()which adds different events, including timer, signal, socket and others.
You are searching for callbacks and if none is found you return EINVAL.
This is exactly the same as if you'd create separate syscalls.
Perhaps even worse, I really don't like demultiplexers, separate syscalls are much cleaner.
Avoiding these callbacks would help reducing the kernel interface, especially for this useless since inferior timer implementation.
Quoted 1 line
I can replace with
-ENOSYSif you like.
It's necessary since we must be able to distinguish the errors.
Quoted 2 lines
No one asked and pain me to create kevent, but it is done.
Probably no the way some people wanted, but it always happend, it is really not that bad.
Nobody says that the work isn't appreciated. But if you don't want it to be critiqued, don't publish it. If you don't want to mask any more changes, fine, say so. I'll find somebody else to do it or will do it myself.
I claim that I know a thing or two about interfaces of the runtime programs expect to use. And I know POSIX and the way the interfaces are designed and how they interact.
Quoted 1 line
Ulrich, tell me the truth, will you kill me if I say that I have an entry in TODO to implement different AIO design (details for interested readers can be found in my blog), and then present it to community? :))
I don't care about the kernel implementation as long as the interface is compatible with what I need for the POSIX AIO implementation. The currently proposed code is going in that direction. Any implementation which like Ben's old one does not allow POSIX AIO to be implemented I will of oppose.
Quoted 5 lines
Quoted 1 line
Then define
SIGEV_KEVENTas a value distinct from the otherSIGEV_values. In the code which handles setup of timers (thetimer_createsyscall), recognizeSIGEV_KEVENTand handle it appropriately. I.e., call into the code to register the event source, just like you'd do with the current interface. Then add the code to post an event to the event queue where currently signals would be sent et voilà.Ok, I see.
It is doable and simple.
I will try to implement it tomorrow.
Thanks, that's progress. And yes, I imagine it's not hard which is why the currently proposed timer interface is so unnecessary.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 6 lines · Ulrich Drepper
Quoted 1 line · Evgeniy Polyakov
Threads are parked in syscalls - which one should be interrupted?
It doesn't matter, use the same policy you use when waking a thread in case of an event. This is not about waking a specific thread, it's about not dropping the event notification.
Event notification is not dropped - thread was awakened, kernel task is completed. Kernel does not know and should not have such knowledge about the fact that selected thread was not good enough. If you want to wakeup another thread - create another event, that is why I proposed userspace notifications, which I actually do not like.
Quoted 11 lines
Quoted 1 line
And what if there were no threads waiting in syscalls?
This is fine, do nothing. It means that the other threads are about to read the ring buffer and will pick up the event.
The case which must be avoided is that of all threads being in the kernel, one threads gets woken, and then is canceled. Without notifying the kernel about the cancellation and in the absence of further events notifications the process is deadlocked.
Quoted 5 lines
A second case which should be avoided is that there is a thread waiting when a thread gets canceled and there are one or more addition threads around, but not in the kernel. But those other threads might not get to the ring buffer anytime soon, so handling the event is unnecessarily delayed.
If those threads are not in the kernel, kernel can not wake hem up. But if there is an event 'wake me up when thread has died' or something like that, when new threads will try to sleep in syscall, they will be immediately awakened, since that event will be ready.
Quoted 7 lines
Quoted 2 lines
It has completely nothing with syscall.
You register a timer to wait until 10:15 that is all.That's a nonsense argument. In this case you would not add any timeout parameter at all. Of course nobody would want that since it's simply too slow. Stop thinking about the absolute timeout as an exceptional case, it might very well not be for some problems.
I repeate - timeout is needed to tell kernel the maximum possible timeframe syscall can live. When you will tell me why you want syscall to be interrupted when some absolute time is on the clock instead of having special event for that, then ok.
I think I know why you want absolute time there - because glibc converts most of the timeouts to absolute time since posix waiting pthread_cond_timedwait() works only with it.
Quoted 4 lines
Beside, I've already mentioned another case where a struct timespec* parameter is needed. There are even two different relative timeouts:
using the monotonis clock or using the realtime clock. The latter is
affected by gettimeofday and ntp.
Kevent convert it to jiffies since it uses wait_event() and friends, jiffies do not carry information about clocks to be used.
Quoted 22 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Kernel use relative timeouts.
Look again. This time at the implementation. For
FUTEX_LOCK_PIthe timeout is an absolute timeout.How come? It just uses timespec.
Correct, it's using the value passed in.
Quoted 11 lines
Quoted 6 lines
The signal mask handling is orthogonal to all this and must be explicit. In some cases explicit pthread_sigmask/sigprocmask calls. But this is not atomic if a signal must be masked/unmasked for the
*_waitcall. This is why we have variants like pselect/ppoll/epoll_pwait which explicitly and *atomically* change the signal mask for the duration of the call.You probably missed kevent signal patch - signal will not be delivered (in special cases) since it will not be copied into signal mask. System just will not know that it happend. Completely. Like putting it into blocked mask.
Quoted 5 lines
I don't really understand what you want to say here.
I looked over the patch and I don't think I miss anything. You just deliver the signal as an event. No signal mask handling at all. This is exactly the problem.
Have you seen specific_send_sig_info():
/* Short-circuit ignored signals. */
if (sig_ignored(p, sig)) {
ret = 1;
goto out;
}almost the same happens when signal is delivered using kevent (special case) - pending mask is not updated.
Quoted 6 lines
Quoted 2 lines
But it is completely irrelevant with kevent signals - there is no race for that case when signal is delivered through file descriptor.
Of course there is a race. You might not want the signal delivered. This is what the signal mask is for. Of the other way around, as I've said before.
Then ignore that event - there is no race between signal delivery and other descriptors reading, and it _is_ when signal is delivered no through the same queue but asynchronously with mask update.
Quoted 4 lines
Quoted 2 lines
It is much better to not know how thing works, then to not be possible to understand how new things can work.
Well, this explains why you don't understand signal masks at all.
Nice :)
I at least try to do something to solve this problem, instead of blindly saying the same again and again without even trying to hear and understand what others say.
Quoted 10 lines
Quoted 1 line
Add kevent signal and do not process that event.
That's not only a horrible hack, it does not work. If I want to ignore a signal for the duration of the call, while you have it occasionally blocked for the rest of the program you would have to register the kevent for the signal, unblock the signal, the
kevent_waitcall, reset the mask, remove the kevent for the signal.. Otherwise it would not be delivered to be ignored. And then you have a race, the same race pselect is designed to prevent. In fact, you have two races.
Quoted 2 lines
There are other scenarios like this. Fact is, signal mask handling is necessary and it cannot be folded into the event handling, it's orthogonal.
You have too narrow look.
Look broader - pselect() has signal mask to prevent race between async signal delivery and file descriptor readiness. With kevent both that events are delivered through the same queue, so there is no race, so kevent syscalls do not need that workaround for 20 years-old design, which can not handle different than fd events.
Quoted 12 lines
Quoted 4 lines
Having special type of kevent signal is the same as putting signal into blocked mask, but signal event will be marked as ready - to indicate that condition was there.
There will not be any race in that case.Nonsense on all counts.
Quoted 3 lines
I think I am a bit blind, probably parts of Leonids are still getting into my brain, but there is one syscall called
kevent_ctl()which adds different events, including timer, signal, socket and others.
Quoted 7 lines
You are searching for callbacks and if none is found you return EINVAL.
This is exactly the same as if you'd create separate syscalls.
Perhaps even worse, I really don't like demultiplexers, separate syscalls are much cleaner.Avoiding these callbacks would help reducing the kernel interface, especially for this useless since inferior timer implementation.
You completely do not want to understand how kevent works and why they are needed, if you would try to think that there are different than yours opinions, then probably we could have some progress.
Those callbacks are neededto support different types of objects, which can produce events, with the same interface.
Quoted 3 lines
Quoted 1 line
I can replace with
-ENOSYSif you like.It's necessary since we must be able to distinguish the errors.
And what if user requests bogus event type - is it invalid condition or normal, but not handled (thus enosys)?
Quoted 8 lines
Quoted 3 lines
No one asked and pain me to create kevent, but it is done. Probably no the way some people wanted, but it always happend, it is really not that bad.
Nobody says that the work isn't appreciated. But if you don't want it to be critiqued, don't publish it. If you don't want to mask any more changes, fine, say so. I'll find somebody else to do it or will do it myself.
I greatly appreciate critics, really. But when it comes to 'this sucks because it sucks, no matter if it is completely different way, it still sucks because others sucked there too' I can not say it is critics, it becomes nonsence.
Quoted 3 lines
I claim that I know a thing or two about interfaces of the runtime programs expect to use. And I know POSIX and the way the interfaces are designed and how they interact.
Well, then I claim that I do not know 'thing or two about interfaces of the runtime programs expect to use', but instead I write those programms and I know my needs. And POSIX interfaces are the last one I prefer to use.
We are on the different positions - theoretical thoughs about world hapinness, and practical usage. I do not say that only one of that approaches must exist, they both can live together, but it requires that people from both sides not only tried to say, that other part is stupid and do not know something or anything, but instead tried to listen and get into account that.
Quoted 9 lines
Quoted 3 lines
Ulrich, tell me the truth, will you kill me if I say that I have an entry in TODO to implement different AIO design (details for interested readers can be found in my blog), and then present it to community? :))
I don't care about the kernel implementation as long as the interface is compatible with what I need for the POSIX AIO implementation. The currently proposed code is going in that direction. Any implementation which like Ben's old one does not allow POSIX AIO to be implemented I will of oppose.
What if it will not be called POSIX AIO, but instead some kind of 'true AIO' or 'real AIO' or maybe 'alternative AIO'? :)
It is quite sure that POSIX AIO interfaces will unlikely to be applied there...
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Then define
SIGEV_KEVENTas a value distinct from the otherSIGEV_values. In the code which handles setup of timers (thetimer_createsyscall), recognizeSIGEV_KEVENTand handle it appropriately. I.e., call into the code to register the event source, just like you'd do with the current interface. Then add the code to post an event to the event queue where currently signals would be sent et voilà.
Quoted 6 lines
Quoted 3 lines
Ok, I see.
It is doable and simple.
I will try to implement it tomorrow.Thanks, that's progress. And yes, I imagine it's not hard which is why the currently proposed timer interface is so unnecessary.
It is the first techical but not political problem we cought in this endless discussion, I separated it in different subthread already. Let's try to think more about it there.
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 1 line · Evgeniy Polyakov
Event notification is not dropped - [...]
Since you said you added the new syscall I'll leave this alone.
Quoted 4 lines
I repeate - timeout is needed to tell kernel the maximum possible timeframe syscall can live. When you will tell me why you want syscall to be interrupted when some absolute time is on the clock instead of having special event for that, then ok.
This goes together with...
Quoted 3 lines
I think I know why you want absolute time there - because glibc converts most of the timeouts to absolute time since posix waiting
pthread_cond_timedwait()works only with it.
I did not make the decision to use absolute timeouts/deadlines. This is what is needed in many situations. It's the more general way to specify delays. These are real-world requirements which were taken into account when designing the interfaces.
For most cases I would agree that when doing AIO you need relative timeouts. But the event handling is not about AIO alone. It's all kinds of events and some/many are wall clock related. And it is definitely necessary in some situations to be able to interrupt if the clock jumps ahead. If a program deals with devices in the real world this be crucial. The new event handling must be generic enough to accommodate all these uses and using struct timespec* plus eventually flags does not add any measurable overhead so there is no reason to not do it right.
Quoted 2 lines
Kevent convert it to jiffies since it uses
wait_event()and friends, jiffies do not carry information about clocks to be used.
Then this points to a place in the implementation which needs changing.
The interface cannot be restricted just because the implementation
currently allow this to be implemented.
Quoted 8 lines
/* Short-circuit ignored signals. */ if (sig_ignored(p, sig)) { ret = 1; goto out; }almost the same happens when signal is delivered using kevent (special case) - pending mask is not updated.
Yes, and how do you set the signal mask atomically wrt to registering and unregistering signals with kevent and the syscall itself? You cannot. But this is exactly which is resolved by adding the signal mask parameter.
Programs which don't need the functionality simply pass a NULL pointer and the cost is once again not measurable. But don't restrict the functionality just because you don't see a use for this in your small world.
Yes, we could (later again) add new syscalls. But this is plain stupid.
I would love to never have the epoll_wait or select syscall and just
have epoll_pwait and pselect since the functionality is a superset. We have a larger kernel ABI. Here we can stop making the same mistake again.
For the userlevel side we might even have separate intterfaces, one with one without signal mask parameter. But that's userlevel, both functions would use the same syscall.
Quoted 8 lines
Quoted 1 line
There are other scenarios like this. Fact is, signal mask handling is necessary and it cannot be folded into the event handling, it's orthogonal.
You have too narrow look.
Look broader -pselect()has signal mask to prevent race between async signal delivery and file descriptor readiness. With kevent both that events are delivered through the same queue, so there is no race, so kevent syscalls do not need that workaround for 20 years-old design, which can not handle different than fd events.
Your failure to understand to signal model leads to wrong conclusions. There are races, several of them, and you cannot do anything without signal mask parameters. I've explained this before.
Quoted 3 lines
Quoted 1 line
Avoiding these callbacks would help reducing the kernel interface, especially for this useless since inferior timer implementation.
You completely do not want to understand how kevent works and why they are needed, if you would try to think that there are different than yours opinions, then probably we could have some progress.
I think I know very well how they work meanwhile.
Quoted 2 lines
Those callbacks are neededto support different types of objects, which can produce events, with the same interface.
Yes, but it is not necessary to expose all the different types in the userlevel APIs. That's the issue. Reduce the exposure of kernel functionality to userlevel APIs.
If you integrate the timer handling into the POSIX timer syscalls the callbacks in your timer patch might not need be there. At least the enqueue callback, if I remember correctly. All enqueue operations are initiated by timer_create calls which can call the function directly. Removing the callback from the list used by add_ctl will reduce the exposed interface.
Quoted 5 lines
Quoted 2 lines
Quoted 1 line
I can replace with
-ENOSYSif you like.It's necessary since we must be able to distinguish the errors.
And what if user requests bogus event type - is it invalid condition or normal, but not handled (thus enosys)?
It's ENOSYS. Just like for system calls. You cannot distinguish completely invalid values from values which are correct only on later kernels. But: the first use is a bug while the later is not a bug and needed to write robust and well performing apps. The former's problems therefore are unimportant.
Quoted 4 lines
Well, then I claim that I do not know 'thing or two about interfaces of the runtime programs expect to use', but instead I write those programms and I know my needs. And POSIX interfaces are the last one I prefer to use.
Well, there it is. You look out for yourself while I make sure that all the bases I can think of are covered.
Again, if you don't want to work on the generalization, fine. That's your right. Nobody will think bad of you for doing this. But don't expect that a) I'll not try to change it and b) I'll not object to the changes being accepted as they are.
Quoted 4 lines
What if it will not be called POSIX AIO, but instead some kind of 'true AIO' or 'real AIO' or maybe 'alternative AIO'? :)
It is quite sure that POSIX AIO interfaces will unlikely to be applied there...
Programmers don't like specialized OS-specific interfaces. AIO users who put up with libaio are rare. The same will happen with any other approach. The Samba use is symptomatic: they need portability even if this costs a minute percentage of performance compared to a highly specialized implementation.
There might be some aspects of POSIX AIO which could be implemented better on Linux. But the important part in the name is the 'P'.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 17 lines · Ulrich Drepper
Quoted 4 lines
I repeate - timeout is needed to tell kernel the maximum possible timeframe syscall can live. When you will tell me why you want syscall to be interrupted when some absolute time is on the clock instead of having special event for that, then ok.
This goes together with...
Quoted 3 lines
I think I know why you want absolute time there - because glibc converts most of the timeouts to absolute time since posix waiting
pthread_cond_timedwait()works only with it.I did not make the decision to use absolute timeouts/deadlines. This is what is needed in many situations. It's the more general way to specify delays. These are real-world requirements which were taken into account when designing the interfaces.
Quoted 9 lines
For most cases I would agree that when doing AIO you need relative timeouts. But the event handling is not about AIO alone. It's all kinds of events and some/many are wall clock related. And it is definitely necessary in some situations to be able to interrupt if the clock jumps ahead. If a program deals with devices in the real world this be crucial. The new event handling must be generic enough to accommodate all these uses and using struct timespec* plus eventually flags does not add any measurable overhead so there is no reason to not do it right.
Timeouts are not about AIO or any other event types (there are a lot of them already as you can see), it is only about syscall itself. Please point me to _any_ syscall out there which uses absolute time (except settimeofday() and similar syscalls).
Quoted 6 lines
Quoted 2 lines
Kevent convert it to jiffies since it uses
wait_event()and friends, jiffies do not carry information about clocks to be used.Then this points to a place in the implementation which needs changing.
The interface cannot be restricted just because the implementation
currently allow this to be implemented.
Btw, do you propose to change all users of wait_event()?
Interface is not restricted, it is just different from what you want it to be, and you did not show why it requires changes.
Btw, there are _no_ interfaces similar to 'wait event with absolute times' in kernel.
Quoted 6 lines
Quoted 6 lines
/* Short-circuit ignored signals. */ if (sig_ignored(p, sig)) { ret = 1; goto out; }
Quoted 7 lines
Quoted 2 lines
almost the same happens when signal is delivered using kevent (special case) - pending mask is not updated.
Yes, and how do you set the signal mask atomically wrt to registering and unregistering signals with kevent and the syscall itself? You cannot. But this is exactly which is resolved by adding the signal mask parameter.
kevent signal registering is atomic with respect to other kevent
syscalls: control syscalls are protected by mutex and waiting syscalls work with queue, which is protected by appropriate lock.
Quoted 4 lines
Programs which don't need the functionality simply pass a NULL pointer and the cost is once again not measurable. But don't restrict the functionality just because you don't see a use for this in your small world.
Quoted 8 lines
Yes, we could (later again) add new syscalls. But this is plain stupid.
I would love to never have theepoll_waitor select syscall and just
haveepoll_pwaitand pselect since the functionality is a superset. We have a larger kernel ABI. Here we can stop making the same mistake again.For the userlevel side we might even have separate intterfaces, one with one without signal mask parameter. But that's userlevel, both functions would use the same syscall.
Let me formulate signal problem here, please point me if it is correct or not.
User registers some async signal notifications and calls poll() waiting for some file descriptors to became ready. When it is interrupted there is no knowledge about what really happend first - signal was delivered or file descriptor was ready.
Is it correct?
In case it is, let me explain why this situation can not happen with
kevent: since signals are not delivered in the old way, but instead they are queued into the same queue where file descriptors are, and queueing is atomic, and pending signal mask is not updated, user will only read one event after another, which automatically (since delivery is atomic) means that what first was read, that was first happend.
So, why in the latter situation we need to specify signal mask, which will block some signals from _async_ delivery, since there is _no_ async delivery?
Quoted 11 lines
Quoted 10 lines
Quoted 3 lines
There are other scenarios like this. Fact is, signal mask handling is necessary and it cannot be folded into the event handling, it's orthogonal.
You have too narrow look.
Look broader -pselect()has signal mask to prevent race between async signal delivery and file descriptor readiness. With kevent both that events are delivered through the same queue, so there is no race, so kevent syscalls do not need that workaround for 20 years-old design, which can not handle different than fd events.
Quoted 3 lines
Your failure to understand to signal model leads to wrong conclusions. There are races, several of them, and you cannot do anything without signal mask parameters. I've explained this before.
Please refer to my above explaination, point me in that example what we are talking about. It seems we do not understand each other.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Avoiding these callbacks would help reducing the kernel interface, especially for this useless since inferior timer implementation.
You completely do not want to understand how kevent works and why they are needed, if you would try to think that there are different than yours opinions, then probably we could have some progress.
I think I know very well how they work meanwhile.
If that would be true, I would be very happy. Definitely.
Quoted 7 lines
Quoted 2 lines
Those callbacks are neededto support different types of objects, which can produce events, with the same interface.
Yes, but it is not necessary to expose all the different types in the userlevel APIs. That's the issue. Reduce the exposure of kernel functionality to userlevel APIs.
Quoted 6 lines
If you integrate the timer handling into the POSIX timer syscalls the callbacks in your timer patch might not need be there. At least the enqueue callback, if I remember correctly. All enqueue operations are initiated by
timer_createcalls which can call the function directly. Removing the callback from the list used byadd_ctlwill reduce the exposed interface.
I posted a patch to implement kevent support for posix timers, it is quite simple in existing model. No need to remove anything, that allows to have flexibility and create different usage models others than what is required by fairly small part of the users.
Quoted 11 lines
Quoted 5 lines
Quoted 2 lines
Quoted 1 line
I can replace with
-ENOSYSif you like.It's necessary since we must be able to distinguish the errors.
And what if user requests bogus event type - is it invalid condition or normal, but not handled (thus enosys)?
It's ENOSYS. Just like for system calls. You cannot distinguish completely invalid values from values which are correct only on later kernels. But: the first use is a bug while the later is not a bug and needed to write robust and well performing apps. The former's problems therefore are unimportant.
I implemented it to return -enosys for the case, when event type is smaller than maximum allowed and no subsystem is registered, and -einval for the case, when requested type is higher.
Quoted 5 lines
Quoted 4 lines
Well, then I claim that I do not know 'thing or two about interfaces of the runtime programs expect to use', but instead I write those programms and I know my needs. And POSIX interfaces are the last one I prefer to use.
Quoted 7 lines
Well, there it is. You look out for yourself while I make sure that all the bases I can think of are covered.
Again, if you don't want to work on the generalization, fine. That's your right. Nobody will think bad of you for doing this. But don't expect that a) I'll not try to change it and b) I'll not object to the changes being accepted as they are.
It is not about generalization, but about those who do practical work and those who prefer to spread theoretical thoughts, which result in several month of unused empty discussions.
Quoted 10 lines
Quoted 4 lines
What if it will not be called POSIX AIO, but instead some kind of 'true AIO' or 'real AIO' or maybe 'alternative AIO'? :)
It is quite sure that POSIX AIO interfaces will unlikely to be applied there...Programmers don't like specialized OS-specific interfaces. AIO users who put up with libaio are rare. The same will happen with any other approach. The Samba use is symptomatic: they need portability even if this costs a minute percentage of performance compared to a highly specialized implementation.
Do not say for everyone - it is not a some kind of feodalism with the only opinion allowed - respect those who do not like/do not want what you propose them to use.
Quoted 2 lines
There might be some aspects of POSIX AIO which could be implemented better on Linux. But the important part in the name is the 'P'.
I will create completely different model, POSIX completely is not designed for that - that model allows to specify set of tasks performed on object completely asycnhronous to user before object is returned - for example specify destination socket and filename, so async sendfile will asynchronously open file, transfer it to remote destination and probably even close (or return file descriptor). The same can be applied to aio read/write.
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 5 lines · Evgeniy Polyakov
On Wed, Nov 22, 2006 at 02:22:15PM -0800, Ulrich Drepper ([email protected]) wrote:
Timeouts are not about AIO or any other event types (there are a lot of them already as you can see), it is only about syscall itself.
Please point me to _any_ syscall out there which uses absolute time (exceptsettimeofday()and similar syscalls).
futex(FUTEX_LOCK_PI).
Quoted 1 line
Btw, do you propose to change all users of
wait_event()?
Which users?
Quoted 2 lines
Interface is not restricted, it is just different from what you want it to be, and you did not show why it requires changes.
No, it is restricted because I cannot express something like an absolute timeout/deadline. If the parameter would be a struct timespec* then at any time we can implement either relative timeouts w/ and w/out observance of settimeofday/ntp and absolute timeouts. This is what makes the interface generic and unrestricted while your current version cannot be used for the latter.
Quoted 3 lines
kevent signal registering is atomic with respect to other kevent
syscalls: control syscalls are protected by mutex and waiting syscalls work with queue, which is protected by appropriate lock.
It is about atomicity wrt to the signal mask manipulation which would have to precede the kevent_wait call and the call itself (and registering a signal for kevent delivery). This is not atomic.
Quoted 2 lines
Let me formulate signal problem here, please point me if it is correct or not.
There are a myriad of different scenarios, it makes no sense to pick one. The interface must be generic to cover them all, I don't know how often I have to repeat this.
Quoted 4 lines
User registers some async signal notifications and calls
poll()waiting for some file descriptors to became ready. When it is interrupted there is no knowledge about what really happend first - signal was delivered or file descriptor was ready.
The order is unimportant. You change the signal mask, for instance, if the time when a thread is waiting in poll() is the only time when a signal can be handled. Or vice versa, it's the time when signals are not wanted. And these are per-thread decisions.
Signal handlers and kevent registrations for signals are process-wide decisions. And furthermore: with kevent delivered signals there is no signal mask anymore (at least you seem to not check it). Even if this would be done it doesn't change the fact that you cannot use signals the way many programs want to.
Fact is that without a signal queue you cannot implement the above cases. You cannot block/unblock a signal for a specific thread. You also cannot work together with signals which cannot be delivered through kevent. This is the case for existing code in a program which happens to use also kevent and it is the case if there is more than one possible recipient. With kevent signals can be attached to one kevent queue only but the recipients (different threads or only different parts of a program) need not use the same kevent queue.
I've said from the start that you cannot possibly expect that programs are not using signal delivery in the current form. And the complete loss of blocking signals for individual threads makes the kevent-based signal delivery incomplete (in a non-fixable form) anyway.
Quoted 6 lines
In case it is, let me explain why this situation can not happen with
kevent: since signals are not delivered in the old way, but instead they are queued into the same queue where file descriptors are, and queueing is atomic, and pending signal mask is not updated, user will only read one event after another, which automatically (since delivery is atomic) means that what first was read, that was first happend.
This really has nothing to do with the problem.
Quoted 2 lines
I posted a patch to implement kevent support for posix timers, it is quite simple in existing model. No need to remove anything,
Surely you don't suggest keeping your original timer patch?
Quoted 2 lines
I implemented it to return -enosys for the case, when event type is
smaller than maximum allowed and no subsystem is registered, and -einval for the case, when requested type is higher.
What is the "maximum allowed"? ENOSYS must be returned for all values which could potentially in future be used as a valid type value. If you limit the values which are treated this way you are setting a fixed upper limit for the type values which _ever_ can be used.
Quoted 3 lines
It is not about generalization, but about those who do practical work and those who prefer to spread theoretical thoughts, which result in several month of unused empty discussions.
I've told you, then don't work on these parts. I'll get the changes I think are needed implemented by somebody else or I'll do it myself. If you say that only those you implement something have a say in the way this is done then this is fine with me. But you have to realize that you're not the one who will make all the final decisions.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 9 lines · Ulrich Drepper
Quoted 6 lines · Evgeniy Polyakov
On Wed, Nov 22, 2006 at 02:22:15PM -0800, Ulrich Drepper ([email protected]) wrote:
Timeouts are not about AIO or any other event types (there are a lot of them already as you can see), it is only about syscall itself. Please point me to _any_ syscall out there which uses absolute time (exceptsettimeofday()and similar syscalls).
futex(FUTEX_LOCK_PI).
It just sets hrtimer with abs time and sleeps - it can achieve the same goals using similar to wait_event() mechanism.
Quoted 3 lines
Quoted 1 line
Btw, do you propose to change all users of
wait_event()?Which users?
Any users which use wait_event() or schedule_timeout(). Futex for example - it perfectly ok lives with relative timeouts provided to schedule_timeout() - the same (roughly saying of course) is done in kevent.
Quoted 9 lines
Quoted 2 lines
Interface is not restricted, it is just different from what you want it to be, and you did not show why it requires changes.
No, it is restricted because I cannot express something like an absolute timeout/deadline. If the parameter would be a struct timespec* then at any time we can implement either relative timeouts w/ and w/out observance of settimeofday/ntp and absolute timeouts. This is what makes the interface generic and unrestricted while your current version cannot be used for the latter.
I think I said already several times that absolute timeouts are not related to syscall execution process. But you seems to not hear me and insist.
Ok, I will change waiting syscalls to have 'flags' parameter and 'struct timespec' as timeout parameter. Special bit in flags will result in additional timer setup which will fire after absolute timeout and will wake up those who wait...
Quoted 7 lines
Quoted 3 lines
kevent signal registering is atomic with respect to other kevent
syscalls: control syscalls are protected by mutex and waiting syscalls work with queue, which is protected by appropriate lock.It is about atomicity wrt to the signal mask manipulation which would have to precede the
kevent_waitcall and the call itself (and registering a signal for kevent delivery). This is not atomic.
If signal mask is updated from userspace it should be done through kevent - add/remove different kevent signals. Signal mask of pending signals is not updated for special kevent signals.
Quoted 6 lines
Quoted 2 lines
Let me formulate signal problem here, please point me if it is correct or not.
There are a myriad of different scenarios, it makes no sense to pick one. The interface must be generic to cover them all, I don't know how often I have to repeat this.
The whole signal mask was added by POSXI exactly for that single practical race in the event dispatching mechanism, which can not handle other types of events like signals.
Quoted 10 lines
Quoted 4 lines
User registers some async signal notifications and calls
poll()waiting for some file descriptors to became ready. When it is interrupted there is no knowledge about what really happend first - signal was delivered or file descriptor was ready.The order is unimportant. You change the signal mask, for instance, if the time when a thread is waiting in
poll()is the only time when a signal can be handled. Or vice versa, it's the time when signals are not wanted. And these are per-thread decisions.
Quoted 5 lines
Signal handlers and kevent registrations for signals are process-wide decisions. And furthermore: with kevent delivered signals there is no signal mask anymore (at least you seem to not check it). Even if this would be done it doesn't change the fact that you cannot use signals the way many programs want to.
There is major contradiction here - you say that programmers will use old-style signal delivery and want me to add signal mask to prevent that delivery, so signals would be in blocked mask, when I say that current kevent signal delivery does not update pending signal mask, which is the same as putting signals into blocked mask, you say that it is not what is required.
Quoted 8 lines
Fact is that without a signal queue you cannot implement the above cases. You cannot block/unblock a signal for a specific thread. You also cannot work together with signals which cannot be delivered through kevent. This is the case for existing code in a program which happens to use also kevent and it is the case if there is more than one possible recipient. With kevent signals can be attached to one kevent queue only but the recipients (different threads or only different parts of a program) need not use the same kevent queue.
Signal queue is replaced with kevent queue, and it is in sync with all other kevents.
Programmers which want to use kevents will use kevents (if miracle will happend and we agree that kevent is good for inclusion), and programmers will know how kevent signal delivery works.
Quoted 4 lines
I've said from the start that you cannot possibly expect that programs are not using signal delivery in the current form. And the complete loss of blocking signals for individual threads makes the kevent-based signal delivery incomplete (in a non-fixable form) anyway.
Having sigmask parameter is the same as creating kevent signal delivery.
And, btw, programmers can change signal mask before calling syscall, since in the syscall there is a gap between start and sigprocmask() call.
Quoted 8 lines
Quoted 6 lines
In case it is, let me explain why this situation can not happen with
kevent: since signals are not delivered in the old way, but instead they are queued into the same queue where file descriptors are, and queueing is atomic, and pending signal mask is not updated, user will only read one event after another, which automatically (since delivery is atomic) means that what first was read, that was first happend.This really has nothing to do with the problem.
It is the only practical example of the need for that signal mask. And it can be perfectly handled by kevent.
Quoted 4 lines
Quoted 2 lines
I posted a patch to implement kevent support for posix timers, it is quite simple in existing model. No need to remove anything,
Surely you don't suggest keeping your original timer patch?
Of course not - kevent timers are more scalable than posix timers (the latter uses idr, which is slower than balanced binary tree, since it looks like it uses similar to radix tree algo), POSIX interface is much-much-much more unconvenient to use than simple add/wait.
Quoted 8 lines
Quoted 3 lines
I implemented it to return -enosys for the case, when event type is smaller than maximum allowed and no subsystem is registered, and -einval for the case, when requested type is higher.
What is the "maximum allowed"? ENOSYS must be returned for all values which could potentially in future be used as a valid type value. If you limit the values which are treated this way you are setting a fixed upper limit for the type values which _ever_ can be used.
Upper limit is for current version - when new type is added limit is increased - just like maximum number of syscalls.
Ok, I will use -ENOSYS for all cases.
Quoted 9 lines
Quoted 3 lines
It is not about generalization, but about those who do practical work and those who prefer to spread theoretical thoughts, which result in several month of unused empty discussions.
I've told you, then don't work on these parts. I'll get the changes I think are needed implemented by somebody else or I'll do it myself. If you say that only those you implement something have a say in the way this is done then this is fine with me. But you have to realize that you're not the one who will make all the final decisions.
Because our void discussion seems to never end, which puts kevent into hung state - I definitely prefer final words made by kernel maintainers about inclusion or declining of the kevents, but they keep silence since they look for not only my decision as author, but also different opinions of the potential users.
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 2 lines · Evgeniy Polyakov
It just sets hrtimer with abs time and sleeps - it can achieve the same goals using similar to
wait_event()mechanism.
I don't follow. Of course it is somehow possible to wait until an absolute deadline. But it's not part of the parameter list and hence easily and _quickly_ usable.
Quoted 6 lines
Quoted 2 lines
Quoted 1 line
Btw, do you propose to change all users of
wait_event()?Which users?
Any users which use
wait_event()orschedule_timeout(). Futex for example - it perfectly ok lives with relative timeouts provided toschedule_timeout()- the same (roughly saying of course) is done in kevent.
No, it does not live perfectly OK with relative timeouts. The userlevel implementation is actually wrong because of this in subtle ways. Some futex interfaces take absolute timeouts and they have to be interrupted if the realtime clock is set forward.
Also, the calls are complicated and slow because the userlevel wrapper has to call clock_gettime/gettimeofday before each futex syscall. If the kernel would accept absolute timeouts as well we would save a syscall and have actually a correct implementation.
Quoted 3 lines
I think I said already several times that absolute timeouts are not related to syscall execution process. But you seems to not hear me and insist.
Because you're wrong. For your use cases it might not be but it's not true in general. And your interface is preventing it from being implemented forever.
Quoted 4 lines
Ok, I will change waiting syscalls to have 'flags' parameter and 'struct timespec' as timeout parameter. Special bit in flags will result in additional timer setup which will fire after absolute timeout and will wake up those who wait...
Thanks a lot.
Quoted 7 lines
Quoted 4 lines
Quoted 3 lines
kevent signal registering is atomic with respect to other kevent
syscalls: control syscalls are protected by mutex and waiting syscalls work with queue, which is protected by appropriate lock.It is about atomicity wrt to the signal mask manipulation which would have to precede the
kevent_waitcall and the call itself (and registering a signal for kevent delivery). This is not atomic.If signal mask is updated from userspace it should be done through kevent - add/remove different kevent signals.
Indeed, this is what I've been saying and why ppoll/pselect/epoll_pwait take the sigset_t parameter.
Adding the signal mask to the queued events (e.g., the signal events) does not work. First of all it's slow, you'd have to find and combine all mask at least every time a signal event is added/removed. Then how do you combine them, OR or AND? Not all threads might want/need the same signal mask.
These are just some of the usability problems. The only clean and usable solution is really to OPTIONALLY pass in the signal mask. Nobody forces anybody to use this feature. Pass a NULL pointer and nothing happens, this is how the other syscalls also work.
Quoted 3 lines
The whole signal mask was added by POSXI exactly for that single practical race in the event dispatching mechanism, which can not handle other types of events like signals.
No. How should this argument make sense ? Signals cannot be used in the current event handling and are therefore used for something completely different. And they will have to be used like this for many applications (.e., thread cancellation, setuid/setgid implementation, etc).
That fact that the new event handling can handle signals is orthogonal (and good). But it does not supersede the old signal use, it's something new. The old uses are still valid.
BTW: there is a little design decision which has to be made: if a signal is registered with kevent and this signal is sent to a specific thread instead of the process (tkill and tgkill), what should happen? I'm currently leaning toward failing the tkill/tgkill syscall if delivery of the signal requires posting to an event queue.
Quoted 3 lines
There is major contradiction here - you say that programmers will use old-style signal delivery and want me to add signal mask to prevent that delivery, so signals would be in blocked mask,
That's one thing you can do. You also can unblock signals.
Quoted 3 lines
when I say that current kevent signal delivery does not update pending signal mask, which is the same as putting signals into blocked mask, you say that it is not what is
required.
First, what is "pending signal mask"? There is one signal mask per thread. And "pending" refers to thread delivery (either per-process or per-thread) which is not the signal mask (well, for non-RT signals it can be a bitmap but this still is no mask).
Second, I'm not talking about signal delivery. Yes, sigaction allows to specify how the signal mask is to be changed when a signal is delivered.
But this is not what I'm talk about. I'm talking about the signal
mask used for the duration of the kevent_wait syscall, regardless of whether signals are waited for or delivered.
Quoted 2 lines
Signal queue is replaced with kevent queue, and it is in sync with all other kevents.
But the signal mask is something completely different and completely independent from the signal queue. There is nothing in the kevent interface to replace that functionality. Nor should this be possible with the events; only a sigset_t parameter to kevent_wait makes sense.
Quoted 1 line
Having sigmask parameter is the same as creating kevent signal delivery.
No, no, no. Not at all.
Quoted 4 lines
Quoted 1 line
Surely you don't suggest keeping your original timer patch?
Of course not - kevent timers are more scalable than posix timers (the latter uses idr, which is slower than balanced binary tree, since it looks like it uses similar to radix tree algo), POSIX interface is much-much-much more unconvenient to use than simple add/wait.
I assume you misread the question. You agree to drop the patch and then
go on listing things why you think it's better to keep them. I don't
think these arguments are in any way sufficient. The interface is already too big and this is 100% duplicate functionality. If there are performance problems with the POSIX timer implementation (and I have yet to see indications) it should be fixed instead of worked around.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 7 lines · Ulrich Drepper
Quoted 2 lines · Evgeniy Polyakov
It just sets hrtimer with abs time and sleeps - it can achieve the same goals using similar to
wait_event()mechanism.I don't follow. Of course it is somehow possible to wait until an absolute deadline. But it's not part of the parameter list and hence easily and _quickly_ usable.
I just described how it is implemented in futex. I will create the same approach - hrtimer which will wakeup wait_event() with infinite timeout.
Quoted 12 lines
Quoted 6 lines
Quoted 2 lines
Quoted 1 line
Btw, do you propose to change all users of
wait_event()?Which users?
Any users which use
wait_event()orschedule_timeout(). Futex for example - it perfectly ok lives with relative timeouts provided toschedule_timeout()- the same (roughly saying of course) is done in kevent.No, it does not live perfectly OK with relative timeouts. The userlevel implementation is actually wrong because of this in subtle ways. Some futex interfaces take absolute timeouts and they have to be interrupted if the realtime clock is set forward.
Quoted 4 lines
Also, the calls are complicated and slow because the userlevel wrapper has to call clock_gettime/gettimeofday before each futex syscall. If the kernel would accept absolute timeouts as well we would save a syscall and have actually a correct implementation.
It is only done for LOCK_PI case, which was specially created to have absolute timeout, i.e. futex does not need it, but there is an option.
I will extend waiting syscalls to have timespec and absolute timeout, I'm just want to stop this (I hope you agree) stupid endless arguing about completely unimportant thing.
Quoted 7 lines
Quoted 3 lines
I think I said already several times that absolute timeouts are not related to syscall execution process. But you seems to not hear me and insist.
Because you're wrong. For your use cases it might not be but it's not true in general. And your interface is preventing it from being implemented forever.
Because I'm right and it will not be used :)
Well, it does not matter anymore, right?
Quoted 6 lines
Quoted 4 lines
Ok, I will change waiting syscalls to have 'flags' parameter and 'struct timespec' as timeout parameter. Special bit in flags will result in additional timer setup which will fire after absolute timeout and will wake up those who wait...
Thanks a lot.
No problem - I always like to spend couple of month arguing about taste and 'right-from-my-point-of-view' theories - doesn't it the best way to waste the time?
...
Quoted 3 lines
Quoted 1 line
Having sigmask parameter is the same as creating kevent signal delivery.
No, no, no. Not at all.
I've dropped a lot, but let me describe signal mask problem in few
words: signal mask provided in sys_pselect() and friends is a mask of signals, which will be put into blocked mask in the task structure in kernel. When new signal is going to be delivered, signal number is being checked if it is in blocked mask, and if so, signal is not put into pending mask of signals, which ends up in not being delivered to userspace. Kevent (with special flag) does exactly the same - but it does not update blocked mask, but instead adds another check if signal is in kevent set of requests, in that case signal is delivered to userspace through kevent queue.
It is _exactly_ the same behaviour from userspace point of view concerning race of delivery signal versus file descriptor readyness. Exactly.
Here is code snippet:specific_send_sig_info()
{
...
/* Short-circuit ignored signals. */
if (sig_ignored(t, sig))
goto out;
...
ret = send_signal(sig, info, t, &t->pending);
if (!ret && !sigismember(&t->blocked, sig))
signal_wake_up(t, sig == SIGKILL);#ifdef CONFIG_KEVENT_SIGNAL
/*
* Kevent allows to deliver signals through kevent queue,
* it is possible to setup kevent to not deliver
* signal through the usual way, in that case send_signal()
* returns 1 and signal is delivered only through kevent queue.
* We simulate successfull delivery notification through this hack:
*/
if (ret == 1)
ret = 0;
#endif
out:
return ret; }
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
Surely you don't suggest keeping your original timer patch?
Of course not - kevent timers are more scalable than posix timers (the latter uses idr, which is slower than balanced binary tree, since it looks like it uses similar to radix tree algo), POSIX interface is much-much-much more unconvenient to use than simple add/wait.
Quoted 6 lines
I assume you misread the question. You agree to drop the patch and then
go on listing things why you think it's better to keep them. I don't
think these arguments are in any way sufficient. The interface is already too big and this is 100% duplicate functionality. If there are performance problems with the POSIX timer implementation (and I have yet to see indications) it should be fixed instead of worked around.
I do _not_ agree to drop kevent timer patch (not posix timer), since from my point of view it is much more convenient interface, it is more scalable, it is generic enough to be used with other kevent methods.
But anyway, we can spend awfull lot of time arguing about taste, which is definitely _NOT_ what we want. So, there are two worlds - posix timers and usual timers, accessible from userspace, first one through create_timer() and friends, second one with kevent interface.
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
In reply to Ulrich DrepperQuoted 19 lines · Ulrich Drepper
Quoted 1 line · Evgeniy Polyakov
Threads are parked in syscalls - which one should be interrupted?
It doesn't matter, use the same policy you use when waking a thread in case of an event. This is not about waking a specific thread, it's about not dropping the event notification.
Quoted 1 line
And what if there were no threads waiting in syscalls?
This is fine, do nothing. It means that the other threads are about to read the ring buffer and will pick up the event.
The case which must be avoided is that of all threads being in the kernel, one threads gets woken, and then is canceled. Without notifying the kernel about the cancellation and in the absence of further events notifications the process is deadlocked.
Quoted 5 lines
A second case which should be avoided is that there is a thread waiting when a thread gets canceled and there are one or more addition threads around, but not in the kernel. But those other threads might not get to the ring buffer anytime soon, so handling the event is unnecessarily delayed.
Ok, to solve the problem in the way which should be good for both I decided to implement additional syscall which will allow to mark any event as ready and thus wake up appropriate threads. If userspace will request zero events to be marked as ready, syscall will just interrupt/wakeup one of the listeners parked in syscall.
Piece?
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 5 lines · Evgeniy Polyakov
Ok, to solve the problem in the way which should be good for both I decided to implement additional syscall which will allow to mark any event as ready and thus wake up appropriate threads. If userspace will request zero events to be marked as ready, syscall will just interrupt/wakeup one of the listeners parked in syscall.
Btw, what about putting aditional multiplexer into add/remove/modify switch? There will be logical 'ready' addon?
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 9 lines · Evgeniy Polyakov
Quoted 5 lines · Evgeniy Polyakov
Ok, to solve the problem in the way which should be good for both I decided to implement additional syscall which will allow to mark any event as ready and thus wake up appropriate threads. If userspace will request zero events to be marked as ready, syscall will just interrupt/wakeup one of the listeners parked in syscall.
Btw, what about putting aditional multiplexer into add/remove/modify switch? There will be logical 'ready' addon?
Something like this.
include/linux/kevent.h +4 -0
@@ -99,6 +99,8 @@ struct kevent_userstruct mutex ctl_mutex;/* Wait until some events are ready. */ wait_queue_head_t wait; +/* Exit from syscall if someone wants us to do it */ +int need_exit;/* Reference counter, increased for each new kevent. */ atomic_t refcnt;
@@ -132,6 +134,8 @@ void kevent_storage_fini(struct kevent_sint kevent_storage_enqueue (struct kevent_storage *st,struct kevent *k);void kevent_storage_dequeue (struct kevent_storage *st,struct kevent *k); +void kevent_ready (struct kevent *k,int ret); +int kevent_user_add_ukevent (struct ukevent *uk,struct kevent_user *u);#ifdef CONFIG_KEVENT_POLL
include/linux/ukevent.h +1 -0
@@ -174,5 +174,6 @@ struct kevent_ring#define KEVENT_CTL_ADD 0#define KEVENT_CTL_REMOVE 1#define KEVENT_CTL_MODIFY 2 +#define KEVENT_CTL_READY 3#endif /* __UKEVENT_H */
kernel/kevent/kevent.c +17 -10
@@ -91,10 +91,10 @@ int kevent_init(struct kevent *k)spin_lock_init (&k->ulock); k->flags =0 ; - if (unlikely(k->event.type >= KEVENT_MAX) +if (unlikely (k->event.type >=KEVENT_MAX ))return kevent_break (k); - if (!kevent_registered_callbacks[k->event.type].callback)) { +if (!kevent_registered_callbacks[k->event.type].callback) {kevent_break (k);return -ENOSYS ; }
@@ -142,16 +142,10 @@ void kevent_storage_dequeue(struct kevenspin_unlock_irqrestore (&st->lock, flags); } -/* -* Call kevent ready callback and queue it into ready queue if needed. -* If kevent is marked as one-shot, then remove it from storage queue. -*/ -static int __kevent_requeue (struct kevent *k,u32 event) +void kevent_ready (struct kevent *k,int ret) { -int ret, rem;unsigned long flags; - - ret = k->callbacks.callback (k); +int rem;spin_lock_irqsave (&k->ulock, flags);if (ret >0 )
@@ -178,6 +172,19 @@ static int __kevent_requeue(struct kevenspin_unlock_irqrestore (&k->user->ready_lock, flags);wake_up (&k->user->wait); } +} + +/* +* Call kevent ready callback and queue it into ready queue if needed. +* If kevent is marked as one-shot, then remove it from storage queue. +*/ +static int __kevent_requeue (struct kevent *k,u32 event) +{ +int ret; + + ret = k->callbacks.callback (k); + +kevent_ready (k, ret);return ret; }
kernel/kevent/kevent_user.c +101 -3
145 lines not on this page · raw message
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
In reply to Evgeniy PolyakovQuoted 6 lines · Evgeniy Polyakov
Quoted 5 lines · Evgeniy Polyakov
Ok, to solve the problem in the way which should be good for both I decided to implement additional syscall which will allow to mark any event as ready and thus wake up appropriate threads. If userspace will request zero events to be marked as ready, syscall will just interrupt/wakeup one of the listeners parked in syscall.
I'll wait for the new code drop to comment.
Quoted 2 lines
Btw, what about putting aditional multiplexer into add/remove/modify switch? There will be logical 'ready' addon?
Is it needed? Usually this is done with a *_wait call with a timeout of zero. That code path might have to be optimized but it should already be there.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 10 lines · Ulrich Drepper
Quoted 7 lines · Evgeniy Polyakov
Quoted 5 lines · Evgeniy Polyakov
Ok, to solve the problem in the way which should be good for both I decided to implement additional syscall which will allow to mark any event as ready and thus wake up appropriate threads. If userspace will request zero events to be marked as ready, syscall will just interrupt/wakeup one of the listeners parked in syscall.
I'll wait for the new code drop to comment.
I posted it.
Quoted 6 lines
Quoted 2 lines
Btw, what about putting aditional multiplexer into add/remove/modify switch? There will be logical 'ready' addon?
Is it needed? Usually this is done with a
*_waitcall with a timeout of zero. That code path might have to be optimized but it should already be there.
It does not allow to mark events as ready.
And current interfaces wake up when either timeout is zero (in this case thread itself does not sleep and can process events), or when there is _new_ work - since there is no _new_ work, when thread awakened to process it was killed, kernel does not think that something is wrong.
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 9 lines · Evgeniy Polyakov
Quoted 3 lines
Quoted 2 lines
Btw, what about putting aditional multiplexer into add/remove/modify switch? There will be logical 'ready' addon?
Is it needed? Usually this is done with a
*_waitcall with a timeout of zero. That code path might have to be optimized but it should already be there.It does not allow to mark events as ready.
And current interfaces wake up when either timeout is zero (in this case thread itself does not sleep and can process events), or when there is _new_ work - since there is no _new_ work, when thread awakened to process it was killed, kernel does not think that something is wrong.
Rather than mark an existing entry as ready, how about a call to inject a new ready event?
This would be useful to implement functionality at userlevel and still use an event queue to announce the availability. Without this type of functionality we'd need to use indirect notification via signal or pipe or something like that.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 13 lines · Ulrich Drepper
Quoted 11 lines · Evgeniy Polyakov
Quoted 5 lines
Quoted 2 lines
Btw, what about putting aditional multiplexer into add/remove/modify switch? There will be logical 'ready' addon?
Is it needed? Usually this is done with a
*_waitcall with a timeout of zero. That code path might have to be optimized but it should already be there.It does not allow to mark events as ready.
And current interfaces wake up when either timeout is zero (in this case thread itself does not sleep and can process events), or when there is _new_ work - since there is no _new_ work, when thread awakened to process it was killed, kernel does not think that something is wrong.
Quoted 7 lines
Rather than mark an existing entry as ready, how about a call to inject a new ready event?
This would be useful to implement functionality at userlevel and still use an event queue to announce the availability. Without this type of functionality we'd need to use indirect notification via signal or pipe or something like that.
With provided patch it is possible to wakeup 'for-free' - just call kevent_ctl(ready) with zero number of ready events, so thread will be awakened if it was in poll(kevent_fd), kevent_wait() or kevent_get_events().
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 5 lines · Evgeniy Polyakov
With provided patch it is possible to wakeup 'for-free' - just call
kevent_ctl(ready)with zero number of ready events, so thread will be awakened if it was inpoll(kevent_fd),kevent_wait()orkevent_get_events().
Yes, I realize that. But I wrote something else:
Quoted 7 lines
Quoted 7 lines
Rather than mark an existing entry as ready, how about a call to inject a new ready event?
This would be useful to implement functionality at userlevel and still use an event queue to announce the availability. Without this type of functionality we'd need to use indirect notification via signal or pipe or something like that.
This is still something which is wanted.
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 12 lines · Ulrich Drepper
Quoted 5 lines · Evgeniy Polyakov
With provided patch it is possible to wakeup 'for-free' - just call
kevent_ctl(ready)with zero number of ready events, so thread will be awakened if it was inpoll(kevent_fd),kevent_wait()orkevent_get_events().Yes, I realize that. But I wrote something else:
Quoted 3 lines
Quoted 3 lines
Rather than mark an existing entry as ready, how about a call to inject a new ready event?
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
This would be useful to implement functionality at userlevel and still use an event queue to announce the availability. Without this type of functionality we'd need to use indirect notification via signal or pipe or something like that.
This is still something which is wanted.
Why do we want to inject _ready_ event, when it is possible to mark event as ready and wakeup thread parked in syscall?
Quoted 3 lines
Signature · 2 lines
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 2 lines · Evgeniy Polyakov
Why do we want to inject _ready_ event, when it is possible to mark event as ready and wakeup thread parked in syscall?
Going back to this old one:
How do you want to mark an event ready if you don't want to introduce yet another layer of data structures? The event notification happens through entries in the ring buffer. Userlevel code should never add anything to the ring buffer directly, this would mean huge synchronization problems. Yes, one could add additional data structures accompanying the ring buffer which can specify userlevel-generated events. But this is a) clumsy and b) a pain to use when the same ring buffer is used in multiple threads (you'd have to have another shared memory segment).
It's much cleaner if the userlevel code can get the kernel to inject a userlevel-generated event. This is the equivalent of userlevel code generating a signal with kill().
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Re: [take24 0/6] kevent: Generic event handling mechanism.
Quoted 16 lines · Ulrich Drepper
Quoted 2 lines · Evgeniy Polyakov
Why do we want to inject _ready_ event, when it is possible to mark event as ready and wakeup thread parked in syscall?
Going back to this old one:
How do you want to mark an event ready if you don't want to introduce yet another layer of data structures? The event notification happens through entries in the ring buffer. Userlevel code should never add anything to the ring buffer directly, this would mean huge synchronization problems. Yes, one could add additional data structures accompanying the ring buffer which can specify userlevel-generated events. But this is a) clumsy and b) a pain to use when the same ring buffer is used in multiple threads (you'd have to have another shared memory segment).
Quoted 3 lines
It's much cleaner if the userlevel code can get the kernel to inject a userlevel-generated event. This is the equivalent of userlevel code generating a signal with
kill().
Existing possibility to mark event as ready works following way:
event is queued into storage queue (socket, inode or some other queue), when readiness condition becomes true, event is queued into ready queue (although it is still in the storage queueu). It happens completely asynchronosu to _any_ kind of userspace processing.
When userspace calls apropriate syscall, event is being copied into ring buffer.
Thus userspace readiness will just mark event as ready, i.e. it queues event into ready queue, so later usersapce will callsyscall to actually get the event.
When one thread is parked in the syscall and there are _no_ events which should be marked as ready (for example only sockets are there, and it is not a good idea to wakeup the whole socket processing state machine), then there is no possibility to receive such event (although it is possible to interrupt and break syscall).
So, according to injecting ready events, it can be done - just an addition of special flag which will force kevent core to move event into ready queue immediately. In this case userspace can event prepare a needed event (like signal event) and deliver it to process, so it will think (only from kevent point of view) that real signal has been arrived.
I will also add special type of events - userspace events - which will not have empty callbacks, which will be intended to use for user-defined way (i.e. for inter thread communications).
Quoted 3 lines
Signature · 1 line
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
Signature · 1 line
Evgeniy Polyakov
cover kevent: Generic event handling mechanism.
In reply to a message not in the archiveGeneric event handling mechanism.
Kevent is a generic subsytem which allows to handle event notifications. It supports both level and edge triggered events. It is similar to poll/epoll in some cases, but it is more scalable, it is faster and allows to work with essentially eny kind of events.
Events are provided into kernel through control syscall and can be read back through ring buffer or using usual syscalls.
Kevent update (i.e. readiness switching) happens directly from internals of the appropriate state machine of the underlying subsytem (like network, filesystem, timer or any other).
Homepage:
http://tservice.net.ru/~s0mbre/old/?section=projects&item=kevent
Documentation page:
http://linux-net.osdl.org/index.php/Kevent
I installed slightly used, but still functional (bought on ebay) remote mind reader, and set it up to read Ulrich's alpha brain waves (I hope he agrees that it is a good decision), which took me the whole week. So I think the last ring buffer implementation is what we all wanted. Details in documentation part.
Changes from 'take24' patchset:
* new (old (new)) ring buffer imeplementation with kernel and user indexes.
* added initialization syscall instead of opening /dev/kevent
* kevent_commit() syscall to commit ring buffer entries
* changed KEVENT_REQ_WAKEUP_ONE flag to KEVENT_REQ_WAKEUP_ALL, kevent wakes only first thread always if that flag is not set
* KEVENT_REQ_ALWAYS_QUEUE flag. If set, kevent will be queued into ready queue instead of copying back to userspace when kevent is ready immediately when it is added.
* lighttpd patch (Hail! Although nothing realy outstanding compared to epoll)
Changes from 'take23' patchset:
* kevent PIPE notifications
* KEVENT_REQ_LAST_CHECK flag, which allows to perform last check at dequeueing time
* fixed poll/select notifications (were broken due to tree manipulations)
* made Documentation/kevent.txt look nice in 80-col terminal
* fix for copy_to_user() failure report for the first kevent (Andrew Morton)
* minor function renames
Changes from 'take22' patchset:
* new ring buffer implementation in process' memory
* wakeup-one-thread flag
* edge-triggered behaviour
Changes from 'take21' patchset:
* minor cleanups (different return values, removed unneded variables, whitespaces and so on)
* fixed bug in kevent removal in case when kevent being removed
is the same as overflow_kevent (spotted by Eric Dumazet)
Changes from 'take20' patchset:
* new ring buffer implementation
* removed artificial limit on possible number of kevents
Changes from 'take19' patchset:
* use __init instead of __devinit
* removed 'default N' from config for user statistic
* removed kevent_user_fini() since kevent can not be unloaded
* use KERN_INFO for statistic output
Changes from 'take18' patchset:
* use __init instead of __devinit
* removed 'default N' from config for user statistic
* removed kevent_user_fini() since kevent can not be unloaded
* use KERN_INFO for statistic output
Changes from 'take17' patchset:
* Use RB tree instead of hash table.
At least for a web sever, frequency of addition/deletion of new kevent is comparable with number of search access, i.e. most of the time events are added, accesed only couple of times and then removed, so it justifies RB tree usage over AVL tree, since the latter does have much slower deletion time (max O(log(N)) compared to 3 ops),
although faster search time (1.44*O(log(N)) vs. 2*O(log(N))).
So for kevents I use RB tree for now and later, when my AVL tree implementation is ready, it will be possible to compare them.
* Changed readiness check for socket notifications.
With both above changes it is possible to achieve more than 3380 req/second compared to 2200, sometimes 2500 req/second for epoll() for trivial web-server and httperf client on the same hardware.
It is possible that above kevent limit is due to maximum allowed kevents in a time limit, which is 4096 events.
Changes from 'take16' patchset:
* misc cleanups (__read_mostly, const ...)
* created special macro which is used for mmap size (number of pages) calculation
* export kevent_socket_notify(), since it is used in network protocols which can be built as modules (IPv6 for example)
Changes from 'take15' patchset:
* converted kevent_timer to high-resolution timers, this forces timer API update at
http://linux-net.osdl.org/index.php/Kevent
* use struct ukevent* instead of void * in syscalls (documentation has been updated)
* added warning in kevent_add_ukevent() if ring has broken index (for testing)
Changes from 'take14' patchset:
* added kevent_wait()
This syscall waits until either timeout expires or at least one event becomes ready. It also commits that @num events from @start are processed by userspace and thus can be be removed or rearmed (depending on it's flags). It can be used for commit events read by userspace through mmap interface. Example userspace code (evtest.c) can be found on project's homepage.
* added socket notifications (send/recv/accept)
Changes from 'take13' patchset:
* do not get lock aroung user data check in __kevent_search()
* fail early if there were no registered callbacks for given type of kevent
* trailing whitespace cleanup
Changes from 'take12' patchset:
* remove non-chardev interface for initialization
* use pointer to kevent_mring instead of unsigned longs
* use aligned 64bit type in raw user data (can be used by high-res timer if needed)
* simplified enqueue/dequeue callbacks and kevent initialization
* use nanoseconds for timeout
* put number of milliseconds into timer's return data
* move some definitions into user-visible header
* removed filenames from comments
Changes from 'take11' patchset:
* include missing headers into patchset
* some trivial code cleanups (use goto instead of if/else games and so on)
* some whitespace cleanups
* check for ready_callback() callback before main loop which should save us some ticks
Changes from 'take10' patchset:
* removed non-existent prototypes
* added helper function for kevent_registered_callbacks
* fixed 80 lines comments issues
* added shared between userspace and kernelspace header instead of embedd them in one
* core restructuring to remove forward declarations
* s o m e w h i t e s p a c e c o d y n g s t y l e c l e a n u p
* use vm_insert_page() instead of remap_pfn_range()
Changes from 'take9' patchset:
* fixed ->nopage method
Changes from 'take8' patchset:
* fixed mmap release bug
* use module_init() instead of late_initcall()
* use better structures for timer notifications
Changes from 'take7' patchset:
* new mmap interface (not tested, waiting for other changes to be acked)
- use nopage() method to dynamically substitue pages
- allocate new page for events only when new added kevent requres it
- do not use ugly index dereferencing, use structure instead
- reduced amount of data in the ring (id and flags), maximum 12 pages on x86 per kevent fd
Changes from 'take6' patchset:
* a lot of comments!
* do not use list poisoning for detection of the fact, that entry is in the list
* return number of ready kevents even if copy*user() fails
* strict check for number of kevents in syscall
* use ARRAY_SIZE for array size calculation
* changed superblock magic number
* use SLAB_PANIC instead of direct panic() call
* changed -E* return values
* a lot of small cleanups and indent fixes
Changes from 'take5' patchset:
* removed compilation warnings about unused wariables when lockdep is not turned on
* do not use internal socket structures, use appropriate (exported) wrappers instead
* removed default 1 second timeout
* removed AIO stuff from patchset
Changes from 'take4' patchset:
* use miscdevice instead of chardevice
* comments fixes
Changes from 'take3' patchset:
* removed serializing mutex from kevent_user_wait()
* moved storage list processing to RCU
* removed lockdep screaming - all storage locks are initialized in the same function, so it was
learned
to differentiate between various cases
* remove kevent from storage if is marked as broken after callback
* fixed a typo in mmaped buffer implementation which would end up in wrong index calcualtion
Changes from 'take2' patchset:
* split kevent_finish_user() to locked and unlocked variants
* do not use KEVENT_STAT ifdefs, use inline functions instead
* use array of callbacks of each type instead of each kevent callback initialization
* changed name of ukevent guarding lock
* use only one kevent lock in kevent_user for all hash buckets instead of per-bucket locks
* do not use kevent_user_ctl structure instead provide needed arguments as syscall parameters
* various indent cleanups
* added optimisation, which is aimed to help when a lot of kevents are being copied from
userspace
* mapped buffer (initial) implementation (no userspace yet)
Changes from 'take1' patchset:
- rebased against 2.6.18-git tree
- removed ioctl controlling
- added new syscall kevent_get_events(int fd, unsigned int min_nr, unsigned int max_nr, unsigned int timeout, void __user *buf, unsigned flags)
- use old syscall kevent_ctl for creation/removing, modification and initial kevent initialization
- use mutuxes instead of semaphores
- added file descriptor check and return error if provided descriptor does not match kevent file operations
- various indent fixes
- removed aio_sendfile() declarations.
Thank you.