fsnotify: Do not share events between notification groups
Subsystems (6)
Jan Kara
maintainerPATCH 1/1 fsnotify: Do not share events between notification groups
Currently fsnotify framework creates one event structure for each notification event and links this event into all interested notification groups. This is done so that we save memory when several notification groups are interested in the event. However the need for event structure shared between inotify & fanotify bloats the event structure so the result is often higher memory consumption.
Another problem is that fsnotify framework keeps path references with outstanding events so that fanotify can return open file descriptors with its events. This has the undesirable effect that filesystem cannot be unmounted while there are outstanding events - a regression for inotify compared to a situation before it was converted to fsnotify framework. For fanotify this problem is hard to avoid and users of fanotify should kind of expect this behavior when they ask for file descriptors from notified files.
This patch changes fsnotify and its users to create separate event structure for each group. This allows for much simpler code (~450 lines removed by this patch) and also smaller event structures. For example on 64-bit system original struct fsnotify_event consumes 120 bytes, plus additional space for file name, additional 24 bytes for second and each subsequent group linking the event, and additional 32 bytes for each inotify group for private data. After the conversion inotify event consumes 48 bytes plus space for file name which is considerably less memory unless file names are long and there are several groups interested in the events (both of which are uncommon). Fanotify event fits in 56 bytes after the conversion (fanotify doesn't care about file names so its events don't have to have it allocated). A win unless there are four or more fanotify groups interested in the event.
The conversion also solves the problem with unmount when only inotify is used as we don't have to grab path references for inotify events.
--- fs/notify/dnotify/dnotify.c | 33 +--- fs/notify/fanotify/fanotify.c | 221 ++++++++++++----------- fs/notify/fanotify/fanotify.h | 18 ++ fs/notify/fanotify/fanotify_user.c | 33 ++-- fs/notify/fsnotify.c | 42 ++--- fs/notify/group.c | 1 + fs/notify/inotify/inotify.h | 21 ++- fs/notify/inotify/inotify_fsnotify.c | 144 ++++++--------- fs/notify/inotify/inotify_user.c | 119 ++++--------- fs/notify/notification.c | 330 +++-------------------------------- include/linux/fsnotify_backend.h | 118 +++---------- kernel/audit_tree.c | 20 +-- kernel/audit_watch.c | 26 ++- 13 files changed, 340 insertions(+), 786 deletions(-) create mode 100644 fs/notify/fanotify/fanotify.h
Guys, what do you think? The patch definitely needs more testing but it survives some basic tests. Also it's rather large so I can try to split it a bit but since this is an infrastructure change it's hard to make it in small steps...
fs/notify/dnotify/dnotify.c +9 -24
@@ -82,20 +82,23 @@ static void dnotify_recalc_inode_mask(struct fsnotify_mark *fsn_mark)* events. */ static int dnotify_handle_event (struct fsnotify_group *group, +struct inode *to_tell,struct fsnotify_mark *inode_mark,struct fsnotify_mark *vfsmount_mark, -struct fsnotify_event *event) +u32 mask,void *data,int data_type, +const unsigned char *file_name) {struct dnotify_mark *dn_mark; -struct inode *to_tell;struct dnotify_struct *dn;struct dnotify_struct **prev;struct fown_struct *fown; -__u32 test_mask = event->mask & ~FS_EVENT_ON_CHILD ; +__u32 test_mask = mask & ~FS_EVENT_ON_CHILD ; -BUG_ON (vfsmount_mark); +/* not a dir, dnotify doesn't care */ +if (!S_ISDIR (to_tell->i_mode)) +return 0 ; - to_tell = event->to_tell; +BUG_ON (vfsmount_mark); dn_mark = container_of(inode_mark, struct dnotify_mark, fsn_mark);
@@ -122,23 +125,6 @@ static int dnotify_handle_event(struct fsnotify_group *group,return 0 ; } -/* -* Given an inode and mask determine if dnotify would be interested in sending -* userspace notification for that pair. -*/ -static bool dnotify_should_send_event (struct fsnotify_group *group, -struct inode *inode, -struct fsnotify_mark *inode_mark, -struct fsnotify_mark *vfsmount_mark, -__u32 mask,void *data,int data_type) -{ -/* not a dir, dnotify doesn't care */ -if (!S_ISDIR (inode->i_mode)) -return false; - -return true; -} -static void dnotify_free_mark (struct fsnotify_mark *fsn_mark) { struct dnotify_mark *dn_mark = container_of(fsn_mark,
@@ -152,10 +138,9 @@ static void dnotify_free_mark(struct fsnotify_mark *fsn_mark)static struct fsnotify_ops dnotify_fsnotify_ops = { .handle_event = dnotify_handle_event, - .should_send_event = dnotify_should_send_event, .free_group_priv =NULL , .freeing_mark =NULL , - .free_event_priv =NULL , + .free_event =NULL , };/*
fs/notify/fanotify/fanotify.c +110 -111
288 lines not on this page · raw message
fs/notify/fanotify/fanotify.h new file +18 -0
@@ -0,0 +1,18 @@ +#include <linux/fsnotify_backend.h> +#include <linux/path.h> +#include <linux/slab.h> + +extern struct kmem_cache *fanotify_event_cachep; + +struct fanotify_event_info { +struct fsnotify_event fse; +/* +* We hold ref to this path so it may be dereferenced at any point +* during this object's lifetime +*/ +struct path path; +struct pid *tgid; +#ifdef CONFIG_FANOTIFY_ACCESS_PERMISSIONS +u32 response;/* userspace answer to question */ +#endif +};
fs/notify/fanotify/fanotify_user.c +16 -17
@@ -19,6 +19,7 @@#include "../../mount.h" #include "../fdinfo.h" +#include "fanotify.h" #define FANOTIFY_DEFAULT_MAX_EVENTS 16384#define FANOTIFY_DEFAULT_MAX_MARKS 8192
@@ -28,6 +29,7 @@ extern const struct fsnotify_ops fanotify_fsnotify_ops;static struct kmem_cache *fanotify_mark_cache__read_mostly ;static struct kmem_cache *fanotify_response_event_cache__read_mostly ; +struct kmem_cache *fanotify_event_cachep__read_mostly ;struct fanotify_response_event {struct list_head list;
@@ -61,8 +63,8 @@ static struct fsnotify_event *get_one_event(struct fsnotify_group *group, }static int create_fd (struct fsnotify_group *group, -struct fsnotify_event *event, -struct file **file) +struct fanotify_event_info *event, +struct file **file) {int client_fd;struct file *new_file;
@@ -73,12 +75,6 @@ static int create_fd(struct fsnotify_group *group,if (client_fd <0 )return client_fd; -if (event->data_type !=FSNOTIFY_EVENT_PATH ) { -WARN_ON (1 ); -put_unused_fd (client_fd); -return -EINVAL ; - } -/* * we need a new file handle for the userspace program so it can read even if it was * originally opened O_WRONLY.
@@ -109,23 +105,25 @@ static int create_fd(struct fsnotify_group *group, }static int fill_event_metadata (struct fsnotify_group *group, -struct fanotify_event_metadata *metadata, -struct fsnotify_event *event, -struct file **file) +struct fanotify_event_metadata *metadata, +struct fsnotify_event *fsn_event, +struct file **file) {int ret =0 ; +struct fanotify_event_info *event;pr_debug ("%s: group=%p metadata=%p event=%p\n" , __func__, - group, metadata, event); + group, metadata, fsn_event); *file =NULL ; + event = container_of(fsn_event, struct fanotify_event_info, fse); metadata->event_len =FAN_EVENT_METADATA_LEN ; metadata->metadata_len =FAN_EVENT_METADATA_LEN ; metadata->vers =FANOTIFY_METADATA_VERSION ; metadata->reserved =0 ; - metadata->mask = event->mask &FAN_ALL_OUTGOING_EVENTS ; + metadata->mask = fsn_event->mask &FAN_ALL_OUTGOING_EVENTS ; metadata->pid =pid_vnr (event->tgid); -if (unlikely (event->mask &FAN_Q_OVERFLOW )) +if (unlikely (fsn_event->mask &FAN_Q_OVERFLOW )) metadata->fd =FAN_NOFD ;else { metadata->fd =create_fd (group, event, file);
@@ -321,7 +319,7 @@ static ssize_t fanotify_read(struct file *file, char __user *buf,if (IS_ERR (kevent))break ; ret =copy_event_to_user (group, kevent, buf); -fsnotify_put_event (kevent); +fsnotify_destroy_event (group, kevent);if (ret <0 )break ; buf += ret;
@@ -409,7 +407,7 @@ static int fanotify_release(struct inode *ignored, struct file *file)static long fanotify_ioctl (struct file *file,unsigned int cmd,unsigned long arg) {struct fsnotify_group *group; -struct fsnotify_event_holder *holder; +struct fsnotify_event *fsn_event;void __user *p;int ret = -ENOTTY ;size_t send_len =0 ;
@@ -421,7 +419,7 @@ static long fanotify_ioctl(struct file *file, unsigned int cmd, unsigned long arswitch (cmd) {case FIONREAD :mutex_lock (&group->notification_mutex); -list_for_each_entry (holder, &group->notification_list, event_list) +list_for_each_entry (fsn_event, &group->notification_list, list) send_len +=FAN_EVENT_METADATA_LEN ;mutex_unlock (&group->notification_mutex); ret =put_user (send_len, (int __user *) p);
@@ -906,6 +904,7 @@ static int __init fanotify_user_setup(void) fanotify_mark_cache =KMEM_CACHE (fsnotify_mark,SLAB_PANIC ); fanotify_response_event_cache =KMEM_CACHE (fanotify_response_event,SLAB_PANIC ); + fanotify_event_cachep =KMEM_CACHE (fanotify_event_info,SLAB_PANIC );return 0 ; }
fs/notify/fsnotify.c +12 -30
84 lines not on this page · raw message
fs/notify/group.c +1 -0
@@ -99,6 +99,7 @@ struct fsnotify_group *fsnotify_alloc_group(const struct fsnotify_ops *ops)INIT_LIST_HEAD (&group->marks_list); group->ops = ops; +fsnotify_init_event (&group->overflow_event,NULL ,FS_Q_OVERFLOW );return group; }
fs/notify/inotify/inotify.h +16 -5
@@ -2,11 +2,12 @@#include <linux/inotify.h> #include <linux/slab.h> /* struct kmem_cache */ -extern struct kmem_cache *event_priv_cachep; - -struct inotify_event_private_data { -struct fsnotify_event_private_data fsnotify_event_priv_data; +struct inotify_event_info { +struct fsnotify_event fse;int wd; +u32 sync_cookie; +int name_len; +char name[]; };struct inotify_inode_mark {
@@ -14,8 +15,18 @@ struct inotify_inode_mark {int wd; }; +static inline struct inotify_event_info *INOTIFY_E (struct fsnotify_event *fse) +{ + return container_of(fse, struct inotify_event_info, fse); +} +extern void inotify_ignored_and_remove_idr (struct fsnotify_mark *fsn_mark,struct fsnotify_group *group); -extern void inotify_free_event_priv (struct fsnotify_event_private_data *event_priv); +extern int inotify_handle_event (struct fsnotify_group *group, +struct inode *to_tell, +struct fsnotify_mark *inode_mark, +struct fsnotify_mark *vfsmount_mark, +u32 mask,void *data,int data_type, +const unsigned char *file_name);extern const struct fsnotify_ops inotify_fsnotify_ops;
fs/notify/inotify/inotify_fsnotify.c +51 -93
195 lines not on this page · raw message
fs/notify/inotify/inotify_user.c +38 -81
197 lines not on this page · raw message
fs/notify/notification.c +26 -304
414 lines not on this page · raw message
include/linux/fsnotify_backend.h +27 -91
172 lines not on this page · raw message
kernel/audit_tree.c +6 -14
@@ -912,12 +912,13 @@ static void evict_chunk(struct audit_chunk *chunk) }static int audit_tree_handle_event (struct fsnotify_group *group, +struct inode *to_tell,struct fsnotify_mark *inode_mark, -struct fsnotify_mark *vfsmonut_mark, -struct fsnotify_event *event) +struct fsnotify_mark *vfsmount_mark, +u32 mask,void *data,int data_type, +const unsigned char *file_name) { -BUG (); -return -EOPNOTSUPP ; +return 0 ; }static void audit_tree_freeing_mark (struct fsnotify_mark *entry,struct fsnotify_group *group)
@@ -933,19 +934,10 @@ static void audit_tree_freeing_mark(struct fsnotify_mark *entry, struct fsnotifyBUG_ON (atomic_read (&entry->refcnt) <1 ); } -static bool audit_tree_send_event (struct fsnotify_group *group,struct inode *inode, -struct fsnotify_mark *inode_mark, -struct fsnotify_mark *vfsmount_mark, -__u32 mask,void *data,int data_type) -{ -return false; -} -static const struct fsnotify_ops audit_tree_ops = { .handle_event = audit_tree_handle_event, - .should_send_event = audit_tree_send_event, .free_group_priv =NULL , - .free_event_priv =NULL , + .free_event =NULL , .freeing_mark = audit_tree_freeing_mark, };
kernel/audit_watch.c +10 -16
@@ -465,35 +465,30 @@ void audit_remove_watch_rule(struct audit_krule *krule) } } -static bool audit_watch_should_send_event (struct fsnotify_group *group,struct inode *inode, -struct fsnotify_mark *inode_mark, -struct fsnotify_mark *vfsmount_mark, -__u32 mask,void *data,int data_type) -{ -return true; -} -/* Update watch data in audit rules based on fsnotify events. */ static int audit_watch_handle_event (struct fsnotify_group *group, +struct inode *to_tell,struct fsnotify_mark *inode_mark,struct fsnotify_mark *vfsmount_mark, -struct fsnotify_event *event) +u32 mask,void *data,int data_type, +const unsigned char *file_name) {struct inode *inode; -__u32 mask = event->mask; -const char *dname = event->file_name; +const char *dname =fsn_name_ptr (&event->file_name);struct audit_parent *parent; parent = container_of(inode_mark, struct audit_parent, mark);BUG_ON (group != audit_watch_group); -switch (event->data_type) { +switch (data_type) {case (FSNOTIFY_EVENT_PATH ): - inode = event->path.dentry->d_inode; +struct path *path = data; + + inode = path->dentry->d_inode;break ;case (FSNOTIFY_EVENT_INODE ): - inode = event->inode; + inode = (struct inode *)data;break ;default :BUG ();
@@ -512,11 +507,10 @@ static int audit_watch_handle_event(struct fsnotify_group *group, }static const struct fsnotify_ops audit_watch_fsnotify_ops = { - .should_send_event = audit_watch_should_send_event, .handle_event = audit_watch_handle_event, .free_group_priv =NULL , .freeing_mark =NULL , - .free_event_priv =NULL , + .free_event =NULL , };static int __init audit_watch_init (void )
Signature · 1 line
1.8.1.4
Quoted 7 lines · On Thu, 14 Nov 2013 01:41:23 +0100 Jan Kara <[email protected]> wrote:
13 files changed, 340 insertions(+), 786 deletions(-) create mode 100644 fs/notify/fanotify/fanotify.hGuys, what do you think? The patch definitely needs more testing but it survives some basic tests. Also it's rather large so I can try to split it a bit but since this is an infrastructure change it's hard to make it in small steps...
It's certainly a large patch, but the description makes it sound good ;)
What have you been using to test it? Do we actually have a decent *notify test suite, or does everyone hack up their own, then let it rot?
Jan Kara
maintainerQuoted 10 lines · Andrew Morton
Quoted 7 lines · On Thu, 14 Nov 2013 01:41:23 +0100 Jan Kara <[email protected]> wrote:
13 files changed, 340 insertions(+), 786 deletions(-) create mode 100644 fs/notify/fanotify/fanotify.hGuys, what do you think? The patch definitely needs more testing but it survives some basic tests. Also it's rather large so I can try to split it a bit but since this is an infrastructure change it's hard to make it in small steps...
Quoted 5 lines
It's certainly a large patch, but the description makes it sound good ;)
What have you been using to test it? Do we actually have a decent *notify test suite, or does everyone hack up their own, then let it rot?
Today, I've submitted a test coverage of fanotify syscalls to LTP :).
inotify syscalls already have a coverage there so I tested them as well. I didn't really test dnotify or audit (besides compilation) but changes there were pretty minimal. Testing caught some minor issues so I'll be sending V2 today or tomorrow (I want to try to split the patch a bit).
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR