fsnotify, dnotify, and inotify
Subsystems (8)
Eric Paris
maintainercover fsnotify, dnotify, and inotify
The following series implements a new generic in kernel filesystem notification system, fsnotify. On top of fsnotify I reimplement dnotify and inotify. I have not finished with the change from inotify although I think inotify_user should be completed. In kernel inotify users (aka audit) still (until I get positive feedback) relay on the old inotify backend. This can be 'easily' fixed.
All of this is in preperation for fanotify and using fanotify as an on access
file scanner. So you better know it's coming.
Why is this useful? Because I actually shrank the struct inode. That's right, my code is smaller and faster. Eat that.
struct inode went from:
unsigned long i_dnotify_mask; /* Directory notify events */
struct dnotify_struct *i_dnotify; /* for directory notifications */
struct list_head inotify_watches; /* watches on this inode */
struct mutex inotify_mutex; /* protects the watches list */to:
__u64 i_fsnotify_mask; /* all events this inode cares about */
struct list_head i_fsnotify_mark_entries; /* fsnotify mark entries */
This is just an RFC as I haven't tested it heavily I might still be able to split patches. I think there are some problem at the moment trying to build fsnotify out. I need to send patch 3 and 4 seperately. Actually I think it's perfectly reasonable to take patches 1-4 as is right now.
if you ever want to see more up2date patches check out:
http://people.redhat.com/~eparis/fsnotify
---
Eric Paris (14):
shit on top for debugging
inotify: reimplement inotify using fsnotify
fsnotify: add correlations between events
fsnotify: include pathnames with entries when possible
fsnotify: generic notification queue and waitq
dnotify: reimplement dnotify using fsnotify
fsnotify: parent event notification
fsnotify: add in inode fsnotify markings
fsnotify: add group priorities
fsnotify: unified filesystem notification backend
fsnotify: use the new open-exec hook for inotify and dnotify
fsnotify: sys_execve and sys_uselib do not call into fsnotify
fsnotify: pass a file instead of an inode to open, read, and write filesystem notification: create fs/notify to contain all fs notification
fs/Kconfig | 39 - fs/Makefile | 5 fs/compat.c | 5 fs/dnotify.c | 194 ------- fs/exec.c | 5 fs/fcntl.c | 4 fs/inode.c | 7 fs/inotify.c | 913 ---------------------------------- fs/inotify_user.c | 778 ----------------------------- fs/nfsd/vfs.c | 4 fs/notify/Kconfig | 14 + fs/notify/Makefile | 4 fs/notify/dnotify/Kconfig | 11 fs/notify/dnotify/Makefile | 1 fs/notify/dnotify/dnotify.c | 350 +++++++++++++ fs/notify/fsnotify.c | 92 +++ fs/notify/fsnotify.h | 115 ++++ fs/notify/group.c | 222 ++++++++ fs/notify/inode_mark.c | 279 ++++++++++ fs/notify/inotify/Kconfig | 31 + fs/notify/inotify/Makefile | 2 fs/notify/inotify/inotify.c | 913 ++++++++++++++++++++++++++++++++++ fs/notify/inotify/inotify.h | 117 ++++ fs/notify/inotify/inotify_fsnotify.c | 190 +++++++ fs/notify/inotify/inotify_kernel.c | 299 +++++++++++ fs/notify/inotify/inotify_user.c | 502 +++++++++++++++++++ fs/notify/notification.c | 279 ++++++++++ fs/open.c | 2 fs/read_write.c | 8 include/linux/dcache.h | 3 include/linux/dnotify.h | 29 - include/linux/fs.h | 6 include/linux/fsnotify.h | 271 ++++++++-- include/linux/fsnotify_backend.h | 152 ++++++ include/linux/inotify.h | 1 35 files changed, 3818 insertions(+), 2029 deletions(-) delete mode 100644 fs/dnotify.c delete mode 100644 fs/inotify.c delete mode 100644 fs/inotify_user.c create mode 100644 fs/notify/Kconfig create mode 100644 fs/notify/Makefile create mode 100644 fs/notify/dnotify/Kconfig create mode 100644 fs/notify/dnotify/Makefile create mode 100644 fs/notify/dnotify/dnotify.c create mode 100644 fs/notify/fsnotify.c create mode 100644 fs/notify/fsnotify.h create mode 100644 fs/notify/group.c create mode 100644 fs/notify/inode_mark.c create mode 100644 fs/notify/inotify/Kconfig create mode 100644 fs/notify/inotify/Makefile create mode 100644 fs/notify/inotify/inotify.c create mode 100644 fs/notify/inotify/inotify.h create mode 100644 fs/notify/inotify/inotify_fsnotify.c create mode 100644 fs/notify/inotify/inotify_kernel.c create mode 100644 fs/notify/inotify/inotify_user.c create mode 100644 fs/notify/notification.c create mode 100644 include/linux/fsnotify_backend.h
Eric Paris
maintainerRFC PATCH 1/14 filesystem notification: create fs/notify to contain all fs notification
Adding yet another filesystem notification system it seemed like a good idea to clean up fs/ by creating an fs/notify and putting everything there.
---
fs/Kconfig | 39 -- fs/Makefile | 5 fs/dnotify.c | 194 -------- fs/inotify.c | 913 -------------------------------------- fs/inotify_user.c | 778 -------------------------------- fs/notify/Kconfig | 2 fs/notify/Makefile | 2 fs/notify/dnotify/Kconfig | 10 fs/notify/dnotify/Makefile | 1 fs/notify/dnotify/dnotify.c | 194 ++++++++ fs/notify/inotify/Kconfig | 27 + fs/notify/inotify/Makefile | 2 fs/notify/inotify/inotify.c | 913 ++++++++++++++++++++++++++++++++++++++ fs/notify/inotify/inotify_user.c | 778 ++++++++++++++++++++++++++++++++ 14 files changed, 1931 insertions(+), 1927 deletions(-) delete mode 100644 fs/dnotify.c delete mode 100644 fs/inotify.c delete mode 100644 fs/inotify_user.c create mode 100644 fs/notify/Kconfig create mode 100644 fs/notify/Makefile create mode 100644 fs/notify/dnotify/Kconfig create mode 100644 fs/notify/dnotify/Makefile create mode 100644 fs/notify/dnotify/dnotify.c create mode 100644 fs/notify/inotify/Kconfig create mode 100644 fs/notify/inotify/Makefile create mode 100644 fs/notify/inotify/inotify.c create mode 100644 fs/notify/inotify/inotify_user.c
fs/Kconfig +1 -38
@@ -270,44 +270,7 @@ config OCFS2_COMPAT_JBD endif # BLOCK -config DNOTIFY - bool "Dnotify support" - default y - help - Dnotify is a directory-based per-fd file change notification system - that uses signals to communicate events to user-space. There exist - superior alternatives, but some applications may still rely on - dnotify. - - If unsure, say Y. - -config INOTIFY - bool "Inotify file change notification support" - default y - ---help--- - Say Y here to enable inotify support. Inotify is a file change - notification system and a replacement for dnotify. Inotify fixes - numerous shortcomings in dnotify and introduces several new features - including multiple file events, one-shot support, and unmount - notification. - - For more information, see <file:Documentation/filesystems/inotify.txt> - - If unsure, say Y. - -config INOTIFY_USER - bool "Inotify support for userspace" - depends on INOTIFY - default y - ---help--- - Say Y here to enable inotify support for userspace, including the - associated system calls. Inotify allows monitoring of both files and - directories via a single open fd. Events are read from the file - descriptor, which is also select()- and poll()-able. - - For more information, see <file:Documentation/filesystems/inotify.txt> - - If unsure, say Y. +source "fs/notify/Kconfig" config QUOTA bool "Quota support"
fs/Makefile +1 -4
@@ -20,8 +20,7 @@ obj-y += no-block.o endif obj-$(CONFIG_BLK_DEV_INTEGRITY) += bio-integrity.o -obj-$(CONFIG_INOTIFY) += inotify.o -obj-$(CONFIG_INOTIFY_USER) += inotify_user.o +obj-y += notify/ obj-$(CONFIG_EPOLL) += eventpoll.o obj-$(CONFIG_ANON_INODES) += anon_inodes.o obj-$(CONFIG_SIGNALFD) += signalfd.o
@@ -57,8 +56,6 @@ obj-$(CONFIG_QFMT_V1) += quota_v1.o obj-$(CONFIG_QFMT_V2) += quota_v2.o obj-$(CONFIG_QUOTACTL) += quota.o -obj-$(CONFIG_DNOTIFY) += dnotify.o - obj-$(CONFIG_PROC_FS) += proc/ obj-y += partitions/ obj-$(CONFIG_SYSFS) += sysfs/
fs/notify/Kconfig new file +2 -0
@@ -0,0 +1,2 @@ +source "fs/notify/dnotify/Kconfig" +source "fs/notify/inotify/Kconfig"
fs/notify/Makefile new file +2 -0
@@ -0,0 +1,2 @@ +obj-y += dnotify/ +obj-y += inotify/
fs/notify/dnotify/Kconfig new file +10 -0
@@ -0,0 +1,10 @@ +config DNOTIFY + bool "Dnotify support" + default y + help + Dnotify is a directory-based per-fd file change notification system + that uses signals to communicate events to user-space. There exist + superior alternatives, but some applications may still rely on + dnotify. + + If unsure, say Y.
fs/notify/dnotify/Makefile new file +1 -0
@@ -0,0 +1 @@ +obj-$(CONFIG_DNOTIFY) += dnotify.o
fs/notify/dnotify/dnotify.c renamed from fs/dnotify.c +0 -0
fs/notify/inotify/Kconfig new file +27 -0
@@ -0,0 +1,27 @@ +config INOTIFY + bool "Inotify file change notification support" + default y + ---help--- + Say Y here to enable inotify support. Inotify is a file change + notification system and a replacement for dnotify. Inotify fixes + numerous shortcomings in dnotify and introduces several new features + including multiple file events, one-shot support, and unmount + notification. + + For more information, see <file:Documentation/filesystems/inotify.txt> + + If unsure, say Y. + +config INOTIFY_USER + bool "Inotify support for userspace" + depends on INOTIFY + default y + ---help--- + Say Y here to enable inotify support for userspace, including the + associated system calls. Inotify allows monitoring of both files and + directories via a single open fd. Events are read from the file + descriptor, which is also select()- and poll()-able. + + For more information, see <file:Documentation/filesystems/inotify.txt> + + If unsure, say Y.
fs/notify/inotify/Makefile new file +2 -0
@@ -0,0 +1,2 @@ +obj-$(CONFIG_INOTIFY) += inotify.o +obj-$(CONFIG_INOTIFY_USER) += inotify_user.o
fs/notify/inotify/inotify.c renamed from fs/inotify.c +0 -0
fs/notify/inotify/inotify_user.c renamed from fs/inotify_user.c +0 -0
Eric Paris
maintainerRFC PATCH 2/14 fsnotify: pass a file instead of an inode to open, read, and write
In reply to Eric Parisfanotify, the upcoming notification system actually needs a f_path so it can do opens in the context of listeners, and it needs a file so it can get f_flags from the original process. Close already was passing a file.
---
fs/compat.c | 5 ++--- fs/nfsd/vfs.c | 4 ++-- fs/open.c | 2 +- fs/read_write.c | 8 ++++---- include/linux/fsnotify.h | 9 ++++++--- 5 files changed, 15 insertions(+), 13 deletions(-)
fs/compat.c +2 -3
@@ -1158,11 +1158,10 @@ out:if (iov != iovstack)kfree (iov);if ((ret + (type ==READ )) >0 ) { -struct dentry *dentry = file->f_path.dentry;if (type ==READ ) -fsnotify_access (dentry); +fsnotify_access (file);else -fsnotify_modify (dentry); +fsnotify_modify (file); }return ret; }
fs/nfsd/vfs.c +2 -2
@@ -940,7 +940,7 @@ nfsd_vfs_read(struct svc_rqst *rqstp, struct svc_fh *fhp, struct file *file, nfsdstats.io_read += host_err; *count = host_err; err =0 ; -fsnotify_access (file->f_path.dentry); +fsnotify_access (file); }else err =nfserrno (host_err); out:
@@ -1007,7 +1007,7 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct file *file,set_fs (oldfs);if (host_err >=0 ) { nfsdstats.io_write += cnt; -fsnotify_modify (file->f_path.dentry); +fsnotify_modify (file); }/* clear setuid/setgid flag after write */
fs/open.c +1 -1
@@ -1020,7 +1020,7 @@ long do_sys_open(int dfd, const char __user *filename, int flags, int mode)put_unused_fd (fd); fd =PTR_ERR (f); }else { -fsnotify_open (f->f_path.dentry); +fsnotify_open (f);fd_install (fd, f); } }
fs/read_write.c +4 -4
@@ -280,7 +280,7 @@ ssize_t vfs_read(struct file *file, char __user *buf, size_t count, loff_t *pos)else ret =do_sync_read (file, buf, count, pos);if (ret >0 ) { -fsnotify_access (file->f_path.dentry); +fsnotify_access (file);add_rchar (current, ret); }inc_syscr (current);
@@ -335,7 +335,7 @@ ssize_t vfs_write(struct file *file, const char __user *buf, size_t count, loff_else ret =do_sync_write (file, buf, count, pos);if (ret >0 ) { -fsnotify_modify (file->f_path.dentry); +fsnotify_modify (file);add_wchar (current, ret); }inc_syscw (current);
@@ -626,9 +626,9 @@ out:kfree (iov);if ((ret + (type ==READ )) >0 ) {if (type ==READ ) -fsnotify_access (file->f_path.dentry); +fsnotify_access (file);else -fsnotify_modify (file->f_path.dentry); +fsnotify_modify (file); }return ret; }
include/linux/fsnotify.h +6 -3
@@ -136,8 +136,9 @@ static inline void fsnotify_mkdir(struct inode *inode, struct dentry *dentry)/* * fsnotify_access - file was read */ -static inline void fsnotify_access (struct dentry *dentry) +static inline void fsnotify_access (struct file *file) { +struct dentry *dentry = file->f_path.dentry;struct inode *inode = dentry->d_inode;u32 mask =IN_ACCESS ;
@@ -152,8 +153,9 @@ static inline void fsnotify_access(struct dentry *dentry)/* * fsnotify_modify - file was modified */ -static inline void fsnotify_modify (struct dentry *dentry) +static inline void fsnotify_modify (struct file *file) { +struct dentry *dentry = file->f_path.dentry;struct inode *inode = dentry->d_inode;u32 mask =IN_MODIFY ;
@@ -168,8 +170,9 @@ static inline void fsnotify_modify(struct dentry *dentry)/* * fsnotify_open - file was opened */ -static inline void fsnotify_open (struct dentry *dentry) +static inline void fsnotify_open (struct file *file) { +struct dentry *dentry = file->f_path.dentry;struct inode *inode = dentry->d_inode;u32 mask =IN_OPEN ;
Eric Paris
maintainerRFC PATCH 3/14 fsnotify: sys_execve and sys_uselib do not call into fsnotify
In reply to Eric Parissys_execve and sys_uselib do not call into fsnotify so inotify, dnotify, and importantly to me fanotify do not see opens on things which are going to be exectued. Create a generic fsnotify hook for these paths.
---
fs/exec.c | 5 +++++ include/linux/fsnotify.h | 7 +++++++ 2 files changed, 12 insertions(+), 0 deletions(-)
fs/exec.c +5 -0
@@ -51,6 +51,7 @@#include <linux/audit.h> #include <linux/tracehook.h> #include <linux/kmod.h> +#include <linux/fsnotify.h> #include <asm/uaccess.h> #include <asm/mmu_context.h>
@@ -135,6 +136,8 @@ asmlinkage long sys_uselib(const char __user * library)if (IS_ERR (file)) goto out; +fsnotify_open_exec (file); + error = -ENOEXEC ;if (file->f_op) {struct linux_binfmt * fmt;
@@ -687,6 +690,8 @@ struct file *open_exec(const char *name)if (IS_ERR (file))return file; +fsnotify_open_exec (file); + err =deny_write_access (file);if (err) {fput (file);
include/linux/fsnotify.h +7 -0
@@ -168,6 +168,13 @@ static inline void fsnotify_modify(struct file *file) }/* +* fsnotify_open_exec - file was opened by execve or uselib +*/ +static inline void fsnotify_open_exec (struct file *file) +{ +} + +/* * fsnotify_open - file was opened */ static inline void fsnotify_open (struct file *file)
Eric Paris
maintainerRFC PATCH 4/14 fsnotify: use the new open-exec hook for inotify and dnotify
In reply to Eric Parisinotify and dnotify did not get access events when their children were accessed for shlib or exec purposes. Trigger on those events as well.
---
include/linux/fsnotify.h | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
include/linux/fsnotify.h +6 -0
@@ -172,6 +172,12 @@ static inline void fsnotify_modify(struct file *file)*/ static inline void fsnotify_open_exec (struct file *file) { +struct dentry *dentry = file->f_path.dentry; +struct inode *inode = dentry->d_inode; + +dnotify_parent (dentry,DN_ACCESS ); +inotify_dentry_parent_queue_event (dentry,IN_ACCESS ,0 , dentry->d_name.name); +inotify_inode_queue_event (inode,IN_ACCESS ,0 ,NULL ,NULL ); }/*
Re: [RFC PATCH -v4 04/14] fsnotify: use the new open-exec hook for inotify and dnotify
Quoted 2 lines · Eric Paris
inotify and dnotify did not get access events when their children were accessed for shlib or exec purposes. Trigger on those events as well.
And requesting it about the third time: Can you please submit this (and the required previous patch) as a bug fix? We'd like to have this in ASAP even if the rest will need some more work.
Eric Paris
maintainerRFC PATCH 5/14 fsnotify: unified filesystem notification backend
In reply to Eric Parisfsnotify is a backend for filesystem notification. fsnotify does not provide any userspace interface but does provide the basis needed for other notification schemes such as dnotify. fsnotify can be extended to be the backend for inotify or the upcoming fsnotify.
---
fs/notify/Kconfig | 12 +++ fs/notify/Makefile | 2 fs/notify/fsnotify.c | 78 +++++++++++++++++++ fs/notify/fsnotify.h | 51 +++++++++++++ fs/notify/group.c | 153 ++++++++++++++++++++++++++++++++++++++ fs/notify/notification.c | 123 +++++++++++++++++++++++++++++++ include/linux/fsnotify.h | 57 ++++++++++++-- include/linux/fsnotify_backend.h | 110 +++++++++++++++++++++++++++ 8 files changed, 578 insertions(+), 8 deletions(-) create mode 100644 fs/notify/fsnotify.c create mode 100644 fs/notify/fsnotify.h create mode 100644 fs/notify/group.c create mode 100644 fs/notify/notification.c create mode 100644 include/linux/fsnotify_backend.h
fs/notify/Kconfig +12 -0
@@ -1,2 +1,14 @@ +config FSNOTIFY + bool "Filesystem notification backend" + default y + ---help--- + fsnotify is a backend for filesystem notification. fsnotify does + not provide any userspace interface but does provide the basis + needed for other notification schemes such as dnotify and fsnotify. + + Say Y here to enable fsnotify suport. + + If unsure, say Y. + source "fs/notify/dnotify/Kconfig" source "fs/notify/inotify/Kconfig"
fs/notify/Makefile +2 -0
@@ -1,2 +1,4 @@ obj-y += dnotify/ obj-y += inotify/ + +obj-$(CONFIG_FSNOTIFY) += fsnotify.o notification.o group.o
fs/notify/fsnotify.c new file +78 -0
78 lines not on this page · raw message
fs/notify/fsnotify.h new file +51 -0
51 lines not on this page · raw message
fs/notify/group.c new file +153 -0
153 lines not on this page · raw message
fs/notify/notification.c new file +123 -0
123 lines not on this page · raw message
include/linux/fsnotify.h +49 -8
190 lines not on this page · raw message
include/linux/fsnotify_backend.h new file +110 -0
110 lines not on this page · raw message
Re: [RFC PATCH -v4 05/14] fsnotify: unified filesystem notification backend
Hi.
fs/notify/group.c line 34
Quoted 4 lines · Eric Paris
fs/notify/group.c new file +4
-0+DEFINE_MUTEX (fsnotify_grp_mutex); +struct srcu_struct fsnotify_grp_srcu_struct; +LIST_HEAD (fsnotify_groups); +__u64 fsnotify_mask;
Those can be static.
Quoted 9 lines
fs/notify/notification.c new file +9
-0+struct fsnotify_event *fsnotify_create_event (struct inode *to_tell,__u64 mask,void *data,int data_is) +{ +struct fsnotify_event *event; + + event =kmem_cache_alloc (event_kmem_cache,GFP_KERNEL ); +if (!event) +return NULL ; + +atomic_set (&event->refcnt,1 );
fs/notify/notification.c line 93
Quoted 10 lines
fs/notify/notification.c new file +10
-0+ +spin_lock_init (&event->lock); + + event->path.dentry =NULL ; + event->path.mnt =NULL ; + event->inode =NULL ; + +INIT_LIST_HEAD (&event->private_data_list); + + event->to_tell = to_tell;
What prevents this inode to be released?
Signature · 1 line
Evgeniy Polyakov
Eric Paris
maintainerRe: [RFC PATCH -v4 05/14] fsnotify: unified filesystem notification backend
Quoted 13 lines · Evgeniy Polyakov
Quoted 13 lines
fs/notify/notification.c new file +13
-0+struct fsnotify_event *fsnotify_create_event (struct inode *to_tell,__u64 mask,void *data,int data_is) +{ +struct fsnotify_event *event; + + event =kmem_cache_alloc (event_kmem_cache,GFP_KERNEL ); +if (!event) +return NULL ; + +atomic_set (&event->refcnt,1 ); + +spin_lock_init (&event->lock); + + event->path.dentry =NULL ;
Quoted 8 lines
fs/notify/notification.c line 93
Quoted 6 lines
fs/notify/notification.c new file +6
-0+ event->path.mnt =NULL ; + event->inode =NULL ; + +INIT_LIST_HEAD (&event->private_data_list); + + event->to_tell = to_tell;What prevents this inode to be released?
Absolutely nothing and I need to document that. Two things, event->to_tell and event->data if event->data_is == INODE are ONLY valid during the call to group-?event_to_notif(). As soon as all groups return from that call those are not valid fields. I could set them NULL when they aren't allowed to be used any more, but it's a wasted operation on a VERY hot path (fsnotify())
As a side note event->data if event->data_is == FILE is perfectly allowed to be used until the event is freed. That's totally pointless for this patch set and I might drop it on the next submission, but it is needed for my fanotify notify patches that I keep in mind as I'm doing this.
I'll remember to better document this 'quirk' in case anyone tries to write a new notifier.
-Eric
Eric Paris
maintainerRFC PATCH 6/14 fsnotify: add group priorities
In reply to Eric ParisIn preperation for blocking fsnotify calls group priorities must be added. When multiple groups request the same event type the lowest priority group will receive the notification first.
---
fs/notify/group.c | 30 ++++++++++++++++++++++++------ include/linux/fsnotify_backend.h | 4 +++- 2 files changed, 27 insertions(+), 7 deletions(-)
fs/notify/group.c +24 -6
@@ -49,7 +49,21 @@ void fsnotify_recalc_global_mask(void)static void fsnotify_add_group (struct fsnotify_group *group) { -list_add_rcu (&group->group_list, &fsnotify_groups); +int priority = group->priority; +struct fsnotify_group *group_iter; + +list_for_each_entry (group_iter, &fsnotify_groups, group_list) { +BUG_ON (list_empty (&fsnotify_groups)); +/* insert in front of this one? */ +if (priority < group_iter->priority) { +/* I used list_add_tail() to insert in front of group_iter... */ +list_add_tail_rcu (&group->group_list, &group_iter->group_list); +return ; + } + } + +/* apparently we need to be the last entry */ +list_add_tail_rcu (&group->group_list, &fsnotify_groups); }void fsnotify_get_group (struct fsnotify_group *group)
@@ -91,14 +105,16 @@ void fsnotify_put_group(struct fsnotify_group *group) } } -static struct fsnotify_group *fsnotify_find_group (unsigned int group_num,__u64 mask,const struct fsnotify_ops *ops) +static struct fsnotify_group *fsnotify_find_group (unsigned int priority,unsigned int group_num, +__u64 mask,const struct fsnotify_ops *ops) {struct fsnotify_group *group_iter;struct fsnotify_group *group =NULL ;list_for_each_entry_rcu (group_iter, &fsnotify_groups, group_list) { -if (group_iter->group_num == group_num) { +if (group_iter->priority == priority) {if ((group_iter->mask == mask) && + (group_iter->group_num == group_num) && (group_iter->ops == ops)) {fsnotify_get_group (group_iter); group = group_iter;
@@ -109,13 +125,14 @@ static struct fsnotify_group *fsnotify_find_group(unsigned int group_num, __u64return group; } -struct fsnotify_group *fsnotify_obtain_group (unsigned int group_num,__u64 mask,const struct fsnotify_ops *ops) +struct fsnotify_group *fsnotify_obtain_group (unsigned int priority,unsigned int group_num, +__u64 mask,const struct fsnotify_ops *ops) {struct fsnotify_group *group, *tgroup;int idx; idx =srcu_read_lock (&fsnotify_grp_srcu_struct); - group =fsnotify_find_group (group_num, mask, ops); + group =fsnotify_find_group (priority, group_num, mask, ops);srcu_read_unlock (&fsnotify_grp_srcu_struct, idx);if (group)return group;
@@ -126,6 +143,7 @@ struct fsnotify_group *fsnotify_obtain_group(unsigned int group_num, __u64 mask,atomic_set (&group->refcnt,1 ); + group->priority = priority; group->group_num = group_num; group->mask = mask;
@@ -133,7 +151,7 @@ struct fsnotify_group *fsnotify_obtain_group(unsigned int group_num, __u64 mask, group->private =NULL ;mutex_lock (&fsnotify_grp_mutex); - tgroup =fsnotify_find_group (group_num, mask, ops); + tgroup =fsnotify_find_group (priority, group_num, mask, ops);/* we raced and something else inserted the same group */ if (tgroup) {mutex_unlock (&fsnotify_grp_mutex);
include/linux/fsnotify_backend.h +3 -1
@@ -85,6 +85,8 @@ struct fsnotify_group {const struct fsnotify_ops *ops;/* how this group handles things */ +unsigned int priority;/* order this group should receive msgs. low first */ +void *private;/* private data for implementers (dnotify, inotify, fanotify) */ };
@@ -95,7 +97,7 @@ extern void fsnotify(struct inode *to_tell, __u64 mask, void *data, int data_is)/* called from fsnotify interfaces, such as fanotify or dnotify */ extern void fsnotify_recalc_global_mask (void ); -extern struct fsnotify_group *fsnotify_obtain_group (unsigned int group_num,__u64 mask,const struct fsnotify_ops *ops); +extern struct fsnotify_group *fsnotify_obtain_group (unsigned int priority,unsigned int group_num,__u64 mask,const struct fsnotify_ops *ops);extern void fsnotify_put_group (struct fsnotify_group *group);extern void fsnotify_get_group (struct fsnotify_group *group);
Eric Paris
maintainerRFC PATCH 7/14 fsnotify: add in inode fsnotify markings
In reply to Eric ParisThis patch creates in inode fsnotify markings. dnotify will make use of in inode markings to mark which inodes it wishes to send events for. fanotify will use this to mark which inodes it does not wish to send events for.
---
fs/inode.c | 6 + fs/notify/Makefile | 2 fs/notify/fsnotify.c | 13 ++ fs/notify/fsnotify.h | 31 ++++ fs/notify/group.c | 24 +++ fs/notify/inode_mark.c | 267 ++++++++++++++++++++++++++++++++++++++ include/linux/fs.h | 5 + include/linux/fsnotify.h | 9 + include/linux/fsnotify_backend.h | 22 +++ 9 files changed, 378 insertions(+), 1 deletions(-) create mode 100644 fs/notify/inode_mark.c
fs/inode.c +6 -0
@@ -21,6 +21,7 @@#include <linux/cdev.h> #include <linux/bootmem.h> #include <linux/inotify.h> +#include <linux/fsnotify.h> #include <linux/mount.h> /*
@@ -183,6 +184,10 @@ static struct inode *alloc_inode(struct super_block *sb) } inode->i_private =NULL ; inode->i_mapping = mapping; +#ifdef CONFIG_FSNOTIFY + inode->i_fsnotify_mask =0 ; +INIT_LIST_HEAD (&inode->i_fsnotify_mark_entries); +#endif }return inode; }
@@ -191,6 +196,7 @@ void destroy_inode(struct inode *inode) {BUG_ON (inode_has_buffers (inode));security_inode_free (inode); +fsnotify_inode_delete (inode);if (inode->i_sb->s_op->destroy_inode) inode->i_sb->s_op->destroy_inode (inode);else
fs/notify/Makefile +1 -1
@@ -1,4 +1,4 @@ obj-y += dnotify/ obj-y += inotify/ -obj-$(CONFIG_FSNOTIFY) += fsnotify.o notification.o group.o +obj-$(CONFIG_FSNOTIFY) += fsnotify.o notification.o group.o inode_mark.o
fs/notify/fsnotify.c +13 -0
@@ -25,6 +25,15 @@#include <linux/fsnotify_backend.h> #include "fsnotify.h" +void __fsnotify_inode_delete (struct inode *inode,int flag) +{ +if (likely (list_empty (&fsnotify_groups))) +return ; + +fsnotify_clear_marks_by_inode (inode, flag); +} +EXPORT_SYMBOL_GPL (__fsnotify_inode_delete); +void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is) {struct fsnotify_group *group;
@@ -37,6 +46,8 @@ void fsnotify(struct inode *to_tell, __u64 mask, void *data, int data_is)if (!(mask & fsnotify_mask))return ; +if (!(mask & to_tell->i_fsnotify_mask)) +return ;/* * SRCU!! the groups list is very very much read only and the path is * very hot (assuming something is using fsnotify) Not blocking while
@@ -52,6 +63,8 @@ void fsnotify(struct inode *to_tell, __u64 mask, void *data, int data_is) idx =srcu_read_lock (&fsnotify_grp_srcu_struct);list_for_each_entry_rcu (group, &fsnotify_groups, group_list) {if (mask & group->mask) { +if (!group->ops->should_send_event (group, to_tell, mask)) +continue ;if (!event) { event =fsnotify_create_event (to_tell, mask, data, data_is);/* shit, we OOM'd and now we can't tell, lets hope something else blows up */
fs/notify/fsnotify.h +31 -0
@@ -40,6 +40,31 @@ struct fsnotify_event {struct list_head private_data_list; }; +/* +* a mark is simply an entry attached to an in core inode which allows an +* fsnotify listener to indicate they are either no longer interested in events +* of a type matching mask or only interested in those events. +* +* these are flushed when an inode is evicted from core and may be flushed +* when the inode is modified (as seen by fsnotify_access). Some fsnotify users +* (such as dnotify) will flush these when the open fd is closed and not at +* inode eviction or modification. +*/ +struct fsnotify_mark_entry { +struct fsnotify_group *group;/* group this mark entry is for */ +__u64 mask;/* mask this mark entry is for */ +struct inode *inode;/* inode this entry is associated with */ +void *private;/* private data for the listener */ +spinlock_t lock;/* protect group, inode, and killme */ +atomic_t refcnt;/* active things looking at this mark */ +int freeme;/* free when this is set and refcnt hits 0 */ +struct list_head i_list;/* list of mark_entries by inode->i_fsnotify_mark_entries */ +struct list_head g_list;/* list of mark_entries by group->i_fsnotify_mark_entries */ +struct list_head free_i_list;/* tmp list used when freeing this mark */ +struct list_head free_g_list;/* tmp list used when freeing this mark */ +void (*free_private)(struct fsnotify_mark_entry *entry);/* called on final put+free */ +}; +extern struct srcu_struct fsnotify_grp_srcu_struct;extern struct list_head fsnotify_groups;extern __u64 fsnotify_mask;
@@ -48,4 +73,10 @@ extern void fsnotify_get_event(struct fsnotify_event *event);extern void fsnotify_put_event (struct fsnotify_event *event);extern struct fsnotify_event_private_data *fsnotify_get_priv_from_event (struct fsnotify_group *group,struct fsnotify_event *event);extern struct fsnotify_event *fsnotify_create_event (struct inode *to_tell,__u64 mask,void *data,int data_is); + +extern void fsnotify_clear_marks_by_group (struct fsnotify_group *group); +extern void fsnotify_clear_marks_by_inode (struct inode *inode,unsigned int flags); +extern void fsnotify_destroy_mark_by_entry (struct fsnotify_mark_entry *entry); +extern void fsnotify_get_mark (struct fsnotify_mark_entry *entry); +extern void fsnotify_put_mark (struct fsnotify_mark_entry *entry);#endif /* _LINUX_FSNOTIFY_PRIVATE_H */
fs/notify/group.c +24 -0
@@ -47,6 +47,24 @@ void fsnotify_recalc_global_mask(void) fsnotify_mask = mask; } +void fsnotify_recalc_group_mask (struct fsnotify_group *group) +{ +__u64 mask =0 ; +unsigned long old_mask = group->mask; +struct fsnotify_mark_entry *entry; + +spin_lock (&group->mark_lock); +list_for_each_entry (entry, &group->mark_entries, g_list) { + mask |= entry->mask; + } +spin_unlock (&group->mark_lock); + + group->mask = mask; + +if (old_mask != mask) +fsnotify_recalc_global_mask (); +} +static void fsnotify_add_group (struct fsnotify_group *group) {int priority = group->priority;
@@ -73,6 +91,9 @@ void fsnotify_get_group(struct fsnotify_group *group)static void fsnotify_destroy_group (struct fsnotify_group *group) { +/* clear all inode mark entries for this group */ +fsnotify_clear_marks_by_group (group); +if (group->ops->free_group_priv) group->ops->free_group_priv (group);
@@ -147,6 +168,9 @@ struct fsnotify_group *fsnotify_obtain_group(unsigned int priority, unsigned int group->group_num = group_num; group->mask = mask; +spin_lock_init (&group->mark_lock); +INIT_LIST_HEAD (&group->mark_entries); + group->ops = ops; group->private =NULL ;
fs/notify/inode_mark.c new file +267 -0
267 lines not on this page · raw message
include/linux/fs.h +5 -0
@@ -665,6 +665,11 @@ struct inode {__u32 i_generation; +#ifdef CONFIG_FSNOTIFY +__u64 i_fsnotify_mask;/* all events this inode cares about */ +struct list_head i_fsnotify_mark_entries;/* fsnotify mark entries */ +#endif +#ifdef CONFIG_DNOTIFY unsigned long i_dnotify_mask;/* Directory notify events */ struct dnotify_struct *i_dnotify;/* for directory notifications */
include/linux/fsnotify.h +9 -0
@@ -99,6 +99,14 @@ static inline void fsnotify_nameremove(struct dentry *dentry, int isdir) }/* +* fsnotify_inode_delete - and inode is being evicted from cache, clean up is needed +*/ +static inline void fsnotify_inode_delete (struct inode *inode) +{ +__fsnotify_inode_delete (inode,FSNOTIFY_INODE_DESTROY ); +} + +/* * fsnotify_inoderemove - an inode is going away */ static inline void fsnotify_inoderemove (struct inode *inode)
@@ -107,6 +115,7 @@ static inline void fsnotify_inoderemove(struct inode *inode)inotify_inode_is_dead (inode);fsnotify (inode,FS_DELETE_SELF , inode,FSNOTIFY_EVENT_INODE ); +__fsnotify_inode_delete (inode,FSNOTIFY_LAST_DENTRY ); }/*
include/linux/fsnotify_backend.h +22 -0
@@ -13,6 +13,7 @@#include <linux/fs.h> #include <linux/list.h> #include <linux/mutex.h> +#include <linux/spinlock.h> #include <linux/wait.h> #include <asm/atomic.h>
@@ -68,13 +69,21 @@#define FSNOTIFY_EVENT_FILE 1#define FSNOTIFY_EVENT_INODE 2 +/* these tell __fsnotify_inode_delete what kind of event this is */ +#define FSNOTIFY_LAST_DENTRY 1 +#define FSNOTIFY_INODE_DESTROY 2 +struct fsnotify_group ;struct fsnotify_event ; +struct fsnotify_mark_entry ;struct fsnotify_ops {int (*event_to_notif)(struct fsnotify_group *group,struct fsnotify_event *event); +void (*mark_clear_inode)(struct fsnotify_mark_entry *entry,struct inode *inode,unsigned int flags); +int (*should_send_event)(struct fsnotify_group *group,struct inode *inode,__u64 mask);void (*free_group_priv)(struct fsnotify_group *group);void (*free_event_priv)(struct fsnotify_group *group,struct fsnotify_event *event); +void (*free_mark_priv)(struct fsnotify_mark_entry *entry); };struct fsnotify_group {
@@ -85,6 +94,10 @@ struct fsnotify_group {const struct fsnotify_ops *ops;/* how this group handles things */ +/* stores all fastapth entries assoc with this group so they can be cleaned on unregister */ +spinlock_t mark_lock;/* protect mark_entries list */ +struct list_head mark_entries;/* all inode mark entries for this group */ +unsigned int priority;/* order this group should receive msgs. low first */ void *private;/* private data for implementers (dnotify, inotify, fanotify) */
@@ -94,17 +107,26 @@ struct fsnotify_group {/* called from the vfs to signal fs events */ extern void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is); +extern void __fsnotify_inode_delete (struct inode *inode,int flag);/* called from fsnotify interfaces, such as fanotify or dnotify */ extern void fsnotify_recalc_global_mask (void ); +extern void fsnotify_recalc_group_mask (struct fsnotify_group *group);extern struct fsnotify_group *fsnotify_obtain_group (unsigned int priority,unsigned int group_num,__u64 mask,const struct fsnotify_ops *ops);extern void fsnotify_put_group (struct fsnotify_group *group);extern void fsnotify_get_group (struct fsnotify_group *group); +extern void fsnotify_recalc_inode_mask (struct inode *inode); +extern struct fsnotify_mark_entry *fsnotify_find_mark_entry (struct fsnotify_group *group,struct inode *inode); +extern struct fsnotify_mark_entry *fsnotify_mark_add (struct fsnotify_group *group,struct inode *inode,__u64 mask);#else static inline void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is); {} + +static inline void __fsnotify_inode_delete (struct inode *inode,int flag) +{} +#endif /* CONFIG_FSNOTIFY */ #endif /* __KERNEL __ */
Re: [RFC PATCH -v4 07/14] fsnotify: add in inode fsnotify markings
fs/notify/inode_mark.c line 74
Quoted 11 lines · Eric Paris
fs/notify/inode_mark.c new file +11
-0+void fsnotify_put_mark (struct fsnotify_mark_entry *entry) +{ +if (atomic_dec_and_test (&entry->refcnt)) { +spin_lock (&entry->lock); +/* entries can only be found by the kernel by searching the +* inode->i_fsnotify_entries or the group->mark_entries lists. +* if freeme is set that means this entry is off both lists. +* if refcnt is 0 that means we are the last thing still +* looking at this entry, so its time to free. +*/ +if (!atomic_read (&entry->refcnt) && entry->freeme) {
Why do you check refcnt again? Does it mean its life cycle does not end when it hits zero and should be freed only under lock, which in turn means it may be accessed under that lock and zero refcnt? Please describe this locking in more details.
Quoted 16 lines
fs/notify/inode_mark.c new file +16
-0+spin_unlock (&entry->lock); +fsnotify_destroy_mark (entry); +return ; + } +spin_unlock (&entry->lock); + } +} + +void fsnotify_clear_marks_by_group (struct fsnotify_group *group) +{ +struct fsnotify_mark_entry *lentry, *entry; +struct inode *inode; +LIST_HEAD (free_list); + +spin_lock (&group->mark_lock); +list_for_each_entry_safe (entry, lentry, &group->mark_entries, g_list) {
fs/notify/inode_mark.c line 100
Quoted 10 lines
fs/notify/inode_mark.c new file +10
-0+list_del_init (&entry->g_list); +list_add (&entry->free_g_list, &free_list); + } +spin_unlock (&group->mark_lock); + +list_for_each_entry_safe (entry, lentry, &free_list, free_g_list) { +fsnotify_get_mark (entry); +spin_lock (&entry->lock); + inode = entry->inode; +if (!inode) {
This inode does not seem to be grabbed previously, or I missed that part? What prevents it from being freed?
Quoted 34 lines
fs/notify/inode_mark.c new file +33
-0+ entry->group =NULL ; +spin_unlock (&entry->lock); +fsnotify_put_mark (entry); +continue ; + } +spin_lock (&inode->i_lock); + +list_del_init (&entry->i_list); + entry->inode =NULL ; +list_del_init (&entry->g_list); + entry->group =NULL ; + entry->freeme =1 ; + +spin_unlock (&inode->i_lock); +spin_unlock (&entry->lock); + +fsnotify_put_mark (entry); + } +} +void fsnotify_clear_marks_by_inode (struct inode *inode,unsigned int flags) +{ +struct fsnotify_mark_entry *lentry, *entry; +LIST_HEAD (free_list); + +spin_lock (&inode->i_lock); +list_for_each_entry_safe (entry, lentry, &inode->i_fsnotify_mark_entries, i_list) { +list_del_init (&entry->i_list); +list_add (&entry->free_i_list, &free_list); + } +spin_unlock (&inode->i_lock); + +/* +* at this point destroy_by_* might race.
fs/notify/inode_mark.c line 177
Quoted 10 lines
fs/notify/inode_mark.c new file +10
-0+* +* we used list_del_init() so it can be list_del_init'd again, no harm. +* we were called from an inode function so we know that other user can +* try to grab entry->inode->i_lock without a problem. +*/ +list_for_each_entry_safe (entry, lentry, &free_list, free_i_list) { +fsnotify_get_mark (entry); + entry->group->ops->mark_clear_inode (entry, inode, flags); +fsnotify_put_mark (entry); + }
Since entry was not grabbed in the above locked list, what prevents it to be freed before fsnotify_get_mark() executed? Could that get moved under lock?
Quoted 6 lines
fs/notify/inode_mark.c new file +6
-0+} + +/* caller must hold inode->i_lock */ +struct fsnotify_mark_entry *fsnotify_find_mark_entry (struct fsnotify_group *group,struct inode *inode) +{ +struct fsnotify_mark_entry *entry;
fs/notify/inode_mark.c line 193
Quoted 10 lines
fs/notify/inode_mark.c new file +10
-0+ +list_for_each_entry (entry, &inode->i_fsnotify_mark_entries, i_list) { +if (entry->group == group) { +fsnotify_get_mark (entry); +return entry; + } + } +return NULL ; +} +/*
Missed newline before comment's start :)
Quoted 30 lines
+ * This is a low use function called when userspace is changing what is being
+ * watched. I don't mind doing the allocation since I'm assuming we will have
+ * more new events than we have adding to old events...
+ *
+ * add (we use |=) the mark to the in core inode mark, if you need to change
+ * rather than | some new bits you needs tofsnotify_destroy_mark_by_inode()
+ * then call this with all the right bits in the mask.
+ */fs/notify/inode_mark.c new file +22
-0+struct fsnotify_mark_entry *fsnotify_mark_add (struct fsnotify_group *group,struct inode *inode,__u64 mask) +{ +/* we initialize entry to shut up the compiler in case we just to out... */ +struct fsnotify_mark_entry *entry =NULL , *lentry; + +/* pre allocate an entry so we can hold the lock */ + entry =fsnotify_alloc_mark (); +if (!entry) +return NULL ; + +/* +* LOCKING ORDER!!!! +* entry->lock +* group->mark_lock +* inode->i_lock +*/ +spin_lock (&group->mark_lock); +spin_lock (&inode->i_lock); + lentry =fsnotify_find_mark_entry (group, inode); +if (lentry) { +/* we didn't use the new entry, kill it */ +fsnotify_destroy_mark (entry);
fs/notify/inode_mark.c line 233
Quoted 10 lines
fs/notify/inode_mark.c new file +10
-0+ entry = lentry; + entry->mask |= mask; + goto out_unlock; + } + +spin_lock_init (&entry->lock); +atomic_set (&entry->refcnt,1 ); + entry->group = group; + entry->mask = mask; + entry->inode = inode;
That's what I talked about previously, what if this inode will be released after inode unlocked?
Quoted 149 lines
fs/notify/inode_mark.c new file +34
-0+ entry->freeme =0 ; + entry->private =NULL ; + entry->free_private = group->ops->free_mark_priv; + +list_add (&entry->i_list, &inode->i_fsnotify_mark_entries); +list_add (&entry->g_list, &group->mark_entries); + +out_unlock: +spin_unlock (&inode->i_lock); +spin_unlock (&group->mark_lock); +return entry; +} + +void fsnotify_recalc_inode_mask (struct inode *inode) +{ +unsigned long new_mask =0 ; +struct fsnotify_mark_entry *entry; + +spin_lock (&inode->i_lock); +list_for_each_entry (entry, &inode->i_fsnotify_mark_entries, i_list) { + new_mask |= entry->mask; + } + inode->i_fsnotify_mask = new_mask; +spin_unlock (&inode->i_lock); +} + + +__init int fsnotify_mark_init (void ) +{ + fsnotify_mark_kmem_cache =kmem_cache_create ("fsnotify_mark_entry" ,sizeof (struct fsnotify_mark_entry ),0 ,SLAB_PANIC ,NULL ); + +return 0 ; +} +subsys_initcall (fsnotify_mark_init);include/linux/fs.h +5
-0@@ -665,6 +665,11 @@ struct inode {__u32 i_generation; +#ifdef CONFIG_FSNOTIFY +__u64 i_fsnotify_mask;/* all events this inode cares about */ +struct list_head i_fsnotify_mark_entries;/* fsnotify mark entries */ +#endif +#ifdef CONFIG_DNOTIFY unsigned long i_dnotify_mask;/* Directory notify events */ struct dnotify_struct *i_dnotify;/* for directory notifications */ include/linux/fsnotify.h +9
-0@@ -99,6 +99,14 @@ static inline void fsnotify_nameremove(struct dentry *dentry, int isdir) }/* +* fsnotify_inode_delete - and inode is being evicted from cache, clean up is needed +*/ +static inline void fsnotify_inode_delete (struct inode *inode) +{ +__fsnotify_inode_delete (inode,FSNOTIFY_INODE_DESTROY ); +} + +/* * fsnotify_inoderemove - an inode is going away */ static inline void fsnotify_inoderemove (struct inode *inode)@@ -107,6 +115,7 @@ static inline void fsnotify_inoderemove(struct inode *inode)inotify_inode_is_dead (inode);fsnotify (inode,FS_DELETE_SELF , inode,FSNOTIFY_EVENT_INODE ); +__fsnotify_inode_delete (inode,FSNOTIFY_LAST_DENTRY ); }/* include/linux/fsnotify_backend.h +22
-0@@ -13,6 +13,7 @@#include <linux/fs.h> #include <linux/list.h> #include <linux/mutex.h> +#include <linux/spinlock.h> #include <linux/wait.h> #include <asm/atomic.h> @@ -68,13 +69,21 @@#define FSNOTIFY_EVENT_FILE 1#define FSNOTIFY_EVENT_INODE 2 +/* these tell __fsnotify_inode_delete what kind of event this is */ +#define FSNOTIFY_LAST_DENTRY 1 +#define FSNOTIFY_INODE_DESTROY 2 +struct fsnotify_group ;struct fsnotify_event ; +struct fsnotify_mark_entry ;struct fsnotify_ops {int (*event_to_notif)(struct fsnotify_group *group,struct fsnotify_event *event); +void (*mark_clear_inode)(struct fsnotify_mark_entry *entry,struct inode *inode,unsigned int flags); +int (*should_send_event)(struct fsnotify_group *group,struct inode *inode,__u64 mask);void (*free_group_priv)(struct fsnotify_group *group);void (*free_event_priv)(struct fsnotify_group *group,struct fsnotify_event *event); +void (*free_mark_priv)(struct fsnotify_mark_entry *entry); };struct fsnotify_group {@@ -85,6 +94,10 @@ struct fsnotify_group {const struct fsnotify_ops *ops;/* how this group handles things */ +/* stores all fastapth entries assoc with this group so they can be cleaned on unregister */ +spinlock_t mark_lock;/* protect mark_entries list */ +struct list_head mark_entries;/* all inode mark entries for this group */ +unsigned int priority;/* order this group should receive msgs. low first */ void *private;/* private data for implementers (dnotify, inotify, fanotify) */ @@ -94,17 +107,26 @@ struct fsnotify_group {/* called from the vfs to signal fs events */ extern void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is); +extern void __fsnotify_inode_delete (struct inode *inode,int flag);/* called from fsnotify interfaces, such as fanotify or dnotify */ extern void fsnotify_recalc_global_mask (void ); +extern void fsnotify_recalc_group_mask (struct fsnotify_group *group);extern struct fsnotify_group *fsnotify_obtain_group (unsigned int priority,unsigned int group_num,__u64 mask,const struct fsnotify_ops *ops);extern void fsnotify_put_group (struct fsnotify_group *group);extern void fsnotify_get_group (struct fsnotify_group *group); +extern void fsnotify_recalc_inode_mask (struct inode *inode); +extern struct fsnotify_mark_entry *fsnotify_find_mark_entry (struct fsnotify_group *group,struct inode *inode); +extern struct fsnotify_mark_entry *fsnotify_mark_add (struct fsnotify_group *group,struct inode *inode,__u64 mask);#else static inline void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is); {} + +static inline void __fsnotify_inode_delete (struct inode *inode,int flag) +{} +#endif /* CONFIG_FSNOTIFY */ #endif /* __KERNEL __ */
Signature · 1 line
Evgeniy Polyakov
Eric Paris
maintainerRe: [RFC PATCH -v4 07/14] fsnotify: add in inode fsnotify markings
Quoted 17 lines · Evgeniy Polyakov
fs/notify/inode_mark.c line 74
Quoted 11 lines · Eric Paris
fs/notify/inode_mark.c new file +11
-0+void fsnotify_put_mark (struct fsnotify_mark_entry *entry) +{ +if (atomic_dec_and_test (&entry->refcnt)) { +spin_lock (&entry->lock); +/* entries can only be found by the kernel by searching the +* inode->i_fsnotify_entries or the group->mark_entries lists. +* if freeme is set that means this entry is off both lists. +* if refcnt is 0 that means we are the last thing still +* looking at this entry, so its time to free. +*/ +if (!atomic_read (&entry->refcnt) && entry->freeme) {Why do you check refcnt again? Does it mean its life cycle does not end when it hits zero and should be freed only under lock, which in turn means it may be accessed under that lock and zero refcnt? Please describe this locking in more details.
Al Viro, please pay attention to this patch. This is the one that provides lifetimes which I hope solves all of inotify's "fun"
marks have very interesting semantics which took me a long time to figure out. Marks can only be found in kernel by walking one of two lists. group->mark_entries or inode->i_fsnotify_mark_entries. These lists are protected by group->mark_lock or inode->i_lost respectively. Everything else inside of a mark is protected by entry->lock. The locking order is
entry->lock
group->mark_lock
inode->i_lock
Entries must not be freed until they are gone from both the group->list and the inode->list and no other process is referencing this entry. The refcnt keeps track of how many processes are looking at this entry. The freeme is set when the entry is removed from both lists. Thus if the refcnt==0 and freeme=1 that means nothing is looking at this entry and it is off both lists so we free it.
The difficulty lies in the fact that we might need to free marks for 3 reasons from 3 different directions.
1) inode disappears (inode deleted or evicted form core, backing FS unmounted)
2) group disappears (close the inotify fd)
3) entry needs to go (inotify unregisters it)
Number 3 is by far the easiest. We grab entry->lock. If entry->group is set grab it's lock. If entry->inode is set we grab that lock. We can then drop this entry from both lists set group and inode to NULL and set freeme. Once everything is finished looking at the entry it will get freed.
Numbers 2 and 3 are much harder lets look at #1 since #2 is similar. There are 3 reasons I can think of that we would need to kill all entries given an inode. the inode is evicted from core (this isn't actually an issue with dnotify or inotify since dnotify holds a reference by the sheer fact an associated file for the inode must be held open, and the existence of a "watch" or "entry" for inotify keep it from being evicted due to memory pressure. It could be evicted because the last dentry pointing to the inode was deleted. The fs backing that inode is being unmounted. In any case, we are given an inode and we are told "all entries associated with this inode need to be freed."
So to find an entry we need to first grab the inode->i_lock and start to walk the inode->i_fsnotify_mark_entries list. Since we hold the i_lock we are not allowed to grab any other locks nor are we allowed to change anything other than entry->i_list. The secret sauce is that we actually move the entry from the inode list to a private list which we can walk and modify lockless. Inside the event we actually have to use a different list, free_i_list, for this operation so nothing else that races with us can mess stuff up. We run the entire inode we are trying to free all entries for an put the entries on the private list. We do NOT modify event->inode.
After this point we can drop inode->i_lock since we are finished with inode->i_fsnotify_mark_entries and that list is empty. Now we can walk the private list we just created lockless. We can thus try to grab in order, all three locks (we don't really need i_lock since we are already off the inode's list) and can safely remove this entry from both lists, set both entry->inode and entry->group to NULL (we hold the event->lock so this is safe) and since we are off both lists set freeme.
As soon as we dropped i_lock remember we could have raced. The cool thing here is that this is fine. The "thing" we are racing with is going to be holding entry->lock. So we are going to block until that gets released. That other task is going to grab entry->inode->i_lock which is also fine. As long as we exist the inode still exists and so the lock is ok. Since we always use list_del_init(entry->i_list) when that other task tries to remove this entry from the inode list it won't really be doing anything, nor hurting anything. This is the reason that we needed the free_i_list. So a racing task can't mess with the lists that we need. Only a process shooting an entry by inode will use this list. That other task is also going to take care of setting entry->inode == NULL.
Eventually we are going to get the entry->lock, will do our best to lock what else we can (we actually shouldn't find entry->inode or entry->group if we lost the race), and we will remove the entry from what lists we can (again it's already off both lists). At the end we know that this entry is off both lists and can be marked to be freed.
It's actually quite slick, we can try to free an entry from all 3 directions at once but the entry won't actually be freed until it is off both lists and nothing is left actively referencing it.
Quoted 22 lines
Quoted 22 lines
fs/notify/inode_mark.c new file +21
-0+spin_unlock (&entry->lock); +fsnotify_destroy_mark (entry); +return ; + } +spin_unlock (&entry->lock); + } +} + +void fsnotify_clear_marks_by_group (struct fsnotify_group *group) +{ +struct fsnotify_mark_entry *lentry, *entry; +struct inode *inode; +LIST_HEAD (free_list); + +spin_lock (&group->mark_lock); +list_for_each_entry_safe (entry, lentry, &group->mark_entries, g_list) { +list_del_init (&entry->g_list); +list_add (&entry->free_g_list, &free_list); + } +spin_unlock (&group->mark_lock); +
Quoted 8 lines
fs/notify/inode_mark.c line 100
Quoted 5 lines
fs/notify/inode_mark.c new file +5
-0+list_for_each_entry_safe (entry, lentry, &free_list, free_g_list) { +fsnotify_get_mark (entry); +spin_lock (&entry->lock); + inode = entry->inode; +if (!inode) {This inode does not seem to be grabbed previously, or I missed that part? What prevents it from being freed?
Here's the cool part. If the inode was free, we would have removed this entry from the inode->i_fsnotify_mark_entries list and set entry->inode to NULL back when that happened.
Quoted 41 lines
Quoted 41 lines
fs/notify/inode_mark.c new file +39
-0+ entry->group =NULL ; +spin_unlock (&entry->lock); +fsnotify_put_mark (entry); +continue ; + } +spin_lock (&inode->i_lock); + +list_del_init (&entry->i_list); + entry->inode =NULL ; +list_del_init (&entry->g_list); + entry->group =NULL ; + entry->freeme =1 ; + +spin_unlock (&inode->i_lock); +spin_unlock (&entry->lock); + +fsnotify_put_mark (entry); + } +} +void fsnotify_clear_marks_by_inode (struct inode *inode,unsigned int flags) +{ +struct fsnotify_mark_entry *lentry, *entry; +LIST_HEAD (free_list); + +spin_lock (&inode->i_lock); +list_for_each_entry_safe (entry, lentry, &inode->i_fsnotify_mark_entries, i_list) { +list_del_init (&entry->i_list); +list_add (&entry->free_i_list, &free_list); + } +spin_unlock (&inode->i_lock); + +/* +* at this point destroy_by_* might race. +* +* we used list_del_init() so it can be list_del_init'd again, no harm. +* we were called from an inode function so we know that other user can +* try to grab entry->inode->i_lock without a problem. +*/ +list_for_each_entry_safe (entry, lentry, &free_list, free_i_list) {
Quoted 8 lines
fs/notify/inode_mark.c line 177
Quoted 4 lines
fs/notify/inode_mark.c new file +4
-0+fsnotify_get_mark (entry); + entry->group->ops->mark_clear_inode (entry, inode, flags); +fsnotify_put_mark (entry); + }Since entry was not grabbed in the above locked list, what prevents it to be freed before
fsnotify_get_mark()executed? Could that get moved under lock?
absolutely nothing. This is a bug. thanks.
Quoted 10 lines
Quoted 10 lines
fs/notify/inode_mark.c new file +10
-0+} + +/* caller must hold inode->i_lock */ +struct fsnotify_mark_entry *fsnotify_find_mark_entry (struct fsnotify_group *group,struct inode *inode) +{ +struct fsnotify_mark_entry *entry; + +list_for_each_entry (entry, &inode->i_fsnotify_mark_entries, i_list) { +if (entry->group == group) { +fsnotify_get_mark (entry);
Quoted 8 lines
fs/notify/inode_mark.c line 193
Quoted 6 lines
fs/notify/inode_mark.c new file +6
-0+return entry; + } + } +return NULL ; +} +/* Missed newline before comment's start :)
don't I wish this was my only problem :)
Quoted 36 lines
Quoted 36 lines
+ * This is a low use function called when userspace is changing what is being
+ * watched. I don't mind doing the allocation since I'm assuming we will have
+ * more new events than we have adding to old events...
+ *
+ * add (we use |=) the mark to the in core inode mark, if you need to change
+ * rather than | some new bits you needs tofsnotify_destroy_mark_by_inode()
+ * then call this with all the right bits in the mask.
+ */fs/notify/inode_mark.c new file +27
-0+struct fsnotify_mark_entry *fsnotify_mark_add (struct fsnotify_group *group,struct inode *inode,__u64 mask) +{ +/* we initialize entry to shut up the compiler in case we just to out... */ +struct fsnotify_mark_entry *entry =NULL , *lentry; + +/* pre allocate an entry so we can hold the lock */ + entry =fsnotify_alloc_mark (); +if (!entry) +return NULL ; + +/* +* LOCKING ORDER!!!! +* entry->lock +* group->mark_lock +* inode->i_lock +*/ +spin_lock (&group->mark_lock); +spin_lock (&inode->i_lock); + lentry =fsnotify_find_mark_entry (group, inode); +if (lentry) { +/* we didn't use the new entry, kill it */ +fsnotify_destroy_mark (entry); + entry = lentry; + entry->mask |= mask; + goto out_unlock; + } +
Quoted 8 lines
fs/notify/inode_mark.c line 233
Quoted 5 lines
fs/notify/inode_mark.c new file +5
-0+spin_lock_init (&entry->lock); +atomic_set (&entry->refcnt,1 ); + entry->group = group; + entry->mask = mask; + entry->inode = inode;That's what I talked about previously, what if this inode will be released after inode unlocked?
__fsnotify_inode_delete() will clean this up when the inode is being booted. since this entry is on the inode->i_fsnotify_mark_entries list.
Al Viro
maintainerRe: [RFC PATCH -v4 07/14] fsnotify: add in inode fsnotify markings
Quoted 10 lines · Eric Paris
So to find an entry we need to first grab the
inode->i_lockand start to walk theinode->i_fsnotify_mark_entrieslist. Since we hold thei_lockwe are not allowed to grab any other locks nor are we allowed to change anything other thanentry->i_list. The secret sauce is that we actually move the entry from the inode list to a private list which we can walk and modify lockless. Inside the event we actually have to use a different list,free_i_list, for this operation so nothing else that races with us can mess stuff up. We run the entire inode we are trying to free all entries for an put the entries on the private list. We do NOT modifyevent->inode.
And just what happens if #3 ("remove entry") hits us in the meanwhile? Freed object sitting on free_i_list?
Eric Paris
maintainerRe: [RFC PATCH -v4 07/14] fsnotify: add in inode fsnotify markings
Quoted 13 lines · Al Viro
Quoted 10 lines · Eric Paris
So to find an entry we need to first grab the
inode->i_lockand start to walk theinode->i_fsnotify_mark_entrieslist. Since we hold thei_lockwe are not allowed to grab any other locks nor are we allowed to change anything other thanentry->i_list. The secret sauce is that we actually move the entry from the inode list to a private list which we can walk and modify lockless. Inside the event we actually have to use a different list,free_i_list, for this operation so nothing else that races with us can mess stuff up. We run the entire inode we are trying to free all entries for an put the entries on the private list. We do NOT modifyevent->inode.
Quoted 2 lines
And just what happens if #3 ("remove entry") hits us in the meanwhile? Freed object sitting on
free_i_list?
In the code on list, nothing. Evgeniy Polyakov pointed out that I grabbed my marks at the wrong time.
http://lkml.org/lkml/2008/12/13/94
The corrected idea is that while under the i_lock I call fsnotify_get_mark() on every inode mark that I am moving to the free_i_list. When #3 races it actually does all of the cleanup, including setting the "freeme" flag. But it won't be able to get the refcnt down to 0. Once we finish 'cleaning up,' we call put on the object and the refcnt will actually go to 0 and be freed.
The memory can't be freed as long as it is on a free_{i,g}_list.
-Eric
Eric Paris
maintainerRFC PATCH 8/14 fsnotify: parent event notification
In reply to Eric Parisinotify and dnotify both use a similar parent notification mechanism. We add a generic parent notification mechanism to fsnotify for both of these to use. This new machanism also adds the dentry flag optimization which exists for inotify to dnotify.
---
fs/notify/inode_mark.c | 2 + include/linux/dcache.h | 3 + include/linux/fsnotify.h | 103 +++++++++++++++++++++++++++++++++++++- include/linux/fsnotify_backend.h | 5 ++ 4 files changed, 109 insertions(+), 4 deletions(-)
fs/notify/inode_mark.c +2 -0
@@ -255,6 +255,8 @@ void fsnotify_recalc_inode_mask(struct inode *inode) } inode->i_fsnotify_mask = new_mask;spin_unlock (&inode->i_lock); + +fsnotify_update_dentry_child_flags (inode); }
include/linux/dcache.h +2 -1
@@ -175,7 +175,8 @@ d_iput: no no no yes#define DCACHE_REFERENCED 0x0008/* Recently used, don't discard. */ #define DCACHE_UNHASHED 0x0010 -#define DCACHE_INOTIFY_PARENT_WATCHED 0x0020/* Parent inode is watched */ +#define DCACHE_INOTIFY_PARENT_WATCHED 0x0020/* Parent inode is watched by inotify */ +#define DCACHE_FSNOTIFY_PARENT_WATCHED 0x0040/* Parent inode is watched by some fsnotify listener */ extern spinlock_t dcache_lock;extern seqlock_t rename_lock;
include/linux/fsnotify.h +100 -3
168 lines not on this page · raw message
include/linux/fsnotify_backend.h +5 -0
@@ -65,6 +65,11 @@#define FS_DN_RENAME 0x1000000000000000ull/* file renamed */ #define FS_DN_MULTISHOT 0x2000000000000000ull/* dnotify multishot */ +#define FS_EVENTS_WITH_CHILD (FS_ACCESS | FS_MODIFY | FS_ATTRIB |\ + FS_CLOSE_WRITE | FS_CLOSE_NOWRITE | FS_OPEN |\ + FS_MOVED_FROM | FS_MOVED_TO | FS_CREATE |\ + FS_DELETE) +/* when calling fsnotify tell it if the data is a file, dentry, or inode */ #define FSNOTIFY_EVENT_FILE 1#define FSNOTIFY_EVENT_INODE 2
Eric Paris
maintainerRFC PATCH 9/14 dnotify: reimplement dnotify using fsnotify
In reply to Eric ParisReimplement dnotify using fsnotify.
---
fs/fcntl.c | 4 + fs/notify/dnotify/Kconfig | 1 fs/notify/dnotify/dnotify.c | 322 ++++++++++++++++++++++++++++++++----------- include/linux/dnotify.h | 29 +--- include/linux/fs.h | 5 - include/linux/fsnotify.h | 71 +++------ 6 files changed, 271 insertions(+), 161 deletions(-)
fs/fcntl.c +4 -0
@@ -327,6 +327,10 @@ static long do_fcntl(int fd, unsigned int cmd, unsigned long arg,break ;case F_NOTIFY : err =fcntl_dirnotify (fd, filp, arg); +if (err) +break ; +if (filp->f_op && filp->f_op->dir_notify) + err = filp->f_op->dir_notify (filp, arg);break ;default :break ;
fs/notify/dnotify/Kconfig +1 -0
@@ -1,5 +1,6 @@ config DNOTIFY bool "Dnotify support" + depends on FSNOTIFY default y help Dnotify is a directory-based per-fd file change notification system
fs/notify/dnotify/dnotify.c +236 -86
401 lines not on this page · raw message
include/linux/dnotify.h +8 -21
@@ -10,7 +10,7 @@struct dnotify_struct {struct dnotify_struct * dn_next; -unsigned long dn_mask; +__u64 dn_mask;int dn_fd;struct file * dn_filp;fl_owner_t dn_owner;
@@ -21,23 +21,18 @@ struct dnotify_struct {#ifdef CONFIG_DNOTIFY -extern void __inode_dir_notify (struct inode *,unsigned long ); +#define ALL_DNOTIFY_EVENTS (FS_DELETE | FS_DELETE_CHILD |\ + FS_MODIFY | FS_MODIFY_CHILD |\ + FS_ACCESS | FS_ACCESS_CHILD |\ + FS_ATTRIB | FS_ATTRIB_CHILD |\ + FS_CREATE | FS_DN_RENAME |\ + FS_MOVED_FROM | FS_MOVED_TO) +extern void dnotify_flush (struct file *,fl_owner_t );extern int fcntl_dirnotify (int ,struct file *,unsigned long ); -extern void dnotify_parent (struct dentry *,unsigned long ); - -static inline void inode_dir_notify (struct inode *inode,unsigned long event) -{ -if (inode->i_dnotify_mask & (event)) -__inode_dir_notify (inode, event); -}#else -static inline void __inode_dir_notify (struct inode *inode,unsigned long event) -{ -} -static inline void dnotify_flush (struct file *filp,fl_owner_t id) { }
@@ -47,14 +42,6 @@ static inline int fcntl_dirnotify(int fd, struct file *filp, unsigned long arg)return -EINVAL ; } -static inline void dnotify_parent (struct dentry *dentry,unsigned long event) -{ -} - -static inline void inode_dir_notify (struct inode *inode,unsigned long event) -{ -} -#endif /* CONFIG_DNOTIFY */ #endif /* __KERNEL __ */
include/linux/fs.h +0 -5
@@ -670,11 +670,6 @@ struct inode {struct list_head i_fsnotify_mark_entries;/* fsnotify mark entries */ #endif -#ifdef CONFIG_DNOTIFY -unsigned long i_dnotify_mask;/* Directory notify events */ -struct dnotify_struct *i_dnotify;/* for directory notifications */ -#endif -#ifdef CONFIG_INOTIFY struct list_head inotify_watches;/* watches on this inode */ struct mutex inotify_mutex;/* protects the watches list */
include/linux/fsnotify.h +22 -49
133 lines not on this page · raw message
Eric Paris
maintainerRFC PATCH 10/14 fsnotify: generic notification queue and waitq
In reply to Eric Parisinotify needs to do asyc notification in which event information is stored on a queue until the listener is ready to receive it. This patch implements a generic notification queue for inotify (and later fanotify) to store events to be sent at a later time.
---
fs/notify/fsnotify.h | 29 +++++++++ fs/notify/group.c | 9 +++ fs/notify/notification.c | 127 ++++++++++++++++++++++++++++++++++++++ include/linux/fsnotify_backend.h | 7 ++ 4 files changed, 172 insertions(+), 0 deletions(-)
fs/notify/fsnotify.h +29 -0
@@ -10,6 +10,20 @@#include <linux/fsnotify.h> #include <asm/atomic.h> +/* +* A single event can be queued in multiple group->notification_lists. +* +* each group->notification_list will point to an event_holder which in turns points +* to the actual event that needs to be sent to userspace. +* +* Seemed cheaper to create a refcnt'd event and a small holder for every group +* than create a different event for every group +* +*/ +struct fsnotify_event_holder { +struct fsnotify_event *event; +struct list_head event_list; +};struct fsnotify_event_private_data {struct fsnotify_group *group;
@@ -23,6 +37,12 @@ struct fsnotify_event_private_data {* listener this structure is where you need to be adding fields. */ struct fsnotify_event { +/* +* If we create an event we are also going to need to create a holder +* to link to a group. So embed one holder in the event. Means only +* one allocation for the common case where we only have one group +*/ +struct fsnotify_event_holder holder;spinlock_t lock;/* protection for the associated event_holder and private_list */ struct inode *to_tell;/*
@@ -72,6 +92,13 @@ extern __u64 fsnotify_mask;extern void fsnotify_get_event (struct fsnotify_event *event);extern void fsnotify_put_event (struct fsnotify_event *event);extern struct fsnotify_event_private_data *fsnotify_get_priv_from_event (struct fsnotify_group *group,struct fsnotify_event *event); + +extern void fsnotify_flush_notif (struct fsnotify_group *group); +extern int fsnotify_add_notif_event (struct fsnotify_group *group,struct fsnotify_event *event,struct fsnotify_event_private_data *priv); +extern int fsnotify_check_notif_queue (struct fsnotify_group *group); +extern struct fsnotify_event *fsnotify_peek_notif_event (struct fsnotify_group *group); +extern struct fsnotify_event *fsnotify_remove_notif_event (struct fsnotify_group *group); +extern struct fsnotify_event *fsnotify_create_event (struct inode *to_tell,__u64 mask,void *data,int data_is);extern void fsnotify_clear_marks_by_group (struct fsnotify_group *group);
@@ -79,4 +106,6 @@ extern void fsnotify_clear_marks_by_inode(struct inode *inode, unsigned int flagextern void fsnotify_destroy_mark_by_entry (struct fsnotify_mark_entry *entry);extern void fsnotify_get_mark (struct fsnotify_mark_entry *entry);extern void fsnotify_put_mark (struct fsnotify_mark_entry *entry); +extern struct fsnotify_event_holder *fsnotify_alloc_event_holder (void ); +extern void fsnotify_destroy_event_holder (struct fsnotify_event_holder *holder);#endif /* _LINUX_FSNOTIFY_PRIVATE_H */
fs/notify/group.c +9 -0
@@ -91,6 +91,9 @@ void fsnotify_get_group(struct fsnotify_group *group)static void fsnotify_destroy_group (struct fsnotify_group *group) { +/* clear the notification queue of all events */ +fsnotify_flush_notif (group); +/* clear all inode mark entries for this group */ fsnotify_clear_marks_by_group (group);
@@ -168,6 +171,12 @@ struct fsnotify_group *fsnotify_obtain_group(unsigned int priority, unsigned int group->group_num = group_num; group->mask = mask; +mutex_init (&group->notification_mutex); +INIT_LIST_HEAD (&group->notification_list); +init_waitqueue_head (&group->notification_waitq); + group->q_len =0 ; + group->max_events =UINT_MAX ; +spin_lock_init (&group->mark_lock);INIT_LIST_HEAD (&group->mark_entries);
fs/notify/notification.c +127 -0
157 lines not on this page · raw message
include/linux/fsnotify_backend.h +7 -0
@@ -99,6 +99,13 @@ struct fsnotify_group {const struct fsnotify_ops *ops;/* how this group handles things */ +/* needed to send notification to userspace */ +struct mutex notification_mutex;/* protect the notification_list */ +struct list_head notification_list;/* list of event_holder this group needs to send to userspace */ +wait_queue_head_t notification_waitq;/* read() on the notification file blocks on this waitq */ +unsigned int q_len;/* events on the queue */ +unsigned int max_events;/* maximum events allowed on the list */ +/* stores all fastapth entries assoc with this group so they can be cleaned on unregister */ spinlock_t mark_lock;/* protect mark_entries list */ struct list_head mark_entries;/* all inode mark entries for this group */
Eric Paris
maintainerRFC PATCH 11/14 fsnotify: include pathnames with entries when possible
In reply to Eric ParisWhen inotify wants to send events to a directory about a child it includes the name of the original file. This patch collects that filename and makes it available for notification.
---
fs/notify/fsnotify.c | 4 ++-- fs/notify/fsnotify.h | 9 ++++++--- fs/notify/notification.c | 19 ++++++++++++++++++- include/linux/fsnotify.h | 34 +++++++++++++++++----------------- include/linux/fsnotify_backend.h | 4 ++-- 5 files changed, 45 insertions(+), 25 deletions(-)
fs/notify/fsnotify.c +2 -2
@@ -34,7 +34,7 @@ void __fsnotify_inode_delete(struct inode *inode, int flag) }EXPORT_SYMBOL_GPL (__fsnotify_inode_delete); -void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is) +void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *file_name) {struct fsnotify_group *group;struct fsnotify_event *event =NULL ;
@@ -66,7 +66,7 @@ void fsnotify(struct inode *to_tell, __u64 mask, void *data, int data_is)if (!group->ops->should_send_event (group, to_tell, mask))continue ;if (!event) { - event =fsnotify_create_event (to_tell, mask, data, data_is); + event =fsnotify_create_event (to_tell, mask, data, data_is, file_name);/* shit, we OOM'd and now we can't tell, lets hope something else blows up */ if (!event)break ;
fs/notify/fsnotify.h +6 -3
@@ -57,6 +57,9 @@ struct fsnotify_event {__u64 mask;/* the type of access */ atomic_t refcnt;/* how many groups still are using/need to send this event */ +char *file_name; +size_t name_len; +struct list_head private_data_list; };
@@ -99,13 +102,13 @@ extern int fsnotify_check_notif_queue(struct fsnotify_group *group);extern struct fsnotify_event *fsnotify_peek_notif_event (struct fsnotify_group *group);extern struct fsnotify_event *fsnotify_remove_notif_event (struct fsnotify_group *group); -extern struct fsnotify_event *fsnotify_create_event (struct inode *to_tell,__u64 mask,void *data,int data_is); +extern struct fsnotify_event *fsnotify_create_event (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *name);extern void fsnotify_clear_marks_by_group (struct fsnotify_group *group);extern void fsnotify_clear_marks_by_inode (struct inode *inode,unsigned int flags);extern void fsnotify_destroy_mark_by_entry (struct fsnotify_mark_entry *entry);extern void fsnotify_get_mark (struct fsnotify_mark_entry *entry);extern void fsnotify_put_mark (struct fsnotify_mark_entry *entry); -extern struct fsnotify_event_holder *fsnotify_alloc_event_holder (void ); -extern void fsnotify_destroy_event_holder (struct fsnotify_event_holder *holder); +extern struct fsnotify_event_holder *fsnotify_alloc_event_holder (void ); +extern void fsnotify_destroy_event_holder (struct fsnotify_event_holder *holder);#endif /* _LINUX_FSNOTIFY_PRIVATE_H */
fs/notify/notification.c +18 -1
@@ -62,6 +62,7 @@ void fsnotify_put_event(struct fsnotify_event *event) event->mask =0 ;BUG_ON (!list_empty (&event->private_data_list)); +kfree (event->file_name);kmem_cache_free (event_kmem_cache, event); } }
@@ -196,7 +197,7 @@ void fsnotify_flush_notif(struct fsnotify_group *group)mutex_unlock (&group->notification_mutex); } -struct fsnotify_event *fsnotify_create_event (struct inode *to_tell,__u64 mask,void *data,int data_is) +struct fsnotify_event *fsnotify_create_event (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *name) {struct fsnotify_event *event;
@@ -204,6 +205,18 @@ struct fsnotify_event *fsnotify_create_event(struct inode *to_tell, __u64 mask,if (!event)return NULL ; +if (name) { + event->file_name =kstrdup (name,GFP_KERNEL ); +if (!event->file_name) { +kmem_cache_free (event_kmem_cache, event); +return NULL ; + } + event->name_len =strlen (event->file_name); + }else { + event->file_name =NULL ; + event->name_len =0 ; + } + event->holder.event =NULL ;INIT_LIST_HEAD (&event->holder.event_list);atomic_set (&event->refcnt,1 );
@@ -222,6 +235,7 @@ struct fsnotify_event *fsnotify_create_event(struct inode *to_tell, __u64 mask,switch (data_is) {case FSNOTIFY_EVENT_FILE : {struct file *file = data; + event->path.dentry = file->f_path.dentry; event->path.mnt = file->f_path.mnt;path_get (&event->path);
@@ -231,6 +245,9 @@ struct fsnotify_event *fsnotify_create_event(struct inode *to_tell, __u64 mask, event->inode = data;break ;default : + event->path.dentry =NULL ; + event->path.mnt =NULL ; + event->inode =NULL ;BUG (); };
include/linux/fsnotify.h +17 -17
@@ -98,7 +98,7 @@ static inline void fsnotify_parent(struct dentry *dentry, __u64 orig_mask)if (p_inode && (p_inode->i_fsnotify_mask & mask)) {dget (parent);spin_unlock (&dentry->d_lock); -fsnotify (p_inode, mask, dentry->d_inode,FSNOTIFY_EVENT_INODE ); +fsnotify (p_inode, mask, dentry->d_inode,FSNOTIFY_EVENT_INODE , dentry->d_name.name);dput (parent); }else {spin_unlock (&dentry->d_lock);
@@ -152,18 +152,18 @@ static inline void fsnotify_move(struct inode *old_dir, struct inode *new_dir,inotify_inode_queue_event (new_dir,IN_MOVED_TO |isdir, cookie, new_name, source); -fsnotify (old_dir, old_dir_mask, old_dir,FSNOTIFY_EVENT_INODE ); -fsnotify (new_dir, new_dir_mask, new_dir,FSNOTIFY_EVENT_INODE ); +fsnotify (old_dir, old_dir_mask, old_dir,FSNOTIFY_EVENT_INODE , old_name); +fsnotify (new_dir, new_dir_mask, new_dir,FSNOTIFY_EVENT_INODE , new_name);if (target) {inotify_inode_queue_event (target,IN_DELETE_SELF ,0 ,NULL ,NULL );inotify_inode_is_dead (target); -fsnotify (target,FS_DELETE , target,FSNOTIFY_EVENT_INODE ); +fsnotify (target,FS_DELETE , target,FSNOTIFY_EVENT_INODE ,NULL ); }if (source) {inotify_inode_queue_event (source,IN_MOVE_SELF ,0 ,NULL ,NULL ); -fsnotify (source,FS_MOVE_SELF , moved->d_inode,FSNOTIFY_EVENT_INODE ); +fsnotify (source,FS_MOVE_SELF , moved->d_inode,FSNOTIFY_EVENT_INODE ,NULL ); }audit_inode_child (new_name, moved, new_dir); }
@@ -196,7 +196,7 @@ static inline void fsnotify_inoderemove(struct inode *inode)inotify_inode_queue_event (inode,IN_DELETE_SELF ,0 ,NULL ,NULL );inotify_inode_is_dead (inode); -fsnotify (inode,FS_DELETE_SELF , inode,FSNOTIFY_EVENT_INODE ); +fsnotify (inode,FS_DELETE_SELF , inode,FSNOTIFY_EVENT_INODE ,NULL );__fsnotify_inode_delete (inode,FSNOTIFY_LAST_DENTRY ); }
@@ -207,7 +207,7 @@ static inline void fsnotify_link_count(struct inode *inode) {inotify_inode_queue_event (inode,IN_ATTRIB ,0 ,NULL ,NULL ); -fsnotify (inode,FS_ATTRIB , inode,FSNOTIFY_EVENT_INODE ); +fsnotify (inode,FS_ATTRIB , inode,FSNOTIFY_EVENT_INODE ,NULL ); }/*
@@ -219,7 +219,7 @@ static inline void fsnotify_create(struct inode *inode, struct dentry *dentry) dentry->d_inode);audit_inode_child (dentry->d_name.name, dentry, inode); -fsnotify (inode,FS_CREATE , dentry->d_inode,FSNOTIFY_EVENT_INODE ); +fsnotify (inode,FS_CREATE , dentry->d_inode,FSNOTIFY_EVENT_INODE , dentry->d_name.name); }/*
@@ -234,7 +234,7 @@ static inline void fsnotify_link(struct inode *dir, struct inode *inode, structfsnotify_link_count (inode);audit_inode_child (new_dentry->d_name.name, new_dentry, dir); -fsnotify (dir,FS_CREATE , inode,FSNOTIFY_EVENT_INODE ); +fsnotify (dir,FS_CREATE , inode,FSNOTIFY_EVENT_INODE , new_dentry->d_name.name); }/*
@@ -246,7 +246,7 @@ static inline void fsnotify_mkdir(struct inode *inode, struct dentry *dentry) dentry->d_name.name, dentry->d_inode);audit_inode_child (dentry->d_name.name, dentry, inode); -fsnotify (inode,FS_CREATE |FS_IN_ISDIR , dentry->d_inode,FSNOTIFY_EVENT_INODE ); +fsnotify (inode,FS_CREATE |FS_IN_ISDIR , dentry->d_inode,FSNOTIFY_EVENT_INODE , dentry->d_name.name); }/*
@@ -265,7 +265,7 @@ static inline void fsnotify_access(struct file *file)inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL );fsnotify_parent (dentry, mask); -fsnotify (inode, mask, file,FSNOTIFY_EVENT_FILE ); +fsnotify (inode, mask, file,FSNOTIFY_EVENT_FILE ,NULL ); }/*
@@ -284,7 +284,7 @@ static inline void fsnotify_modify(struct file *file)inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL );fsnotify_parent (dentry, mask); -fsnotify (inode, mask, file,FSNOTIFY_EVENT_FILE ); +fsnotify (inode, mask, file,FSNOTIFY_EVENT_FILE ,NULL ); }/*
@@ -299,7 +299,7 @@ static inline void fsnotify_open_exec(struct file *file)inotify_inode_queue_event (inode,IN_ACCESS ,0 ,NULL ,NULL );fsnotify_parent (dentry,FS_ACCESS ); -fsnotify (inode,FS_ACCESS , file,FSNOTIFY_EVENT_FILE ); +fsnotify (inode,FS_ACCESS , file,FSNOTIFY_EVENT_FILE ,NULL ); }/*
@@ -318,7 +318,7 @@ static inline void fsnotify_open(struct file *file)inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL );fsnotify_parent (dentry, mask); -fsnotify (inode, mask, file,FSNOTIFY_EVENT_FILE ); +fsnotify (inode, mask, file,FSNOTIFY_EVENT_FILE ,NULL ); }/*
@@ -339,7 +339,7 @@ static inline void fsnotify_close(struct file *file)inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL );fsnotify_parent (dentry, mask); -fsnotify (inode, mask, file,FSNOTIFY_EVENT_FILE ); +fsnotify (inode, mask, file,FSNOTIFY_EVENT_FILE ,NULL ); }/*
@@ -357,7 +357,7 @@ static inline void fsnotify_xattr(struct dentry *dentry)inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL );fsnotify_parent (dentry, mask); -fsnotify (inode, mask, inode,FSNOTIFY_EVENT_INODE ); +fsnotify (inode, mask, inode,FSNOTIFY_EVENT_INODE ,NULL ); }/*
@@ -395,7 +395,7 @@ static inline void fsnotify_change(struct dentry *dentry, unsigned int ia_valid) dentry->d_name.name);fsnotify_parent (dentry, mask); -fsnotify (inode, mask, inode,FSNOTIFY_EVENT_INODE ); +fsnotify (inode, mask, inode,FSNOTIFY_EVENT_INODE ,NULL ); } }
include/linux/fsnotify_backend.h +2 -2
@@ -118,7 +118,7 @@ struct fsnotify_group {#ifdef CONFIG_FSNOTIFY /* called from the vfs to signal fs events */ -extern void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is); +extern void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *name);extern void __fsnotify_inode_delete (struct inode *inode,int flag);/* called from fsnotify interfaces, such as fanotify or dnotify */
@@ -133,7 +133,7 @@ extern struct fsnotify_mark_entry *fsnotify_find_mark_entry(struct fsnotify_grouextern struct fsnotify_mark_entry *fsnotify_mark_add (struct fsnotify_group *group,struct inode *inode,__u64 mask);#else -static inline void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is); +static inline void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *name); {}static inline void __fsnotify_inode_delete (struct inode *inode,int flag)
Re: [RFC PATCH -v4 11/14] fsnotify: include pathnames with entries when possible
Quoted 3 lines · Eric Paris
When inotify wants to send events to a directory about a child it includes the name of the original file. This patch collects that filename and makes it available for notification.
What about extending fsnotify with attributes so that there would be no problems extending it with new events and data sent to userspace?
Signature · 1 line
Evgeniy Polyakov
Eric Paris
maintainerRe: [RFC PATCH -v4 11/14] fsnotify: include pathnames with entries when possible
Quoted 7 lines · Evgeniy Polyakov
Quoted 3 lines · Eric Paris
When inotify wants to send events to a directory about a child it includes the name of the original file. This patch collects that filename and makes it available for notification.
What about extending fsnotify with attributes so that there would be no problems extending it with new events and data sent to userspace?
Since struct fsnotify_event is not exported to userspace information may be collected in there at event creation time. I actually have a number of later patches that do this for information that I want to send for fanotify.
Information can also be stored in the event->private area if only one
group really cares.... (kinda how I store wd for inotify)
Information which is useful to more than 1 group should be added to struct fsnotify_event and its up to the individual listeners to export that to userspace anyway they want to.
So absolutely this is completely extensible.
Eric Paris
maintainerRFC PATCH 12/14 fsnotify: add correlations between events
In reply to Eric Parisinotify sends userspace a correlation between events when they are related (aka when dentries are moved). This adds that same support for all fsnotify events.
---
fs/notify/fsnotify.c | 4 ++-- fs/notify/fsnotify.h | 3 ++- fs/notify/notification.c | 12 ++++++++++- include/linux/fsnotify.h | 41 +++++++++++++++++++------------------- include/linux/fsnotify_backend.h | 10 +++++++-- 5 files changed, 44 insertions(+), 26 deletions(-)
fs/notify/fsnotify.c +2 -2
@@ -34,7 +34,7 @@ void __fsnotify_inode_delete(struct inode *inode, int flag) }EXPORT_SYMBOL_GPL (__fsnotify_inode_delete); -void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *file_name) +void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *file_name,u32 cookie) {struct fsnotify_group *group;struct fsnotify_event *event =NULL ;
@@ -66,7 +66,7 @@ void fsnotify(struct inode *to_tell, __u64 mask, void *data, int data_is, constif (!group->ops->should_send_event (group, to_tell, mask))continue ;if (!event) { - event =fsnotify_create_event (to_tell, mask, data, data_is, file_name); + event =fsnotify_create_event (to_tell, mask, data, data_is, file_name, cookie);/* shit, we OOM'd and now we can't tell, lets hope something else blows up */ if (!event)break ;
fs/notify/fsnotify.h +2 -1
@@ -57,6 +57,7 @@ struct fsnotify_event {__u64 mask;/* the type of access */ atomic_t refcnt;/* how many groups still are using/need to send this event */ +u32 sync_cookie;char *file_name;size_t name_len;
@@ -102,7 +103,7 @@ extern int fsnotify_check_notif_queue(struct fsnotify_group *group);extern struct fsnotify_event *fsnotify_peek_notif_event (struct fsnotify_group *group);extern struct fsnotify_event *fsnotify_remove_notif_event (struct fsnotify_group *group); -extern struct fsnotify_event *fsnotify_create_event (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *name); +extern struct fsnotify_event *fsnotify_create_event (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *name,u32 cookie);extern void fsnotify_clear_marks_by_group (struct fsnotify_group *group);extern void fsnotify_clear_marks_by_inode (struct inode *inode,unsigned int flags);
fs/notify/notification.c +11 -1
@@ -20,6 +20,7 @@#include <linux/init.h> #include <linux/kernel.h> #include <linux/list.h> +#include <linux/module.h> #include <linux/mount.h> #include <linux/mutex.h> #include <linux/namei.h>
@@ -34,6 +35,13 @@static struct kmem_cache *event_kmem_cache;static struct kmem_cache *event_holder_kmem_cache; +static atomic_t fsnotify_sync_cookie =ATOMIC_INIT (0 ); + +u32 fsnotify_get_cookie (void ) +{ +return atomic_inc_return (&fsnotify_sync_cookie); +} +EXPORT_SYMBOL_GPL (fsnotify_get_cookie);int fsnotify_check_notif_queue (struct fsnotify_group *group) {
@@ -197,7 +205,7 @@ void fsnotify_flush_notif(struct fsnotify_group *group)mutex_unlock (&group->notification_mutex); } -struct fsnotify_event *fsnotify_create_event (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *name) +struct fsnotify_event *fsnotify_create_event (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *name,u32 cookie) {struct fsnotify_event *event;
@@ -217,6 +225,8 @@ struct fsnotify_event *fsnotify_create_event(struct inode *to_tell, __u64 mask, event->name_len =0 ; } + event->sync_cookie = cookie; + event->holder.event =NULL ;INIT_LIST_HEAD (&event->holder.event_list);atomic_set (&event->refcnt,1 );
include/linux/fsnotify.h +21 -20
142 lines not on this page · raw message
include/linux/fsnotify_backend.h +8 -2
@@ -118,8 +118,9 @@ struct fsnotify_group {#ifdef CONFIG_FSNOTIFY /* called from the vfs to signal fs events */ -extern void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *name); +extern void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *name,u32 cookie);extern void __fsnotify_inode_delete (struct inode *inode,int flag); +extern u32 fsnotify_get_cookie (void );/* called from fsnotify interfaces, such as fanotify or dnotify */ extern void fsnotify_recalc_global_mask (void );
@@ -133,12 +134,17 @@ extern struct fsnotify_mark_entry *fsnotify_find_mark_entry(struct fsnotify_grouextern struct fsnotify_mark_entry *fsnotify_mark_add (struct fsnotify_group *group,struct inode *inode,__u64 mask);#else -static inline void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *name); +static inline void fsnotify (struct inode *to_tell,__u64 mask,void *data,int data_is,const char *name,u32 cookie); {}static inline void __fsnotify_inode_delete (struct inode *inode,int flag) {} +static inline u32 fsnotify_get_cookie (void ) +{ +return 0 ; +} +#endif /* CONFIG_FSNOTIFY */ #endif /* __KERNEL __ */
Re: [RFC PATCH -v4 12/14] fsnotify: add correlations between events
Quoted 11 lines · Eric Paris
inotify sends userspace a correlation between events when they are related (aka when dentries are moved). This adds that same support for all fsnotify events.
fs/notify/notification.c +0
-0@@ -34,6 +35,13 @@static struct kmem_cache *event_kmem_cache;static struct kmem_cache *event_holder_kmem_cache;
fs/notify/notification.c line 47
Quoted 10 lines
fs/notify/notification.c +7
-0+static atomic_t fsnotify_sync_cookie =ATOMIC_INIT (0 ); + +u32 fsnotify_get_cookie (void ) +{ +return atomic_inc_return (&fsnotify_sync_cookie); +} +EXPORT_SYMBOL_GPL (fsnotify_get_cookie);int fsnotify_check_notif_queue (struct fsnotify_group *group) {
atomic_inc_return seems rather expensive to put on a hot path in almost every fs operation. On a multiprocessor system, the cache line for fsnotify_sync_cookie would be ping-ponging constantly between processors. The canonical solution is to form the cookie by concatenating the processor number with a per-processor cookie, so that generating a new cookie would not require synchronization between processors. Surely this code already exists to be used somewhere in Linux.
--scott
Signature · 1 line
( http://cscott.net/ )
Eric Paris
maintainerRe: [RFC PATCH -v4 12/14] fsnotify: add correlations between events
Quoted 20 lines · C. Scott Ananian
Quoted 19 lines · Eric Paris
inotify sends userspace a correlation between events when they are related (aka when dentries are moved). This adds that same support for all fsnotify events.
fs/notify/notification.c +7
-0@@ -34,6 +35,13 @@static struct kmem_cache *event_kmem_cache;static struct kmem_cache *event_holder_kmem_cache; +static atomic_t fsnotify_sync_cookie =ATOMIC_INIT (0 ); + +u32 fsnotify_get_cookie (void ) +{ +return atomic_inc_return (&fsnotify_sync_cookie); +} +EXPORT_SYMBOL_GPL (fsnotify_get_cookie);
Quoted 11 lines
fs/notify/notification.c line 47
Quoted 2 lines
fs/notify/notification.c +0
-0int fsnotify_check_notif_queue (struct fsnotify_group *group) {
atomic_inc_returnseems rather expensive to put on a hot path in almost every fs operation. On a multiprocessor system, the cache line forfsnotify_sync_cookiewould be ping-ponging constantly between processors. The canonical solution is to form the cookie by concatenating the processor number with a per-processor cookie, so that generating a new cookie would not require synchronization between processors. Surely this code already exists to be used somewhere in Linux.
A) this isn't a hot path, it's only when a file is renamed.
B) It's still a great idea (even if not here at least where I plan to use an atomic_inc_return in my later fanotify patches). Does anyone know of an example of something like this in kernel?
Here I'm slightly concerned wasting enough bits of the 32 (we can't grow it, because inotify has a fixed abi) for processors, but I'd be surprised to find we'd ever have enough renames on a single processor (my 2 second math would be we'd need about 13-14 bits for processors which still leaves 262k renames per processor before anything wrapped) to be a problem.
anyone who uses inotify think there would be a problem with cookie reuse coming this soon?
Since I'm using a 64bit cookie and the lifetime of use in my later code is measured in seconds I think this would be a great improvement. Thanks
-Eric
Re: [RFC PATCH -v4 12/14] fsnotify: add correlations between events
Quoted 2 lines · Eric Paris
anyone who uses inotify think there would be a problem with cookie reuse coming this soon?
In my experience we only care about cookies to be the same, and not monotinically grow up. So this may just jump up and down, and since events are potentially distributed in time (although follow one after another), wrapping should roughly cover number of processes running (and potentially renaming objects), so having 20 bits for the counter and 12 for the CPU id should be enough for now.
Signature · 1 line
Evgeniy Polyakov
Re: [RFC PATCH -v4 12/14] fsnotify: add correlations between events
In reply to Eric ParisQuoted 1 line · Eric Paris
A) this isn't a hot path, it's only when a file is renamed.
Yes, sorry, I misread the description of that patch to indicate that you planned to include a newly-generated cookie in all fsevents. Re-reading the patch I see that you are still only generating cookies for renames. But yeah, better cookie generation will help scalability if you plan to use them more broadly in fanotify.
--scott
Signature · 1 line
( http://cscott.net/ )
Eric Paris
maintainerRFC PATCH 13/14 inotify: reimplement inotify using fsnotify
In reply to Eric ParisYes, holy shit, I'm trying to reimplement inotify as fsnotify...
---
fs/inode.c | 1 fs/notify/inotify/Kconfig | 20 + fs/notify/inotify/Makefile | 2 fs/notify/inotify/inotify.h | 117 +++++++ fs/notify/inotify/inotify_fsnotify.c | 183 +++++++++++ fs/notify/inotify/inotify_kernel.c | 293 +++++++++++++++++ fs/notify/inotify/inotify_user.c | 591 +++++++++------------------------- include/linux/fsnotify.h | 39 +- include/linux/inotify.h | 1 9 files changed, 783 insertions(+), 464 deletions(-) create mode 100644 fs/notify/inotify/inotify.h create mode 100644 fs/notify/inotify/inotify_fsnotify.c create mode 100644 fs/notify/inotify/inotify_kernel.c
fs/inode.c +1 -0
@@ -372,6 +372,7 @@ int invalidate_inodes(struct super_block * sb)mutex_lock (&iprune_mutex);spin_lock (&inode_lock);inotify_unmount_inodes (&sb->s_inodes); +fsn_inotify_unmount_inodes (&sb->s_inodes); busy =invalidate_list (&sb->s_inodes, &throw_away);spin_unlock (&inode_lock);
fs/notify/inotify/Kconfig +12 -8
@@ -1,26 +1,30 @@ config INOTIFY bool "Inotify file change notification support" - default y + default n ---help--- - Say Y here to enable inotify support. Inotify is a file change - notification system and a replacement for dnotify. Inotify fixes - numerous shortcomings in dnotify and introduces several new features - including multiple file events, one-shot support, and unmount - notification. + Say Y here to enable legacy in kernel inotify support. Inotify is a + file change notification system. It is a replacement for dnotify. + This option only provides the legacy inotify in kernel API. There + are no in tree kernel users of this interface since it is deprecated. + You only need this if you are loading an out of tree kernel module + that uses inotify. For more information, see <file:Documentation/filesystems/inotify.txt> - If unsure, say Y. + If unsure, say N. config INOTIFY_USER bool "Inotify support for userspace" - depends on INOTIFY + depends on FSNOTIFY default y ---help--- Say Y here to enable inotify support for userspace, including the associated system calls. Inotify allows monitoring of both files and directories via a single open fd. Events are read from the file descriptor, which is also select()- and poll()-able. + Inotify fixes numerous shortcomings in dnotify and introduces several + new features including multiple file events, one-shot support, and + unmount notification. For more information, see <file:Documentation/filesystems/inotify.txt>
fs/notify/inotify/Makefile +1 -1
@@ -1,2 +1,2 @@ obj-$(CONFIG_INOTIFY) += inotify.o -obj-$(CONFIG_INOTIFY_USER) += inotify_user.o +obj-$(CONFIG_INOTIFY_USER) += inotify_fsnotify.o inotify_kernel.o inotify_user.o
fs/notify/inotify/inotify.h new file +117 -0
117 lines not on this page · raw message
fs/notify/inotify/inotify_fsnotify.c new file +183 -0
183 lines not on this page · raw message
fs/notify/inotify/inotify_kernel.c new file +293 -0
293 lines not on this page · raw message
fs/notify/inotify/inotify_user.c +156 -435
790 lines not on this page · raw message
include/linux/fsnotify.h +19 -20
@@ -257,10 +257,10 @@ static inline void fsnotify_access(struct file *file) {struct dentry *dentry = file->f_path.dentry;struct inode *inode = dentry->d_inode; -__u64 mask =IN_ACCESS ; +__u64 mask =FS_ACCESS ;if (S_ISDIR (inode->i_mode)) - mask |=IN_ISDIR ; + mask |=FS_IN_ISDIR ;inotify_dentry_parent_queue_event (dentry, mask,0 , dentry->d_name.name);inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL );
@@ -276,10 +276,10 @@ static inline void fsnotify_modify(struct file *file) {struct dentry *dentry = file->f_path.dentry;struct inode *inode = dentry->d_inode; -__u64 mask =IN_MODIFY ; +__u64 mask =FS_MODIFY ;if (S_ISDIR (inode->i_mode)) - mask |=IN_ISDIR ; + mask |=FS_IN_ISDIR ;inotify_dentry_parent_queue_event (dentry, mask,0 , dentry->d_name.name);inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL );
@@ -310,10 +310,10 @@ static inline void fsnotify_open(struct file *file) {struct dentry *dentry = file->f_path.dentry;struct inode *inode = dentry->d_inode; -__u64 mask =IN_OPEN ; +__u64 mask =FS_OPEN ;if (S_ISDIR (inode->i_mode)) - mask |=IN_ISDIR ; + mask |=FS_IN_ISDIR ;inotify_dentry_parent_queue_event (dentry, mask,0 , dentry->d_name.name);inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL );
@@ -329,14 +329,13 @@ static inline void fsnotify_close(struct file *file) {struct dentry *dentry = file->f_path.dentry;struct inode *inode = dentry->d_inode; -const char *name = dentry->d_name.name;fmode_t mode = file->f_mode; -__u64 mask = (mode &FMODE_WRITE ) ?IN_CLOSE_WRITE :IN_CLOSE_NOWRITE ; +__u64 mask = (mode &FMODE_WRITE ) ?FS_CLOSE_WRITE :FS_CLOSE_NOWRITE ;if (S_ISDIR (inode->i_mode)) - mask |=IN_ISDIR ; + mask |=FS_IN_ISDIR ; -inotify_dentry_parent_queue_event (dentry, mask,0 , name); +inotify_dentry_parent_queue_event (dentry, mask,0 , dentry->d_name.name);inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL );fsnotify_parent (dentry, mask);
@@ -349,10 +348,10 @@ static inline void fsnotify_close(struct file *file)static inline void fsnotify_xattr (struct dentry *dentry) {struct inode *inode = dentry->d_inode; -__u64 mask =IN_ATTRIB ; +__u64 mask =FS_ATTRIB ;if (S_ISDIR (inode->i_mode)) - mask |=IN_ISDIR ; + mask |=FS_IN_ISDIR ;inotify_dentry_parent_queue_event (dentry, mask,0 , dentry->d_name.name);inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL );
@@ -371,26 +370,26 @@ static inline void fsnotify_change(struct dentry *dentry, unsigned int ia_valid)__u64 mask =0 ;if (ia_valid &ATTR_UID ) - mask |=IN_ATTRIB ; + mask |=FS_ATTRIB ;if (ia_valid &ATTR_GID ) - mask |=IN_ATTRIB ; + mask |=FS_ATTRIB ;if (ia_valid &ATTR_SIZE ) - mask |=IN_MODIFY ; + mask |=FS_MODIFY ;/* both times implies a utime(s) call */ if ((ia_valid & (ATTR_ATIME |ATTR_MTIME )) == (ATTR_ATIME |ATTR_MTIME )) - mask |=IN_ATTRIB ; + mask |=FS_ATTRIB ;else if (ia_valid &ATTR_ATIME ) - mask |=IN_ACCESS ; + mask |=FS_ACCESS ;else if (ia_valid &ATTR_MTIME ) - mask |=IN_MODIFY ; + mask |=FS_MODIFY ;if (ia_valid &ATTR_MODE ) - mask |=IN_ATTRIB ; + mask |=FS_ATTRIB ;if (mask) {if (S_ISDIR (inode->i_mode)) - mask |=IN_ISDIR ; + mask |=FS_IN_ISDIR ;inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL );inotify_dentry_parent_queue_event (dentry, mask,0 , dentry->d_name.name);
include/linux/inotify.h +1 -0
@@ -112,6 +112,7 @@ extern void inotify_inode_queue_event(struct inode *, __u32, __u32,extern void inotify_dentry_parent_queue_event (struct dentry *,__u32 ,__u32 ,const char *);extern void inotify_unmount_inodes (struct list_head *); +extern void fsn_inotify_unmount_inodes (struct list_head *);extern void inotify_inode_is_dead (struct inode *);extern u32 inotify_get_cookie (void );
Re: [RFC PATCH -v4 13/14] inotify: reimplement inotify using fsnotify
Quoted 1 line · Eric Paris
Yes, holy shit, I'm trying to reimplement inotify as fsnotify...
While you are at it, please update inode_setattr() so that it dropped inotify watches if new permissions do not allow to read data.
Signature · 1 line
Evgeniy Polyakov
Eric Paris
maintainerRe: [RFC PATCH -v4 13/14] inotify: reimplement inotify using fsnotify
Quoted 5 lines · Evgeniy Polyakov
Quoted 1 line · Eric Paris
Yes, holy shit, I'm trying to reimplement inotify as fsnotify...
While you are at it, please update
inode_setattr()so that it dropped inotify watches if new permissions do not allow to read data.
Now this isn't so easy.... Do you have suggestions how to do this with
inotify as it stands today? the vfs needs to know what's hanging on the
open fd in userspace... Not a bad idea. I'll look, but clearly not a
core patch....
-Eric
Re: [RFC PATCH -v4 13/14] inotify: reimplement inotify using fsnotify
Quoted 7 lines · Eric Paris
Quoted 2 lines
While you are at it, please update
inode_setattr()so that it dropped inotify watches if new permissions do not allow to read data.Now this isn't so easy.... Do you have suggestions how to do this with
inotify as it stands today? the vfs needs to know what's hanging on the
open fd in userspace... Not a bad idea. I'll look, but clearly not a
core patch....
I could put inotify check into notify_change(), which will check if new permissions do not allow reading for the users stored in inode->inotify_watches, each one would be dereferenced to inotify device, which holds a user context, which added watch object, so it can be checked the same way generic_permission() works.
Signature · 1 line
Evgeniy Polyakov
Eric Paris
maintainerRFC PATCH 14/14 shit on top for debugging
In reply to Eric Paris---
fs/notify/dnotify/dnotify.c | 6 ++++++ fs/notify/fsnotify.c | 1 + fs/notify/group.c | 18 ++++++++++++++++++ fs/notify/inode_mark.c | 10 ++++++++++ fs/notify/inotify/inotify_fsnotify.c | 7 +++++++ fs/notify/inotify/inotify_kernel.c | 6 ++++++ fs/notify/inotify/inotify_user.c | 3 +++ fs/notify/notification.c | 2 ++ include/linux/fsnotify.h | 1 + 9 files changed, 54 insertions(+), 0 deletions(-)
fs/notify/dnotify/dnotify.c +6 -0
@@ -66,6 +66,8 @@ static void dnotify_recalc_inode_mask(struct fsnotify_mark_entry *entry)unsigned long new_mask;struct dnotify_struct *dn; +printk (KERN_CRIT "%s: inode=%p\n" , __func__, entry->inode); + new_mask =0 ; dn = (struct dnotify_struct *)entry->private;for (; dn !=NULL ; dn = dn->dn_next)
@@ -174,6 +176,8 @@ void dnotify_flush(struct file *filp, fl_owner_t id)if (!entry)return ; +printk (KERN_CRIT "%s: found entry=%p entry->group=%p\n" , __func__, entry, entry->group); +spin_lock (&entry->lock); prev = (struct dnotify_struct **)&entry->private;while ((dn = *prev) !=NULL ) {
@@ -241,6 +245,8 @@ int fcntl_dirnotify(int fd, struct file *filp, unsigned long arg)if (!dir_notify_enable)return -EINVAL ; +printk (KERN_CRIT "%s: fd=%d filp=%p arg=%lx\n" , __func__, fd, filp, arg); +if ((arg & ~DN_MULTISHOT ) ==0 ) {dnotify_flush (filp, id);return 0 ;
fs/notify/fsnotify.c +1 -0
@@ -65,6 +65,7 @@ void fsnotify(struct inode *to_tell, __u64 mask, void *data, int data_is, constif (mask & group->mask) {if (!group->ops->should_send_event (group, to_tell, mask))continue ; +printk (KERN_CRIT "%s: to_tell=%p data=%p data_is=%d mask=%llx fsnotify_mask=%llx\n" , __func__, to_tell, data, data_is, mask, fsnotify_mask);if (!event) { event =fsnotify_create_event (to_tell, mask, data, data_is, file_name, cookie);/* shit, we OOM'd and now we can't tell, lets hope something else blows up */
fs/notify/group.c +18 -0
@@ -39,12 +39,17 @@ void fsnotify_recalc_global_mask(void)__u64 mask =0 ;int idx; +printk (KERN_CRIT "%s: starting fsnotify_mask=%llx\n" , __func__, fsnotify_mask); + idx =srcu_read_lock (&fsnotify_grp_srcu_struct);list_for_each_entry_rcu (group, &fsnotify_groups, group_list) { +printk (KERN_CRIT "%s: group=%p group->mask=%llx\n" , __func__, group, group->mask); mask |= group->mask; }srcu_read_unlock (&fsnotify_grp_srcu_struct, idx); fsnotify_mask = mask; + +printk (KERN_CRIT "%s: ending fsnotify_mask=%llx\n" , __func__, fsnotify_mask); }void fsnotify_recalc_group_mask (struct fsnotify_group *group)
@@ -54,13 +59,19 @@ void fsnotify_recalc_group_mask(struct fsnotify_group *group)struct fsnotify_mark_entry *entry;spin_lock (&group->mark_lock); + +printk (KERN_CRIT "%s: group=%p starting group->mask=%llx\n" , __func__, group, group->mask); +list_for_each_entry (entry, &group->mark_entries, g_list) { +printk (KERN_CRIT "%s: entry=%p entry->mask=%llx\n" , __func__, entry, entry->mask); mask |= entry->mask; }spin_unlock (&group->mark_lock); group->mask = mask; +printk (KERN_CRIT "%s: group=%p finishing group->mask=%llx\n" , __func__, group, group->mask); +if (old_mask != mask)fsnotify_recalc_global_mask (); }
@@ -87,10 +98,14 @@ static void fsnotify_add_group(struct fsnotify_group *group)void fsnotify_get_group (struct fsnotify_group *group) {atomic_inc (&group->refcnt); + +printk (KERN_CRIT "%s: group=%p refcnt=%d\n" , __func__, group,atomic_read (&group->refcnt)); }static void fsnotify_destroy_group (struct fsnotify_group *group) { +printk (KERN_CRIT "%s: group=%p refcnt=%d\n" , __func__, group,atomic_read (&group->refcnt)); +/* clear the notification queue of all events */ fsnotify_flush_notif (group);
@@ -105,6 +120,7 @@ static void fsnotify_destroy_group(struct fsnotify_group *group)void fsnotify_put_group (struct fsnotify_group *group) { +printk (KERN_CRIT "%s: group=%p refcnt=%d\n" , __func__, group,atomic_read (&group->refcnt));if (atomic_dec_and_test (&group->refcnt)) {mutex_lock (&fsnotify_grp_mutex);list_del_rcu (&group->group_list);
@@ -200,5 +216,7 @@ struct fsnotify_group *fsnotify_obtain_group(unsigned int priority, unsigned intif (mask)fsnotify_recalc_global_mask (); +printk (KERN_CRIT "%s: group=%p refcnt=%d\n" , __func__, group,atomic_read (&group->refcnt)); +return group; }
fs/notify/inode_mark.c +10 -0
@@ -58,11 +58,13 @@ static struct fsnotify_mark_entry *fsnotify_alloc_mark(void)void fsnotify_get_mark (struct fsnotify_mark_entry *entry) { +printk (KERN_CRIT "%s: entry=%p refcnt BEFORE the get=%x\n" , __func__, entry,atomic_read (&entry->refcnt));atomic_inc (&entry->refcnt); }void fsnotify_put_mark (struct fsnotify_mark_entry *entry) { +printk (KERN_CRIT "%s: entry=%p refcnt BEFORE the put=%x\n" , __func__, entry,atomic_read (&entry->refcnt));if (atomic_dec_and_test (&entry->refcnt)) {spin_lock (&entry->lock);/* entries can only be found by the kernel by searching the
@@ -95,6 +97,7 @@ void fsnotify_clear_marks_by_group(struct fsnotify_group *group)list_for_each_entry_safe (entry, lentry, &free_list, free_g_list) {fsnotify_get_mark (entry); +printk (KERN_CRIT "%s: entry=%p entry->inode=%p entry->group=%p\n" , __func__, entry, entry->inode, entry->group);spin_lock (&entry->lock); inode = entry->inode;if (!inode) {
@@ -127,6 +130,8 @@ void fsnotify_destroy_mark_by_entry(struct fsnotify_mark_entry *entry)spin_lock (&entry->lock); +printk (KERN_CRIT "%s: entry=%p entry->inode=%p entry->group=%p\n" , __func__, entry, entry->inode, entry->group); + group = entry->group;if (group)spin_lock (&group->mark_lock);
@@ -156,6 +161,9 @@ void fsnotify_clear_marks_by_inode(struct inode *inode, unsigned int flags)struct fsnotify_mark_entry *lentry, *entry;LIST_HEAD (free_list); +if (!list_empty (&inode->i_fsnotify_mark_entries)) +printk (KERN_CRIT "%s: inode=%p flags=%d\n" , __func__, inode, flags); +spin_lock (&inode->i_lock);list_for_each_entry_safe (entry, lentry, &inode->i_fsnotify_mark_entries, i_list) {list_del_init (&entry->i_list);
@@ -241,6 +249,8 @@ struct fsnotify_mark_entry *fsnotify_mark_add(struct fsnotify_group *group, stru out_unlock:spin_unlock (&inode->i_lock);spin_unlock (&group->mark_lock); + +printk (KERN_CRIT "%s: group=%p inode=%p entry=%p mask=%llx\n" , __func__, group, inode, entry, mask);return entry; }
fs/notify/inotify/inotify_fsnotify.c +7 -0
@@ -51,6 +51,8 @@ static int inotify_event_to_notif(struct fsnotify_group *group, struct fsnotify_struct inotify_mark_private_data *mark_priv;int wd, ret =0 ; +printk (KERN_CRIT "%s: group=%p event=%p\n" , __func__, group, event); + to_tell = event->to_tell;spin_lock (&to_tell->i_lock);
@@ -100,7 +102,10 @@ static void inotify_mark_clear_inode(struct fsnotify_mark_entry *entry, struct ilist_add (&entry->i_list, &inode->i_fsnotify_mark_entries);spin_unlock (&inode->i_lock); +printk (KERN_CRIT "%s: entry=%p inode=%p flags=%d\n" , __func__, entry, inode, flags); +fsnotify (inode,FS_IN_IGNORED , inode,FSNOTIFY_EVENT_INODE ,NULL ,0 ); +inotify_destroy_mark_entry (entry); }
@@ -162,6 +167,8 @@ static void inotify_free_mark_priv(struct fsnotify_mark_entry *entry)struct inotify_mark_private_data *mark_priv = entry->private;struct inode *inode = mark_priv->inode; +printk (KERN_CRIT "%s: entry=%p\n" , __func__, entry); +BUG_ON (!entry->private); mark_priv = entry->private;
fs/notify/inotify/inotify_kernel.c +6 -0
@@ -56,6 +56,10 @@ int find_inode(const char __user *dirname, struct path *path, unsigned flags) {int error; +char *tmp =getname (dirname); +printk (KERN_CRIT "%s: pathname=%s\n" , __func__, tmp); +putname (tmp); + error =user_path_at (AT_FDCWD , dirname, flags, path);if (error)return error;
@@ -74,6 +78,8 @@ void inotify_destroy_mark_entry(struct fsnotify_mark_entry *entry)struct idr *idr;int wd; +printk (KERN_CRIT "%s: entry=%p refct=%d\n" , __func__, entry,atomic_read (&entry->refcnt)); +spin_lock (&entry->lock); mark_priv = entry->private;
fs/notify/inotify/inotify_user.c +3 -0
@@ -202,6 +202,9 @@ static ssize_t inotify_read(struct file *file, char __user *buf,__inotify_free_event_priv (priv);spin_unlock (&event->lock); +printk (KERN_CRIT "%s: event->wd=%d event->mask=%x event->cookie=%d event->len=%d event->file_name=%s\n" , + __func__, inotify_event.wd, inotify_event.mask, inotify_event.cookie, inotify_event.len, event->file_name); +if (copy_to_user (buf, &inotify_event, event_size)) { ret = -EFAULT ;break ;
fs/notify/notification.c +2 -0
@@ -263,6 +263,8 @@ struct fsnotify_event *fsnotify_create_event(struct inode *to_tell, __u64 mask, event->mask = mask; +printk (KERN_CRIT "%s: event=%p event->to_tell=%p event->data=%p event->flag=%d event->file=%s\n" , __func__, event, event->to_tell, event->inode, event->flag, event->file_name); +return event; }
include/linux/fsnotify.h +1 -0
@@ -96,6 +96,7 @@ static inline void fsnotify_parent(struct dentry *dentry, __u64 orig_mask) p_inode = parent->d_inode;if (p_inode && (p_inode->i_fsnotify_mask & mask)) { +printk (KERN_CRIT "%s: dentry=%p orig_mask=%llx mask=%llx parent=%p parent->inode=%p p_inode->i_fsnotify_mask=%llx\n" , __func__, dentry, orig_mask, mask, parent, p_inode, p_inode->i_fsnotify_mask);dget (parent);spin_unlock (&dentry->d_lock);fsnotify (p_inode, mask, dentry->d_inode,FSNOTIFY_EVENT_INODE , dentry->d_name.name,0 );
Re: [RFC PATCH -v4 14/14] shit on top for debugging
Quoted 1 line · Eric Paris
Given that this code will probably need some general debugging, perhaps use pr_debug.
Signature · 2 lines
James Morris <[email protected]>
Eric Paris
maintainerRe: [RFC PATCH -v4 14/14] shit on top for debugging
Quoted 6 lines · James Morris
Quoted 1 line · Eric Paris
Given that this code will probably need some general debugging, perhaps use
pr_debug.
an accidental patch left on top of my stack which I didn't mean to send. But sure, that's a great idea. I'll fix the comments and do it that way....
-Eroc
Quoted 10 lines · Eric Paris
The following series implements a new generic in kernel filesystem notification system, fsnotify. On top of fsnotify I reimplement dnotify and inotify. I have not finished with the change from inotify although I think
inotify_usershould be completed. In kernel inotify users (aka audit) still (until I get positive feedback) relay on the old inotify backend. This can be 'easily' fixed.
All of this is in preperation for fanotify and using fanotify as an on access
file scanner. So you better know it's coming.
Why is this useful? Because I actually shrank the struct inode. That's right, my code is smaller and faster. Eat that.
As a desktop-search-and-indexing developer, it doesn't seem like fanotify is going to give me anything I want. The inotify/dnotify restructuring to fsnotify seems reasonable, but not exciting (to me).
Quoted 1 line
From a desktop search perspective, my wishlist reads like this:
1) An 'autoadd' option added to inotify directory watches, so that
newly-created subdirectories get atomically added to the watch. That would prevent missed IN_MOVED_FROM and IN_MOVED_TO in a newly created directory.
2) A reasonable interface to map inode #s to "the current path to
this inode" -- or failing that, an iopen or ilink syscall. This would allow the search index to maintain inode #s instead of path names for files, which saves a lot of IN_MOVE processing and races (especially for directory moves, which require recursive path renaming).
3) Dream case: in-kernel dirty bits. I don't *really* want to know
all the details; I just want to know which inotify watches are dirty, so I can rescan them. To avoid races, the query should return a dirty watch and atomically clear its dirty flag, so that if it is updated again during my indexing, I'd be told to scan it again.
Quoted 1 line
From the indexing perspective, dealing with a sequence of operations like:
1) mkdir -p abc/def
2) echo "foo" > abc/def/ghi
3) mv abc xyz
4) mv xyz/def/ghi jkl
is entirely too much "fun". Depending on the races between kernel and user space, I might get only:
CREATE abc
IN_MOVED_TO jkl
and if I do get my watches put in place after step (1) I've still got to deal with a recursive rename in step (3), the possibility of getting notification of (2) after the rename in (3) (so abc/def/ghi no longer exists), and other delights.
As far as I can tell, fanotify only helps with the 'recursive watch' problem (which could be solved with 'autoadd' or just using the
algorithm in http://mail.gnome.org/archives/dashboard-hackers/2004-October/msg00022.html
), and doesn't give me any tools to deal with the actual hard races or path-maintenance problems.
--scott
Signature · 1 line
( http://cscott.net/ )
Eric Paris
maintainerQuoted 46 lines · C. Scott Ananian
Quoted 10 lines · Eric Paris
The following series implements a new generic in kernel filesystem notification system, fsnotify. On top of fsnotify I reimplement dnotify and inotify. I have not finished with the change from inotify although I think
inotify_usershould be completed. In kernel inotify users (aka audit) still (until I get positive feedback) relay on the old inotify backend. This can be 'easily' fixed.
All of this is in preperation for fanotify and using fanotify as an on access
file scanner. So you better know it's coming.
Why is this useful? Because I actually shrank the struct inode. That's right, my code is smaller and faster. Eat that.As a desktop-search-and-indexing developer, it doesn't seem like fanotify is going to give me anything I want. The inotify/dnotify restructuring to fsnotify seems reasonable, but not exciting (to me).
From a desktop search perspective, my wishlist reads like this:
1) An 'autoadd' option added to inotify directory watches, so that
newly-created subdirectories get atomically added to the watch. That would prevent missedIN_MOVED_FROMandIN_MOVED_TOin a newly created directory.
2) A reasonable interface to map inode #s to "the current path to
this inode" -- or failing that, an iopen or ilink syscall. This would allow the search index to maintain inode #s instead of path names for files, which saves a lot ofIN_MOVEprocessing and races (especially for directory moves, which require recursive path renaming).
3) Dream case: in-kernel dirty bits. I don't *really* want to know
all the details; I just want to know which inotify watches are dirty, so I can rescan them. To avoid races, the query should return a dirty watch and atomically clear its dirty flag, so that if it is updated again during my indexing, I'd be told to scan it again.From the indexing perspective, dealing with a sequence of operations like:
1) mkdir -p abc/def
2) echo "foo" > abc/def/ghi
3) mv abc xyz
4) mv xyz/def/ghi jkl
is entirely too much "fun". Depending on the races between kernel and user space, I might get only:
CREATE abc
IN_MOVED_TOjkl
and if I do get my watches put in place after step (1) I've still got to deal with a recursive rename in step (3), the possibility of getting notification of (2) after the rename in (3) (so abc/def/ghi no longer exists), and other delights.
Quoted 5 lines
As far as I can tell, fanotify only helps with the 'recursive watch' problem (which could be solved with 'autoadd' or just using the
algorithm in http://mail.gnome.org/archives/dashboard-hackers/2004-October/msg00022.html
), and doesn't give me any tools to deal with the actual hard races or path-maintenance problems.
You are absolutely correct that fanotify doesn't help with object movement or path maintenance. Neither had been requested, but notification (that an inode moved) shouldn't be impossible (although the hooks are going to be a lot more involved and will probably take some fighting with the VFS people, my current fanotify hooks use what is already being handed to fsnotify_* today) To directly answer you requests
1) autoadd isn't really what I'm looking at, but maybe someday I could take a peek, at first glance it doesn't seem unreasonable an idea, but I don't see how the userspace interface could work. Without the call the inotify_init to get the watch descriptor how can userspace know what these new events are? Only possibility I see for this is if inotify got an extensible userspace interface. In any case I'd be hard pressed to call it a high priority since it's already possible to get this and the intention of the addition is to make userspace code easier.
2) major vfs and every FS redesign me thinks.
3) What you want is IN_MODIFY from every inode but you want them to all coallese until you grab one instead of only merging the same event type if they are the last 2 in the notification queue. Not sure of a particularly clean/fast way to implement that right offhand, we'd have to run the entire notification queue every time before we add an event to the end, but at least this is doable with the constraints of the inotify user interface.
Can't this already be done in userspace just by draining the queue, matching events, throwing duplicates away, and then processing whatever is left? You know there is atomicity since removal of an event and the addition of an event both hold the inotify_dev->ev_mutex.
In any case, I'm going to let your thoughts rattle around in my brain while I'm still trying to rewrite inotify and dnotify to a better base. My first inclination is to stop using inotify and start using fanotify. Abandon filenames and start using device/inode pairs and you get
everything you need. But I'm certain that isn't that case :)
-Eric
Quoted 10 lines · Eric Paris
Quoted 9 lines · C. Scott Ananian
Quoted 8 lines · Eric Paris
The following series implements a new generic in kernel filesystem notification system, fsnotify. On top of fsnotify I reimplement dnotify and inotify. I have not finished with the change from inotify although I think
inotify_usershould be completed. In kernel inotify users (aka audit) still (until I get positive feedback) relay on the old inotify backend. This can be 'easily' fixed.
All of this is in preperation for fanotify and using fanotify as an on access
file scanner. So you better know it's coming.
Eric,
I was also getting my hands dirty with inotify (and friends) but I see that you are already ahead. So, I thought I would rather list my immediate requirements and see how far we can go with that.
My application is interested in only filesystem events (read/write/delete) but:
1) wants notification of only those events which are generated by the calling process and it's children.
2) Subject to the constraint as mentioned above (1), it would be nice if I can just say that I need to be notified of events on all filesystem objects (perhaps within a namespace) rather than specifying each and every file/dir that I am interested in. This is somewhat similar to the "recursive watch" problem that others have mentioned.
The first part is fairly easy to achieve and I had already created a patch for that on top of inotify. (See patch below). I was still thinking about second part when I saw your patches. Can you tell me whether your work is going to achive this? Maybe I can modify my patch to work on top of what you are proposing ...
-Niraj
--------------------------------
Inotify: filter events only by the current process and it's children.
Adds a new flag (to be used with sys_inotify_init1) which restricts notifications of only those events which has been generated by the calling process and it's children.
This patch is on top of 2.6.28-rc5.
fs/inotify.c +18 -1
@@ -80,6 +80,8 @@ struct inotify_handle {struct list_head watches;/* list of watches */ atomic_t count;/* reference count */ u32 last_wd;/* the last wd allocated */ +struct task_struct *owner_task;/* owner task_struct */ +u32 flags;/* operations flags */ const struct inotify_operations *in_ops;/* inotify caller operations */ };
@@ -310,6 +312,18 @@ void inotify_inode_queue_event(struct inode *inode, u32 mask, u32 cookie,u32 watch_mask = watch->mask;if (watch_mask & mask) {struct inotify_handle *ih= watch->ih; +struct task_struct *curr = current; +int found =0 ; +if (ih->flags &IN_ONLY_CHILD ) { +for (curr = current; curr != &init_task; + curr = curr->parent) +if (ih->owner_task == curr) { + found =1 ; +break ; + } +if (!found) +continue ; + }mutex_lock (&ih->mutex);if (watch_mask &IN_ONESHOT )remove_watch_no_event (watch, ih);
@@ -467,7 +481,8 @@ EXPORT_SYMBOL_GPL(inotify_inode_is_dead);* inotify_init - allocate and initialize an inotify instance * @ops: caller's inotify operations */ -struct inotify_handle *inotify_init (const struct inotify_operations *ops) +struct inotify_handle *inotify_init (const struct inotify_operations *ops, +int flags) {struct inotify_handle *ih;
@@ -479,6 +494,8 @@ struct inotify_handle *inotify_init(const struct inotify_operations *ops)INIT_LIST_HEAD (&ih->watches);mutex_init (&ih->mutex); ih->last_wd =0 ; + ih->owner_task = current; + ih->flags = flags; ih->in_ops = ops;atomic_set (&ih->count,0 );get_inotify_handle (ih);
fs/inotify_user.c +2 -2
@@ -588,7 +588,7 @@ asmlinkage long sys_inotify_init1(int flags)BUILD_BUG_ON (IN_CLOEXEC !=O_CLOEXEC );BUILD_BUG_ON (IN_NONBLOCK !=O_NONBLOCK ); -if (flags & ~(IN_CLOEXEC |IN_NONBLOCK )) +if (flags & ~(IN_CLOEXEC |IN_NONBLOCK |IN_ONLY_CHILD ))return -EINVAL ; fd =get_unused_fd_flags (flags &O_CLOEXEC );
@@ -614,7 +614,7 @@ asmlinkage long sys_inotify_init1(int flags) goto out_free_uid; } - ih =inotify_init (&inotify_user_ops); + ih =inotify_init (&inotify_user_ops, flags);if (IS_ERR (ih)) { ret =PTR_ERR (ih); goto out_free_dev;
include/linux/inotify.h +6 -3
@@ -66,8 +66,10 @@ struct inotify_event { IN_MOVE_SELF)/* Flags for sys_inotify_init1. */ -#define IN_CLOEXEC O_CLOEXEC -#define IN_NONBLOCK O_NONBLOCK +#define IN_CLOEXEC O_CLOEXEC/* 02000000 */ +#define IN_NONBLOCK O_NONBLOCK/* 00004000 */ +#define IN_ONLY_CHILD 0x00000002/* only notify event generated by this +process and children. */ #ifdef __KERNEL__
@@ -117,7 +119,8 @@ extern u32 inotify_get_cookie(void);/* Kernel Consumer API */ -extern struct inotify_handle *inotify_init (const struct inotify_operations *); +extern struct inotify_handle *inotify_init (const struct inotify_operations *, +int );extern void inotify_init_watch (struct inotify_watch *);extern void inotify_destroy (struct inotify_handle *); extern __s32 inotify_find_watch(struct inotify_handle *, struct inode *,
kernel/audit.c +1 -1
@@ -983,7 +983,7 @@ static int __init audit_init(void)audit_log (NULL ,GFP_KERNEL ,AUDIT_KERNEL ,"initialized" );#ifdef CONFIG_AUDITSYSCALL - audit_ih =inotify_init (&audit_inotify_ops); + audit_ih =inotify_init (&audit_inotify_ops,0 );if (IS_ERR (audit_ih))audit_panic ("cannot initialize inotify handle" );#endif
kernel/audit_tree.c +1 -1
@@ -907,7 +907,7 @@ static int __init audit_tree_init(void) {int i; - rtree_ih =inotify_init (&rtree_inotify_ops); + rtree_ih =inotify_init (&rtree_inotify_ops,0 );if (IS_ERR (rtree_ih))audit_panic ("cannot initialize inotify handle for rectree watches" );
Quoted 15 lines · Eric Paris
Quoted 2 lines · C. Scott Ananian
As a desktop-search-and-indexing developer, it doesn't seem like fanotify is going to give me anything I want. [...]
You are absolutely correct that fanotify doesn't help with object movement or path maintenance. Neither had been requested, but notification (that an inode moved) shouldn't be impossible (although the hooks are going to be a lot more involved and will probably take some fighting with the VFS people, my current fanotify hooks use what is already being handed to fsnotify_* today) To directly answer you requests
Quoted 4 lines
1) An 'autoadd' option added to inotify directory watches, so that
newly-created subdirectories get atomically added to the watch. That would prevent missedIN_MOVED_FROMandIN_MOVED_TOin a newly created directory.
Quoted 8 lines
1) autoadd isn't really what I'm looking at, but maybe someday I could take a peek, at first glance it doesn't seem unreasonable an idea, but I don't see how the userspace interface could work. Without the call the
inotify_initto get the watch descriptor how can userspace know what these new events are? Only possibility I see for this is if inotify got an extensible userspace interface. In any case I'd be hard pressed to call it a high priority since it's already possible to get this and the intention of the addition is to make userspace code easier.
Multiple calls to inotify_add_watch are allowed to return the same watch descriptor, since the descriptor is unique to a pathname. I think you would pass IN_DIR_AUTO_ADD as part of the 'mask' when you set up the watch, and when a subdirectory is added you generate the IN_CREATE as before but also atomically create a new watch on that directory, adding it to the same inotify instance. Since inotify maintains an ordered queue, the userland will eventually get the IN_CREATE, call inotify_add_watch on the subdirectory as before, and get the automatically created watch descriptor. Later events in this directory which are already on this queue use this same descriptor, so it "just works".
If you don't properly process the IN_CREATE or don't process queue events in order, you could get events referring to a watch descriptor you don't know about yet, but you can just defer them until you process the IN_CREATE and discover the descriptor id. These problems would be of your own making, of course, but are solvable. If you do the obvious thing, you don't have to worry.
This is a narrow fix which removes a race condition and enables straightforward code to "just work".
Quoted 6 lines
Quoted 5 lines
2) A reasonable interface to map inode #s to "the current path to
this inode" -- or failing that, an iopen or ilink syscall. This would allow the search index to maintain inode #s instead of path names for files, which saves a lot ofIN_MOVEprocessing and races (especially for directory moves, which require recursive path renaming).2) major vfs and every FS redesign me thinks.
I'm not convinced of that. I'm pretty certain one could export symlinks /proc/<pid>/mountinfo/<dev>/<inode> -> <absolute path in processes' namespace> with very little trouble, and no violence done to the VFS, which already has an iget() function which does the heavy lifting.
Would it be efficient? Well, more efficient (and reliable!) than try to maintain this same information in userland. We only need to use this when some action is actually taken by the desktop search (like launching an application with some document resulting from a search), which is less frequent than indexing operations.
Quoted 5 lines
Quoted 5 lines
3) Dream case: in-kernel dirty bits. I don't *really* want to know
all the details; I just want to know which inotify watches are dirty, so I can rescan them. To avoid races, the query should return a dirty watch and atomically clear its dirty flag, so that if it is updated again during my indexing, I'd be told to scan it again.
Quoted 7 lines
3) What you want is
IN_MODIFYfrom every inode but you want them to all coallese until you grab one instead of only merging the same event type if they are the last 2 in the notification queue. Not sure of a particularly clean/fast way to implement that right offhand, we'd have to run the entire notification queue every time before we add an event to the end, but at least this is doable with the constraints of the inotify user interface.
Yes, this sounds about right. There are details to be hashed out: If a/foo is modified and then moved to a/bar, do I get a combined event, reordered events (IN_MOVE a/foo -> a/bar, then IN_MODIFY a/bar), or dirty bits (IN_MOVE a/foo -> a/bar with an IN_DIRTY flag set on the event). But there are still atomicity concerns (next paragraph):
Quoted 4 lines
Can't this already be done in userspace just by draining the queue, matching events, throwing duplicates away, and then processing whatever is left? You know there is atomicity since removal of an event and the addition of an event both hold the
inotify_dev->ev_mutex.
No, you break atomicity between draining the event and doing the processing. I can drain the queue but then have a/foo moved to a/bar before I try to index a/foo. Then indexing fails, and I have to maintain some complicated user-space data structure to say, "oh, when you find out where a/bar went to, you should index it".
Proper dirty bits would have an atomic "fetch and clear" operation. So a/foo would be dirty, it would be returned on the queue and the dirty bit would be atomically cleared. If it was then moved to a/bar before I got around to indexing, the index operation would fail, but I'd know that a/bar would have its dirty bit set implicitly by the move, and so would show up on the queue again.
More tricky details: The dirty bit would probably actually be set on the directory 'a', and then when I scanned it I'd discover the 'foo->bar' rename. But I'd still have to "remember" (in userspace) that I didn't successfully scan a/foo and so scan a/bar. This dirty list could be a short list of "still dirty" inode numbers, and it would only be used in this particular move-after-dirty-read-and-before-index race. If a/foo was modified and then moved to a/bar, I'd simply see that both directory 'a' was dirty and file 'bar' was dirty, and wouldn't need to use the "failed index" list.
Quoted 5 lines
In any case, I'm going to let your thoughts rattle around in my brain while I'm still trying to rewrite inotify and dnotify to a better base. My first inclination is to stop using inotify and start using fanotify. Abandon filenames and start using device/inode pairs and you get
everything you need. But I'm certain that isn't that case :)
Well, except for being able to recreate the path from the inode, without which ability inode numbers without directory notifications are pretty useless.
BTW, I had some difficulty discovering the exact userland API you were proposing for fanotify. I eventually found in it the 'v1' and 'v2' set of fanotify patches, before the split to fsnotify, but it would be nice to see it restated in an easier-to-find place. 'google fanotify' turns up:
http://lwn.net/Articles/303277/
as the second hit, which is reasonable, but
http://people.redhat.com/~eparis/fanotify/21-fanotify-documentation.patch
seems better? I note that fanotify doesn't actually seem to return the relevant inode number from (say) a CLOSE_WAS_WRITABLE event; I've got to stat /proc/self/fd/<fd> to get that?
--scott
Signature · 1 line
( http://cscott.net/ )
Eric Paris
maintainerQuoted 11 lines · C. Scott Ananian
Quoted 5 lines · Eric Paris
In any case, I'm going to let your thoughts rattle around in my brain while I'm still trying to rewrite inotify and dnotify to a better base. My first inclination is to stop using inotify and start using fanotify. Abandon filenames and start using device/inode pairs and you get
everything you need. But I'm certain that isn't that case :)Well, except for being able to recreate the path from the inode, without which ability inode numbers without directory notifications are pretty useless.
fanotify doesn't have directory notifications. You get notifications about the individual inodes. I don't have mv tracking, and I'm not sure how much trouble it'll be. Maybe not that bad since a mv doesn't have to deliver the fd to userspace. I promise to think about how to make this better for you. In any case, you can get pathnames, dev, and inode very quickly.
Quoted 8 lines
BTW, I had some difficulty discovering the exact userland API you were proposing for fanotify. I eventually found in it the 'v1' and 'v2' set of fanotify patches, before the split to fsnotify, but it would be nice to see it restated in an easier-to-find place. 'google fanotify' turns up:
http://lwn.net/Articles/303277/as the second hit, which is reasonable, but
http://people.redhat.com/~eparis/fanotify/21-fanotify-documentation.patch
This is probably the best there is ATM. It includes a program which uses all of the fanotify functionality I wrote.
Quoted 3 lines
seems better? I note that fanotify doesn't actually seem to return the relevant inode number from (say) a
CLOSE_WAS_WRITABLEevent; I've got to stat /proc/self/fd/<fd> to get that?
For pathname you readlink on /proc/self/fd/[event->fd]
For dev and ino you stat [event->fd]
where event was filled from your getsockopt call.
In any case, event coallessing seems like it needs to work by walking the notification queue starting at the back and working forwards. If you find a duplicate just drop. If you find a mv, place this one at the end. Kinda sucks that we are taking and O(1) operation and making it O(n).
At least with fanotify you don't have mv races, since you have an open fd which still gives you the access you need even if it mv'd.
don't worry I won't forget your thoughts.
-Eric
Quoted 6 lines · Eric Paris
fanotify doesn't have directory notifications. You get notifications about the individual inodes. I don't have mv tracking, and I'm not sure how much trouble it'll be. Maybe not that bad since a mv doesn't have to deliver the fd to userspace. I promise to think about how to make this better for you. In any case, you can get pathnames, dev, and inode very quickly.
For consistency, directory events could appear as mutations (write/close_was_writable) of the directory inodes, with an open file descriptor to the directory.
It would be a bit annoying, but you wouldn't even have to pass the 'from' and 'to' names. You can then readdir the open file descriptor to the directory find out what's changed for yourself. (If you do pass 'from' and 'to' path information, then the open file descriptor to the directory is unnecessary.)
Capturing a coherent directory listing on startup (or while processing a directory mutation) is the other tricky bit; being able to block-for-ack on a rename would make that much easier, returning SEND_DELAY for example. The basic ideas seem coherent with what you've got in fanotify. A write_need_access_decision on directories would allow userspace locking during the scan, and you'd set a fastpath (with SURVIVE_WRITE cleared) after the scan was complete.
The write_need_access_decision method would, as testiment to the generality of the mechanism, allow userspace implementations of copy-on-write and versioning file systems. A versioning filesystem could check in the 'new version' of a file on close_was_writable; write_need_access would just need to block until userland had completed the checkin to prevent races between kernel and userland. For a copy-on-write system based on hardlinks, write_need_access_decision would break a hardlink if necessary, and regardless install a fast path skipping future checks.
Quoted 8 lines
In any case, event coallessing seems like it needs to work by walking the notification queue starting at the back and working forwards. If you find a duplicate just drop. If you find a mv, place this one at the end. Kinda sucks that we are taking and O(1) operation and making it O(n).
At least with fanotify you don't have mv races, since you have an open fd which still gives you the access you need even if it mv'd.
On reflection event coalescing is not necessary so long as you're passing open file descriptors around. The kernel mechanisms are sufficient to ensure that the file descriptors are still valid even if unlink/move operations occur after the mutation.
To briefly summarize my fanotify wishlist:
1) write/close_was_writable events on directory inodes triggered by
create/rename/unlink.
2) write_need_access_decision event applicable to both regular files
and directories.
--scott
Signature · 1 line
( http://cscott.net/ )
Al Viro
maintainerQuoted 7 lines · C. Scott Ananian
Quoted 1 line
2) major vfs and every FS redesign me thinks.
I'm not convinced of that. I'm pretty certain one could export symlinks /proc/<pid>/mountinfo/<dev>/<inode> -> <absolute path in processes' namespace> with very little trouble, and no violence done to the VFS, which already has an
iget()function which does the heavy lifting.
There is no such thing as absolute path of inode *anywhere*, process namespace or not. Not on any UNIX. Period. End of story.
Al, once again astonished by the ability of desktop developers to post from the alternative realities they apparently inhabit...
Quoted 11 lines · Al Viro
Quoted 7 lines · C. Scott Ananian
Quoted 1 line
2) major vfs and every FS redesign me thinks.
I'm not convinced of that. I'm pretty certain one could export symlinks /proc/<pid>/mountinfo/<dev>/<inode> -> <absolute path in processes' namespace> with very little trouble, and no violence done to the VFS, which already has an
iget()function which does the heavy lifting.There is no such thing as absolute path of inode *anywhere*, process namespace or not. Not on any UNIX. Period. End of story.
That's not correct, as /proc/self/fd/<num> and the getcwd syscall make clear. struct inode has a i_dentry member, and via its d_parent links you can reconstruct the path, as __d_path in fs/dcache.c does.
Of course, this only works as long as the inode is in the kernel's cache (which implies all the appropriate dentry items are there, too).
For an open file descriptor, this works well. If you index the open
file now and then return sometime later to try to open it, there's no guarantee.
If the inode represents a directory, we *could* use the magic '..' and '.' entries to reconstruct its path, even without its being in the cache. I haven't looked hard, but that might involve VFS violence. But having to manually track what directory inode a given file was in is still an improvement over having to duplicate the entire directory tree.
We can also debate whether finding *a* name for the inode if sufficient, or if you really need to know *all* names for the inode. One is plenty if you're just looking for a path to give on the command line to (say) emacs to open the file after a successful search. But then emacs may not be happy when it tries to write a backup file to an arbitrarily chosen directory.
Quoted 1 line
Al, once again astonished by the ability of desktop developers to post from the alternative realities they apparently inhabit...
I think you're prejudging. I'm certainly posting from a weird twisted alternate reality, but by and large it's not the "desktop developer" one. ;-)
There are a paucity of good solutions. We can follow the example of old-school UNIX dump and open the filesystem's block device directly to map inodes to paths. Or we can use 'find' over the entire filesystem. Or we can duplicate the filesystem's directory structure in userspace so that we can correctly associate index information with the current path to the given file?
inotify *almost* lets us do that last thing (though the code duplication pains me) but is too racy for reliable use. Give me a kernel interface without races and I'll call it a good start. If you can save me the trouble of duplicating all of the filesystem's directory information in my userspace database in order to handle directory moves, I'll actually grin a little.
--scott
Signature · 1 line
( http://cscott.net/ )
Al Viro
maintainerQuoted 3 lines · C. Scott Ananian
That's not correct, as /proc/self/fd/<num> and the getcwd syscall make clear. struct inode has a
i_dentrymember, and via itsd_parentlinks you can reconstruct the path, as__d_pathinfs/dcache.cdoes.
Think for a minute
a) you might have any number of links to given inode, *including* *zero*
b) any subset of these links may be in dcache (including empty)
c) any number of bindings might exist to file in question *or* to its ancestors (again, including zero).
d) none of the relative paths from fs root to file has to be visible as a path leading to file in question from anywhere (again, think of bindings)
e) all of that can change before information reaches userland.
Quoted 6 lines
inotify *almost* lets us do that last thing (though the code duplication pains me) but is too racy for reliable use. Give me a kernel interface without races and I'll call it a good start. If you can save me the trouble of duplicating all of the filesystem's directory information in my userspace database in order to handle directory moves, I'll actually grin a little.
_WHAT_ interface without races? Anything along the lines of "somebody had done something to <pathname>" is racy by definition. Simply because the next operation might have changed the mapping from pathnames to files.
What are you actually trying to do?
Quoted 2 lines · C. Scott Ananian
That's not correct, as /proc/self/fd/<num> and the getcwd syscall make clear.
No. These take a file descriptor into account, which _does_ have a unique path.
Quoted 2 lines
struct inode has a
i_dentrymember, and via itsd_parentlinks you can reconstruct the path, as__d_pathinfs/dcache.cdoes.
reconstruct _a_ path inside the same filesystem, ignoring which link is wanted, and inside which mount.
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · C. Scott Ananian
That's not correct, as /proc/self/fd/<num> and the getcwd syscall make clear.
No. These take a file descriptor into account, which _does_ have a unique path.
getcwd doesn't actually hold a file descriptor to the working directory. If you reread my message, you'll find that I was explicit about where the information was stored.
Quoted 5 lines
Quoted 2 lines
struct inode has a
i_dentrymember, and via itsd_parentlinks you can reconstruct the path, as__d_pathinfs/dcache.cdoes.reconstruct _a_ path inside the same filesystem, ignoring which link is wanted, and inside which mount.
Right. If you go back and reread my message, I made clear that that was sufficient.
But Al's question is more relevant, and I'll try to restate the problem for him, since it's clear that none of the existing *notify interfaces was written with an eye towards making possible races easily manageable.
--scott
Signature · 1 line
( http://cscott.net/ )
Al Viro
maintainerQuoted 3 lines · C. Scott Ananian
getcwd doesn't actually hold a file descriptor to the working directory. If you reread my message, you'll find that I was explicit about where the information was stored.
Indeed - explicit, persistent and wrong. For current directory of a process we store vfsmount and dentry. And use those in getcwd() rather than playing hopeless games with inodes.
Quoted 8 lines · Al Viro
Quoted 3 lines · C. Scott Ananian
getcwd doesn't actually hold a file descriptor to the working directory. If you reread my message, you'll find that I was explicit about where the information was stored.
Indeed - explicit, persistent and wrong. For current directory of a process we store vfsmount and dentry. And use those in
getcwd()rather than playing hopeless games with inodes.
Geez. Please don't treat me as if I can't read source code.
I suggested a Mach-like iopen mechanism to address some inotify races.
In order to show that extreme VFS violence might not be necessary, I
pointed out that *in some cases* you can derive *some paths* to the file from the inode number, using the iget()->i_dentry list. But you've driven me far off-topic.
Let's get back to the problem at hand. The most obvious problem with
inotify is the race between mkdir/IN_CREATE and the userspace process adding the watch on the new directory. I proposed an 'autoadd' mechanism earlier in this thread to address this (stolen from the racy userspace version of this in python-inotify); the "Love-Trowbridge algorithm" from:
http://mail.gnome.org/archives/dashboard-hackers/2004-October/msg00022.html
is also targetted at this race.
But this isn't the only problem. The inotify interface on directories returns (in effect) a <directory inode>,<filename> pair (the directory watch is per inode; the event includes a filename). This means that:
echo foo >a/b; echo bar >a/c; mv a/c a/b
has an inherent race. Our index service drains the inotify queue and attempts to open and index a/b. After the indexing, we check the queue and discover IN_MOVED_FROM c and IN_MOVED_TO b. There is no way for the userspace process to know whether it managed to index the file before or after the move. (We're forced to track renames to detect this situation and then attempt to reindex a/b, and of course we can have another race; we must repeat until we finally succeed.) If inotify provided an inode number or file descriptor instead of a path name, we'd be able to tell if we were indexing the thing we expected.
But this isn't the end. How about:
mkdir -p a/b a/c ; touch a/b/foo a/c/foo
<read inotify queue here>
mv a/b a/bb ; mv a/c a/b
When we index a/b/foo, we won't know whether this is the original a/b/foo or the original a/c/foo. In this case we can open 'a/b' and check that the inode number is what we expect before using openat to open 'foo' (but remember that the previous race means that we're still not sure 'foo' is what we expect it to be, so we still need to use that detection algorithm as well).
And remember that we're still expected to keep and update a map in userspace mapping from directory watch ids to path names, and presumably keep path name information updated in our search index as well. When a directory is moved, we need to recursively update path information for all files in the index -- unless we keep path information as <directory inode>;<filename> pairs, which avoids the recursive update at the expense of having to maintain a redundant copy of the filesystem's directory structure in userspace.
(These are the races I've found; it's possible there are others.)
As far as I can tell, none of the existing Linux desktop search tools attempt to deal with these races. (Beagle handles the 'mkdir' race, but not the other rename races.) This is acceptable only if an unreliable file index is acceptable.
Some possible improvements to the situtation (all bad, in various ways -- better suggestions wanted!):
a) do nothing. Most developers will ignore the races in inotify out
of ignorance or complexity, and most applications which use inotify will be unreliable as a result.
b) use inode numbers rather than path names uniformly, in both
inotify and the userland search index, along with an iopen() syscall, as in Mach. This decouples path maintenance from indexing. This was discussed in (for example)
http://www.coda.cs.cmu.edu/maillists/codalist/codalist-1998/0217.html
by Peter J. Braam and Ted Ts'o, but Al Viro has been objecting to the idea here. (If all you need to do is open found files after a search, you can skip path maintenance entirely.)
c) Pass file descriptors in the notification API from the kernel.
This solves the races associated with renames before indexing. Userland still has to maintain its own copy of all the direntries for indexed content, but at least this task is decoupled. (The proposed fanotify API passes file descriptors, but provides no mechanism (yet) for path maintenance.)
d) Do all indexing in the filesystem. BeOS used this option; in Linux-land, this would probably be a thin FUSE shim which layered over an existing filesystem. The shim could grab the appropriate locks to manage the races and ensure that the index's path information was consistent with the filesystem.
Returning to fanotify, I'll recant some of my earlier judgement:
fanotify already solves the 'mkdir' race in inotify (by virtue of not requiring separate watches on each directory) and the 'mv before index' race (by passing an open file descriptor to userland). If it provided some basic directory-change support so that path information can be maintained, it would be a clear win for desktop search, since by simply processing events in order we can produce a coherent index
state. The only remaining races would be during the initial scan.
If one wanted the simplest possible correct userspace, perhaps move and create can be deferred by userland using the fanotify 'approval' mechanism until the scan is complete.
--scott
Signature · 1 line
( http://cscott.net/ )
Al Viro
maintainerQuoted 11 lines · C. Scott Ananian
Quoted 3 lines
Indeed - explicit, persistent and wrong. For current directory of a process we store vfsmount and dentry. And use those in
getcwd()rather than playing hopeless games with inodes.Geez. Please don't treat me as if I can't read source code.
I suggested a Mach-like iopen mechanism to address some inotify races.
In order to show that extreme VFS violence might not be necessary, I
pointed out that *in some cases* you can derive *some paths* to the file from the inode number, using the iget()->i_dentry list. But you've driven me far off-topic.
BTW, XFS open-by-inode implementation is severely broken - try using it for directories and you'll get all kinds of hell breaking loose. And that's the only attempt at iopen in Linux.
While we are at it, iget() is not a generic interface - it's strictly up to specific filesystem whether that sucker will work at all. Or what kind of data will have to be required for it to work on given fs, assuming it will work to start with.
Quoted 2 lines
Let's get back to the problem at hand. The most obvious problem with
inotify is the race between mkdir/IN_CREATE and the userspace process
[snip]
Quoted 4 lines
As far as I can tell, none of the existing Linux desktop search tools attempt to deal with these races. (Beagle handles the 'mkdir' race, but not the other rename races.) This is acceptable only if an unreliable file index is acceptable.
... and inotify is unreliable by design, or what passes for it anyway.
Quoted 7 lines
Some possible improvements to the situtation (all bad, in various ways -- better suggestions wanted!):
a) do nothing. Most developers will ignore the races in inotify out
of ignorance or complexity, and most applications which use inotify will be unreliable as a result.
Quoted 8 lines
b) use inode numbers rather than path names uniformly, in both
inotify and the userland search index, along with aniopen()syscall, as in Mach. This decouples path maintenance from indexing. This was discussed in (for example)
http://www.coda.cs.cmu.edu/maillists/codalist/codalist-1998/0217.html
by Peter J. Braam and Ted Ts'o, but Al Viro has been objecting to the idea here. (If all you need to do is open found files after a search, you can skip path maintenance entirely.)
For one thing, opened directory can be fchdir'ed to. Or passed toopenat() et.al. For another, go ahead, show me how to implement that sucker for something like NFS. Or CIFS. Or FAT, while we are at it. Or anything that does have stable numbers identifying fs objects, but where those numbers are huge.
Quoted 7 lines
c) Pass file descriptors in the notification API from the kernel.
This solves the races associated with renames before indexing. Userland still has to maintain its own copy of all the direntries for indexed content, but at least this task is decoupled. (The proposed fanotify API passes file descriptors, but provides no mechanism (yet) for path maintenance.)
Quoted 5 lines
d) Do all indexing in the filesystem. BeOS used this option; in Linux-land, this would probably be a thin FUSE shim which layered over an existing filesystem. The shim could grab the appropriate locks to manage the races and ensure that the index's path information was consistent with the filesystem.
e) start with providing a higher-level description of what you want to achieve. While we are at it, is it supposed to be fs-agnostic? What kinds of filesystems could in principle be used with that?
So far you've mentioned use of blatantly inadequate tool and far too low-level description of changes that might possibly make it more tolerable for unspecified use you have in mind. I'm sorry, but that's exactly how a bunch of bad APIs (including inotify) got pushed into the tree and I would rather not repeat the experience.
Interfaces must make sense. And "we need <list of things> for our project, nevermind what are they for, here's what we would like to see" is a recipe for kludges. Inotify, dnotify, F_SETLEASE, etc. Sigh...
Quoted 6 lines · Al Viro
Quoted 4 lines
As far as I can tell, none of the existing Linux desktop search tools attempt to deal with these races. (Beagle handles the 'mkdir' race, but not the other rename races.) This is acceptable only if an unreliable file index is acceptable.
... and inotify is unreliable by design, or what passes for it anyway.
Really? I don't see that in the documentation anywhere (except as an aside in a comment you added to inotify's pin-to-kill method). Nor do I find it supported by code review. Sure, the queue is limited size, but you are guaranteed an overflow event when messages are dropped; that's all that's needed for reliability. Can you provide some specific example where events are lost "by design"? There are userland races in inotify, but they are all solvable, and I've described above how to do so. The problem is that the workarounds make practical programming with inotify very cumbersome, and the result is that most implementors haven't bothered to make things correct (ie, moving a directory breaks all search tools I've looked at). I'm interested in determining whether there are minor tweaks which can make correct userlands easier to write. That's why I've been beating the "autoadd" drum in this thread: it's a very small patch would would remove one category of race entirely (leaving only the rename races for userland to deal with).
Quoted 11 lines
Quoted 8 lines
b) use inode numbers rather than path names uniformly, in both
inotify and the userland search index, along with aniopen()syscall, as in Mach. This decouples path maintenance from indexing. This was discussed in (for example)
http://www.coda.cs.cmu.edu/maillists/codalist/codalist-1998/0217.html
by Peter J. Braam and Ted Ts'o, but Al Viro has been objecting to the idea here. (If all you need to do is open found files after a search, you can skip path maintenance entirely.)For one thing, opened directory can be fchdir'ed to. Or passed to
openat()et.al.
You seem to be confusing inotify watch descriptors with file descriptors. Directories are not held open in inotify, as you should know.
Adding an inotify_open_wd() syscall would indeed eliminate the last of the 3 races I described above (a/b/c rename to a/d/c). It wouldn't do anything about a/b -> a/c rename races.
Quoted 3 lines
For another, go ahead, show me how to implement that
sucker for something like NFS. Or CIFS. Or FAT, while we are at it.
Or anything that does have stable numbers identifying fs objects, but where those numbers are huge.
Sure. Because not all filesystems support xattr (say) we should refuse to add it to any? As you know well, The standard solution to the inode problem (in NFS or FUSE, say) is to synthesize stable inode numbers in the kernel (or in metadata in something like vfat), with the understanding that these numbers will be different on every mount.
For the purposes of race-avoidance in inotify, transient inode
numbers are acceptable. And we can always fall back to "old racy inotify" or manual filesystem scans for minority filesystems.
Quoted 6 lines
Quoted 6 lines
c) Pass file descriptors in the notification API from the kernel.
This solves the races associated with renames before indexing. Userland still has to maintain its own copy of all the direntries for indexed content, but at least this task is decoupled. (The proposed fanotify API passes file descriptors, but provides no mechanism (yet) for path maintenance.)
You didn't comment on this alternative, but as I've continued thinking about the problem, this alternative has seemed best.
Quoted 3 lines
e) start with providing a higher-level description of what you want to achieve. While we are at it, is it supposed to be fs-agnostic? What kinds of filesystems could in principle be used with that?
The high level problem should be obvious, but I'll restate it for you anyway:
* Maintain a auxiliary set of content-derived metadata for each file
in some subset (under a directory, mount-point, matching a wildcard, etc). This metadata must be updated whenever the content changes, and must include/maintain some stable reference to the file so that searches on the metadata can be mapped back to their corresponding files later.
Implementors vary on the definition of 'content' (do file permissions count? xattrs? etc), the best "update" API (is notify-on-close and reindexing entire files sufficient, or do we want to do incremental updates on each dirty block/byte), and whether the notification is synchronous or asynchronous (eg: Eric Paris wants to act on 'bad' content, so he needs a synchronous interface to prevent races from acting on bad content). In Linux search tools I've looked at, the 'stable reference' notion has usually been kept as a file:// URL (ie, absolute path), but Mac OS X has file handles closer to inode numbers (although I'm not positive that's what they are using).
Filesystem-agnostic tools are preferable, but "works best on filesystem X" search tools are already common. The BeFS was an example of non-agnostic search, and Beagle "works best" on filesystems that support xattrs. The i_version field will also be useful for search tools, for filesystems which support it (only ext4 so far?). So give me filesystem-specific search if it is compelling, or agnostic search where possible.
Quoted 5 lines
So far you've mentioned use of blatantly inadequate tool and far too low-level description of changes that might possibly make it more tolerable for unspecified use you have in mind. I'm sorry, but that's exactly how a bunch of bad APIs (including inotify) got pushed into the tree and I would rather not repeat the experience.
So, you don't understand the problem, and you'd rather not have inotify in tree because you think it is "blatantly inadequate" -- yet you are using inotify in your audit subsystem? I think I am beginning to get the impression that you're just not going to be much help.
Quoted 3 lines
Interfaces must make sense. And "we need <list of things> for our project, nevermind what are they for, here's what we would like to see" is a recipe for kludges. Inotify, dnotify,
F_SETLEASE, etc. Sigh...
I don't think this is a sound technical argument. We've got at least two different APIs on the table (fanotify and inotify), a well-understood problem description, *and* I've made at least 4 different proposals for how the current situation might be improved. Please feel free to suggest concrete alternatives, but I don't plan to continue this thread while you are just taking bogus potshots.
--scott
Signature · 1 line
( http://cscott.net/ )