fanotify: flush outstanding perm requests on group destroy
Subsystems (2)
PATCH 1/4 fanotify: flush outstanding perm requests on group destroy
When an fanotify listener is closing it may cause a deadlock between the listener and the original task doing an fs operation. If the original task is waiting for a permissions response it will be holding the srcu lock. The listener cannot clean up and exit until after that srcu lock is syncronized. Thus deadlock. The fix introduced here is to stop accepting new permissions events when a listener is shutting down and to grant permission for all outstanding events. Thus the original task will eventually release the srcu lock and the listener can complete shutdown.
--- fs/notify/fanotify/fanotify_user.c | 27 +++++++++++++++++++++++++++ include/linux/fanotify.h | 7 ------- include/linux/fsnotify_backend.h | 1 + 3 files changed, 28 insertions(+), 7 deletions(-)
fs/notify/fanotify/fanotify_user.c +27 -0
@@ -195,6 +195,14 @@ static int prepare_for_access_response(struct fsnotify_group *group, re->fd = fd;mutex_lock (&group->fanotify_data.access_mutex); + +if (group->fanotify_data.bypass_perm) { +mutex_unlock (&group->fanotify_data.access_mutex); +kmem_cache_free (fanotify_response_event_cache, re); + event->response =FAN_ALLOW ; +return 0 ; + } +list_add_tail (&re->list, &group->fanotify_data.access_list);mutex_unlock (&group->fanotify_data.access_mutex);
@@ -364,9 +372,28 @@ static ssize_t fanotify_write(struct file *file, const char __user *buf, size_tstatic int fanotify_release (struct inode *ignored,struct file *file) {struct fsnotify_group *group = file->private_data; +struct fanotify_response_event *re, *lre;pr_debug ("%s: file=%p group=%p\n" , __func__, file, group); +#ifdef CONFIG_FANOTIFY_ACCESS_PERMISSIONS +mutex_lock (&group->fanotify_data.access_mutex); + + group->fanotify_data.bypass_perm = true; + +list_for_each_entry_safe (re, lre, &group->fanotify_data.access_list, list) { +pr_debug ("%s: found group=%p re=%p event=%p\n" , __func__, group, + re, re->event); + +list_del_init (&re->list); + re->event->response =FAN_ALLOW ; + +kmem_cache_free (fanotify_response_event_cache, re); + } +mutex_unlock (&group->fanotify_data.access_mutex); + +wake_up (&group->fanotify_data.access_waitq); +#endif /* matches the fanotify_init->fsnotify_alloc_group */ fsnotify_put_group (group);
include/linux/fanotify.h +0 -7
@@ -95,11 +95,4 @@ struct fanotify_response { (long)(meta)->event_len >= (long)FAN_EVENT_METADATA_LEN && \ (long)(meta)->event_len <= (long)(len)) -#ifdef __KERNEL__ - -struct fanotify_wait { -struct fsnotify_event *event; -__s32 fd; -}; -#endif /* __KERNEL__ */ #endif /* _LINUX_FANOTIFY_H */
include/linux/fsnotify_backend.h +1 -0
@@ -156,6 +156,7 @@ struct fsnotify_group {struct mutex access_mutex;struct list_head access_list;wait_queue_head_t access_waitq; +bool bypass_perm;/* protected by access_mutex */ #endif /* CONFIG_FANOTIFY_ACCESS_PERMISSIONS */ int f_flags; } fanotify_data;
Signature · 1 line
1.6.5.3
PATCH 2/4 fanotify: drop duplicate pr_debug statement
From: Tvrtko Ursulin <[email protected]>
This reminded me... you have two pr_debugs in fanotify_should_send_event which output redundant information. Maybe you intended it like that so it is selectable how much log spam you want, or if not you may want to apply this patch.
--- fs/notify/fanotify/fanotify.c | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-)
fs/notify/fanotify/fanotify.c +0 -3
@@ -165,9 +165,6 @@ static bool fanotify_should_send_event(struct fsnotify_group *group,"mask=%x data=%p data_type=%d\n" , __func__, group, to_tell, inode_mark, vfsmnt_mark, event_mask, data, data_type); -pr_debug ("%s: group=%p vfsmount_mark=%p inode_mark=%p mask=%x\n" , - __func__, group, vfsmnt_mark, inode_mark, event_mask); -/* sorry, fanotify only gives a damn about files and dirs */ if (!S_ISREG (to_tell->i_mode) && !S_ISDIR (to_tell->i_mode))
Signature · 1 line
1.6.5.3
PATCH 3/4 fanotify: resize pid and reorder structure
From: Tvrtko Ursulin <[email protected]>
resize pid and reorder the fanotify_event_metadata so it is naturally aligned and we can work towards dropping the packed attributed
--- include/linux/fanotify.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
include/linux/fanotify.h +2 -2
@@ -70,9 +70,9 @@struct fanotify_event_metadata {__u32 event_len;__u32 vers; -__s32 fd;__u64 mask; -__s64 pid; +__s32 fd; +__s32 pid; } __attribute__ ((packed));struct fanotify_response {
Signature · 1 line
1.6.5.3
PATCH 4/4 fanotify: drops the packed attribute from userspace event metadata
The userspace event metadata structure was packed so when sent from a kernel with a certain set of alignment rules to a userspace listener with a different set of alignment rules the userspace process would be able to use the structure. On some arches just using packed, even if it doesn't do anything to the alignment can cause a severe performance hit. From now on we are not going to set the packed attribute and will just need to be very careful to make sure the structure is naturally aligned and that explicit padding is used when necessary. To make sure noone gets this wrong in the future, we enforce this fact, at build time, using a similar structure that is packed and comparing their sizes.
--- fs/notify/fanotify/fanotify_user.c | 2 ++ include/linux/fanotify.h | 22 ++++++++++++++++++++-- 2 files changed, 22 insertions(+), 2 deletions(-)
fs/notify/fanotify/fanotify_user.c +2 -0
@@ -778,6 +778,8 @@ SYSCALL_ALIAS(sys_fanotify_mark, SyS_fanotify_mark);*/ static int __init fanotify_user_setup (void ) { +BUILD_BUG_ON (sizeof (struct fanotify_event_metadata ) != +sizeof (struct fan_event_meta_packed )); fanotify_mark_cache =KMEM_CACHE (fsnotify_mark,SLAB_PANIC ); fanotify_response_event_cache =KMEM_CACHE (fanotify_response_event,SLAB_PANIC );
include/linux/fanotify.h +20 -2
@@ -66,14 +66,21 @@ FAN_Q_OVERFLOW)#define FANOTIFY_METADATA_VERSION 1 - +/* +* This structue must be naturally aligned so that a 32 bit userspace process +* will find the offsets the same as a 64bit process. If there would be padding +* in the structure it must be added explictly by hand. Please note that +* anything added to this structure must also be added to the fan_event_meta_packed +* struct, which is used to enforce the alignment and padding rules at build +* time. +*/ struct fanotify_event_metadata {__u32 event_len;__u32 vers;__u64 mask;__s32 fd;__s32 pid; -} __attribute__ ((packed)); +};struct fanotify_response {__s32 fd;
@@ -95,4 +102,15 @@ struct fanotify_response { (long)(meta)->event_len >= (long)FAN_EVENT_METADATA_LEN && \ (long)(meta)->event_len <= (long)(len)) +#ifdef __KERNEL__ +/* see struct fanotify_event_metadata for the reason this exists */ +struct fan_event_meta_packed { +__u32 event_len; +__u32 vers; +__u64 mask; +__s32 fd; +__s32 pid; +} __attribute__ ((packed)); + +#endif /* __KERNEL__ */ #endif /* _LINUX_FANOTIFY_H */
Signature · 1 line
1.6.5.3
Re: [PATCH 4/4] fanotify: drops the packed attribute from userspace event metadata
Quoted 1 line · On Monday 23 August 2010 02:37:10 Eric Paris wrote:
#define
FANOTIFY_METADATA_VERSION1
FANOTIFY_METADATA_VERSION should be incremented, too.
include/linux/fanotify.h line 80
Quoted 4 lines
struct
fanotify_event_metadata{include/linux/fanotify.h +0
-0__u32 event_len;__u32 vers;__u64 mask;
We don't actually care if there are any holes in this structure; all we care about is that the structure has the same alignment on 32-bit and 64-bit architectures.
Using type aligned_u64 here instead of __u64 will do the trick.
include/linux/fanotify.h struct fanotify_response { · line 113
Quoted 9 lines
include/linux/fanotify.h +9
-0+#ifdef __KERNEL__ +/* see struct fanotify_event_metadata for the reason this exists */ +struct fan_event_meta_packed { +__u32 event_len; +__u32 vers; +__u64 mask; +__s32 fd; +__s32 pid; +} __attribute__ ((packed));
This does not add much value; the two structures can still go out of sync A note to be careful about changes to struct fanotify_event_metadata should really be warning enough.
Andreas
Quoted 8 lines · On Monday 23 August 2010 02:37:07 Eric Paris wrote:
When an fanotify listener is closing it may cause a deadlock between the listener and the original task doing an fs operation. If the original task is waiting for a permissions response it will be holding the srcu lock. The listener cannot clean up and exit until after that srcu lock is syncronized. Thus deadlock. The fix introduced here is to stop accepting new permissions events when a listener is shutting down and to grant permission for all outstanding events. Thus the original task will eventually release the srcu lock and the listener can complete shutdown.
This seems to work now.
The one remaining issue is that processes blocked on perm events cannot be killed even with SIGKILL. I don't know how hard it will be to fix this.
Thanks,
Andreas
Quoted 11 lines · On Tuesday 24 Aug 2010 02:13:11 Andreas Gruenbacher wrote:
Quoted 9 lines · On Monday 23 August 2010 02:37:07 Eric Paris wrote:
When an fanotify listener is closing it may cause a deadlock between the listener and the original task doing an fs operation. If the original task is waiting for a permissions response it will be holding the srcu lock. The listener cannot clean up and exit until after that srcu lock is syncronized. Thus deadlock. The fix introduced here is to stop accepting new permissions events when a listener is shutting down and to grant permission for all outstanding events. Thus the original task will eventually release the srcu lock and the listener can complete shutdown.
Quoted 4 lines
This seems to work now.
The one remaining issue is that processes blocked on perm events cannot be killed even with SIGKILL. I don't know how hard it will be to fix this.
I think just switching to interruptible sleep in fanotify_get_response_from_access should be fine. And it should probably deny the current event when signal is received.
Tvrtko
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom. Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Quoted 3 lines · On Tuesday 24 August 2010 10:49:45 Tvrtko Ursulin wrote:
I think just switching to interruptible sleep in
fanotify_get_response_from_accessshould be fine. And it should probably deny the current event when signal is received.
Well the result would be -EINTR from the system call that blocked on the perm event, the same as with an interruptible nfs mount. The process would never get -EPERM. Processes may
not be prepared to handle -EINTR in all cases, and so it may make more sense to use the same behavior as NFS and only allow SIGKILL to kill a process blocked on a perm event (which the blocked process will never see).
Quoted 9 lines
From the nfs man page:
intr / nointrSelects whether to allow signals to interrupt file operations on this mount point. If neither option is specified (or if nointr is specified), signals do not interrupt NFS file operations. If intr is specified, system calls return EINTR if an in-progress NFS operation is interrupted by a signal.
Quoted 6 lines
Using the intr option is preferred to using the soft option because it is significantly less likely to result in data corruption. The intr / nointr
mount option is deprecated after kernel 2.6.25. Only SIGKILL can
interrupt a pending NFS operation on these kernels, and if specified, this mount option is ignored to provide backwards compatibility with older kernels.
Andreas
Quoted 5 lines · On Tuesday 24 Aug 2010 10:36:29 Andreas Gruenbacher wrote:
Quoted 3 lines · On Tuesday 24 August 2010 10:49:45 Tvrtko Ursulin wrote:
I think just switching to interruptible sleep in
fanotify_get_response_from_accessshould be fine. And it should probably deny the current event when signal is received.
Quoted 7 lines
Well the result would be
-EINTRfrom the system call that blocked on the perm event, the same as with an interruptible nfs mount. The process would never get-EPERM. Processes may
not be prepared to handle-EINTRin all cases, and so it may make more sense to use the same behavior as NFS and only allow SIGKILL to kill a process blocked on a perm event (which the blocked process will never see).
That would be wait_event_killable then, even simpler change.
I agree that hiding -EINTR from open, from userspace code is probably a more compatible way of doing it. If I remember correctly POSIX does allow EINTR from open but form our experience there are indeed applications which do not handle it. Some version of X immediately spring to mind who used to set up a periodic signal delivery (something like internal jiffy, I think they called it smart scheduler) to themselves and would not handle -EINTR from open. Samba also had a problem here in specific circumstances.
Tvrtko
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom. Company Reg No 2096520. VAT Reg No GB 348 3873 20.