fanotify: Fix list corruption in fanotify_get_response()
Subsystems (4)
Jan Kara
maintainercover fanotify: Fix list corruption in fanotify_get_response()
Hi,
these patches should fix the corruption in fanotify_get_response() which Miklos has spotted. Miklos, can you please test them? I have tried writing my own stress-tester to hit this issue but I was not able to trigger the bug...
Honza
Jan Kara
maintainerPATCH 1/3 fsnotify: Use enum for return values of fsnotify_add_event()
Currently there are three possible returns from fsnotify_add_event(). We will be adding a fourth one. Let's change magic numbers to enum to make things clearer.
--- fs/notify/fanotify/fanotify.c | 7 ++++--- fs/notify/inotify/inotify_fsnotify.c | 4 ++-- fs/notify/notification.c | 20 +++++++++++--------- include/linux/fsnotify_backend.h | 14 ++++++++++---- 4 files changed, 27 insertions(+), 18 deletions(-)
fs/notify/fanotify/fanotify.c +4 -3
@@ -192,6 +192,7 @@ static int fanotify_handle_event(struct fsnotify_group *group, const unsigned char *file_name, u32 cookie) { int ret = 0; +enum fsn_add_event_ret ae_ret;struct fanotify_event_info *event;struct fsnotify_event *fsn_event;
@@ -218,10 +219,10 @@ static int fanotify_handle_event(struct fsnotify_group *group,return -ENOMEM ; fsn_event = &event->fse; - ret =fsnotify_add_event (group, fsn_event, fanotify_merge); -if (ret) { + ae_ret =fsnotify_add_event (group, fsn_event, fanotify_merge); +if (ae_ret !=AE_INSERTED ) {/* Permission events shouldn't be merged */ -BUG_ON (ret ==1 && mask &FAN_ALL_PERM_EVENTS ); +BUG_ON (ae_ret ==AE_MERGED && mask &FAN_ALL_PERM_EVENTS );/* Our event wasn't used in the end. Free it. */ fsnotify_destroy_event (group, fsn_event);
fs/notify/inotify/inotify_fsnotify.c +2 -2
@@ -72,7 +72,7 @@ int inotify_handle_event(struct fsnotify_group *group,struct inotify_inode_mark *i_mark;struct inotify_event_info *event;struct fsnotify_event *fsn_event; -int ret; +enum fsn_add_event_ret ret;int len =0 ;int alloc_len =sizeof (struct inotify_event_info );
@@ -109,7 +109,7 @@ int inotify_handle_event(struct fsnotify_group *group,strcpy (event->name, file_name); ret =fsnotify_add_event (group, fsn_event, inotify_merge); -if (ret) { +if (ret !=AE_INSERTED ) {/* Our event wasn't used in the end. Free it. */ fsnotify_destroy_event (group, fsn_event); }
fs/notify/notification.c +11 -9
@@ -84,12 +84,12 @@ void fsnotify_destroy_event(struct fsnotify_group *group,* added to the queue, 1 if the event was merged with some other queued event, * 2 if the queue of events has overflown. */ -int fsnotify_add_event (struct fsnotify_group *group, -struct fsnotify_event *event, -int (*merge)(struct list_head *, -struct fsnotify_event *)) +enum fsn_add_event_ret fsnotify_add_event ( +struct fsnotify_group *group, +struct fsnotify_event *event, +int (*merge)(struct list_head *,struct fsnotify_event *)) { -int ret =0 ; +enum fsn_add_event_ret ret =AE_INSERTED ;struct list_head *list = &group->notification_list;pr_debug ("%s: group=%p event=%p\n" , __func__, group, event);
@@ -97,7 +97,7 @@ int fsnotify_add_event(struct fsnotify_group *group,mutex_lock (&group->notification_mutex);if (group->q_len >= group->max_events) { - ret =2 ; + ret =AE_OVERFLOW ;/* Queue overflow event only if it isn't already queued */ if (!list_empty (&group->overflow_event->list)) {mutex_unlock (&group->notification_mutex);
@@ -108,10 +108,12 @@ int fsnotify_add_event(struct fsnotify_group *group, }if (!list_empty (list) && merge) { - ret =merge (list, event); -if (ret) { +int merge_ret; + + merge_ret =merge (list, event); +if (merge_ret) {mutex_unlock (&group->notification_mutex); -return ret; +return AE_MERGED ; } }
include/linux/fsnotify_backend.h +10 -4
@@ -299,11 +299,17 @@ extern int fsnotify_fasync(int fd, struct file *file, int on);/* Free event from memory */ extern void fsnotify_destroy_event (struct fsnotify_group *group,struct fsnotify_event *event); +/* Return values of fsnotify_add_event() */ +enum fsn_add_event_ret { +AE_INSERTED ,/* Event was added in the queue */ +AE_MERGED ,/* Event was merged with another event, passed event unused */ +AE_OVERFLOW ,/* Queue overflow, passed event unused */ +};/* attach the event to the group notification queue */ -extern int fsnotify_add_event (struct fsnotify_group *group, -struct fsnotify_event *event, -int (*merge)(struct list_head *, -struct fsnotify_event *)); +extern enum fsn_add_event_ret fsnotify_add_event ( +struct fsnotify_group *group, +struct fsnotify_event *event, +int (*merge)(struct list_head *,struct fsnotify_event *));/* Remove passed event from groups notification queue */ extern void fsnotify_remove_event (struct fsnotify_group *group,struct fsnotify_event *event);/* true if the group notification queue is empty */
Signature · 1 line
2.6.6
Re: [PATCH 1/3] fsnotify: Use enum for return values of fsnotify_add_event()
Quoted 3 lines · Jan Kara
Currently there are three possible returns from
fsnotify_add_event(). We will be adding a fourth one. Let's change magic numbers to enum to make things clearer.
This cleanup should go last, otherwise it'll just make backporting the fix harder.
Also I don't really see the value of having a big enum with 3 of the
values having the same meaning. Yeah, there's that BUG_ON for having
merged a permission event. But it checks the very obvious condition at the top of fanotify_merge() with the big comment, not very interesting.
So I'd kill the BUG_ON, drop the merged/overflowed/shutdown distinction and return a two way value (doing it with an enum still makes the code more readable, though).
Thanks,
Miklos
Quoted 133 lines
Signed-off-by Jan Kara <[email protected]>--- fs/notify/fanotify/fanotify.c | 7 ++++--- fs/notify/inotify/inotify_fsnotify.c | 4 ++-- fs/notify/notification.c | 20 +++++++++++--------- include/linux/fsnotify_backend.h | 14 ++++++++++---- 4 files changed, 27 insertions(+), 18 deletions(-)fs/notify/fanotify/fanotify.c +4
-3@@ -192,6 +192,7 @@ static int fanotify_handle_event(struct fsnotify_group *group, const unsigned char *file_name, u32 cookie) { int ret = 0; +enum fsn_add_event_ret ae_ret;struct fanotify_event_info *event;struct fsnotify_event *fsn_event;@@ -218,10 +219,10 @@ static int fanotify_handle_event(struct fsnotify_group *group,return -ENOMEM ; fsn_event = &event->fse; - ret =fsnotify_add_event (group, fsn_event, fanotify_merge); -if (ret) { + ae_ret =fsnotify_add_event (group, fsn_event, fanotify_merge); +if (ae_ret !=AE_INSERTED ) {/* Permission events shouldn't be merged */ -BUG_ON (ret ==1 && mask &FAN_ALL_PERM_EVENTS ); +BUG_ON (ae_ret ==AE_MERGED && mask &FAN_ALL_PERM_EVENTS );/* Our event wasn't used in the end. Free it. */ fsnotify_destroy_event (group, fsn_event);fs/notify/inotify/inotify_fsnotify.c +2
-2@@ -72,7 +72,7 @@ int inotify_handle_event(struct fsnotify_group *group,struct inotify_inode_mark *i_mark;struct inotify_event_info *event;struct fsnotify_event *fsn_event; -int ret; +enum fsn_add_event_ret ret;int len =0 ;int alloc_len =sizeof (struct inotify_event_info );@@ -109,7 +109,7 @@ int inotify_handle_event(struct fsnotify_group *group,strcpy (event->name, file_name); ret =fsnotify_add_event (group, fsn_event, inotify_merge); -if (ret) { +if (ret !=AE_INSERTED ) {/* Our event wasn't used in the end. Free it. */ fsnotify_destroy_event (group, fsn_event); }fs/notify/notification.c +11
-9@@ -84,12 +84,12 @@ void fsnotify_destroy_event(struct fsnotify_group *group,* added to the queue, 1 if the event was merged with some other queued event, * 2 if the queue of events has overflown. */ -int fsnotify_add_event (struct fsnotify_group *group, -struct fsnotify_event *event, -int (*merge)(struct list_head *, -struct fsnotify_event *)) +enum fsn_add_event_ret fsnotify_add_event ( +struct fsnotify_group *group, +struct fsnotify_event *event, +int (*merge)(struct list_head *,struct fsnotify_event *)) { -int ret =0 ; +enum fsn_add_event_ret ret =AE_INSERTED ;struct list_head *list = &group->notification_list;pr_debug ("%s: group=%p event=%p\n" , __func__, group, event);@@ -97,7 +97,7 @@ int fsnotify_add_event(struct fsnotify_group *group,mutex_lock (&group->notification_mutex);if (group->q_len >= group->max_events) { - ret =2 ; + ret =AE_OVERFLOW ;/* Queue overflow event only if it isn't already queued */ if (!list_empty (&group->overflow_event->list)) {mutex_unlock (&group->notification_mutex);@@ -108,10 +108,12 @@ int fsnotify_add_event(struct fsnotify_group *group, }if (!list_empty (list) && merge) { - ret =merge (list, event); -if (ret) { +int merge_ret; + + merge_ret =merge (list, event); +if (merge_ret) {mutex_unlock (&group->notification_mutex); -return ret; +return AE_MERGED ; } }include/linux/fsnotify_backend.h +10
-5@@ -299,11 +299,17 @@ extern int fsnotify_fasync(int fd, struct file *file, int on);/* Free event from memory */ extern void fsnotify_destroy_event (struct fsnotify_group *group,struct fsnotify_event *event); +/* Return values of fsnotify_add_event() */ +enum fsn_add_event_ret { +AE_INSERTED ,/* Event was added in the queue */ +AE_MERGED ,/* Event was merged with another event, passed event unused */ +AE_OVERFLOW ,/* Queue overflow, passed event unused */ +};/* attach the event to the group notification queue */ -extern int fsnotify_add_event (struct fsnotify_group *group, -struct fsnotify_event *event, -int (*merge)(struct list_head *, -struct fsnotify_event *)); +extern enum fsn_add_event_ret fsnotify_add_event ( +struct fsnotify_group *group, +struct fsnotify_event *event, +int (*merge)(struct list_head *,struct fsnotify_event *));/* Remove passed event from groups notification queue */ extern void fsnotify_remove_event (struct fsnotify_group *group,struct fsnotify_event *event);/* true if the group notification queue is empty */ --2.6.6
Jan Kara
maintainerRe: [PATCH 1/3] fsnotify: Use enum for return values of fsnotify_add_event()
Quoted 14 lines · Miklos Szeredi
Quoted 3 lines · Jan Kara
Currently there are three possible returns from
fsnotify_add_event(). We will be adding a fourth one. Let's change magic numbers to enum to make things clearer.This cleanup should go last, otherwise it'll just make backporting the fix harder.
Also I don't really see the value of having a big enum with 3 of the
values having the same meaning. Yeah, there's thatBUG_ONfor having
merged a permission event. But it checks the very obvious condition at the top offanotify_merge()with the big comment, not very interesting.
Quoted 3 lines
So I'd kill the
BUG_ON, drop the merged/overflowed/shutdown distinction and return a two way value (doing it with an enum still makes the code more readable, though).
Fair enough, I'll fix this up and resend the series (likely early next week).
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Jan Kara
maintainerPATCH 2/3 fsnotify: Add a way to stop queueing events on group shutdown
In reply to Jan KaraImplement a function that can be called when a group is being shutdown to stop queueing new events to the group. Fanotify will use this.
--- fs/notify/group.c | 19 +++++++++++++++++++ fs/notify/notification.c | 5 +++++ include/linux/fsnotify_backend.h | 4 ++++ 3 files changed, 28 insertions(+)
fs/notify/group.c +19 -0
@@ -40,6 +40,17 @@ static void fsnotify_final_destroy_group(struct fsnotify_group *group) }/* +* Stop queueing new events for this group. Once this function returns +* fsnotify_add_event() will not add any new events to the group's queue. +*/ +void fsnotify_group_stop_queueing (struct fsnotify_group *group) +{ +mutex_lock (&group->notification_mutex); + group->shutdown = true; +mutex_unlock (&group->notification_mutex); +} + +/* * Trying to get rid of a group. Remove all marks, flush all events and release * the group reference. * Note that another thread calling fsnotify_clear_marks_by_group() may still
@@ -47,6 +58,14 @@ static void fsnotify_final_destroy_group(struct fsnotify_group *group)*/ void fsnotify_destroy_group (struct fsnotify_group *group) { +/* +* Stop queueing new events. The code below is careful enough to not +* require this but fanotify needs to stop queuing events even before +* fsnotify_destroy_group() is called and this makes the other callers +* of fsnotify_destroy_group() to see the same behavior. +*/ +fsnotify_group_stop_queueing (group); +/* clear all inode marks for this group, attach them to destroy_list */ fsnotify_detach_group_marks (group);
fs/notify/notification.c +5 -0
@@ -96,6 +96,11 @@ enum fsn_add_event_ret fsnotify_add_event(mutex_lock (&group->notification_mutex); +if (group->shutdown) { +mutex_unlock (&group->notification_mutex); +return AE_SHUTDOWN ; + } +if (group->q_len >= group->max_events) { ret =AE_OVERFLOW ;/* Queue overflow event only if it isn't already queued */
include/linux/fsnotify_backend.h +4 -0
@@ -148,6 +148,7 @@ struct fsnotify_group {#define FS_PRIO_1 1/* fanotify content based access control */ #define FS_PRIO_2 2/* fanotify pre-content access */ unsigned int priority; +bool shutdown;/* group is being shut down, don't queue more events */ /* stores all fastpath marks assoc with this group so they can be cleaned on unregister */ struct mutex mark_mutex;/* protect marks_list */
@@ -292,6 +293,8 @@ extern struct fsnotify_group *fsnotify_alloc_group(const struct fsnotify_ops *opextern void fsnotify_get_group (struct fsnotify_group *group);/* drop reference on a group from fsnotify_alloc_group */ extern void fsnotify_put_group (struct fsnotify_group *group); +/* group destruction begins, stop queuing new events */ +extern void fsnotify_group_stop_queueing (struct fsnotify_group *group);/* destroy group */ extern void fsnotify_destroy_group (struct fsnotify_group *group);/* fasync handler function */
@@ -304,6 +307,7 @@ enum fsn_add_event_ret {AE_INSERTED ,/* Event was added in the queue */ AE_MERGED ,/* Event was merged with another event, passed event unused */ AE_OVERFLOW ,/* Queue overflow, passed event unused */ + AE_SHUTDOWN, /* Group is being released, passed event unused */ };/* attach the event to the group notification queue */ extern enum fsn_add_event_ret fsnotify_add_event(
Signature · 1 line
2.6.6
Re: [PATCH 2/3] fsnotify: Add a way to stop queueing events on group shutdown
s/AE_SHUTDOWN/1/
Quoted 95 lines · Jan Kara
Implement a function that can be called when a group is being shutdown to stop queueing new events to the group. Fanotify will use this.
Signed-off-by Jan Kara <[email protected]>--- fs/notify/group.c | 19 +++++++++++++++++++ fs/notify/notification.c | 5 +++++ include/linux/fsnotify_backend.h | 4 ++++ 3 files changed, 28 insertions(+)fs/notify/group.c +19
-0@@ -40,6 +40,17 @@ static void fsnotify_final_destroy_group(struct fsnotify_group *group) }/* +* Stop queueing new events for this group. Once this function returns +* fsnotify_add_event() will not add any new events to the group's queue. +*/ +void fsnotify_group_stop_queueing (struct fsnotify_group *group) +{ +mutex_lock (&group->notification_mutex); + group->shutdown = true; +mutex_unlock (&group->notification_mutex); +} + +/* * Trying to get rid of a group. Remove all marks, flush all events and release * the group reference. * Note that another thread calling fsnotify_clear_marks_by_group() may still @@ -47,6 +58,14 @@ static void fsnotify_final_destroy_group(struct fsnotify_group *group)*/ void fsnotify_destroy_group (struct fsnotify_group *group) { +/* +* Stop queueing new events. The code below is careful enough to not +* require this but fanotify needs to stop queuing events even before +* fsnotify_destroy_group() is called and this makes the other callers +* of fsnotify_destroy_group() to see the same behavior. +*/ +fsnotify_group_stop_queueing (group); +/* clear all inode marks for this group, attach them to destroy_list */ fsnotify_detach_group_marks (group);fs/notify/notification.c +5
-0@@ -96,6 +96,11 @@ enum fsn_add_event_ret fsnotify_add_event(mutex_lock (&group->notification_mutex); +if (group->shutdown) { +mutex_unlock (&group->notification_mutex); +return AE_SHUTDOWN ; + } +if (group->q_len >= group->max_events) { ret =AE_OVERFLOW ;/* Queue overflow event only if it isn't already queued */ include/linux/fsnotify_backend.h +4
-1@@ -148,6 +148,7 @@ struct fsnotify_group {#define FS_PRIO_1 1/* fanotify content based access control */ #define FS_PRIO_2 2/* fanotify pre-content access */ unsigned int priority; +bool shutdown;/* group is being shut down, don't queue more events */ /* stores all fastpath marks assoc with this group so they can be cleaned on unregister */ struct mutex mark_mutex;/* protect marks_list */ @@ -292,6 +293,8 @@ extern struct fsnotify_group *fsnotify_alloc_group(const struct fsnotify_ops *opextern void fsnotify_get_group (struct fsnotify_group *group);/* drop reference on a group from fsnotify_alloc_group */ extern void fsnotify_put_group (struct fsnotify_group *group); +/* group destruction begins, stop queuing new events */ +extern void fsnotify_group_stop_queueing (struct fsnotify_group *group);/* destroy group */ extern void fsnotify_destroy_group (struct fsnotify_group *group);/* fasync handler function */ @@ -304,6 +307,7 @@ enum fsn_add_event_ret {AE_INSERTED ,/* Event was added in the queue */ AE_MERGED ,/* Event was merged with another event, passed event unused */ AE_OVERFLOW ,/* Queue overflow, passed event unused */ + AE_SHUTDOWN, /* Group is being released, passed event unused */ };/* attach the event to the group notification queue */ extern enum fsn_add_event_ret fsnotify_add_event( --2.6.6
Jan Kara
maintainerPATCH 3/3 fanotify: Fix list corruption in fanotify_get_response()
In reply to Jan Karafanotify_get_response() calls fsnotify_remove_event() when it finds that group is being released from fanotify_release() (bypass_perm is set). However the event it removes need not be only in the group's notification queue but it can have already moved to access_list (userspace read the event before closing the fanotify instance fd) which is protected by a different lock. Thus when fsnotify_remove_event() races with fanotify_release() operating on access_list, the list can get corrupted.
Fix the problem by moving all the logic removing permission events from the lists to one place - fanotify_release().
--- fs/notify/fanotify/fanotify.c | 13 +------------ fs/notify/fanotify/fanotify_user.c | 36 ++++++++++++++++++++++++------------ include/linux/fsnotify_backend.h | 1 - 3 files changed, 25 insertions(+), 25 deletions(-)
fs/notify/fanotify/fanotify.c +1 -12
@@ -67,18 +67,7 @@ static int fanotify_get_response(struct fsnotify_group *group,pr_debug ("%s: group=%p event=%p\n" , __func__, group, event); -wait_event (group->fanotify_data.access_waitq, event->response || -atomic_read (&group->fanotify_data.bypass_perm)); - -if (!event->response) {/* bypass_perm set */ -/* -* Event was canceled because group is being destroyed. Remove -* it from group's event list because we are responsible for -* freeing the permission event. -*/ -fsnotify_remove_event (group, &event->fae.fse); -return 0 ; - } +wait_event (group->fanotify_data.access_waitq, event->response);/* userspace responded, convert to something usable */ switch (event->response) {
fs/notify/fanotify/fanotify_user.c +24 -12
@@ -358,16 +358,20 @@ static int fanotify_release(struct inode *ignored, struct file *file)#ifdef CONFIG_FANOTIFY_ACCESS_PERMISSIONS struct fanotify_perm_event_info *event, *next; +struct fsnotify_event *fsn_event;/* -* There may be still new events arriving in the notification queue -* but since userspace cannot use fanotify fd anymore, no event can -* enter or leave access_list by now. +* Stop new events from arriving in the notification queue. since +* userspace cannot use fanotify fd anymore, no event can enter or +* leave access_list by now either. */ -spin_lock (&group->fanotify_data.access_lock); - -atomic_inc (&group->fanotify_data.bypass_perm); +fsnotify_group_stop_queueing (group); +/* +* Process all permission events on access_list and notification queue +* and simulate reply from userspace. +*/ +spin_lock (&group->fanotify_data.access_lock);list_for_each_entry_safe (event, next, &group->fanotify_data.access_list, fae.fse.list) { pr_debug("%s: found group=%p event=%p\n", __func__, group,
@@ -379,12 +383,21 @@ static int fanotify_release(struct inode *ignored, struct file *file)spin_unlock (&group->fanotify_data.access_lock);/* -* Since bypass_perm is set, newly queued events will not wait for -* access response. Wake up the already sleeping ones now. -* synchronize_srcu() in fsnotify_destroy_group() will wait for all -* processes sleeping in fanotify_handle_event() waiting for access -* response and thus also for all permission events to be freed. +* Destroy all non-permission events. For permission events just +* dequeue them and set the response. They will be freed once the +* response is consumed and fanotify_get_response() returns. */ +mutex_lock (&group->notification_mutex); +while (!fsnotify_notify_queue_is_empty (group)) { + fsn_event =fsnotify_remove_first_event (group); +if (!(fsn_event->mask &FAN_ALL_PERM_EVENTS )) +fsnotify_destroy_event (group, fsn_event); +else +FANOTIFY_PE (fsn_event)->response =FAN_ALLOW ; + } +mutex_unlock (&group->notification_mutex); + +/* Response for all permission events it set, wakeup waiters */ wake_up (&group->fanotify_data.access_waitq);#endif
@@ -755,7 +768,6 @@ SYSCALL_DEFINE2(fanotify_init, unsigned int, flags, unsigned int, event_f_flags)spin_lock_init (&group->fanotify_data.access_lock);init_waitqueue_head (&group->fanotify_data.access_waitq);INIT_LIST_HEAD (&group->fanotify_data.access_list); -atomic_set (&group->fanotify_data.bypass_perm,0 );#endif switch (flags &FAN_ALL_CLASS_BITS ) {case FAN_CLASS_NOTIF :
include/linux/fsnotify_backend.h +0 -1
@@ -180,7 +180,6 @@ struct fsnotify_group {spinlock_t access_lock;struct list_head access_list;wait_queue_head_t access_waitq; -atomic_t bypass_perm;#endif /* CONFIG_FANOTIFY_ACCESS_PERMISSIONS */ int f_flags;unsigned int max_marks;
Signature · 1 line
2.6.6
Quoted 10 lines · Jan Kara
fanotify_get_response()callsfsnotify_remove_event()when it finds that group is being released fromfanotify_release()(bypass_permis set). However the event it removes need not be only in the group's notification queue but it can have already moved toaccess_list(userspace read the event before closing the fanotify instance fd) which is protected by a different lock. Thus whenfsnotify_remove_event()races withfanotify_release()operating onaccess_list, the list can get corrupted.Fix the problem by moving all the logic removing permission events from the lists to one place -
fanotify_release().
This should fix the reported issue, and it's cleaner than my solution. I'll backport and ask QA to test. I haven't tried reproducing it myself.
However the theoretical memory access ordering issue (which might possibly be impossible to trigger) is still there AFAICS:
CPU1:list_del_init(&event->fae.fse.list);event->response = FAN_ALLOW;
CPU2:wait_event(group->fanotify_data.access_waitq, event->response);
...
WARN_ON(!list_empty(&event->list));
So I think we still need a separate patch adding smp_wmb() before setting event->response and smp_rmb() after the wait_event().
Thanks,
Miklos
Quoted 114 lines
Reported-by Miklos Szeredi <[email protected]>Signed-off-by Jan Kara <[email protected]>--- fs/notify/fanotify/fanotify.c | 13 +------------ fs/notify/fanotify/fanotify_user.c | 36 ++++++++++++++++++++++++------------ include/linux/fsnotify_backend.h | 1 - 3 files changed, 25 insertions(+), 25 deletions(-)fs/notify/fanotify/fanotify.c +1
-12@@ -67,18 +67,7 @@ static int fanotify_get_response(struct fsnotify_group *group,pr_debug ("%s: group=%p event=%p\n" , __func__, group, event); -wait_event (group->fanotify_data.access_waitq, event->response || -atomic_read (&group->fanotify_data.bypass_perm)); - -if (!event->response) {/* bypass_perm set */ -/* -* Event was canceled because group is being destroyed. Remove -* it from group's event list because we are responsible for -* freeing the permission event. -*/ -fsnotify_remove_event (group, &event->fae.fse); -return 0 ; - } +wait_event (group->fanotify_data.access_waitq, event->response);/* userspace responded, convert to something usable */ switch (event->response) {fs/notify/fanotify/fanotify_user.c +24
-12@@ -358,16 +358,20 @@ static int fanotify_release(struct inode *ignored, struct file *file)#ifdef CONFIG_FANOTIFY_ACCESS_PERMISSIONS struct fanotify_perm_event_info *event, *next; +struct fsnotify_event *fsn_event;/* -* There may be still new events arriving in the notification queue -* but since userspace cannot use fanotify fd anymore, no event can -* enter or leave access_list by now. +* Stop new events from arriving in the notification queue. since +* userspace cannot use fanotify fd anymore, no event can enter or +* leave access_list by now either. */ -spin_lock (&group->fanotify_data.access_lock); - -atomic_inc (&group->fanotify_data.bypass_perm); +fsnotify_group_stop_queueing (group); +/* +* Process all permission events on access_list and notification queue +* and simulate reply from userspace. +*/ +spin_lock (&group->fanotify_data.access_lock);list_for_each_entry_safe (event, next, &group->fanotify_data.access_list, fae.fse.list) { pr_debug("%s: found group=%p event=%p\n", __func__, group,@@ -379,12 +383,21 @@ static int fanotify_release(struct inode *ignored, struct file *file)spin_unlock (&group->fanotify_data.access_lock);/* -* Since bypass_perm is set, newly queued events will not wait for -* access response. Wake up the already sleeping ones now. -* synchronize_srcu() in fsnotify_destroy_group() will wait for all -* processes sleeping in fanotify_handle_event() waiting for access -* response and thus also for all permission events to be freed. +* Destroy all non-permission events. For permission events just +* dequeue them and set the response. They will be freed once the +* response is consumed and fanotify_get_response() returns. */ +mutex_lock (&group->notification_mutex); +while (!fsnotify_notify_queue_is_empty (group)) { + fsn_event =fsnotify_remove_first_event (group); +if (!(fsn_event->mask &FAN_ALL_PERM_EVENTS )) +fsnotify_destroy_event (group, fsn_event); +else +FANOTIFY_PE (fsn_event)->response =FAN_ALLOW ; + } +mutex_unlock (&group->notification_mutex); + +/* Response for all permission events it set, wakeup waiters */ wake_up (&group->fanotify_data.access_waitq);#endif @@ -755,7 +768,6 @@ SYSCALL_DEFINE2(fanotify_init, unsigned int, flags, unsigned int, event_f_flags)spin_lock_init (&group->fanotify_data.access_lock);init_waitqueue_head (&group->fanotify_data.access_waitq);INIT_LIST_HEAD (&group->fanotify_data.access_list); -atomic_set (&group->fanotify_data.bypass_perm,0 );#endif switch (flags &FAN_ALL_CLASS_BITS ) {case FAN_CLASS_NOTIF :include/linux/fsnotify_backend.h +0
-2@@ -180,7 +180,6 @@ struct fsnotify_group {spinlock_t access_lock;struct list_head access_list;wait_queue_head_t access_waitq; -atomic_t bypass_perm;#endif /* CONFIG_FANOTIFY_ACCESS_PERMISSIONS */ int f_flags;unsigned int max_marks; --2.6.6
Jan Kara
maintainerQuoted 9 lines · Miklos Szeredi
Quoted 8 lines · Jan Kara
fanotify_get_response()callsfsnotify_remove_event()when it finds that group is being released fromfanotify_release()(bypass_permis set). However the event it removes need not be only in the group's notification queue but it can have already moved toaccess_list(userspace read the event before closing the fanotify instance fd) which is protected by a different lock. Thus whenfsnotify_remove_event()races withfanotify_release()operating onaccess_list, the list can get corrupted.
Quoted 8 lines
Quoted 2 lines
Fix the problem by moving all the logic removing permission events from the lists to one place -
fanotify_release().Reviewed-by Miklos Szeredi <[email protected]>This should fix the reported issue, and it's cleaner than my solution. I'll backport and ask QA to test. I haven't tried reproducing it myself.
OK, thanks!
Quoted 7 lines
However the theoretical memory access ordering issue (which might possibly be impossible to trigger) is still there AFAICS:
CPU1:
list_del_init(&event->fae.fse.list);event->response=FAN_ALLOW;
Quoted 7 lines
CPU2:
wait_event(group->fanotify_data.access_waitq, event->response);
...
WARN_ON(!list_empty(&event->list));So I think we still need a separate patch adding
smp_wmb()before settingevent->responseandsmp_rmb()after thewait_event().
Ugh, this is really theoretical ;) But I agree nothing really prevents it. The core reason for this seems to be the lockless check of event->list. I dislike adding barriers just to accommodate that WARN_ON() (although that is a useful one so I wouldn't like to lose it either). I'll think about it. Thanks for spotting this.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Hi,
Quoted 8 lines · Jan Kara
fanotify_get_response()callsfsnotify_remove_event()when it finds that group is being released fromfanotify_release()(bypass_permis set). However the event it removes need not be only in the group's notification queue but it can have already moved toaccess_list(userspace read the event before closing the fanotify instance fd) which is protected by a different lock. Thus whenfsnotify_remove_event()races withfanotify_release()operating onaccess_list, the list can get corrupted.
Quoted 5 lines
Fix the problem by moving all the logic removing permission events from the lists to one place -
fanotify_release().Reported-by Miklos Szeredi <[email protected]>Signed-off-by Jan Kara <[email protected]>
I think all in all this patchset implements a sane solution. Nevertheless some comments/questions concerning the new code in fanotify_release below:
Quoted 10 lines
fs/notify/fanotify/fanotify_user.c +1
-5@@ -379,12 +383,21 @@ static int fanotify_release(struct inode *ignored, struct file *file)spin_unlock (&group->fanotify_data.access_lock);/* -* Since bypass_perm is set, newly queued events will not wait for -* access response. Wake up the already sleeping ones now. -* synchronize_srcu() in fsnotify_destroy_group() will wait for all -* processes sleeping in fanotify_handle_event() waiting for access -* response and thus also for all permission events to be freed. +* Destroy all non-permission events. For permission events just
fs/notify/fanotify/fanotify_user.c static int fanotify_release(struct inode *ignored, struct file *file) · line 396
Quoted 10 lines
fs/notify/fanotify/fanotify_user.c +9
-0+* dequeue them and set the response. They will be freed once the +* response is consumed and fanotify_get_response() returns. */ +mutex_lock (&group->notification_mutex); +while (!fsnotify_notify_queue_is_empty (group)) { + fsn_event =fsnotify_remove_first_event (group); +if (!(fsn_event->mask &FAN_ALL_PERM_EVENTS )) +fsnotify_destroy_event (group, fsn_event); +else +FANOTIFY_PE (fsn_event)->response =FAN_ALLOW ;
Why do we have to handle (i.e destroy) non-permission events here? Is this meant as some kind of optimization?
Also, we now set the response during both iterations. I assume the assignment during the iteration of the access list is only a leftover and can be skipped.
Why dont we set the shutdown flag directly in release() as soon as we grep the notification mutex. I dont see any reason to do this seperated from the removal of events from the notification list (or do I miss something?). Similar situation in destroy_group(): We can set the flag in flush_notify instead of destroy_group (we then do not even need the dedicated function stop_queueing() then).
fsnotify_remove_event() is not used any more now, so we should remove this function.
Regards,
Lino
Jan Kara
maintainerQuoted 15 lines · Lino Sanfilippo
Quoted 15 lines
fs/notify/fanotify/fanotify_user.c +5
-5@@ -379,12 +383,21 @@ static int fanotify_release(struct inode *ignored, struct file *file)spin_unlock (&group->fanotify_data.access_lock);/* -* Since bypass_perm is set, newly queued events will not wait for -* access response. Wake up the already sleeping ones now. -* synchronize_srcu() in fsnotify_destroy_group() will wait for all -* processes sleeping in fanotify_handle_event() waiting for access -* response and thus also for all permission events to be freed. +* Destroy all non-permission events. For permission events just +* dequeue them and set the response. They will be freed once the +* response is consumed and fanotify_get_response() returns. */ +mutex_lock (&group->notification_mutex); +while (!fsnotify_notify_queue_is_empty (group)) {
Quoted 8 lines
fs/notify/fanotify/fanotify_user.c static int fanotify_release(struct inode *ignored, struct file *file) · line 396
Quoted 5 lines
fs/notify/fanotify/fanotify_user.c +5
-0+ fsn_event =fsnotify_remove_first_event (group); +if (!(fsn_event->mask &FAN_ALL_PERM_EVENTS )) +fsnotify_destroy_event (group, fsn_event); +else +FANOTIFY_PE (fsn_event)->response =FAN_ALLOW ;Why do we have to handle (i.e destroy) non-permission events here? Is this meant as some kind of optimization?
Where else would they be destroyed? Normal events are destroyed when userspace reads them but nobody is going to read these anymore. For permission events things are more complex - they get destroyed by the process creating these events since that process waits for the response.
Quoted 3 lines
Also, we now set the response during both iterations. I assume the assignment during the iteration of the access list is only a leftover and can be skipped.
Both are currently needed because I've changed fanotify_get_response() to check only the response value.
Quoted 6 lines
Why dont we set the shutdown flag directly in
release()as soon as we grep the notification mutex. I dont see any reason to do this seperated from the removal of events from the notification list (or do I miss something?). Similar situation indestroy_group(): We can set the flag inflush_notifyinstead ofdestroy_group(we then do not even need the dedicated functionstop_queueing()then).
We could do it like that but I didn't want fanotify to hook into generic fsnotify internals and rather wanted to encapsulate the functionality in a function. Since this is not really performance critical, extra round trip on notification_mutex should be fine.
Quoted 2 lines
fsnotify_remove_event()is not used any more now, so we should remove this function.
Yeah, will fix that.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 25 lines · Jan Kara
Quoted 15 lines · Lino Sanfilippo
Quoted 15 lines
fs/notify/fanotify/fanotify_user.c +5
-5@@ -379,12 +383,21 @@ static int fanotify_release(struct inode *ignored, struct file *file)spin_unlock (&group->fanotify_data.access_lock);/* -* Since bypass_perm is set, newly queued events will not wait for -* access response. Wake up the already sleeping ones now. -* synchronize_srcu() in fsnotify_destroy_group() will wait for all -* processes sleeping in fanotify_handle_event() waiting for access -* response and thus also for all permission events to be freed. +* Destroy all non-permission events. For permission events just +* dequeue them and set the response. They will be freed once the +* response is consumed and fanotify_get_response() returns. */ +mutex_lock (&group->notification_mutex); +while (!fsnotify_notify_queue_is_empty (group)) {Quoted 8 lines
fs/notify/fanotify/fanotify_user.c static int fanotify_release(struct inode *ignored, struct file *file) · line 396
Quoted 5 lines
fs/notify/fanotify/fanotify_user.c +5
-0+ fsn_event =fsnotify_remove_first_event (group); +if (!(fsn_event->mask &FAN_ALL_PERM_EVENTS )) +fsnotify_destroy_event (group, fsn_event); +else +FANOTIFY_PE (fsn_event)->response =FAN_ALLOW ;Why do we have to handle (i.e destroy) non-permission events here? Is this meant as some kind of optimization?
Quoted 4 lines
Where else would they be destroyed? Normal events are destroyed when userspace reads them but nobody is going to read these anymore. For permission events things are more complex - they get destroyed by the process creating these events since that process waits for the response.
I think he meant that normal events get destroyed anyway in fsnotify_destroy_group(). Which is true, but then we'd need a proper iterator for picking just permission events and it would be a lot more complicated in the end.
Thanks,
Miklos
Quoted 8 lines · Jan Kara
Quoted 7 lines
Why dont we set the shutdown flag directly in
release()as soon as we grep the notification mutex. I dont see any reason to do this seperated from the removal of events from the notification list (or do I miss something?). Similar situation indestroy_group(): We can set the flag inflush_notifyinstead ofdestroy_group(we then do not even need the dedicated functionstop_queueing()then).
Quoted 4 lines
We could do it like that but I didn't want fanotify to hook into generic fsnotify internals and rather wanted to encapsulate the functionality in a function. Since this is not really performance critical, extra round trip on
notification_mutexshould be fine.
I understood that flag as an extension for generic fsnotify groups, not only for fanotify. Sure, there is nothing else that uses it right now, but "being destroyed" is a state that is valid for all types of fsnotify groups, and it could
be useful in future to know when a group is in this state. But maybe it depends
on the point of view if this should be treated as a fanotify or fsnotify extension. And its up to you, of course.
Regards,
Lino
The backported patches test out OK.
I asked permission to publish the reproducer.
Thanks,
Miklos
Quoted 3 lines · Miklos Szeredi
The backported patches test out OK.
I asked permission to publish the reproducer.
Fujitsu provided the attached reproducer. Entry point is "./run.sh"
Thanks,
Miklos