Skip to contentSkip to conversation

fsnotify: reduce coupling of permission and non permission events

RFC PATCH In review Amir Goldstein 2 participants 6 messages Nov 14, 2016 – Nov 16, 2016

Subsystems (3)
lore
Share & tools
lore

Amir Goldstein

author reviewer
#0 lore raw

cover fsnotify: reduce coupling of permission and non permission events

The issue reported by Miklos Szeredi was processes blocking when trying to close an inotify file descriptor, while another fanotify listener is failing to handle permission events.

Miklos included a test program with the report.
His test program with some modifications is available on my github:
https://github.com/amir73il/fsnotify-utils/blob/master/fanotify_bug.c

Ideally, we would want that destruction of a group would only block if an event handled by this group is in progress, but this is not easy to achieve. Instead, we make sure that destrurction of a group would only block if an event handled by a group of similar class (priority) is in progress.

Amir Goldstein (2):
fsnotify: separate fsnotify_mark_srcu for groups with permission
events
fsnotify: handle permission events without holding
fsnotify_mark_srcu[0]

 fs/notify/fanotify/fanotify.c | 15 ++++++---
 fs/notify/fsnotify.c          | 57 ++++++++++++++++++++++++++++----
 fs/notify/fsnotify.h          | 17 ++++++++--
 fs/notify/group.c             |  2 +-
 fs/notify/mark.c              | 77 ++++++++++++++++++++++++++++++++-----------
 5 files changed, 134 insertions(+), 34 deletions(-)
Signature · 1 line
2.7.4
author reviewer
#1 lore raw

RFC PATCH 1/2 fsnotify: separate fsnotify_mark_srcu for groups with permission events

fsnotify_mark_srcu[1] protects reads of the first half of inode/vfsmount mark lists, which consist of the high priority (>0) marks, whose masks may contain permission events.

fsnotify_mark_srcu[0] protects reads of the second half of inode/vfsmount mark lists, which consist of the low priority (0) marks, whose masks do not contain permission events.

High priority marks and low priority marks are added to different destroy lists and freed by different reapers, who synchronize only the relevant srcu.

A follow up change will use this separation to guaranty that destroying low priority groups will not block on handling of permission events.

---
 fs/notify/fsnotify.c | 34 ++++++++++++++++++-----
 fs/notify/fsnotify.h | 17 ++++++++++--
 fs/notify/group.c    |  2 +-
 fs/notify/mark.c     | 77 +++++++++++++++++++++++++++++++++++++++-------------
 4 files changed, 100 insertions(+), 30 deletions(-)
fs/notify/fsnotify.c +27 -7
index db39de2..af5c523a 100644
@@ -192,9 +192,10 @@ int fsnotify(struct inode *to_tell, __u32 mask, void *data, int data_is,
 {
 	struct hlist_node *inode_node = NULL, *vfsmount_node = NULL;
 	struct fsnotify_mark *inode_mark = NULL, *vfsmount_mark = NULL;
-	struct fsnotify_group *inode_group, *vfsmount_group;
+	struct fsnotify_group *inode_group, *vfsmount_group, *group;
 	struct mount *mnt;
-	int idx, ret = 0;
+	int perm_idx, idx;
+	int ret = 0;
 	/* global tests shouldn't care about events on child only the specific event */
 	__u32 test_mask = (mask & ~FS_EVENT_ON_CHILD);
 
@@ -223,7 +224,14 @@ int fsnotify(struct inode *to_tell, __u32 mask, void *data, int data_is,
 	    !(mnt && test_mask & mnt->mnt_fsnotify_mask))
 		return 0;
 
-	idx = srcu_read_lock(&fsnotify_mark_srcu);
+	/*
+	 * First mark on the list may be either low or high priority, so
+	 * when traversing to first mark we must hold read side of both srcu.
+	 * When we reach the first low priority mark, we may drop the
+	 * read side of fsnotify_mark_srcu[1]
+	 */
+	idx = srcu_read_lock(&fsnotify_mark_srcu[0]);
+	perm_idx = srcu_read_lock(&fsnotify_mark_srcu[1]);
 
 	if ((mask & FS_MODIFY) ||
 	    (test_mask & to_tell->i_fsnotify_mask))
@@ -252,13 +260,13 @@ int fsnotify(struct inode *to_tell, __u32 mask, void *data, int data_is,
 		if (inode_node) {
 			inode_mark = hlist_entry(srcu_dereference(inode_node, &fsnotify_mark_srcu),
 						 struct fsnotify_mark, obj_list);
-			inode_group = inode_mark->group;
+			group = inode_group = inode_mark->group;
 		}
 
 		if (vfsmount_node) {
 			vfsmount_mark = hlist_entry(srcu_dereference(vfsmount_node, &fsnotify_mark_srcu),
 						    struct fsnotify_mark, obj_list);
-			vfsmount_group = vfsmount_mark->group;
+			group = vfsmount_group = vfsmount_mark->group;
 		}
 
 		if (inode_group && vfsmount_group) {
@@ -270,8 +278,16 @@ int fsnotify(struct inode *to_tell, __u32 mask, void *data, int data_is,
 			} else if (cmp < 0) {
 				vfsmount_group = NULL;
 				vfsmount_mark = NULL;
+				group = inode_group;
 			}
 		}
+
+		if (group->priority == 0 && perm_idx >= 0) {
+			/* We are done iterating the high priority marks */
+			srcu_read_unlock(&fsnotify_mark_srcu[1], perm_idx);
+			perm_idx = -1;
+		}
+
 		ret = send_to_group(to_tell, inode_mark, vfsmount_mark, mask,
 				    data, data_is, cookie, file_name);
 
@@ -287,7 +303,9 @@ int fsnotify(struct inode *to_tell, __u32 mask, void *data, int data_is,
 	}
 	ret = 0;
 out:
-	srcu_read_unlock(&fsnotify_mark_srcu, idx);
+	srcu_read_unlock(&fsnotify_mark_srcu[0], idx);
+	if (perm_idx >= 0)
+		srcu_read_unlock(&fsnotify_mark_srcu[1], perm_idx);
 
 	return ret;
 }
@@ -299,7 +317,9 @@ static __init int fsnotify_init(void)
 
 	BUG_ON(hweight32(ALL_FSNOTIFY_EVENTS) != 23);
 
-	ret = init_srcu_struct(&fsnotify_mark_srcu);
+	ret = init_srcu_struct(&fsnotify_mark_srcu[0]);
+	if (!ret)
+		ret = init_srcu_struct(&fsnotify_mark_srcu[1]);
 	if (ret)
 		panic("initializing fsnotify_mark_srcu");
 
fs/notify/fsnotify.h +14 -3
index 0a3bc2c..c1d6ae6 100644
@@ -11,8 +11,19 @@
 /* destroy all events sitting in this groups notification queue */
 extern void fsnotify_flush_notify(struct fsnotify_group *group);
 
-/* protects reads of inode and vfsmount marks list */
-extern struct srcu_struct fsnotify_mark_srcu;
+/*
+ * fsnotify_mark_srcu[1] protects reads of the first half of inode/vfsmount
+ * mark lists, which consist of the high priority (>0) marks, whose masks
+ * may contain permission events.
+ *
+ * fsnotify_mark_srcu[0] protects reads of the second half of inode/vfsmount
+ * mark lists, which consist of the low priority (0) marks, whose masks
+ * do not contain permission events.
+ */
+#define FS_PRIO_SRCU(prio)	((int)(prio > FS_PRIO_0))
+#define FS_PRIO_SRCU_NUM	2
+
+extern struct srcu_struct fsnotify_mark_srcu[FS_PRIO_SRCU_NUM];
 
 /* Calculate mask of events for a list of marks */
 extern u32 fsnotify_recalc_mask(struct hlist_head *head);
@@ -61,7 +72,7 @@ extern void fsnotify_detach_group_marks(struct fsnotify_group *group);
 /*
  * wait for fsnotify_mark_srcu period to end and free all marks in destroy_list
  */
-extern void fsnotify_mark_destroy_list(void);
+extern void fsnotify_mark_destroy_list(int priority);
 
 /*
  * update the dentry->d_flags of all of inode's children to indicate if inode cares
fs/notify/group.c +1 -1
index fbe3cbe..ef946c0 100644
@@ -73,7 +73,7 @@ void fsnotify_destroy_group(struct fsnotify_group *group)
 	 * Wait for fsnotify_mark_srcu period to end and free all marks in
 	 * destroy_list
 	 */
-	fsnotify_mark_destroy_list();
+	fsnotify_mark_destroy_list(group->priority);
 
 	/*
 	 * Since we have waited for fsnotify_mark_srcu in
fs/notify/mark.c +58 -19

123 lines not on this page · raw message

Signature · 1 line
2.7.4
author reviewer
#2 lore raw

RFC PATCH 2/2 fsnotify: handle permission events without holding fsnotify_mark_srcu[0]

In reply to Amir Goldstein

Handling fanotify events does not entail dereferencing fsnotify_mark beyond the point of fanotify_should_send_event().

For the case of permission events, which may block indefinitely, return -EAGAIN and then fsnotify() will call handle_event() again without a reference to the mark.

Without a reference to the mark, there is no need to call handle_event() under fsnotify_mark_srcu[0] read side lock, so we drop fsnotify_mark_srcu[0] while handling the event and grab it back before continuing to the next mark.

After this change, a blocking permission event will no longer block closing of any file descriptors of 0 priority groups, i.e: inotify and fanotify groups of class FAN_CLASS_NOTIF.

---
 fs/notify/fanotify/fanotify.c | 15 +++++++++++----
 fs/notify/fsnotify.c          | 23 +++++++++++++++++++++++
 2 files changed, 34 insertions(+), 4 deletions(-)
fs/notify/fanotify/fanotify.c +11 -4
index e0e5f7c..c7689ad 100644
@@ -176,7 +176,7 @@ init: __maybe_unused
 static int fanotify_handle_event(struct fsnotify_group *group,
 				 struct inode *inode,
 				 struct fsnotify_mark *inode_mark,
-				 struct fsnotify_mark *fanotify_mark,
+				 struct fsnotify_mark *vfsmnt_mark,
 				 u32 mask, void *data, int data_type,
 				 const unsigned char *file_name, u32 cookie)
 {
@@ -195,9 +195,16 @@ static int fanotify_handle_event(struct fsnotify_group *group,
 	BUILD_BUG_ON(FAN_ACCESS_PERM != FS_ACCESS_PERM);
 	BUILD_BUG_ON(FAN_ONDIR != FS_ISDIR);
 
-	if (!fanotify_should_send_event(inode_mark, fanotify_mark, mask, data,
-					data_type))
-		return 0;
+	if (inode_mark || vfsmnt_mark) {
+		if (!fanotify_should_send_event(inode_mark, vfsmnt_mark, mask,
+						data, data_type))
+			return 0;
+#ifdef CONFIG_FANOTIFY_ACCESS_PERMISSIONS
+		/* Ask to be called again without a reference to mark */
+		if (mask & FAN_ALL_PERM_EVENTS)
+			return -EAGAIN;
+#endif
+	}
 
 	pr_debug("%s: group=%p inode=%p mask=%x\n", __func__, group, inode,
 		 mask);
fs/notify/fsnotify.c +23 -0
index af5c523a..5b9a248 100644
@@ -291,6 +291,29 @@ int fsnotify(struct inode *to_tell, __u32 mask, void *data, int data_is,
 		ret = send_to_group(to_tell, inode_mark, vfsmount_mark, mask,
 				    data, data_is, cookie, file_name);
 
+		/*
+		 * If handle_event() is going to block, we call it again
+		 * witout holding fsnotify_mark_srcu[0], which is protecting
+		 * the low priority mark lists.
+		 * We are still holding fsnotify_mark_srcu[1], which
+		 * is protecting the high priority marks in the first half
+		 * of the mark list, which is where we are at.
+		 */
+		if (group->priority > 0 && ret == -EAGAIN) {
+			srcu_read_unlock(&fsnotify_mark_srcu[0], idx);
+
+			ret = group->ops->handle_event(group, to_tell,
+						       NULL, NULL,
+						       mask, data, data_is,
+						       file_name, cookie);
+
+			/*
+			 * We need to hold fsnotify_mark_srcu[0], because
+			 * next mark may be low priority.
+			 */
+			idx = srcu_read_lock(&fsnotify_mark_srcu[0]);
+		}
+
 		if (ret && (mask & ALL_FSNOTIFY_PERM_EVENTS))
 			goto out;
 
Signature · 1 line
2.7.4
maintainer
#3 lore raw

Re: [RFC][PATCH 2/2] fsnotify: handle permission events without holding fsnotify_mark_srcu[0]

Quoted 12 lines · Amir Goldstein

Handling fanotify events does not entail dereferencing fsnotify_mark beyond the point of fanotify_should_send_event().

For the case of permission events, which may block indefinitely, return -EAGAIN and then fsnotify() will call handle_event() again without a reference to the mark.

Without a reference to the mark, there is no need to call handle_event() under fsnotify_mark_srcu[0] read side lock, so we drop fsnotify_mark_srcu[0] while handling the event and grab it back before continuing to the next mark.

Quoted 6 lines

After this change, a blocking permission event will no longer block closing of any file descriptors of 0 priority groups, i.e: inotify and fanotify groups of class FAN_CLASS_NOTIF.

Well, this has a similar problem as my attempt to fix the issue. The current mark can get removed from the mark list while waiting for userspace response. ->next pointer is still valid at that moment but ->next->pprev already points to mark preceding us (that's how rcu lists work). When ->next mark then gets removed from the list and destroyed (it may be protected by the second srcu), our ->next pointer points to freed memory.

Furthermore I don't like the scheme of ->handle_event returning -EAGAIN and then dropping the srcu lock - I'd rather have some helpers provided by the generic fsnotify code to drop srcu lock. That needs some propagation of information inside the ->handle_event and then the helper but that's IMO cleaner. Anyway, that is just a technical detail.

I have some ideas how to fix up issues with my refcounting approach - refcount would pin marks not only in memory but also in lists but I have yet to see whether that works out sensibly (it would mean that dropping mark reference would then need to take group->mark_mutex and that may cause lock ordering issues).

Honza

Quoted 70 lines
fs/notify/fanotify/fanotify.c +11 -4
index e0e5f7c..c7689ad 100644
@@ -176,7 +176,7 @@ init: __maybe_unused
 static int fanotify_handle_event(struct fsnotify_group *group,
 				 struct inode *inode,
 				 struct fsnotify_mark *inode_mark,
-				 struct fsnotify_mark *fanotify_mark,
+				 struct fsnotify_mark *vfsmnt_mark,
 				 u32 mask, void *data, int data_type,
 				 const unsigned char *file_name, u32 cookie)
 {
@@ -195,9 +195,16 @@ static int fanotify_handle_event(struct fsnotify_group *group,
 	BUILD_BUG_ON(FAN_ACCESS_PERM != FS_ACCESS_PERM);
 	BUILD_BUG_ON(FAN_ONDIR != FS_ISDIR);
 
-	if (!fanotify_should_send_event(inode_mark, fanotify_mark, mask, data,
-					data_type))
-		return 0;
+	if (inode_mark || vfsmnt_mark) {
+		if (!fanotify_should_send_event(inode_mark, vfsmnt_mark, mask,
+						data, data_type))
+			return 0;
+#ifdef CONFIG_FANOTIFY_ACCESS_PERMISSIONS
+		/* Ask to be called again without a reference to mark */
+		if (mask & FAN_ALL_PERM_EVENTS)
+			return -EAGAIN;
+#endif
+	}
 
 	pr_debug("%s: group=%p inode=%p mask=%x\n", __func__, group, inode,
 		 mask);
fs/notify/fsnotify.c +23 -0
index af5c523a..5b9a248 100644
@@ -291,6 +291,29 @@ int fsnotify(struct inode *to_tell, __u32 mask, void *data, int data_is,
 		ret = send_to_group(to_tell, inode_mark, vfsmount_mark, mask,
 				    data, data_is, cookie, file_name);
 
+		/*
+		 * If handle_event() is going to block, we call it again
+		 * witout holding fsnotify_mark_srcu[0], which is protecting
+		 * the low priority mark lists.
+		 * We are still holding fsnotify_mark_srcu[1], which
+		 * is protecting the high priority marks in the first half
+		 * of the mark list, which is where we are at.
+		 */
+		if (group->priority > 0 && ret == -EAGAIN) {
+			srcu_read_unlock(&fsnotify_mark_srcu[0], idx);
+
+			ret = group->ops->handle_event(group, to_tell,
+						       NULL, NULL,
+						       mask, data, data_is,
+						       file_name, cookie);
+
+			/*
+			 * We need to hold fsnotify_mark_srcu[0], because
+			 * next mark may be low priority.
+			 */
+			idx = srcu_read_lock(&fsnotify_mark_srcu[0]);
+		}
+
 		if (ret && (mask & ALL_FSNOTIFY_PERM_EVENTS))
 			goto out;
 
Signature · 1 line
2.7.4
Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
author reviewer
#4 lore raw

Re: [RFC][PATCH 2/2] fsnotify: handle permission events without holding fsnotify_mark_srcu[0]

Quoted 20 lines · Jan Kara
Quoted 12 lines · Amir Goldstein

Handling fanotify events does not entail dereferencing fsnotify_mark beyond the point of fanotify_should_send_event().

For the case of permission events, which may block indefinitely, return -EAGAIN and then fsnotify() will call handle_event() again without a reference to the mark.

Without a reference to the mark, there is no need to call handle_event() under fsnotify_mark_srcu[0] read side lock, so we drop fsnotify_mark_srcu[0] while handling the event and grab it back before continuing to the next mark.

Quoted 6 lines

After this change, a blocking permission event will no longer block closing of any file descriptors of 0 priority groups, i.e: inotify and fanotify groups of class FAN_CLASS_NOTIF.

Quoted 6 lines

Well, this has a similar problem as my attempt to fix the issue. The current mark can get removed from the mark list while waiting for userspace response. ->next pointer is still valid at that moment but ->next->pprev already points to mark preceding us (that's how rcu lists work). When ->next mark then gets removed from the list and destroyed (it may be protected by the second srcu), our ->next pointer points to freed memory.

Oh! I missed the fact that the SRCU does not protect mark->obj_list. Can resurrecting mark->free_list buy us anything?
Perhaps keep the marks on obj_list without FLAG_ATTACHED
and then remove marks from both free_list and obj_list post srcu_synchronize()? I think that removing mark->obj_list was optimization of good faith and not because it really hurts the system's memory usage that much?

Quoted 7 lines

Furthermore I don't like the scheme of ->handle_event returning -EAGAIN and then dropping the srcu lock - I'd rather have some helpers provided by the generic fsnotify code to drop srcu lock. That needs some propagation of information inside the ->handle_event and then the helper but that's IMO cleaner. Anyway, that is just a technical detail.

Yeh, that's just the minimal LOC implementation. I figured the implementation may be rejected for more profound flaws. Although personally,
I do like the so called O_NONBLOCKING semantics here.
I debated with myself if I should use EAGAIN or EWOULDBLOCK
for sake of readability.

Quoted 6 lines

I have some ideas how to fix up issues with my refcounting approach - refcount would pin marks not only in memory but also in lists but I have yet to see whether that works out sensibly (it would mean that dropping mark reference would then need to take group->mark_mutex and that may cause lock ordering issues).

It sounds like a chicken and egg problem, but I suppose you don't mean the actual mark refcount, but a secondary "pinned refcount"?

Anyway, if you have something ready, my test setup is already in place..

Cheers,
Amir.

maintainer
#5 lore raw

Re: [RFC][PATCH 2/2] fsnotify: handle permission events without holding fsnotify_mark_srcu[0]

Quoted 28 lines · Amir Goldstein
Quoted 20 lines · Jan Kara
Quoted 12 lines · Amir Goldstein

Handling fanotify events does not entail dereferencing fsnotify_mark beyond the point of fanotify_should_send_event().

For the case of permission events, which may block indefinitely, return -EAGAIN and then fsnotify() will call handle_event() again without a reference to the mark.

Without a reference to the mark, there is no need to call handle_event() under fsnotify_mark_srcu[0] read side lock, so we drop fsnotify_mark_srcu[0] while handling the event and grab it back before continuing to the next mark.

Quoted 6 lines

After this change, a blocking permission event will no longer block closing of any file descriptors of 0 priority groups, i.e: inotify and fanotify groups of class FAN_CLASS_NOTIF.

Quoted 6 lines

Well, this has a similar problem as my attempt to fix the issue. The current mark can get removed from the mark list while waiting for userspace response. ->next pointer is still valid at that moment but ->next->pprev already points to mark preceding us (that's how rcu lists work). When ->next mark then gets removed from the list and destroyed (it may be protected by the second srcu), our ->next pointer points to freed memory.

Quoted 6 lines

Oh! I missed the fact that the SRCU does not protect mark->obj_list. Can resurrecting mark->free_list buy us anything?
Perhaps keep the marks on obj_list without FLAG_ATTACHED
and then remove marks from both free_list and obj_list post srcu_synchronize()? I think that removing mark->obj_list was optimization of good faith and not because it really hurts the system's memory usage that much?

You have to be really careful here. Minimally you'd need then another srcu_synchronize() call after removing mark from the list and before freeing the mark so that you are sure no process iterating the list can have a reference to a mark being freed but even then it needs careful checking whether it would work. The joy of lockless algorithms...

Quoted 8 lines
Quoted 5 lines

I have some ideas how to fix up issues with my refcounting approach - refcount would pin marks not only in memory but also in lists but I have yet to see whether that works out sensibly (it would mean that dropping mark reference would then need to take group->mark_mutex and that may cause lock ordering issues).

It sounds like a chicken and egg problem, but I suppose you don't mean the actual mark refcount, but a secondary "pinned refcount"?

No, I mean the original refcount. I have patches that already postpone part of the mark cleanup to happen only once the refcount drops to 0.

Quoted 1 line

Anyway, if you have something ready, my test setup is already in place..

Thanks for an offer. I don't have anything yet, hopefully later today or tomorrow.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR