fsnotify: Fix fsnotify_mark_connector race
Subsystems (2)
PATCH 1/1 fsnotify: Fix fsnotify_mark_connector race
fsnotify() acquires a reference to a fsnotify_mark_connector through the SRCU-protected pointer to_tell->i_fsnotify_marks. However, it appears that no precautions are taken in fsnotify_put_mark() to ensure that fsnotify() drops its reference to this fsnotify_mark_connector before assigning a value to its 'destroy_next' field. This can result in fsnotify_put_mark() assigning a value to a connector's 'destroy_next' field right before fsnotify() tries to traverse the linked list referenced by the connector's 'list' field. Since these two fields are members of the same union, this behavior results in a kernel panic.
This issue is resolved by calling synchronize_srcu() before updating the connector's 'destroy_next' field in fsnotify_put_mark(). Since the relevant section of fsnotify() occurs in a SRCU read-side critical section, this will force fsnotify_put_mark() to wait for fsnotify() to finish operating on the connector before updating its 'destroy_next' field. Since fsnotify_put_mark() removes references to the connector before assigning its 'destroy_next' field, it shouldn't be possible for another thread to acquire a reference to the connector while fsnotify_put_mark() waits for fsnotify() to finish its work.
The offending behavior here is extremely unlikely; since fsnotify_put_mark() removes references to a connector (specifically, it ensures that the connector is unreachable from the inode it was formerly attached to) before updating its 'destroy_next' field, a sizeable chunk of code in fsnotify_put_mark() has to execute in the short window between when fsnotify() acquires the connector reference and saves the value of its 'list' field. On the HEAD kernel, I've only been able to reproduce this by inserting a udelay(1) in fsnotify(). However, I've been able to reproduce this issue without inserting a udelay(1) anywhere on older unmodified release kernels, so I believe it's worth fixing at HEAD.
Fixes bug 199437: https://bugzilla.kernel.org/show_bug.cgi?id=199437
--- fs/notify/mark.c | 1 + 1 file changed, 1 insertion(+)
fs/notify/mark.c +1 -0
@@ -227,6 +227,7 @@ void fsnotify_put_mark(struct fsnotify_mark *mark)iput (inode);if (free_conn) { +synchronize_srcu (&fsnotify_mark_srcu);spin_lock (&destroy_lock); conn->destroy_next = connector_destroy_list; connector_destroy_list = conn;
Signature · 1 line
2.17.0.484.g0c8726318c-goog
Jan Kara
maintainerQuoted 10 lines · Robert Kolchmeyer
fsnotify()acquires a reference to afsnotify_mark_connectorthrough the SRCU-protected pointerto_tell->i_fsnotify_marks. However, it appears that no precautions are taken infsnotify_put_mark()to ensure thatfsnotify()drops its reference to thisfsnotify_mark_connectorbefore assigning a value to its 'destroy_next' field. This can result infsnotify_put_mark()assigning a value to a connector's 'destroy_next' field right beforefsnotify()tries to traverse the linked list referenced by the connector's 'list' field. Since these two fields are members of the same union, this behavior results in a kernel panic.
Ahh, I was reading this code just two days ago trying to find some issues (due to reports of softlockups in fsnotify()) but this didn't occur to me. Great spotting!
Quoted 10 lines
This issue is resolved by calling
synchronize_srcu()before updating the connector's 'destroy_next' field infsnotify_put_mark(). Since the relevant section offsnotify()occurs in a SRCU read-side critical section, this will forcefsnotify_put_mark()to wait forfsnotify()to finish operating on the connector before updating its 'destroy_next' field. Sincefsnotify_put_mark()removes references to the connector before assigning its 'destroy_next' field, it shouldn't be possible for another thread to acquire a reference to the connector whilefsnotify_put_mark()waits forfsnotify()to finish its work.
Quoted 11 lines
The offending behavior here is extremely unlikely; since
fsnotify_put_mark()removes references to a connector (specifically, it ensures that the connector is unreachable from the inode it was formerly attached to) before updating its 'destroy_next' field, a sizeable chunk of code infsnotify_put_mark()has to execute in the short window between whenfsnotify()acquires the connector reference and saves the value of its 'list' field. On the HEAD kernel, I've only been able to reproduce this by inserting a udelay(1) infsnotify(). However, I've been able to reproduce this issue without inserting a udelay(1) anywhere on older unmodified release kernels, so I believe it's worth fixing at HEAD.
Oh, the bug definitely needs fixing. I somewhat dislike synchronize_srcu() when destroying the connector though. The whole point of connector_destroy_list is to batch synchronize_srcu() calls and move them out of the relative fast path. So what I'd do instead is to move 'destroy_next' from a union with 'list' to a union with 'inode' and 'mnt'. Those are not safe to access under srcu anyway - you have to hold conn->lock to get them and use conn->flags to determine the type under the same lock. So in that union we are actually safe to use the space for anything once conn->flags are cleared by fsnotify_detach_connector_from_object(). Will you send a patch for that so that you get a proper credit for all the debugging you did? :) Thanks!
Honza
Quoted 23 lines
Fixes bug 199437: https://bugzilla.kernel.org/show_bug.cgi?id=199437
Fixes 08991e83b7286635167bab40927665a90fb00d81Signed-off-by Robert Kolchmeyer <[email protected]>--- fs/notify/mark.c | 1 + 1 file changed, 1 insertion(+)fs/notify/mark.c +1
-0@@ -227,6 +227,7 @@ void fsnotify_put_mark(struct fsnotify_mark *mark)iput (inode);if (free_conn) { +synchronize_srcu (&fsnotify_mark_srcu);spin_lock (&destroy_lock); conn->destroy_next = connector_destroy_list; connector_destroy_list = conn;Signature · 1 line
2.17.0.484.g0c8726318c-goog
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Sounds great. I'll send out that patch shortly. Thanks!
Quoted 78 lines · Jan Kara
Quoted 10 lines · Robert Kolchmeyer
fsnotify()acquires a reference to afsnotify_mark_connectorthrough the SRCU-protected pointerto_tell->i_fsnotify_marks. However, it appears that no precautions are taken infsnotify_put_mark()to ensure thatfsnotify()drops its reference to thisfsnotify_mark_connectorbefore assigning a value to its 'destroy_next' field. This can result infsnotify_put_mark()assigning a value to a connector's 'destroy_next' field right beforefsnotify()tries to traverse the linked list referenced by the connector's 'list' field. Since these two fields are members of the same union, this behavior results in a kernel panic.Ahh, I was reading this code just two days ago trying to find some issues (due to reports of softlockups in
fsnotify()) but this didn't occur to me. Great spotting!Quoted 10 lines
This issue is resolved by calling
synchronize_srcu()before updating the connector's 'destroy_next' field infsnotify_put_mark(). Since the relevant section offsnotify()occurs in a SRCU read-side critical section, this will forcefsnotify_put_mark()to wait forfsnotify()to finish operating on the connector before updating its 'destroy_next' field. Sincefsnotify_put_mark()removes references to the connector before assigning its 'destroy_next' field, it shouldn't be possible for another thread to acquire a reference to the connector whilefsnotify_put_mark()waits forfsnotify()to finish its work.Quoted 11 lines
The offending behavior here is extremely unlikely; since
fsnotify_put_mark()removes references to a connector (specifically, it ensures that the connector is unreachable from the inode it was formerly attached to) before updating its 'destroy_next' field, a sizeable chunk of code infsnotify_put_mark()has to execute in the short window between whenfsnotify()acquires the connector reference and saves the value of its 'list' field. On the HEAD kernel, I've only been able to reproduce this by inserting a udelay(1) infsnotify(). However, I've been able to reproduce this issue without inserting a udelay(1) anywhere on older unmodified release kernels, so I believe it's worth fixing at HEAD.Oh, the bug definitely needs fixing. I somewhat dislike
synchronize_srcu()when destroying the connector though. The whole point ofconnector_destroy_listis to batchsynchronize_srcu()calls and move them out of the relative fast path. So what I'd do instead is to move 'destroy_next' from a union with 'list' to a union with 'inode' and 'mnt'. Those are not safe to access under srcu anyway - you have to holdconn->lockto get them and useconn->flagsto determine the type under the same lock. So in that union we are actually safe to use the space for anything onceconn->flagsare cleared byfsnotify_detach_connector_from_object(). Will you send a patch for that so that you get a proper credit for all the debugging you did? :) Thanks!Honza
Quoted 21 lines
Fixes bug 199437: https://bugzilla.kernel.org/show_bug.cgi?id=199437
Fixes 08991e83b7286635167bab40927665a90fb00d81Signed-off-by Robert Kolchmeyer <[email protected]>--- fs/notify/mark.c | 1 + 1 file changed, 1 insertion(+)fs/notify/mark.c +1
-1@@ -227,6 +227,7 @@ void fsnotify_put_mark(struct fsnotify_mark *mark)iput (inode);if (free_conn) { +synchronize_srcu (&fsnotify_mark_srcu);spin_lock (&destroy_lock); conn->destroy_next = connector_destroy_list; connector_destroy_list = conn; --Quoted 2 lines
2.17.0.484.g0c8726318c-goog
--
Jan Kara <[email protected]>
SUSE Labs, CR
PATCH v2 1/1 fsnotify: Fix fsnotify_mark_connector race
In reply to Robert Kolchmeyerfsnotify() acquires a reference to a fsnotify_mark_connector through the SRCU-protected pointer to_tell->i_fsnotify_marks. However, it appears that no precautions are taken in fsnotify_put_mark() to ensure that fsnotify() drops its reference to this fsnotify_mark_connector before assigning a value to its 'destroy_next' field. This can result in fsnotify_put_mark() assigning a value to a connector's 'destroy_next' field right before fsnotify() tries to traverse the linked list referenced by the connector's 'list' field. Since these two fields are members of the same union, this behavior results in a kernel panic.
This issue is resolved by moving the connector's 'destroy_next' field into the object pointer union. This should work since the object pointer access is protected by both a spinlock and the value of the 'flags' field, and the 'flags' field is cleared while holding the spinlock in fsnotify_put_mark() before 'destroy_next' is updated. It shouldn't be possible for another thread to accidentally read from the object pointer after the 'destroy_next' field is updated.
The offending behavior here is extremely unlikely; since fsnotify_put_mark() removes references to a connector (specifically, it ensures that the connector is unreachable from the inode it was formerly attached to) before updating its 'destroy_next' field, a sizeable chunk of code in fsnotify_put_mark() has to execute in the short window between when fsnotify() acquires the connector reference and saves the value of its 'list' field. On the HEAD kernel, I've only been able to reproduce this by inserting a udelay(1) in fsnotify(). However, I've been able to reproduce this issue without inserting a udelay(1) anywhere on older unmodified release kernels, so I believe it's worth fixing at HEAD.
Fixes bug 199437: https://bugzilla.kernel.org/show_bug.cgi?id=199437
---
Changelog since v1:
- Solve problem by moving 'destroy_next' field into a different union instead of calling synchronize_srcu().
include/linux/fsnotify_backend.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
include/linux/fsnotify_backend.h +1 -3
@@ -217,12 +217,10 @@ struct fsnotify_mark_connector {union {/* Object pointer [lock] */ struct inode *inode;struct vfsmount *mnt; - }; -union { -struct hlist_head list;/* Used listing heads to free after srcu period expires */ struct fsnotify_mark_connector *destroy_next; }; +struct hlist_head list; };/*
Signature · 1 line
2.17.0.484.g0c8726318c-goog
Jan Kara
maintainerQuoted 31 lines · Robert Kolchmeyer
fsnotify()acquires a reference to afsnotify_mark_connectorthrough the SRCU-protected pointerto_tell->i_fsnotify_marks. However, it appears that no precautions are taken infsnotify_put_mark()to ensure thatfsnotify()drops its reference to thisfsnotify_mark_connectorbefore assigning a value to its 'destroy_next' field. This can result infsnotify_put_mark()assigning a value to a connector's 'destroy_next' field right beforefsnotify()tries to traverse the linked list referenced by the connector's 'list' field. Since these two fields are members of the same union, this behavior results in a kernel panic.This issue is resolved by moving the connector's '
destroy_next' field into the object pointer union. This should work since the object pointer access is protected by both a spinlock and the value of the 'flags' field, and the 'flags' field is cleared while holding the spinlock infsnotify_put_mark()before 'destroy_next' is updated. It shouldn't be possible for another thread to accidentally read from the object pointer after the 'destroy_next' field is updated.The offending behavior here is extremely unlikely; since
fsnotify_put_mark()removes references to a connector (specifically, it ensures that the connector is unreachable from the inode it was formerly attached to) before updating its 'destroy_next' field, a sizeable chunk of code infsnotify_put_mark()has to execute in the short window between whenfsnotify()acquires the connector reference and saves the value of its 'list' field. On the HEAD kernel, I've only been able to reproduce this by inserting a udelay(1) infsnotify(). However, I've been able to reproduce this issue without inserting a udelay(1) anywhere on older unmodified release kernels, so I believe it's worth fixing at HEAD.
Quoted 4 lines
Fixes bug 199437: https://bugzilla.kernel.org/show_bug.cgi?id=199437
Fixes 08991e83b7286635167bab40927665a90fb00d81Signed-off-by Robert Kolchmeyer <[email protected]>
Thanks! I've added the patch to my tree and will push it to Linus for -rc3.
Honza
Quoted 29 lines
---
Changelog since v1:
- Solve problem by moving 'destroy_next' field into a different union instead of callingsynchronize_srcu().include/linux/fsnotify_backend.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)include/linux/fsnotify_backend.h +1
-3@@ -217,12 +217,10 @@ struct fsnotify_mark_connector {union {/* Object pointer [lock] */ struct inode *inode;struct vfsmount *mnt; - }; -union { -struct hlist_head list;/* Used listing heads to free after srcu period expires */ struct fsnotify_mark_connector *destroy_next; }; +struct hlist_head list; };/* Signature · 1 line
2.17.0.484.g0c8726318c-goog
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR