ceph: get snap_rwsem read lock in handle_cap_export for ceph_add_cap
PATCH 1/1 ceph: get snap_rwsem read lock in handle_cap_export for ceph_add_cap
ceph_add_cap says in its function documentation that the caller should hold the read lock on the session snap_rwsem. Furthermore, not only ceph_add_cap needs that lock, when it calls to ceph_lookup_snap_realm it eventually calls ceph_get_snap_realm which states via lockdep that snap_rwsem needs to be held. handle_cap_export calls ceph_add_cap without that mdsc->snap_rwsem held. Thus, since ceph_get_snap_realm and ceph_add_cap both need the lock, the common place to acquire that lock is inside handle_cap_export.
--- fs/ceph/caps.c | 2 ++ 1 file changed, 2 insertions(+)
fs/ceph/caps.c +2 -0
@@ -3903,8 +3903,10 @@ static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex,/* add placeholder for the export tagert */ int flag = (cap == ci->i_auth_cap) ?CEPH_CAP_FLAG_AUTH :0 ; tcap = new_cap; +down_read (&mdsc->snap_rwsem);ceph_add_cap (inode, tsession, t_cap_id, issued,0 , t_seq -1 , t_mseq, (u64)-1 , flag, &new_cap); +up_read (&mdsc->snap_rwsem);if (!list_empty (&ci->i_cap_flush_list) && ci->i_auth_cap == tcap) {
Signature · 1 line
2.35.1
Quoted 19 lines · Niels Dossche
ceph_add_capsays in its function documentation that the caller should hold the read lock on the sessionsnap_rwsem. Furthermore, not onlyceph_add_capneeds that lock, when it calls toceph_lookup_snap_realmit eventually callsceph_get_snap_realmwhich states via lockdep thatsnap_rwsemneeds to be held.handle_cap_exportcallsceph_add_capwithout thatmdsc->snap_rwsemheld. Thus, sinceceph_get_snap_realmandceph_add_capboth need the lock, the common place to acquire that lock is insidehandle_cap_export.Signed-off-by Niels Dossche <[email protected]>--- fs/ceph/caps.c | 2 ++ 1 file changed, 2 insertions(+)fs/ceph/caps.c +0
-0
fs/ceph/caps.c static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex, · line 3912
Quoted 10 lines
fs/ceph/caps.c +2
-0@@ -3903,8 +3903,10 @@ static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex,/* add placeholder for the export tagert */ int flag = (cap == ci->i_auth_cap) ?CEPH_CAP_FLAG_AUTH :0 ; tcap = new_cap; +down_read (&mdsc->snap_rwsem);ceph_add_cap (inode, tsession, t_cap_id, issued,0 , t_seq -1 , t_mseq, (u64)-1 , flag, &new_cap); +up_read (&mdsc->snap_rwsem);if (!list_empty (&ci->i_cap_flush_list) && ci->i_auth_cap == tcap) {
LGTM.
Quoted 19 lines · Niels Dossche
ceph_add_capsays in its function documentation that the caller should hold the read lock on the sessionsnap_rwsem. Furthermore, not onlyceph_add_capneeds that lock, when it calls toceph_lookup_snap_realmit eventually callsceph_get_snap_realmwhich states via lockdep thatsnap_rwsemneeds to be held.handle_cap_exportcallsceph_add_capwithout thatmdsc->snap_rwsemheld. Thus, sinceceph_get_snap_realmandceph_add_capboth need the lock, the common place to acquire that lock is insidehandle_cap_export.Signed-off-by Niels Dossche <[email protected]>--- fs/ceph/caps.c | 2 ++ 1 file changed, 2 insertions(+)fs/ceph/caps.c +0
-0
fs/ceph/caps.c static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex, · line 3912
Quoted 10 lines
fs/ceph/caps.c +2
-0@@ -3903,8 +3903,10 @@ static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex,/* add placeholder for the export tagert */ int flag = (cap == ci->i_auth_cap) ?CEPH_CAP_FLAG_AUTH :0 ; tcap = new_cap; +down_read (&mdsc->snap_rwsem);ceph_add_cap (inode, tsession, t_cap_id, issued,0 , t_seq -1 , t_mseq, (u64)-1 , flag, &new_cap); +up_read (&mdsc->snap_rwsem);if (!list_empty (&ci->i_cap_flush_list) && ci->i_auth_cap == tcap) {
Looks good. The other ceph_add_cap callsites already hold this.
Merged into ceph testing branch.
Thanks!
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 28 lines · Jeff Layton
Quoted 27 lines · Niels Dossche
ceph_add_capsays in its function documentation that the caller should hold the read lock on the sessionsnap_rwsem. Furthermore, not onlyceph_add_capneeds that lock, when it calls toceph_lookup_snap_realmit eventually callsceph_get_snap_realmwhich states via lockdep thatsnap_rwsemneeds to be held.handle_cap_exportcallsceph_add_capwithout thatmdsc->snap_rwsemheld. Thus, sinceceph_get_snap_realmandceph_add_capboth need the lock, the common place to acquire that lock is insidehandle_cap_export.Signed-off-by Niels Dossche <[email protected]>--- fs/ceph/caps.c | 2 ++ 1 file changed, 2 insertions(+)fs/ceph/caps.c +2
-0@@ -3903,8 +3903,10 @@ static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex,/* add placeholder for the export tagert */ int flag = (cap == ci->i_auth_cap) ?CEPH_CAP_FLAG_AUTH :0 ; tcap = new_cap; +down_read (&mdsc->snap_rwsem);ceph_add_cap (inode, tsession, t_cap_id, issued,0 , t_seq -1 , t_mseq, (u64)-1 , flag, &new_cap); +up_read (&mdsc->snap_rwsem);
Quoted 7 lines
fs/ceph/caps.c static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex, · line 3912
Quoted 2 lines
fs/ceph/caps.c +0
-0if (!list_empty (&ci->i_cap_flush_list) && ci->i_auth_cap == tcap) {Looks good. The other
ceph_add_capcallsites already hold this.Merged into ceph testing branch.
Oops, spoke too soon. This patch calls down_read (a potentially sleeping function) while holding the i_ceph_lock spinlock. I think you'll need to take the rwsem earlier in the function, before taking the spinlock.
Dropped from testing branch for now...
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 38 lines · Jeff Layton
Quoted 28 lines · Jeff Layton
Quoted 27 lines · Niels Dossche
ceph_add_capsays in its function documentation that the caller should hold the read lock on the sessionsnap_rwsem. Furthermore, not onlyceph_add_capneeds that lock, when it calls toceph_lookup_snap_realmit eventually callsceph_get_snap_realmwhich states via lockdep thatsnap_rwsemneeds to be held.handle_cap_exportcallsceph_add_capwithout thatmdsc->snap_rwsemheld. Thus, sinceceph_get_snap_realmandceph_add_capboth need the lock, the common place to acquire that lock is insidehandle_cap_export.Signed-off-by Niels Dossche <[email protected]>--- fs/ceph/caps.c | 2 ++ 1 file changed, 2 insertions(+)fs/ceph/caps.c +2
-0@@ -3903,8 +3903,10 @@ static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex,/* add placeholder for the export tagert */ int flag = (cap == ci->i_auth_cap) ?CEPH_CAP_FLAG_AUTH :0 ; tcap = new_cap; +down_read (&mdsc->snap_rwsem);ceph_add_cap (inode, tsession, t_cap_id, issued,0 , t_seq -1 , t_mseq, (u64)-1 , flag, &new_cap); +up_read (&mdsc->snap_rwsem);Quoted 7 lines
fs/ceph/caps.c static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex, · line 3912
Quoted 2 lines
fs/ceph/caps.c +0
-0if (!list_empty (&ci->i_cap_flush_list) && ci->i_auth_cap == tcap) {Looks good. The other
ceph_add_capcallsites already hold this.Merged into ceph testing branch.
Quoted 5 lines
Oops, spoke too soon. This patch calls
down_read(a potentially sleeping function) while holding thei_ceph_lockspinlock. I think you'll need to take the rwsem earlier in the function, before taking the spinlock.Dropped from testing branch for now...
Ah my bad. I notice that handle_cap_export is actually called with the i_ceph_lock spinlock.
I can send a v2 which acquires the down_read lock just before the i_ceph_lock spinlock is taken (i.e. just under the retry label). Does that work for you? If so, I'll send a v2.
Thanks!
Quoted 43 lines · Niels Dossche
Quoted 42 lines · Jeff Layton
Quoted 28 lines · Jeff Layton
Quoted 27 lines · Niels Dossche
ceph_add_capsays in its function documentation that the caller should hold the read lock on the sessionsnap_rwsem. Furthermore, not onlyceph_add_capneeds that lock, when it calls toceph_lookup_snap_realmit eventually callsceph_get_snap_realmwhich states via lockdep thatsnap_rwsemneeds to be held.handle_cap_exportcallsceph_add_capwithout thatmdsc->snap_rwsemheld. Thus, sinceceph_get_snap_realmandceph_add_capboth need the lock, the common place to acquire that lock is insidehandle_cap_export.Signed-off-by Niels Dossche <[email protected]>--- fs/ceph/caps.c | 2 ++ 1 file changed, 2 insertions(+)fs/ceph/caps.c +2
-0@@ -3903,8 +3903,10 @@ static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex,/* add placeholder for the export tagert */ int flag = (cap == ci->i_auth_cap) ?CEPH_CAP_FLAG_AUTH :0 ; tcap = new_cap; +down_read (&mdsc->snap_rwsem);ceph_add_cap (inode, tsession, t_cap_id, issued,0 , t_seq -1 , t_mseq, (u64)-1 , flag, &new_cap); +up_read (&mdsc->snap_rwsem);Quoted 7 lines
fs/ceph/caps.c static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex, · line 3912
Quoted 2 lines
fs/ceph/caps.c +0
-0if (!list_empty (&ci->i_cap_flush_list) && ci->i_auth_cap == tcap) {Looks good. The other
ceph_add_capcallsites already hold this.Merged into ceph testing branch.
Oops, spoke too soon. This patch calls
down_read(a potentially sleeping function) while holding thei_ceph_lockspinlock. I think you'll need to take the rwsem earlier in the function, before taking the spinlock.
Quoted 6 lines
Quoted 1 line
Dropped from testing branch for now...
Ah my bad. I notice that
handle_cap_exportis actually called with thei_ceph_lockspinlock.
I can send a v2 which acquires thedown_readlock just before thei_ceph_lockspinlock is taken (i.e. just under the retry label). Does that work for you? If so, I'll send a v2.
Thanks!
That should be fine.
Signature · 1 line
Jeff Layton <[email protected]>