ceph: if we are blacklisted, __do_request returns directly
PATCH v3 1/1 ceph: if we are blacklisted, __do_request returns directly
If we mount cephfs by the recover_session option,__do_request can return directly until the client automatically reconnects.
--- fs/ceph/mds_client.c | 6 ++++++ 1 file changed, 6 insertions(+)
fs/ceph/mds_client.c +6 -0
@@ -2708,6 +2708,12 @@ static void __do_request(struct ceph_mds_client *mdsc,put_request_session (req); +if (mdsc->fsc->blacklisted && +ceph_test_mount_opt (mdsc->fsc,CLEANRECOVER )) { + err = -EBLACKLISTED ; + goto finish; + } + mds =__choose_mds (mdsc, req, &random);if (mds <0 ||ceph_mdsmap_get_state (mdsc->mdsmap, mds) <CEPH_MDS_STATE_ACTIVE ) {
Signature · 1 line
2.24.2 (Apple Git-127)
Quoted 13 lines · Yanhu Cao
If we mount cephfs by the
recover_sessionoption,__do_requestcan return directly until the client automatically reconnects.Signed-off-by Yanhu Cao <[email protected]>--- fs/ceph/mds_client.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/ceph/mds_client.c +0
-0
fs/ceph/mds_client.c static void __do_request(struct ceph_mds_client *mdsc, · line 2716
Quoted 9 lines
fs/ceph/mds_client.c +6
-0@@ -2708,6 +2708,12 @@ static void __do_request(struct ceph_mds_client *mdsc,put_request_session (req); +if (mdsc->fsc->blacklisted && +ceph_test_mount_opt (mdsc->fsc,CLEANRECOVER )) { + err = -EBLACKLISTED ; + goto finish; + } +
Why check for CLEANRECOVER? If we're mounted with recover_session=no wouldn't we want to do the same thing here?
Either way, it's still blacklisted. The only difference is that it won't attempt to automatically recover the session that way.
Quoted 3 lines
fs/ceph/mds_client.c +0
-0mds =__choose_mds (mdsc, req, &random);if (mds <0 ||ceph_mdsmap_get_state (mdsc->mdsmap, mds) <CEPH_MDS_STATE_ACTIVE ) {
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 28 lines · Jeff Layton
Quoted 13 lines · Yanhu Cao
If we mount cephfs by the
recover_sessionoption,__do_requestcan return directly until the client automatically reconnects.Signed-off-by Yanhu Cao <[email protected]>--- fs/ceph/mds_client.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/ceph/mds_client.c +0
-0fs/ceph/mds_client.c static void __do_request(struct ceph_mds_client *mdsc, · line 2716
Quoted 9 lines
fs/ceph/mds_client.c +6
-0@@ -2708,6 +2708,12 @@ static void __do_request(struct ceph_mds_client *mdsc,put_request_session (req); +if (mdsc->fsc->blacklisted && +ceph_test_mount_opt (mdsc->fsc,CLEANRECOVER )) { + err = -EBLACKLISTED ; + goto finish; + } +Why check for CLEANRECOVER? If we're mounted with recover_session=no wouldn't we want to do the same thing here?
Quoted 2 lines
Either way, it's still blacklisted. The only difference is that it won't attempt to automatically recover the session that way.
I think mds will clear the blacklist. In addition to loading cephfs via recover_session=clean, I didn't find a location where fsc->blacklisted is set to false. If the client has been blacklisted, should it always be blacklisted (fsc->blacklisted=true)? Or is there another way to set fsc->blacklised to false?
Quoted 8 lines
fs/ceph/mds_client.c static void __do_request(struct ceph_mds_client *mdsc, · line 2719
Quoted 5 lines
fs/ceph/mds_client.c +0
-0mds =__choose_mds (mdsc, req, &random);if (mds <0 ||ceph_mdsmap_get_state (mdsc->mdsmap, mds) <CEPH_MDS_STATE_ACTIVE ) {--
Jeff Layton <[email protected]>
Quoted 28 lines · Yanhu Cao
Quoted 27 lines · Jeff Layton
Quoted 13 lines · Yanhu Cao
If we mount cephfs by the
recover_sessionoption,__do_requestcan return directly until the client automatically reconnects.Signed-off-by Yanhu Cao <[email protected]>--- fs/ceph/mds_client.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/ceph/mds_client.c +0
-0fs/ceph/mds_client.c static void __do_request(struct ceph_mds_client *mdsc, · line 2716
Quoted 9 lines
fs/ceph/mds_client.c +6
-0@@ -2708,6 +2708,12 @@ static void __do_request(struct ceph_mds_client *mdsc,put_request_session (req); +if (mdsc->fsc->blacklisted && +ceph_test_mount_opt (mdsc->fsc,CLEANRECOVER )) { + err = -EBLACKLISTED ; + goto finish; + } +Why check for CLEANRECOVER? If we're mounted with recover_session=no wouldn't we want to do the same thing here?
Quoted 9 lines
Quoted 2 lines
Either way, it's still blacklisted. The only difference is that it won't attempt to automatically recover the session that way.
I think mds will clear the blacklist. In addition to loading cephfs via recover_session=clean, I didn't find a location where
fsc->blacklistedis set to false. If the client has been blacklisted, should it always be blacklisted (fsc->blacklisted=true)? Or is there another way to setfsc->blacklisedto false?
Basically, this patch is just changing it so that when the client is blacklisted and the mount is done with recover_session=clean, we'll shortcut the rest of the __do_request and just return -EBLACKLISTED.
My question is: why do we need to test for recover_session=clean here?
If the client _knows_ that it is blacklisted, why would it want to continue with __do_request in the recover_session=no case? Would it make more sense to always return early in __do_request when the client is blacklisted?
Quoted 7 lines
Quoted 7 lines
fs/ceph/mds_client.c static void __do_request(struct ceph_mds_client *mdsc, · line 2719
Quoted 4 lines
fs/ceph/mds_client.c +0
-0mds =__choose_mds (mdsc, req, &random);if (mds <0 ||ceph_mdsmap_get_state (mdsc->mdsmap, mds) <CEPH_MDS_STATE_ACTIVE ) {--
Jeff Layton <[email protected]>
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 40 lines · Jeff Layton
Quoted 28 lines · Yanhu Cao
Quoted 27 lines · Jeff Layton
Quoted 13 lines · Yanhu Cao
If we mount cephfs by the
recover_sessionoption,__do_requestcan return directly until the client automatically reconnects.Signed-off-by Yanhu Cao <[email protected]>--- fs/ceph/mds_client.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/ceph/mds_client.c +0
-0fs/ceph/mds_client.c static void __do_request(struct ceph_mds_client *mdsc, · line 2716
Quoted 9 lines
fs/ceph/mds_client.c +6
-0@@ -2708,6 +2708,12 @@ static void __do_request(struct ceph_mds_client *mdsc,put_request_session (req); +if (mdsc->fsc->blacklisted && +ceph_test_mount_opt (mdsc->fsc,CLEANRECOVER )) { + err = -EBLACKLISTED ; + goto finish; + } +Why check for CLEANRECOVER? If we're mounted with recover_session=no wouldn't we want to do the same thing here?
Quoted 9 lines
Quoted 2 lines
Either way, it's still blacklisted. The only difference is that it won't attempt to automatically recover the session that way.
I think mds will clear the blacklist. In addition to loading cephfs via recover_session=clean, I didn't find a location where
fsc->blacklistedis set to false. If the client has been blacklisted, should it always be blacklisted (fsc->blacklisted=true)? Or is there another way to setfsc->blacklisedto false?
Quoted 5 lines
Basically, this patch is just changing it so that when the client is blacklisted and the mount is done with recover_session=clean, we'll shortcut the rest of the
__do_requestand just return-EBLACKLISTED.My question is: why do we need to test for recover_session=clean here?
I thought that fsc->blacklisted is related to recovery_session=clean. If we test it, the client can do the rest of __do_request. It seems useless now because kcephfs cannot resume the session like ceph-fuse when mds cleared the blacklist.
Quoted 5 lines
If the client _knows_ that it is blacklisted, why would it want to continue with
__do_requestin the recover_session=no case? Would it make more sense to always return early in__do_requestwhen the client is blacklisted?
Makes sense. if there is no problem. I will patch the next commit and return -EBLACKLISTED only when fsc->blacklisted=true.
Quoted 13 lines
Quoted 9 lines
Quoted 9 lines
fs/ceph/mds_client.c static void __do_request(struct ceph_mds_client *mdsc, · line 2719
Quoted 6 lines
fs/ceph/mds_client.c +0
-0mds =__choose_mds (mdsc, req, &random);if (mds <0 ||ceph_mdsmap_get_state (mdsc->mdsmap, mds) <CEPH_MDS_STATE_ACTIVE ) {--
Jeff Layton <[email protected]>--
Jeff Layton <[email protected]>
Quoted 44 lines · Yanhu Cao
Quoted 43 lines · Jeff Layton
Quoted 28 lines · Yanhu Cao
Quoted 27 lines · Jeff Layton
Quoted 13 lines · Yanhu Cao
If we mount cephfs by the
recover_sessionoption,__do_requestcan return directly until the client automatically reconnects.Signed-off-by Yanhu Cao <[email protected]>--- fs/ceph/mds_client.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/ceph/mds_client.c +0
-0fs/ceph/mds_client.c static void __do_request(struct ceph_mds_client *mdsc, · line 2716
Quoted 9 lines
fs/ceph/mds_client.c +6
-0@@ -2708,6 +2708,12 @@ static void __do_request(struct ceph_mds_client *mdsc,put_request_session (req); +if (mdsc->fsc->blacklisted && +ceph_test_mount_opt (mdsc->fsc,CLEANRECOVER )) { + err = -EBLACKLISTED ; + goto finish; + } +Why check for CLEANRECOVER? If we're mounted with recover_session=no wouldn't we want to do the same thing here?
Quoted 9 lines
Quoted 2 lines
Either way, it's still blacklisted. The only difference is that it won't attempt to automatically recover the session that way.
I think mds will clear the blacklist. In addition to loading cephfs via recover_session=clean, I didn't find a location where
fsc->blacklistedis set to false. If the client has been blacklisted, should it always be blacklisted (fsc->blacklisted=true)? Or is there another way to setfsc->blacklisedto false?Basically, this patch is just changing it so that when the client is blacklisted and the mount is done with recover_session=clean, we'll shortcut the rest of the
__do_requestand just return-EBLACKLISTED.
Quoted 7 lines
Quoted 1 line
My question is: why do we need to test for recover_session=clean here?
I thought that
fsc->blacklistedis related to recovery_session=clean. If we test it, the client can do the rest of__do_request. It seems useless now because kcephfs cannot resume the session like ceph-fuse when mds cleared the blacklist.
fsc->blacklisted just indicates that the client has detected that it has been blacklisted. With recover_session=clean it can reconnect and continue on (with some limitations). See:
https://ceph.io/community/automatic-cephfs-recovery-after-blacklisting/
Quoted 8 lines
Quoted 4 lines
If the client _knows_ that it is blacklisted, why would it want to continue with
__do_requestin the recover_session=no case? Would it make more sense to always return early in__do_requestwhen the client is blacklisted?Makes sense. if there is no problem. I will patch the next commit and return
-EBLACKLISTEDonly when fsc->blacklisted=true.
Sure. To be clear, I don't see this as a bug, but rather just an optimization. If the client is blacklisted then we don't really need to do all of the work or attempt to send the request until that's been cleared. That's the case regardless of the recover_session= option.
Quoted 11 lines
Quoted 11 lines
Quoted 7 lines
Quoted 7 lines
fs/ceph/mds_client.c static void __do_request(struct ceph_mds_client *mdsc, · line 2719
Quoted 4 lines
fs/ceph/mds_client.c +0
-0mds =__choose_mds (mdsc, req, &random);if (mds <0 ||ceph_mdsmap_get_state (mdsc->mdsmap, mds) <CEPH_MDS_STATE_ACTIVE ) {--
Jeff Layton <[email protected]>--
Jeff Layton <[email protected]>
Signature · 1 line
Jeff Layton <[email protected]>