fd/ceph/mds_client: add per-request lock for completion
PATCH 1/1 fd/ceph/mds_client: add per-request lock for completion
ceph_mdsc_wait_request() took ceph_mds_client.mutex only to resolve a request's terminal state after the wait returned: either a reply had arrived and handle_reply() had set r_reply/CEPH_MDS_R_GOT_RESULT, or there was an error and the waiter had to abort the request by setting r_err/CEPH_MDS_R_ABORTED so a late reply is discarded. It never touched any ceph_mds_client fields; it used the big mutex purely as the serialization point against the matching section at the tail of handle_reply(), and (indirectly) against __do_request() and handle_forward().
ceph_mds_client.mutex is a coarse lock, also covering the request tree, tid counters, the session table and the mdsmap. Making the waiter take it means request completion contends with request submission (ceph_mdsc_submit_request() holds it across __register_request() and __do_request()) even though the two only share a handful of per-request fields.
This patch adds a per-request spinlock that guards a request's terminal state: r_err, r_reply, CEPH_MDS_R_GOT_RESULT and CEPH_MDS_R_ABORTED.
Lock ordering is ceph_mds_client.mutex, r_fill_mutex, r_completion_lock; the spinlock is always innermost and no sleeping lock is taken while it is held.
The two BUG_ON()s in handle_forward()'s resend branch are removed. They asserted that a request being resent had neither errored nor completed, an invariant that held only because the client mutex serialized the resend against the waiter's abort. Now that the waiter aborts under r_completion_lock without the client mutex, a forward can race an abort; this is harmless because __do_request() re-checks the terminal state under r_completion_lock and unregisters the request instead of resending it.
This weakens one invariant: previously, the abort path held ceph_mds_client.mutex, so it was mutually exclusive with __do_request() and a request could never be sent once CEPH_MDS_R_ABORTED was set. With the abort now protected only by r_completion_lock, the bit can be set concurrently with a resend-path __do_request() (kick_requests(), __wake_requests(), handle_forward()) after that function's one-time entry check, so an aborted request may still be sent or queued. This is harmless:ABORTED is re-checked at every consumer: handle_reply() discards the reply, ceph_fill_trace() skips the dcache work that depends on locks held by the (now returned) caller, and the next __do_request() unregisters it. This matches the existing property that an interrupted request may still execute on the MDS.
--- fs/ceph/locks.c | 21 ++++++---- fs/ceph/mds_client.c | 99 ++++++++++++++++++++++++++++++++++---------- fs/ceph/mds_client.h | 10 +++++ 3 files changed, 102 insertions(+), 28 deletions(-)
fs/ceph/locks.c +14 -7
@@ -177,25 +177,32 @@ static int ceph_lock_wait_for_completion(struct ceph_mds_client *mdsc,doutc (cl,"request %llu was interrupted\n" , req->r_tid); +/* +* mdsc->mutex guards the r_session check below. The terminal-state +* arbitration against a concurrent reply/forward handler is done under +* r_completion_lock (the same lock ceph_mdsc_wait_request() uses), +* since those handlers no longer serialize on mdsc->mutex. +* r_fill_mutex additionally fences ceph_fill_trace() / +* ceph_readdir_prepopulate(), which consult CEPH_MDS_R_ABORTED while +* relying on locks (dir mutex) held by our caller. +* Lock ordering: mdsc->mutex -> r_fill_mutex -> r_completion_lock. +*/ mutex_lock (&mdsc->mutex); +mutex_lock (&req->r_fill_mutex); +spin_lock (&req->r_completion_lock);if (test_bit (CEPH_MDS_R_GOT_RESULT , &req->r_req_flags)) { err =0 ; }else { -/* -* ensure we aren't running concurrently with -* ceph_fill_trace or ceph_readdir_prepopulate, which -* rely on locks (dir mutex) held by our caller. -*/ -mutex_lock (&req->r_fill_mutex); req->r_err = err;set_bit (CEPH_MDS_R_ABORTED , &req->r_req_flags); -mutex_unlock (&req->r_fill_mutex);if (!req->r_session) {// haven't sent the request err =0 ; } } +spin_unlock (&req->r_completion_lock); +mutex_unlock (&req->r_fill_mutex);mutex_unlock (&mdsc->mutex);if (!err)return 0 ;
fs/ceph/mds_client.c +78 -21
182 lines not on this page · raw message
fs/ceph/mds_client.h +10 -0
@@ -394,6 +394,16 @@ struct ceph_mds_request {struct ceph_msg *r_reply;struct ceph_mds_reply_info_parsed r_reply_info;int r_err; + +/* +* Serializes the request's terminal state (r_err, r_reply, +* CEPH_MDS_R_GOT_RESULT and CEPH_MDS_R_ABORTED) between the reply and +* forward handlers, __do_request() and ceph_mdsc_wait_request(). +* +* Lock ordering: mdsc->mutex -> r_fill_mutex -> r_completion_lock. +*/ +spinlock_t r_completion_lock; +u32 r_readdir_offset;struct page *r_locked_page;
Signature · 1 line
2.47.3