Skip to contentSkip to conversation

fd/ceph/mds_client: add per-request lock for completion

PATCH New Max Kellermann 1 participant 1 message Jul 8 Ceph client

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fd/ceph/mds_client: add per-request lock for completion

ceph_mdsc_wait_request() took ceph_mds_client.mutex only to resolve a request's terminal state after the wait returned: either a reply had arrived and handle_reply() had set r_reply/CEPH_MDS_R_GOT_RESULT, or there was an error and the waiter had to abort the request by setting r_err/CEPH_MDS_R_ABORTED so a late reply is discarded. It never touched any ceph_mds_client fields; it used the big mutex purely as the serialization point against the matching section at the tail of handle_reply(), and (indirectly) against __do_request() and handle_forward().

ceph_mds_client.mutex is a coarse lock, also covering the request tree, tid counters, the session table and the mdsmap. Making the waiter take it means request completion contends with request submission (ceph_mdsc_submit_request() holds it across __register_request() and __do_request()) even though the two only share a handful of per-request fields.

This patch adds a per-request spinlock that guards a request's terminal state: r_err, r_reply, CEPH_MDS_R_GOT_RESULT and CEPH_MDS_R_ABORTED.

Lock ordering is ceph_mds_client.mutex, r_fill_mutex, r_completion_lock; the spinlock is always innermost and no sleeping lock is taken while it is held.

The two BUG_ON()s in handle_forward()'s resend branch are removed. They asserted that a request being resent had neither errored nor completed, an invariant that held only because the client mutex serialized the resend against the waiter's abort. Now that the waiter aborts under r_completion_lock without the client mutex, a forward can race an abort; this is harmless because __do_request() re-checks the terminal state under r_completion_lock and unregisters the request instead of resending it.

This weakens one invariant: previously, the abort path held ceph_mds_client.mutex, so it was mutually exclusive with __do_request() and a request could never be sent once CEPH_MDS_R_ABORTED was set. With the abort now protected only by r_completion_lock, the bit can be set concurrently with a resend-path __do_request() (kick_requests(), __wake_requests(), handle_forward()) after that function's one-time entry check, so an aborted request may still be sent or queued. This is harmless:
ABORTED is re-checked at every consumer: handle_reply() discards the reply, ceph_fill_trace() skips the dcache work that depends on locks held by the (now returned) caller, and the next __do_request() unregisters it. This matches the existing property that an interrupted request may still execute on the MDS.

---
 fs/ceph/locks.c      | 21 ++++++----
 fs/ceph/mds_client.c | 99 ++++++++++++++++++++++++++++++++++----------
 fs/ceph/mds_client.h | 10 +++++
 3 files changed, 102 insertions(+), 28 deletions(-)
fs/ceph/locks.c +14 -7
index 677221bd64e0..35cadcd7cb2a 100644
@@ -177,25 +177,32 @@ static int ceph_lock_wait_for_completion(struct ceph_mds_client *mdsc,
 
 	doutc(cl, "request %llu was interrupted\n", req->r_tid);
 
+	/*
+	 * mdsc->mutex guards the r_session check below.  The terminal-state
+	 * arbitration against a concurrent reply/forward handler is done under
+	 * r_completion_lock (the same lock ceph_mdsc_wait_request() uses),
+	 * since those handlers no longer serialize on mdsc->mutex.
+	 * r_fill_mutex additionally fences ceph_fill_trace() /
+	 * ceph_readdir_prepopulate(), which consult CEPH_MDS_R_ABORTED while
+	 * relying on locks (dir mutex) held by our caller.
+	 * Lock ordering: mdsc->mutex -> r_fill_mutex -> r_completion_lock.
+	 */
 	mutex_lock(&mdsc->mutex);
+	mutex_lock(&req->r_fill_mutex);
+	spin_lock(&req->r_completion_lock);
 	if (test_bit(CEPH_MDS_R_GOT_RESULT, &req->r_req_flags)) {
 		err = 0;
 	} else {
-		/*
-		 * ensure we aren't running concurrently with
-		 * ceph_fill_trace or ceph_readdir_prepopulate, which
-		 * rely on locks (dir mutex) held by our caller.
-		 */
-		mutex_lock(&req->r_fill_mutex);
 		req->r_err = err;
 		set_bit(CEPH_MDS_R_ABORTED, &req->r_req_flags);
-		mutex_unlock(&req->r_fill_mutex);
 
 		if (!req->r_session) {
 			// haven't sent the request
 			err = 0;
 		}
 	}
+	spin_unlock(&req->r_completion_lock);
+	mutex_unlock(&req->r_fill_mutex);
 	mutex_unlock(&mdsc->mutex);
 	if (!err)
 		return 0;
fs/ceph/mds_client.c +78 -21

182 lines not on this page · raw message

fs/ceph/mds_client.h +10 -0
index 731d6ad04956..449b5e28b2de 100644
@@ -394,6 +394,16 @@ struct ceph_mds_request {
 	struct ceph_msg  *r_reply;
 	struct ceph_mds_reply_info_parsed r_reply_info;
 	int r_err;
+
+	/*
+	 * Serializes the request's terminal state (r_err, r_reply,
+	 * CEPH_MDS_R_GOT_RESULT and CEPH_MDS_R_ABORTED) between the reply and
+	 * forward handlers, __do_request() and ceph_mdsc_wait_request().
+	 *
+	 * Lock ordering: mdsc->mutex -> r_fill_mutex -> r_completion_lock.
+	 */
+	spinlock_t r_completion_lock;
+
 	u32               r_readdir_offset;
 
 	struct page *r_locked_page;
Signature · 1 line
2.47.3