Skip to contentSkip to conversation

ceph: defer stopping the mdsc delayed_work

PATCH Superseded [email protected] 2 participants 5 messages Jul 24, 2023

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 ceph: defer stopping the mdsc delayed_work

From: Xiubo Li <[email protected]>

Flushing the dirty buffer may take a long time if the Rados is overloaded or if there is network issue. So we should ping the MDSs periodically to keep alive, else the MDS will blocklist the kclient.

Cc: [email protected]
Cc: Venky Shankar <[email protected]>
URL: https://tracker.ceph.com/issues/61843

---

V2:
- Fixed a typo in commit comment, thanks Milind.

 fs/ceph/mds_client.c | 2 +-
 fs/ceph/mds_client.h | 3 ++-
 fs/ceph/super.c      | 7 ++++---
 3 files changed, 7 insertions(+), 5 deletions(-)
fs/ceph/mds_client.c +1 -1
index 65230ebefd51..70987b3c198a 100644
@@ -5192,7 +5192,7 @@ static void delayed_work(struct work_struct *work)
 
 	doutc(mdsc->fsc->client, "mdsc delayed_work\n");
 
-	if (mdsc->stopping)
+	if (mdsc->stopping >= CEPH_MDSC_STOPPING_FLUSHED)
 		return;
 
 	mutex_lock(&mdsc->mutex);
fs/ceph/mds_client.h +2 -1
index 5d02c8c582fd..befbd384428e 100644
@@ -400,7 +400,8 @@ struct cap_wait {
 
 enum {
 	CEPH_MDSC_STOPPING_BEGIN = 1,
-	CEPH_MDSC_STOPPING_FLUSHED = 2,
+	CEPH_MDSC_STOPPING_FLUSHING = 2,
+	CEPH_MDSC_STOPPING_FLUSHED = 3,
 };
 
 /*
fs/ceph/super.c +4 -3
index 0a370852cf02..49fd17fbba9f 100644
@@ -1477,7 +1477,7 @@ static int ceph_init_fs_context(struct fs_context *fc)
 static bool __inc_stopping_blocker(struct ceph_mds_client *mdsc)
 {
 	spin_lock(&mdsc->stopping_lock);
-	if (mdsc->stopping >= CEPH_MDSC_STOPPING_FLUSHED) {
+	if (mdsc->stopping >= CEPH_MDSC_STOPPING_FLUSHING) {
 		spin_unlock(&mdsc->stopping_lock);
 		return false;
 	}
@@ -1490,7 +1490,7 @@ static void __dec_stopping_blocker(struct ceph_mds_client *mdsc)
 {
 	spin_lock(&mdsc->stopping_lock);
 	if (!atomic_dec_return(&mdsc->stopping_blockers) &&
-	    mdsc->stopping >= CEPH_MDSC_STOPPING_FLUSHED)
+	    mdsc->stopping >= CEPH_MDSC_STOPPING_FLUSHING)
 		complete_all(&mdsc->stopping_waiter);
 	spin_unlock(&mdsc->stopping_lock);
 }
@@ -1551,7 +1551,7 @@ static void ceph_kill_sb(struct super_block *s)
 	sync_filesystem(s);
 
 	spin_lock(&mdsc->stopping_lock);
-	mdsc->stopping = CEPH_MDSC_STOPPING_FLUSHED;
+	mdsc->stopping = CEPH_MDSC_STOPPING_FLUSHING;
 	wait = !!atomic_read(&mdsc->stopping_blockers);
 	spin_unlock(&mdsc->stopping_lock);
 
@@ -1565,6 +1565,7 @@ static void ceph_kill_sb(struct super_block *s)
 			pr_warn_client(cl, "umount was killed, %ld\n", timeleft);
 	}
 
+	mdsc->stopping = CEPH_MDSC_STOPPING_FLUSHED;
 	kill_anon_super(s);
 
 	fsc->client->extra_mon_dispatch = NULL;
Signature · 1 line
2.40.1
maintainer
#1 lore raw
Quoted 9 lines · On Mon, Jul 24, 2023 at 10:44 AM <[email protected]> wrote:

From: Xiubo Li <[email protected]>

Flushing the dirty buffer may take a long time if the Rados is overloaded or if there is network issue. So we should ping the MDSs periodically to keep alive, else the MDS will blocklist the kclient.

Hi Xiubo,

The stable tag doesn't make sense here as this commit enhances commit 2789c08342f7 ("ceph: drop the messages from MDS when unmounting") which isn't upstream. It should probably just be folded there.

Thanks,

Ilya

author
#2 lore raw
Quoted 8 lines · Ilya Dryomov
Quoted 7 lines · On Mon, Jul 24, 2023 at 10:44 AM <[email protected]> wrote:

From: Xiubo Li <[email protected]>

Flushing the dirty buffer may take a long time if the Rados is overloaded or if there is network issue. So we should ping the MDSs periodically to keep alive, else the MDS will blocklist the kclient.

Quoted 6 lines
Quoted 1 line

Hi Xiubo,

The stable tag doesn't make sense here as this commit enhances commit 2789c08342f7 ("ceph: drop the messages from MDS when unmounting") which isn't upstream. It should probably just be folded there.

No, Ilya. This is not an enhancement for commit 2789c08342f7.

They are for different issues here. This patch just based on that. We can apply this first and then I can rebase the testing branch.

Thanks

- Xiubo

Quoted 4 lines

Thanks,

Ilya

maintainer
#3 lore raw
Quoted 14 lines · Xiubo Li
Quoted 11 lines · Ilya Dryomov
Quoted 8 lines · On Mon, Jul 24, 2023 at 10:44 AM <[email protected]> wrote:

From: Xiubo Li <[email protected]>

Flushing the dirty buffer may take a long time if the Rados is overloaded or if there is network issue. So we should ping the MDSs periodically to keep alive, else the MDS will blocklist the kclient.

Hi Xiubo,

Quoted 8 lines
Quoted 3 lines

The stable tag doesn't make sense here as this commit enhances commit 2789c08342f7 ("ceph: drop the messages from MDS when unmounting") which isn't upstream. It should probably just be folded there.

No, Ilya. This is not an enhancement for commit 2789c08342f7.

They are for different issues here. This patch just based on that. We can apply this first and then I can rebase the testing branch.

Ah, thanks for letting me know. Please go ahead and structure it that way in the testing branch. As it is, it conflicts heavily as the enum that it adds a new member to was added in commit 2789c08342f7.

Ilya

author
#4 lore raw
Quoted 17 lines · Ilya Dryomov
Quoted 16 lines · Xiubo Li
Quoted 14 lines · Ilya Dryomov
Quoted 8 lines · On Mon, Jul 24, 2023 at 10:44 AM <[email protected]> wrote:

From: Xiubo Li <[email protected]>

Flushing the dirty buffer may take a long time if the Rados is overloaded or if there is network issue. So we should ping the MDSs periodically to keep alive, else the MDS will blocklist the kclient.

Hi Xiubo,

The stable tag doesn't make sense here as this commit enhances commit 2789c08342f7 ("ceph: drop the messages from MDS when unmounting") which isn't upstream. It should probably just be folded there.

Quoted 7 lines
Quoted 4 lines

No, Ilya. This is not an enhancement for commit 2789c08342f7.

They are for different issues here. This patch just based on that. We can apply this first and then I can rebase the testing branch.

Ah, thanks for letting me know. Please go ahead and structure it that way in the testing branch. As it is, it conflicts heavily as the enum that it adds a new member to was added in commit 2789c08342f7.

Sure. Thanks

Quoted 3 lines

Ilya