ceph: cancel delayed work instead of flushing on mdsc teardown
PATCH 1/1 ceph: cancel delayed work instead of flushing on mdsc teardown
The first thing metric_delayed_work does is check mdsc->stopping, and then return immediately if it's set...which is good since we would have already torn down the metric structures at this point, otherwise.
Worse yet, it's possible that the ceph_metric_destroy call could race with the delayed_work, in which case we could end up a end up accessing destroyed percpu variables.
At this point in the mdsc teardown, the "stopping" flag has already been set, so there's no benefit to flushing the work. Just cancel it instead, and do so before we tear down the metrics structures.
--- fs/ceph/mds_client.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/ceph/mds_client.c +1 -1
@@ -4977,9 +4977,9 @@ void ceph_mdsc_destroy(struct ceph_fs_client *fsc)ceph_mdsc_stop (mdsc); +cancel_delayed_work_sync (&mdsc->metric.delayed_work);ceph_metric_destroy (&mdsc->metric); -flush_delayed_work (&mdsc->metric.delayed_work); fsc->mdsc =NULL ;kfree (mdsc);dout ("mdsc_destroy %p done\n" , mdsc);
Signature · 1 line
2.31.1
Quoted 23 lines · Jeff Layton
The first thing
metric_delayed_workdoes is checkmdsc->stopping, and then return immediately if it's set...which is good since we would have already torn down the metric structures at this point, otherwise.Worse yet, it's possible that the
ceph_metric_destroycall could race with thedelayed_work, in which case we could end up a end up accessing destroyed percpu variables.At this point in the mdsc teardown, the "stopping" flag has already been set, so there's no benefit to flushing the work. Just cancel it instead, and do so before we tear down the metrics structures.
Cc Xiubo Li <[email protected]>Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/mds_client.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/ceph/mds_client.c +0
-0
fs/ceph/mds_client.c void ceph_mdsc_destroy(struct ceph_fs_client *fsc) · line 4982
Quoted 6 lines
fs/ceph/mds_client.c +1
-0@@ -4977,9 +4977,9 @@ void ceph_mdsc_destroy(struct ceph_fs_client *fsc)ceph_mdsc_stop (mdsc); +cancel_delayed_work_sync (&mdsc->metric.delayed_work);ceph_metric_destroy (&mdsc->metric);
In the "ceph_metric_destroy()" it will also do "cancel_delayed_work_sync(&mdsc->metric.delayed_work)".
We can just move the it to the front of the _destory().
Quoted 4 lines
fs/ceph/mds_client.c +0
-1-flush_delayed_work (&mdsc->metric.delayed_work); fsc->mdsc =NULL ;kfree (mdsc);dout ("mdsc_destroy %p done\n" , mdsc);
Quoted 27 lines · Xiubo Li
Quoted 26 lines · Jeff Layton
The first thing
metric_delayed_workdoes is checkmdsc->stopping, and then return immediately if it's set...which is good since we would have already torn down the metric structures at this point, otherwise.Worse yet, it's possible that the
ceph_metric_destroycall could race with thedelayed_work, in which case we could end up a end up accessing destroyed percpu variables.At this point in the mdsc teardown, the "stopping" flag has already been set, so there's no benefit to flushing the work. Just cancel it instead, and do so before we tear down the metrics structures.
Cc Xiubo Li <[email protected]>Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/mds_client.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/ceph/mds_client.c +0
-0@@ -4977,9 +4977,9 @@ void ceph_mdsc_destroy(struct ceph_fs_client *fsc)ceph_mdsc_stop (mdsc);
Quoted 10 lines
fs/ceph/mds_client.c void ceph_mdsc_destroy(struct ceph_fs_client *fsc) · line 4982
Quoted 3 lines
fs/ceph/mds_client.c +1
-0+cancel_delayed_work_sync (&mdsc->metric.delayed_work);ceph_metric_destroy (&mdsc->metric);In the "
ceph_metric_destroy()" it will also do "cancel_delayed_work_sync(&mdsc->metric.delayed_work)".We can just move the it to the front of the
_destory().
Good point! I'll send a v2 after I test it out.
Quoted 6 lines
fs/ceph/mds_client.c void ceph_mdsc_destroy(struct ceph_fs_client *fsc) · line 4985
Quoted 5 lines
fs/ceph/mds_client.c +0
-1-flush_delayed_work (&mdsc->metric.delayed_work); fsc->mdsc =NULL ;kfree (mdsc);dout ("mdsc_destroy %p done\n" , mdsc);
Thanks,
Signature · 1 line
Jeff Layton <[email protected]>