drbd: do not finish the stable resync when repl state is L_PAUSED_SYNC_T
PATCH 1/1 drbd: do not finish the stable resync when repl state is L_PAUSED_SYNC_T
The assumption was, when a unstable resync start first, then a stable resync starts and be paused. If the stable resync finish ahead of the unstable one from paused directly, it will leaving the resync end into INCONSISTENT finally.
So don't finish the stable resync when repl state is L_PAUSED_SYNC_T, finish it when the repl state turns into L_SYNC_TARGET.
--- drbd/drbd_actlog.c | 15 +++++++++++++++ drbd/drbd_int.h | 1 + drbd/drbd_sender.c | 5 +++++ 3 files changed, 21 insertions(+)
drbd/drbd_actlog.c +15 -0
@@ -1117,7 +1117,22 @@ static int update_sync_bits(struct drbd_peer_device *peer_device,if (mode ==SET_IN_SYNC ) {unsigned long still_to_go =drbd_bm_total_weight (peer_device);bool rs_is_done = (still_to_go <= peer_device->rs_failed); +enum drbd_repl_state repl_state = peer_device->repl_state[NOW ];drbd_advance_rs_marks (peer_device, still_to_go); +if (rs_is_done && repl_state ==L_PAUSED_SYNC_T && !test_bit (UNSTABLE_RESYNC , &peer_device->flags)) { +struct drbd_device *device = peer_device->device; +struct drbd_peer_device *p; +for_each_peer_device (p, device) { +if (p == peer_device) +continue ; + +if (test_bit (UNSTABLE_RESYNC , &p->flags) && p->repl_state[NOW ] ==L_SYNC_TARGET ) { + rs_is_done = false; + peer_device->finish_resync_delay = true; +break ; + } + } + }if (cleared || rs_is_done)maybe_schedule_on_disk_bitmap_update (peer_device, rs_is_done); }else if (mode ==RECORD_RS_FAILED ) {
drbd/drbd_int.h +1 -0
@@ -1095,6 +1095,7 @@ struct drbd_peer_device {bool resync_susp_peer[2 ];bool resync_susp_dependency[2 ];bool resync_susp_other_c[2 ]; +bool finish_resync_delay;enum drbd_repl_state negotiation_result;/* To find disk state after attach */ unsigned int send_cnt;unsigned int recv_cnt;
drbd/drbd_sender.c +5 -0
@@ -735,6 +735,7 @@ next_sector: goto request_done; } + peer_device->finish_resync_delay = false; sector =BM_BIT_TO_SECT (bit);if (drbd_try_rs_begin_io (peer_device, sector, true)) {
@@ -833,6 +834,10 @@ request_done:* resync data block, and the last bit is cleared. * until then resync "work" is "inactive" ... */ +if (peer_device->finish_resync_delay) { +drbd_resync_finished (peer_device,D_MASK ); + peer_device->finish_resync_delay = false; + }put_ldev (device);return 0 ; }
Signature · 1 line
2.25.1
Hi,
Thanks for the contribution. Please describe the steps necessary to reproduce the issue. I tried briefly to reproduce it without success; getting a paused sync to finish before a running one is not that easy. A test in https://github.com/LINBIT/drbd9-tests would be ideal, but a written description is OK too.
Please also verify that the issue occurs on the latest drbd-9.0 or master branch. I see from the patch that you are working on a version that is at least a couple of months old.
Best regards,
Joel
Hi Joel,<br/> Thanks for the reply and sorry for the delay. After some more <br/>investigation, I found the root cause of this problem we met is that <br/>there is a race condition in update_sync_bits() and receive_bitmap().<br/>This race condition will lead the problem I was trying to solve in this <br/>patch above.<br/><br/>We confirmed the race condition existing in upstream and I have sent a <br/>patch for it:<br/>https://lists.linbit.com/pipermail/drbd-dev/2021-September/005840.html<br/><br/>So you can go to review my new patch directly.<br/><br/>Thanks,<br/>Xu
From: Joel Colledge <[email protected]>
Date: 2021-09-01 16:40:32
To: Rui Xu <[email protected]>
Cc: Philipp Reisner <[email protected]>,[email protected]
Subject: Re: [Drbd-dev] [PATCH] drbd: do not finish the stable resync when repl state is L_PAUSED_SYNC_T
Hi Joel,
I'm sorry there were some format problems in my last email.
Thanks for the reply and sorry for the delay. After some more investigation, I found the root cause
of this problem we met is that there is a race condition in update_sync_bits() and receive_bitmap(). This race condition will lead the problem I was trying to solve in this patch above.
We confirmed the race condition existing in upstream and I have sent a patch for it:
https://lists.linbit.com/pipermail/drbd-dev/2021-September/005840.html
So you can go to review my new patch directly.
Thanks,
Xu
From: Joel Colledge <[email protected]>
Date: 2021-09-01 16:40:32
To: Rui Xu <[email protected]>
Cc: Philipp Reisner <[email protected]>,[email protected]
Subject: Re: [Drbd-dev] [PATCH] drbd: do not finish the stable resync when repl state is L_PAUSED_SYNC_T>Hi,
Quoted 13 lines
Thanks for the contribution. Please describe the steps necessary to reproduce the issue. I tried briefly to reproduce it without success; getting a paused sync to finish before a running one is not that easy. A test in https://github.com/LINBIT/drbd9-tests would be ideal, but a written description is OK too.
Please also verify that the issue occurs on the latest drbd-9.0 or master branch. I see from the patch that you are working on a version that is at least a couple of months old.
Best regards,
Joel