Doesn't "writes" do what resync does ?
Hi List,
As per my previous email, I am busy shifting a bunch of data around. I have just created a new RAID1 set, following which MD started resyncing it.
Before that completed, I started shifting data onto it. I always thought that writing data to a new array, does "exactly" what resyncing does (it has to do all the same calcs etc)
This doesn't seem to be the case:
md2 : active raid1 sdd2[1] sda2[0]
243567616 blocks super 1.2 [2/2] [UU]
[===>.................] resync = 17.2% (41949184/243567616) finish=122.3min speed=27454K/sec
yet:
--- Physical volume ---
PV Name /dev/md2 VG Name RAID PV Size 232.28 GiB / not usable 35.00 MiB Allocatable yes PE Size 4.00 MiB Total PE 59456 Free PE 811 Allocated PE 58645 PV UUID my6BsX-cSPD-MTKK-8U0t-ULVl-MKcA-LcAW2D
All the data on here was moved from another RAID device and currently contains about 200gig of data.
Should that resync not have had more completed ?
Cheers,
Pieter
Quoted 29 lines · Pieter De Wit
Hi List,
As per my previous email, I am busy shifting a bunch of data around. I have just created a new RAID1 set, following which MD started resyncing it.
Before that completed, I started shifting data onto it. I always thought that writing data to a new array, does "exactly" what resyncing does (it has to do all the same calcs etc)
This doesn't seem to be the case:
md2 : active raid1 sdd2[1] sda2[0]
243567616 blocks super 1.2 [2/2] [UU] [===>.................] resync = 17.2% (41949184/243567616)finish=122.3min speed=27454K/sec
yet:
--- Physical volume ---
PV Name /dev/md2 VG Name RAID PV Size 232.28 GiB / not usable 35.00 MiB Allocatable yes PE Size 4.00 MiB Total PE 59456 Free PE 811 Allocated PE 58645 PV UUID my6BsX-cSPD-MTKK-8U0t-ULVl-MKcA-LcAW2D
Quoted 4 lines
All the data on here was moved from another RAID device and currently contains about 200gig of data.
Should that resync not have had more completed ?
Your question is invalid. What you meant to ask is
"Why are pvdisplay and mdstat reporting what seems to be conflicting state data?"
Did you also ask on the lvm list why pvdisplay says most PEs are consumed, yet mdstat says resync is only 17% complete?
Signature · 1 line
Stan
Quoted 11 lines · Stan Hoeppner
Quoted 2 lines · Pieter De Wit
<snip>
Should that resync not have had more completed ?Your question is invalid. What you meant to ask is
"Why are pvdisplay and mdstat reporting what seems to be conflicting state data?"
Did you also ask on the lvm list why pvdisplay says most PEs are consumed, yet mdstat says resync is only 17% complete?
Hi again Stan,
pvdisplay says most PEs are consumed because I moved that data to the device. My question, rephased then:
Shouldn't writes to a RAID device count as resyncs ?
Cheers,
Pieter
Quoted 12 lines · Pieter De Wit
Quoted 11 lines · Stan Hoeppner
Quoted 2 lines · Pieter De Wit
<snip>
Should that resync not have had more completed ?Your question is invalid. What you meant to ask is
"Why are pvdisplay and mdstat reporting what seems to be conflicting state data?"
Did you also ask on the lvm list why pvdisplay says most PEs are consumed, yet mdstat says resync is only 17% complete?
Quoted 6 lines
Hi again Stan,
pvdisplay says most PEs are consumed because I moved that data to the device. My question, rephased then:
Shouldn't writes to a RAID device count as resyncs ?
The resync process is independent of normal IO. It starts at the beginning and soldiers on to the end doing a read of each sector pair then comparing them (for RAID1). So no, writes don't count as resync operations. md doesn't perform write/read/verify in normal operation, only write. Linux relies on hardware to report write errors, and assumes the data hit the disk intact if no error. There is no mechanism to pass a new write as verified to the resync process. If you think it should you may want to shoot the idea past Neil. Though I'm sure he's already considered that and rejected it for various reasons.
And unless you restart the resync, it won't verify any sectors you wrote up to its current position. Any sectors you wrote after that point it will be verifying. You don't need to restart the resync or do another one. I'm simply explaining how the resync is independent of normal write IO.
Signature · 1 line
Stan
Quoted 29 lines · Stan Hoeppner
Quoted 12 lines · Pieter De Wit
Quoted 11 lines · Stan Hoeppner
Quoted 2 lines · Pieter De Wit
<snip>
Should that resync not have had more completed ?Your question is invalid. What you meant to ask is
"Why are pvdisplay and mdstat reporting what seems to be conflicting state data?"
Did you also ask on the lvm list why pvdisplay says most PEs are consumed, yet mdstat says resync is only 17% complete?
Quoted 6 lines
Hi again Stan,
pvdisplay says most PEs are consumed because I moved that data to the device. My question, rephased then:
Shouldn't writes to a RAID device count as resyncs ?
The resync process is independent of normal IO. It starts at the beginning and soldiers on to the end doing a read of each sector pair then comparing them (for RAID1). So no, writes don't count as resync operations. md doesn't perform write/read/verify in normal operation, only write. Linux relies on hardware to report write errors, and assumes the data hit the disk intact if no error. There is no mechanism to pass a new write as verified to the resync process. If you think it should you may want to shoot the idea past Neil. Though I'm sure he's already considered that and rejected it for various reasons.
Quoted 6 lines
And unless you restart the resync, it won't verify any sectors you wrote up to its current position. Any sectors you wrote after that point it will be verifying. You don't need to restart the resync or do another one. I'm simply explaining how the resync is independent of normal write IO.
Hi Stan,
I will leave the email intact for Neil. Neil, please point out anything I have missed :)
I understand, but why is the following not the same then (let's stick to RAID 1 for now, just to make it easier)
* Read both sectors, yip, they are the same
* Write both sectors with the same data, none reported an error
The one outstanding thing I can think of is that this will require a major re-write of the code. I *assume* that MD doesn't track the resync progress by bitmap, but rather by sector count.
The second is that it could become hell to update a bitmap if there in threads involved, I am also guessing it will create havoc for the scheduler, since a write now has to update the bitmap, but in order to do so, it needs a memory lock, which means that it might have to wait for the sync read to complete.
Cheers,
Pieter