Skip to contentSkip to conversation

Doesn't "writes" do what resync does ?

DISCUSSION Pieter De Wit 2 participants 5 messages Dec 29, 2013 – Dec 30, 2013 MD RAID

lore
Share & tools
lore
#0 lore raw

Hi List,

As per my previous email, I am busy shifting a bunch of data around. I have just created a new RAID1 set, following which MD started resyncing it.

Before that completed, I started shifting data onto it. I always thought that writing data to a new array, does "exactly" what resyncing does (it has to do all the same calcs etc)

This doesn't seem to be the case:

md2 : active raid1 sdd2[1] sda2[0]

      243567616 blocks super 1.2 [2/2] [UU]
      [===>.................]  resync = 17.2% (41949184/243567616) 

finish=122.3min speed=27454K/sec

yet:

--- Physical volume ---

  PV Name               /dev/md2
  VG Name               RAID
  PV Size               232.28 GiB / not usable 35.00 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              59456
  Free PE               811
  Allocated PE          58645
  PV UUID               my6BsX-cSPD-MTKK-8U0t-ULVl-MKcA-LcAW2D

All the data on here was moved from another RAID device and currently contains about 200gig of data.

Should that resync not have had more completed ?

Cheers,

Pieter

#1 lore raw
Quoted 29 lines · Pieter De Wit

Hi List,

As per my previous email, I am busy shifting a bunch of data around. I have just created a new RAID1 set, following which MD started resyncing it.

Before that completed, I started shifting data onto it. I always thought that writing data to a new array, does "exactly" what resyncing does (it has to do all the same calcs etc)

This doesn't seem to be the case:

md2 : active raid1 sdd2[1] sda2[0]

      243567616 blocks super 1.2 [2/2] [UU]
      [===>.................]  resync = 17.2% (41949184/243567616)

finish=122.3min speed=27454K/sec

yet:

--- Physical volume ---

  PV Name               /dev/md2
  VG Name               RAID
  PV Size               232.28 GiB / not usable 35.00 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              59456
  Free PE               811
  Allocated PE          58645
  PV UUID               my6BsX-cSPD-MTKK-8U0t-ULVl-MKcA-LcAW2D
Quoted 4 lines

All the data on here was moved from another RAID device and currently contains about 200gig of data.

Should that resync not have had more completed ?

Your question is invalid. What you meant to ask is

"Why are pvdisplay and mdstat reporting what seems to be conflicting state data?"

Did you also ask on the lvm list why pvdisplay says most PEs are consumed, yet mdstat says resync is only 17% complete?

Signature · 1 line
Stan
#2 lore raw
Quoted 11 lines · Stan Hoeppner
Quoted 2 lines · Pieter De Wit

<snip>
Should that resync not have had more completed ?

Your question is invalid. What you meant to ask is

"Why are pvdisplay and mdstat reporting what seems to be conflicting state data?"

Did you also ask on the lvm list why pvdisplay says most PEs are consumed, yet mdstat says resync is only 17% complete?

Hi again Stan,

pvdisplay says most PEs are consumed because I moved that data to the device. My question, rephased then:

Shouldn't writes to a RAID device count as resyncs ?

Cheers,

Pieter

#3 lore raw
Quoted 12 lines · Pieter De Wit
Quoted 11 lines · Stan Hoeppner
Quoted 2 lines · Pieter De Wit

<snip>
Should that resync not have had more completed ?

Your question is invalid. What you meant to ask is

"Why are pvdisplay and mdstat reporting what seems to be conflicting state data?"

Did you also ask on the lvm list why pvdisplay says most PEs are consumed, yet mdstat says resync is only 17% complete?

Quoted 6 lines

Hi again Stan,

pvdisplay says most PEs are consumed because I moved that data to the device. My question, rephased then:

Shouldn't writes to a RAID device count as resyncs ?

The resync process is independent of normal IO. It starts at the beginning and soldiers on to the end doing a read of each sector pair then comparing them (for RAID1). So no, writes don't count as resync operations. md doesn't perform write/read/verify in normal operation, only write. Linux relies on hardware to report write errors, and assumes the data hit the disk intact if no error. There is no mechanism to pass a new write as verified to the resync process. If you think it should you may want to shoot the idea past Neil. Though I'm sure he's already considered that and rejected it for various reasons.

And unless you restart the resync, it won't verify any sectors you wrote up to its current position. Any sectors you wrote after that point it will be verifying. You don't need to restart the resync or do another one. I'm simply explaining how the resync is independent of normal write IO.

Signature · 1 line
Stan
#4 lore raw
Quoted 29 lines · Stan Hoeppner
Quoted 12 lines · Pieter De Wit
Quoted 11 lines · Stan Hoeppner
Quoted 2 lines · Pieter De Wit

<snip>
Should that resync not have had more completed ?

Your question is invalid. What you meant to ask is

"Why are pvdisplay and mdstat reporting what seems to be conflicting state data?"

Did you also ask on the lvm list why pvdisplay says most PEs are consumed, yet mdstat says resync is only 17% complete?

Quoted 6 lines

Hi again Stan,

pvdisplay says most PEs are consumed because I moved that data to the device. My question, rephased then:

Shouldn't writes to a RAID device count as resyncs ?

The resync process is independent of normal IO. It starts at the beginning and soldiers on to the end doing a read of each sector pair then comparing them (for RAID1). So no, writes don't count as resync operations. md doesn't perform write/read/verify in normal operation, only write. Linux relies on hardware to report write errors, and assumes the data hit the disk intact if no error. There is no mechanism to pass a new write as verified to the resync process. If you think it should you may want to shoot the idea past Neil. Though I'm sure he's already considered that and rejected it for various reasons.

Quoted 6 lines

And unless you restart the resync, it won't verify any sectors you wrote up to its current position. Any sectors you wrote after that point it will be verifying. You don't need to restart the resync or do another one. I'm simply explaining how the resync is independent of normal write IO.

Hi Stan,

I will leave the email intact for Neil. Neil, please point out anything I have missed :)

I understand, but why is the following not the same then (let's stick to RAID 1 for now, just to make it easier)

* Read both sectors, yip, they are the same
* Write both sectors with the same data, none reported an error

The one outstanding thing I can think of is that this will require a major re-write of the code. I *assume* that MD doesn't track the resync progress by bitmap, but rather by sector count.

The second is that it could become hell to update a bitmap if there in threads involved, I am also guessing it will create havoc for the scheduler, since a write now has to update the bitmap, but in order to do so, it needs a memory lock, which means that it might have to wait for the sync read to complete.

Cheers,

Pieter