Skip to contentSkip to conversation

Moving MD/LVM from PPC to x86

DISCUSSION Turbo Fredriksson 4 participants 10 messages Jun 28, 2007 – Jul 11, 2007

lore
Share & tools
lore
#0 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'm trying to move some disks from my PPC desktop to a dedicated server. Yesterday my system looked like this:

1. /dev/md0 372.61Gb (sda1+missing)
2. /dev/hdb 74.53Gb
3. /dev/sdb 298.09Gb

Now with the paycheck, I bought a third SATA disk (400Gb). I new from the start that disk wouldn't fit in my desktopbox, so I also bought a 500W PSU (I had a huge server case without PSU from previously).

I was a little impatient, so I added the new disk (physically just dangling outside the box :) to the LVM (as md1+missing).

The idea was to re-create sdb as mdX+missing, so I tried pvmove sdb. But instantly received a lot of errors. Now I can't remove it from the VG!

My new server is a i386, and I already had a disk with Linux installed for Intel, so that disk was also moved (from another server I don't need any more - it was a Pentium 1 - to slow! :).

In with the system disk, in with the LVM disk and started. The system had to be upgraded (from Ubuntu 6.xx to 7.04 - Feisty). Took a couple of hours, and when that was done, I started to get the MD's working. No go... One of the disks didn't even have a partition table!? None of the disks had a MD superblock!

I wasn't really scared, I knew I haven't tried to modify the disks. Put everything back into the PPC, and yes, everything works as expected (exept the read errors from sdc/md1).

So, long winded explanation of what i've done, any idea about:

1. How do I remove md1/sdc before I get data on it (i have the LVM unmounted for the time being).

2. How do I move a VG/PV/LV from PPC to x86?
I.e., how do I fix/solve/get around the problem with endianness so I can move the disks between the machines?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8 <http://mailcrypt.sourceforge.net/>

iD8DBQFGhCHomlWzPKccHgARApC7AJ9eFBMpNObARyIiFptbwChKCWzaXACfSr6z m6k0jAvBS0PtC/1Is8OWTvk=
=qrJb
-----END PGP SIGNATURE-----

#1 lore raw
Quoted 1 line · Turbo Fredriksson

2. How do I move a VG/PV/LV from PPC to x86?

The on-disk LVM2 metadata should be accessible from both architectures.

Alasdair

Signature · 1 line
[email protected]
#2 lore raw

Quoting Alasdair G Kergon <[email protected]>:

Quoted 5 lines · Turbo Fredriksson
Quoted 1 line

2. How do I move a VG/PV/LV from PPC to x86?

The on-disk LVM2 metadata should be accessible from both architectures.

Well, when I move the disks, the intel machine say that one of the disks don't have a partition table and that the other don't have any MD superblock...

#3 lore raw
Quoted 11 lines · On Saturday June 30, [email protected] wrote:

Quoting Alasdair G Kergon <[email protected]>:

Quoted 5 lines · Turbo Fredriksson
Quoted 1 line

2. How do I move a VG/PV/LV from PPC to x86?

The on-disk LVM2 metadata should be accessible from both architectures.

Well, when I move the disks, the intel machine say that one of the disks don't have a partition table and that the other don't have any MD superblock...

Version 0.90 MD superblocks (still the default) uses host-endian values so you cannot move between architectures directly. However isn't too hard to make it work.
Firstly, use
mdadm --examine --metadata=0.swap /dev/DEVICE

to check that you have the right devices.
Then

mdadm --assemble /dev/md0 --update=byteorder /dev/DEV0 /dev/DEV1 ....

That should assemble the array and update the superblocks so that they are in the right byteorder and will assemble easily in future.

NeilBrown

#4 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Neil" == Neil Brown <[email protected]> writes:

Quoted 11 lines · Neil> On Saturday June 30, [email protected] wrote:
Quoted 11 lines

Quoting Alasdair G Kergon <[email protected]>:

Quoted 5 lines · Turbo Fredriksson
Quoted 1 line

2. How do I move a VG/PV/LV from PPC to x86?

The on-disk LVM2 metadata should be accessible from both architectures.

Well, when I move the disks, the intel machine say that one of the disks don't have a partition table and that the other don't have any MD superblock...

Neil> Version 0.90 MD superblocks (still the default) uses host-endian Neil> values so you cannot move between architectures directly. Neil> However isn't too hard to make it work.

So if we have version 1.0, 1.1 and 1.2 superblocks, why aren't they the default yet? I assume they're useful, so let's use them!

John

#5 lore raw
In reply to Neil Brown

Quoting Neil Brown <[email protected]>:

Quoted 2 lines

Firstly, use
mdadm --examine --metadata=0.swap /dev/DEVICE

This should be done on the PPC, or can it be done on the Intel?

Tried this command on some other machines, and there the '--metadata' is an unknown option... This on mdadm v1.9.0.

Quoted 2 lines

Then
mdadm --assemble /dev/md0 --update=byteorder /dev/DEV0 /dev/DEV1 ....

And this on the Intel?

#6 lore raw
In reply to Neil Brown

Quoting Neil Brown <[email protected]>:

Quoted 5 lines

Version 0.90 MD superblocks (still the default) uses host-endian values so you cannot move between architectures directly. However isn't too hard to make it work.
Firstly, use
mdadm --examine --metadata=0.swap /dev/DEVICE

----- s n i p -----

ppc:~# mdadm --examine --metadata=0.swap /dev/md0
mdadm: No super block found on /dev/md0 (Expected magic a92b4efc, got 00000000)
ppc:~# mdadm --examine --metadata=0.swap /dev/md1
mdadm: No super block found on /dev/md1 (Expected magic a92b4efc, got 00000000)
ppc:~# mdadm --examine --metadata=0.swap /dev/md2
mdadm: No super block found on /dev/md2 (Expected magic a92b4efc, got 00000000)
ppc:~# mdadm --examine --metadata=0.swap /dev/sda1
mdadm: No super block found on /dev/sda1 (Expected magic a92b4efc, got fc4e2ba9)
ppc:~# mdadm --examine --metadata=0.swap /dev/sda
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got 00000000)
ppc:~# mdadm --examine --metadata=0.swap /dev/sdb
mdadm: No super block found on /dev/sdb (Expected magic a92b4efc, got 00000000)
ppc:~# mdadm --examine --metadata=0.swap /dev/sdb1
mdadm: No super block found on /dev/sdb1 (Expected magic a92b4efc, got fc4e2ba9)
ppc:~# mdadm --examine --metadata=0.swap /dev/sdc
mdadm: No super block found on /dev/sdc (Expected magic a92b4efc, got 00000000)
ppc:~# mdadm --examine --metadata=0.swap /dev/sdc1
mdadm: No super block found on /dev/sdc1 (Expected magic a92b4efc, got fc4e2ba9)
ppc:~# cat /proc/mdstat | grep ^md
md2 : active raid1 sdb1[0]
md1 : active raid1 sdc1[0]
md0 : active raid1 sda1[0]
----- s n i p -----

Now what?

Quoted 7 lines

to check that you have the right devices.
Then

mdadm --assemble /dev/md0 --update=byteorder /dev/DEV0 /dev/DEV1 ....

That should assemble the array and update the superblocks so that they are in the right byteorder and will assemble easily in future.

Is this safe, changing the byteorder on all the physical devices (that are part of my MD's)?
Will it still work on the PPC?

#7 lore raw
Quoted 8 lines · On Sunday July 8, [email protected] wrote:

Quoting Neil Brown <[email protected]>:

Quoted 5 lines

Version 0.90 MD superblocks (still the default) uses host-endian values so you cannot move between architectures directly. However isn't too hard to make it work.
Firstly, use
mdadm --examine --metadata=0.swap /dev/DEVICE

Quoted 7 lines

----- s n i p -----

ppc:~# mdadm --examine --metadata=0.swap /dev/md0
mdadm: No super block found on /dev/md0 (Expected magic a92b4efc, got 00000000)
ppc:~# mdadm --examine --metadata=0.swap /dev/md1
mdadm: No super block found on /dev/md1 (Expected magic a92b4efc, got 00000000)
ppc:~# mdadm --examine --metadata=0.swap /dev/md2
mdadm: No super block found on /dev/md2 (Expected magic a92b4efc, got 00000000)

You --examine component devices of an array, not the whole array.

Quoted 2 lines
ppc:~# mdadm --examine --metadata=0.swap /dev/sda1
mdadm: No super block found on /dev/sda1 (Expected magic a92b4efc, got fc4e2ba9)

This suggests that the superblock is currently the correct byteorder for the current host. You use
mdadm --examine --metadata=0.swap /dev/sda1

when you have moved a devices from one host to different host with the opposite endian-ness (e.g. bigendian to littlendian).

Quoted 4 lines
Quoted 4 lines

to check that you have the right devices.
Then

Quoted 8 lines
Quoted 4 lines

mdadm --assemble /dev/md0 --update=byteorder /dev/DEV0 /dev/DEV1 ....

That should assemble the array and update the superblocks so that they are in the right byteorder and will assemble easily in future.

Is this safe, changing the byteorder on all the physical devices (that are part of my MD's)?
Will it still work on the PPC?

I cannot remember where you were moving 'from' or 'to', but what you have to do is:

1/ move the devices to the new computer.
2/ use "mdadm --examine" to make sure they are where you expect them
to be. Use "--metdata=0.swap" if the byteorder is different on the new machine.
3/ Once you are satisfied that things look right, use
mdadm --assemble /dev/md0 --update=byteorder ....
to assemble the array. This will change the byteorder in the superblocks. After you have done this, the array will assemble normally on the new machine, but will not if you move it back to the old machine. If you want to use the old machine again, you need to use --update=byteorder again.

Does that make it clear?

NeilBrown

#8 lore raw

Quoting Neil Brown <[email protected]>:

Quoted 5 lines

This suggests that the superblock is currently the correct byteorder for the current host. You use
mdadm --examine --metadata=0.swap /dev/sda1
when you have moved a devices from one host to different host with the opposite endian-ness (e.g. bigendian to littlendian).

I see, thanx for clarifying this.

Quoted 1 line

Does that make it clear?

Absolutely. I just want to be very clear. I don't have a full backup (have no means to backup 1Tb), so I want to minimize the risk of loosing data. I thank you for explaining so that even I can understand :)

PS. No sarcasm intended, even IF it should sound like it! I _am_ greatfull.

Signature · 8 lines
Noriega Honduras Marxist South Africa president AK-47 FSF radar FBI
congress class struggle SEAL Team 6 colonel Cocaine DES
[See http://www.aclu.org/echelonwatch/index.html for more about this]
[Or http://www.europarl.eu.int/tempcom/echelon/pdf/rapport_echelon_en.pdf]
If neither of these works, try http://www.aclu.org and search for echelon.
Note. This is a real, not fiction.
http://www.theregister.co.uk/2001/09/06/eu_releases_echelon_spying_report/
http://www.aclu.org/safefree/nsaspying/23989res20060131.html#echelon
#9 lore raw
In reply to Neil Brown

Quoting Neil Brown <[email protected]>:

Quoted 3 lines

3/ Once you are satisfied that things look right, use
mdadm --assemble /dev/md0 --update=byteorder ....
to assemble the array.

This worked. After a lot of tries. I kept getting 'resource busy' (or 'device busy' or something other in that effect. But eventually I got all my MD's moved.

Thanx for all the help.

Signature · 9 lines
Rule Psix Mossad class struggle tritium DES counter-intelligence
Treasury fissionable NORAD Kennedy smuggle genetic cracking subway
pits
[See http://www.aclu.org/echelonwatch/index.html for more about this]
[Or http://www.europarl.eu.int/tempcom/echelon/pdf/rapport_echelon_en.pdf]
If neither of these works, try http://www.aclu.org and search for echelon.
Note. This is a real, not fiction.
http://www.theregister.co.uk/2001/09/06/eu_releases_echelon_spying_report/
http://www.aclu.org/safefree/nsaspying/23989res20060131.html#echelon