Move md raid5 from intel to sparc?
Hello everyone,
In the interest of keeping around some otherwise-functional Sparcs, I was asked if it would be possible to move a linux md raid set from an Intel box to a Sun box. I said, "Sure," and then immediately wondered about byte ordering -- will this work? Take the SCSI box off of the intel, plug it into the sparc, and try to run the array...
Then my next thought was that someone here would know the answer. :)
Thanks!
Signature · 4 lines
Matthew Mitchell Systems Programmer/Administrator [email protected] Geophysical Development Corporation phone 713 782 1234 1 Riverway Suite 2100, Houston, TX 77056 fax 713 782 1829
Quoted 10 lines · Matthew Mitchell
Hello everyone,
In the interest of keeping around some otherwise-functional Sparcs, I was asked if it would be possible to move a linux md raid set from an Intel box to a Sun box. I said, "Sure," and then immediately wondered about byte ordering -- will this work? Take the SCSI box off of the intel, plug it into the sparc, and try to run the array...
Then my next thought was that someone here would know the answer. :)
It will not work. The on-disk MD superblock is written using cpu arch format. Furthermore, let's say you are able to re-create the raid5 array after moving it to the Sun box. What about the filesystem and data? At least you need a filesystem which manipulates its metadata in neutral format for this kind of "move" to work.
Regards,
Mike Tran
Quoted 10 lines · Mike Tran
Quoted 6 lines · Matthew Mitchell
Hello everyone,
In the interest of keeping around some otherwise-functional Sparcs, I was asked if it would be possible to move a linux md raid set from an Intel box to a Sun box. I said, "Sure," and then immediately wondered about byte ordering -- will this work? Take the SCSI box off of the intel, plug it into the sparc, and try to run the array...
Then my next thought was that someone here would know the answer. :)
Quoted 5 lines
It will not work. The on-disk MD superblock is written using cpu arch format. Furthermore, let's say you are able to re-create the raid5 array after moving it to the Sun box. What about the filesystem and data? At least you need a filesystem which manipulates its metadata in neutral format for this kind of "move" to work.
As I feared.
What _about_ the filesystem and data, though? Some filesystems are certainly written in a known byte-order, like ISO 9660. Are there any of (ext3, XFS, JFS, reiserfs) that this is true for? Or are they all written in cpu native byte-order?
Signature · 4 lines
Matthew Mitchell Systems Programmer/Administrator [email protected] Geophysical Development Corporation phone 713 782 1234 1 Riverway Suite 2100, Houston, TX 77056 fax 713 782 1829
Quoted 4 lines · Matthew Mitchell
What _about_ the filesystem and data, though? Some filesystems are certainly written in a known byte-order, like ISO 9660. Are there any of (ext3, XFS, JFS, reiserfs) that this is true for? Or are they all written in cpu native byte-order?
I know XFS always uses big-endian on disk; one of SGI's selling points was that you can move disks back and forth between MIPS and Intel computers. I think ext2/3 also uses the same format between big and little endian archs.
Signature · 2 lines
--
-Matt StegmanQuoted 10 lines · Matt Stegman
Quoted 4 lines · Matthew Mitchell
What _about_ the filesystem and data, though? Some filesystems are certainly written in a known byte-order, like ISO 9660. Are there any of (ext3, XFS, JFS, reiserfs) that this is true for? Or are they all written in cpu native byte-order?
I know XFS always uses big-endian on disk; one of SGI's selling points was that you can move disks back and forth between MIPS and Intel computers. I think ext2/3 also uses the same format between big and little endian archs.
EXT2 has two on-disk byte orders, and kernel code understands both. The MD physical format could be handled similarly. Present code (drivers/md/md.c) becomes rather upset when it sees unexpected things.
This quick review was based on 2.4.20, and doesn't say anything about 2.5 series.
Just for the kicks from linux/fs/ext2/super.c:
* Big-endian to little-endian byte-swapping/bitmaps by
* David S. Miller ([email protected]), 1995
Yes, it was made to support same disk at SPARC and at i386 ;-)
Quoted 2 lines
Signature · 1 line
-Matt Stegman
/Matti Aarnio
Quoted 3 lines · Matti Aarnio
EXT2 has two on-disk byte orders, and kernel code understands both. The MD physical format could be handled similarly. Present code (
drivers/md/md.c) becomes rather upset when it sees unexpected things.
many years ago, there was ext2-be on m68k and ppc. that was an exception, though, used before the proper cpu_to_le macros were added to the ext2 code. ext2 is specified as le and always has been.
Signature · 2 lines
Tom Vier <[email protected]> DSA Key ID 0xE6CB97DA
Quoted 4 lines · Matthew Mitchell
What _about_ the filesystem and data, though? Some filesystems are certainly written in a known byte-order, like ISO 9660. Are there any of (ext3, XFS, JFS, reiserfs) that this is true for? Or are they all written in cpu native byte-order?
Just look at the current releases of kernel, 2.4 and 2.5, all of the above filesytems seem to write metadata using one known byte-order (ei. always BE or always LE). Now all we need is testing.
-Mike Tran
I am doing a sparc thingin. I have to build the disk images on a x86 system, and move the disk to my sparc, for debug/test. Everhting works fine, except the MD's SB. I tweaked a bit on MD driver, then everything is good. I would think that MD should take care the SB format in next release.
----- Original Message -----
From: "Mike Tran" <[email protected]>
To: "Matthew Mitchell" <[email protected]>
Cc: <[email protected]>
Sent: Wednesday, June 11, 2003 12:17 PM
Subject: Re: Move md raid5 from intel to sparc?
Quoted 18 lines · Matthew Mitchell
Quoted 5 lines
What _about_ the filesystem and data, though? Some filesystems are certainly written in a known byte-order, like ISO 9660. Are there any of (ext3, XFS, JFS, reiserfs) that this is true for? Or are they all written in cpu native byte-order?
Just look at the current releases of kernel, 2.4 and 2.5, all of the above filesytems seem to write metadata using one known byte-order (ei. always BE or always LE). Now all we need is testing.
(quoted diff) +0
-2-Mike Tran -To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 7 lines · On Wed, 2003-06-11 at 14:31, 3tcdgwg3 wrote:
I am doing a sparc thingin. I have to build the disk images on a x86 system, and move the disk to my sparc, for debug/test. Everhting works fine, except the MD's SB. I tweaked a bit on MD driver, then everything is good. I would think that MD should take care the SB format in next release.
This work should start from the MD kernel driver. That is the MD driver has to write the SB in one known format and converts the SB to the native cpu after reading it from disk. Next, the MD tools (mdadm, mdreconf, etc.) will need to follow.
Neil B, Do you have this work item in mind?
-Mike Tran
Quoted 10 lines · Mike Tran
Quoted 7 lines · On Wed, 2003-06-11 at 14:31, 3tcdgwg3 wrote:
I am doing a sparc thingin. I have to build the disk images on a x86 system, and move the disk to my sparc, for debug/test. Everhting works fine, except the MD's SB. I tweaked a bit on MD driver, then everything is good. I would think that MD should take care the SB format in next release.
Quoted 6 lines
This work should start from the MD kernel driver. That is the MD driver has to write the SB in one known format and converts the SB to the native cpu after reading it from disk. Next, the MD tools (mdadm, mdreconf, etc.) will need to follow.
Neil B, Do you have this work item in mind?
Neil's new version 1 superblock, which is present in the 2.5 kernel, addresses the byte ordering issues that are a problem in the current 0.90 superblock of the 2.4 md driver.
The new superblock code stores all numeric values in little endian on-disk and converts them to CPU endianness before use.
--
Paul
Quoted 17 lines · Paul Clements
Quoted 15 lines · Mike Tran
Quoted 7 lines · On Wed, 2003-06-11 at 14:31, 3tcdgwg3 wrote:
I am doing a sparc thingin. I have to build the disk images on a x86 system, and move the disk to my sparc, for debug/test. Everhting works fine, except the MD's SB. I tweaked a bit on MD driver, then everything is good. I would think that MD should take care the SB format in next release.
This work should start from the MD kernel driver. That is the MD driver has to write the SB in one known format and converts the SB to the native cpu after reading it from disk. Next, the MD tools (mdadm, mdreconf, etc.) will need to follow.
Quoted 8 lines
Quoted 1 line
Neil B, Do you have this work item in mind?
Neil's new version 1 superblock, which is present in the 2.5 kernel, addresses the byte ordering issues that are a problem in the current 0.90 superblock of the 2.4 md driver.
The new superblock code stores all numeric values in little endian on-disk and converts them to CPU endianness before use.
Any possibility of a backport? If it's not too hairy I don't mind attempting it myself, but I haven't even looked at md in 2.5.
Signature · 4 lines
Matthew Mitchell Systems Programmer/Administrator [email protected] Geophysical Development Corporation phone 713 782 1234 1 Riverway Suite 2100, Houston, TX 77056 fax 713 782 1829
Quoted 10 lines · Matthew Mitchell
Quoted 5 lines · Paul Clements
Neil's new version 1 superblock, which is present in the 2.5 kernel, addresses the byte ordering issues that are a problem in the current 0.90 superblock of the 2.4 md driver.
The new superblock code stores all numeric values in little endian on-disk and converts them to CPU endianness before use.Any possibility of a backport? If it's not too hairy I don't mind attempting it myself, but I haven't even looked at md in 2.5.
If you just wanted to rip out the superblock code from the 2.5 md.c and use that in 2.4, it probably wouldn't be too bad. The superblock handling code has been improved and modularized quite a bit, and the md driver can now handle various different superblock formats.
--
Paul
Quoted 21 lines · On Thursday June 12, [email protected] wrote:
Quoted 20 lines · Paul Clements
Quoted 16 lines · Mike Tran
Quoted 7 lines · On Wed, 2003-06-11 at 14:31, 3tcdgwg3 wrote:
I am doing a sparc thingin. I have to build the disk images on a x86 system, and move the disk to my sparc, for debug/test. Everhting works fine, except the MD's SB. I tweaked a bit on MD driver, then everything is good. I would think that MD should take care the SB format in next release.
This work should start from the MD kernel driver. That is the MD driver has to write the SB in one known format and converts the SB to the native cpu after reading it from disk. Next, the MD tools (mdadm, mdreconf, etc.) will need to follow.
Neil B, Do you have this work item in mind?
Quoted 9 lines
Quoted 5 lines
Neil's new version 1 superblock, which is present in the 2.5 kernel, addresses the byte ordering issues that are a problem in the current 0.90 superblock of the 2.4 md driver.
The new superblock code stores all numeric values in little endian on-disk and converts them to CPU endianness before use.Any possibility of a backport? If it's not too hairy I don't mind attempting it myself, but I haven't even looked at md in 2.5.
A back-port is really out of the question.
There was massive changes to md in 2.5 in order to prepare the code for this (and other) extensions.
In 2.4, knowledge of the superblock layout is spread throughout the code. In 2.5 it is localised, making multiple superblock formats a possibility.
If you wanted something like this in 2.4, you would need to get the code that reads the superblock to look at it, determine what format it was, and then translate it inplace to the required format (e.g. do byte swapping). Then when writing out you would need to translate it to the original format before writing.
It is certainly do-able, but it wouldn't be a back-port. It would be writing brand new code to do a vaguely similar task in a completely different way.
And to answer the question:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Neil B, Do you have this work item in mind?
No. I am not really interested in anything other than bug fixes for 2.4 md code. If someone else wants to write some improvements, I'm happy to review them and sponsor them into the kernel, but I'm not interested in pushing anything myself.
NeilBrown
I'd be interested in how to get RAID5 to "swing both ways", from Sparc to Intel and back to Sparc with minimal disruption in service.
I've got an Ultra II running Aurora Linux which was recently raised from the dead after a case of heat stroke. During the time it was down I attempted to connect it to an Intel machine running RedHat Linux, both running 2.4 kernels. As expected, it came up with errors.
The logs showed that the raid superblock didn't match the device names on the new system (i.e /dev/sda1 became /dev/sdd1). To fix this I knew I'd have to edit the RAID table using "mdadm". I was sure I could do this since I've done it before when I setup the raid. What I wasn't sure about was whether the Intel kernel could read the Sun disk labels and use them as is. Since I wasn't able to get a good backup of the RAID prior to the failure, I didn't want to chance corrupting the disk labels.
Now that our Sparc is back on-line, I'd like to get a procedure in place to allow connecting the RAID to an Intel Linux machine in case our Sparc is stricken with another case of SIDS (Sudden Information Denial Syndrome).
--Cal Webster
Network Manager
NavAir TSD ISEO CPNC
[email protected]
Quoted 30 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Matthew Mitchell
Sent: Wednesday, June 11, 2003 10:02 AM
To: [email protected]
Subject: Move md raid5 from intel to sparc?Hello everyone,
In the interest of keeping around some otherwise-functional Sparcs, I was asked if it would be possible to move a linux md raid set from an Intel box to a Sun box. I said, "Sure," and then immediately wondered about byte ordering -- will this work? Take the SCSI box off of the intel, plug it into the sparc, and try to run the array...
Then my next thought was that someone here would know the answer. :)
Thanks!
--
Matthew Mitchell
Systems Programmer/Administrator [email protected]
Geophysical Development Corporation phone 713 782 1234
1 Riverway Suite 2100, Houston, TX 77056 fax 713 782 1829-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html