mkfs.xfs, lvm, multi-terrabyte hardware array and luks

When I try to format partitions above 2-3gb my opteron experiences heavy io wait; the mkfs.xfs fails, and I receive the following....

"mkfs.xfs: libxfs_device_zero write failed: Input/output error"

When I format partions below 2-3gb, there is no problem whatsoever. I can mkfs.xfs on a +2-3GB non-luks formated partition without a problem... any thoughts?

View this message in context: http://www.nabble.com/mkfs.xfs%2C-lvm%2C-multi-terrabyte-hardware-array-and-luks-tf3262532.html#a9068053
Sent from the linux-xfs mailing list archive at Nabble.com.

Sounds like a LUKS problem, maybe it can't do those large offsets? xfs certainly can...

I bet you'll find that the 2GB size is the threshold... xfs is just trying a write():

        if ((bytes = write(fd, z, bytes)) < 0) {
                fprintf(stderr, _("%s: %s write failed: %s\n"),
                        progname, __FUNCTION__, strerror(errno));

maybe try a simple dd write at the end of your large luks device, see how that goes.

-Eric

Thank you for the quick response. I have posted on a few luks forums to try to delve into this issue a little deeper; if they are aware of a resolution I'll make sure to post it. The interesting thing is that when I mkfs.ext3 on luks partitions above 2-3gb all is fine; I wish xfs and luks would play nice.....

View this message in context: http://www.nabble.com/mkfs.xfs%2C-lvm%2C-multi-terrabyte-hardware-array-and-luks-tf3262532.html#a9069238
Sent from the linux-xfs mailing list archive at Nabble.com.

mkfs.xfs actually does writes out at the end of the device and verifies them; I'm not sure ext3 is doing the same. You may find yourself in trouble on ext3 post-mkfs (or you may not...) I seem to recall that xfs has shaken out similar problems on other block devices for this reason.

I'd do some simple device-level read/write tests around 2GB just for fun, see how it goes.

-Eric

XFS.. HERE I COME!!!!!!! The LUKS partition was indeed the culprit. The LUKS partition was not accurately representing the size of the underlying logical volume (when over 2gb) automatically; in order to handle partitions over 2gb it is necessary to explicitly specify the desired size of the LUKS partition (size<lv) to the LUKS partition itself (via the LUKS "resize" command). It seems that mkfs.xfs relies on an accurate LUKS size in order to work on logical volumes greater than or equal to 2gb. I updated this info on the LUKS wiki and other forums...

So just to recap.... if you are getting the error:

"mkfs.xfs: libxfs_device_zero write failed: Input/output error"

make sure that you have resized the underlying LUKS partition to the desired size.

Thanks for the help; much appreciated.

View this message in context: http://www.nabble.com/mkfs.xfs%2C-lvm%2C-multi-terrabyte-hardware-array-and-luks-tf3262532.html#a9074554
Sent from the linux-xfs mailing list archive at Nabble.com.

Eric thanks for the help. It seems that you are likely correct in regard to the LUKS 2gb threshold. I am in the process of urandoming (via dd) a large [20gb] LUKS based ext3 partition to see if the writes fail while less than 20gb... I haven't gotten the results yet... because it takes forever. If this fails to fill the 20gb ext3 with random data then it would be fair to conclude that LUKS has a problem with large partitions [and ext3 has a problem verifying the underlying partition size], right?

In the meantime I have executed a few trial and error tests (by creating various sized lv/LUKS partitions and then mkfs.xfs on them); the results overwhelming suggest that there is a 2gb xfs limitation on LUKS partitions (probably more politically correct to say it the other way around). Besides relying on LUKS forums are there any other resources that you know of that/who would/could provide a quick solution?

Is there anyway that lvm2 could be contributing to this problem? Are there any mkfs.xfs commands that I must issue in this [lvm2/LUKS/Large hardware raid6 array] environment? Are there certain properties that must remain congruent in regard to the raid array, the lvm2 environment, LUKS and/or XFS?

Again, thanks for your time and help.

View this message in context: http://www.nabble.com/mkfs.xfs%2C-lvm%2C-multi-terrabyte-hardware-array-and-luks-tf3262532.html#a9073464
Sent from the linux-xfs mailing list archive at Nabble.com.