Bad disk?
Yesterday I added a hard drive (to put extra stuff on it) to my ubuntu 10.10 box and created a LVM in it. Then copied some files to it and restarted the machine to see if it would mount into the right mountpoint. It didn't. So I decided to see if it was there (vg in question is export):
raub@strangepork:~$ sudo vgscan Reading all physical volumes. This may take a while... /dev/dm-0: read failed after 0 of 4096 at 429496664064: Input/output error /dev/dm-0: read failed after 0 of 4096 at 429496721408: Input/output error /dev/dm-0: read failed after 0 of 4096 at 0: Input/output error /dev/dm-0: read failed after 0 of 4096 at 4096: Input/output error /dev/dm-0: read failed after 0 of 4096 at 0: Input/output error Found volume group "export" using metadata type lvm2 Found volume group "root" using metadata type lvm2 raub@strangepork:~$
Those dm-0 messages do not make me happy. dmesg and vgchange make me think the problem is on the new drive:
[ 268.024593] scsi 0:0:0:0: Direct-Access ATA ST3500320NS SN04 PQ: 0 ANSI: 5 [ 268.024900] sd 0:0:0:0: [sdc] 976773168 512-byte logical blocks: (500 GB/465 GiB) [ 268.024918] sd 0:0:0:0: Attached scsi generic sg2 type 0 [ 268.024996] sd 0:0:0:0: [sdc] Write Protect is off [ 268.025003] sd 0:0:0:0: [sdc] Mode Sense: 00 3a 00 00 [ 268.025046] sd 0:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 268.025377] sdc: sdc1 [ 268.049853] sd 0:0:0:0: [sdc] Attached SCSI disk [ 335.467482] quiet_error: 3 callbacks suppressed [ 335.467492] Buffer I/O error on device dm-0, logical block 104857584 [ 335.467540] Buffer I/O error on device dm-0, logical block 104857584 [ 335.467589] Buffer I/O error on device dm-0, logical block 104857598 [ 335.467615] Buffer I/O error on device dm-0, logical block 104857598 [ 335.467647] Buffer I/O error on device dm-0, logical block 0 [ 335.467671] Buffer I/O error on device dm-0, logical block 0 [ 335.467703] Buffer I/O error on device dm-0, logical block 1 [ 335.467734] Buffer I/O error on device dm-0, logical block 104857599 [ 335.467762] Buffer I/O error on device dm-0, logical block 104857599 [ 335.467788] Buffer I/O error on device dm-0, logical block 104857599
raub@strangepork:~$ sudo vgchange -a y /dev/dm-0: read failed after 0 of 4096 at 0: Input/output error 1 logical volume(s) in volume group "export" now active 4 logical volume(s) in volume group "root" now active raub@strangepork:~$
Glancing through http://readlist.com/lists/centos.org/centos/1/5704.html I wondered if I had the wrong file system descriptor:
raub@strangepork:~$ sudo sfdisk -l /dev/sdc
Disk /dev/sdc: 60801 cylinders, 255 heads, 63 sectors/track Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System /dev/sdc1 0+ 60800 60801- 488384001 8e Linux LVM /dev/sdc2 0 - 0 0 0 Empty /dev/sdc3 0 - 0 0 0 Empty /dev/sdc4 0 - 0 0 0 Empty
raub@strangepork:~$
I guess that rules that out. Does that mean the disk is bad? If so, is there a way to retrieve the data I put in it?
Quoted 5 lines · Mauricio Tavares
Yesterday I added a hard drive (to put extra stuff on it) to my ubuntu 10.10 box and created a LVM in it. Then copied some files to it and restarted the machine to see if it would mount into the right mountpoint. It didn't. So I decided to see if it was there (vg in question is export):
None of the errors you posted mention a physical disk at all, so there isn't anything wrong with them. To find out what dm-0 is you need to poke around with dmsetup.
Quoted 10 lines · Phillip Susi
Quoted 5 lines · Mauricio Tavares
Yesterday I added a hard drive (to put extra stuff on it) to my ubuntu 10.10 box and created a LVM in it. Then copied some files to it and restarted the machine to see if it would mount into the right mountpoint. It didn't. So I decided to see if it was there (vg in question is export):
None of the errors you posted mention a physical disk at all, so there isn't anything wrong with them. To find out what dm-0 is you need to poke around with dmsetup.
Now that might be interesting since I honestly do not know how to use
it (and my google-fu seems weak today). But this is a bit more info I found:
raub@strangepork:~$ sudo pvs -a /dev/dm-0: read failed after 0 of 4096 at 0: Input/output error /dev/dm-0: read failed after 0 of 4096 at 429496664064: Input/output error /dev/dm-0: read failed after 0 of 4096 at 429496721408: Input/output error /dev/dm-0: read failed after 0 of 4096 at 0: Input/output error /dev/dm-0: read failed after 0 of 4096 at 4096: Input/output error /dev/dm-0: read failed after 0 of 4096 at 0: Input/output error PV VG Fmt Attr PSize PFree /dev/dm-0 -- 0 0 /dev/dm-1 root lvm2 a- 277.05g 192.55g /dev/dm-2 -- 0 0 /dev/dm-3 -- 0 0 /dev/dm-4 -- 0 0 /dev/dm-5 -- 0 0 /dev/ram0 -- 0 0 /dev/ram1 -- 0 0 /dev/ram10 -- 0 0 /dev/ram11 -- 0 0 /dev/ram12 -- 0 0 /dev/ram13 -- 0 0 /dev/ram14 -- 0 0 /dev/ram15 -- 0 0 /dev/ram2 -- 0 0 /dev/ram3 -- 0 0 /dev/ram4 -- 0 0 /dev/ram5 -- 0 0 /dev/ram6 -- 0 0 /dev/ram7 -- 0 0 /dev/ram8 -- 0 0 /dev/ram9 -- 0 0 /dev/root -- 0 0 /dev/sda1 -- 0 0 /dev/sda3 -- 0 0 /dev/sdc1 export lvm2 a- 465.76g 65.76g raub@strangepork:~$
FYI, the root vg in inside /dev/sda3, which is an encrypted partition. How can I find which drive /dev/dm-0 is?
Quoted 1 line · Mauricio Tavares
How can I find which drive
/dev/dm-0is?
Try:
cat /sys/block/dm-0/dm/name
On my system, I created dm2device so I can tell which snapshot is full or whatever device has a problem:
# cat /usr/local/bin/dm2device #!/bin/sh
cat /sys/block/$1/dm/name
--
Ray Morris
[email protected]
Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/
Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/
Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php
Quoted 61 lines · Mauricio Tavares
Quoted 10 lines · Phillip Susi
Quoted 5 lines · Mauricio Tavares
Yesterday I added a hard drive (to put extra stuff on it) to my ubuntu 10.10 box and created a LVM in it. Then copied some files to it and restarted the machine to see if it would mount into the right mountpoint. It didn't. So I decided to see if it was there (vg in question is export):
None of the errors you posted mention a physical disk at all, so there isn't anything wrong with them. To find out what dm-0 is you need to poke around with dmsetup.
Now that might be interesting since I honestly do not know how
to use it (and my google-fu seems weak today). But this is a bit more info I found:raub@strangepork:~$ sudo pvs -a /dev/dm-0: read failed after 0 of 4096 at 0: Input/output error /dev/dm-0: read failed after 0 of 4096 at 429496664064: Input/output error /dev/dm-0: read failed after 0 of 4096 at 429496721408: Input/output error /dev/dm-0: read failed after 0 of 4096 at 0: Input/output error /dev/dm-0: read failed after 0 of 4096 at 4096: Input/output error /dev/dm-0: read failed after 0 of 4096 at 0: Input/output error PV VG Fmt Attr PSize PFree /dev/dm-0 -- 0 0 /dev/dm-1 root lvm2 a- 277.05g 192.55g /dev/dm-2 -- 0 0 /dev/dm-3 -- 0 0 /dev/dm-4 -- 0 0 /dev/dm-5 -- 0 0 /dev/ram0 -- 0 0 /dev/ram1 -- 0 0 /dev/ram10 -- 0 0 /dev/ram11 -- 0 0 /dev/ram12 -- 0 0 /dev/ram13 -- 0 0 /dev/ram14 -- 0 0 /dev/ram15 -- 0 0 /dev/ram2 -- 0 0 /dev/ram3 -- 0 0 /dev/ram4 -- 0 0 /dev/ram5 -- 0 0 /dev/ram6 -- 0 0 /dev/ram7 -- 0 0 /dev/ram8 -- 0 0 /dev/ram9 -- 0 0 /dev/root -- 0 0 /dev/sda1 -- 0 0 /dev/sda3 -- 0 0 /dev/sdc1 export lvm2 a- 465.76g 65.76g raub@strangepork:~$FYI, the root vg in inside
/dev/sda3, which is an encrypted partition. How can I find which drive/dev/dm-0is?_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Quoted 7 lines · Ray Morris
Quoted 1 line · Mauricio Tavares
How can I find which drive
/dev/dm-0is?Try:
cat/sys/block/dm-0/dm/name
Quoted 8 lines
On my system, I created dm2device so I can tell which snapshot is full or whatever device has a problem:
# cat /usr/local/bin/dm2device #!/bin/shcat
/sys/block/$1/dm/name
Thanks!
cat /sys/block/dm-0/dm/name returned export-vms, which is /dev/sdc1.
Quoted 84 lines
--
Ray Morris
[email protected]Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.phpQuoted 54 lines · Mauricio Tavares
Quoted 10 lines · Phillip Susi
Quoted 5 lines · Mauricio Tavares
Yesterday I added a hard drive (to put extra stuff on it) to my ubuntu 10.10 box and created a LVM in it. Then copied some files to it and restarted the machine to see if it would mount into the right mountpoint. It didn't. So I decided to see if it was there (vg in question is export):
None of the errors you posted mention a physical disk at all, so there isn't anything wrong with them. To find out what dm-0 is you need to poke around with dmsetup.
Now that might be interesting since I honestly do not know how to use it (and my google-fu seems weak today). But this is a bit more info I found:
raub@strangepork:~$ sudo pvs -a /dev/dm-0: read failed after 0 of 4096 at 0: Input/output error /dev/dm-0: read failed after 0 of 4096 at 429496664064: Input/output error /dev/dm-0: read failed after 0 of 4096 at 429496721408: Input/output error /dev/dm-0: read failed after 0 of 4096 at 0: Input/output error /dev/dm-0: read failed after 0 of 4096 at 4096: Input/output error /dev/dm-0: read failed after 0 of 4096 at 0: Input/output error PV VG Fmt Attr PSize PFree /dev/dm-0 -- 0 0 /dev/dm-1 root lvm2 a- 277.05g 192.55g /dev/dm-2 -- 0 0 /dev/dm-3 -- 0 0 /dev/dm-4 -- 0 0 /dev/dm-5 -- 0 0 /dev/ram0 -- 0 0 /dev/ram1 -- 0 0 /dev/ram10 -- 0 0 /dev/ram11 -- 0 0 /dev/ram12 -- 0 0 /dev/ram13 -- 0 0 /dev/ram14 -- 0 0 /dev/ram15 -- 0 0 /dev/ram2 -- 0 0 /dev/ram3 -- 0 0 /dev/ram4 -- 0 0 /dev/ram5 -- 0 0 /dev/ram6 -- 0 0 /dev/ram7 -- 0 0 /dev/ram8 -- 0 0 /dev/ram9 -- 0 0 /dev/root -- 0 0 /dev/sda1 -- 0 0 /dev/sda3 -- 0 0 /dev/sdc1 export lvm2 a- 465.76g 65.76g raub@strangepork:~$Quoted 9 lines
FYI, the root vg in inside
/dev/sda3, which is an encrypted partition. How can I find which drive/dev/dm-0is?_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Quoted 2 lines · Mauricio Tavares
Those dm-0 messages do not make me happy. dmesg and vgchange make me think the problem is on the new drive:
Those dm-0 messages are probably a logical error. For instance, a snapshot that is full would give those errors. You need to tell us what dm-0 is mapped to. Look in /dev/mapper for starters.
Quoted 11 lines
[ 268.024593] scsi 0:0:0:0: Direct-Access ATA ST3500320NS SN04 PQ: 0 ANSI: 5 [ 268.024900] sd 0:0:0:0: [sdc] 976773168 512-byte logical blocks: (500 GB/465 GiB) [ 268.024918] sd 0:0:0:0: Attached scsi generic sg2 type 0 [ 268.024996] sd 0:0:0:0: [sdc] Write Protect is off [ 268.025003] sd 0:0:0:0: [sdc] Mode Sense: 00 3a 00 00 [ 268.025046] sd 0:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 268.025377] sdc: sdc1 [ 268.049853] sd 0:0:0:0: [sdc] Attached SCSI disk
This is normal for your new disk.
Quoted 11 lines
[ 335.467482] quiet_error: 3 callbacks suppressed [ 335.467492] Buffer I/O error on device dm-0, logical block 104857584 [ 335.467540] Buffer I/O error on device dm-0, logical block 104857584 [ 335.467589] Buffer I/O error on device dm-0, logical block 104857598 [ 335.467615] Buffer I/O error on device dm-0, logical block 104857598 [ 335.467647] Buffer I/O error on device dm-0, logical block 0 [ 335.467671] Buffer I/O error on device dm-0, logical block 0 [ 335.467703] Buffer I/O error on device dm-0, logical block 1 [ 335.467734] Buffer I/O error on device dm-0, logical block 104857599 [ 335.467762] Buffer I/O error on device dm-0, logical block 104857599 [ 335.467788] Buffer I/O error on device dm-0, logical block 104857599
Again, this is on dm-0, not sdc.
Quoted 10 lines
raub@strangepork:~$ sudo sfdisk -l /dev/sdcDisk
/dev/sdc: 60801 cylinders, 255 heads, 63 sectors/track Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0Device Boot Start End #cyls #blocks Id System /dev/sdc1 0+ 60800 60801- 488384001 8e Linux LVM /dev/sdc2 0 - 0 0 0 Empty /dev/sdc3 0 - 0 0 0 Empty /dev/sdc4 0 - 0 0 0 Empty
This is normal, not sure what is has to do with filesystem desciptors.
Tell us exactly what you mean by "put a LVM on it". Did you run pvcreate? vgcreate? lvcreate? You might find the output of "pvs" enlightening. That will tell us what PVs you have created.
And list /dev/mapper so we know what dm-0 is, and include the output of "lvs".
Signature · 4 lines
Stuart D. Gathman <[email protected]> Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154 "Confutatis maledictis, flammis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial.
Quoted 9 lines · Stuart D. Gathman
Quoted 2 lines · Mauricio Tavares
Those dm-0 messages do not make me happy. dmesg and vgchange make me think the problem is on the new drive:
Those dm-0 messages are probably a logical error. For instance, a snapshot that is full would give those errors. You need to tell us what dm-0 is mapped to. Look in
/dev/mapperfor starters.
Sorry for that: I did not think it would be there. But, as you said, it is:
lrwxrwxrwx 1 root root 7 2010-11-10 09:30 export-vms -> ../dm-0
the export vg is in /dev/sdc1.
Quoted 39 lines
Quoted 11 lines
[ 268.024593] scsi 0:0:0:0: Direct-Access ATA ST3500320NS SN04 PQ: 0 ANSI: 5 [ 268.024900] sd 0:0:0:0: [sdc] 976773168 512-byte logical blocks: (500 GB/465 GiB) [ 268.024918] sd 0:0:0:0: Attached scsi generic sg2 type 0 [ 268.024996] sd 0:0:0:0: [sdc] Write Protect is off [ 268.025003] sd 0:0:0:0: [sdc] Mode Sense: 00 3a 00 00 [ 268.025046] sd 0:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 268.025377] sdc: sdc1 [ 268.049853] sd 0:0:0:0: [sdc] Attached SCSI diskThis is normal for your new disk.
Quoted 11 lines
[ 335.467482] quiet_error: 3 callbacks suppressed [ 335.467492] Buffer I/O error on device dm-0, logical block 104857584 [ 335.467540] Buffer I/O error on device dm-0, logical block 104857584 [ 335.467589] Buffer I/O error on device dm-0, logical block 104857598 [ 335.467615] Buffer I/O error on device dm-0, logical block 104857598 [ 335.467647] Buffer I/O error on device dm-0, logical block 0 [ 335.467671] Buffer I/O error on device dm-0, logical block 0 [ 335.467703] Buffer I/O error on device dm-0, logical block 1 [ 335.467734] Buffer I/O error on device dm-0, logical block 104857599 [ 335.467762] Buffer I/O error on device dm-0, logical block 104857599 [ 335.467788] Buffer I/O error on device dm-0, logical block 104857599Again, this is on dm-0, not sdc.
Quoted 10 lines
raub@strangepork:~$ sudo sfdisk -l /dev/sdcDisk
/dev/sdc: 60801 cylinders, 255 heads, 63 sectors/track Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0Device Boot Start End #cyls #blocks Id System
/dev/sdc1 0+ 60800 60801- 488384001 8e Linux LVM /dev/sdc2 0 - 0 0 0 Empty /dev/sdc3 0 - 0 0 0 Empty /dev/sdc4 0 - 0 0 0 Empty
Quoted 7 lines
This is normal, not sure what is has to do with filesystem desciptors.
Tell us exactly what you mean by "put a LVM on it". Did you run pvcreate? vgcreate? lvcreate? You might find the output of "pvs" enlightening. That will tell us what PVs you have created.
And list/dev/mapperso we know what dm-0 is, and include the output of "lvs".
Let me put this way, I thought I did. I mean, after creating the
partition, setting it to LVM (8e), then running
pvcreate /dev/sdc1
vgcreate export /dev/sdc1
lvcreate -L 400G --name vms export
I used mkfs.ext4 to create partition (on /dev/mapper/export-vms) and off I went. Do you think I missed a step?
Quoted 9 lines · Mauricio Tavares
Quoted 6 lines
Tell us exactly what you mean by "put a LVM on it". Did you run pvcreate? vgcreate? lvcreate? You might find the output of "pvs" enlightening. That will tell us what PVs you have created. And list
/dev/mapperso we know what dm-0 is, and include the output of "lvs".Let me put this way, I thought I did. I mean, after creating the
partition, setting it to LVM (8e), then running
Quoted 6 lines
pvcreate
/dev/sdc1
vgcreate export/dev/sdc1
lvcreate -L 400G --name vms exportI used mkfs.ext4 to create partition (on
/dev/mapper/export-vms) and off I went. Do you think I missed a step?
Great. Now include output of "lvs"
BTW, if you really suspect a disk error, test for it directly. E.g., you can run
# dd if=/dev/sdc1 of=/dev/null bs=256k
to read through the partition or
# smartctl -t long /dev/sdc
To initiate a long self test of the disk (need smartmontools installed).
A brand new disk that flunks self test is indeed defective.
However, for real physical I/O errors, there would be errors logged in /var/log/messages referencing sdc (as opposed to dm-0), so I still think it is a logical error.
Signature · 4 lines
Stuart D. Gathman <[email protected]> Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154 "Confutatis maledictis, flammis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial.
Quoted 11 lines · Stuart D. Gathman
Quoted 9 lines · Mauricio Tavares
Quoted 6 lines
Tell us exactly what you mean by "put a LVM on it". Did you run pvcreate? vgcreate? lvcreate? You might find the output of "pvs" enlightening. That will tell us what PVs you have created. And list
/dev/mapperso we know what dm-0 is, and include the output of "lvs".Let me put this way, I thought I did. I mean, after creating the
partition, setting it to LVM (8e), then running
Quoted 8 lines
Quoted 6 lines
pvcreate
/dev/sdc1
vgcreate export/dev/sdc1
lvcreate -L 400G --name vms exportI used mkfs.ext4 to create partition (on
/dev/mapper/export-vms) and off I went. Do you think I missed a step?Great. Now include output of "lvs"
raub@strangepork:~$ sudo lvs export /dev/dm-0: read failed after 0 of 4096 at 0: Input/output error LV VG Attr LSize Origin Snap% Move Log Copy% Convert vms export -wi-a- 400.00g raub@strangepork:~$
Quoted 17 lines
BTW, if you really suspect a disk error, test for it directly. E.g., you can run
# dd if=/dev/sdc1 of=/dev/null bs=256kto read through the partition or
# smartctl -t long /dev/sdcTo initiate a long self test of the disk (need smartmontools installed).
A brand new disk that flunks self test is indeed defective.
However, for real physical I/O errors, there would be errors logged in
/var/log/messagesreferencing sdc (as opposed to dm-0), so I still think it is a logical error.