Skip to contentSkip to conversation

pvremove won't work

linux-lvm Stefan Radovanovici 3 participants 7 messages Jun 9, 2009 – Jun 10, 2009

lore
Share & tools
lore
#0 lore raw

Hi All,

I have a LVM over 3 partitions (3 separate disks) and one of them is failing so I need to remove it from the group. I moved the content to the rest of the disks (pvmove) and now pvdisplay shows:

root@neriak:~# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sdb1
  VG Name               storage
  PV Size               186.31 GB / not usable 4.00 MB
  Allocatable           yes (but full)
  PE Size (KByte)       4096
  Total PE              47694
  Free PE               0
  Allocated PE          47694
  PV UUID               QUmigK-SCow-nMKo-NEcC-cOQY-bAO9-3k6g9K

--- Physical volume ---

  PV Name               /dev/sdc1
  VG Name               storage
  PV Size               149.05 GB / not usable 1.31 MB
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              38156
  Free PE               19483
  Allocated PE          18673
  PV UUID               idHUfs-rd5i-fVf7-UzE0-YGJT-X1k4-mrp3be

"/dev/sda4" is a new physical volume of "200.87 GB" --- NEW Physical volume ---
PV Name /dev/sda4
VG Name

  PV Size               200.87 GB
  Allocatable           NO
  PE Size (KByte)       0
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               TNsOa7-2Ibb-IW9i-C1s5-1NI9-L5e3-WlKnSH

However, I can't remove /dev/sda4 (the partition on the failing disk) from the group:

root@neriak:~# pvremove /dev/sda4
  Can't open /dev/sda4 exclusively - not removing. Mounted filesystem?

I don't see it mounted anywhere. The LVM is also unmounted. Any ideas of what I might be doing wrong ? I am running LVM 2.02.39 on Ubuntu 9.04 64bit

Thanks in advance,
Stefan

#1 lore raw
Quoted 3 lines · Stefan Radovanovici

Hi All,

I have a LVM over 3 partitions (3 separate disks) and one of them is failing so I need to remove it from the group. I moved the content to the rest of the disks (pvmove) and now pvdisplay shows:

....

Quoted 4 lines

However, I can't remove /dev/sda4 (the partition on the failing disk) from the group:

root@neriak:~# pvremove /dev/sda4
 Can't open /dev/sda4 exclusively - not removing. Mounted filesystem?

what is the use of doing pvremove on sda4?? you just rip the disk off and dump it in the wastebin.

to remove a pv from a vg you use vgreduce, but i assume you already did that, judging from the pvdisplay.

Quoted 1 line

I don't see it mounted anywhere. The LVM is also unmounted. Any ideas of what I might be doing wrong ? I am running LVM 2.02.39 on Ubuntu 9.04 64bit

check /sys/block/sda/sda4/holders

Signature · 6 lines
Luca Berra -- [email protected]
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \
#2 lore raw
Quoted 6 lines · Luca Berra
Quoted 4 lines

However, I can't remove /dev/sda4 (the partition on the failing disk) from the group:

root@neriak:~# pvremove /dev/sda4
 Can't open /dev/sda4 exclusively - not removing. Mounted filesystem?

what is the use of doing pvremove on sda4?? you just rip the disk off and dump it in the wastebin.

Well, yeah, I just wanted to make sure that removing the disk while some of its partitions are part of the LVM won't cause any problems. I can't remove it just yet anyway, the OS is also installed on other partitions on sda. So basically as long as I did a vgreduce, I can take the disk out, is that right? I am assuming /dev/sda4 is still showing in the pvdisplay output because it's still a LVM-type partition.

Quoted 1 line

check /sys/block/sda/sda4/holders

I did:

root@neriak:/sys/block/sda/sda4/holders# l
total 0
drwxr-xr-x 2 root root 0 2009-06-09 08:55 .
drwxr-xr-x 4 root root 0 2009-06-09 08:43 ..
lrwxrwxrwx 1 root root 0 2009-06-09 08:55 dm-1 -> ../../../../../../../../../virtual/block/dm-1
root@neriak:/sys/block/sda/sda4/holders# cd dm-1
root@neriak:/sys/block/sda/sda4/holders/dm-1# l
total 0
drwxr-xr-x  5 root root    0 2009-06-09 08:43 .
drwxr-xr-x 28 root root    0 2009-06-09 08:30 ..
lrwxrwxrwx  1 root root    0 2009-06-09 08:55 bdi -> ../../bdi/252:1
-r--r--r--  1 root root 4096 2009-06-09 08:55 capability
-r--r--r--  1 root root 4096 2009-06-09 08:55 dev
-r--r--r--  1 root root 4096 2009-06-09 08:55 ext_range
drwxr-xr-x  2 root root    0 2009-06-09 08:55 holders
drwxr-xr-x  2 root root    0 2009-06-09 08:55 power
-r--r--r--  1 root root 4096 2009-06-09 08:55 range
-r--r--r--  1 root root 4096 2009-06-09 08:55 removable
-r--r--r--  1 root root 4096 2009-06-09 08:55 ro
-r--r--r--  1 root root 4096 2009-06-09 08:55 size
drwxr-xr-x  2 root root    0 2009-06-09 08:55 slaves
-r--r--r--  1 root root 4096 2009-06-09 08:55 stat
lrwxrwxrwx  1 root root    0 2009-06-09 08:55 subsystem -> ../../../../class/block
-rw-r--r--  1 root root 4096 2009-06-09 08:43 uevent

I have to admit I don't know what I am looking at. What's the dm-1 ?

Regards,
Stefan

#3 lore raw
Quoted 9 lines · Stefan Radovanovici
Quoted 6 lines · Luca Berra
Quoted 4 lines

However, I can't remove /dev/sda4 (the partition on the failing disk) from the group:

root@neriak:~# pvremove /dev/sda4
 Can't open /dev/sda4 exclusively - not removing. Mounted filesystem?

what is the use of doing pvremove on sda4?? you just rip the disk off and dump it in the wastebin.

Quoted 7 lines

Well, yeah, I just wanted to make sure that removing the disk while
some of its partitions are part of the LVM won't cause any problems. I can't remove it just yet anyway, the OS is also installed on other partitions on sda. So basically as long as I did a vgreduce, I can take the disk out, is that right? I am assuming /dev/sda4 is still showing in the pvdisplay output because it's still a LVM-type partition.

It is still labeled. LVM2 PV metadata has two components; a label in the 2nd sector that identifies the device as an LVM2 PV and includes the UUID and a metadata buffer further into the device that stores details like the VG the PV belongs to and the LVs that are allocated.

Running vgreduce for a PV edits the metadata on all PVs in the appropriate VG to remove references to the PV being removed.

The pvremove command just wipes the label from the PV.

Quoted 13 lines
Quoted 1 line

check /sys/block/sda/sda4/holders

I did:

root@neriak:/sys/block/sda/sda4/holders# l
total 0
drwxr-xr-x 2 root root 0 2009-06-09 08:55 .
drwxr-xr-x 4 root root 0 2009-06-09 08:43 ..
lrwxrwxrwx 1 root root 0 2009-06-09 08:55 dm-1 -> ../../../../../../../../../virtual/block/dm-1
root@neriak:/sys/block/sda/sda4/holders# cd dm-1
root@neriak:/sys/block/sda/sda4/holders/dm-1# l
total 0
Quoted 3 lines

lrwxrwxrwx 1 root root 0 2009-06-09 08:55 bdi -> ../../bdi/252:1

I have to admit I don't know what I am looking at. What's the dm-1 ?

Use dmsetup:

$ dmsetup info --maj 252 --min 1

(the output above shows dm is using 252 as its major number for this device on your system).

At a guess it looks like you somehow removed the PV from the VG while a logical volume was mapped on it.

Regards,
Bryn.

#4 lore raw
Quoted 9 lines · Bryn M. Reeves

Use dmsetup:

$ dmsetup info --maj 252 --min 1

(the output above shows dm is using 252 as its major number for this device on your system).

At a guess it looks like you somehow removed the PV from the VG while a logical volume was mapped on it.

First of all, thanks for the info. I am not a LVM expert and this is useful information. dmsetup shows this:

root@neriak:/sys/block/sda/sda4/holders/dm-1# dmsetup info --maj 252 --min 1
Name:              storage-pvmove0
State:             ACTIVE
Read Ahead:        256
Tables present:    LIVE
Open count:        0
Event number:      1
Major, minor:      252, 1
Number of targets: 1
UUID: LVM-SS2wO5K12BwSo1iceXVSiTaYPQzdgEuUoOPgGYKKRLO109x4837eAHZeiSmzzEqW

I think I somehow messed up while trying to get rid of /dev/sda4. I used a GUI for this not the CLI, first I tried to move the extents to other PVs (which I think is that pvmove does) and then to actually remove it from the group (which is probably what vgreduce does). If I am not mistaken, the first operation gave an error even though it appeared to have worked somehow at the time.

Any way I can fix this ?

Thanks,
Stefan

#5 lore raw
Quoted 26 lines · Stefan Radovanovici
Quoted 9 lines · Bryn M. Reeves

Use dmsetup:

$ dmsetup info --maj 252 --min 1

(the output above shows dm is using 252 as its major number for this device on your system).

At a guess it looks like you somehow removed the PV from the VG while a logical volume was mapped on it.

First of all, thanks for the info. I am not a LVM expert and this is useful information. dmsetup shows this:

root@neriak:/sys/block/sda/sda4/holders/dm-1# dmsetup info --maj 252 --min 1
Name:              storage-pvmove0
State:             ACTIVE
Read Ahead:        256
Tables present:    LIVE
Open count:        0
Event number:      1
Major, minor:      252, 1
Number of targets: 1
UUID: LVM-SS2wO5K12BwSo1iceXVSiTaYPQzdgEuUoOPgGYKKRLO109x4837eAHZeiSmzzEqW
Quoted 2 lines

I think I somehow messed up while trying to get rid of /dev/sda4. I used a GUI for this not the CLI, first I tried to move the extents to other PVs (which I think is that pvmove does) and then to actually remove it from the group (which is probably what vgreduce does). If I am not mistaken, the first operation gave an error even though it appeared to have worked somehow at the time.

it seems to be some leftover from the pvmove operation, unfortunately i have no idea if some data was not actually moved correctly. did you record the error message the pvmove operation reported?

Regards,
L.

Signature · 6 lines
Luca Berra -- [email protected]
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \
#6 lore raw
Quoted 10 lines · Luca Berra
Quoted 7 lines

I think I somehow messed up while trying to get rid of /dev/sda4. I used a GUI for this not the CLI, first I tried to move the extents to other PVs (which I think is that pvmove does) and then to actually remove it from the group (which is probably what vgreduce does). If I am not mistaken, the first operation gave an error even though it appeared to have worked somehow at the time.

it seems to be some leftover from the pvmove operation, unfortunately i have no idea if some data was not actually moved correctly. did you record the error message the pvmove operation reported?

No, unfortunately not, it was just a popup in that GUI. The LVM is mounted now, with only the 2 remaining partitions as PVs, and so far the data seems to be in order so maybe pvmove worked. I guess I will live it at that and when the time comes, I will just remove the failing disk from the system.

Regards,
Stefan