Skip to contentSkip to conversation

Saying goodbye to LVM

linux-lvm Xen 4 participants 7 messages Feb 1, 2018 – Feb 7, 2018

lore
Share & tools
lore

Xen

author
#0 lore raw

You are probably happy about this, but...

I managed to get LVM to corrupt my data on Ubuntu Xenial 3 times now.

All of that is related to:

- LVM not checking or behaving correctly when a duplicate PV appears
- LVM not checking or behaving correctly when a cache volume is out of sync with its origin.

In addition the thin DM target of kernel 3.x was so buggy I couldn't compile anything big without the system hanging.

I will probably become a ZFS user.

Goodbye, and thanks for all the fish.

#1 lore raw

If you think zfs will be the answer... then good luck. So where did a new PV come from that had the same UUID as the existing ones? Basically the more details you give... then we can try to help.

But in any case, good luck.

Sent from my iPhone

Quoted 22 lines · Xen

You are probably happy about this, but...

I managed to get LVM to corrupt my data on Ubuntu Xenial 3 times now.

All of that is related to:

- LVM not checking or behaving correctly when a duplicate PV appears
- LVM not checking or behaving correctly when a cache volume is out of sync with its origin.

In addition the thin DM target of kernel 3.x was so buggy I couldn't compile anything big without the system hanging.

I will probably become a ZFS user.

Goodbye, and thanks for all the fish.

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

#2 lore raw
In reply to Xen

Il 01-02-2018 17:45 Xen ha scritto:

Quoted 10 lines

You are probably happy about this, but...

I managed to get LVM to corrupt my data on Ubuntu Xenial 3 times now.

All of that is related to:

- LVM not checking or behaving correctly when a duplicate PV appears
- LVM not checking or behaving correctly when a cache volume is out of sync with its origin.

Quoted 6 lines

In addition the thin DM target of kernel 3.x was so buggy I couldn't compile anything big without the system hanging.

I will probably become a ZFS user.

Goodbye, and thanks for all the fish.

I am both a LVM/ThinLVM and ZFS heavy user, so I hope to be impartial here...

LVM and its lvmthin counterpart are *rock solid* in my experience, except in the (very) edge case of full thin pool (this was lengthly discussed in the past, and both Zednek and Jonathan gave accurate suggestions on how to avoid that).

However, in my experience, the only distribution which keep updated version of lvm kernel and user space utilities is RHEL/CentOS. I found Debian and Ubuntu based distributions particularly *bad* at managing LVM and device mapper targets in general.

I am not using lvmcache, so I can not speak for it.

That said, ZFS really is outstanding (especially checksum and compression, albeit is sorely lacks reflinks). I really have high hopes for stratis (https://github.com/stratis-storage), which plan to provide ZFS-like feature using stacked device mapper targets (which our beloved LVM targets on top).

Regards.

Signature · 5 lines
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: [email protected] - [email protected]
GPG public key ID: FF5F32A8
#3 lore raw

When do we run out of turtles?

  Original Message
From: Gionatan Danti
Sent: Wednesday, February 7, 2018 13:50
To: LVM general discussion and development
Reply To: LVM general discussion and development
Cc: Xen
Subject: Re: [linux-lvm] Saying goodbye to LVM

‎ally have high hopes
for stratis (https://github.com/stratis-storage), which plan to provide ZFS-like feature using stacked device mapper targets (which our beloved LVM targets on top)

author
#4 lore raw
In reply to Gionatan Danti

Gionatan Danti schreef op 07-02-2018 19:42:

Quoted 11 lines

I am both a LVM/ThinLVM and ZFS heavy user, so I hope to be impartial here...

LVM and its lvmthin counterpart are *rock solid* in my experience, except in the (very) edge case of full thin pool (this was lengthly discussed in the past, and both Zednek and Jonathan gave accurate suggestions on how to avoid that).

However, in my experience, the only distribution which keep updated version of lvm kernel and user space utilities is RHEL/CentOS. I found Debian and Ubuntu based distributions particularly *bad* at managing LVM and device mapper targets in general.

This is the reason for the problems but LVM released bad products all the same with the solution being to not use them for very long, or rather, to upgrade.

Yes Ubuntu runs a long time behind and Debian also.

As a user, I can't help that, upgrading LVM just like that to have less "there is a pit here but we won't tell you about that" simply seems also fraught with peril.

For example, upgrading LVM slightly to 160 caused udev problems I didn't have before.

So you can blame the distributions, you can also blame features being released first and proper protection only being added much later.

So if you're on Xenial, you are stuck with the features but without the protection.

In particular there is a quagmire of situations you can end up with wrt the shielding and dual activation of the same vg, many times of which you can only get out of the situation with dmsetup remove, but I didn't know this at first.

Or you end up in the situation where a PV is missing and you cannot edit the VG, but in order to remove the PV you have to edit the VG.

A missing cache device cannot be removed without the missing cache device being present.

I meant to say, you can have 2 disks out of sync and resolving it is not possible other than by editing config on disk and doing vgcfgrestore.

But you can't do vgcfgrestore without removing a missing PV first.

There is a huge amount of chicken and egg problems because physically a VG sits inside a PV but logically a PV sits inside a VG and this constantly causes issues.

LVM just has conceptual problems.

Quoted 5 lines

That said, ZFS really is outstanding (especially checksum and compression, albeit is sorely lacks reflinks). I really have high hopes for stratis (https://github.com/stratis-storage), which plan to provide ZFS-like feature using stacked device mapper targets (which our beloved LVM targets on top).

I cannot write more yet because I don't have ZFS setup yet.

I don't like ZFS too much because it's opaque and Linux support seems to be flimsy (for example boot support) and the only good documentation is Oracle but it often does not apply.

#5 lore raw

Il 07-02-2018 21:37 Xen ha scritto:

Quoted 34 lines

This is the reason for the problems but LVM released bad products all the same with the solution being to not use them for very long, or rather, to upgrade.

Yes Ubuntu runs a long time behind and Debian also.

As a user, I can't help that, upgrading LVM just like that to have less "there is a pit here but we won't tell you about that" simply seems also fraught with peril.

For example, upgrading LVM slightly to 160 caused udev problems I didn't have before.

So you can blame the distributions, you can also blame features being released first and proper protection only being added much later.

So if you're on Xenial, you are stuck with the features but without the protection.

In particular there is a quagmire of situations you can end up with wrt the shielding and dual activation of the same vg, many times of which you can only get out of the situation with dmsetup remove, but I didn't know this at first.

Or you end up in the situation where a PV is missing and you cannot edit the VG, but in order to remove the PV you have to edit the VG.

A missing cache device cannot be removed without the missing cache device being present.

I meant to say, you can have 2 disks out of sync and resolving it is not possible other than by editing config on disk and doing vgcfgrestore.

Quoted 7 lines

But you can't do vgcfgrestore without removing a missing PV first.

There is a huge amount of chicken and egg problems because physically a VG sits inside a PV but logically a PV sits inside a VG and this constantly causes issues.

LVM just has conceptual problems.

As a CentOS user, I *never* encountered such problems. I really think these are caused by the lack of proper integration testing from Debian/Ubuntu. But hey - all key LVM developers are RedHat people, so it should be expected (for the better/worse).

I can not say anything about lvmcache, though.

Quoted 6 lines

I cannot write more yet because I don't have ZFS setup yet.

I don't like ZFS too much because it's opaque and Linux support seems to be flimsy (for example boot support) and the only good documentation is Oracle but it often does not apply.

True. I never use it with boot device. LVM and XFS are, on the other hand, extremely well integrated into mainline kernel/userspace utilities.

Hence my great interest in stratis...
Regards.

Signature · 5 lines
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: [email protected] - [email protected]
GPG public key ID: FF5F32A8
author
#6 lore raw

Gionatan Danti schreef op 07-02-2018 22:19:

Quoted 5 lines
Quoted 1 line

LVM just has conceptual problems.

As a CentOS user, I *never* encountered such problems. I really think these are caused by the lack of proper integration testing from Debian/Ubuntu.

That would only apply to udev/boot problems, not the tooling issues.

If you never make DD copies, you never run into such issues.

And if you don't use Cache you won't have those missing PV issues either.

Maybe I am just great at finding missing features but LVM has in the end cost me a lot more time than it has saved me.

I mean, if I had just stuck to regular partitions I would have been further ahead in life by now ;-).

Including any lack of LVM expertise I would have had by then. Which, in the end, I don't think is worth it.

Quoted 2 lines

But hey - all key LVM developers are RedHat people, so it should be expected (for the better/worse).

The denialist nature of Linux people ensures that even if LVM upstream says UPGRADE, Ubuntu will say "why? everything works fine for me".

Or, "I never ran into such issues" ;-).

Quoted 1 line

True. I never use it with boot device.

Even on Solaris it is limited, for example the root pool cannot have an external log device (that means SLOG). Then, you have no clue if this is also going to be the case on Linux or not ;-). And Grub supports booting from a root dataset but only barely, I don't think anything else (e.g. a ZVOL) is any kind of realism. The biggest downside is inflexibility in shrinking pools, and people complain about ZVOL snapshots requiring a lot of space.

Btrfs, on the other hand, supports removing disks from raid sets and just reorganizing what's left.

Quoted 3 lines

LVM and XFS are, on the other
hand, extremely well integrated into mainline kernel/userspace utilities.

Except that apparently there are (or were, or can be) extreme initramfs/udev issues and the Ubuntu support/integration has been flimsy at best -- what's not flimsy is Grub support, it will even load an embedded LVM just fine.

I mean you can have an LV on a PV that is an LV on a PV and Grub will be able to read it, the Ubuntu initramfs will not.

Quoted 1 line

Hence my great interest in stratis...

I don't deny you there but I wonder if I'm not better off sticking to ordinary partitions ;-).

But my main idea is to use compressed ZVOLs if I can.

You can just stick partition tables on those too. ZFS has a lot of different "models".