Skip to contentSkip to conversation

lvm-vdo, snapshots and cache

linux-lvm Gionatan Danti 2 participants 6 messages Nov 15, 2022 – Nov 16, 2022

lore
Share & tools
lore
#0 lore raw

Dear all,
as previous vdo utils are gone in RHEL9, VDO volumes must be created via lvm-vdo and associated lvm commands.

What is not clear to me is how to combine lvm-vdo with lvmthin (for fast CoW snapshots) and/or lvmcache (for SSD caching of an HDD pool). For example, with the old /dev/mapper/vdo style volumes (ie: the one created via vdo create) one can do something as physical_dev -> vdo -> lvmthin -> lvmcache.

How to do the same with lvm-vdo?
Regards.

Signature · 11 lines
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: [email protected] - [email protected]
GPG public key ID: FF5F32A8

_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
#1 lore raw

Dne 15. 11. 22 v 18:42 Gionatan Danti napsal(a):

Quoted 6 lines

Dear all,
as previous vdo utils are gone in RHEL9, VDO volumes must be created via lvm-vdo and associated lvm commands.

What is not clear to me is how to combine lvm-vdo with lvmthin (for fast CoW snapshots) and/or lvmcache (for SSD caching of an HDD pool). For example, with the old /dev/mapper/vdo style volumes (ie: the one created via vdo create) one can do something as physical_dev -> vdo -> lvmthin -> lvmcache.

How to do the same with lvm-vdo?

Hi

You could try 'vg' on top of another 'vg' - however I'd not recommend to use it this way (and it's unssuported (&unsupportable) by lvm2 in general)

We aim to support multiple VDO LV within a single VDOPOOL - will hopefully happen relatively soon (so it should work like with thin pools)

IMHO I'd not recommend to combine 2 provisioning technologies - it's already hard to resolve 'out-of-space' troubles with just one technology...

We will see if the 'VDO thick snapshot' will also go with multiple VDO LV support - likely not - but will be added later on.

Caching should be already support - on VDO LV as well as on VDOPOOL LV (both work somewhat differently).

Regards

Zdenek

_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

#2 lore raw

Il 2022-11-15 22:56 Zdenek Kabelac ha scritto:

Quoted 3 lines

You could try 'vg' on top of another 'vg' - however I'd not recommend to use it this way (and it's unssuported (&unsupportable) by lvm2 in general)

Hi Zdenek,
yeah, I would strongly avoid that outside lab testing.

Quoted 3 lines

IMHO I'd not recommend to combine 2 provisioning technologies - it's already hard to resolve 'out-of-space' troubles with just one technology...

The issue is that fast snapshots are only provided by dm-thin, meaning that lvmthin is almost mandatory for implementing a rolling/continuous snapshot scheme. And, on a more basic level, if we can not stack LVM on top of a VDO LV (in a supported manner), how can we take a snapshot of user data residing on that specific volume?

This seems a clear regression vs a separate VDO device. Am I missing something?

Quoted 2 lines

Caching should be already support - on VDO LV as well as on VDOPOOL LV (both work somewhat differently).

Good, thanks for confirming. I will do some tests... Regards.

Signature · 11 lines
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: [email protected] - [email protected]
GPG public key ID: FF5F32A8

_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
#3 lore raw

Dne 15. 11. 22 v 23:41 Gionatan Danti napsal(a):

Quoted 5 lines

Il 2022-11-15 22:56 Zdenek Kabelac ha scritto:

Quoted 3 lines

You could try  'vg' on top of another 'vg' - however I'd not recommend to use it this way (and it's unssuported (&unsupportable) by lvm2 in general)

Quoted 8 lines

Hi Zdenek,
yeah, I would strongly avoid that outside lab testing.

Quoted 3 lines

IMHO I'd not recommend to combine 2 provisioning technologies - it's already hard to resolve 'out-of-space' troubles with just one technology...

The issue is that fast snapshots are only provided by dm-thin, meaning that lvmthin is almost mandatory for implementing a rolling/continuous snapshot scheme. And, on a more basic level, if we can not stack LVM on top of a VDO LV (in a supported manner), how can we take a snapshot of user data residing on that specific volume?

Well - as said - vg on vg is basically equivalent of the original LV on top of VDO manager.

But still these fast snapshot do not solve you a problem of double out-of-space fault.

Quoted 1 line

This seems a clear regression vs a separate VDO device. Am I missing something?

It's good for experimenting - but I'd not suggest to use this setup.

Regards

Zdenek

_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

#4 lore raw

Il 2022-11-16 11:50 Zdenek Kabelac ha scritto:

Quoted 2 lines

Well - as said - vg on vg is basically equivalent of the original LV on top of VDO manager.

Hi Zdenek,
it seems clunkier to manage two nested VG/LV if you ask me.

Quoted 2 lines

But still these fast snapshot do not solve you a problem of double out-of-space fault.

Yeah, but available space requires constant monitor even when dealing with lvmthin.

Quoted 1 line

It's good for experimenting - but I'd not suggest to use this setup.

In RHEL7 it was specifically supported - see here[1]:

"As a general rule, you should place certain storage layers under VDO and others on top of VDO:
Under VDO: DM-Multipath, DM-Crypt, and software RAID (LVM or mdraid).
On top of VDO: LVM cache, LVM snapshots, and LVM Thin Provisioning"

For a practical example: in the past I experimented with a storage stack composed of raid -> vdo -> lvmthin -> xfs serving qemu/kvm virtual disk files. The lvmthin layer enabled rolling snapshots, and was a key feature of the test setup.

Now the only method to replicate that is to have two nested LVM instances, one running vdo and one lvmthin - right?

Regards.

[1] https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/storage_administration_guide/vdo-qs-requirements

Signature · 11 lines
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: [email protected] - [email protected]
GPG public key ID: FF5F32A8

_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
#5 lore raw

Dne 16. 11. 22 v 15:38 Gionatan Danti napsal(a):

Quoted 6 lines

Il 2022-11-16 11:50 Zdenek Kabelac ha scritto:

Quoted 2 lines

Well - as said - vg on vg is basically equivalent of the original LV on top of VDO manager.

Hi Zdenek,
it seems clunkier to manage two nested VG/LV if you ask me.

IMHO not different from lvm on top of vdo managed volume.
It should be mostly equal (and in few cases possibly even easier or at least more OS friendly)

Quoted 10 lines
Quoted 3 lines

But still these fast snapshot do not solve you a problem of double out-of-space fault.

Yeah, but available space requires constant monitor even when dealing with lvmthin.

Quoted 1 line

It's good for experimenting - but I'd not suggest to use this setup.

In RHEL7 it was specifically supported - see here[1]:

Quoted 8 lines

"As a general rule, you should place certain storage layers under VDO and others on top of VDO:
Under VDO: DM-Multipath, DM-Crypt, and software RAID (LVM or mdraid).
On top of VDO: LVM cache, LVM snapshots, and LVM Thin Provisioning"

For a practical example: in the past I experimented with a storage stack composed of raid -> vdo -> lvmthin -> xfs serving qemu/kvm virtual disk files. The lvmthin layer enabled rolling snapshots, and was a key feature of the test setup.

Now the only method to replicate that is to have two nested LVM instances, one running vdo and one lvmthin - right?

Well - it's not difficult 'coding' to enable using 'VDO LV' as thinpool data LV, however we currently lack a lot on support for 'VDO LV' recovery alone in lvm2 - so until this will get polished - we try to avoid shot ourself to our foots with enabling these complicated stacks ATM :)

We also have some ideas for different snapshoting inside VDO - so we will see how that will go - it looks more supportable then combining VDO+THIN together.

Regards

Zdenek

_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/