Running thin_trim before activating a thin pool
Is it possible to configure LVM2 so that it runs thin_trim before it activates a thin pool? Qubes OS currently runs blkdiscard on every thin volume before deleting it, which is slow and unreliable. Would running thin_trim during system startup provide a better alternative?
Signature · 3 lines
Sincerely, Demi Marie Obenour (she/her/hers) Invisible Things Lab
_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Dne 29. 01. 22 v 19:52 Demi Marie Obenour napsal(a):
Quoted 4 lines
Is it possible to configure LVM2 so that it runs
thin_trimbefore it activates a thin pool? Qubes OS currently runs blkdiscard on every thin volume before deleting it, which is slow and unreliable. Would runningthin_trimduring system startup provide a better alternative?
Hi
Nope there is currently no support from lvm2 side for this. Feel free to open RFE.
I guess this would possibly justify some form of support for 'writable' component activation.
Regards
Zdenek
_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Quoted 11 lines · Zdenek Kabelac
Dne 29. 01. 22 v 19:52 Demi Marie Obenour napsal(a):
Quoted 4 lines
Is it possible to configure LVM2 so that it runs
thin_trimbefore it activates a thin pool? Qubes OS currently runs blkdiscard on every thin volume before deleting it, which is slow and unreliable. Would runningthin_trimduring system startup provide a better alternative?Hi
Nope there is currently no support from lvm2 side for this. Feel free to open RFE.
Done: https://bugzilla.redhat.com/show_bug.cgi?id=2048160
Signature · 3 lines
Sincerely, Demi Marie Obenour (she/her/hers) Invisible Things Lab
_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Dne 29. 01. 22 v 21:09 Demi Marie Obenour napsal(a):
Quoted 7 lines · Zdenek Kabelac
Quoted 6 lines
Dne 29. 01. 22 v 19:52 Demi Marie Obenour napsal(a):
Quoted 4 lines
Is it possible to configure LVM2 so that it runs
thin_trimbefore it activates a thin pool? Qubes OS currently runs blkdiscard on every thin volume before deleting it, which is slow and unreliable. Would runningthin_trimduring system startup provide a better alternative?
Quoted 9 lines
Quoted 5 lines
Hi
Nope there is currently no support from lvm2 side for this. Feel free to open RFE.
Thanks
Although your use-case Thinpool on top of VDO is not really a good plan and there is a good reason behind why lvm2 does not support this device stack directly (aka thin-pool data LV as VDO LV). I'd say you are stepping on very very thin ice...
Also I assume you have already checked performance of discard on VDO, but I would not want to run this operation frequently on any larger volume...
Regards
Zdenek
_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Quoted 18 lines · Zdenek Kabelac
Dne 29. 01. 22 v 21:09 Demi Marie Obenour napsal(a):
Quoted 7 lines · Zdenek Kabelac
Quoted 6 lines
Dne 29. 01. 22 v 19:52 Demi Marie Obenour napsal(a):
Quoted 4 lines
Is it possible to configure LVM2 so that it runs
thin_trimbefore it activates a thin pool? Qubes OS currently runs blkdiscard on every thin volume before deleting it, which is slow and unreliable. Would runningthin_trimduring system startup provide a better alternative?Quoted 9 lines
Quoted 5 lines
Hi
Nope there is currently no support from lvm2 side for this. Feel free to open RFE.
Quoted 6 lines
Thanks
Although your use-case Thinpool on top of VDO is not really a good plan and there is a good reason behind why lvm2 does not support this device stack directly (aka thin-pool data LV as VDO LV).
I'd say you are stepping on very very thin ice...
Thin pool on VDO is not my actual use-case. The actual reason for the ticket is slow discards of thin devices that are about to be deleted; you can find more details in the linked GitHub issue. That said, now I am curious why you state that dm-thin on top of dm-vdo (that is, userspace/filesystem/VM/etc ⇒ dm-thin data (*not* metadata) ⇒ dm-vdo ⇒ hardware/dm-crypt/etc) is a bad idea. It seems to be a decent way to add support for efficient snapshots of data stored on a VDO volume, and to have multiple volumes on top of a single VDO volume. Furthermore,
https://access.redhat.com/articles/2106521#vdo recommends exactly this use-case. Or am I misunderstanding you?
Quoted 2 lines
Also I assume you have already checked performance of discard on VDO, but I would not want to run this operation frequently on any larger volume...
I have never actually used VDO myself, although the documentation does warn about this.
Signature · 3 lines
Sincerely, Demi Marie Obenour (she/her/hers) Invisible Things Lab
_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Dne 30. 01. 22 v 2:20 Demi Marie Obenour napsal(a):
Quoted 21 lines · Zdenek Kabelac
Quoted 20 lines
Dne 29. 01. 22 v 21:09 Demi Marie Obenour napsal(a):
Quoted 7 lines · Zdenek Kabelac
Quoted 6 lines
Dne 29. 01. 22 v 19:52 Demi Marie Obenour napsal(a):
Quoted 4 lines
Is it possible to configure LVM2 so that it runs
thin_trimbefore it activates a thin pool? Qubes OS currently runs blkdiscard on every thin volume before deleting it, which is slow and unreliable. Would runningthin_trimduring system startup provide a better alternative?Quoted 9 lines
Quoted 5 lines
Hi
Nope there is currently no support from lvm2 side for this. Feel free to open RFE.
Thanks
Quoted 7 lines
Quoted 4 lines
Although your use-case Thinpool on top of VDO is not really a good plan and there is a good reason behind why lvm2 does not support this device stack directly (aka thin-pool data LV as VDO LV).
I'd say you are stepping on very very thin ice...Thin pool on VDO is not my actual use-case. The actual reason for the ticket is slow discards of thin devices that are about to be deleted;
Hi
Discard of thins itself is AFAIC pretty fast - unless you have massively sized thin devices with many GiB of metadata - obviously you cannot process this amount of metadata in nanoseconds (and there are prepared kernel patches to make it even faster)
What is the problem is the speed of discard of physical devices. You could actually try to feel difference with:
lvchange --discards passdown|nopassdown thinpool
Also it's very important to keep metadata on fast storage device (SSD/NVMe)! Keeping metadata on same hdd spindle as data is always going to feel slow (in fact it's quite pointless to talk about performance and use hdd...)
Quoted 4 lines
you can find more details in the linked GitHub issue. That said, now I am curious why you state that dm-thin on top of dm-vdo (that is, userspace/filesystem/VM/etc ⇒ dm-thin data (*not* metadata) ⇒ dm-vdo ⇒ hardware/dm-crypt/etc) is a bad idea. It seems to be a decent way to
Out-of-space recoveries are ATM much harder then what we want.
So as long as user can maintain free space of your VDO and thin-pool it's ok. Once user runs out of space - recovery is pretty hard task (and there is reason we have support...)
Quoted 2 lines
add support for efficient snapshots of data stored on a VDO volume, and to have multiple volumes on top of a single VDO volume. Furthermore,
We hope we will add some direct 'snapshot' support to VDO so users will not need to combine both technologies together.
Thin is more oriented towards extreme speed.
VDO is more about 'compression & deduplication' - so space efficiency.
Combining both together is kind of harming their advantages.
Quoted 2 lines
https://access.redhat.com/articles/2106521#vdo recommends exactly this use-case. Or am I misunderstanding you?
There are many paths to Rome...
So as mentioned above - you need to pick performance/space effieciency.
And since you want to write your own thin volume managing software, I'm guessing you care about performance a lot (so we do - but with our given constrains that are limiting us to some level)...
Quoted 5 lines
Quoted 2 lines
Also I assume you have already checked performance of discard on VDO, but I would not want to run this operation frequently on any larger volume...
I have never actually used VDO myself, although the documentation does warn about this.
It's been purely related to the initial BZ description which cares a lot about thin discard performance and following comment adds VDO discard into same equation... :)
Regards
Zdenek
_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Quoted 30 lines · Zdenek Kabelac
Dne 30. 01. 22 v 2:20 Demi Marie Obenour napsal(a):
Quoted 21 lines · Zdenek Kabelac
Quoted 20 lines
Dne 29. 01. 22 v 21:09 Demi Marie Obenour napsal(a):
Quoted 7 lines · Zdenek Kabelac
Quoted 6 lines
Dne 29. 01. 22 v 19:52 Demi Marie Obenour napsal(a):
Quoted 4 lines
Is it possible to configure LVM2 so that it runs
thin_trimbefore it activates a thin pool? Qubes OS currently runs blkdiscard on every thin volume before deleting it, which is slow and unreliable. Would runningthin_trimduring system startup provide a better alternative?Quoted 9 lines
Quoted 5 lines
Hi
Nope there is currently no support from lvm2 side for this. Feel free to open RFE.
Thanks
Quoted 7 lines
Quoted 4 lines
Although your use-case Thinpool on top of VDO is not really a good plan and there is a good reason behind why lvm2 does not support this device stack directly (aka thin-pool data LV as VDO LV).
I'd say you are stepping on very very thin ice...Thin pool on VDO is not my actual use-case. The actual reason for the ticket is slow discards of thin devices that are about to be deleted;
Quoted 6 lines
Hi
Discard of thins itself is AFAIC pretty fast - unless you have massively sized thin devices with many GiB of metadata - obviously you cannot process this amount of metadata in nanoseconds (and there are prepared kernel patches to make it even faster)
Would you be willing and able to share those patches?
Quoted 3 lines
What is the problem is the speed of discard of physical devices. You could actually try to feel difference with:
lvchange --discards passdown|nopassdown thinpool
In Qubes OS I believe we do need the discards to be passed down eventually, but I doubt it needs to be synchronous. Being able to run the equivalent of fstrim -av periodically would be amazing. I’m CC’ing Marek Marczykowski-Górecki (Qubes OS project lead) in case he has something to say.
Quoted 3 lines
Also it's very important to keep metadata on fast storage device (SSD/NVMe)! Keeping metadata on same hdd spindle as data is always going to feel slow (in fact it's quite pointless to talk about performance and use hdd...)
That explains why I had such a horrible experience with my initial (split between NVMe and HDD) install. I would not be surprised if some or all of the metadata volume wound up on the spinning disk.
Quoted 6 lines
Quoted 4 lines
you can find more details in the linked GitHub issue. That said, now I am curious why you state that dm-thin on top of dm-vdo (that is, userspace/filesystem/VM/etc ⇒ dm-thin data (*not* metadata) ⇒ dm-vdo ⇒ hardware/dm-crypt/etc) is a bad idea. It seems to be a decent way to
Out-of-space recoveries are ATM much harder then what we want.
Okay, thanks! Will this be fixed in a future version?
Quoted 3 lines
So as long as user can maintain free space of your VDO and thin-pool it's ok. Once user runs out of space - recovery is pretty hard task (and there is reason we have support...)
Out of space is already a tricky issue in Qubes OS. I certainly would not want to make it worse.
Quoted 5 lines
Quoted 2 lines
add support for efficient snapshots of data stored on a VDO volume, and to have multiple volumes on top of a single VDO volume. Furthermore,
We hope we will add some direct 'snapshot' support to VDO so users will not need to combine both technologies together.
Does that include support for splitting a VDO volume into multiple, individually-snapshottable volumes, the way thin works?
Quoted 4 lines
Thin is more oriented towards extreme speed.
VDO is more about 'compression & deduplication' - so space efficiency.Combining both together is kind of harming their advantages.
That makes sense.
Signature · 3 lines
Sincerely, Demi Marie Obenour (she/her/hers) Invisible Things Lab
_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Dne 30. 01. 22 v 18:30 Demi Marie Obenour napsal(a):
Quoted 5 lines · Zdenek Kabelac
Quoted 4 lines
Dne 30. 01. 22 v 2:20 Demi Marie Obenour napsal(a):
Quoted 3 lines · Zdenek Kabelac
Quoted 2 lines
Dne 29. 01. 22 v 21:09 Demi Marie Obenour napsal(a):
Quoted 1 line · Zdenek Kabelac
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Dne 29. 01. 22 v 19:52 Demi Marie Obenour napsal(a):
Discard of thins itself is AFAIC pretty fast - unless you have massively sized thin devices with many GiB of metadata - obviously you cannot process this amount of metadata in nanoseconds (and there are prepared kernel patches to make it even faster)
Would you be willing and able to share those patches?
Then are always landing in upstream kernel once they are all validated & tested (recent kernel already has many speed enhancements).
Quoted 10 lines
Quoted 4 lines
What is the problem is the speed of discard of physical devices. You could actually try to feel difference with:
lvchange --discards passdown|nopassdown thinpoolIn Qubes OS I believe we do need the discards to be passed down eventually, but I doubt it needs to be synchronous. Being able to run the equivalent of
fstrim -avperiodically would be amazing. I’m CC’ing Marek Marczykowski-Górecki (Qubes OS project lead) in case he has something to say.
You could easily run in parallel individual blkdiscards for your thin LVs.... For most modern drives thought it's somewhat waste of time...
Those trimming tools should be used when they are solving some real problems, running them just for fun is just energy & performance waste....
Quoted 8 lines
Quoted 4 lines
Also it's very important to keep metadata on fast storage device (SSD/NVMe)! Keeping metadata on same hdd spindle as data is always going to feel slow (in fact it's quite pointless to talk about performance and use hdd...)
That explains why I had such a horrible experience with my initial (split between NVMe and HDD) install. I would not be surprised if some or all of the metadata volume wound up on the spinning disk.
With lvm2 user can always 'pvmove' any LV to any desired PV. There is not yet any 'smart' logic to do it automatically.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
add support for efficient snapshots of data stored on a VDO volume, and to have multiple volumes on top of a single VDO volume. Furthermore,
We hope we will add some direct 'snapshot' support to VDO so users will not need to combine both technologies together.
Does that include support for splitting a VDO volume into multiple, individually-snapshottable volumes, the way thin works?
Yes - that's the plan - to have multiple VDO LV in a single VDOPool.
Regards
Zdenek
_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Quoted 11 lines · Zdenek Kabelac
Dne 30. 01. 22 v 18:30 Demi Marie Obenour napsal(a):
Quoted 7 lines · Zdenek Kabelac
Quoted 4 lines
Discard of thins itself is AFAIC pretty fast - unless you have massively sized thin devices with many GiB of metadata - obviously you cannot process this amount of metadata in nanoseconds (and there are prepared kernel patches to make it even faster)
Would you be willing and able to share those patches?
Then are always landing in upstream kernel once they are all validated & tested (recent kernel already has many speed enhancements).
Thanks! Which mailing list should I be watching?
Quoted 10 lines
Quoted 9 lines
Quoted 3 lines
What is the problem is the speed of discard of physical devices. You could actually try to feel difference with:
lvchange --discards passdown|nopassdown thinpoolIn Qubes OS I believe we do need the discards to be passed down eventually, but I doubt it needs to be synchronous. Being able to run the equivalent of
fstrim -avperiodically would be amazing. I’m CC’ing Marek Marczykowski-Górecki (Qubes OS project lead) in case he has something to say.
Quoted 5 lines
You could easily run in parallel individual blkdiscards for your thin LVs.... For most modern drives thought it's somewhat waste of time...
Those trimming tools should be used when they are solving some real problems, running them just for fun is just energy & performance waste....
My understanding (which could be wrong) is that periodic trim is necessary for SSDs.
Quoted 10 lines
Quoted 7 lines
Quoted 3 lines
Also it's very important to keep metadata on fast storage device (SSD/NVMe)! Keeping metadata on same hdd spindle as data is always going to feel slow (in fact it's quite pointless to talk about performance and use hdd...)
That explains why I had such a horrible experience with my initial (split between NVMe and HDD) install. I would not be surprised if some or all of the metadata volume wound up on the spinning disk.
With lvm2 user can always 'pvmove' any LV to any desired PV. There is not yet any 'smart' logic to do it automatically.
Good point. I was probably unware of that at the time.
Signature · 3 lines
Sincerely, Demi Marie Obenour (she/her/hers) Invisible Things Lab
_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Dne 30. 01. 22 v 19:01 Demi Marie Obenour napsal(a):
Quoted 7 lines · Zdenek Kabelac
Quoted 4 lines
Dne 30. 01. 22 v 18:30 Demi Marie Obenour napsal(a):
Quoted 1 line · Zdenek Kabelac
Then are always landing in upstream kernel once they are all validated & tested (recent kernel already has many speed enhancements).
Thanks! Which mailing list should I be watching?
lkml
Quoted 8 lines
Quoted 5 lines
You could easily run in parallel individual blkdiscards for your thin LVs.... For most modern drives thought it's somewhat waste of time...
Those trimming tools should be used when they are solving some real problems, running them just for fun is just energy & performance waste....
My understanding (which could be wrong) is that periodic trim is necessary for SSDs.
This was useful for archaic SSDs. Modern SSD/NVMe drives are much smarter...
Regards
Zdenek
_______________________________________________
linux-lvm mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Il 2022-01-30 12:18 Zdenek Kabelac ha scritto:
Quoted 4 lines
Thin is more oriented towards extreme speed.
VDO is more about 'compression & deduplication' - so space efficiency.Combining both together is kind of harming their advantages.
Unfortunately, it is the only (current) solution to have snapshotting with data compression/deduplication. Integrating snapshot capability into VDO would be awesome!
Thanks.
Signature · 11 lines
Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: [email protected] - [email protected] GPG public key ID: FF5F32A8 _______________________________________________ linux-lvm mailing list [email protected] https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/