determining snapshot overhead
Hello,
I am trying to create snapshots of thick volumes that can't break because they overflow (thin volumes are not an option right now). I can't find information on how to calculate (in advance) the metadata overhead of a snapshot when all blocks in the origin volume are changed (e.g. the snapshot is 100% full). Is there a way to do this?
-Bram
Dne 14.1.2015 v 08:24 Bram Klein Gunnewiek napsal(a):
Quoted 7 lines
Hello,
I am trying to create snapshots of thick volumes that can't break because they overflow (thin volumes are not an option right now). I can't find information on how to calculate (in advance) the metadata overhead of a snapshot when all blocks in the origin volume are changed (e.g. the snapshot is 100% full). Is there a way to do this?
Recent versions of lvm2 (let's say >= 111) are able to determine this size automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
Regards
Zdenek
Thanks for your reply. I tested it out and it does not seem to work (lvm2 version used is 2.02.98-6ubuntu2):
lvcreate -n testvolume3 -L 1G vg
lvcreate -n testvolume3_snapshot -s /dev/vg/testvolume3 -l100%ORIGIN dd if=/dev/zero of=/dev/vg/testvolume3 bs=1G count=1
After this the snapshot is invalid:
testvolume3_snapshot vg swi-I-s-- 1.00g testvolume3 100.00
I checked out the lvm2 source code and found the methods cow_max_extents and _cow_max_size in snapshot_manip.c. Do these methods calculate the exact max size of the snapshot meta data?
Quoted 21 lines · Zdenek Kabelac
Dne 14.1.2015 v 08:24 Bram Klein Gunnewiek napsal(a):
Quoted 7 lines
Hello,
I am trying to create snapshots of thick volumes that can't break because they overflow (thin volumes are not an option right now). I can't find information on how to calculate (in advance) the metadata overhead of a snapshot when all blocks in the origin volume are changed (e.g. the snapshot is 100% full). Is there a way to do this?
Recent versions of lvm2 (let's say >= 111) are able to determine this size automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
Regards
Zdenek
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Dne 14.1.2015 v 11:18 Bram Klein Gunnewiek napsal(a):
Quoted 3 lines
Thanks for your reply. I tested it out and it does not seem to work (lvm2 version used is 2.02.98-6ubuntu2):
yep - too old for this feature being supported.
Quoted 11 lines
lvcreate -n testvolume3 -L 1G vg
lvcreate -ntestvolume3_snapshot-s/dev/vg/testvolume3-l100%ORIGIN dd if=/dev/zero of=/dev/vg/testvolume3 bs=1G count=1After this the snapshot is invalid:
testvolume3_snapshotvg swi-I-s-- 1.00g testvolume3 100.00I checked out the lvm2 source code and found the methods
cow_max_extentsand_cow_max_sizeinsnapshot_manip.c. Do these methods calculate the exact max size of the snapshot meta data?
yes.
It's probably worth to note there has been some related kernel fixes for old snaps - otherwise there has been theoretical path to create much bigger snapshots then the origin size was - though it's nearly impossible to reach such state.
Zdenek
Quoted 31 lines · Zdenek Kabelac
Quoted 15 lines
Dne 14.1.2015 v 08:24 Bram Klein Gunnewiek napsal(a):
Quoted 7 lines
Hello,
I am trying to create snapshots of thick volumes that can't break because they overflow (thin volumes are not an option right now). I can't find information on how to calculate (in advance) the metadata overhead of a snapshot when all blocks in the origin volume are changed (e.g. the snapshot is 100% full). Is there a way to do this?
Recent versions of lvm2 (let's say >= 111) are able to determine this size automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
Regards
Quoted 7 lines
Zdenek
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Quoted 6 lines · Zdenek Kabelac
Dne 14.1.2015 v 11:18 Bram Klein Gunnewiek napsal(a):
Quoted 3 lines
Thanks for your reply. I tested it out and it does not seem to work (lvm2 version used is 2.02.98-6ubuntu2):
yep - too old for this feature being supported.
Ok, then we have to do it ourself since we don't want to go with backports / custom versions on our systems
Quoted 9 lines
Quoted 9 lines
lvcreate -n testvolume3 -L 1G vg
lvcreate -ntestvolume3_snapshot-s/dev/vg/testvolume3-l100%ORIGIN dd if=/dev/zero of=/dev/vg/testvolume3 bs=1G count=1After this the snapshot is invalid:
testvolume3_snapshotvg swi-I-s-- 1.00g testvolume3 100.00
Quoted 7 lines
Quoted 3 lines
I checked out the lvm2 source code and found the methods
cow_max_extentsand_cow_max_sizeinsnapshot_manip.c. Do these methods calculate the exact max size of the snapshot meta data?yes.
It's probably worth to note there has been some related kernel fixes for old snaps - otherwise there has been theoretical path to create much bigger snapshots then the origin size was - though it's nearly impossible to reach such state.
Thanks again. I did some tests and calculations and it seems that the overhead is ~0.39% of the origin size when chunk sizes of 4KiB are used. Bigger chunk size means less overhead. If we use 0.5% of the origin space as meta data overhead we should be on the safe side I think?
Quoted 40 lines
Zdenek
Quoted 25 lines · Zdenek Kabelac
Quoted 15 lines
Dne 14.1.2015 v 08:24 Bram Klein Gunnewiek napsal(a):
Quoted 7 lines
Hello,
I am trying to create snapshots of thick volumes that can't break because they overflow (thin volumes are not an option right now). I can't find information on how to calculate (in advance) the metadata overhead of a snapshot when all blocks in the origin volume are changed (e.g. the snapshot is 100% full). Is there a way to do this?
Recent versions of lvm2 (let's say >= 111) are able to determine this size automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
Regards
Quoted 7 lines
Zdenek
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/Quoted 6 lines
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Dne 14.1.2015 v 13:38 Bram Klein Gunnewiek napsal(a):
Quoted 29 lines · Zdenek Kabelac
Quoted 6 lines
Dne 14.1.2015 v 11:18 Bram Klein Gunnewiek napsal(a):
Quoted 3 lines
Thanks for your reply. I tested it out and it does not seem to work (lvm2 version used is 2.02.98-6ubuntu2):
yep - too old for this feature being supported.
Ok, then we have to do it ourself since we don't want to go with backports / custom versions on our systems
Quoted 13 lines
Quoted 5 lines
lvcreate -n testvolume3 -L 1G vg
lvcreate -ntestvolume3_snapshot-s/dev/vg/testvolume3-l100%ORIGIN dd if=/dev/zero of=/dev/vg/testvolume3 bs=1G count=1Quoted 7 lines
After this the snapshot is invalid:
testvolume3_snapshotvg swi-I-s-- 1.00g testvolume3 100.00I checked out the lvm2 source code and found the methods
cow_max_extentsand_cow_max_sizeinsnapshot_manip.c. Do these methods calculate the exact max size of the snapshot meta data?Quoted 6 lines
yes.
It's probably worth to note there has been some related kernel fixes for old snaps - otherwise there has been theoretical path to create much bigger snapshots then the origin size was - though it's nearly impossible to reach such state.
Quoted 4 lines
Thanks again. I did some tests and calculations and it seems that the overhead is ~0.39% of the origin size when chunk sizes of 4KiB are used. Bigger chunk size means less overhead. If we use 0.5% of the origin space as meta data overhead we should be on the safe side I think?
It cannot be defined as some 'percentage' value - as there are even combinations of LV sizes and extent size where the percentage might be over 50% (thought those are not really usable in real life).
lvm2 code is the only valid calculation how to estimate full max size of snapshot for various combinations of those parameters.
Zdenek
Quoted 52 lines
Quoted 40 lines
Zdenek
Quoted 28 lines · Zdenek Kabelac
Quoted 18 lines
Dne 14.1.2015 v 08:24 Bram Klein Gunnewiek napsal(a):
Quoted 10 lines
Hello,
I am trying to create snapshots of thick volumes that can't break because they
overflow (thin volumes are not an option right now). I can't find information
on how to calculate (in advance) the metadata overhead of a snapshot when all
blocks in the origin volume are changed (e.g. the snapshot is 100% full). Is there a way to do this?Recent versions of lvm2 (let's say >= 111) are able to determine this size automatically - just use: 'lvcreate -s -l100%ORIGIN vg/origin'
Regards
Quoted 7 lines
Zdenek
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/Quoted 6 lines
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/Quoted 5 lines
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/