LVM cache/dm-cache questions.
Hi,
I have a few simple questions regarding relatively new feature LVM cache/dm-cache.
From second hand information on internet, it appears that you cannot resize cached volume.
Please correct me if I am wrong: to resize a cached volume, one needs to lvremove the cache, resize and recreate the cache?
In scenario where cache device fails, how safe is the data on the cached LVM?
Is it possible to make snapshots of the cached device?
Before I test these things I will need to upgrade distribution (as the release I am on does not have the LVM cache), which means switching to systemd (I would rather avoid, if LVM cache does not fail gracefully).
Thank you very much!
Sergei.
Quoted 9 lines · sergei
Hi,
I have a few simple questions regarding relatively new feature LVM cache/dm-cache.
From second hand information on internet, it appears that you cannot resize cached volume.
Please correct me if I am wrong: to resize a cached volume, one needs to lvremove the cache, resize and recreate the cache?
This is true at the moment. You are likely to be able to simply resize online in the future.
Quoted 1 line
In scenario where cache device fails, how safe is the data on the cached LVM?
If you are using writethrough, quite safe. If you are using ‘writeback’, probably not (RAID for your cache in this case would probably be wise).
Quoted 1 line
Is it possible to make snapshots of the cached device?
easy answer is “no”.
complicated answer includes a ‘yes’. You can cache a poolDataLV, then your thinLV and all its thinSnaps would be cached.
brassow
Thank you!
That clears up a few questions!
Regards.
Sergei.
Quoted 29 lines · Brassow Jonathan
Quoted 9 lines · sergei
Hi,
I have a few simple questions regarding relatively new feature LVM cache/dm-cache.
From second hand information on internet, it appears that you cannot resize cached volume.
Please correct me if I am wrong: to resize a cached volume, one needs to lvremove the cache, resize and recreate the cache?This is true at the moment. You are likely to be able to simply resize online in the future.
Quoted 1 line
In scenario where cache device fails, how safe is the data on the cached LVM?
If you are using writethrough, quite safe. If you are using ‘writeback’, probably not (RAID for your cache in this case would probably be wise).
Quoted 1 line
Is it possible to make snapshots of the cached device?
easy answer is “no”.
complicated answer includes a ‘yes’. You can cache a poolDataLV, then your thinLV and all its thinSnaps would be cached.
brassow
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
question I'd like to ask is: how to LUKS encrypt cache pool LV ?
many thanks.
L.
Quoted 25 lines · Brassow Jonathan
Quoted 8 lines · sergei
Hi,
I have a few simple questions regarding relatively new feature LVM cache/dm-cache.
From second hand information on internet, it appears that you cannot resize cached volume.
Please correct me if I am wrong: to resize a cached volume, one needs to lvremove the cache, resize and recreate the cache?This is true at the moment. You are likely to be able to simply resize online in the future.
Quoted 1 line
In scenario where cache device fails, how safe is the data on the cached LVM?
If you are using writethrough, quite safe. If you are using ‘writeback’, probably not (RAID for your cache in this case would probably be wise).
Quoted 1 line
Is it possible to make snapshots of the cached device?
easy answer is “no”.
complicated answer includes a ‘yes’. You can cache a poolDataLV, then your thinLV and all its thinSnaps would be cached.
brassow
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
lejeczek schreef op 25-08-2016 13:51:
Quoted 4 lines
question I'd like to ask is: how to LUKS encrypt cache pool LV ?
many thanks.
L.
In general without being any expert here you will need to encrypt the origin device (PV) on which it resides and likely also the PV of the cache pool itself. It's no different from any other volume you would use.
You can allow LUKS to pass-through discards if you open the volume with --allow-discards on the cryptsetup prompt (command).
In case you have an SSD underneath. It was said that the cache itself does not utilize discards just yet, but that it was in the planning.
well, I prefer to encrypt LV itself, and I'm trying the same what worked always with my "regular" LVs, yet - "cache pools LVs" fail to encrypt with: Command failed with code 22.
Quoted 17 lines · Xen
lejeczek schreef op 25-08-2016 13:51:
Quoted 4 lines
question I'd like to ask is: how to LUKS encrypt cache pool LV ?
many thanks.
L.In general without being any expert here you will need to encrypt the origin device (PV) on which it resides and likely also the PV of the cache pool itself. It's no different from any other volume you would use.
You can allow LUKS to pass-through discards if you open the volume with --allow-discards on the cryptsetup prompt (command).
In case you have an SSD underneath. It was said that the cache itself does not utilize discards just yet, but that it was in the planning.
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
lejeczek schreef op 26-08-2016 14:45:
Quoted 3 lines
well, I prefer to encrypt LV itself, and I'm trying the same what worked always with my "regular" LVs, yet - "cache pools LVs" fail to encrypt with: Command failed with code 22.
If you are going to encrypt an LV it will no longer be an LV but an (opened) LUKS container.
whatever you might call it, it works, luks encrypting, opening & mounting @boot - so I only wonder (which was my question) why not cache pool LVs. Is it not supported... would be great if a developer sees this question, I'm not sure jut yet about filing a bug report.
Quoted 12 lines · Xen
lejeczek schreef op 26-08-2016 14:45:
Quoted 3 lines
well, I prefer to encrypt LV itself, and I'm trying the same what worked always with my "regular" LVs, yet - "cache pools LVs" fail to encrypt with: Command failed with code 22.
If you are going to encrypt an LV it will no longer be an LV but an (opened) LUKS container.
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Quoted 5 lines · lejeczek
whatever you might call it, it works, luks encrypting, opening & mounting @boot - so I only wonder (which was my question) why not cache pool LVs. Is it not supported... would be great if a developer sees this question, I'm not sure jut yet about filing a bug report.
In general LVM2 doesn't auto-activate or interpret unknown device types. LUKS header is considered unknown from LVM2 perspective. Simply put LVM2 doesn't understand LUKS header data. Not sure what you tried to do with cache pool LV, but in my opinion any effort to encrypt (live or detached) cache pool LV may end with severe data corruption...
As of now I think you have in general two options:
a) encrypt both PVs because obviously if you only encrypt the origin PV you end up with decrypted plaintext data stored in cache pool. Probably this is the exact scenario you were about to avoid?
Unfortunately a) is suboptimal with regard to performance since you'd perform the encryption of data blocks twice.
Option b): encrypt the top level LV (the one constructed from both cache and origin LV). This way ciphertext would be stored twice in cache PV and origin PV but the encryption would be performed only once.
Regards
Ondrej
lejeczek schreef op 26-08-2016 16:01:
Quoted 5 lines
whatever you might call it, it works, luks encrypting, opening & mounting @boot - so I only wonder (which was my question) why not cache pool LVs. Is it not supported...
would be great if a developer sees this question, I'm not sure jut yet about filing a bug report.
Ondrej has it down.
You can only encrypt the combined volume, not the individual parts, unless you encrypt those at the PV level.
Quoted 9 lines · Ondrej Kozina
Quoted 5 lines · lejeczek
whatever you might call it, it works, luks encrypting, opening & mounting @boot - so I only wonder (which was my question) why not cache pool LVs. Is it not supported... would be great if a developer sees this question, I'm not sure jut yet about filing a bug report.
In general LVM2 doesn't auto-activate or interpret unknown device types. LUKS header is considered unknown from LVM2 perspective. Simply put LVM2 doesn't understand LUKS header data. Not sure what you tried to do with cache pool LV, but in my opinion any effort to encrypt (live or detached) cache pool LV may end with severe data corruption...
Quoted 8 lines
As of now I think you have in general two options:
a) encrypt both PVs because obviously if you only encrypt the origin PV you end up with decrypted plaintext data stored in cache pool. Probably this is the exact scenario you were about to avoid?
Unfortunately a) is suboptimal with regard to performance since you'd perform the encryption of data blocks twice.
Option b): encrypt the top level LV (the one constructed from both cache and origin LV). This way ciphertext would be stored twice in cache PV and origin PV but the encryption would be performed only once.
gee, guys, thanks Ondrej,
this I was saying from the beginning did not work - option b - does not work. I can Not encrypt top level cache pool LV. It does work with any other LV I have, but cache pool fails (like I said earlier) with:
Command failed with code 22.
And me speculating on my own - whether it is a bug or just limitation of implementation (Centos 7.2, lvm2-2.02.130-5.el7_2.4.x86_64, cryptsetup-1.6.7-1.el7.x86_64)- I thought instead I should seek clarification.
Quoted 8 lines
Regards
Ondrej_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Quoted 33 lines · lejeczek
Quoted 22 lines · Ondrej Kozina
Quoted 5 lines · lejeczek
whatever you might call it, it works, luks encrypting, opening & mounting @boot - so I only wonder (which was my question) why not cache pool LVs. Is it not supported... would be great if a developer sees this question, I'm not sure jut yet about filing a bug report.
In general LVM2 doesn't auto-activate or interpret unknown device types. LUKS header is considered unknown from LVM2 perspective. Simply put LVM2 doesn't understand LUKS header data. Not sure what you tried to do with cache pool LV, but in my opinion any effort to encrypt (live or detached) cache pool LV may end with severe data corruption...
As of now I think you have in general two options:
a) encrypt both PVs because obviously if you only encrypt the origin PV you end up with decrypted plaintext data stored in cache pool. Probably this is the exact scenario you were about to avoid?
Quoted 8 lines
Unfortunately a) is suboptimal with regard to performance since you'd perform the encryption of data blocks twice.
Option b): encrypt the top level LV (the one constructed from both cache and origin LV). This way ciphertext would be stored twice in cache PV and origin PV but the encryption would be performed only once.
Quoted 7 lines
gee, guys, thanks Ondrej,
this I was saying from the beginning did not work - option b
- does not work. I can Not encrypt top level cache pool LV. It does work with any other LV I have, but cache pool fails (like I said earlier) with:Command failed with code 22.
Could you post here 'lsblk' and 'lvs' output together with exact cryptsetup command you have executed to luksFormat your top level cached LV? Please add also --debug option among your original cryptsetup options.
Regards
Ondrej
$ lvs -a h200front LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert 0 h200front Cwi-aoC--- 9.10t [cache0] [0_corig] 100.00 8.18 0.00 [0_corig] h200front rwi-aoC--- 9.10t 100.00 [0_corig_rimage_0] h200front iwi-aor--- 1.82t [0_corig_rimage_1] h200front iwi-aor--- 1.82t [0_corig_rimage_2] h200front iwi-aor--- 1.82t [0_corig_rimage_3] h200front iwi-aor--- 1.82t [0_corig_rimage_4] h200front iwi-aor--- 1.82t [0_corig_rimage_5] h200front iwi-aor--- 1.82t [0_corig_rmeta_0] h200front ewi-aor--- 4.00m [0_corig_rmeta_1] h200front ewi-aor--- 4.00m [0_corig_rmeta_2] h200front ewi-aor--- 4.00m [0_corig_rmeta_3] h200front ewi-aor--- 4.00m [0_corig_rmeta_4] h200front ewi-aor--- 4.00m [0_corig_rmeta_5] h200front ewi-aor--- 4.00m [cache0] h200front Cwi---C--- 220.00g 100.00 8.18 0.00 [cache0_cdata] h200front Cwi-aor--- 220.00g 100.00 [cache0_cdata_rimage_0] h200front iwi-aor--- 220.00g [cache0_cdata_rimage_1] h200front iwi-aor--- 220.00g [cache0_cdata_rmeta_0] h200front ewi-aor--- 4.00m [cache0_cdata_rmeta_1] h200front ewi-aor--- 4.00m [cache0_cmeta] h200front ewi-aor--- 512.00m 100.00 [cache0_cmeta_rimage_0] h200front iwi-aor--- 512.00m [cache0_cmeta_rimage_1] h200front iwi-aor--- 512.00m [cache0_cmeta_rmeta_0] h200front ewi-aor--- 4.00m [cache0_cmeta_rmeta_1] h200front ewi-aor--- 4.00m [lvol0_pmspare] h200front ewi------- 512.00m
I cannot debug now as for now I've given up the idea to encrypt this LV, but I would say is should be easily reproducible (maybe even waste of time looking at my setup) I simply tried:
cryptsetup -v luksFormat /dev/mapper/h200front-0
and that works(ed) with "regurlar" LVs.
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 238.5G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 237.5G 0 part
├─SL-root 253:0 0 150G 0 lvm /
└─SL-home 253:1 0 10G 0 lvm
sdb 8:16 0 223.6G 0 disk
├─h200front-cache0_cdata_rmeta_0 253:4 0 4M 0 lvm
│ └─h200front-cache0_cdata 253:8 0 220G 0 lvm
│ └─h200front-0 253:29 0 9.1T 0 lvm
├─h200front-cache0_cdata_rimage_0 253:5 0 220G 0 lvm
│ └─h200front-cache0_cdata 253:8 0 220G 0 lvm
│ └─h200front-0 253:29 0 9.1T 0 lvm
├─h200front-cache0_cmeta_rmeta_0 253:9 0 4M 0 lvm
│ └─h200front-cache0_cmeta 253:13 0 512M 0 lvm
│ └─h200front-0 253:29 0 9.1T 0 lvm
└─h200front-cache0_cmeta_rimage_0 253:10 0 512M 0 lvm
└─h200front-cache0_cmeta 253:13 0 512M 0 lvm
└─h200front-0 253:29 0 9.1T 0 lvm
sdc 8:32 0 223.6G 0 disk
├─h200front-cache0_cdata_rmeta_1 253:6 0 4M 0 lvm
│ └─h200front-cache0_cdata 253:8 0 220G 0 lvm
│ └─h200front-0 253:29 0 9.1T 0 lvm
├─h200front-cache0_cdata_rimage_1 253:7 0 220G 0 lvm
│ └─h200front-cache0_cdata 253:8 0 220G 0 lvm
│ └─h200front-0 253:29 0 9.1T 0 lvm
├─h200front-cache0_cmeta_rmeta_1 253:11 0 4M 0 lvm
│ └─h200front-cache0_cmeta 253:13 0 512M 0 lvm
│ └─h200front-0 253:29 0 9.1T 0 lvm
└─h200front-cache0_cmeta_rimage_1 253:12 0 512M 0 lvm
└─h200front-cache0_cmeta 253:13 0 512M 0 lvm
└─h200front-0 253:29 0 9.1T 0 lvm
sdd 8:48 0 894.3G 0 disk
└─h300Int1-0 253:2 0 1.8T 0 lvm
└─h300Int1.0_crypt 253:27 0 1.8T 0
crypt /__.aLocalStorages/2
sde 8:64 0 894.3G 0 disk
└─h300Int1-0 253:2 0 1.8T 0 lvm
└─h300Int1.0_crypt 253:27 0 1.8T 0
crypt /__.aLocalStorages/2
sdf 8:80 0 447.1G 0 disk
└─h300Int0-0 253:3 0 1.3T 0 lvm
└─h300Int0.0_crypt 253:28 0 1.3T 0
crypt /__.aLocalStorages/1
sdg 8:96 0 447.1G 0 disk
└─h300Int0-0 253:3 0 1.3T 0 lvm
└─h300Int0.0_crypt 253:28 0 1.3T 0
crypt /__.aLocalStorages/1
sdh 8:112 0 447.1G 0 disk
└─h300Int0-0 253:3 0 1.3T 0 lvm
└─h300Int0.0_crypt 253:28 0 1.3T 0
crypt /__.aLocalStorages/1
sdi 8:128 0 1.8T 0 disk
├─h200front-0_corig_rmeta_0 253:14 0 4M 0 lvm
│ └─h200front-0_corig 253:26 0 9.1T 0 lvm
│ └─h200front-0 253:29 0 9.1T 0 lvm
└─h200front-0_corig_rimage_0 253:15 0 1.8T 0 lvm
└─h200front-0_corig 253:26 0 9.1T 0 lvm
└─h200front-0 253:29 0 9.1T 0 lvm
sdj 8:144 0 1.8T 0 disk
├─h200front-0_corig_rmeta_1 253:16 0 4M 0 lvm
│ └─h200front-0_corig 253:26 0 9.1T 0 lvm
│ └─h200front-0 253:29 0 9.1T 0 lvm
└─h200front-0_corig_rimage_1 253:17 0 1.8T 0 lvm
└─h200front-0_corig 253:26 0 9.1T 0 lvm
└─h200front-0 253:29 0 9.1T 0 lvm
sdk 8:160 0 1.8T 0 disk
├─h200front-0_corig_rmeta_2 253:18 0 4M 0 lvm
│ └─h200front-0_corig 253:26 0 9.1T 0 lvm
│ └─h200front-0 253:29 0 9.1T 0 lvm
└─h200front-0_corig_rimage_2 253:19 0 1.8T 0 lvm
└─h200front-0_corig 253:26 0 9.1T 0 lvm
└─h200front-0 253:29 0 9.1T 0 lvm
sdl 8:176 0 1.8T 0 disk
├─h200front-0_corig_rmeta_3 253:20 0 4M 0 lvm
│ └─h200front-0_corig 253:26 0 9.1T 0 lvm
│ └─h200front-0 253:29 0 9.1T 0 lvm
└─h200front-0_corig_rimage_3 253:21 0 1.8T 0 lvm
└─h200front-0_corig 253:26 0 9.1T 0 lvm
└─h200front-0 253:29 0 9.1T 0 lvm
sdm 8:192 0 1.8T 0 disk
├─h200front-0_corig_rmeta_4 253:22 0 4M 0 lvm
│ └─h200front-0_corig 253:26 0 9.1T 0 lvm
│ └─h200front-0 253:29 0 9.1T 0 lvm
└─h200front-0_corig_rimage_4 253:23 0 1.8T 0 lvm
└─h200front-0_corig 253:26 0 9.1T 0 lvm
└─h200front-0 253:29 0 9.1T 0 lvm
sdn 8:208 0 1.8T 0 disk
├─h200front-0_corig_rmeta_5 253:24 0 4M 0 lvm
│ └─h200front-0_corig 253:26 0 9.1T 0 lvm
│ └─h200front-0 253:29 0 9.1T 0 lvm
└─h200front-0_corig_rimage_5 253:25 0 1.8T 0 lvm
└─h200front-0_corig 253:26 0 9.1T 0 lvm
└─h200front-0 253:29 0 9.1T 0 lvm
sdo 8:224 0 9.1T 0 disk /__.aLocalStorages/3LV is working fine, being a host for all sorts of data, only cryptsetup does not work on it. many thanks.
Quoted 53 lines · Ondrej Kozina
Quoted 33 lines · lejeczek
Quoted 22 lines · Ondrej Kozina
Quoted 5 lines · lejeczek
whatever you might call it, it works, luks encrypting, opening & mounting @boot - so I only wonder (which was my question) why not cache pool LVs. Is it not supported... would be great if a developer sees this question, I'm not sure jut yet about filing a bug report.
In general LVM2 doesn't auto-activate or interpret unknown device types. LUKS header is considered unknown from LVM2 perspective. Simply put LVM2 doesn't understand LUKS header data. Not sure what you tried to do with cache pool LV, but in my opinion any effort to encrypt (live or detached) cache pool LV may end with severe data corruption...
As of now I think you have in general two options:
a) encrypt both PVs because obviously if you only encrypt the origin PV you end up with decrypted plaintext data stored in cache pool. Probably this is the exact scenario you were about to avoid?
Quoted 8 lines
Unfortunately a) is suboptimal with regard to performance since you'd perform the encryption of data blocks twice.
Option b): encrypt the top level LV (the one constructed from both cache and origin LV). This way ciphertext would be stored twice in cache PV and origin PV but the encryption would be performed only once.
Quoted 7 lines
gee, guys, thanks Ondrej,
this I was saying from the beginning did not work - option b
- does not work. I can Not encrypt top level cache pool LV. It does work with any other LV I have, but cache pool fails (like I said earlier) with:Command failed with code 22.
Could you post here 'lsblk' and 'lvs' output together with exact cryptsetup command you have executed to luksFormat your top level cached LV? Please add also --debug option among your original cryptsetup options.
Regards
Ondrej_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Thank you for the lsblk and lvs output. At least now we know that raid is somehow involved in the setup...
Quoted 4 lines · lejeczek
I cannot debug now as for now I've given up the idea to encrypt this LV, but I would say is should be easily reproducible (maybe even waste of time looking at my setup) I simply tried:
Well, without the debug output from cryptsetup I can't be of any more help, I'm sorry. It's like starring into crystal ball guessing what may have gone wrong. The -22 error code is general "wrong parameters" answer. I don't even know if cryptsetup has got into code path where it's supposed to open the device for write.
But let me give it one last shot. Did you by any chance end up with 22 right after the prompt:
"Are you sure? (Type uppercase yes):"
If so, are you sure that your answer was 'YES' instead of lowercase 'yes'?
Regards
Ondrej
lejeczek schreef op 29-08-2016 16:22:
Quoted 3 lines
I cannot debug now as for now I've given up the idea to encrypt this LV, but I would say is should be easily reproducible (maybe even waste of time looking at my setup)
I can definitely say I have encrypted a cached volume without issue.
I have had a disk with two cached volumes, if you want to know, both caches originating from the same SSD.
Main caches (main origin volume) were on a HDD in a simple regular non-thin LVM.
now when you say that - (I) believe it or not - I think it's possible. I cannot test it, for now the LV is in production but! I few days ago introduced an external usb keboard which has not LED indicators at all and now when I play with it it seems there might be problem on my f24, maybe the drivers(?). Gee, open source with all its greatness and the luck of officially Christened, own hardware(laptop) can be a reason of great frustration sometimes. I'll put together another cache pool some days and then shall provide clarification to my own frustration.
Quoted 24 lines · Ondrej Kozina
Thank you for the lsblk and lvs output. At least now we know that raid is somehow involved in the setup...
Quoted 4 lines · lejeczek
I cannot debug now as for now I've given up the idea to encrypt this LV, but I would say is should be easily reproducible (maybe even waste of time looking at my setup) I simply tried:
Well, without the debug output from cryptsetup I can't be of any more help, I'm sorry. It's like starring into crystal ball guessing what may have gone wrong. The -22 error code is general "wrong parameters" answer. I don't even know if cryptsetup has got into code path where it's supposed to open the device for write.
But let me give it one last shot. Did you by any chance end up with 22 right after the prompt:
"Are you sure? (Type uppercase yes):"
If so, are you sure that your answer was 'YES' instead of lowercase 'yes'?
Regards
Ondrej_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/