Device $dev is not a valid LUKS device.
Hi all,
I've been using dm-crypt for quite some years now, and every now and then I am very unpleasantly surprised by the message that my entire volume can no longer be read:
Device /dev/disk/by-path/ip-foo.bar:mainstorage-lun-1 is not a valid LUKS device.
In this case the iscsi server was rebooted while the volume was still mounted. The volume is used for making backups and no backups were currently made, so no actual writes got lost. Nevertheless after reboot it no longer recognizes my luks volume.
I've had this kind of malice before and at that time was unable to fix it. So in the end I gave up, created a new luks volume and started from scratch.
However I hope that is not how every future power failure is going to end up. Is there anything I can do to fix this volume or anything I can do to prevent such a simple event from causing such catastrophic results?
Kind regards,
Erik.
Hi Erik,
first, you can make a header-backup, see the FAQ. Second, you can investigate what destroys the header. The same thing could well destroy a filesystem, RAID-superblock, etc.
Or put differently: Something is very, very wrong in your set-up. Loss of the LUKS header can only happen if einther the user does it (directly or indirectly) or somethign is broken.
Arno
Quoted 29 lines · Erik Logtenberg
Hi all,
I've been using dm-crypt for quite some years now, and every now and then I am very unpleasantly surprised by the message that my entire volume can no longer be read:
Device /dev/disk/by-path/ip-foo.bar:mainstorage-lun-1 is not a valid LUKS device.
In this case the iscsi server was rebooted while the volume was still mounted. The volume is used for making backups and no backups were currently made, so no actual writes got lost. Nevertheless after reboot it no longer recognizes my luks volume.
I've had this kind of malice before and at that time was unable to fix it. So in the end I gave up, created a new luks volume and started from scratch.
However I hope that is not how every future power failure is going to end up. Is there anything I can do to fix this volume or anything I can do to prevent such a simple event from causing such catastrophic results?
Kind regards,
Erik.
_______________________________________________
dm-crypt mailing list
[email protected]
http://www.saout.de/mailman/listinfo/dm-crypt
Signature · 6 lines
Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: [email protected] GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718 ---- One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision. -- Bertrand Russell
Hi Arno,
Do you have some pointers on how I can investigate that?
In my previous setup, I had my luks device directly on top of my iscsi device. That worked great in the beginning, but after a power failure on the iscsi server, my volume was destroyed. Since the luks device was the only contents on the iscsi device, there was not much testing/investigating that I could do.
So I changed the setup: I created an LVM volume on my iscsi device, and the luks device inside the LVM volume. That way, I thought, I would at least know what part of the setup caused the malfunction. I mean, if the iscsi device itself would become corrupted, I wouldn't be able to activate the LVM volume anymore, but if dm-crypt were to make some mistake, I would expect to still be able to activate the LVM volume, but then not be able to read the luks volume.
Right? Well, it turns out I can still activate the LVM volume just fine, so apparently there are no media errors / overwritten headers, at least no overwritten LVM headers.
But then, after I activated the LVM volume, dm-crypt can't read the luks volume. So apparently -only- the luks volume is damaged in some way. So... how can I investigate this further?
Kind regards,
Erik.
Quoted 43 lines · Arno Wagner
Hi Erik,
first, you can make a header-backup, see the FAQ. Second, you can investigate what destroys the header. The same thing could well destroy a filesystem, RAID-superblock, etc.
Or put differently: Something is very, very wrong in your set-up. Loss of the LUKS header can only happen if einther the user does it (directly or indirectly) or somethign is broken.Arno
Quoted 29 lines · Erik Logtenberg
Hi all,
I've been using dm-crypt for quite some years now, and every now and then I am very unpleasantly surprised by the message that my entire volume can no longer be read:
Device /dev/disk/by-path/ip-foo.bar:mainstorage-lun-1 is not a valid LUKS device.
In this case the iscsi server was rebooted while the volume was still mounted. The volume is used for making backups and no backups were currently made, so no actual writes got lost. Nevertheless after reboot it no longer recognizes my luks volume.
I've had this kind of malice before and at that time was unable to fix it. So in the end I gave up, created a new luks volume and started from scratch.
However I hope that is not how every future power failure is going to end up. Is there anything I can do to fix this volume or anything I can do to prevent such a simple event from causing such catastrophic results?
Kind regards,
Erik.
_______________________________________________
dm-crypt mailing list
[email protected]
http://www.saout.de/mailman/listinfo/dm-crypt
Lol, whoops. I found out what was the cause of my problem. My mount script wasn't functioning quite as it should, it tried to luksOpen the iscsi device directly instead of the lvm volume. Argl.
Fixed my mount script, able to mount my backup volume after all. Nice.
Still, I think I'll make a backup of that header just to be sure, for future use.
Kind regards,
Erik.
Quoted 80 lines · Erik Logtenberg
Hi Arno,
Do you have some pointers on how I can investigate that?
In my previous setup, I had my luks device directly on top of my iscsi device. That worked great in the beginning, but after a power failure on the iscsi server, my volume was destroyed. Since the luks device was the only contents on the iscsi device, there was not much testing/investigating that I could do.
So I changed the setup: I created an LVM volume on my iscsi device, and the luks device inside the LVM volume. That way, I thought, I would at least know what part of the setup caused the malfunction. I mean, if the iscsi device itself would become corrupted, I wouldn't be able to activate the LVM volume anymore, but if dm-crypt were to make some mistake, I would expect to still be able to activate the LVM volume, but then not be able to read the luks volume.
Right? Well, it turns out I can still activate the LVM volume just fine, so apparently there are no media errors / overwritten headers, at least no overwritten LVM headers.
But then, after I activated the LVM volume, dm-crypt can't read the luks volume. So apparently -only- the luks volume is damaged in some way. So... how can I investigate this further?Kind regards,
Erik.
Quoted 10 lines · Arno Wagner
Hi Erik,
first, you can make a header-backup, see the FAQ. Second, you can investigate what destroys the header. The same thing could well destroy a filesystem, RAID-superblock, etc.
Or put differently: Something is very, very wrong in your set-up. Loss of the LUKS header can only happen if einther the user does it (directly or indirectly) or somethign is broken.Quoted 33 lines
Arno
Quoted 29 lines · Erik Logtenberg
Hi all,
I've been using dm-crypt for quite some years now, and every now and then I am very unpleasantly surprised by the message that my entire volume can no longer be read:
Device /dev/disk/by-path/ip-foo.bar:mainstorage-lun-1 is not a valid LUKS device.
In this case the iscsi server was rebooted while the volume was still mounted. The volume is used for making backups and no backups were currently made, so no actual writes got lost. Nevertheless after reboot it no longer recognizes my luks volume.
I've had this kind of malice before and at that time was unable to fix it. So in the end I gave up, created a new luks volume and started from scratch.
However I hope that is not how every future power failure is going to end up. Is there anything I can do to fix this volume or anything I can do to prevent such a simple event from causing such catastrophic results?
Kind regards,
Erik.
_______________________________________________
dm-crypt mailing list
[email protected]
http://www.saout.de/mailman/listinfo/dm-crypt_______________________________________________
dm-crypt mailing list
[email protected]
http://www.saout.de/mailman/listinfo/dm-crypt
Hi Erik,
basically the LUKS header (see FAQ for brief explanation of the on-disk format) gets overwritten. The question is by what. Often, looking at the data that is now there in place of the header guves a clue, sometimes not. So look at the start of the (former) LUKS device.
Quoted 18 lines · Erik Logtenberg
Hi Arno,
Do you have some pointers on how I can investigate that?
In my previous setup, I had my luks device directly on top of my iscsi device. That worked great in the beginning, but after a power failure on the iscsi server, my volume was destroyed. Since the luks device was the only contents on the iscsi device, there was not much testing/investigating that I could do.
So I changed the setup: I created an LVM volume on my iscsi device, and the luks device inside the LVM volume. That way, I thought, I would at least know what part of the setup caused the malfunction. I mean, if the iscsi device itself would become corrupted, I wouldn't be able to activate the LVM volume anymore, but if dm-crypt were to make some mistake, I would expect to still be able to activate the LVM volume, but then not be able to read the luks volume.
Quoted 3 lines
Right? Well, it turns out I can still activate the LVM volume just fine, so apparently there are no media errors / overwritten headers, at least no overwritten LVM headers.
I do not use LVM (complicates things which is bad IMO). I also failed to find any description of its superblock or the superblock location on the web, pretty bad. However, in an experiment, pvcreate put the header at 1k offset, while the LUKS header is at the beginning of the device. So whatever kills the LUKS header may leave the LVM superblock undamaged. It is interesting, however, that the LUKS header seems to have been damaged inside the LVM container.
Quoted 3 lines
But then, after I activated the LVM volume, dm-crypt can't read the luks volume. So apparently -only- the luks volume is damaged in some way. So... how can I investigate this further?
Look at what is there instead of the LUKS header. Then find out what put it there. I sould also submit that until you find out, you shpuld not trust the iSCSI device, something is clearly wring with it.
Arno
Quoted 49 lines · Arno Wagner
Quoted 10 lines
Hi Erik,
first, you can make a header-backup, see the FAQ. Second, you can investigate what destroys the header. The same thing could well destroy a filesystem, RAID-superblock, etc.
Or put differently: Something is very, very wrong in your set-up. Loss of the LUKS header can only happen if einther the user does it (directly or indirectly) or somethign is broken.Quoted 33 lines
Arno
Quoted 29 lines · Erik Logtenberg
Hi all,
I've been using dm-crypt for quite some years now, and every now and then I am very unpleasantly surprised by the message that my entire volume can no longer be read:
Device /dev/disk/by-path/ip-foo.bar:mainstorage-lun-1 is not a valid LUKS device.
In this case the iscsi server was rebooted while the volume was still mounted. The volume is used for making backups and no backups were currently made, so no actual writes got lost. Nevertheless after reboot it no longer recognizes my luks volume.
I've had this kind of malice before and at that time was unable to fix it. So in the end I gave up, created a new luks volume and started from scratch.
However I hope that is not how every future power failure is going to end up. Is there anything I can do to fix this volume or anything I can do to prevent such a simple event from causing such catastrophic results?
Kind regards,
Erik.
_______________________________________________
dm-crypt mailing list
[email protected]
http://www.saout.de/mailman/listinfo/dm-crypt_______________________________________________
dm-crypt mailing list
[email protected]
http://www.saout.de/mailman/listinfo/dm-crypt
Signature · 6 lines
Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: [email protected] GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718 ---- One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision. -- Bertrand Russell