Skip to contentSkip to conversation

Hosed encrypted drive. Is disaster recovery possible?

dm-crypt Willie 5 participants 12 messages Aug 1, 2010 – Aug 2, 2010

lore
Share & tools
lore

Willie

author
#0 lore raw

Evening all,

I'm not very hopeful of a positive response, but having just made my worst mistake in thirty years of computing I thought this would be where most of the relevant knowledge is.

I have an external 1.5TB Seagate drive, encrypted with dm-crypt/luks and formatted xfs.

In a state of dog-tiredness, thinking I was pointing at a USB stick, I have inadvertently wiped a few hundred MB of the beginning of this disk with:

dd if=./archlinux-2010.05-netinstall-i686.iso of=/dev/sdc

My question, as you might guess - is there any possibility of recovering the vast amount of data still on the drive? I could do it with an unencrypted disk, but I have no idea how to proceed in this case.

Thanks for any suggestions. (I've managed not to cry so far...)

Willie

Signature · 2 lines
http://www.fastmail.fm - Accessible with your email software
                          or over the web
#1 lore raw
Quoted 3 lines · Willie

My question, as you might guess - is there any possibility of recovering the vast amount of data still on the drive? I could do it with an unencrypted disk, but I have no idea how to proceed in this case.

If you have no backups of the dm-crypt key (and in the case of LUKS: the LUKS header + the passphrase required to decrypt the volume key) you have no chance (at least unless the cipher you've used is broken ;) ).

Cheers,
Chris.

#2 lore raw
In reply to Willie
Quoted 15 lines · Willie

Evening all,

I'm not very hopeful of a positive response, but having just made my worst mistake in thirty years of computing I thought this would be where most of the relevant knowledge is.

I have an external 1.5TB Seagate drive, encrypted with dm-crypt/luks and formatted xfs.

In a state of dog-tiredness, thinking I was pointing at a USB stick, I have inadvertently wiped a few hundred MB of the beginning of this disk with:

   dd if=./archlinux-2010.05-netinstall-i686.iso of=/dev/sdc
Quoted 7 lines

My question, as you might guess - is there any possibility of recovering the vast amount of data still on the drive? I could do it with an unencrypted disk, but I have no idea how to proceed in this case.

Thanks for any suggestions. (I've managed not to cry so far...)

Willie

Hi Willie,

sorry, but you will have wiped the salt in the header, which makes recovery impossible. You will also have wiped all keys (they take about the first 8.5MB), which again does make recovery impossible. In fact, any recovery from this would mean that LUKS is badly broken security-wise.

The only protection against this type of error is (besides a conventional backup), a header backup, see the FAQ at
http://code.google.com/p/cryptsetup/wiki/FrequentlyAskedQuestions

I did something similar recently, (tired and thinking I was blanking an USB stick), fortunately I had a backup of the whole disk. But the lession to me was: Hands away from dd and family when tired.

Arno

Signature · 7 lines
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: [email protected] 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 
author
#3 lore raw
Quoted 36 lines · Arno Wagner
Quoted 15 lines · Willie

Evening all,

I'm not very hopeful of a positive response, but having just made my worst mistake in thirty years of computing I thought this would be where most of the relevant knowledge is.

I have an external 1.5TB Seagate drive, encrypted with dm-crypt/luks and formatted xfs.

In a state of dog-tiredness, thinking I was pointing at a USB stick, I have inadvertently wiped a few hundred MB of the beginning of this disk with:

   dd if=./archlinux-2010.05-netinstall-i686.iso of=/dev/sdc
Quoted 7 lines

My question, as you might guess - is there any possibility of recovering the vast amount of data still on the drive? I could do it with an unencrypted disk, but I have no idea how to proceed in this case.

Thanks for any suggestions. (I've managed not to cry so far...)

Willie

Hi Willie,

sorry, but you will have wiped the salt in the header, which makes recovery impossible. You will also have wiped all keys (they take about the first 8.5MB), which again does make recovery impossible. In fact, any recovery from this would mean that LUKS is badly broken security-wise.

The only protection against this type of error is (besides a conventional backup), a header backup, see the FAQ at
http://code.google.com/p/cryptsetup/wiki/FrequentlyAskedQuestions

Quoted 6 lines

I did something similar recently, (tired and thinking I was blanking an USB stick), fortunately I had a backup of the whole disk. But the lession to me was: Hands away from dd and family when tired.

Arno

Oh well. Some you lose. Chin up. Onward and upward, etc etc...

Thanks for the replies lads.

Signature · 1 line
http://www.fastmail.fm - IMAP accessible web-mail
#4 lore raw
Quoted 2 lines · Willie

Oh well. Some you lose. Chin up. Onward and upward, etc etc...

That's the spirit. Only those doing nothing of importance do not screw up sometimes.

Arno

Signature · 7 lines
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: [email protected] 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 
author
#5 lore raw
Quoted 10 lines · Arno Wagner
Quoted 2 lines · Willie

Oh well. Some you lose. Chin up. Onward and upward, etc etc...

That's the spirit. Only those doing nothing of importance do not screw up sometimes.

Arno

Well, it's kind of strange. I aborted the dd as soon as I became aware of my moment of crass stupidity and I can still see part of the directory structure, with quite a few file names in there showing proper file sizes, but these seem to be empty of data.

I still have "/dev/mapper/udisks-luks-uuid-c1a534b4-d1ba-40d0-adb8-6d2490f06ade-uid9330 on /media/Seagate_1.5 type xfs (rw,nosuid,nodev,uhelper=udisks)", and obviously I have the pass phrase.

I suppose it's the key that's gone, and there's the kicker, right?

Signature · 1 line
http://www.fastmail.fm - Email service worth paying for. Try it for free
#6 lore raw

Is this LUKS? If it is, it should refuse to open becaue of missing header. If it is plain dm--crypt, you just lost some data at the beginning of the disk and can use the usual recovery methods...

Arno

Quoted 36 lines · Willie
Quoted 10 lines · Arno Wagner
Quoted 2 lines · Willie

Oh well. Some you lose. Chin up. Onward and upward, etc etc...

That's the spirit. Only those doing nothing of importance do not screw up sometimes.

Arno

Well, it's kind of strange. I aborted the dd as soon as I became aware of my moment of crass stupidity and I can still see part of the directory structure, with quite a few file names in there showing proper file sizes, but these seem to be empty of data.

I still have "/dev/mapper/udisks-luks-uuid-c1a534b4-d1ba-40d0-adb8-6d2490f06ade-uid9330 on /media/Seagate_1.5 type xfs (rw,nosuid,nodev,uhelper=udisks)", and obviously I have the pass phrase.

I suppose it's the key that's gone, and there's the kicker, right?

Signature · 6 lines
http://www.fastmail.fm - Email service worth paying for. Try it for free

_______________________________________________
dm-crypt mailing list
[email protected]
http://www.saout.de/mailman/listinfo/dm-crypt
Signature · 7 lines
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: [email protected] 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 
#7 lore raw
In reply to Willie
Quoted 4 lines · Willie

I still have "/dev/mapper/udisks-luks-uuid-c1a534b4-d1ba-40d0-adb8-6d2490f06ade-uid9330 on /media/Seagate_1.5 type xfs (rw,nosuid,nodev,uhelper=udisks)", and obviously I have the pass phrase.

If you have live mapping still, you are not lost completely yet. Do not reboot! First run "dmsetup table --showkeys" and "dmsetup info -c" and store the full mapping to some file.

If you see dm-crypt mapping there mapped to proper drive, you can still recreate LUKS header with some some magic.

(If you have saved that dmsetup mapping, I'll describe hot to save you data
- still depends how many of fs was overwritten.)

If you want help with that, paste here "dmsetup table" (*without* using --showkeys, we do not want see your master key:-). For recovery you will need to know that key, so be sure you have full table with key stored as written above.

(Table is created according to LUKS header which is lost, so after reboot you are lost completely. BTW Passphrase will not help here at all.)

Milan

#8 lore raw

ma, 2010-08-02 kello 11:58 +0200, Milan Broz kirjoitti:

Quoted 3 lines

If you have live mapping still, you are not lost completely yet. Do not reboot! First run "dmsetup table --showkeys" and "dmsetup info -c" and store the full mapping to some file.

_After_ doing that to have the mapping safe, it occurs to one that if the mapping is indeed live, you could just dump the decrypted block device image elsewhere (either re-encrypted or not). This would reduce the problem into a generic filesystem recovery one, with no need to tinker with reconstructing the original mapping.

Signature · 2 lines
Mikko Rauhala <[email protected]>
University of Helsinki
#9 lore raw

[dm-crypt] How to gather LUKS parameters from active device (if LUKS header lost)

In reply to Milan Broz
Quoted 2 lines · Milan Broz

If you see dm-crypt mapping there mapped to proper drive, you can still recreate LUKS header with some some magic.

Well, here is the idea how to reconstruct LUKS header from active mapping if header is lost but mapping is still active.
(Note: if device is not active, recovery is impossible).

- it will change LUKS UUID!
- no passphrase needed, it asks for new one (root access required of course)
- cryptsetup 1.1.x required.

Do not save master key file (second param) to unencrypted filesystem!

I'll add something similar to cryptsetup distro into DOC install, for now take this as an idea - see attached script (it will not touch device, only saves master key to file and print required parameters for cryptsetup).

BEWARE: NO GUARANTEES AT ALL. NOT PROPERLY TESTED.

Example:
If you have mapped device named "luks_sdb", script will produce this:

# <script> luks_sdb /mnt/safedisk/sdb_master_key

Generating master key to file /mnt/safedisk/sdb_master_key.
You can now try to reformat LUKS device using:
cryptsetup luksFormat -c aes-cbc-essiv:sha256 -s 256 --align-payload=2056 --master-key-file=/mnt/safedisk/sdb_master_key /dev/sdb

Milan

[---cut here---]
#!/bin/bash

# Try to get LUKS info and master key from active mapping and prepare parameters for cryptsetup" # (C) 2010 Milan Broz <[email protected]>

fail() { echo -e $1 ; exit 1 ; }
field() { echo $(dmsetup table --target crypt --showkeys $DEVICE | cut -d' ' -f$1) ; } field_cryptsetup() { echo $(cryptsetup status $DEVICE | grep $1 | sed "s/.*$1:\s*//;s/\ .*//") ; }

which xxd >/dev/null || fail "You need xxd (part of vim package) installed to convert key."

[ -z "$2" ] && fail "LUKS header from active mapping, use:\n $0 crypt_mapped_device mk_file_name";

DEVICE=$1
MK_FILE=$2

[ -z "$(field 4)" ] && fail "Mapping $1 not active or it is not crypt target."

CIPHER=$(field_cryptsetup cipher)
OFFSET=$(field_cryptsetup offset)
REAL_DEVICE=$(field_cryptsetup device)
KEY_SIZE=$(field_cryptsetup keysize)
KEY=$(field 5)

[ -z "$CIPHER" -o -z "$OFFSET" -o "$OFFSET" -le 383 -o -z "$KEY" ] && fail "Incompatible device, sorry."

echo "Generating master key to file $MK_FILE."
echo -E -n $KEY| xxd -r -p >$MK_FILE

echo "You can now try to reformat LUKS device using:"
echo " cryptsetup luksFormat -c $CIPHER -s $KEY_SIZE --align-payload=$OFFSET --master-key-file=$MK_FILE $REAL_DEVICE"

author
#10 lore raw

Re: [dm-crypt] How to gather LUKS parameters from active device (if LUKS header lost)

Quoted 73 lines · Milan Broz
Quoted 2 lines · Milan Broz

If you see dm-crypt mapping there mapped to proper drive, you can still recreate LUKS header with some some magic.

Well, here is the idea how to reconstruct LUKS header from active mapping if header is lost but mapping is still active.
(Note: if device is not active, recovery is impossible).

- it will change LUKS UUID!
- no passphrase needed, it asks for new one (root access required of course)
- cryptsetup 1.1.x required.

Do not save master key file (second param) to unencrypted filesystem!

I'll add something similar to cryptsetup distro into DOC install, for now take this as an idea - see attached script (it will not touch device,
only saves master key to file and print required parameters for cryptsetup).

BEWARE: NO GUARANTEES AT ALL. NOT PROPERLY TESTED.

Example:
If you have mapped device named "luks_sdb", script will produce this:

# <script> luks_sdb /mnt/safedisk/sdb_master_key

Generating master key to file /mnt/safedisk/sdb_master_key. You can now try to reformat LUKS device using:
cryptsetup luksFormat -c aes-cbc-essiv:sha256 -s 256 --align-payload=2056 --master-key-file=/mnt/safedisk/sdb_master_key /dev/sdb

Milan

[---cut here---]
#!/bin/bash

# Try to get LUKS info and master key from active mapping and prepare parameters for cryptsetup"
# (C) 2010 Milan Broz <[email protected]>

fail() { echo -e $1 ; exit 1 ; }
field() { echo $(dmsetup table --target crypt --showkeys $DEVICE | cut -d' ' -f$1) ; }
field_cryptsetup() { echo $(cryptsetup status $DEVICE | grep $1 | sed "s/.*$1:\s*//;s/\ .*//") ; }

which xxd >/dev/null || fail "You need xxd (part of vim package) installed to convert key."

[ -z "$2" ] && fail "LUKS header from active mapping, use:\n $0 crypt_mapped_device mk_file_name";

DEVICE=$1
MK_FILE=$2

[ -z "$(field 4)" ] && fail "Mapping $1 not active or it is not crypt target."

CIPHER=$(field_cryptsetup cipher)
OFFSET=$(field_cryptsetup offset)
REAL_DEVICE=$(field_cryptsetup device)
KEY_SIZE=$(field_cryptsetup keysize)
KEY=$(field 5)

[ -z "$CIPHER" -o -z "$OFFSET" -o "$OFFSET" -le 383 -o -z "$KEY" ] && fail "Incompatible device, sorry."

Quoted 7 lines

echo "Generating master key to file $MK_FILE."
echo -E -n $KEY| xxd -r -p >$MK_FILE

echo "You can now try to reformat LUKS device using:" echo " cryptsetup luksFormat -c $CIPHER -s $KEY_SIZE --align-payload=$OFFSET --master-key-file=$MK_FILE $REAL_DEVICE"

It gets worse and worse: I go to work, come back and my woman has turned off the computer. Whatever I was seeing earlier today is no longer there
- just the iso image I wrote to the disk.

I think I'm stuffed, but very very grateful for the helpful replies here.

Signature · 2 lines
http://www.fastmail.fm - Faster than the air-speed velocity of an
                          unladen european swallow
#11 lore raw

Re: [dm-crypt] How to gather LUKS parameters from active device (if LUKS header lost)

In reply to Milan Broz

Hmm, this was still mounted? Ah, I see. Another thing to add to my list to things not to do when tired: Giving advice in critical situations. Sorry.

One thing you can do before trying Milan's instructions is to make an image backup (with dd/dd_rescue) of the decrypted device, i.e. the device in /dev/mapper/<something>.
That will fix the current state in case something goes wrong and you can do conventional recovery on the image.

Arno

Quoted 73 lines · Milan Broz
Quoted 2 lines · Milan Broz

If you see dm-crypt mapping there mapped to proper drive, you can still recreate LUKS header with some some magic.

Well, here is the idea how to reconstruct LUKS header from active mapping if header is lost but mapping is still active.
(Note: if device is not active, recovery is impossible).

- it will change LUKS UUID!
- no passphrase needed, it asks for new one (root access required of course)
- cryptsetup 1.1.x required.

Do not save master key file (second param) to unencrypted filesystem!

I'll add something similar to cryptsetup distro into DOC install, for now take this as an idea - see attached script (it will not touch device, only saves master key to file and print required parameters for cryptsetup).

BEWARE: NO GUARANTEES AT ALL. NOT PROPERLY TESTED.

Example:
If you have mapped device named "luks_sdb", script will produce this:

# <script> luks_sdb /mnt/safedisk/sdb_master_key

Generating master key to file /mnt/safedisk/sdb_master_key.
You can now try to reformat LUKS device using:
cryptsetup luksFormat -c aes-cbc-essiv:sha256 -s 256 --align-payload=2056 --master-key-file=/mnt/safedisk/sdb_master_key /dev/sdb

Milan

[---cut here---]
#!/bin/bash

# Try to get LUKS info and master key from active mapping and prepare parameters for cryptsetup" # (C) 2010 Milan Broz <[email protected]>

fail() { echo -e $1 ; exit 1 ; }
field() { echo $(dmsetup table --target crypt --showkeys $DEVICE | cut -d' ' -f$1) ; } field_cryptsetup() { echo $(cryptsetup status $DEVICE | grep $1 | sed "s/.*$1:\s*//;s/\ .*//") ; }

which xxd >/dev/null || fail "You need xxd (part of vim package) installed to convert key."

[ -z "$2" ] && fail "LUKS header from active mapping, use:\n $0 crypt_mapped_device mk_file_name";

DEVICE=$1
MK_FILE=$2

[ -z "$(field 4)" ] && fail "Mapping $1 not active or it is not crypt target."

CIPHER=$(field_cryptsetup cipher)
OFFSET=$(field_cryptsetup offset)
REAL_DEVICE=$(field_cryptsetup device)
KEY_SIZE=$(field_cryptsetup keysize)
KEY=$(field 5)

[ -z "$CIPHER" -o -z "$OFFSET" -o "$OFFSET" -le 383 -o -z "$KEY" ] && fail "Incompatible device, sorry."

echo "Generating master key to file $MK_FILE."
echo -E -n $KEY| xxd -r -p >$MK_FILE

echo "You can now try to reformat LUKS device using:"
echo " cryptsetup luksFormat -c $CIPHER -s $KEY_SIZE --align-payload=$OFFSET --master-key-file=$MK_FILE $REAL_DEVICE"

_______________________________________________
dm-crypt mailing list
[email protected]
http://www.saout.de/mailman/listinfo/dm-crypt

Signature · 7 lines
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: [email protected] 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier