Skip to contentSkip to conversation

Impossible or Possible to Securely Erase File on Btrfs?

DISCUSSION Kyle 8 participants 13 messages Mar 18, 2013 – Mar 20, 2013 Btrfs file system

lore
Share & tools
lore

Kyle

author
#0 lore raw

Hi,

After reading through the btrfs documentation I'm curious to know if it's possible to ever securely erase a file from a btrfs filesystem (or ZFS for that matter). On non-COW filesystems atop regular HDDs one can simply overwrite the file with zeros or random data using dd or some other tool and rest assured that the blocks which contained the sensitive information have been wiped. However on btrfs it would seem any such attempt would write the zeros/random data to a new location, leaving the old blocks with the sensitive data intact. Further, since specifying NOCOW is only possible for newly created files, there seems to be no way to overwrite the appropriate blocks short of deleting the associated file and then filling the entire free filesystem space with zeros/random data such that the old blocks are eventually overwritten. What's the verdict on this?

Regards,

Kyle

#1 lore raw
Quoted 3 lines · Kyle

After reading through the btrfs documentation I'm curious to know if it's possible to ever securely erase a file from a btrfs filesystem (or ZFS for that matter).

Not really.

It gets even worse for SSDs, because the SSD itself can be
effectively CoW, with old pages lurking away in the flash storage where (with a bit of physical persuasion of the hardware) they can be read. So you have the same problem on SSDs even with non-CoW filesystems.

Quoted 4 lines

On non-COW filesystems atop regular HDDs one can simply overwrite the file with zeros or random data using dd or some other tool and rest assured that the blocks which contained the sensitive information have been wiped.

Assuming that anything that's modified the file since its creation
hasn't written a copy and then overwritten it with rename(), of course...

Quoted 8 lines

However on btrfs it would seem any such attempt would write the zeros/random data to a new location, leaving the old blocks with the sensitive data intact. Further, since specifying NOCOW is only possible for newly created files, there seems to be no way to overwrite the appropriate blocks short of deleting the associated file and then filling the entire free filesystem space with zeros/random data such that the old blocks are eventually overwritten. What's the verdict on this?

Your analysis is pretty much correct, but should be expanded to a
whole load of other data-leakage paths in generalised storage systems. Overwriting file contents hasn't really been a reliable method of erasing files for many years (if ever).

Hugo.

Signature · 3 lines
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
   --- Great oxymorons of the world, no.  3: Military Intelligence ---   
#2 lore raw
Quoted 5 lines · Hugo Mills
Quoted 3 lines · Kyle

After reading through the btrfs documentation I'm curious to know if it's possible to ever securely erase a file from a btrfs filesystem (or ZFS for that matter).

Quoted 7 lines

Not really.

It gets even worse for SSDs, because the SSD itself can be
effectively CoW, with old pages lurking away in the flash storage where (with a bit of physical persuasion of the hardware) they can be read. So you have the same problem on SSDs even with non-CoW filesystems.

Encryption is the solution, either dmcrypt, encryptfs, or Opal SED.

With even consumer SSDs now, Opal 2.0 is increasingly common, so data on the SSD is already encrypted whether locked or not. By default the drive is unlocked, so the DEK is exposed but data on the chips is encrypted. Unfortunately I'm not aware of user space tools to manage Opal drives on linux, it's a side topic question but if anyone knows of such a tool I'd like to know about it. I don't think hdparm does, rather it manages the ATA Security features like Secure Erase, Secure Enhanced Erase, and I think it can also cause DEK regeneration on SED drives which effectively is encryption erase (removes the existing DEK, creates a new one, thereby rendering the drive effectively erased in seconds). But I don't think it manages KEKs and the locking/unlocking of the drive.

Booting from these Opal compliant SEDs is difficult because it requires firmware or bootloader support; and then also kernel and user space tools to support to lock/unlock when the computer is put to sleep/hibernation and then woken. For that use case it's necessary to use software encryption, dmcrypt/LUKS on Linux.

Another possibility for the OP's use case is encryptfs which encrypts above btrfs. The file contents are encrypted, but the file names and file system itself (and metadata) are not. In this case, deletion is functionally equivalent to having filled the blocks with random data (short of the DEK escaping into the wild).

Chris Murphy

author
#3 lore raw
Quoted 18 lines · Chris Murphy
Quoted 5 lines · Hugo Mills
Quoted 3 lines · Kyle

After reading through the btrfs documentation I'm curious to know if it's possible to ever securely erase a file from a btrfs filesystem (or ZFS for that matter).

Quoted 7 lines

Not really.

It gets even worse for SSDs, because the SSD itself can be
effectively CoW, with old pages lurking away in the flash storage where (with a bit of physical persuasion of the hardware) they can be read. So you have the same problem on SSDs even with non-CoW filesystems.

Encryption is the solution, either dmcrypt, encryptfs, or Opal SED.

Quoted 8 lines

With even consumer SSDs now, Opal 2.0 is increasingly common, so data on the SSD is already encrypted whether locked or not. By default the drive is unlocked, so the DEK is exposed but data on the chips is encrypted. Unfortunately I'm not aware of user space tools to manage Opal drives on linux, it's a side topic question but if anyone knows of such a tool I'd like to know about it. I don't think hdparm does, rather it manages the ATA Security features like Secure Erase, Secure Enhanced Erase, and I think it can also cause DEK regeneration on SED drives which effectively is encryption erase (removes the existing DEK, creates a new one, thereby rendering the drive effectively erased in seconds). But I don't think it manages KEKs and the locking/unlocking of the drive.

Booting from these Opal compliant SEDs is difficult because it requires firmware or bootloader support; and then also kernel and user space tools to support to lock/unlock when the computer is put to sleep/hibernation and then woken. For that use case it's necessary to use software encryption, dmcrypt/LUKS on Linux.

Another possibility for the OP's use case is encryptfs which encrypts above btrfs. The file contents are encrypted, but the file names and file system itself (and metadata) are not. In this case, deletion is functionally equivalent to having filled the blocks with random data (short of the DEK escaping into the wild).

Chris Murphy

I'm doubtful that encryption would be as effective as what Chris Mason was suggesting, which is to either find and use a disk with secure erase trim or rotate out the drives and securely erase them so that the data is destroyed entirely rather than encrypted.

Simply having the data exist on a disk, even if it's encrypted, still leaves it open to various attacks. For instance, multiple side channel attacks exist which could reveal the decryption key, permitting all data on the disk to be read including deleted but not yet overwritten files. Physical access to the machine may not even be required: A remote attacker able to exploit the machine and read the disk blocks would be able to recover the data as well.

Granted these types of attacks are rare, however ultimately the only way to secure unneeded sensitive data against a determined attacker is to destroy it outright. For encrypted data, it's true the decryption keys could be destroyed, thus essentially destroying the data (subject to the security of the cryptographic algorithm itself) however this would destroy the entire filesystem, not securely erase a single file.

Hugo made the point that a multitude of other data-leakage paths exist, and while this is true, at a minimum one should be able to destroy individual files beyond recovery so that should data leakage occur in the future, previously securely erased data is not recoverable. At least, in an ideal world.

For the time being I'll be thankful I'm not a person tasked with protecting/erasing such highly sensitive data. Aside from the confidence-inspiring crunching of decommissioned hard disks going through an industrial shredder, it must be a nightmare.

-Kyle

#4 lore raw
Quoted 2 lines · Kyle

I'm doubtful that encryption would be as effective as what Chris Mason was suggesting, which is to either find and use a disk with secure erase trim or rotate out the drives and securely erase them so that the data is destroyed entirely rather than encrypted.

Well, I'll argue it's more effective in that it's more likely to be used than it is to be cracked in the time frame of the data mattering if the proper setup is being used.

Quoted 2 lines

Simply having the data exist on a disk, even if it's encrypted, still leaves it open to various attacks. For instance, multiple side channel attacks exist which could reveal the decryption key, permitting all data on the disk to be read including deleted but not yet overwritten files.

Not if you're using LUKS with either a high entropy passphrase, or increasing PBKDF2 iterations, or both. LUKS and at least Opal SEDs don't store keys or passphrases in plaintext. If you're using plain dmcrypt (without LUKS) then you be concerned.

Quoted 1 line

Physical access to the machine may not even be required: A remote attacker able to exploit the machine and read the disk blocks would be able to recover the data as well.

If it's an HDD, or the SSD's garbage collection after trim is issue is slow, yes anyone who has either file system or block level access can obtain unencrypted data from a mapped encrypted block device. But if the fall back position to a remote take over is counting on deleted files being irrecoverable, well, you still have a remote attacker stealing visible files. Chances are if the deleted files are that sensitive, the non-deleted files are too. So I don't see the gain of going to extraordinary lengths to zero the data in released blocks for active use drives.

Quoted 2 lines

Granted these types of attacks are rare, however ultimately the only way to secure unneeded sensitive data against a determined attacker is to destroy it outright.

This is probably specious. It's sorta like saying the only way to secure a computer containing sensitive data is to disconnect it from the internet. Or from the wall outlet.

Quoted 1 line

Hugo made the point that a multitude of other data-leakage paths exist, and while this is true, at a minimum one should be able to destroy individual files beyond recovery so that should data leakage occur in the future, previously securely erased data is not recoverable. At least, in an ideal world.

Well at the moment we are SOL on that front with either SSDs or COW file systems. You'd need to use HDDs and a file system that'll support overwriting files with random data and then marking them as deleted.

I think the overwhelming concern is someone gaining physical access to the drive: theft of a laptop, or disposal of a dead drive (for which ATA security erase unit commands aren't going to work). And for that encryption is quite useful and important.

Chris Murphy

#5 lore raw
In reply to Kyle

Quoting Kyle (2013-03-18 14:15:17)

Quoted 15 lines

Hi,

After reading through the btrfs documentation I'm curious to know if it's possible to ever securely erase a file from a btrfs filesystem (or ZFS for that matter). On non-COW filesystems atop regular HDDs one can simply overwrite the file with zeros or random data using dd or some other tool and rest assured that the blocks which contained the sensitive information have been wiped. However on btrfs it would seem any such attempt would write the zeros/random data to a new location, leaving the old blocks with the sensitive data intact. Further, since specifying NOCOW is only possible for newly created files, there seems to be no way to overwrite the appropriate blocks short of deleting the associated file and then filling the entire free filesystem space with zeros/random data such that the old blocks are eventually overwritten. What's the verdict on this?

We don't do this now for other reasons mentioned in the thread. The best path to get here is to use trim, and to find a device that supports a secure erase trim (I don't know if this even exists, sorry).

Outside of that, the only way to do this securely is to delete the files, rotate in new drives, rotate out the old drives, and run a full drive secure delete (not very practical).

-chris

#6 lore raw

Would it make sense for btrfs to support a write zeros to empty space erase? I know it would be slow as it would have to write to all the free space in the file system but it would be useful.

It's probably pretty far down the priority list for development though I expect.

Quoted 25 lines · Chris Mason

Quoting Kyle (2013-03-18 14:15:17)

Quoted 15 lines

Hi,

After reading through the btrfs documentation I'm curious to know if it's possible to ever securely erase a file from a btrfs filesystem (or ZFS for that matter). On non-COW filesystems atop regular HDDs one can simply overwrite the file with zeros or random data using dd or some other tool and rest assured that the blocks which contained the sensitive information have been wiped. However on btrfs it would seem any such attempt would write the zeros/random data to a new location, leaving the old blocks with the sensitive data intact. Further, since specifying NOCOW is only possible for newly created files, there seems to be no way to overwrite the appropriate blocks short of deleting the associated file and then filling the entire free filesystem space with zeros/random data such that the old blocks are eventually overwritten. What's the verdict on this?

We don't do this now for other reasons mentioned in the thread. The best path to get here is to use trim, and to find a device that supports a secure erase trim (I don't know if this even exists, sorry).

Outside of that, the only way to do this securely is to delete the files, rotate in new drives, rotate out the old drives, and run a full drive secure delete (not very practical).

Quoted 5 lines

-chris
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
Gareth Pye
Level 2 Judge, Melbourne, Australia
Australian MTG Forum: mtgau.com [email protected] - www.rockpaperdynamite.wordpress.com "Dear God, I would like to file a bug report"

#7 lore raw
Quoted 5 lines · Gareth Pye

Would it make sense for btrfs to support a write zeros to empty space erase? I know it would be slow as it would have to write to all the free space in the file system but it would be useful.

It's probably pretty far down the priority list for development though I expect.

I have prototyped this some time ago

http://repo.or.cz/w/linux-2.6/btrfs-unstable.git/commit/55ec5c00022be8f2bbed06b99b5f4be5832a5451

It reuses the trim code that already traverses the free space but calls blkdev_issue_zeroout instead. The overwrite pattern could be extended to random data, some posion patterns (like leaf that has 0 items and could be recognized in the code to catch misdirected writes; similarly for tree node blocks), or the secure discard.

(I haven't resolved what to do if there's a mixture of HDDs and SSDs, rewriting the free space has different impact on the underlying devices.)

The existing FITRIM ioctl structure fstrim_range does not contain any spare bytes so either we need to go with own ioctl or propose FITRIM_V2.

david

#8 lore raw
In reply to Chris Mason
Quoted 4 lines · Chris Mason

We don't do this now for other reasons mentioned in the thread. The best path to get here is to use trim, and to find a device that supports a secure erase trim (I don't know if this even exists, sorry).

I'm not finding a reference to "secure erase trim". In any case I wouldn't expect it to differ from trim. The requirement for ATA SECURITY ERASE UNIT is to replace LBA 0 to READ NATIVE MAX contents with either 0s or 1s.

ENHANCED SECURITY ERASE UNIT additional requires data overwrite, including sectors not in use. But there's no requirement for what kind of data is used for overwriting.

Chris Murphy

#10 lore raw
Quoted 7 lines · David Sterba
Quoted 2 lines · Chris Murphy

I'm not finding a reference to "secure erase trim". In any case I wouldn't expect it to differ from trim.

There's a 'secure discard' trim command in linux,

http://lkml.indiana.edu/hypermail/linux/kernel/1007.1/02355.html

"Secure discard is the same as discard except that all copies of the discarded sectors (perhaps created by garbage collection) must also be erased."

The erase mechanism seems to be the same "data set management" trim in ACS-2 in either case.

Chris Murphy

#11 lore raw
In reply to Kyle

Hi,

just reading chattr manpage..

Quoted 15 lines · On Monday 18 March 2013 14:15:17 you wrote:

Hi,

After reading through the btrfs documentation I'm curious to know if it's possible to ever securely erase a file from a btrfs filesystem (or ZFS for that matter). On non-COW filesystems atop regular HDDs one can simply overwrite the file with zeros or random data using dd or some other tool and rest assured that the blocks which contained the sensitive information have been wiped. However on btrfs it would seem any such attempt would write the zeros/random data to a new location, leaving the old blocks with the sensitive data intact. Further, since specifying NOCOW is only possible for newly created files, there seems to be no way to overwrite the appropriate blocks short of deleting the associated file and then filling the entire free filesystem space with zeros/random data such that the old blocks are eventually overwritten. What's the verdict on this?

what would chattr +s do?

"
When a file with the `s' attribute set is deleted, its blocks are zeroed and written back to the disk. Note: please make sure to read the bugs and limitations section at the end of this document. "

Nice spring to all of you! :)
Mark

Quoted 8 lines

Regards,

Kyle
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 1 line
Marek Otahal :o)
#12 lore raw

Am Dienstag, 19. März 2013 schrieb Marek Otahal:

Quoted 21 lines

Hi,

just reading chattr manpage..

Quoted 15 lines · On Monday 18 March 2013 14:15:17 you wrote:

Hi,

After reading through the btrfs documentation I'm curious to know if it's possible to ever securely erase a file from a btrfs filesystem (or ZFS for that matter). On non-COW filesystems atop regular HDDs one can simply overwrite the file with zeros or random data using dd or some other tool and rest assured that the blocks which contained the sensitive information have been wiped. However on btrfs it would seem any such attempt would write the zeros/random data to a new location, leaving the old blocks with the sensitive data intact. Further, since specifying NOCOW is only possible for newly created files, there seems to be no way to overwrite the appropriate blocks short of deleting the associated file and then filling the entire free filesystem space with zeros/random data such that the old blocks are eventually overwritten. What's the verdict on this?

Quoted 8 lines

what would chattr +s do?

"
When a file with the `s' attribute set is deleted, its blocks are zeroed and written back to the disk. Note: please make sure to read the bugs and limitations section at the end of this document. "

Nice spring to all of you! :)

Did you read on as suggested?

BUGS AND LIMITATIONS
The c', 's', and u' attributes are not honored by the ext2 and
ext3 filesystems as implemented in the current mainline Linux kernels.

But well question still stands: Does BTRFS honor it?

My bet is: It doesn´t.

Thanks,

Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7