Skip to contentSkip to conversation

PATCH - ext2fs privacy (i.e. secure deletion) patch

PATCH the grugq 10 participants 44 messages Jan 28, 2004 – Feb 18, 2004

Subsystems (2)
lore
Share & tools
lore

the grugq

author
#0 lore raw

Hi,

I've written a quick patch to the ext2fs which ensures that inodes, data
blocks, and directory entries are overwritten with null bytes when
they are deleted. The patch adds a couple of inline functions to overwrite the appropriate data with zeroes. Its quite simple and self-explanitory. Both attached patches are to a vanilla 2.4.24 kernel.

inodes: ext2_delete_inode() now uses the inline function delete_inode() to mark the on-disk inode as 'deleted'. If the privacy option is not enabled, then the i_dtime is set as per usual operation, otherwise the meta-data are all set to 0. After the inode has been truncated, the i_data is set to 0 by delete_blocks() and the inode is written to disk again.

Splitting the delete_inode() and delete_blocks() functions is not ideal, I would prefer that one function remove the meta-data and the inode be written out just once. The problem is that ext2_truncate() needs the i_data information from the struct inode, and we want to remove this information from the inode on the disk. To ensure that the i_data is 0 by the time it hits the disk, I write the inode to disk twice, which is not optimal.

directory entries: ext2_delete_entry() now uses the inline function mark_delete_dir() to indicate that the directory entry is no longer valid. If the privacy patch is not enabled, then the dir->inode is set to 0 as per normal operation, otherwise the entry is memset 0. Very straightforward.

blocks: ext2_free_blocks() now uses the inline function destroy_block() before it updates the group meta-data. The destroy_block() function overwrites the block with zeroes. I'm not sure that it is implemented correctly, I don't know anything about buffer_heads. The block is only overwritten with a single pass of zeroes. It is, of course, possible to have a full 37 pass overwriter with full bit toggling maps and all that, but I think it might make more sense to add a new option for that, something like: CONFIG_EXT2_FS_PRIVACY_EXTREME_PARANOIA. For preventing simple forensic analysis (that is, what the police, employers and even most government agencies do) a single pass with zeroes is sufficient. Users can use userland tools like wipe or srm for more thorough overwriting.

There is also a patch to ext3fs, which adds the same functionality (in the same way) as the ext2fs privacy patch. However, the main problem with privacy on the ext3fs is the journal. I am not familiar enough with how the journal transactions are handled to implement what seems to me the obvious solution: when a transaction is complete (i.e. doesn't need to be replayed by fsck) overwrite it with null bytes. If done correctly, it is possible to leave sufficient information for fsck to find the right transaction to begin replaying, without leaving any data which would be of use to a forensic analyst.

Given the existing global political climate, privacy control over a file system seems like a no-brainer to me. Full privacy options would include things like mounting disks with noatime, and possibly normalizing the mtime and ctime of all inodes (setting everthing to an arbitrary date, e.g. 0). Certainly this will break tools like make which require atime, but it would provide even less information to a forensic analyst examining the file system.

peace,

--gq

ps. apologies for posting to the kernel mailing list, but I'm not sure who is the maintainer for ext2/3.

fs/Config.in-ext2-privacy renamed from fs/Config.in +1 -0
@@ -92,6 +92,7 @@
 tristate 'ROM file system support' CONFIG_ROMFS_FS
 
 tristate 'Second extended fs support' CONFIG_EXT2_FS
+dep_mbool '  ext2 privacy support' CONFIG_EXT2_FS_PRIVACY $CONFIG_EXT2_FS
 
 tristate 'System V/Xenix/V7/Coherent file system support' CONFIG_SYSV_FS
 
fs/ext2-privacy/balloc.c renamed from fs/ext2/balloc.c +19 -0
@@ -247,6 +247,23 @@
 	return slot;
 }
 
+static inline void destroy_block(struct inode *inode, unsigned long block)
+{
+#ifdef CONFIG_EXT2_FS_PRIVACY
+	struct buffer_head	* bh;
+
+	bh = sb_getblk(inode->i_sb, block);
+
+	memset(bh->b_data, 0x00, bh->b_size);
+
+	mark_buffer_dirty(bh);
+	wait_on_buffer(bh);
+	brelse(bh);
+
+	return;
+#endif
+}
+
 /* Free given blocks, update quota and i_blocks field */
 void ext2_free_blocks (struct inode * inode, unsigned long block,
 		       unsigned long count)
@@ -319,6 +336,8 @@
 				   "bit already cleared for block %lu", block);
 		else {
 			DQUOT_FREE_BLOCK(inode, 1);
+
+			destroy_block(sb, block);
 			gdp->bg_free_blocks_count =
 				cpu_to_le16(le16_to_cpu(gdp->bg_free_blocks_count)+1);
 			es->s_free_blocks_count =
fs/ext2-privacy/dir.c renamed from fs/ext2/dir.c +10 -1
@@ -468,6 +468,15 @@
 	return err;
 }
 
+static inline mark_deleted_dir(struct ext2_dir_entry_2 *dir)
+{
+#ifndef CONFIG_EXT2_FS_PRIVACY
+	dir->inode = 0;
+#else
+	memset(dir, 0, dir->rec_len);
+#endif
+}
+
 /*
  * ext2_delete_entry deletes a directory entry by merging it with the
  * previous entry. Page is up-to-date. Releases the page.
@@ -495,7 +504,7 @@
 		BUG();
 	if (pde)
 		pde->rec_len = cpu_to_le16(to-from);
-	dir->inode = 0;
+	mark_deleted_dir(dir);
 	err = ext2_commit_chunk(page, from, to);
 	UnlockPage(page);
 	ext2_put_page(page);
fs/ext2-privacy/inode.c renamed from fs/ext2/inode.c +37 -1
@@ -46,6 +46,36 @@
 	ext2_discard_prealloc (inode);
 }
 
+static inline void delete_inode(struct inode *inode)
+{
+#ifndef CONFIG_EXT2_FS_PRIVACY
+	inode->u.ext2_i.i_dtime	= CURRENT_TIME;
+#else
+	inode->i_mode	= 0;
+	inode->i_uid	= 0;
+	inode->i_gid	= 0;
+	inode->i_nlink	= 0;
+	inode->i_atime	= 0;
+	inode->i_ctime	= 0;
+	inode->i_mtime	= 0;
+	inode->u.ext2_i.i_dtime	= 0;
+	inode->u.ext2_i.i_flags	= 0;
+	inode->u.ext2_i.i_faddr	= 0;
+	inode->u.ext2_i.i_frag_no	= 0;
+	inode->u.ext2_i.i_frag_size	= 0;
+	inode->u.ext2_i.i_file_acl	= 0;
+	inode->i_generation	= 0;
+#endif
+}
+
+static inline void delete_blocks(struct inode *inode)
+{
+#ifdef CONFIG_EXT2_FS_PRIVACY
+	inode->i_blocks	= 0;
+	memset(&inode->u.ext2_i.i_data, 0x00, sizeof(inode->u.ext2_i.i_data));
+#endif
+}
+
 /*
  * Called at the last iput() if i_nlink is zero.
  */
@@ -57,12 +87,18 @@
 	    inode->i_ino == EXT2_ACL_IDX_INO ||
 	    inode->i_ino == EXT2_ACL_DATA_INO)
 		goto no_delete;
-	inode->u.ext2_i.i_dtime	= CURRENT_TIME;
+
+	delete_inode(inode);
 	mark_inode_dirty(inode);
 	ext2_update_inode(inode, IS_SYNC(inode));
+
 	inode->i_size = 0;
 	if (inode->i_blocks)
 		ext2_truncate (inode);
+	delete_blocks(inode);
+       mark_inode_dirty(inode);
+       ext2_update_inode(inode, IS_SYNC(inode));
+
 	ext2_free_inode (inode);
 
 	unlock_kernel();
fs/Config.in-ext3-privacy renamed from fs/Config.in +1 -0
@@ -34,6 +34,7 @@
 # dep_tristate '  Journal Block Device support (JBD for ext3)' CONFIG_JBD $CONFIG_EXT3_FS
 define_bool CONFIG_JBD $CONFIG_EXT3_FS
 dep_mbool '  JBD (ext3) debugging support' CONFIG_JBD_DEBUG $CONFIG_JBD
+dep_mbool '  ext3 privacy support' CONFIG_EXT3_FS_PRIVACY $CONFIG_EXT3_FS
 
 # msdos file systems
 tristate 'DOS FAT fs support' CONFIG_FAT_FS
fs/ext3-privacy/balloc.c renamed from fs/ext3/balloc.c +18 -0
@@ -252,6 +252,23 @@
 	return slot;
 }
 
+static inline void destroy_block(struct inode *inode, unsigned long block)
+{
+#ifdef CONFIG_EXT3_FS_PRIVACY
+	struct buffer_head	* bh;
+
+	bh = sb_getblk(inode->i_sb, block);
+
+	memset(bh->b_data, 0x00, bh->b_size);
+
+	mark_buffer_dirty(bh);
+	wait_on_buffer(bh);
+	brelse(bh);
+
+	return;
+#endif
+}
+
 /* Free given blocks, update quota and i_blocks field */
 void ext3_free_blocks (handle_t *handle, struct inode * inode,
 			unsigned long block, unsigned long count)
@@ -370,6 +387,7 @@
 			BUFFER_TRACE(bitmap_bh, "bit already cleared");
 		} else {
 			dquot_freed_blocks++;
+			destroy_block(sb, block);
 			gdp->bg_free_blocks_count =
 			  cpu_to_le16(le16_to_cpu(gdp->bg_free_blocks_count)+1);
 			es->s_free_blocks_count =
fs/ext3-privacy/inode.c renamed from fs/ext3/inode.c +23 -1
@@ -172,6 +172,28 @@
 	ext3_discard_prealloc (inode);
 }
 
+static inline void delete_inode(struct inode *inode)
+{
+#ifndef CONFIG_EXT3_FS_PRIVACY
+	inode->u.ext3_i.i_dtime	= CURRENT_TIME;
+#else
+	inode->i_mode	= 0;
+	inode->i_uid	= 0;
+	inode->i_gid	= 0;
+	inode->i_nlink	= 0;
+	inode->i_size	= 0;
+	inode->i_blocks	= 0;
+	inode->i_atime	= 0;
+	inode->i_ctime	= 0;
+	inode->i_mtime	= 0;
+	inode->u.ext3_i.i_dtime	= 0;
+	inode->u.ext3_i.i_flags	= 0;
+	inode->u.ext3_i.i_file_acl	= 0;
+	inode->i_generation	= 0;
+	memset(&inode->u.ext3_i.i_data, 0x00, sizeof(inode->u.ext3_i.i_data));
+#endif
+}
+
 /*
  * Called at the last iput() if i_nlink is zero.
  */
@@ -211,7 +233,7 @@
 	 * (Well, we could do this if we need to, but heck - it works)
 	 */
 	ext3_orphan_del(handle, inode);
-	inode->u.ext3_i.i_dtime	= CURRENT_TIME;
+	delete_inode(inode);
 
 	/* 
 	 * One subtle ordering requirement: if anything has gone wrong
#1 lore raw

Hi!

Quoted 9 lines

}

fs/ext2-privacy/balloc.c renamed from fs/ext2/balloc.c +7 -0
+static inline void destroy_block(struct inode *inode, unsigned long block)
+{
+#ifdef CONFIG_EXT2_FS_PRIVACY
+	struct buffer_head	* bh;
+
+	bh = sb_getblk(inode->i_sb, block);
+

fs/ext2-privacy/balloc.c line 266

Quoted 10 lines
fs/ext2-privacy/balloc.c renamed from fs/ext2/balloc.c +10 -0
+	memset(bh->b_data, 0x00, bh->b_size);
+
+	mark_buffer_dirty(bh);
+	wait_on_buffer(bh);
+	brelse(bh);
+
+	return;
+#endif
+}
+

Perhaps this should still be controlled by (chattr(1)) [its already documented, just not yet implemented].

When a file with the `s' attribute set is deleted, its blocks
are zeroed and written back to the disk.

...at which point config option is not really neccessary.

Pavel

Signature · 2 lines
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
author
#2 lore raw

Hey,

Quoted 8 lines

Perhaps this should still be controlled by (chattr(1)) [its already documented, just not yet implemented].

When a file with the `s' attribute set is deleted, its blocks
are zeroed and written back to the disk.

...at which point config option is not really neccessary.

You're not the first person to mention this to me, Pádraig, brought this up on the day I posted. I certainly thing the 's' options should be implemented, however for a privacy patch I believe that the user shouldn't have to intervene to ensure a file is securely erased. It makes more sense to me, as a lazy person, that the file system should be set to always remove the file content... that way the user doesn't need to get involved.

All that said, the user's content is something that the user could be considered responsible for erasing themselves. The meta-data is the part of the file which they dont' have access to, so having privacy capabilities for meta-data erasure is a requirement. User data erasure... I can take it or leave it. I think it should be automatic if at all, but I'm not really that bothered about it.

peace,

--gq

#3 lore raw

Hi!

Quoted 17 lines
Quoted 8 lines

Perhaps this should still be controlled by (chattr(1)) [its already documented, just not yet implemented].

When a file with the `s' attribute set is deleted, its blocks
are zeroed and written back to the disk.

...at which point config option is not really neccessary.

You're not the first person to mention this to me, Pádraig, brought this up on the day I posted. I certainly thing the 's' options should be implemented, however for a privacy patch I believe that the user shouldn't have to intervene to ensure a file is securely erased. It makes more sense to me, as a lazy person, that the file system should be set to always remove the file content... that way the user doesn't need to get involved.

Quoted 6 lines

All that said, the user's content is something that the user could be considered responsible for erasing themselves. The meta-data is the part of the file which they dont' have access to, so having privacy capabilities for meta-data erasure is a requirement. User data erasure... I can take it or leave it. I think it should be automatic if at all, but I'm not really that bothered about it.

Well, doing it on-demand means one less config option, and possibility to include it into 2.7... It should be easy to have tiny patch forcing that option always own for your use...
Pavel

Signature · 2 lines
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
author
#4 lore raw

Hey,

Quoted 4 lines

Well, doing it on-demand means one less config option, and possibility to include it into 2.7... It should be easy to have tiny patch forcing that option always own for your use...

Works for me. Should I implement the chattr 's' handling then for the data blocks? Or do you think it should also include the meta-data deletion as well?

peace,

--gq

#5 lore raw

Hi!

Quoted 7 lines
Quoted 3 lines

Well, doing it on-demand means one less config option, and possibility to include it into 2.7... It should be easy to have tiny patch forcing that option always own for your use...

Works for me. Should I implement the chattr 's' handling then for the data blocks? Or do you think it should also include the meta-data deletion as well?

[Perhaps I got confused somewhere in between]

I think chattr +s file should be zeroed, along with its metadata. I thought your patch already does that. If not, doing that would be great...
Pavel

Signature · 2 lines
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
author
#6 lore raw

Hey,

Quoted 2 lines

[Perhaps I got confused somewhere in between]

nope, just me.

Quoted 4 lines

I think chattr +s file should be zeroed, along with its metadata. I thought your patch already does that. If not, doing that would be great...

it does already zero the meta-data. I was simply asking if you think the whole "erase" operation should be under the control of chattr 's', or just a subset (i.e. only data overwriting is optional). Its clear now that you want the whole thing to be controled by chattr 's'. I'll knock that up then, and re-submit.

peace,

--gq

#7 lore raw
Quoted 5 lines · the grugq

it does already zero the meta-data. I was simply asking if you think the whole "erase" operation should be under the control of chattr 's', or just a subset (i.e. only data overwriting is optional). Its clear now that you want the whole thing to be controled by chattr 's'. I'll knock that up then, and re-submit.

Yes, some people want to *keep* as much information so that it can be undeleted (think user mistakes).

#25 lore raw
In reply to the grugq
Quoted 3 lines · the grugq

All that said, the user's content is something that the user could be considered responsible for erasing themselves. The meta-data is the part of the file which they dont' have access to, so having privacy

Actually, I have encountered file systems where two successive write() calls from userspace to the same offset in the file wouldn't end up in the same physical location on the disk (AIX's JFS with compression). It would LZ compress each 4K block, and then find a contiguous set of 512-byte sectors to write it. So one write might compress down to 6 sectors and be written in one place, the next time it doesn't compress as well and takes 7 - so it ends up elsewhere because the previous hole was exactly 6 - and if you try to zero it by writing a block of zeros, that would compress down to 1 sector and fail to overwrite the others...

If we ever do a filesystem with compression, we'll have the same issue.

#8 lore raw
In reply to Pavel Machek
Quoted 10 lines · Pavel Machek
Quoted 6 lines

All that said, the user's content is something that the user could be considered responsible for erasing themselves. The meta-data is the part of the file which they dont' have access to, so having privacy capabilities for meta-data erasure is a requirement. User data erasure... I can take it or leave it. I think it should be automatic if at all, but I'm not really that bothered about it.

Well, doing it on-demand means one less config option, and possibility to include it into 2.7... It should be easy to have tiny patch forcing that option always own for your use...

The obvious thing to do would be to make it a mount option, so that
(a) recompilation is not necessary in order to use the feature, and
(b) the feature can be turned on or off on a per-filesystem feature. In 2.6, it's possible to specify certain mount option to be specifed by default on a per-filesystem basis (via a new field in the superblock).

So if you do things that way, then secure deletion would take place either if the secure deletion flag is set (so it can be enabled on a per-file basis), or if the filesystem is mounted with the secure-deletion mount option.

- Ted

author
#9 lore raw
Quoted 9 lines

The obvious thing to do would be to make it a mount option, so that
(a) recompilation is not necessary in order to use the feature, and
(b) the feature can be turned on or off on a per-filesystem feature. In 2.6, it's possible to specify certain mount option to be specifed by default on a per-filesystem basis (via a new field in the superblock).

Quoted 4 lines

So if you do things that way, then secure deletion would take place either if the secure deletion flag is set (so it can be enabled on a per-file basis), or if the filesystem is mounted with the secure-deletion mount option.

Makes sense to me. If either the file system, or the file, are in 'secure delete' mode, then erase everything about the file. Allowing the paranoid to have the option as default, and the concerned to target specific files. I like it.

peace,

--gq

#10 lore raw
Quoted 13 lines · the grugq
Quoted 11 lines

The obvious thing to do would be to make it a mount option, so that
(a) recompilation is not necessary in order to use the feature, and
(b) the feature can be turned on or off on a per-filesystem feature. In 2.6, it's possible to specify certain mount option to be specifed by default on a per-filesystem basis (via a new field in the superblock). So if you do things that way, then secure deletion would take place either if the secure deletion flag is set (so it can be enabled on a per-file basis), or if the filesystem is mounted with the secure-deletion mount option.

Quoted 1 line

Makes sense to me. If either the file system, or the file, are in 'secure delete' mode, then erase everything about the file. Allowing the paranoid to have the option as default, and the concerned to target specific files. I like it.

It would be useful to have this as a directory option, so that all files in directory would be protected. I think wherever you do it you have to prevent hard links, so that unlink really removes the data.

Signature · 3 lines
bill davidsen <[email protected]>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979
#11 lore raw
Quoted 3 lines · Bill Davidsen

It would be useful to have this as a directory option, so that all files in directory would be protected. I think wherever you do it you have to prevent hard links, so that unlink really removes the data.

This of course implies that 'chattr +s' (or whatever it was) has to fail if the link count isn't exactly one. Also makes for lots of uglyiness if it's a directory option - you then have to walk all the entries in the directory and check *their* link counts. Bad Juju doing it in the kernel if you have a directory with a million entries - and racy as hell if you do it in userspace.

#12 lore raw
Quoted 8 lines · [email protected] wrote:
Quoted 1 line · Bill Davidsen

It would be useful to have this as a directory option, so that all files in directory would be protected. I think wherever you do it you have to prevent hard links, so that unlink really removes the data.

This of course implies that 'chattr +s' (or whatever it was) has to fail if the link count isn't exactly one.

Do you disagree that the count does need to be one?

Quoted 5 lines

Also makes for lots of uglyiness
if it's a directory option - you then have to walk all the entries in the directory and check *their* link counts. Bad Juju doing it in the kernel if you have a directory with a million entries - and racy as hell if you do it in userspace.

I agree with everything you said, "useful" doesn't always map to "easy." But if you agree that the count needs to be one on files, then you could also fail if you tried to add it to a directory which was not empty.

In case I didn't make it clear, the use I was considering was to create a single directory in which created files would really go away when deleted. I hadn't considered doing it after files were present, what you say about overhead is clearly an issue. I think I could even envision some bizarre race conditions if the kernel had to do marking of each file, so perhaps it's impractical.

But what happens when the 'setgid' bit is put on a directory? At least in 2.4 existing files do NOT get the group set, only files newly created. So unless someone feels that's a bug which needs immediate fixing, I can point to it as a model by which the feature could be practically implemented.

Comment?

Signature · 3 lines
bill davidsen <[email protected]>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979
author
#13 lore raw
Quoted 7 lines

But what happens when the 'setgid' bit is put on a directory? At least in 2.4 existing files do NOT get the group set, only files newly created. So unless someone feels that's a bug which needs immediate fixing, I can point to it as a model by which the feature could be practically implemented.

Comment?

So the proposal is specifically:

if a directory has its chattr 's' "bit" set, then all newly created files within that directory inherit the chattr 's' "bit".

Personally, I think that extends beyond simply privacy protection, and addresses how chattr values are handled by the file system. Privacy protection wouldn't affect the way in which the chattr values are handled, so if they are inherited through directories then fine... but I don't see this as specific to secure deletion.

--gq

author
#14 lore raw
In reply to Bill Davidsen
Quoted 3 lines

But what happens when the 'setgid' bit is put on a directory? At least in 2.4 existing files do NOT get the group set, only files newly created. So unless someone feels that's a bug which needs immediate fixing, I can point to it as a model by which the feature could be practically implemented.

Implementing the privacy patch based on the suggestions put forward (chattr +s && mount options), i've hit something of a snag.

If a directory has "chattr +s" then whenever a directory entry is deleted, should the dirent contents be overwritten, or should only freed blocks be overwritten? It makes sense to me that the directory entry should be overwritten because it is inaccessible meta-data and exposes information about the file system. Since the user obviously wants the directory to be securely deleted, that could be construed as implying they want sensitive directory content securely deleted as well.

--gq

#17 lore raw
In reply to [email protected]
Quoted 10 lines · On Wed, Feb 04, 2004 at 12:14:18PM -0500, [email protected] wrote:
Quoted 3 lines

It would be useful to have this as a directory option, so that all files in directory would be protected. I think wherever you do it you have to prevent hard links, so that unlink really removes the data.

This of course implies that 'chattr +s' (or whatever it was) has to fail if the link count isn't exactly one. Also makes for lots of uglyiness if it's a directory option - you then have to walk all the entries in the directory and check *their* link counts. Bad Juju doing it in the kernel if you have a directory with a million entries - and racy as hell if you do it in userspace.

Disagree. Unless you also want to make it illegal to establish a hard link if the +s option is set.

A easier set of semantics is that when the inode count drops to zero, the inode is securely deleted, but that if there are hard links, there are hard links. Basically the flag applies to inodes, and that in some cases, where you create a file in a parent directory, the inode may inherit the secure deletion flag. But a creating a hard link doesn't count as creating a new inode.

- Ted

#15 lore raw
In reply to Bill Davidsen
Quoted 4 lines · Bill Davidsen
Quoted 2 lines

This of course implies that 'chattr +s' (or whatever it was) has to fail if the link count isn't exactly one.

Do you disagree that the count does need to be one?

I'm not prepared to say that there's no scenario where we *dont* care how many links there are, as long as the file *does* get wiped when the last one goes away.

The MH mail handler stores each message in a file - so a mail message is easily stored in multiple folders by simply using multiple hard links. I could easily see having mail that I want to +s and go away when I remove it from the last folder it was in....

Quoted 3 lines

I agree with everything you said, "useful" doesn't always map to "easy." But if you agree that the count needs to be one on files, then you could also fail if you tried to add it to a directory which was not empty.

Yes you could. The question is whether that's a desired semantic or not.

Quoted 6 lines

In case I didn't make it clear, the use I was considering was to create a single directory in which created files would really go away when deleted. I hadn't considered doing it after files were present, what you say about overhead is clearly an issue. I think I could even envision some bizarre race conditions if the kernel had to do marking of each file, so perhaps it's impractical.

As I said, ugly and murky....

Quoted 5 lines

But what happens when the 'setgid' bit is put on a directory? At least in 2.4 existing files do NOT get the group set, only files newly created. So unless someone feels that's a bug which needs immediate fixing, I can point to it as a model by which the feature could be practically implemented.

Ahh.. but now you're suggesting a different model than "directory must be empty". Obviously more discussion of what we *want* it to do is needed ;)

author
#18 lore raw
In reply to Theodore Ts'o

Ok, I've update the secure delete patch to now check

is_secure_delete(inode)

to see if either the chattr +s bit is set, or if the superblock has EXT2FS_MOUNT_SECRM (which I've added) set.

I believe I've correctly modified the super.c to support a "secrm" mount option, but I haven't tested it so I can't say for sure. That is to say it compiles, but I haven't tried to run it. This rev of the patch is to see if the direction (and logic) of the secure delete functionality is in line with what people are thinking.

There is some complexity around directory files. If a directory file has chattr +s set, then any directory entry within that file will be erased when the entry is removed. If a file has chattr +s set, or the fs was mounted with SECRM then the directory entry is removed. I would like people to pay particular attention to this part of the patch, its the area I'm least confident with.

A directory entry's rec_len is preserved, in case we are removing the first entry for a directory block.

The rest of the patch is basicly unchanged from the previous patchs. I haven't done a patch for ext3, when this one is agreed on then I'll it port acroos to ext3.

peace,

--gq

include/linux/ext2_fs.h-secrm renamed from include/linux/ext2_fs.h +1 -0
@@ -310,6 +310,7 @@
 #define EXT2_MOUNT_MINIX_DF		0x0080	/* Mimics the Minix statfs */
 #define EXT2_MOUNT_NOBH			0x0100	/* No buffer_heads */
 #define EXT2_MOUNT_NO_UID32		0x0200  /* Disable 32-bit UIDs */
+#define	EXT2_MOUNT_SECRM		0x0400  /* Securely delete files */
 #define EXT2_MOUNT_XATTR_USER		0x4000	/* Extended user attributes */
 #define EXT2_MOUNT_POSIX_ACL		0x8000	/* POSIX Access Control Lists */
 
fs/ext2-secrm/balloc.c renamed from fs/ext2/balloc.c +19 -1
@@ -173,6 +173,21 @@
 	}
 }
 
+static inline void delete_block(struct super_block *sb, unsigned long block)
+{
+	struct buffer_head	* bh;
+
+	bh = sb_getblk(sb, block);
+
+	memset(bh->b_data, 0, bh->b_size);
+
+	mark_buffer_dirty(bh);
+	wait_on_buffer(bh); /* XXX is this necessary? */
+	brelse(bh);
+
+	return;
+}
+
 /* Free given blocks, update quota and i_blocks field */
 void ext2_free_blocks (struct inode * inode, unsigned long block,
 		       unsigned long count)
@@ -240,8 +255,11 @@
 			ext2_error (sb, "ext2_free_blocks",
 				      "bit already cleared for block %lu",
 				      block + i);
-		else
+		else {
+			if (is_secure_delete(inode))
+				delete_block(inode->i_sb, block + i);
 			group_freed++;
+		}
 	}
 
 	mark_buffer_dirty(bitmap_bh);
fs/ext2-secrm/dir.c renamed from fs/ext2/dir.c +18 -2
@@ -530,6 +530,7 @@
 {
 	struct address_space *mapping = page->mapping;
 	struct inode *inode = mapping->host;
+	struct inode *dino;
 	char *kaddr = page_address(page);
 	unsigned from = ((char*)dir - kaddr) & ~(ext2_chunk_size(inode)-1);
 	unsigned to = ((char*)dir - kaddr) + le16_to_cpu(dir->rec_len);
@@ -555,9 +556,24 @@
 		BUG();
 	if (pde)
 		pde->rec_len = cpu_to_le16(to-from);
-	dir->inode = 0;
+
+	dino = iget(inode->i_sb, dir->inode);
+	if (!dino || (!is_secure_delete(dino) && !is_secure_delete(inode))) {
+		dir->inode = 0;
+		inode->i_ctime = inode->i_mtime = CURRENT_TIME;
+	} else {
+		unsigned short rec_len = dir->rec_len;
+
+		memset(dir, 0, dir->rec_len);
+		dir->rec_len = rec_len;
+	}
+
+	if (dino)
+		iput(dino);
+
 	err = ext2_commit_chunk(page, from, to);
-	inode->i_ctime = inode->i_mtime = CURRENT_TIME;
+
+ 	/* XXX should this be in the if (!is_secure_delete()) above? */
 	EXT2_I(inode)->i_flags &= ~EXT2_BTREE_FL;
 	mark_inode_dirty(inode);
 out:
fs/ext2-secrm/ext2.h renamed from fs/ext2/ext2.h +10 -0
@@ -79,6 +79,16 @@
 	return container_of(inode, struct ext2_inode_info, vfs_inode);
 }
 
+static inline int is_secure_delete(struct inode *inode)
+{
+	/* expanded for readibility */
+	if (EXT2_I(inode)->i_flags & EXT2_SECRM_FL)
+		return 1;
+	if (test_opt(inode->i_sb, SECRM))
+		return 1;
+	return 0;
+}
+
 /* balloc.c */
 extern int ext2_bg_has_super(struct super_block *sb, int group);
 extern unsigned long ext2_bg_num_gdb(struct super_block *sb, int group);
fs/ext2-secrm/inode.c renamed from fs/ext2/inode.c +39 -1
@@ -64,6 +64,36 @@
 		ext2_discard_prealloc(inode);
 }
 
+static inline void delete_inode(struct inode *inode)
+{
+	if (!is_secure_delete(inode))
+		EXT2_I(inode)->i_dtime = get_seconds();
+	else {
+		inode->i_mode   = 0;
+		inode->i_uid    = 0;
+		inode->i_gid    = 0;
+		inode->i_nlink  = 0;
+		inode->i_atime.tv_sec  = 0;
+		inode->i_atime.tv_nsec  = 0;
+		inode->i_ctime.tv_sec  = 0;
+		inode->i_ctime.tv_nsec  = 0;
+		inode->i_mtime.tv_sec  = 0;
+		inode->i_mtime.tv_nsec  = 0;
+		EXT2_I(inode)->i_dtime = 0;
+		EXT2_I(inode)->i_faddr = 0;
+		EXT2_I(inode)->i_frag_no       = 0;
+		EXT2_I(inode)->i_frag_size     = 0;
+		EXT2_I(inode)->i_file_acl      = 0;
+		inode->i_generation     = 0;
+	}
+}
+
+static inline void delete_blocks(struct inode *inode)
+{
+	inode->i_blocks = 0;
+	memset(EXT2_I(inode)->i_data, 0, sizeof(EXT2_I(inode)->i_data));
+}
+
 /*
  * Called at the last iput() if i_nlink is zero.
  */
@@ -71,13 +101,21 @@
 {
 	if (is_bad_inode(inode))
 		goto no_delete;
-	EXT2_I(inode)->i_dtime	= get_seconds();
+	delete_inode(inode);
 	mark_inode_dirty(inode);
 	ext2_update_inode(inode, inode_needs_sync(inode));
 
 	inode->i_size = 0;
 	if (inode->i_blocks)
 		ext2_truncate (inode);
+
+	if (is_secure_delete(inode)) {
+		EXT2_I(inode)->i_flags = 0;
+
+		delete_blocks(inode);
+		mark_inode_dirty(inode);
+		ext2_update_inode(inode, inode_needs_sync(inode));
+	}
 	ext2_free_inode (inode);
 
 	return;
fs/ext2-secrm/super.c renamed from fs/ext2/super.c +5 -1
@@ -270,7 +270,7 @@
 	Opt_bsd_df, Opt_minix_df, Opt_grpid, Opt_nogrpid,
 	Opt_resgid, Opt_resuid, Opt_sb, Opt_err_cont, Opt_err_panic, Opt_err_ro,
 	Opt_nouid32, Opt_check, Opt_nocheck, Opt_debug, Opt_oldalloc, Opt_orlov, Opt_nobh,
-	Opt_user_xattr, Opt_nouser_xattr, Opt_acl, Opt_noacl,
+	Opt_user_xattr, Opt_nouser_xattr, Opt_acl, Opt_noacl, Opt_secrm,
 	Opt_ignore, Opt_err,
 };
 
@@ -299,6 +299,7 @@
 	{Opt_nouser_xattr, "nouser_xattr"},
 	{Opt_acl, "acl"},
 	{Opt_noacl, "noacl"},
+	{Opt_secrm, "secrm"},
 	{Opt_ignore, "grpquota"},
 	{Opt_ignore, "noquota"},
 	{Opt_ignore, "quota"},
@@ -410,6 +411,9 @@
 			printk("EXT2 (no)acl options not supported\n");
 			break;
 #endif
+		case Opt_secrm:
+			set_opt(sbi->s_mount_opt, SECRM);
+			break;
 		case Opt_ignore:
 			break;
 		default:
#26 lore raw
In reply to [email protected]
Quoted 3 lines · [email protected] wrote:

Actually, I have encountered file systems where two successive write() calls from userspace to the same offset in the file wouldn't end up in the same physical location on the disk (AIX's JFS with compression).

See also:

- ext3 with data journalling

- reiser4 with wandering logs

- experimental ext? patches for tail-packing small files

-- Jamie

#27 lore raw
Quoted 15 lines · Jamie Lokier
Quoted 5 lines · [email protected] wrote:

Actually, I have encountered file systems where two successive write() calls from userspace to the same offset in the file wouldn't end up in the same physical location on the disk (AIX's JFS with compression).

See also:

- ext3 with data journalling

- reiser4 with wandering logs

Quoted 12 lines

- experimental ext? patches for tail-packing small files

-- Jamie
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

reiser4 probably does not need secure deletion as much as others, because once the encryption plugins are debugged we will most likely encourage users to use encryption by default. Perhaps someone will show the error in my thinking though, I am not trying to be rigid here....

author
#28 lore raw

Well, I think secure deletion should be an option for everyone. Using encryption is a data hiding technique, you prevent people for detemining what sort of data is being stored there. Now, admittedly I dont know at what level the reiser4 encryption appears, but I would think its safer to have complete erasure when a file deleted regardless of how well protected its contents were.

just a thought.

--gq

Quoted 31 lines · Hans Reiser
Quoted 15 lines · Jamie Lokier
Quoted 4 lines · [email protected] wrote:

Actually, I have encountered file systems where two successive write() calls from userspace to the same offset in the file wouldn't end up in the same physical location on the disk (AIX's JFS with compression).

See also:

- ext3 with data journalling

- reiser4 with wandering logs

Quoted 12 lines

- experimental ext? patches for tail-packing small files

-- Jamie
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

reiser4 probably does not need secure deletion as much as others, because once the encryption plugins are debugged we will most likely encourage users to use encryption by default. Perhaps someone will show the error in my thinking though, I am not trying to be rigid here....

#43 lore raw
In reply to Hans Reiser
Quoted 4 lines · Hans Reiser

reiser4 probably does not need secure deletion as much as others, because once the encryption plugins are debugged we will most likely encourage users to use encryption by default. Perhaps someone will show the error in my thinking though, I am not trying to be rigid here....

With encrypted block devices, there is the possibility that someone may discover your key, or gain access to your computer (e.g. steal your laptop while it's switched on, or someone puts a gun to your head and makes you enter the key).

If someone gets access you might be glad you securely deleted some files by overwriting the blocks.

When encryption is implemented in the filesystem itself, this is preventable.

There is a cryptographic way to ensure deleted files cannot be recovered even when someone knows the filesystem key, without needing to overwrite the files. This is even better than overwriting, because it resists signal processing methods on the hard disk platter, and is effective with virtual devices where overwriting does not actually erase the original data (e.g. VMware or Bochs copy-on-write disk image; LVM snapshots; some SAN devices).

Thanks in advance for the implementation :)

-- Jamie

#16 lore raw
In reply to [email protected]
Quoted 8 lines · [email protected] wrote:
Quoted 4 lines · Bill Davidsen
Quoted 2 lines

This of course implies that 'chattr +s' (or whatever it was) has to fail if the link count isn't exactly one.

Do you disagree that the count does need to be one?

Quoted 8 lines

I'm not prepared to say that there's no scenario where we *dont* care how many links there are, as long as the file *does* get wiped when the last one goes away.

The MH mail handler stores each message in a file - so a mail message is easily stored in multiple folders by simply using multiple hard links. I could easily see having mail that I want to +s and go away when I remove it from the last folder it was in....

This is then a question of what we want it to do, and I assume that either of the obvious behaviours is not only possibe but practical. The question is if the objective is to make the data go away when the last link is removed, or to make removal of sach a file (unlink) remove the contents at the time of the unlink. I think the latter clearly implies allowing a single link to the inode.

Since I could make a case for either, I'd like to hear other feedback.

Quoted 6 lines
Quoted 3 lines

I agree with everything you said, "useful" doesn't always map to "easy." But if you agree that the count needs to be one on files, then you could also fail if you tried to add it to a directory which was not empty.

Yes you could. The question is whether that's a desired semantic or not.

Given my use of the 'setgid' bit as a possible model, I would say that having new files created with the attribute is useful, and the user putting the attribute on the directory should control the content at the time the attribute is set. In other words it will be as useful if we just make it apply on files as they are created.

Quoted 6 lines
Quoted 3 lines

In case I didn't make it clear, the use I was considering was to create a single directory in which created files would really go away when deleted. I hadn't considered doing it after files were present, what you say about overhead is clearly an issue. I think I could even envision some bizarre race conditions if the kernel had to do marking of each file, so perhaps it's impractical.

As I said, ugly and murky....

And not needed.

Quoted 7 lines
Quoted 3 lines

But what happens when the 'setgid' bit is put on a directory? At least in 2.4 existing files do NOT get the group set, only files newly created. So unless someone feels that's a bug which needs immediate fixing, I can point to it as a model by which the feature could be practically implemented.

Ahh.. but now you're suggesting a different model than "directory must be empty". Obviously more discussion of what we *want* it to do is needed ;)

Yes, the person setting the attribute can control this. Unlike the setgid bit it can't be done (currently) as a part of creat().

Signature · 3 lines
bill davidsen <[email protected]>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979
#29 lore raw
In reply to the grugq

There is an extensive literature on how you can recover deleted files from media that has been erased a dozen times, but breaking encryption is harder. It is more secure to not put the data on disk unencrypted at all is my point.....

Hans

Quoted 4 lines · the grugq

Well, I think secure deletion should be an option for everyone. Using encryption is a data hiding technique, you prevent people for detemining what sort of data is being stored there. Now, admittedly I dont know at what level the reiser4 encryption appears, but I would think its safer to have complete erasure when a file deleted regardless of how well protected its contents were.

just a thought.

author
#30 lore raw

Sure, but to a large extent it depends on your threat model. If you say "an extremely well funded government entity wants to examine your hard drive" then they will have access to insane technologies like electron microscopes. This makes any software solution to secure deletion practically impossible (if we trust the literature).

If, on the other hand, we have a threat model of, say, the police, then things are very different. In the UK, there is a law which requires you to turn over your encryption keys when the court demands them. The police have a tactic for extracting keys which involves physical violence and intimidation. These are very effective against encryption. However, the police do not have access to hardware based analysis technologies, they are just too expensive. So, while they can recover something that has been encrypted, they can't recover something that has been securely deleted. (Not without begging the .gov to perform a hardware analysis, something which would be quite unlikely in the normal course of events).

The majority of forensic analysis is performed with software tools on a bit image of the hard drive. If this bit image doesn't contain the data, the software tools won't see it, and the forensic analysis will fail. I would suggest adding secure deletion, because it does provide a pretty good level of protection. It is not a 100% solution, but neither is encryption. The two in combination are complementary technologies.

peace,

--gq

Quoted 12 lines · Hans Reiser

There is an extensive literature on how you can recover deleted files from media that has been erased a dozen times, but breaking encryption is harder. It is more secure to not put the data on disk unencrypted at all is my point.....

Hans

Quoted 4 lines · the grugq

Well, I think secure deletion should be an option for everyone. Using encryption is a data hiding technique, you prevent people for detemining what sort of data is being stored there. Now, admittedly I dont know at what level the reiser4 encryption appears, but I would think its safer to have complete erasure when a file deleted regardless of how well protected its contents were.

just a thought.

#31 lore raw
Quoted 5 lines · the grugq

If, on the other hand, we have a threat model of, say, the police, then things are very different. In the UK, there is a law which requires you to turn over your encryption keys when the court demands them. The police have a tactic for extracting keys which involves physical violence and intimidation. These are very effective against encryption.

This is how to implement secure deletion cryptographically:

- Each time a file is created, choose a random number.

- Encrypt the number with your filesystem key and store the encrypted version in the inode.

- The number is used for encrypting that file.

Secure deletion is then a matter of securely deleting the inode. The file data does not have to be overwritten.

This is secure against many attacks that "secure deletion" by overwriting is weak against. This includes electron microscopes looking at the data, and UK law. (The police can demand your filesystem key, but nobody knows the random number that belonged to a new-deleted inode).

There is a chance the electron microscope may recover the number from the securely deleted inode. That is the weakness of this system, therefore the inode data should be very thoroughly erased or itself subject to careful cryptographic hding.

-- Jamie

author
#32 lore raw

Yes, the allocation of the inode and data blocks should be randomized for security, but that would lead to performance impacts. Implementing that should definately be a compile time option.

I would advocate erasing all meta-data on a file, and also erasing the data. The end-user can be responsible for erasing the data, they can access it with write(), but they can't access the meta-data (not without directly accessing the file system). Thats why I'm putting these patches forward. The file system should be responsible for removing meta-data when a file is deleted. This secure deletion doesn't have to incorporate data block erasure (although my implemenation does).

Your suggestion would certainly work, but I think the performance impact of using random inodes and data blocks would dissuade many from having it enabled by default. Simple secure deletion of the data and meta-data would have a lower impact, and be more likely to be used on more file systems.

peace,

--gq

Quoted 23 lines

This is how to implement secure deletion cryptographically:

- Each time a file is created, choose a random number.

- Encrypt the number with your filesystem key and store the encrypted version in the inode.

- The number is used for encrypting that file.

Secure deletion is then a matter of securely deleting the inode. The file data does not have to be overwritten.

This is secure against many attacks that "secure deletion" by overwriting is weak against. This includes electron microscopes looking at the data, and UK law. (The police can demand your filesystem key, but nobody knows the random number that belonged to a new-deleted inode).

There is a chance the electron microscope may recover the number from the securely deleted inode. That is the weakness of this system, therefore the inode data should be very thoroughly erased or itself subject to careful cryptographic hding.

#33 lore raw
Quoted 3 lines · the grugq

Yes, the allocation of the inode and data blocks should be randomized for security, but that would lead to performance impacts. Implementing that should definately be a compile time option.

What do you mean?

I haven't mentioned randomising block allocations at all.

The random number is an encryption key, private to the inode, used to encrypt the data blocks. The blocks are allocated efficiently as usual.

Quoted 5 lines

Your suggestion would certainly work, but I think the performance impact of using random inodes and data blocks would dissuade many from having it enabled by default. Simple secure deletion of the data and meta-data would have a lower impact, and be more likely to be used on more file systems.

My suggestion would be much more efficient than yours: for every file created and deleted, you do twice the I/O I do.

-- Jamie

author
#34 lore raw
Quoted 9 lines

What do you mean?

I haven't mentioned randomising block allocations at all.

The random number is an encryption key, private to the inode, used to encrypt the data blocks. The blocks are allocated efficiently as usual.

I didn't understand your proposal. nm.

As I now understand, you are proposing a file system which has per file encryption where the key is stored in the inode. The inode is then the only location with senstive data which needs to be removed.

What about directory files? That is, how would you propose handling the directory entries of deleted files?

Also, this proposal seems to me more related to how to implement an encrypted file system, than how to implement secure deletion on existing file systems.

Quoted 3 lines

My suggestion would be much more efficient than yours: for every file created and deleted, you do twice the I/O I do.

Sorry, per file encryption is more efficient than deffered block writes after deletion? What you are proposing is unrelated to secure deletion. Its an encrypted file system implementation. Comparing efficiency between secure deletion on ext2, for example, and encrypted files on some unimplemented file system doesn't make sense.

Now, given that my comments were on what I thought you were proposing (randomly allocating inodes and blocks to prevent an analyst being able to piece a file back toghether via guess work) not what you actually were proposing (an encrypted file system implementation), ignore the previous email.

peace,

--gq

#35 lore raw
Quoted 3 lines · the grugq

As I now understand, you are proposing a file system which has per file encryption where the key is stored in the inode. The inode is then the only location with senstive data which needs to be removed.

Yes.

Quoted 3 lines

Also, this proposal seems to me more related to how to implement an encrypted file system, than how to implement secure deletion on existing file systems.

Not really, this is pointing out an alternative means of secure deletion _if_ you have encryption. The points I wanted to make were, most important first:

- Overwriting data does not always do what you think it does. Several block devices _do not_ overwrite the same storage blocks. Thus it is dangerous to call something "secure deletion" when it might not do anything at all.

- Filesystems on top of encrypted block devices _do_ need overwriting-based secure deletion, if they are of the type where the block encryption key is derived from the block offset and device key only.

- Eraseable per-file keys may be a more secure way of destroying data than overwriting data on a magnetic medium.

-- Jamie

#40 lore raw
In reply to Jamie Lokier

Exactly right.

Hans

Quoted 40 lines · Jamie Lokier
Quoted 3 lines · the grugq

If, on the other hand, we have a threat model of, say, the police, then things are very different. In the UK, there is a law which requires you to turn over your encryption keys when the court demands them. The police have a tactic for extracting keys which involves physical violence and intimidation. These are very effective against encryption.

This is how to implement secure deletion cryptographically:

- Each time a file is created, choose a random number.

- Encrypt the number with your filesystem key and store the encrypted version in the inode.

- The number is used for encrypting that file.

Secure deletion is then a matter of securely deleting the inode. The file data does not have to be overwritten.

This is secure against many attacks that "secure deletion" by overwriting is weak against. This includes electron microscopes looking at the data, and UK law. (The police can demand your filesystem key, but nobody knows the random number that belonged to a new-deleted inode).

There is a chance the electron microscope may recover the number from the securely deleted inode. That is the weakness of this system, therefore the inode data should be very thoroughly erased or itself subject to careful cryptographic hding.

-- Jamie
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#39 lore raw
In reply to the grugq
Quoted 10 lines · the grugq
Quoted 9 lines

What do you mean?

I haven't mentioned randomising block allocations at all.

The random number is an encryption key, private to the inode, used to encrypt the data blocks. The blocks are allocated efficiently as usual.

Quoted 7 lines

I didn't understand your proposal. nm.

As I now understand, you are proposing a file system which has per file encryption where the key is stored in the inode. The inode is then the only location with senstive data which needs to be removed.

What about directory files? That is, how would you propose handling the directory entries of deleted files?

Also, this proposal seems to me more related to how to implement an encrypted file system, than how to implement secure deletion on existing file systems.

It will be easy to code on reiser4 which has encryption being built into it.

Quoted 7 lines
Quoted 4 lines

My suggestion would be much more efficient than yours: for every file created and deleted, you do twice the I/O I do.

Sorry, per file encryption is more efficient than deffered block writes after deletion?

Oh yes. Way so.

Quoted 10 lines

What you are proposing is unrelated to secure deletion. Its an encrypted file system implementation. Comparing efficiency between secure deletion on ext2, for example, and encrypted files on some unimplemented file system doesn't make sense.

Now, given that my comments were on what I thought you were proposing (randomly allocating inodes and blocks to prevent an analyst being able to piece a file back toghether via guess work) not what you actually were proposing (an encrypted file system implementation), ignore the previous email.

peace,

--gq

#36 lore raw
In reply to Jamie Lokier
Quoted 7 lines · Jamie Lokier

- Eraseable per-file keys may be a more secure way of destroying data than overwriting data on a magnetic medium.

I agree.

#37 lore raw
In reply to Jamie Lokier

Hi!

Quoted 10 lines
Quoted 3 lines

As I now understand, you are proposing a file system which has per file encryption where the key is stored in the inode. The inode is then the only location with senstive data which needs to be removed.

Yes.

Quoted 3 lines

Also, this proposal seems to me more related to how to implement an encrypted file system, than how to implement secure deletion on existing file systems.

Quoted 8 lines

Not really, this is pointing out an alternative means of secure deletion _if_ you have encryption. The points I wanted to make were, most important first:

- Overwriting data does not always do what you think it does. Several block devices _do not_ overwrite the same storage blocks. Thus it is dangerous to call something "secure deletion" when it might not do anything at all.

But you have same vulnerability, crypto does not help here. If your i-node happens to be put on other place, attacker still gets the key intact etc.

There's not much you can do. [It may be even worse with that crypto... If you kick the table while your top-secret .mpg.tgz collection is accessed, you are likely to cause bad sector within i-node, attacker can get the key, and decrypt it all. With on-place overwriting he only gets one block.]
Pavel

Signature · 2 lines
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
#38 lore raw
Quoted 8 lines · Pavel Machek
Quoted 4 lines

- Overwriting data does not always do what you think it does. Several block devices _do not_ overwrite the same storage blocks. Thus it is dangerous to call something "secure deletion" when it might not do anything at all.

But you have same vulnerability, crypto does not help here. If your i-node happens to be put on other place, attacker still gets the key intact etc.

Yes, you're right. I wanted to draw attention to the fact that ext2's "secure deletion" attribute, when implemented, can be misleading. I should have pointed out that an inode-private key has the same weakness.

Quoted 5 lines

There's not much you can do. [It may be even worse with that crypto... If you kick the table while your top-secret .mpg.tgz collection is accessed, you are likely to cause bad sector within i-node, attacker can get the key, and decrypt it all. With on-place overwriting he only gets one block.]

There is something which can help:

First, the idea of an inode private key, which is itself encrypted by a whole filesystem key, may be seen as a two level tree structure (the filesystem key is its root). You can also have deeper key structures. The important thing is that without the root key, you cannot follow any path to the tree's branches. The key tree does not have to be the same as the directory tree, in fact it's better if it isn't, so you can keep the key tree balanced.

You can regularly change branch keys within the filesystem, updating all the dependent keys when that's done. That can be done incrementally, as you make changes to the flesystem. So for example, whenever you want to erase an inode (secure deletion), you need to modify its parent key in the key tree and re-encrypt all the inode key's siblings with the new parent key. Repeat all the way to the root.

If you do those updates every time, then you confine the "snapshotting block device" weakness to the problem of destroying old versions of the root key - for which you should insist on choosing an appropriate device. You might also use multiple root keys, so that a single remapped block on a hard disk which is used to hold the root key(s) is not enough to decrypt old versions of the filesystem.

-- Jamie

#41 lore raw
In reply to Hans Reiser
Quoted 35 lines · Hans Reiser

Exactly right.

Hans

Quoted 28 lines · Jamie Lokier
Quoted 7 lines · the grugq

If, on the other hand, we have a threat model of, say, the police, then things are very different. In the UK, there is a law which requires you to turn over your encryption keys when the court demands them. The police have a tactic for extracting keys which involves physical violence and intimidation. These are very effective against encryption.

This is how to implement secure deletion cryptographically:

- Each time a file is created, choose a random number.

- Encrypt the number with your filesystem key and store the encrypted version in the inode.

- The number is used for encrypting that file.

Secure deletion is then a matter of securely deleting the inode. The file data does not have to be overwritten.

This is secure against many attacks that "secure deletion" by overwriting is weak against. This includes electron microscopes looking at the data, and UK law. (The police can demand your filesystem key, but nobody knows the random number that belonged to a new-deleted inode).

Also they will demand this random number since the court can consider it as a part of your secret key. So just delete your secret key without creating meaningless infrastructure ;)
Edward.

Quoted 16 lines
Quoted 16 lines

There is a chance the electron microscope may recover the number from the securely deleted inode. That is the weakness of this system, therefore the inode data should be very thoroughly erased or itself subject to careful cryptographic hding.

-- Jamie
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#42 lore raw
Quoted 7 lines · Edward Shishkin

Also they will demand this random number since the court can consider it as a part of your secret key. So just delete your secret key without creating meaningless infrastructure ;)
Edward.

This is a good point. A court may be able to demand your pass phrase, but if you rm the key itself, they can hardly expect you to have memorized that.....

#19 lore raw
In reply to Theodore Ts'o

At which point, in the presence of the theoretical mount option, it becomes easy to reduce the work load to "a long list of block operations" by making the shredding unconditional.

That is, instead of thinking of it as shredding a file, the (arbitrarily named) "zerofree" mount option is passed and every block that is released from active file system use is zeroed. E.g. file blocks, directory blocks, attribute blocks, everything. That just takes (at the worst) a list/queue and a block-write of a block-sized page containing all zeros (or, better yet, an optionally-user-supplied squeegee pattern.) So take/make the free_block() routine and make it submit a "dirty block" to the I/O buffering system. If the block is immediately re-used then even the write expense amortizes to near zero for clearing that block (or unwritten fragment).

The down side of the simple form comes up when the user unlinks that 20 gig file.

And of course there is no such thing as un-erase for such a media.

On the more positive side this would be _*outstanding*_ for my NV-RAM keychain drive where the files are warranted to be small and I don't want some random person who finds my lost keychain even able to guess about that pesky project I was working on last month.

Meh... 8-)

Rob.

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Theodore Ts'o
Sent: Tuesday, February 03, 2004 10:30 PM
To: Pavel Machek
Cc: the grugq; [email protected]
Subject: Re: PATCH - ext2fs privacy (i.e. secure deletion) patch

Quoted 11 lines · Pavel Machek
Quoted 7 lines

All that said, the user's content is something that the user could be considered responsible for erasing themselves. The meta-data is the part

of the file which they dont' have access to, so having privacy capabilities for meta-data erasure is a requirement. User data erasure... I can take it or leave it. I think it should be automatic if at all, but I'm not really that bothered about it.

Well, doing it on-demand means one less config option, and possibility to include it into 2.7... It should be easy to have tiny patch forcing that option always own for your use...

The obvious thing to do would be to make it a mount option, so that
(a) recompilation is not necessary in order to use the feature, and
(b) the feature can be turned on or off on a per-filesystem feature. In 2.6, it's possible to specify certain mount option to be specifed by default on a per-filesystem basis (via a new field in the superblock).

So if you do things that way, then secure deletion would take place either if the secure deletion flag is set (so it can be enabled on a per-file basis), or if the filesystem is mounted with the secure-deletion mount option.

- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#20 lore raw
Quoted 4 lines · Robert White

On the more positive side this would be _*outstanding*_ for my NV-RAM keychain drive where the files are warranted to be small and I don't want some random person who finds my lost keychain even able to guess about that pesky project I was working on last month.

An encrypted and/or steganographic filesystem would be much better for your NVRAM keychain drive, because that hides your files even when you _haven't_ deleted them.

-- Jamie

#21 lore raw

For some forms of better, yes. 8-) The disposable inode key and the fully encrypted file system are clearly more potent and better. Nice and bank-vault. Sometimes though, you just want and only need a $3 chain on your door to keep out the neighbor children. Were the fully vault-like option available you can bet I'd partition my keychain to do both. Meanwhile...

A generic mount option that does the write-over-on-free operation could be easily ported to/integrated with the other file-systems like, say, vfat. Which, in turn, would continue to let me use my keychain to carry things to non-linux environments while giving me some anti-leave-behind protection when I delete things (in linux). (So I am evil in trying to get the poor-man's feature in for general consideration. 8-)

"Truly secure" is a good thing but "casually secure and pervasive" is relatively important too.

Consider the simple "strings /dev/sda1" "attack" on a raw device. Even with the VMWare journaling and such and its implications, from inside the environment write-over-on-erase would go a long way to block the stupid leave behind vulnerability.

[Discussion-Fork]

I have actually been looking into the removable media issues more than anything else. My exact area of focus has been the idea of "local root" for a (removable) file system. Basically it seems that there needs to be a way to communicate a list of otherwise unassigned user ID numbers into the kernel. Then when a flagged file system were mounted, the real user ID numbers in the file system image would map to known-unused numbers. The root-on-disk would become something arbitrary like 21901 and then user ID 21901 would enjoy root-like write permissions on that media, but no other media in the system. All the other UID/GID numbers would likewise be translated to unused "real" nubers. This way applications could "run setuid" and such but in their own sandbox, leaving the file system internally consistent after use, but never endangering the overall system security state. (e.g. setuid root is bad, setuid 21901 not so much. 8-)

Think "application cards" or, say, student terminals where the whole OS is installed but the portable (firewire/usb/etc) drives the students use would remain consistent and capable of running their apps.

I have many thoughts about the requirements. A callback daemon a-la hotplug to coordinate UID and GID pools. A shim over the "real" file system calls (kind of like the way loop shims as a block device) to do the bidirectional translations of ID number. That sort of thing.

I could see wanting to drop signatures and otherwise "taint" the file system image with a temporary/scratch file full of information during this application, or god knows what else. Casual wipe then applies and is highly utilitarian because you can unlink-on-unmount and rely on the buffer flush to do the dirty work, rather than have a window of time when the file system is mounted but the scratch space is being de-constructed. Yes, not Big-S secure, but "highly utilitarian" if for no reason other than preventing an fsck from casually restoring a scratch file with valid contents.

[/Discussion-Fork]

So meanwhile, having a simple zero-out/expunge on last unlink (ne block free) flag for a whole file system serves the dual purpose of mistake prevention and exceedingly trivial security, at the sole cost of "no norton-grade casual un-erase".

Regardless of the "I don't want the government recovering evidence of my illicit activities" elements to be gained by a cryptographically complete solution to secure file removal; there is real and immediate utility to a simple wipe-on-erase. More-so if it were to become "expected to be available" on all supported filesystem types.

Rob.

-----Original Message-----
From: Jamie Lokier [mailto:[email protected]]
Sent: Thursday, February 12, 2004 7:41 PM
To: Robert White
Cc: 'Theodore Ts'o'; 'Pavel Machek'; 'the grugq'; [email protected]
Subject: Re: PATCH - ext2fs privacy (i.e. secure deletion) patch

Quoted 3 lines · Robert White

On the more positive side this would be _*outstanding*_ for my NV-RAM keychain drive where the files are warranted to be small and I don't want some random person who finds my lost keychain even able to guess about

that

Quoted 1 line

pesky project I was working on last month.

An encrypted and/or steganographic filesystem would be much better for your NVRAM keychain drive, because that hides your files even when you _haven't_ deleted them.

-- Jamie

#24 lore raw
In reply to Robert White

Hi!

Quoted 4 lines

At which point, in the presence of the theoretical mount option, it becomes easy to reduce the work load to "a long list of block operations" by making the shredding unconditional.

Quoted 10 lines

That is, instead of thinking of it as shredding a file, the (arbitrarily named) "zerofree" mount option is passed and every block that is released from active file system use is zeroed. E.g. file blocks, directory blocks, attribute blocks, everything. That just takes (at the worst) a list/queue and a block-write of a block-sized page containing all zeros (or, better yet, an optionally-user-supplied squeegee pattern.) So take/make the free_block() routine and make it submit a "dirty block" to the I/O buffering system. If the block is immediately re-used then even the write expense amortizes to near zero for clearing that block (or unwritten fragment).

I see a small problem with that: some "interesting" data, such as file names, are smaller than a block. With this kind of implementation, you'd never wipe those.
Pavel
[un-erase no longer works even on ext3, that's nothing new]

Signature · 2 lines
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
#22 lore raw
In reply to Jamie Lokier
Quoted 11 lines · Jamie Lokier
Quoted 4 lines · Robert White

On the more positive side this would be _*outstanding*_ for my NV-RAM keychain drive where the files are warranted to be small and I don't want some random person who finds my lost keychain even able to guess about that pesky project I was working on last month.

An encrypted and/or steganographic filesystem would be much better for your NVRAM keychain drive, because that hides your files even when you _haven't_ deleted them.

But these are not mutually exclusive. Even if the data are encrypted, when I delete something I want it GONE. That way even if a TLA with resources to break the crypto steals my device, my previous password list is safe.

Signature · 3 lines
bill davidsen <[email protected]>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979
#23 lore raw
Quoted 4 lines · Bill Davidsen

But these are not mutually exclusive. Even if the data are encrypted, when I delete something I want it GONE. That way even if a TLA with resources to break the crypto steals my device, my previous password list is safe.

Fair point. Per-inode keys won't protect against a resourceful TLA (if such a TLA ever exists).

-- Jamie