"Write once only but read many" filesystem
Quoted 27 lines · Scott Lovenberg
Quoted 26 lines · Jörn Engel
Quoted 6 lines · Peter Teoh
Quoted 2 lines
Or do you want individual files/directories to be immutable -
chattr?chattr is not good enough, as root can still modify it. So if
current feature is not there, then some small development may be needed.Quoted 8 lines
Quoted 2 lines
And in either case, what problem do you want to solve with a
read-only filesystem?Simple: i want to record down everything that a user does, or a
database does, or any applications running - just record down its state permanently securely into the filesystem, knowing that for sure, there is not way to modify the data, short of recreating the
filesystem again. Sound logical? Or is there any loophole in this
concept?The loophole is called root. In a normal setup, root can do anything, including writing directly to the device your filesystem resides in, writing to kernel memory, etc.
It may be rather inconvenient to change a filesystem by writing to the block device, but far from impossible. If you want to make such changes impossible, you are facing an uphill battle that I personally don't care about. And if inconvenience is good enough, wouldn't chattr be sufficiently inconvenient?
Quoted 6 lines
Quoted 2 lines
Jörn
How about mounting an isofs via loopback? This has the added benefit of being ready to be exported to disc. You can make it with mkisofs on a directory structure and mount it to the tree with a normal mount(1). If it asks for fs type on mount, I think its 'iso9660'.
Thanks for the idea. Based on this idea, I will start looking at the implementation of isofs, and how is it made to be readonly.......my ultimate aim is to make the filesystem readonly upon after being written, and the file closed. Not sure if it can be done, but I envisaged a lot of audit journalling are of these nature. Of course, it is always possible to "dd" the filesystem to modify the content, but then if given design into its protection mechanism (like incremental checksum - current checksum based on previous checksum, generated and stored together with the file, upon after every writing of data) we can always protect its integrity. Aim is to set it to readonly......anyone can read....but not modifiable. As a start I will try out patching ext2, hoping that it is much simpler than ext3/ext4.
Upon changes to its content (via dd) it will invalidate the immediate future incremental checksum. Similarly, if u patch the current checksum (which depends on a hash function of previous data, and previous checksum), it will not be valid, as the current checksum is also dependent on the history of previous checksum. So everytime u change the content via dd, u will need to modify the next checksum, which is calculated based on this data, and the current checksum, which again provide the seed for the next checksum etc. U will have to modify a lot of data, unless u are near the tip of the latest modification. The smaller the chunk of data per checksum, the more difficult to keep up with the rate of modification. Tradeoff is more work for CPU.
The userspace tool part will then always validate the checksum with the data being read, if modified, checksum will not be valid. Since it is a hash function, given the modified data, and the previous checksum, it is not possible calculate the current checksum.
/Of course, if the intruder is root, then it is as good as not having all these complex calculations, so our assumption is that the machine is not compromised yet.
Then of course the "chattr" can be used as well - if it is not compromised. True, but the possibility that it can be modified infinitely via chattr also exists - which is what write-once-read-many is against - to provide the assurance that it has not been overwritten the second time (possible technically, but very very difficult).
An equivalent requirements of such a filesystem will be: a filesystem such that upon every changes made, a log of dates of changes are made. Overhead for these feature is high, so a lightweighted version will be the date/history only it is first written.
On an existing ext2 filesystem, once the '"worm" kernel module is loaded, the feature immediately take effect - content becomes read-only but not modifiable, but modifiable for new contents. And old contents may or may not have a checksum to protect it, but if the checksum exists, it will come from the last time "worm" was running and generating checksum.
So u have the best of both world - ext2/3/4 with/without worm. If worm-enabled, it may also be configured at the directory level - some directory can be solely dedicated for worm-journalling.
What do u guys think - any conceptual errors?
/
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
You're still being really vague on your requirements. Sometimes you talk about a filesystem that can't be modified after it has been written, and other times you talk about a filesystem that can be updated, but only to add information. Sometimes you talk about a read-only restriction that is effective against the superuser (not possible) and other times say that it's OK if the superuser can modify the filesystem without a trace.
In addition to new ext3 features, using isofs, and chattr, mounting read-only and using file permissions also effect read-only status, and you'd have to explain how any of these don't meet your requirements. But rest assured that none of them is effective against the superuser.
Quoted 2 lines
Upon changes to its content (via dd) it will invalidate the immediate future incremental checksum.
I couldn't tell what "it" is in this sentence, or what checksums you're thinking of.
--
Bryan Henderson IBM Almaden Research Center
San Jose CA Filesystems
Quoted 2 lines · Bryan Henderson
You're still being really vague on your requirements. Sometimes you talk about a filesystem that can't be modified after it has been written, and
Sorry for the confusion. The requirement from upfront is that it is ALWAYS WRITE-ONCE only. But many of you will say that it is impossible, as it can be written over again via other means. Yes, the other means is via "dd", but that will require that you unmount the filesystem first, then using dd and hexeditor to modify the filesystem. Not sure if I had made myself clear?
Quoted 2 lines
other times you talk about a filesystem that can be updated, but only to
NO updating is possible. Write once, and the file attributes defined from the first time it is written, and not modifiable after that - access control is from the kernel, and therefore userspace cannot do anything, once mounted of course.
Quoted 3 lines
add information. Sometimes you talk about a read-only restriction that is effective against the superuser (not possible) and other times say that it's OK if the superuser can modify the filesystem without a trace. In addition to new ext3 features, using isofs, and chattr, mounting read-only and using file permissions also effect read-only status, and you'd have to explain how any of these don't meet your requirements. But
May be I will describe an auditing scenario to make myself clear - why the above does not meet the requirement. First, an arbitrary application does something, then the auditing daemon will record it down, into a file. Once the file is open for write, it is possible to continuosly write to it. But once close, it automatically becomes read-only. But permanently no userspace tool can reverse the read-only attribute - controlled from the kernel. But while being opened and written, it is not possible to move back the file pointer as well - ie, once written, is written, not possible to overwrite. Concurrently, other files can be opened on the same directory for writing. Can any of the abovementioned userspace tool, or existing ext2/3/4 configuration can satisfy the above requirement?
Quoted 3 lines
rest assured that none of them is effective against the superuser.
Agreed - once the filesystem is unmounted, u can do anything to the filesystem. But modifying any bytes of it will be difficult, the control mechanism as laid out in my earlier email.
Quoted 6 lines
Quoted 2 lines
Upon changes to its content (via dd) it will invalidate the immediate future incremental checksum.
I couldn't tell what "it" is in this sentence, or what checksums you're thinking of.
Erh......Not sure what u mean? What I mean is that if u unmount the filesystem, and attempt to modify any bytes of the write-once filesystem, it will most likely invalidate the integrity of subsequent data, illustrated by following diagram:
hash_functionA(checksumA, dataA) ==> checksumB.
hash_functionA(checksumB, dataB) ==> checksumC.
hash_functionA(checksumC, dataC) ==> checksumD.
Eg, if u attempt to modify the checksumB, 2 ways of detection is possible:
1. next: u can be detected by a mismatch of checksumC stored on the disk. 2. previous: u can be detected by a mismatch of checksumB stored on disk (based on previous checksumA and dataA).
Or if u attempt to modify the dataB, similarly the two ways of detection applies as well.
So if u really want to modify a single bytes of data, u have to start from the very first item in the chain. And with an added "salt" to the hash_functionA (random number), it is possible to branch off the hash_functionA(checksumB, dataB) into many different branches, thus implementing the concurrent open files concept. And the algorithm of calculation as described easily facilitate a forward looking concept - no previous data have to be modified - so whatever written on disk, will be written, not modifiable anymore.
Is the concept clearer now?
Quoted 7 lines
--
Bryan Henderson IBM Almaden Research Center
San Jose CA Filesystems
Quoted 6 lines · Peter Teoh
Sorry for the confusion. The requirement from upfront is that it is
ALWAYS WRITE-ONCE only. But many of you will say that it is
impossible, as it can be written over again via other means. Yes, the
other means is via "dd", but that will require that you unmount the filesystem first, then using dd and hexeditor to modify the
filesystem. Not sure if I had made myself clear?
You're confused. The kernel cannot prevent root from modifying a block device while a filesystem is mounted.
Sounds to me like what you really want is a printer. You can't modify what's been printed.
You could also use a second computer and log to that.
Quoted 4 lines
May be I will describe an auditing scenario to make myself clear - why
the above does not meet the requirement. First, an arbitrary
application does something, then the auditing daemon will record it
down, into a file. Once the file is open for write, it is possible to
Quoted 8 lines
continuosly write to it. But once close, it automatically becomes
read-only. But permanently no userspace tool can reverse the read-only
attribute - controlled from the kernel. But while being opened and
written, it is not possible to move back the file pointer as well - ie,
once written, is written, not possible to overwrite. Concurrently,
other files can be opened on the same directory for writing. Can any
of the abovementioned userspace tool, or existing ext2/3/4 configuration can satisfy the above requirement?
Can't be done. root can ptrace the auditing daemon and change what it wants to write.
Signature · 4 lines
Intel are signing my paycheques ... these opinions are still mine "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step."
Quoted 4 lines
Sounds to me like what you really want is a printer. You can't modify what's been printed.
You could also use a second computer and log to that.
Don't forget write-once storage servers and file servers. That's what's actually used in the real world for this.
--
Bryan Henderson IBM Almaden Research Center
San Jose CA Filesystems
Quoted 3 lines
Quoted 1 line
other times you talk about a filesystem that can be updated, ...
NO updating is possible.
I'm still not convinced you mean that, because the example you give is specifically of a file that becomes immutable after some writing to it. How about creating a new file? That is a filesystem update. Do you want to allow that? How about directory modifications, such as rename and unlink?
Quoted 9 lines
Once the file is open for write, it is possible to
continuosly write to it. But once close, it automatically becomes
read-only. But permanently no userspace tool can reverse the read-only
attribute - controlled from the kernel. But while being opened and
written, it is not possible to move back the file pointer as well - ie,
once written, is written, not possible to overwrite. Concurrently,
other files can be opened on the same directory for writing. Can any
of the abovementioned userspace tool, or existing ext2/3/4 configuration can satisfy the above requirement?
No. And the first thing in the list to which you refer is what I believe
your proposal is: adding a feature to ext3. Modifying ext3 cannot
satisfy the above requirements.
Quoted 8 lines
hash_functionA(checksumA, dataA)==> checksumB.
hash_functionA(checksumB, dataB)==> checksumC.
hash_functionA(checksumC, dataC)==> checksumD.So if u really want to modify a single bytes of data, u have to start from the very first item in the chain.
Yeah, that's what the bad guy will do. So you haven't prevented someone from undetectably modifying previously written data.
Quoted 2 lines
And with an added "salt" to the
hash_functionA(random number)
The bad guy would be sure to use the same salt, or change it.
--
Bryan Henderson IBM Almaden Research Center
San Jose CA Filesystems
Quoted 6 lines · Bryan Henderson
Quoted 4 lines
Quoted 2 lines
other times you talk about a filesystem that can be updated, ...
NO updating is possible.
I'm still not convinced you mean that, because the example you give is specifically of a file that becomes immutable after some writing to it. How about creating a new file?
Yes - always allowed.
Quoted 3 lines
That is a filesystem update. Do you want
to allow that? How about directory modifications, such as rename and unlink?
Creating directories - always allowed. Modification - rename/unlink etc will be disallowed.
Quoted 3 lines
Yeah, that's what the bad guy will do. So you haven't prevented someone from undetectably modifying previously written data.
This is the only difference between the current discussion, and the hardware WORM storage solution u mentioned in the previous email, due to it software vs hardware implementation aspect. I have also found this:
ftp://reports.stanford.edu/pub/cstr/reports/cs/tr/87/1177/CS-TR-87-1177.pdf
Well, I would like to thank you and Matthew for the discussion (I think using the printer is not advisable - the output is not really so immutable after all - JUST reprint the modified content will do. Similarly for WORM storage medium - just write into another CD.)
U agree with me?. Sorry for the rubbish talk :-).
Anyway, I shall attempt to write some proof-of-concept patches to try out the idea. I may fail.
Thanks.
Quoted 4 lines
(I think
using the printer is not advisable - the output is not really so immutable after all - JUST reprint the modified content will do. Similarly for WORM storage medium - just write into another CD.)
In these systems, the untrusted person doesn't have physical access to the medium, though he has total access to the interface for adding data to it.
One example would be where the untrusted person is a hacker from the
Internet. Another would be where the medium is in the possession of an independent auditing firm.
WORM storage media is actually of little interest these days for security since you can accomplish the same thing with programming -- as long as the untrusted person doesn't have any access to the storage except a communication wire, the server can make data immutable even while using ordinary disk drives -- if you have enough confidence in the engineering.
--
Bryan Henderson IBM Almaden Research Center
San Jose CA Filesystems