TALPA - a threat model? well sorta.
So after about a week of trying to squeeze information out of anti-malware companies I'm starting to feel like I can better speak for their needs (although they probably don't like what I have to say). I would like to point out that many enterprises are going to run this stuff on their machines. Period. End of story. Personally I'd rather support a clean interface than have to try to support support problems my customers have when their hacked fragile systems have trouble.
So, what is it that anti-malware companies do? They scan files. That's it. Many of the arguments they make and marketing speak we hear or think we hear claim more grandious things. But from what I've seen they aren't the real deal. Most of the security model descriptions that people on list want actually are framed under the belief that these products need to or attempt to completely block some class of attacks. They don't. If you think they do you need to fix your frame of reference. The only class of attacks this interface is supposed to address is those that can be found by scanning files. This is NOT an LSM. This is NOT an IDS. This is a file scanner. There may be all sorts of marketing material or general beliefs that they provide security against all sorts of wide and varied threats (and they do), but in all reality the only threats they provide any help against are those that can be found by scanning files. Simple as that. Some may argue this isn't "good" security and I'm not going to make a strong argument to the contrary, I can stand here for days and show cases where this is highly useful but no one can provide a threat model more than to say, "we attempt to locate files which may be harmful somewhere in the digital ecosystem and try to deny access to that data."
It can be easily shown (and I can come up with LOTS of example) that this is of value. Lets consider a simple Linux worm:
http://www.ca.com/us/securityadvisor/virusinfo/virus.aspx?id=47869
This worm needs 4 vulnerabilities for it to work and a default RHEL SELinux policy would likely, although I haven't verified this fact, also stop the propagation. I believe that the right way to "fix" this is to patch your software and to run an LSM. But notice that 'scanning files' is also enough to break the chain. When the download is completed nothing would be able to execute the file in question since the anti-malware software would prevent it. Assuming some new vulnerability was found to replace one of the patched problems its possible the anti-malware code could still recognize enough of the binary to stop it. I'm told by one AV company that they look for 95 Linux-specific threats composed of 42 trojans and 53 viruses or worms. At least 2 other companies specifically stated that they look for malware which targets Linux systems although didn't give hard numbers. All three vendors who have spoken with me also admit that most of their threats are found on Windows.
I also gave a previous example how file scanning can help with subverted distro repos. That one is purely theoretical as opposed to the real attack shown above, but clearly value can be easily demonstrated.
The value of a file scanning interface is not in stopping active attacks. Its in making the Linux platform a more difficult location for the storage and propagation of bad data. I think its reasonable to think that we all agree we don't want to be the preferred hosting platforms for trojan binaries intended to attack other non-Linux systems. Why would one want consciencely choose to leave Linux as a safe haven for the existence of malware? Even though the malware is not attacking the Linux platform do we as the Linux community really want to be a breeding and hosting ground as long as the costs are not too high? (yes Ted, I'm talking directly to you here)
Assuming the costs are not too high, we should add this file scanning subsystem to help prevent the storage and dissemination of malware across our systems. If I'm running a hosting company I wouldn't want people to be able to upload their trojans and host them on my Linux server even if those trojans will only ever be downloaded by another OS. We are part of the larger computing ecosystem. If we can make our systems inhospitable to malware like data we help ourselves a little, but the greater computing community a lot.
The real goal is to get files into to some userspace scanner and let them do whatever they want. Remember, this isn't a new LSM. The goal isn't to provide perfect security. The goal isn't to stop already running malicious programs. The one and only goal is to scan files. We should not be considering timing attacks, we should not be considering processes actively trying to get around the system for small periods of time. We should certainly not be considering root processes being able to sneak things by. The idea is that a file exists on disk and we want some userspace program to give a best effort at scanning it. Yes, we might be able to prevent a root process from doing bad things, but a bad root process already won.
Now costs:
1) Kernel or userspace. If we actually want a relatively robust ecosystem of malware storage hardening it needs to be in kernel. knfsd makes it plainly obvious that userspace scanners will leave big gaping holes. Not to mention there cannot be any checks for files opened by suid apps in userspace. And while no specific claims are being made about intentionally malicious code, putting this in kernel makes programs which call syscalls directly much more difficult to be used to circumvent the scanning.
2) I think the "best" time to do scanning is at read and write. Any disagreements there? But what's the cost? The cost is lots of scanning and a huge performance hit on concurrent file access. I'm willing to put hard numbers behind this, but it does seem quite obvious that open and close happen a lot less than read and write. Sure that means between open and read data may have become bad. We aren't an LSM. We aren't and IDS. We aren't making solid claims about local system security. We are claiming to harden the machine's storage and propagation of 'bad' files.
3) code maintenance. I'm here. Sophos appears to be willing to help and one of the other AV vendors said they may be willing to commit to helping with support. If nothing else we know the AV vendors will hop to get stuff fixed every time an enterprise distro is about to release *smile*
I've yet to hear a technical argument against my patch set (although I do plan some changes like using the audit system and changing the kernel->userspace interface) Andi pointed out that I completely didn't consider rawio on the block device but I have decided to just accept that as a "you've already lost" condition. I've explained my mmap support about 10 times, no one has said I'm wrong yet (I assume people who can't read) keep talking about mmap.
No, this is not a new security system. This is not some great wonderful panacea that's going to stop every exploit. Its a file scanner. It's not bad. It clearly can help. This isn't just a pipe dream, I've got code. Where's the problem?
-Eric
Quoted 2 lines
So, what is it that anti-malware companies do? They scan files. That's it.
Good so lets instead have a discussion about making the file event notification more scalable. That is the same thing I want for content indexing. It is the same thing you want for certain kinds of smart archiving, for on-line asynchronous backup and other stuff.
It ought to be a simple clean syscall interface.
Alan
Quoted 9 lines · Alan Cox
Quoted 2 lines
So, what is it that anti-malware companies do? They scan files. That's it.
Good so lets instead have a discussion about making the file event notification more scalable. That is the same thing I want for content indexing. It is the same thing you want for certain kinds of smart archiving, for on-line asynchronous backup and other stuff.
It ought to be a simple clean syscall interface.
Are you willing to make it blocking? I'm not sure how to make what we have capable of assuring that the object you got a notification about is actually the object you are acting on. Thoughts on how to accomplish that? I'm here to code and I'm willing to throw all my work in the garbage if someone can show me how to actually do it better.
-Eric
Quoted 9 lines · On Wed, 13 Aug 2008 12:47:45 -0400 Eric Paris <[email protected]> wrote:
Quoted 8 lines · Alan Cox
Quoted 2 lines
So, what is it that anti-malware companies do? They scan files. That's it.
Good so lets instead have a discussion about making the file event notification more scalable. That is the same thing I want for content indexing. It is the same thing you want for certain kinds of smart archiving, for on-line asynchronous backup and other stuff.
Quoted 7 lines
Quoted 1 line
It ought to be a simple clean syscall interface.
Are you willing to make it blocking? I'm not sure how to make what we have capable of assuring that the object you got a notification about is actually the object you are acting on. Thoughts on how to accomplish that? I'm here to code and I'm willing to throw all my work in the garbage if someone can show me how to actually do it better.
I don't think you need to be blocking if you passed up a file handle ?
fd = fileeventmumble(somestuff); do_stuff close(fd);
[taking care not to end up recursing as a result]
Quoted 14 lines · Alan Cox
Quoted 11 lines · On Wed, 13 Aug 2008 12:47:45 -0400 Eric Paris <[email protected]> wrote:
Quoted 9 lines · Alan Cox
Quoted 2 lines
So, what is it that anti-malware companies do? They scan files. That's it.
Good so lets instead have a discussion about making the file event notification more scalable. That is the same thing I want for content indexing. It is the same thing you want for certain kinds of smart archiving, for on-line asynchronous backup and other stuff.
It ought to be a simple clean syscall interface.
Quoted 7 lines
Quoted 5 lines
Are you willing to make it blocking? I'm not sure how to make what we have capable of assuring that the object you got a notification about is actually the object you are acting on. Thoughts on how to accomplish that? I'm here to code and I'm willing to throw all my work in the garbage if someone can show me how to actually do it better.
I don't think you need to be blocking if you passed up a file handle ?
Without blocking and waiting how do you deny access? Maybe I needed another thing they do. "They do file scanning and deny access to bad files."
async scanning on close/write is great. but you need blocking/access control on open/read.....
Quoted 5 lines
fd = fileeventmumble(somestuff); do_stuff close(fd);[taking care not to end up recursing as a result]
[you pointed out the whole point of process exclusions in the original work]
Quoted 5 lines
Quoted 1 line
I don't think you need to be blocking if you passed up a file handle ?
Without blocking and waiting how do you deny access? Maybe I needed another thing they do. "They do file scanning and deny access to bad files."
Denying access is easy enough - chmod it or set an SELinux label on it.
There are a limited number of useful arrangements I can see here
'opened for write deny [some set of] access until verified'
Thats an selinux state transition on open for write, and one from the user space end on the other
'run around after things'
Which is simply a notifier and a relabel as 'contaminated' or similar (or a chmod or mv ...) by the daemon end of things.
Now I can see why you might want a 'has been open for a write and dirty for a while' notifier - again for indexing as well as virus scanners and the like.
What other semantic do you have in mind given that you have to deal with open by me, open by writer, content modified.. after I have it open anyway ? It seems the rest is a discussion about time intervals ?
Quoted 11 lines
-----Original Message-----
From: [email protected] [mailto:malware-list- [email protected]] On Behalf Of Alan Cox
Sent: Wednesday, August 13, 2008 3:59 PM
To: Eric Paris
Cc: [email protected]; [email protected]; malware- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [malware-list] TALPA - a threat model? well sorta.Quoted 1 line
Quoted 1 line
I don't think you need to be blocking if you passed up a file
handle ?
Quoted 3 lines
Quoted 3 lines
Without blocking and waiting how do you deny access? Maybe I needed another thing they do. "They do file scanning and deny access to
bad
Quoted 3 lines
Quoted 1 line
files."
Denying access is easy enough - chmod it or set an SELinux label on
it.
I may be missing something about your suggestion, but I don't see how this would work. Who does the chmod?
Here's a sequence:
- Application opens file
- AV scanner notified in some way without blocking
- Application reads file into memory
- AV scanner determines file is infected.
- AV scanner chmod's file -- oops, too late.
- Application sends file over the wire to another machine with a more vulnerable OS
How would this be prevented?
Jon Press
Quoted 8 lines
I may be missing something about your suggestion, but I don't see how this would work. Who does the chmod?
Here's a sequence:
- Application opens file
- AV scanner notified in some way without blocking
- Application reads file into memory
The discussion has been about scanning on write.
Quoted 6 lines
- AV scanner determines file is infected.
- AV scanner chmod's file -- oops, too late.
- Application sends file over the wire to another machine with a more vulnerable OSHow would this be prevented?
I don't think you can. In your case how does your AV scanner deal with the case where the application opens the file while another user has it open and the other user (or even other task with the same handle) changes the content possibly via mmap.
Content may also directly be shared between users of a file using the mmap interfaces so your scan on read model is rather dysfunctional.
Quoted 2 lines · Jonathan
I may be missing something about your suggestion, but I don't see how this would work. Who does the chmod?
Chmod is also not a solution to the hierarchical storage (or incremental restore from backup) problem.
I believe we really do need the block-on-open.
Signature · 1 line
All Rights Reversed
Quoted 10 lines · On Wed, 13 Aug 2008 17:35:29 -0400 Rik van Riel <[email protected]> wrote:
Quoted 2 lines · Jonathan
I may be missing something about your suggestion, but I don't see how this would work. Who does the chmod?
Chmod is also not a solution to the hierarchical storage (or incremental restore from backup) problem.
I believe we really do need the block-on-open.
The block on open is orthogonal really. Useful for HSM, useful for certain very primitive scanning but not much else that I can see.
And its a minor mod to the security hooks to allow it as far as I can see
Quoted 12 lines · Alan Cox
Quoted 9 lines · On Wed, 13 Aug 2008 17:35:29 -0400 Rik van Riel <[email protected]> wrote:
Quoted 2 lines · Jonathan
I may be missing something about your suggestion, but I don't see how this would work. Who does the chmod?
Chmod is also not a solution to the hierarchical storage (or incremental restore from backup) problem.
Quoted 6 lines
Quoted 1 line
I believe we really do need the block-on-open.
The block on open is orthogonal really. Useful for HSM, useful for certain very primitive scanning but not much else that I can see.
And its a minor mod to the security hooks to allow it as far as I can see
So here's where I run into trouble. Lets assume I want to be helpful and engineer in a vacuum for this unknown HSM user as well. Clearly file scanners need the file to be there which means the inode ("on disk" kind) needs to be there and stuff like that. I assume that the HSM user is going to need to hook long before these things even exist. Where would they need to hook?
Should I just design for my own needs and include stacking and a priority number and hopefully the HSM people can use it later? I don't know the details of what might someday by needed for a project I know nothing about *smile*
-Eric
Quoted 10 lines · Rik van Riel
Quoted 2 lines · Jonathan
I may be missing something about your suggestion, but I don't see how this would work. Who does the chmod?
Chmod is also not a solution to the hierarchical storage (or incremental restore from backup) problem.
I believe we really do need the block-on-open.
Or use either the Fuse or Coda kernel modules and handle such requests in userspace. With FUSE you should even be able to block on a per-page granularity, Coda only has session semantics so it will only notify userspace of the open and close events, while read and write and mmap are passed directly to the underlying file.
Jan
Quoted 1 line · On Fri, 15 Aug 2008 16:16:22 -0400 Jan Harkes <[email protected]> wrote:
Quoted 1 line
I believe we really do need the block-on-open.
I believe more that we need block-on-read ;-) (on open we can start an async scan to cut latency)
read() (or mmap etc) is where the actual use/transfer of contaminated data happens, not in the open.
Signature · 3 lines
If you want to reach me at my work email, use [email protected] For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 10 lines · Arjan van de Ven
Quoted 1 line · On Fri, 15 Aug 2008 16:16:22 -0400 Jan Harkes <[email protected]> wrote:
Quoted 1 line
I believe we really do need the block-on-open.
I believe more that we need block-on-read ;-) (on open we can start an async scan to cut latency)
read()(or mmap etc) is where the actual use/transfer of contaminated data happens, not in the open.
I could probably buy that, but I don't know how an HSM would work. Would we have everything we need at open for them to fire off?
/me is HSM clueless and trying to include their needs is proving a challenge.
-Eric
Quoted 9 lines · On Sun, 17 Aug 2008 19:19:37 -0400 Eric Paris <[email protected]> wrote:
Quoted 8 lines · Arjan van de Ven
Quoted 1 line · On Fri, 15 Aug 2008 16:16:22 -0400 Jan Harkes <[email protected]> wrote:
Quoted 1 line
I believe we really do need the block-on-open.
I believe more that we need block-on-read ;-) (on open we can start an async scan to cut latency)
Quoted 8 lines
Quoted 2 lines
read()(or mmap etc) is where the actual use/transfer of contaminated data happens, not in the open.I could probably buy that, but I don't know how an HSM would work. Would we have everything we need at open for them to fire off?
/me is HSM clueless and trying to include their needs is proving a challenge.
I bet it's the same; until you read/mmap... no need to get the data.
you wouldn't want an "ls -l" or "find" to restore your entire system from tape after all :)
Signature · 3 lines
If you want to reach me at my work email, use [email protected] For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 10 lines · Arjan van de Ven
Quoted 2 lines · Eric Paris
/me is HSM clueless and trying to include their needs is proving a challenge.
I bet it's the same; until you read/mmap... no need to get the data.
you wouldn't want an "ls -l" or "find" to restore your entire system from tape after all :)
The HSMs I'm familiar with keep the inode and the first block of the file on-disk, so that stat and file commands work. For Windows, you also need to keep the pointer to the icon to use valid, or create a fake one.
--dave
Signature · 4 lines
David Collier-Brown | Always do right. This will gratify Sun Microsystems, Toronto | some people and astonish the rest [email protected] | -- Mark Twain cell: (647) 833-9377, bridge: (877) 385-4099 code: 506 9191#
Quoted 5 lines
I could probably buy that, but I don't know how an HSM would work. Would we have everything we need at open for them to fire off?
/me is HSM clueless and trying to include their needs is proving a challenge.
So don't bother. Its a theoretical use for the most part so we can mangle the interface later.
Alan
Quoted 8 lines · On Mon, 18 Aug 2008 16:33:13 +0100 Alan Cox <[email protected]> wrote:
Quoted 5 lines
I could probably buy that, but I don't know how an HSM would work. Would we have everything we need at open for them to fire off?
/me is HSM clueless and trying to include their needs is proving a challenge.
So don't bother. Its a theoretical use for the most part so we can mangle the interface later.
Think of a consumer HSM application: backup to rsync.net or Amazon S3.
Instead of waiting for the whole backup to be restored, you can start using the filesystem immediately. The block-on-open hook can be used by the restore program to fetch files from the remote backup site on an as-needed basis, with a full restore going on in the background.
If the block-on-open hook can be used for that (even with additional magic, like creating empty HSM inodes with a special attr to notify "the data lives elsewhere"), HSM should be good.
The "data lives elsewhere" bit/xattr/whatever could also be used on directories, so not even the whole directory tree would have to be restored right on restore :)
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: HSM (was Re: [malware-list] TALPA - a threat model? well sorta.)
Quoted 29 lines · On Tue, 19 Aug 2008 09:14:16 +0200 Pavel Machek <[email protected]> wrote:
Quoted 28 lines · Rik van Riel
Quoted 9 lines · On Mon, 18 Aug 2008 16:33:13 +0100 Alan Cox <[email protected]> wrote:
Quoted 6 lines
I could probably buy that, but I don't know how an HSM would work. Would we have everything we need at open for them to fire off?
/me is HSM clueless and trying to include their needs is proving a challenge.
So don't bother. Its a theoretical use for the most part so we can mangle the interface later.
Think of a consumer HSM application: backup to rsync.net or Amazon S3.
Instead of waiting for the whole backup to be restored, you can start using the filesystem immediately. The block-on-open hook can be used by the restore program to fetch files from the remote backup site on an as-needed basis, with a full restore going on in the background.
If the block-on-open hook can be used for that (even with additional magic, like creating empty HSM inodes with a special attr to notify "the data lives elsewhere"), HSM should be good.
Quoted 6 lines
Quoted 3 lines
The "data lives elsewhere" bit/xattr/whatever could also be used on directories, so not even the whole directory tree would have to be restored right on restore :)
But is this really needed to be cross-filesystem thing? I'd expect this to be implemented with FUSE, maybe FUSE+unionfs...
If you think FUSE+unionfs is a cleaner solution than one hook in the VFS, I've got a bridge to sell you.
Signature · 1 line
All rights reversed.
Re: HSM (was Re: [malware-list] TALPA - a threat model? well sorta.)
Quoted 36 lines · Rik van Riel
Quoted 33 lines · On Tue, 19 Aug 2008 09:14:16 +0200 Pavel Machek <[email protected]> wrote:
Quoted 31 lines · Rik van Riel
Quoted 9 lines · On Mon, 18 Aug 2008 16:33:13 +0100 Alan Cox <[email protected]> wrote:
Quoted 6 lines
I could probably buy that, but I don't know how an HSM would work. Would we have everything we need at open for them to fire off?
/me is HSM clueless and trying to include their needs is proving a challenge.
So don't bother. Its a theoretical use for the most part so we can mangle the interface later.
Think of a consumer HSM application: backup to rsync.net or Amazon S3.
Instead of waiting for the whole backup to be restored, you can start using the filesystem immediately. The block-on-open hook can be used by the restore program to fetch files from the remote backup site on an as-needed basis, with a full restore going on in the background.
If the block-on-open hook can be used for that (even with additional magic, like creating empty HSM inodes with a special attr to notify "the data lives elsewhere"), HSM should be good.
The "data lives elsewhere" bit/xattr/whatever could also be used on directories, so not even the whole directory tree would have to be restored right on restore :)
Quoted 5 lines
Quoted 2 lines
But is this really needed to be cross-filesystem thing? I'd expect this to be implemented with FUSE, maybe FUSE+unionfs...
If you think FUSE+unionfs is a cleaner solution than one hook in the VFS, I've got a bridge to sell you.
If you can do it with one clean enough hook, I'll buy that bridge.
[If you want to do 'list directory before files are there' -- and you seem to want to from description above -- fuse seems like a way to go.]
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Re: HSM (was Re: [malware-list] TALPA - a threat model? well sorta.)
Quoted 8 lines · On Tue, 19 Aug 2008 21:20:27 +0200 Pavel Machek <[email protected]> wrote:
Quoted 2 lines
If you think FUSE+unionfs is a cleaner solution than one hook in the VFS, I've got a bridge to sell you.
If you can do it with one clean enough hook, I'll buy that bridge.
[If you want to do 'list directory before files are there' -- and you seem to want to from description above -- fuse seems like a way to go.]
For a poor man's HSM, it may be a possibility to simply restore all the files as sparse files, fetching the content on demand.
The files can be marked as HSM-empty files with some kind of bit flag or xattr.
Restoring sparse files takes very little time.
This scheme could even work for directories, so you can start using the filesystem before the whole directory tree has been restored.
All it needs is block-on-open when the file has a special flag or xattr set.
Signature · 1 line
All rights reversed.
Re: HSM (was Re: [malware-list] TALPA - a threat model? well sorta.)
Quoted 6 lines · Rik van Riel
Quoted 3 lines · On Tue, 19 Aug 2008 21:20:27 +0200 Pavel Machek <[email protected]> wrote:
Quoted 2 lines
If you think FUSE+unionfs is a cleaner solution than one hook in the VFS, I've got a bridge to sell you.
Quoted 8 lines
Quoted 5 lines
If you can do it with one clean enough hook, I'll buy that bridge.
[If you want to do 'list directory before files are there' -- and you seem to want to from description above -- fuse seems like a way to go.]
For a poor man's HSM, it may be a possibility to simply restore all the files as sparse files, fetching the content on demand.
Agreed but... I'd rather have rich man's HSM and rich man's antivirus. You know, I'm going to get very rich, owning that bridge ;-).
[Seriously, FUSE seems to be better way of implementing HSM, and single hook at open is not enough to implement non-racy antivirus...]
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 7 lines · Alan Cox
Quoted 2 lines
So, what is it that anti-malware companies do? They scan files. That's it.
Good so lets instead have a discussion about making the file event notification more scalable. That is the same thing I want for content indexing. It is the same thing you want for certain kinds of smart archiving, for on-line asynchronous backup and other stuff.
Also for hierachial storage management, which also shares they other requirement with the AV crowd that it want to be able to block the calling process until the notification is ACKed (for recalling data from offline media).
Quoted 1 line
It ought to be a simple clean syscall interface.
I was wondering whether to piggy-back on the audit code was the best idea here..
Quoted 9 lines · Alan Cox
Quoted 2 lines
So, what is it that anti-malware companies do? They scan files. That's it.
Good so lets instead have a discussion about making the file event notification more scalable. That is the same thing I want for content indexing. It is the same thing you want for certain kinds of smart archiving, for on-line asynchronous backup and other stuff.
It ought to be a simple clean syscall interface.
In this case, it seems to be the same kind of requirement that lead to dnotify and inotify, right?
So in order to get the third notification syscall interface right, the obvious questions are:
1. How do you make it cleaner than inotify?
2. How do you make it more scalable than inotify?
3. Do you need it to do anything that inotify doesn't do, beyond 1. and 2.?
Arnd <><
Quoted 4 lines · Eric Paris
No, this is not a new security system. This is not some great wonderful panacea that's going to stop every exploit. Its a file scanner. It's not bad. It clearly can help. This isn't just a pipe dream, I've got code. Where's the problem?
Just repost your patches, with a description of what is happening with them, taking into account the previously mentioned comments and we can review it again :)
No problem at all, just a normal patch lifecycle...
thanks,
greg k-h
On Wed, 13 Aug 2008 12:36:15 -0400
Eric Paris <[email protected]> wrote:
[finally good description snipped]
Quoted 2 lines
1) Kernel or userspace.
this is kind of the last question to ask, not the first ;)
Quoted 2 lines
Not to mention there cannot be any checks for files opened by suid apps in userspace.
setuid apps do not use glibc? news to me.
Quoted 4 lines
And while no specific claims are
being made about intentionally malicious code, putting this in kernel makes programs which call syscalls directly much more difficult to be used to circumvent the scanning.
[so lets ignore this one]
Quoted 2 lines
2) I think the "best" time to do scanning is at read and write. Any disagreements there?
I agree about the "read" case.
"write" is tricky... because there's many vectors to write, from write() system call to mmap() system call to truncate() system call. (yes you can write a stream of bytes that passes virus scan, and then truncate to be suddenly a live virus)
I would propose to use the word "dirty" instead, we all know what we mean by that (it's all of them) and doesn't tie our thinking to the write() system call.
I would like to introduce a concept in your discussion you did not mention yet in this email: the difference between synchronous scanning and asynchronous scanning.
It's clear from the protection model that you described that on 'read' you want to wait until the scan is done before you give the data to the process asking for it... and that's totally reasonable: "Do not give out bad data" is a very clear line in terms of security.
for the "dirty" case it gets muddy. You clearly want to scan "some time" after the write, from the principle of getting rid of malware that's on the disk, but it's unclear if this HAS to be synchronous. (obviously, synchronous behavior hurts performance bigtime so lets do as little as we can of that without hurting the protection). One advantage of doing the dirty case async (and a little time delayed) is that repeated writes will get lumped up into one scan in practice, saving a ton of performance.
(scan-on-close is just another way of implementing "delay the dirty scan").
Based on Alans comments, to me this sounds like we should have an efficient mechanism to notify userspace of "dirty events"; this is not virus scan specific in any way or form. And this mechanism likely will need to allow multiple subscribers.
for the open() case, I would argue that you don't need synchronous behavior as long as the read() case is synchronous. I can imagine that open() kicks off an async scan, and if it's done by the time the first read() happens, no blocking at all happens.
For efficiency the kernel ought to keep track of which files have been declared clean, and it needs to track of a 'generation' of the scan with which it has been found clean (so that if you update your virus definitions, you can invalidate all previous scanning just by bumping the 'generation' number in whatever format we use).
Clearly the kernel needs to wipe this clean generation number on any modification to the file (inode) of any kind. And clearly this needs to be done inside the kernel because tracking dirty operations in any other way is just insane.
Now this to me we have a few basic building blocks:
1) We need an efficient mechanism to notify userspace of files that get dirtied. Virus scanners will subscribe to this for the async dirty scanning; indexing agents also will subscribe to this.
2) We very likely should have a mechanism for a userspace app to request a scan on a file, both sync or async (O_SYNC flag?). This is useful regardless because it allows the source of many things to do the right thing.
3) we need a mechanism in the kernel to track "scanned with generation X of signatures" that invalidates on any dirty operation. The syscall from 2) will use this as a cache to be quick.
I think few people will disagree about this.
Open questions now are
4) do we have the kernel kick off an async scan in open() or do we have glibc do this
5) do we have the kernel do the sync scan on read/mmap/.. or do we have glibc do this
I think this is where the whole debate is about now.
And a few hard ones
6) how do we deal with multiple scanning agents in parallel
7) how do we prevent malware from pretending to be a virus scanner
Signature · 3 lines
If you want to reach me at my work email, use [email protected] For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 5 lines · Arjan van de Ven
for the "dirty" case it gets muddy. You clearly want to scan "some time" after the write, from the principle of getting rid of malware that's on the disk, but it's unclear if this HAS to be synchronous. (obviously, synchronous behavior hurts performance bigtime so lets do as little as we can of that without hurting the protection).
Something else to think about is what happens if the file is naturally written in pieces. For example, I've been playing with bittorrent recently, and it appears that trackerd will do something... not very intelligent in that it will repeatedly try to index a file which is being written in pieces, and in some cases, it will do things like call pdftext that aren't exactly cheap. A timeout *can* help (i.e., don't try to scan/index this file until 15 minutes after the last write), but it won't help if the torrent is very large, or the download bitrate is very slow. One very simple workaround is to disable trackerd altogether while you are downloading the file, but that's not very pleasant solution; it's horribly manual.
Most of this may end up being outside of the kernel (i.e.,some kind of interface where a bittorrent client can say, "look this file is still being downloaded, so it's don't bother scanning it unless some process *other* than the bittorrent client tries to access the file". And maybe there should be some other more complex policies, such as the bittorrent client explicitly telling the indexer/scanner that the file is has been completely downloaded, so it's safe to index it now.
But what this points out is that if you want a good solution, (a) it probably shouldn't all be in the kernel, since trying to get all of this complexity into the kernel will be painful, and (b) the policy about whether or not a bittorrent client should be allowed to say, "it's OK not to check the file until it's completely downloaded, even if I am handing out pieces to other people over the network --- after all the entire file has its own SHA checksum for data integrity verification --- is very much a policy question where different system administrators will come down on different sides about what should and shouldn't be allowed --- and therefore this kind of policy decision should ****NOT**** be in the kernel.
Quoted 5 lines
For efficiency the kernel ought to keep track of which files have been declared clean, and it needs to track of a 'generation' of the scan with which it has been found clean (so that if you update your virus definitions, you can invalidate all previous scanning just by bumping the 'generation' number in whatever format we use).
We have an i_version support for NFSv4, so we have that already as far as the version of the file. We can have a single bit which means "block on open" that is stored on a file, and some kind of policy which dictates whether or not any modification to the file contens should automatically set the bit.
However, questions of which version of virus database was used to scan a particular file should be stored outside of the filesystem, since each product will have its own version namespace, and the questions of what happens if a user switches from one version checker to another is going to be messy. So better that this be done in userspace, and that this information be stored in some on-disk database.
- Ted
Quoted 28 lines · On Wed, 13 Aug 2008 14:15:49 -0400 Theodore Tso <[email protected]> wrote:
Quoted 5 lines · Arjan van de Ven
for the "dirty" case it gets muddy. You clearly want to scan "some time" after the write, from the principle of getting rid of malware that's on the disk, but it's unclear if this HAS to be synchronous. (obviously, synchronous behavior hurts performance bigtime so lets do as little as we can of that without hurting the protection).
Something else to think about is what happens if the file is naturally written in pieces. For example, I've been playing with bittorrent recently, and it appears that trackerd will do something... not very intelligent in that it will repeatedly try to index a file which is being written in pieces, and in some cases, it will do things like call pdftext that aren't exactly cheap. A timeout *can* help (i.e., don't try to scan/index this file until 15 minutes after the last write), but it won't help if the torrent is very large, or the download bitrate is very slow. One very simple workaround is to disable trackerd altogether while you are downloading the file, but that's not very pleasant solution; it's horribly manual.
Most of this may end up being outside of the kernel (i.e.,some kind of interface where a bittorrent client can say, "look this file is still being downloaded, so it's don't bother scanning it unless some process *other* than the bittorrent client tries to access the file". And maybe there should be some other more complex policies, such as the bittorrent client explicitly telling the indexer/scanner that the file is has been completely downloaded, so it's safe to index it now.
Quoted 4 lines
verification --- is very much a policy question where different system administrators will come down on different sides about what should and shouldn't be allowed --- and therefore this kind of policy decision should ****NOT**** be in the kernel.
exactly. Even more, since this is async work, the scheduling of the order of work also is a policy.. and userland is again the right place for that.
Quoted 7 lines
Quoted 6 lines
For efficiency the kernel ought to keep track of which files have been declared clean, and it needs to track of a 'generation' of the scan with which it has been found clean (so that if you update your virus definitions, you can invalidate all previous scanning just by bumping the 'generation' number in whatever format we use).
Quoted 8 lines
We have an
i_versionsupport for NFSv4, so we have that already as far as the version of the file. We can have a single bit which means "block on open" that is stored on a file, and some kind of policy which dictates whether or not any modification to the file contens should automatically set the bit.However, questions of which version of virus database was used to scan a particular file should be stored outside of the filesystem, since
well I was assuming we only store this in memory (say in the inode) and just rescan the file if we destroy the in memory inode.
I don't see the need for this to be persistent data; in fact I assume (Eric, please confirm) that this data is not *supposed* to be persistent.
Quoted 2 lines
each product will have its own version namespace, and the questions of what happens if a user switches from one version checker to another is
yes that's a hard question; what if you have 2 virus scanners active.
(they could register a version of the database with the kernel, and the in kernel version-cookie could be a hash of all registered versions I suppose.. if anything changes ever we just rehash and scan as if we have to do that)
Signature · 3 lines
If you want to reach me at my work email, use [email protected] For development, discussion and tips for power savings, visit http://www.lesswatts.org
Arjan van de Ven wrote on 13/08/2008 19:21:49:
Quoted 8 lines
Quoted 2 lines
However, questions of which version of virus database was used to scan a particular file should be stored outside of the filesystem, since
well I was assuming we only store this in memory (say in the inode) and just rescan the file if we destroy the in memory inode.
I don't see the need for this to be persistent data; in fact I assume (Eric, please confirm) that this data is not *supposed* to be persistent.
Correct.
Quoted 9 lines
Quoted 2 lines
each product will have its own version namespace, and the questions of what happens if a user switches from one version checker to another is
yes that's a hard question; what if you have 2 virus scanners active.
(they could register a version of the database with the kernel, and the in kernel version-cookie could be a hash of all registered versions I suppose.. if anything changes ever we just rehash and scan as if we have to do that)
It is in fact really simple with the proposed inode versioning approach. Any client which wants to invalidate the cache just needs to bump the global version number which will force a rescan on next access.
Tvrtko
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Quoted 27 lines · Theodore Tso
Quoted 5 lines · Arjan van de Ven
for the "dirty" case it gets muddy. You clearly want to scan "some time" after the write, from the principle of getting rid of malware that's on the disk, but it's unclear if this HAS to be synchronous. (obviously, synchronous behavior hurts performance bigtime so lets do as little as we can of that without hurting the protection).
Something else to think about is what happens if the file is naturally written in pieces. For example, I've been playing with bittorrent recently, and it appears that trackerd will do something... not very intelligent in that it will repeatedly try to index a file which is being written in pieces, and in some cases, it will do things like call pdftext that aren't exactly cheap. A timeout *can* help (i.e., don't try to scan/index this file until 15 minutes after the last write), but it won't help if the torrent is very large, or the download bitrate is very slow. One very simple workaround is to disable trackerd altogether while you are downloading the file, but that's not very pleasant solution; it's horribly manual.
Most of this may end up being outside of the kernel (i.e.,some kind of interface where a bittorrent client can say, "look this file is still being downloaded, so it's don't bother scanning it unless some process *other* than the bittorrent client tries to access the file". And maybe there should be some other more complex policies, such as the bittorrent client explicitly telling the indexer/scanner that the file is has been completely downloaded, so it's safe to index it now.
Quoted 11 lines
But what this points out is that if you want a good solution, (a) it probably shouldn't all be in the kernel, since trying to get all of this complexity into the kernel will be painful, and (b) the policy about whether or not a bittorrent client should be allowed to say, "it's OK not to check the file until it's completely downloaded, even if I am handing out pieces to other people over the network --- after all the entire file has its own SHA checksum for data integrity verification --- is very much a policy question where different system administrators will come down on different sides about what should and shouldn't be allowed --- and therefore this kind of policy decision should ****NOT**** be in the kernel.
I never suggested putting a scanner in kernel. Sound like you want the "allow don't cache" response from your userspace scanner while this is going on. The kernel doesn't need to be making decisions about when to send events, nor should userspace tell the kernel not to send events. Its up to whatever the scanner is to agree not to actually do any scanning...
Quoted 6 lines
However, questions of which version of virus database was used to scan a particular file should be stored outside of the filesystem, since each product will have its own version namespace, and the questions of what happens if a user switches from one version checker to another is going to be messy. So better that this be done in userspace, and that this information be stored in some on-disk database.
No. How in the heck can some out of kernel database store information about what inodes have been scanned in any even slightly sane way? And people think the race between open and read is too large and you suggest moving clean/dirty marking to a userspace database? I MUCH prefer my (and it sounds like arjan agrees) clean/dirty versioned flag in inode.
-Eris
Quoted 6 lines · Eric Paris
I never suggested putting a scanner in kernel. Sound like you want the "allow don't cache" response from your userspace scanner while this is going on. The kernel doesn't need to be making decisions about when to send events, nor should userspace tell the kernel not to send events. Its up to whatever the scanner is to agree not to actually do any scanning...
And if the system isn't running a virus checker, but just a file indexer (ala tracker), it shouldn't go to userspace at all. In that case all that is necessary is an asynchronous notification.
Also something else that is needed is support for multiple clients. (i.e., what happens if the user runs two virus checkers, or a virus checker plus a hierarchical storage manager driving a tape robot, or all of the above plus trackerd --- where some clients need to block open(2) access, and some do not need block open(2) --- and in the case of HSM, ordering becomes important; you want to retrieve the file from the tape robot first, *then* scan it using the virus checker. :-)
Quoted 5 lines
No. How in the heck can some out of kernel database store information about what inodes have been scanned in any even slightly sane way? And people think the race between open and read is too large and you suggest moving clean/dirty marking to a userspace database? I MUCH prefer my (and it sounds like arjan agrees) clean/dirty versioned flag in inode.
Don't ask me; I think most AV checkers for linux are security theater and not very much use (other than making money for the AV company's shareholders) anyway. I thought you were the one who wanted to record information about which version of the virus db a particular file had been scanned against. The place where I can see this being useful is what happens you get a new virus DB, and so you need to start scanning all of the files in your 5TB enterprise file server --- and then the system crashes or it needs to be taken down for scheduled maintenance.
You want to have *some* off-line database for storing this information, since it would be silly to want to have the first thing that happens after a new virus DB gets downloaded is to interate over the entire filesystem, clearing a persistent the "clean" bit --- that would take *forever* on a 5TB filerserver; and what happens if you crash in the middle of clearing the "clean" bit.. And if the system gets shutdown in the middle of the scan, you need some way of remembering which inodes have been scanned using the "new" db, and which ones haven't yet been scanned via the new virus db. All of this should be kept in userpsace, and is strictly speaking Not Our Problem.
I'm just arguing that there should be absolutely *no* support in the kernel for solving this particular problem, since the question of whether a file has been scanned with a particular version of the virus DB is purely a userspace problem.
- Ted
Quoted 12 lines
-----Original Message-----
From: [email protected] [mailto:malware-list- [email protected]] On Behalf Of Theodore Tso
Sent: Wednesday, August 13, 2008 3:29 PM
To: Eric Paris
Cc: [email protected]; [email protected]; malware- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Arjan van de Ven
Subject: Re: [malware-list] TALPA - a threat model? well sorta.
Quoted 7 lines
Also something else that is needed is support for multiple clients. (i.e., what happens if the user runs two virus checkers, or a virus checker plus a hierarchical storage manager driving a tape robot, or all of the above plus trackerd --- where some clients need to block open(2) access, and some do not need block open(2) --- and in the case of HSM, ordering becomes important; you want to retrieve the file from the tape robot first, *then* scan it using the virus checker. :-)
The issue of multiple clients does need to be accounted for. However, I will mention that it is unusual (at least in my experience) to actually run two AV products at the same time in "realtime" mode. We strongly recommend that anyone who installs our product should remove any other AV product on the system -- for technical reasons, not financial -- since they've already paid for ours by the time they get to this point. I am not aware of anyone objecting to that idea.
Quoted 4 lines
I'm just arguing that there should be absolutely *no* support in the kernel for solving this particular problem, since the question of whether a file has been scanned with a particular version of the virus DB is purely a userspace problem.
Caching of previous results can be done in either user space or kernel space. We have seen it both ways. Wherever it is done, I would say that rather than record AV signature file version numbers, there should be a mechanism for the application to invalidate or flush the cache whenever a signature update is done. There are other circumstances where that would also be useful -- such as if the user changes a scanning option in a way that increases the strictness of the scan. In other words, a file that was marked as clean based on one level of strictness may not be clean under a stricter scan. You wouldn't want the cache to prevent it from being scanned under such circumstances.
Jon Press
Theodore Tso wrote on 13/08/2008 20:29:22:
Quoted 2 lines · Eric Paris
Quoted 1 line
I never suggested putting a scanner in kernel. Sound like you want
the
Quoted 2 lines
Quoted 2 lines
"allow don't cache" response from your userspace scanner while this is going on. The kernel doesn't need to be making decisions about when
to
Quoted 8 lines
Quoted 3 lines
send events, nor should userspace tell the kernel not to send events. Its up to whatever the scanner is to agree not to actually do any scanning...
And if the system isn't running a virus checker, but just a file indexer (ala tracker), it shouldn't go to userspace at all. In that case all that is necessary is an asynchronous notification.
Quoted 7 lines
Also something else that is needed is support for multiple clients. (i.e., what happens if the user runs two virus checkers, or a virus checker plus a hierarchical storage manager driving a tape robot, or all of the above plus trackerd --- where some clients need to block open(2) access, and some do not need block open(2) --- and in the case of HSM, ordering becomes important; you want to retrieve the file from the tape robot first, *then* scan it using the virus checker. :-)
Hm, maybe by implementing a facility with which a client can register it's interface usage intent? Something like:
register(I_HAVE_NO_INTEREST_IN_CONTENT);register(I_WANT_TO_EXAMINE_CONTENT);
All former ones would run first because they only want to have the opportunity to block and do something unrelated to file content (like HSMs), and later group would be ran last since they want to examine the content.
Ordering inside those two groups is not important because I don't see how a model other than restrictive can make sense with content security scanning.
Quoted 3 lines
Quoted 3 lines
No. How in the heck can some out of kernel database store information about what inodes have been scanned in any even slightly sane way? And people think the race between open and read is too large and you
suggest
Quoted 23 lines
Quoted 2 lines
moving clean/dirty marking to a userspace database? I MUCH prefer my (and it sounds like arjan agrees) clean/dirty versioned flag in inode.
Don't ask me; I think most AV checkers for linux are security theater and not very much use (other than making money for the AV company's shareholders) anyway. I thought you were the one who wanted to record information about which version of the virus db a particular file had been scanned against. The place where I can see this being useful is what happens you get a new virus DB, and so you need to start scanning all of the files in your 5TB enterprise file server --- and then the system crashes or it needs to be taken down for scheduled maintenance.
You want to have *some* off-line database for storing this information, since it would be silly to want to have the first thing that happens after a new virus DB gets downloaded is to interate over the entire filesystem, clearing a persistent the "clean" bit --- that would take *forever* on a 5TB filerserver; and what happens if you crash in the middle of clearing the "clean" bit.. And if the system gets shutdown in the middle of the scan, you need some way of remembering which inodes have been scanned using the "new" db, and which ones haven't yet been scanned via the new virus db. All of this should be kept in userpsace, and is strictly speaking Not Our Problem.
Quoted 4 lines
I'm just arguing that there should be absolutely *no* support in the kernel for solving this particular problem, since the question of whether a file has been scanned with a particular version of the virus DB is purely a userspace problem.
The thing is the idea never was for clean-dirty "database" to be persistent but to have the same lifetime as the inode (in memory one). And once the cache/database gets invalidated re-scanning happens on-demand so the 5TB problem does not exist. Concerns about multiple clients where every has a different versioning scheme are also not relevant with the proposed versioning scheme (see my reply to Arjan).
So this problem can be solved in kernel in a much more performant and secure way than it would be possible in userspace and all that with just a handfull of lines of code.
--
Tvrtko A. Ursulin
Senior Software Engineer, Sophos
"Views and opinions expressed in this email are strictly those of the author.
The contents has not been reviewed or approved by Sophos."
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Quoted 12 lines
-----Original Message-----
From: [email protected] [mailto:malware-list- [email protected]] On Behalf Of [email protected]
Sent: Thursday, August 14, 2008 5:31 AM
To: Theodore Tso
Cc: [email protected]; [email protected]; malware- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Arjan van de Ven
Subject: Re: [malware-list] TALPA - a threat model? well sorta.
Quoted 1 line
Hm, maybe by implementing a facility with which a client can register
it's
Quoted 8 lines
interface usage intent? Something like:
register(I_HAVE_NO_INTEREST_IN_CONTENT);register(I_WANT_TO_EXAMINE_CONTENT);All former ones would run first because they only want to have the opportunity to block and do something unrelated to file content (like HSMs), and later group would be ran last since they want to examine
the
Quoted 3 lines
content.
Ordering inside those two groups is not important because I don't see
how
Quoted 2 lines
a model other than restrictive can make sense with content security scanning.
I'm not sure I understand why "interest in content" means not blocking, and vice versa. However, I think this is a good idea if made more explicit, i.e.:
Quoted 2 lines
register(BLOCK);
register(DON'T_BLOCK);
Jon Press
Jonathan Press wrote on 14/08/2008 13:03:40:
Quoted 13 lines
Quoted 1 line
Hm, maybe by implementing a facility with which a client can register
it's
Quoted 8 lines
interface usage intent? Something like:
register(I_HAVE_NO_INTEREST_IN_CONTENT);register(I_WANT_TO_EXAMINE_CONTENT);All former ones would run first because they only want to have the opportunity to block and do something unrelated to file content (like HSMs), and later group would be ran last since they want to examine
the
Quoted 2 lines
content.
Quoted 8 lines
Quoted 1 line
Ordering inside those two groups is not important because I don't see
how
Quoted 2 lines
a model other than restrictive can make sense with content security scanning.
I'm not sure I understand why "interest in content" means not blocking, and vice versa. However, I think this is a good idea if made more explicit, i.e.:
Small misunderstanding because both would block. If you go back to Ted's original post I was replying to, he was worried about how would anti-malware scanning interact with HSM since both may end up using the same interface. HSM, as far as I understand it, needs to block on open and "plant" the right file in place, while anti-malware also needs to block and examine the right content. That is why ordering matters, anti-malware needs to run after the content is put in place. And that is what my idea solves (slight overstatement since I spent only seconds on it) by separating them in two groups of clients. First which has no interest in content and second which does.
--
Tvrtko A. Ursulin
Senior Software Engineer, Sophos
"Views and opinions expressed in this email are strictly those of the author.
The contents has not been reviewed or approved by Sophos."
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Hi!
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
a model other than restrictive can make sense with content security scanning.
I'm not sure I understand why "interest in content" means not blocking, and vice versa. However, I think this is a good idea if made more explicit, i.e.:
Quoted 10 lines
Small misunderstanding because both would block. If you go back to Ted's original post I was replying to, he was worried about how would anti-malware scanning interact with HSM since both may end up using the same interface. HSM, as far as I understand it, needs to block on open and "plant" the right file in place, while anti-malware also needs to block and examine the right content. That is why ordering matters, anti-malware needs to run after the content is put in place. And that is what my idea solves (slight overstatement since I spent only seconds on it) by separating them in two groups of clients. First which has no interest in content and second which does.
I don't see why HSMs are mixed into this discussion -- they appear very different. For one, you probably have just one filesystem with tape storage, so it makes sense to implement HSM at filesystem level; probably with something like FUSE+unionfs.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 6 lines · On Thu, Aug 14, 2008 at 10:30:56AM +0100, [email protected] wrote:
The thing is the idea never was for clean-dirty "database" to be persistent but to have the same lifetime as the inode (in memory one). And once the cache/database gets invalidated re-scanning happens on-demand so the 5TB problem does not exist. Concerns about multiple clients where every has a different versioning scheme are also not relevant with the proposed versioning scheme (see my reply to Arjan).
So in essence, what I hear you saying is that all AV products want to work in a mode where the moment the inode falls out of the inode cache, and we lose the "clean" bit, when the inode is brought back into the cache, it will be scanned again. That is, the "clean" bit is never persistent, and never needs to be stored in memory.
That seems fair; if it turns out there is an AV product that wants to optimize this a bit further, as long as we provide a persistent inode version/generation number, they can always do their own persistent database in userspace.
- Ted
Quoted 8 lines · Theodore Tso
Quoted 6 lines · On Thu, Aug 14, 2008 at 10:30:56AM +0100, [email protected] wrote:
The thing is the idea never was for clean-dirty "database" to be persistent but to have the same lifetime as the inode (in memory one). And once the cache/database gets invalidated re-scanning happens on-demand so the 5TB problem does not exist. Concerns about multiple clients where every has a different versioning scheme are also not relevant with the proposed versioning scheme (see my reply to Arjan).
Quoted 5 lines
So in essence, what I hear you saying is that all AV products want to work in a mode where the moment the inode falls out of the inode cache, and we lose the "clean" bit, when the inode is brought back into the cache, it will be scanned again. That is, the "clean" bit is never persistent, and never needs to be stored in memory.
There needs to be a way to say that an inode in cache needs to be rescanned. 3 states this flag can be. Clean, Dirty, Infected. The current talpa solution involves a global monotomically increasing counter every time you change virus defs or make some "interesting" change. If global == inode flag we are clean. If global == negative inode flag we are infected. if global > inode flag we are dirty and need a scan.
Quoted 4 lines
That seems fair; if it turns out there is an AV product that wants to optimize this a bit further, as long as we provide a persistent inode version/generation number, they can always do their own persistent database in userspace.
exporting i_version might be useful for better userspace caching, although I've yet to see any reasonable description of how a userspace database can map between data on disk and what they have in userspace. How can a userspace process, given 2 file descriptors know they are actually the same thing on disk?
Quoted 8 lines · Eric Paris
There needs to be a way to say that an inode in cache needs to be rescanned. 3 states this flag can be. Clean, Dirty, Infected. The current talpa solution involves a global monotomically increasing counter every time you change virus defs or make some "interesting" change. If global == inode flag we are clean. If global == negative inode flag we are infected. if global > inode flag we are dirty and need a scan.
"Infected" just means to instantly return an error when the file is opened or if an already opened file descriptor is read or mmap'ed, right? If file is already mmaped(), what's the plan? Send a kill -9 to the process, even if it ends up kill off an emacs or openoffice process?
Quoted 11 lines
Quoted 4 lines
That seems fair; if it turns out there is an AV product that wants to optimize this a bit further, as long as we provide a persistent inode version/generation number, they can always do their own persistent database in userspace.
exporting
i_versionmight be useful for better userspace caching, although I've yet to see any reasonable description of how a userspace database can map between data on disk and what they have in userspace. How can a userspace process, given 2 file descriptors know they are actually the same thing on disk?
If a userspace database knows that inode X, i_version Y was checked a day ago, and inode X still has i_version Y, even if that inode has been evicted from memory, the contents will be the same absent root messing about with direct access to the block device. If there was an intervening boot, the someone could remove the disk, edit the disk block directly -- but that person could also add a backdoor to the kernel while they were at it.
If your threat model is, "we do file scanning; that's it", then having an external database which uses the inode number and i_version as a tuple makes a lot of sense --- for filesystems where i_version is getting bumped on every disk write, which is needed to support NFSv4 cache support, anyway.
- Ted
Quoted 10 lines · Theodore Tso
Quoted 8 lines · Eric Paris
There needs to be a way to say that an inode in cache needs to be rescanned. 3 states this flag can be. Clean, Dirty, Infected. The current talpa solution involves a global monotomically increasing counter every time you change virus defs or make some "interesting" change. If global == inode flag we are clean. If global == negative inode flag we are infected. if global > inode flag we are dirty and need a scan.
Quoted 5 lines
"Infected" just means to instantly return an error when the file is opened or if an already opened file descriptor is read or mmap'ed, right? If file is already
mmaped(), what's the plan? Send a kill -9 to the process, even if it ends up kill off an emacs or openoffice process?
We don't have a revocation mechanism in linux and this isn't about adding one. Your trying to turn this into something it isn't. If you have it opened and mmap'd you've got that file for as long as you want. I've already said that given Arjan's belief that we can move it read/write instead of open/close we are moving the open->read race to a mmap->fault race. It isn't perfect at stopping bad data flow, but its darn sure a lot better than nothing.
Quoted 13 lines
Quoted 6 lines
Quoted 5 lines
That seems fair; if it turns out there is an AV product that wants to optimize this a bit further, as long as we provide a persistent inode version/generation number, they can always do their own persistent database in userspace.
Quoted 6 lines
exporting
i_versionmight be useful for better userspace caching, although I've yet to see any reasonable description of how a userspace database can map between data on disk and what they have in userspace. How can a userspace process, given 2 file descriptors know they are actually the same thing on disk?
Quoted 7 lines
If a userspace database knows that inode X,
i_versionY was checked a day ago, and inode X still hasi_versionY, even if that inode has been evicted from memory, the contents will be the same absent root messing about with direct access to the block device. If there was an intervening boot, the someone could remove the disk, edit the disk block directly -- but that person could also add a backdoor to the kernel while they were at it.
is i_version an on disk think? didn't realize that and just assumed it was in in core thing. I wouldn't have an issue sending i_version to the userspace scanner for them to use as they like.
-Eric
Quoted 5 lines · Eric Paris
is
i_versionan on disk think? didn't realize that and just assumed it was in in core thing. I wouldn't have an issue sendingi_versionto the userspace scanner for them to use as they like.
It's on-disk for some filesystems, in order to support NFsv4 advanced caching semantics (which means i_version has to survive a reboot, which means it has to be on disk). It is *not* on disk for ext3, although it is for ext4.
- Ted
Quoted 11 lines · Theodore Tso
Quoted 5 lines · Eric Paris
is
i_versionan on disk think? didn't realize that and just assumed it was in in core thing. I wouldn't have an issue sendingi_versionto the userspace scanner for them to use as they like.It's on-disk for some filesystems, in order to support NFsv4 advanced caching semantics (which means
i_versionhas to survive a reboot, which means it has to be on disk). It is *not* on disk for ext3, although it is for ext4.
Is there some pragmatic way in kernel to know if the inode I have corresponds to a persistent or in core only inode? I'll poke around myself and look.
-Eris
Quoted 11 lines · Theodore Tso
Quoted 5 lines · Eric Paris
is
i_versionan on disk think? didn't realize that and just assumed it was in in core thing. I wouldn't have an issue sendingi_versionto the userspace scanner for them to use as they like.It's on-disk for some filesystems, in order to support NFsv4 advanced caching semantics (which means
i_versionhas to survive a reboot, which means it has to be on disk). It is *not* on disk for ext3, although it is for ext4.
It's not used at all on regular files except for ext4 with a non-default, undocumented mount option. XFS will grow it soon in a similar way as ext4, except that it will be documented or I might have even figured out by then how to just enabled it from nfsd.
Quoted 5 lines · Christoph Hellwig
It's not used at all on regular files except for ext4 with a non-default, undocumented mount option. XFS will grow it soon in a similar way as ext4, except that it will be documented or I might have even figured out by then how to just enabled it from nfsd.
We do need a standardized way of enabling it (since it does cost you something in performance, so not everyone will want it on), and a standardized way of reading i_version out to userspace. Maybe a mount option is the right way to do it, maybe not.
We may want to take this to the linux-fs list and try to get agreements on these points; the main reason why it's not enabled by default in ext4 is because the NFSv4 advanced caching code is in common use (is it even in mainline)?
- Ted
Quoted 9 lines · Theodore Tso
We do need a standardized way of enabling it (since it does cost you something in performance, so not everyone will want it on), and a standardized way of reading
i_versionout to userspace. Maybe a mount option is the right way to do it, maybe not.We may want to take this to the linux-fs list and try to get agreements on these points; the main reason why it's not enabled by default in ext4 is because the NFSv4 advanced caching code is in common use (is it even in mainline)?
I have not yet seen code actually using it at all, neither in mainline nor on one of the many nfs development lists.
Quoted 6 lines · Christoph Hellwig
Quoted 5 lines · Theodore Tso
We do need a standardized way of enabling it (since it does cost you something in performance, so not everyone will want it on), and a standardized way of reading
i_versionout to userspace. Maybe a mount option is the right way to do it, maybe not.
Quoted 7 lines
Quoted 4 lines
We may want to take this to the linux-fs list and try to get agreements on these points; the main reason why it's not enabled by default in ext4 is because the NFSv4 advanced caching code is in common use (is it even in mainline)?
I have not yet seen code actually using it at all, neither in mainline nor on one of the many nfs development lists.
Oops, I'd love to, and it should be very easy. How do I find out if i_version is supported on a given superblock?
There's nothing particularly "advanced" about this, by the way--this is a very minor variation on the caching model that nfs has always had, and our nfsv4 server is currently pretty broken without it.
Actually, it's pretty broken even on nfsv2/v3 for filesystems with poor time resolution.
--b.
Quoted 5 lines · J. Bruce Fields
Quoted 2 lines
I have not yet seen code actually using it at all, neither in mainline nor on one of the many nfs development lists.
Oops, I'd love to, and it should be very easy. How do I find out if
i_versionis supported on a given superblock?
We don't have a way of exporting this fact at the moment. I assume the best way would be to add a flag in struct super.
Quoted 3 lines
There's nothing particularly "advanced" about this, by the way--this is a very minor variation on the caching model that nfs has always had, and our nfsv4 server is currently pretty broken without it.
Well, if you're willing to try it out, as I've mentioned on my blog[1][2], ext4 is working pretty well on my laptop --- I'm running it as my primary filesystem. There are a few problems with ext3 filesystems converted to use ext4, as opposed to starting afresh via "mke2fs -t ext4dev /dev/hdXX" that we've just found in the past week (and fixed within a day or two, although they haven't been pushed to Linus yet), but overall, it's been pretty stable.
So this would be a good time for someone who is familiar wiht NFSv4 to try it out and let us know if the i_version support is as you would like in ext4 --- we're in the bugfixing/stablization phase right now, so this would be an ideal time to get that feedback. For more information, on how to get started, please see:
http://ext4.wiki.kernel.org/index.php/Ext4_Howto
for instructions, and mount the filesystem with the "-o i_version" mount option.
Quoted 2 lines
Actually, it's pretty broken even on nfsv2/v3 for filesystems with poor time resolution.
And that's fixed in ext4 as well....
- Ted
[1] http://thunk.org/tytso/blog/2008/06/30/ext4-is-now-the-primary-filesystem-on-my-laptop/
[2] http://thunk.org/tytso/blog/2008/08/08/fast-ext4-fsck-times/
Quoted 28 lines · Theodore Tso
Quoted 5 lines · J. Bruce Fields
Quoted 2 lines
I have not yet seen code actually using it at all, neither in mainline nor on one of the many nfs development lists.
Oops, I'd love to, and it should be very easy. How do I find out if
i_versionis supported on a given superblock?We don't have a way of exporting this fact at the moment. I assume the best way would be to add a flag in struct super.
Quoted 3 lines
There's nothing particularly "advanced" about this, by the way--this is a very minor variation on the caching model that nfs has always had, and our nfsv4 server is currently pretty broken without it.
Well, if you're willing to try it out, as I've mentioned on my blog[1][2], ext4 is working pretty well on my laptop --- I'm running it as my primary filesystem. There are a few problems with ext3 filesystems converted to use ext4, as opposed to starting afresh via "mke2fs -t ext4dev
/dev/hdXX" that we've just found in the past week (and fixed within a day or two, although they haven't been pushed to Linus yet), but overall, it's been pretty stable.So this would be a good time for someone who is familiar wiht NFSv4 to try it out and let us know if the
i_versionsupport is as you would like in ext4 --- we're in the bugfixing/stablization phase right now, so this would be an ideal time to get that feedback. For more information, on how to get started, please see:
Quoted 4 lines
http://ext4.wiki.kernel.org/index.php/Ext4_Howto
for instructions, and mount the filesystem with the "-o
i_version" mount option.
Neato, thanks. I set up a toy ext4 filesystem (just a 512 meg sparse file loopback mounted) on one of my test machines--so I just need to add the superblock flag and a bit of nfsd code and then I should be able to try it out.
Quoted 4 lines
Quoted 2 lines
Actually, it's pretty broken even on nfsv2/v3 for filesystems with poor time resolution.
And that's fixed in ext4 as well....
That's an improvement, yes. Of course the time is still updated only every jiffy, so there's still a race:
file updated
client checks ctime
file updated again within a jiffy
client checks ctime again, concludes file hasn't changed.
--b.
Quoted 12 lines · Theodore Tso
Quoted 5 lines · Christoph Hellwig
It's not used at all on regular files except for ext4 with a non-default, undocumented mount option. XFS will grow it soon in a similar way as ext4, except that it will be documented or I might have even figured out by then how to just enabled it from nfsd.
We do need a standardized way of enabling it (since it does cost you something in performance, so not everyone will want it on), and a standardized way of reading
i_versionout to userspace. Maybe a mount option is the right way to do it, maybe not.
Quoted 4 lines
We may want to take this to the linux-fs list and try to get agreements on these points; the main reason why it's not enabled by default in ext4 is because the NFSv4 advanced caching code is in common use (is it even in mainline)?
could you do something like defining a namespace inside posix attributes and then setting up a mechanism in the kernel to alert if the attributes change (with the entire namespace getting cleared if the file gets dirtied)?
this would have the advantage of storing the clean/dirty state in a known location on the filesystem (you would need to enable posix attributes, but that should be an acceptable limit), and would allow multiple scanners (virii, indexing, etc) to do their own thing at their own schedule when things get dirtied.
the userspace library calls that open the files would be able to take care of the issue of deciding which of the configured scanners on a system should be called for each attribute that's not set on a file.
This would seem to require minimal kernel support
1. reserve a attribute namespace
2. clear that namespace if the file is dirtied
3. provide a notification mechanism for programs to subscribe to that lists all files where the namespace was not already empty when the file was dirtied.
everything else can be userspace.
David Lang
Quoted 4 lines · On Thu, Aug 14, 2008 at 06:44:33PM -0700, [email protected] wrote:
could you do something like defining a namespace inside posix attributes and then setting up a mechanism in the kernel to alert if the attributes change (with the entire namespace getting cleared if the file gets dirtied)?
According to Eric Paris the clean/dirty state is only stored in memory. We could use the extended attribute interface as a way of not defining a new system call, or some other interface, but I'm not sure it's such a great match given that the extended attributes interface are designed for persistent data.
I agree that doesn't actually work very well for the tracker use case, where you the clean/dirty bit to be persistent (in case the tracker is disabled due to the fact you are running on battery, for example, and then you reboot).
- Ted
Quoted 12 lines · On Thu, 14 Aug 2008 22:04:00 -0400 Theodore Tso <[email protected]> wrote:
Quoted 4 lines · On Thu, Aug 14, 2008 at 06:44:33PM -0700, [email protected] wrote:
could you do something like defining a namespace inside posix attributes and then setting up a mechanism in the kernel to alert if the attributes change (with the entire namespace getting cleared if the file gets dirtied)?
According to Eric Paris the clean/dirty state is only stored in memory. We could use the extended attribute interface as a way of not defining a new system call, or some other interface, but I'm not sure it's such a great match given that the extended attributes interface are designed for persistent data.
Quoted 5 lines
I agree that doesn't actually work very well for the tracker use case, where you the clean/dirty bit to be persistent (in case the tracker is disabled due to the fact you are running on battery, for example, and then you reboot).
but we need a "give me all dirty files" solution, not a "is this file dirty" solution.
I do not want a virus scanner to constantly have to poll the whole fs for dirty files ;-)
Signature · 3 lines
If you want to reach me at my work email, use [email protected] For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 21 lines · Arjan van de Ven
Quoted 12 lines · On Thu, 14 Aug 2008 22:04:00 -0400 Theodore Tso <[email protected]> wrote:
Quoted 4 lines · On Thu, Aug 14, 2008 at 06:44:33PM -0700, [email protected] wrote:
could you do something like defining a namespace inside posix attributes and then setting up a mechanism in the kernel to alert if the attributes change (with the entire namespace getting cleared if the file gets dirtied)?
According to Eric Paris the clean/dirty state is only stored in memory. We could use the extended attribute interface as a way of not defining a new system call, or some other interface, but I'm not sure it's such a great match given that the extended attributes interface are designed for persistent data.
Quoted 5 lines
I agree that doesn't actually work very well for the tracker use case, where you the clean/dirty bit to be persistent (in case the tracker is disabled due to the fact you are running on battery, for example, and then you reboot).
Quoted 5 lines
but we need a "give me all dirty files" solution, not a "is this file dirty" solution.
I do not want a virus scanner to constantly have to poll the whole fs for dirty files ;-)
I'm not sure.
there are two situations (with the transition between them)
1. unscanned system, we want to do everything. (this happens immediatly after a new signature file is deployed)
here you do just want to filter out the files that have been scanned from the list of everything, and you probably want to check at the time of scanning the file in case it was opened (and scanned) in the meantime.
2. mostly scanned system, we only want to scan files that have been dirtied.
here you don't need to scan everything, you only need to scan in two cases
2a. the file was dirtied (you learn about it and add it to the queue of files to scan when you get around to it)
2b. an unscanned file is opened (the library detects that the file was not marked approved by all the current scanners, so it invokes the scanners on this file before completing the open, or copy for mmap, or whatever)
In the first case the attributes work "don't bother scanning me".
In the second case they also work (becouse you aren't trying to scan everything)
the only time there is a headache is in the transition between them when you have scanned a lot of the system, but not all of it, and the machine was rebooted so you lost track of what you had scanned.
it shouldn't be too hard to deal with this. even if you never resume the scan you are still safe (becouse of the scan-on-open), but it's also possible to either do a find f(or equivalent) or files without the attribute and store the results (similar to updatedb) and then updating the file to mark the files as being scanned (update in place, change the first character or something to be fairly crash safe). after the full list of files has been scanned shift to the second mode.
the sweep scan should be a background task, possibly disabled when on battery power.
why would this not satisfy the requirements?
David Lang
Hi,
Quoted 46 lines · [email protected] writes:
Quoted 21 lines · Arjan van de Ven
Quoted 12 lines · On Thu, 14 Aug 2008 22:04:00 -0400 Theodore Tso <[email protected]> wrote:
Quoted 4 lines · On Thu, Aug 14, 2008 at 06:44:33PM -0700, [email protected] wrote:
could you do something like defining a namespace inside posix attributes and then setting up a mechanism in the kernel to alert if the attributes change (with the entire namespace getting cleared if the file gets dirtied)?
According to Eric Paris the clean/dirty state is only stored in memory. We could use the extended attribute interface as a way of not defining a new system call, or some other interface, but I'm not sure it's such a great match given that the extended attributes interface are designed for persistent data.
Quoted 5 lines
I agree that doesn't actually work very well for the tracker use case, where you the clean/dirty bit to be persistent (in case the tracker is disabled due to the fact you are running on battery, for example, and then you reboot).
Quoted 5 lines
but we need a "give me all dirty files" solution, not a "is this file dirty" solution.
I do not want a virus scanner to constantly have to poll the whole fs for dirty files ;-)
I'm not sure.
there are two situations (with the transition between them)
1. unscanned system, we want to do everything. (this happens immediatly after a new signature file is deployed)
here you do just want to filter out the files that have been scanned from the list of everything, and you probably want to check at the time of scanning the file in case it was opened (and scanned) in the meantime.
2. mostly scanned system, we only want to scan files that have been dirtied.
here you don't need to scan everything, you only need to scan in two cases
Quoted 7 lines
2a. the file was dirtied (you learn about it and add it to the queue of files to scan when you get around to it)
2b. an unscanned file is opened (the library detects that the file was not marked approved by all the current scanners, so it invokes the scanners on this file before completing the open, or copy for mmap, or whatever)
2b could also be used as a general lazy scanning mechanism, no? If there is a new signature or the file is dirty or unknown, scan it. Always.
So this should all just be one case, no?
Hannes
Quoted 58 lines · Johannes Weiner
Hi,
Quoted 46 lines · [email protected] writes:
Quoted 21 lines · Arjan van de Ven
Quoted 12 lines · On Thu, 14 Aug 2008 22:04:00 -0400 Theodore Tso <[email protected]> wrote:
Quoted 4 lines · On Thu, Aug 14, 2008 at 06:44:33PM -0700, [email protected] wrote:
could you do something like defining a namespace inside posix attributes and then setting up a mechanism in the kernel to alert if the attributes change (with the entire namespace getting cleared if the file gets dirtied)?
According to Eric Paris the clean/dirty state is only stored in memory. We could use the extended attribute interface as a way of not defining a new system call, or some other interface, but I'm not sure it's such a great match given that the extended attributes interface are designed for persistent data.
Quoted 5 lines
I agree that doesn't actually work very well for the tracker use case, where you the clean/dirty bit to be persistent (in case the tracker is disabled due to the fact you are running on battery, for example, and then you reboot).
Quoted 5 lines
but we need a "give me all dirty files" solution, not a "is this file dirty" solution.
I do not want a virus scanner to constantly have to poll the whole fs for dirty files ;-)
I'm not sure.
there are two situations (with the transition between them)
1. unscanned system, we want to do everything. (this happens immediatly after a new signature file is deployed)
here you do just want to filter out the files that have been scanned from the list of everything, and you probably want to check at the time of scanning the file in case it was opened (and scanned) in the meantime.
2. mostly scanned system, we only want to scan files that have been dirtied.
here you don't need to scan everything, you only need to scan in two cases
Quoted 7 lines
2a. the file was dirtied (you learn about it and add it to the queue of files to scan when you get around to it)
2b. an unscanned file is opened (the library detects that the file was not marked approved by all the current scanners, so it invokes the scanners on this file before completing the open, or copy for mmap, or whatever)
Quoted 5 lines
2b could also be used as a general lazy scanning mechanism, no? If there is a new signature or the file is dirty or unknown, scan it. Always.
So this should all just be one case, no?
this depends on the policy in userspace.
you could:
1. plan on doing all scanning as files are opened (this is going to be noticable to users)
2. run the background scanner continuously as a low priority (as soon as you finish one scan, do the next)
3. do one background scan when you get a new signature, and after you complete that only scan files that change.
and probably others
#3 has the advantage that the only time a user will hit a delay when opening a file is if the file was just recently modified and the scanner daemon(s) haven't had a chance to scan it yet.
David Lang
Quoted 54 lines · On Thu, 14 Aug 2008, [email protected] wrote:
Quoted 21 lines · Arjan van de Ven
Quoted 12 lines · On Thu, 14 Aug 2008 22:04:00 -0400 Theodore Tso <[email protected]> wrote:
Quoted 4 lines · On Thu, Aug 14, 2008 at 06:44:33PM -0700, [email protected] wrote:
could you do something like defining a namespace inside posix attributes and then setting up a mechanism in the kernel to alert if the attributes change (with the entire namespace getting cleared if the file gets dirtied)?
According to Eric Paris the clean/dirty state is only stored in memory. We could use the extended attribute interface as a way of not defining a new system call, or some other interface, but I'm not sure it's such a great match given that the extended attributes interface are designed for persistent data.
Quoted 5 lines
I agree that doesn't actually work very well for the tracker use case, where you the clean/dirty bit to be persistent (in case the tracker is disabled due to the fact you are running on battery, for example, and then you reboot).
Quoted 5 lines
but we need a "give me all dirty files" solution, not a "is this file dirty" solution.
I do not want a virus scanner to constantly have to poll the whole fs for dirty files ;-)
I'm not sure.
there are two situations (with the transition between them)
1. unscanned system, we want to do everything. (this happens immediatly after a new signature file is deployed)
here you do just want to filter out the files that have been scanned from the list of everything, and you probably want to check at the time of scanning the file in case it was opened (and scanned) in the meantime.
2. mostly scanned system, we only want to scan files that have been dirtied.
here you don't need to scan everything, you only need to scan in two cases
2a. the file was dirtied (you learn about it and add it to the queue of files to scan when you get around to it)
2b. an unscanned file is opened (the library detects that the file was not marked approved by all the current scanners, so it invokes the scanners on this file before completing the open, or copy for mmap, or whatever)
In the first case the attributes work "don't bother scanning me".
In the second case they also work (becouse you aren't trying to scan everything)
the only time there is a headache is in the transition between them when you have scanned a lot of the system, but not all of it, and the machine was rebooted so you lost track of what you had scanned.
Quoted 7 lines
it shouldn't be too hard to deal with this. even if you never resume the scan you are still safe (becouse of the scan-on-open), but it's also possible to either do a find f(or equivalent) or files without the attribute and store the results (similar to updatedb) and then updating the file to mark the files as being scanned (update in place, change the first character or something to be fairly crash safe). after the full list of files has been scanned shift to the second mode.
the sweep scan should be a background task, possibly disabled when on battery power.
why would this not satisfy the requirements?
David Lang
one way this approach would be a pain is that as signature files got updated the attributes would accumulate.
a couple ways of dealing with this.
1. scanners clean up after themselves (when they go to add a new one the remove the old one)
2. background sweep through the system removing all tags that aren't in the current 'blessed' set
David Lang
Quoted 12 lines · Theodore Tso
Quoted 4 lines · On Thu, Aug 14, 2008 at 06:44:33PM -0700, [email protected] wrote:
could you do something like defining a namespace inside posix attributes and then setting up a mechanism in the kernel to alert if the attributes change (with the entire namespace getting cleared if the file gets dirtied)?
According to Eric Paris the clean/dirty state is only stored in memory. We could use the extended attribute interface as a way of not defining a new system call, or some other interface, but I'm not sure it's such a great match given that the extended attributes interface are designed for persistent data.
Quoted 4 lines
I agree that doesn't actually work very well for the tracker use case, where you the clean/dirty bit to be persistent (in case the tracker is disabled due to the fact you are running on battery, for example, and then you reboot).
why would you not want the results of the viris scan to be persistant?
assuming that you are not dual-booting( or otherwise taking explicit action to go around the OS), the fact that you scanned the file with signature set 123 is just as valid now as it was yesterday.
If you are not trying to work around the actions of root, why are you trying to work around the actions of the person who has physical control of the system? (who could easily become root if they wanted to)
David Lang
Quoted 5 lines
According to Eric Paris the clean/dirty state is only stored in memory. We could use the extended attribute interface as a way of not defining a new system call, or some other interface, but I'm not sure it's such a great match given that the extended attributes interface are designed for persistent data.
This is another application layer matter. At the end of the day why does the kernel care where this data is kept. For all we know someone might want to centralise it or even distribute it between nodes on a clustered file system.
Alan
Quoted 10 lines · Theodore Tso
Quoted 8 lines · Eric Paris
There needs to be a way to say that an inode in cache needs to be rescanned. 3 states this flag can be. Clean, Dirty, Infected. The current talpa solution involves a global monotomically increasing counter every time you change virus defs or make some "interesting" change. If global == inode flag we are clean. If global == negative inode flag we are infected. if global > inode flag we are dirty and need a scan.
Quoted 3 lines
"Infected" just means to instantly return an error when the file is opened or if an already opened file descriptor is read or mmap'ed, right? If file is already
mmaped(), what's the plan? Send a kill -9
Me thinks about naming my machine host-inline-real-virus-or-at-least-identifiable-part-here.cz, then having fun with people not able to access their apache logs.
Returning an error when you think you see a virus is a great invitation to DoS attack, really.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 6 lines · Arjan van de Ven
On Wed, 13 Aug 2008 12:36:15 -0400
Eric Paris <[email protected]> wrote:[finally good description snipped]
Quoted 1 line
Quoted 8 lines
Quoted 1 line
1) Kernel or userspace.
this is kind of the last question to ask, not the first ;)
Quoted 2 lines
Not to mention there cannot be any checks for files opened by suid apps in userspace.
setuid apps do not use glibc? news to me.
I was thinking about LD_PRELOAD. you're right for most things it could be glibc. I still have knfsd on my side though.
Quoted 6 lines
Quoted 4 lines
And while no specific claims are
being made about intentionally malicious code, putting this in kernel makes programs which call syscalls directly much more difficult to be used to circumvent the scanning.[so lets ignore this one]
/me doesn't want to ignore things that help his argument, even if they are weak. :)
Quoted 9 lines
Quoted 2 lines
2) I think the "best" time to do scanning is at read and write. Any disagreements there?
I agree about the "read" case.
"write" is tricky... because there's many vectors to write, fromwrite()system call tommap()system call totruncate()system call. (yes you can write a stream of bytes that passes virus scan, and then truncate to be suddenly a live virus)
Quoted 7 lines
I would propose to use the word "dirty" instead, we all know what we mean by that (it's all of them) and doesn't tie our thinking to the
write()system call.I would like to introduce a concept in your discussion you did not mention yet in this email: the difference between synchronous scanning and asynchronous scanning.
I already have that concept! my open time scanning is sync and can return an EACCESS. my close time scanning is async and only updates the in inode clean/dirty mark. It makes no access decisions. That way on the next open we are good to go! And yes yes, mmap write after close invalides the async clean mark.
Quoted 5 lines
It's clear from the protection model that you described that on 'read' you want to wait until the scan is done before you give the data to the process asking for it... and that's totally reasonable: "Do not give out bad data" is a very clear line in terms of security.
Quoted 14 lines
for the "dirty" case it gets muddy. You clearly want to scan "some time" after the write, from the principle of getting rid of malware that's on the disk, but it's unclear if this HAS to be synchronous. (obviously, synchronous behavior hurts performance bigtime so lets do as little as we can of that without hurting the protection). One advantage of doing the dirty case async (and a little time delayed) is that repeated writes will get lumped up into one scan in practice, saving a ton of performance.
(scan-on-close is just another way of implementing "delay the dirty scan").
Based on Alans comments, to me this sounds like we should have an efficient mechanism to notify userspace of "dirty events"; this is not virus scan specific in any way or form. And this mechanism likely will need to allow multiple subscribers.
I'm certainly willing to go down the inotify'ish path for async notification of 'dirty' inodes instead of implement my own async mechanism if I can find a way to do it.
Quoted 4 lines
for the
open()case, I would argue that you don't need synchronous behavior as long as theread()case is synchronous. I can imagine thatopen()kicks off an async scan, and if it's done by the time the firstread()happens, no blocking at all happens.
An interesting addition. Trying to keep these queues of events gets much more complex, but if people really think the open to read race is that important I've always said it wasn't impossible to close.
Quoted 9 lines
For efficiency the kernel ought to keep track of which files have been declared clean, and it needs to track of a 'generation' of the scan with which it has been found clean (so that if you update your virus definitions, you can invalidate all previous scanning just by bumping the 'generation' number in whatever format we use).
Clearly the kernel needs to wipe this clean generation number on any modification to the file (inode) of any kind. And clearly this needs to be done inside the kernel because tracking dirty operations in any other way is just insane.
I agree! i_talpa_seq_no I added to struct inode is exactly that!
Quoted 8 lines
Now this to me we have a few basic building blocks:
1) We need an efficient mechanism to notify userspace of files that get dirtied. Virus scanners will subscribe to this for the async dirty scanning; indexing agents also will subscribe to this.
2) We very likely should have a mechanism for a userspace app to request a scan on a file, both sync or async (O_SYNCflag?). This is useful regardless because it allows the source of many things to do the right thing.
This means what exactly? Some new syscall? You want some program to say "notify userspace about this inode" and then every userspace thing that wants the notification gets it? Not sure how much brains I want to build into generic userspace apps. I certainly wouldn't want to make any userspace app easily be able to avoid AV scanning even if I make no promises about malicious apps. Just because I don't make gurantees doesn't mean I should make it easy. Just because I can't be 100% that a fence won't keep the kids from falling in the pool doesn't mean I shouldn't build it. Differentiating between don't AV scan and don't index is going to be hard in a single interface, but not unsolvable.
Quoted 3 lines
3) we need a mechanism in the kernel to track "scanned with generation X of signatures" that invalidates on any dirty operation. The syscall from 2) will use this as a cache to be quick.
i_talpa_seq_no
Quoted 7 lines
I think few people will disagree about this.
Open questions now are
4) do we have the kernel kick off an async scan inopen()or do we have glibc do this
5) do we have the kernel do the sync scan on read/mmap/.. or do we have glibc do this
scan on mmap read? How do I implement this? Do we update atime on mmap read? can I do it there like I do for writes with mtime?
Quoted 5 lines
I think this is where the whole debate is about now.
And a few hard ones
6) how do we deal with multiple scanning agents in parallel
My first suggestion is we don't, and worry about it as soon as we have the first parallel users.
Quoted 1 line
7) how do we prevent malware from pretending to be a virus scanner
We don't. Let the LSM do its job?
Quoted 8 lines · On Wed, 13 Aug 2008 14:57:44 -0400 Eric Paris <[email protected]> wrote:
Quoted 4 lines
for the
open()case, I would argue that you don't need synchronous behavior as long as theread()case is synchronous. I can imagine thatopen()kicks off an async scan, and if it's done by the time the firstread()happens, no blocking at all happens.An interesting addition. Trying to keep these queues of events gets much more complex, but if people really think the open to read race is that important I've always said it wasn't impossible to close.
it's not "just" about open-to-read race.
it's about open being non-blocking, and if read is not immediate, never hitting the latency at all.
The real point is that "read" is the actual point you want to intercept, not "open" (you even wrote that in your description).. so why not just do that ?
The open case then is just a performance optimization.
Quoted 7 lines
Quoted 5 lines
Open questions now are
4) do we have the kernel kick off an async scan inopen()or do we have glibc do this
5) do we have the kernel do the sync scan on read/mmap/.. or do we have glibc do thisscan on mmap read? How do I implement this?
on calling mmap(); not at fault time.
Signature · 3 lines
If you want to reach me at my work email, use [email protected] For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 12 lines · Arjan van de Ven
Quoted 8 lines · On Wed, 13 Aug 2008 14:57:44 -0400 Eric Paris <[email protected]> wrote:
Quoted 4 lines
for the
open()case, I would argue that you don't need synchronous behavior as long as theread()case is synchronous. I can imagine thatopen()kicks off an async scan, and if it's done by the time the firstread()happens, no blocking at all happens.An interesting addition. Trying to keep these queues of events gets much more complex, but if people really think the open to read race is that important I've always said it wasn't impossible to close.
Quoted 8 lines
it's not "just" about open-to-read race.
it's about open being non-blocking, and if read is not immediate, never hitting the latency at all.The real point is that "read" is the actual point you want to intercept, not "open" (you even wrote that in your description).. so why not just do that ?
The open case then is just a performance optimization.
I've been thinking about this more over night. I really like the idea for perf reasons but I'm scared of programs not expecting and thus poorly handling -EACCESS from read. Every program ever is going to expect that back from open, but once you have the fd open its not common.
The idea of multiple concurrent outstanding async notifications is going to be much more difficult to code, but hey, who am I to complain. We could have outstanding async scanning requests for any number of writes, any number of closes, and any number of opens all at the same time. The current interface believes that every request out requires a response but this type of interface basically requires some sort of coalescing.
Would you be opposed to a first patch round that did blocking enforcement on open like we have today and do sync scanning (blocking or return EWOULDBLOCK) on read if needed due to concurrent writes?
Quoted 24 lines · On Thu, 14 Aug 2008 10:12:13 -0400 Eric Paris <[email protected]> wrote:
Quoted 14 lines · Arjan van de Ven
Quoted 10 lines · On Wed, 13 Aug 2008 14:57:44 -0400 Eric Paris <[email protected]> wrote:
Quoted 5 lines
for the
open()case, I would argue that you don't need synchronous behavior as long as theread()case is synchronous. I can imagine thatopen()kicks off an async scan, and if it's done by the time the firstread()happens, no blocking at all happens.An interesting addition. Trying to keep these queues of events gets much more complex, but if people really think the open to read race is that important I've always said it wasn't impossible to close.
Quoted 8 lines
it's not "just" about open-to-read race.
it's about open being non-blocking, and if read is not immediate, never hitting the latency at all.The real point is that "read" is the actual point you want to intercept, not "open" (you even wrote that in your description).. so why not just do that ?
The open case then is just a performance optimization.
Quoted 5 lines
I've been thinking about this more over night. I really like the idea for perf reasons but I'm scared of programs not expecting and thus poorly handling
-EACCESSfrom read. Every program ever is going to expect that back from open, but once you have the fd open its not common.
you could stretch things and report -EIO.
Quoted 3 lines
The idea of multiple concurrent outstanding async notifications is going to be much more difficult to code, but hey, who am I to
well... do you really need a response?
you could just write it to a netlink socket...
Quoted 5 lines
complain. We could have outstanding async scanning requests for any number of writes, any number of closes, and any number of opens all at the same time. The current interface believes that every request out requires a response but this type of interface basically requires some sort of coalescing.
or fire-and-forget.
Quoted 4 lines
Would you be opposed to a first patch round that did blocking enforcement on open like we have today and do sync scanning (blocking or return EWOULDBLOCK) on read if needed due to concurrent writes?
I think we need to sort the interface issue on this for sure, and probably from the start...
Quoted 1 line
Signature · 3 lines
If you want to reach me at my work email, use [email protected] For development, discussion and tips for power savings, visit http://www.lesswatts.org
It seems to me that this "scan on file open" business is the wrong way to do things - because it reduces performance.
If you scan on file open, then your security sw is too late and getting in the way.
It is better to scan in advance. Most machines has lots of idle time. Use that time to scan in advance, and mark the files as "clean".
A "clean" file can be opened without further checking anytime - giving normal high performance. A file that gets written to becomes "dirty" until checked again. Some mechanism for making a clean copy of a clean file might help avoid excessive "dirtying".
"Scan on open" might still be useful for cases when the system haven't kept up with writing, but please don't aim to have this be the _primary_ mode of scanning. A file server
where most of the stuff is pre-scanned will likely perform much better than one scanning everything on open.
Helge Hafting
Quoted 14 lines · Helge Hafting
It seems to me that this "scan on file open" business is the wrong way to do things - because it reduces performance.
If you scan on file open, then your security sw is too late and getting in the way.
It is better to scan in advance. Most machines has lots of idle time. Use that time to scan in advance, and mark the files as "clean".
A "clean" file can be opened without further checking anytime - giving normal high performance. A file that gets written to becomes "dirty" until checked again. Some mechanism for making a clean copy of a clean file might help avoid excessive "dirtying".
Quoted 5 lines
"Scan on open" might still be useful for cases when the system haven't kept up with writing, but please don't aim to have this be the _primary_ mode of scanning. A file server
where most of the stuff is pre-scanned will likely perform much better than one scanning everything on open.
This is the source of all that scan on write/close rambling I've seen go past the last couple of days..
So I'd not worry about this particular aspect
Quoted 6 lines
-----Original Message-----
From: [email protected] [mailto:malware-list- [email protected]] On Behalf Of Peter Zijlstra
Sent: Friday, August 15, 2008 6:37 AM
To: Helge Hafting
Cc: [email protected]; [email protected];
Quoted 1 line
[email protected]; Arjan van
Quoted 9 lines
de Ven
Subject: Re: [malware-list] TALPA - a threat model? well sorta.Quoted 5 lines · Helge Hafting
It seems to me that this "scan on file open" business is the wrong way to do things - because it reduces performance.
If you scan on file open, then your security sw is too late and getting in the way.
The problem is that you have to account for the cases where the malware made it onto the system even if you were trying to catch it ahead of time. For example:
- Administrator turns off or reduces AV protection for some reason for some period of time. It happens all the time.
- New infection makes it onto the machine before the signatures have caught up with it. This also happens. There is an ongoing PR race among AV vendors about who was faster on the draw to get out signatures to detect some new malware. The fact that this race exists reflects that reality that there is some window during which new malware will make it onto some number of machines before the scanners catch up.
Quoted 1 line
Quoted 1 line
It is better to scan in advance. Most machines has lots of idle
time.
Quoted 3 lines
Quoted 3 lines
Use that time to scan in advance, and mark the files as "clean".
A "clean" file can be opened without further checking anytime -
giving
Quoted 1 line
Quoted 1 line
normal high performance. A file that gets written to becomes
"dirty"
Quoted 7 lines
Quoted 7 lines
until checked again. Some mechanism for making a clean copy of a clean file might help avoid excessive "dirtying".
"Scan on open" might still be useful for cases when the system haven't kept up with writing, but please don't aim to have this be the _primary_ mode of scanning. A file server where most of the stuff is pre-scanned will likely perform much
better
Quoted 1 line
Quoted 1 line
than one scanning everything on open.
It isn't the primary mode. It's the mode that catches things as they arrive, and as they are about to be used. Most sites will also employ the applications' features for running regular scans of the whole system on a scheduled basis, to catch anything that may have slipped through.
Jon Press
Jon Press wrote on 2008-08-15 14:10:21:
Quoted 26 lines
Quoted 6 lines
-----Original Message-----
From: [email protected] [mailto:malware-list- [email protected]] On Behalf Of Peter Zijlstra
Sent: Friday, August 15, 2008 6:37 AM
To: Helge Hafting
Cc: [email protected]; [email protected];Quoted 1 line
[email protected]; Arjan van
Quoted 9 lines
de Ven
Subject: Re: [malware-list] TALPA - a threat model? well sorta.Quoted 5 lines · Helge Hafting
It seems to me that this "scan on file open" business is the wrong way to do things - because it reduces performance.
If you scan on file open, then your security sw is too late and getting in the way.
The problem is that you have to account for the cases where the malware made it onto the system even if you were trying to catch it ahead of time. For example:
- Administrator turns off or reduces AV protection for some reason for some period of time. It happens all the time.
Quoted 7 lines
- New infection makes it onto the machine before the signatures have caught up with it. This also happens. There is an ongoing PR race among AV vendors about who was faster on the draw to get out signatures to detect some new malware. The fact that this race exists reflects that reality that there is some window during which new malware will make it onto some number of machines before the scanners catch up.
Not to mention removable media - it might be old hat, but infected/malware files can come in on floppies, CDs or USB flash discs careless left on the pavement outside an office.
Signature · 6 lines
Douglas Leeder Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom. Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Quoted 6 lines · On Fri, Aug 15, 2008 at 02:18:12PM +0100, [email protected] wrote:
Quoted 6 lines
- New infection makes it onto the machine before the signatures have caught up with it. This also happens. There is an ongoing PR race among AV vendors about who was faster on the draw to get out signatures to detect some new malware. The fact that this race exists reflects that reality that there is some window during which new malware will make it onto some number of machines before the scanners catch up.
Let's go back to the threat model. The Threat Model which Eric Paris has suggested is that we are only trying to solve the Scanning Problem. Just Scanning.
That implies if the malware has been written to the disk, we will catch it once AV catching is turned on and the user attempts to run or otherwise access the file with the bad content. However, if the malware starts running, then regardless of whether the malware is running with user privileges, or manages to get root privileges via some buffer overflow that wasn't caught via LSM/SELinux/AppAmor/whatever, this is out of scope of Eric's proposal.
Are we agreed on that? There may be other components of the solution such as LSM, SELinux, etc., that will very likely be useful in protecting the system once the malware starts running. But I thought Eric's proposal proposed excluding that from the Threat Model for the purposes of the interface we are trying to solve. If that's not true, let's deal with it now.
Quoted 3 lines
Not to mention removable media - it might be old hat, but infected/malware files can come in on floppies, CDs or USB flash discs careless left on the pavement outside an office.
That's not a problem given the scanning model proposed by Eric; when you insert removable media, it will get scanned when it is first accessed.
- Ted
Quoted 5 lines
-----Original Message-----
From: Theodore Tso [mailto:[email protected]]
Sent: Friday, August 15, 2008 1:05 PM
To: [email protected]
Cc: Press, Jonathan; [email protected]; [email protected];
Arjan van de
Quoted 1 line
Ven; [email protected]; Helge Hafting; [email protected];
malware-
Quoted 4 lines
[email protected]; Peter Zijlstra; [email protected]
Subject: Re: [malware-list] TALPA - a threat model? well sorta.Quoted 1 line
Not to mention removable media - it might be old hat, but
infected/malware
Quoted 1 line
Quoted 1 line
files can come in on floppies, CDs or USB flash discs careless left
on the
Quoted 5 lines
Quoted 1 line
pavement outside an office.
That's not a problem given the scanning model proposed by Eric; when you insert removable media, it will get scanned when it is first accessed.
That is exactly the idea. However, the context of this particular thread was the following statement by Helge Hafting:
It seems to me that this "scan on file open" business is the
wrong
way to do things - because it reduces performance.
If you scan on file open, then your security sw is too late and getting in the way.
We were just pointing out that this is not a good argument in practical terms AGAINST scanning on open. In fact, your reply completely reinforces that point.
Jon Press
Quoted 38 lines · Jonathan
Quoted 6 lines
-----Original Message-----
From: Theodore Tso [mailto:[email protected]]
Sent: Friday, August 15, 2008 1:05 PM
To: [email protected]
Cc: Press, Jonathan; [email protected]; [email protected];Arjan van de
Quoted 2 lines
Ven; [email protected]; Helge Hafting; [email protected];
malware-
Quoted 6 lines
[email protected]; Peter Zijlstra; [email protected]
Subject: Re: [malware-list] TALPA - a threat model? well sorta.Quoted 2 lines
Not to mention removable media - it might be old hat, but
infected/malware
Quoted 2 lines
Quoted 2 lines
files can come in on floppies, CDs or USB flash discs careless left
on the
Quoted 6 lines
Quoted 2 lines
pavement outside an office.
That's not a problem given the scanning model proposed by Eric; when you insert removable media, it will get scanned when it is first accessed.
That is exactly the idea. However, the context of this particular thread was the following statement by Helge Hafting:
It seems to me that this "scan on file open" business is the
wrong
way to do things - because it reduces performance.
Quoted 8 lines
If you scan on file open, then your security sw is too late and getting in the way.
We were just pointing out that this is not a good argument in practical terms AGAINST scanning on open. In fact, your reply completely reinforces that point.
Scanning on open should be a last resort. Scan in advance when you can. Of course, removable media cannot be scanned until it is inserted and mounted, that is obvious. The scanning can start as soon as the filesystem is mounted though, there is no reason to wait until users try to access something.
A CD inserted into a CD-server may not necessarily be needed immediately, so scanning in advance will help here too. The user inserting a CD in a home computer may start to use stuff right away, or perhaps he spends
some time reading the docs before a complicated install. Sill room for some scanning in advance, which also may end up with the nice effect of caching the CD.
Helge Hafting
Quoted 8 lines · Helge Hafting
Scanning on open should be a last resort. Scan in advance when you can. Of course, removable media cannot be scanned until it is inserted and mounted,
that is obvious. The scanning can start as soon as the filesystem is mounted though,
there is no reason to wait until users try to access something.
Quoted 6 lines
A CD inserted into a CD-server may not necessarily be needed immediately, so scanning in advance will help here too. The user inserting a CD in a home computer may start to use stuff right away, or perhaps he spends some time reading the docs before a complicated install. Sill room for some scanning in advance, which also may end up with the nice effect of caching the CD.
Hmm, then there is the issue that an active scanner will avoid me from removing the media again. Most annoysome when the machine won't promptly give back the disk.
Imagine me browsing through a stack of unmarked dvd-rw media trying to find the one with the right file on it, but having to wait every time for the whole media to get scanned..
Not a nice picture.
Peter Zijlstra wrote on 18/08/2008 11:14:00:
Quoted 3 lines · Helge Hafting
Quoted 2 lines
Scanning on open should be a last resort. Scan in advance when you
can.
Quoted 7 lines
Quoted 7 lines
Of course, removable media cannot be scanned until it is inserted and mounted,
that is obvious. The scanning can start as soon as the filesystem is mounted though,
there is no reason to wait until users try to access something.A CD inserted into a CD-server may not necessarily be needed
immediately, so
Quoted 1 line
Quoted 1 line
scanning in advance will help here too. The user inserting a CD in a
home
Quoted 2 lines
Quoted 2 lines
computer may start to use stuff right away, or perhaps he spends some time reading the docs before a complicated install. Sill room for
some
Quoted 7 lines
Quoted 2 lines
scanning in advance, which also may end up with the nice effect of caching the CD.
Hmm, then there is the issue that an active scanner will avoid me from removing the media again. Most annoysome when the machine won't promptly give back the disk.
Quoted 5 lines
Imagine me browsing through a stack of unmarked dvd-rw media trying to find the one with the right file on it, but having to wait every time for the whole media to get scanned..
Not a nice picture.
I agree. And not only that but also the fact that most of the time I would not want mount to trigger a background scan of everything because not all might be accessed later. Therefore it would be just wasting resources and ruining user experience. This argument is flawed in a way that is unsolvable in the same way normal preload to page cache is unsolvable because it is impossible to predict the usage pattern.
--
Tvrtko A. Ursulin
Senior Software Engineer, Sophos
"Views and opinions expressed in this email are strictly those of the author.
The contents has not been reviewed or approved by Sophos."
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Helge Hafting <[email protected]> wrote on 2008-08-18 11:09:25:
Quoted 8 lines
Scanning on open should be a last resort. Scan in advance when you can. Of course, removable media cannot be scanned until it is inserted and mounted,
that is obvious. The scanning can start as soon as the filesystem is mounted though,
there is no reason to wait until users try to access something.A CD inserted into a CD-server may not necessarily be needed
immediately, so
Quoted 1 line
scanning in advance will help here too. The user inserting a CD in a
home
Quoted 2 lines
computer may start to use stuff right away, or perhaps he spends some time reading the docs before a complicated install. Sill room for
some
Quoted 2 lines
scanning in advance, which also may end up with the nice effect of caching the CD.
It might be useful to scan in advance, on mount, or scanner start. But that
can be simulated using a user-space program, and kernel-based on-open blocking scans:
Just have a background program open the files, and they will be scanned and
cached.
The background program could also do clever things like check if running on battery, check
load level, keep hot-lists of files to scan/scan first.
All of which can be implemented on top of the Talpa interface already specified.
In practice scan on-open with blocking is fast enough for general usage, even using our existing
hacky kernel module approach.
Signature · 6 lines
Douglas Leeder Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom. Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Quoted 19 lines · Jonathan
Quoted 6 lines
-----Original Message-----
From: [email protected] [mailto:malware-list- [email protected]] On Behalf Of Peter Zijlstra
Sent: Friday, August 15, 2008 6:37 AM
To: Helge Hafting
Cc: [email protected]; [email protected];Quoted 1 line
[email protected]; Arjan van
Quoted 9 lines
de Ven
Subject: Re: [malware-list] TALPA - a threat model? well sorta.Quoted 5 lines · Helge Hafting
It seems to me that this "scan on file open" business is the wrong way to do things - because it reduces performance.
If you scan on file open, then your security sw is too late and getting in the way.
Quoted 6 lines
The problem is that you have to account for the cases where the malware made it onto the system even if you were trying to catch it ahead of time. For example:
- Administrator turns off or reduces AV protection for some reason for some period of time. It happens all the time.
according to the threat model actions of the administrator do not matter.
David Lang
Quoted 5 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]
Sent: Friday, August 15, 2008 12:25 PM
To: Press, Jonathan
Cc: Peter Zijlstra; Helge Hafting; [email protected];
malware-
Quoted 6 lines
[email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Arjan van de Ven
Subject: RE: [malware-list] TALPA - a threat model? well sorta.Quoted 1 line
The problem is that you have to account for the cases where the
malware
Quoted 4 lines
Quoted 4 lines
made it onto the system even if you were trying to catch it ahead of time. For example:
- Administrator turns off or reduces AV protection for some reason
for
Quoted 3 lines
Quoted 1 line
some period of time. It happens all the time.
according to the threat model actions of the administrator do not
matter.
Sorry, I don't know what you mean.
Jon
Quoted 7 lines · Jonathan
Quoted 3 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]Quoted 1 line
The problem is that you have to account for the cases where the
malware
Quoted 3 lines
Quoted 3 lines
made it onto the system even if you were trying to catch it ahead of time. For example:
Quoted 8 lines
Quoted 1 line
Quoted 1 line
- Administrator turns off or reduces AV protection for some reason
for
Quoted 3 lines
Quoted 1 line
some period of time. It happens all the time.
according to the threat model actions of the administrator do not
matter.
Sorry, I don't know what you mean.
the threat model that was posted two days ago in the initial message of this thread specificly stated that actions of root are not something that this is trying to defend against.
David Lang
Quoted 5 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]
Sent: Friday, August 15, 2008 1:33 PM
To: Press, Jonathan
Cc: Peter Zijlstra; Helge Hafting; [email protected];
malware-
Quoted 9 lines
[email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Arjan van de Ven
Subject: RE: [malware-list] TALPA - a threat model? well sorta.Quoted 3 lines · Jonathan
Quoted 3 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]Quoted 1 line
The problem is that you have to account for the cases where the
malware
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
made it onto the system even if you were trying to catch it ahead
of
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
time. For example:
- Administrator turns off or reduces AV protection for some reason
for
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
some period of time. It happens all the time.
according to the threat model actions of the administrator do not
matter.
Quoted 4 lines
Quoted 2 lines
Sorry, I don't know what you mean.
the threat model that was posted two days ago in the initial message
of
Quoted 1 line
this thread specificly stated that actions of root are not something
that
Quoted 1 line
this is trying to defend against.
I think you may have missed the point of any such statement.
Just to clarify...
The model does not exclude root-owned processes from the notification and scanning sequence. If root attempts to execute a file, that file would be scanned before the execution is allowed. If a root-owned process attempts to open a file, that access would be blocked until the file is scanned. If a root-owned process closes a file that has been written to, that file would be scanned.
In addition, to generalize from the incorrect idea that the actions of root are not being defended against to the idea that the possible impacts of an administrator's actions in configuring an application should not be accounted for at all in our thinking doesn't make sense to me anyway.
Jon Press
Quoted 30 lines · Jonathan
Quoted 8 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]Quoted 3 lines · Jonathan
Quoted 3 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]Quoted 1 line
The problem is that you have to account for the cases where the
malware
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
made it onto the system even if you were trying to catch it ahead
of
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
time. For example:
- Administrator turns off or reduces AV protection for some reason
for
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
some period of time. It happens all the time.
according to the threat model actions of the administrator do not
matter.
Quoted 4 lines
Quoted 2 lines
Sorry, I don't know what you mean.
the threat model that was posted two days ago in the initial message
of
Quoted 1 line
this thread specificly stated that actions of root are not something
that
Quoted 1 line
this is trying to defend against.
I think you may have missed the point of any such statement.
Quoted 8 lines
Just to clarify...
The model does not exclude root-owned processes from the notification and scanning sequence. If root attempts to execute a file, that file would be scanned before the execution is allowed. If a root-owned process attempts to open a file, that access would be blocked until the file is scanned. If a root-owned process closes a file that has been written to, that file would be scanned.
correct so far.
Quoted 5 lines
In addition, to generalize from the incorrect idea that the actions of root are not being defended against to the idea that the possible impacts of an administrator's actions in configuring an application should not be accounted for at all in our thinking doesn't make sense to me anyway.
questions had been raised about how this model could defend against all the tricky things that root can do, the answer was that they are not trying to defend against root doing tricky things.
turning off the scanner, letting things get infected, and turning it back on would fall in the same catagory as marking a file that the scanner marked as bad as sucessfully scanned.
in any case the vunerability is limited as the next time the signatures are updated the files would get scanned again, so I don't think it's a big problem in practice.
David Lang
Quoted 3 lines · On Fri, 15 Aug 2008 10:47:15 PDT, [email protected] said:
in any case the vunerability is limited as the next time the signatures are updated the files would get scanned again, so I don't think it's a big problem in practice.
This problem is actually identical to "new file scanned, but you don't have the signature available yet so malware isn't detected".
Those of us who have seen large mail servers pile up queues in the 10s of millions in the 45 minutes between when the worm went critical-mass and when we got a signature might disagree on it not being a big problem in practice.
Of course, if that's considered "outside" the threat model, somebody better start writing down exactly what small corner of threat model this is actually helping against...
Quoted 9 lines · On Fri, 15 Aug 2008, [email protected] wrote:
Quoted 3 lines · On Fri, 15 Aug 2008 10:47:15 PDT, [email protected] said:
in any case the vunerability is limited as the next time the signatures are updated the files would get scanned again, so I don't think it's a big problem in practice.
This problem is actually identical to "new file scanned, but you don't have the signature available yet so malware isn't detected".
Quoted 7 lines
Those of us who have seen large mail servers pile up queues in the 10s of millions in the 45 minutes between when the worm went critical-mass and when we got a signature might disagree on it not being a big problem in practice.
Of course, if that's considered "outside" the threat model, somebody better start writing down exactly what small corner of threat model this is actually helping against...
go back to the beginning of this thread. that is defining the (very limited) thread model that they are trying to defend against.
the rest of us are not trying to defend against this threat model. we are trying to identify the appropriate infrastructure that could be used by the TALPA folks for their work, that can also be reasonable for inclusion in the kernel (which includes being suitable for other similar purposes, such as filesystem indexing)
we agree that the threat they are trying to defend against is a very small portion of the overall threat, but there are other components (including SELinux) that are available to deal with other portions of the overall threat.
if you want to say that the solution is too limited to be worth while, then you need to write a new threat model that you think is what should be defended against and then we can start discussing how to defend against it.
David Lang
Quoted 6 lines · On Fri, Aug 15, 2008 at 02:06:47PM -0400, [email protected] wrote:
This problem is actually identical to "new file scanned, but you don't have the signature available yet so malware isn't detected".
Those of us who have seen large mail servers pile up queues in the 10s of millions in the 45 minutes between when the worm went critical-mass and when we got a signature might disagree on it not being a big problem in practice.
For a mail server, I really think something specialized like ClamAV is a much better solution than something in userspace, which will probably decide it has to rescan every single file that gets written,
including your mail server logs. :-)
A specialized solution for a mail server is *always* going to be able to a more efficient, more practical, and be able to do application-specialized things (such as refusing the e-mail while the connection is still open, so you don't have to worry about being RFC compliant about sending bounce mails when the SMTP return-path is most likely bogus).
- Ted
Quoted 5 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]
Sent: Friday, August 15, 2008 1:47 PM
To: Press, Jonathan
Cc: Peter Zijlstra; Helge Hafting; [email protected];
malware-
Quoted 6 lines
[email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Arjan van de Ven
Subject: RE: [malware-list] TALPA - a threat model? well sorta.Quoted 1 line · Jonathan
In addition, to generalize from the incorrect idea that the actions
of
Quoted 3 lines
Quoted 3 lines
root are not being defended against to the idea that the possible impacts of an administrator's actions in configuring an application should not be accounted for at all in our thinking doesn't make
sense to
Quoted 3 lines
Quoted 1 line
me anyway.
questions had been raised about how this model could defend against
all
Quoted 4 lines
the tricky things that root can do, the answer was that they are not trying to defend against root doing tricky things.
turning off the scanner, letting things get infected, and turning it
back
Quoted 2 lines
on would fall in the same catagory as marking a file that the scanner marked as bad as sucessfully scanned.
Well, I agree that there are things you can't prevent, that's for sure. But the point is to build the "threat model" and application functionality around the idea that IF they happen, you want to be able to plug the resulting holes as well as you can. You can't simply close your eyes to the possibility.
Jon Press
Quoted 26 lines · Jonathan
Quoted 5 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]
Sent: Friday, August 15, 2008 1:47 PM
To: Press, Jonathan
Cc: Peter Zijlstra; Helge Hafting; [email protected];malware-
Quoted 6 lines
[email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Arjan van de Ven
Subject: RE: [malware-list] TALPA - a threat model? well sorta.Quoted 1 line · Jonathan
In addition, to generalize from the incorrect idea that the actions
of
Quoted 3 lines
Quoted 3 lines
root are not being defended against to the idea that the possible impacts of an administrator's actions in configuring an application should not be accounted for at all in our thinking doesn't make
sense to
Quoted 3 lines
Quoted 1 line
me anyway.
questions had been raised about how this model could defend against
all
Quoted 4 lines
the tricky things that root can do, the answer was that they are not trying to defend against root doing tricky things.
turning off the scanner, letting things get infected, and turning it
back
Quoted 8 lines
Quoted 2 lines
on would fall in the same catagory as marking a file that the scanner marked as bad as sucessfully scanned.
Well, I agree that there are things you can't prevent, that's for sure. But the point is to build the "threat model" and application functionality around the idea that IF they happen, you want to be able to plug the resulting holes as well as you can. You can't simply close your eyes to the possibility.
correct, and the threat model that was documented as what we are trying to defend against explicity says that actions by root (or other running programs, including running malware) are not part of the threat model they are trying to address (it doesn't mean that they aren't important, just that they are not trying to deal with them). All they are trying to do is to have a way to scan files. Since there is use in having support for this (for security theater, a little real security, and for other purposes like indexing) we are figuing out how to provide tools that would allow this.
David Lang
Quoted 25 lines · Jonathan
Quoted 7 lines
-----Original Message-----
From: [email protected] [mailto:malware-list- [email protected]] On Behalf Of Peter Zijlstra
Sent: Friday, August 15, 2008 6:37 AM
To: Helge Hafting
Cc: [email protected]; [email protected];Quoted 2 lines
[email protected]; Arjan van
Quoted 11 lines
de Ven
Subject: Re: [malware-list] TALPA - a threat model? well sorta.Quoted 6 lines · Helge Hafting
It seems to me that this "scan on file open" business is the wrong way to do things - because it reduces performance.
If you scan on file open, then your security sw is too late and getting in the way.
Quoted 7 lines
The problem is that you have to account for the cases where the malware made it onto the system even if you were trying to catch it ahead of time. For example:
- Administrator turns off or reduces AV protection for some reason for some period of time. It happens all the time.
Everything is "dirty" again when AV sw restarts.
Quoted 7 lines
- New infection makes it onto the machine before the signatures have caught up with it. This also happens. There is an ongoing PR race among AV vendors about who was faster on the draw to get out signatures to detect some new malware. The fact that this race exists reflects that reality that there is some window during which new malware will make it onto some number of machines before the scanners catch up.
And when new signatures arrive, everything is dirty again. Well, you can optimize and say that everything only need to be checked with the new signatures, since they passed the old ones already. Maybe that helps performance a little.
The "clean" concept works fine, just make sure dirtying happens in all these extra cases.
Helge Hafting
Helge Hafting wrote on 15/08/2008 11:07:48:
Quoted 2 lines
It seems to me that this "scan on file open" business is the wrong way to do things - because it reduces performance.
It can never be free - what ever you do it has to happen some time and that can and will clash with something else. So your reason why it is wrong is a bit to simplistic.
Quoted 11 lines
If you scan on file open, then your security sw is too late and getting in the way.
It is better to scan in advance. Most machines has lots of idle time. Use that time to scan in advance, and mark the files as "clean".
A "clean" file can be opened without further checking anytime - giving normal high performance. A file that gets written to becomes "dirty" until checked again. Some mechanism for making a clean copy of a clean file might help avoid excessive "dirtying".
Quoted 5 lines
"Scan on open" might still be useful for cases when the system haven't kept up with writing, but please don't aim to have this be the _primary_ mode of scanning. A file server
where most of the stuff is pre-scanned will likely perform much better than one scanning everything on open.
The first thing you have to make clear is whether in your vision inode clean-dirty-unknown status is persistent or not? But in any case I think you are making a problem where there isn't one, think about it a bit.
But the idea about the ability to make a clean copy is interesting. So we would need a copy done without userspace intervention and preserving the inode cache status alongside it. Maybe splice or tee could easily do it? Although I am not sure how often copying would happen and whether this would be such a gain.
--
Tvrtko A. Ursulin
Senior Software Engineer, Sophos
"Views and opinions expressed in this email are strictly those of the author.
The contents has not been reviewed or approved by Sophos."
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Eric Paris wrote on 13/08/2008 19:57:44:
Quoted 2 lines
Quoted 2 lines
It's clear from the protection model that you described that on 'read' you want to wait until the scan is done before you give the data to
the
Quoted 9 lines
Quoted 9 lines
process asking for it... and that's totally reasonable: "Do not give out bad data" is a very clear line in terms of security.
for the "dirty" case it gets muddy. You clearly want to scan "some time" after the write, from the principle of getting rid of malware that's on the disk, but it's unclear if this HAS to be synchronous. (obviously, synchronous behavior hurts performance bigtime so lets do as little as we can of that without hurting the protection). One advantage of doing the dirty case async (and a little time
delayed)
Quoted 9 lines
Quoted 8 lines
is that repeated writes will get lumped up into one scan in practice, saving a ton of performance.
(scan-on-close is just another way of implementing "delay the dirty scan").
Based on Alans comments, to me this sounds like we should have an efficient mechanism to notify userspace of "dirty events"; this is not virus scan specific in any way or form. And this mechanism likely will need to allow multiple subscribers.
Quoted 3 lines
I'm certainly willing to go down the inotify'ish path for async notification of 'dirty' inodes instead of implement my own async mechanism if I can find a way to do it.
Do I understand correctly that everyone agrees scanning whenever an inode gets dirty would be a terrible thing for performance?
Another thing we have here is that malware could not be neccessariliy identified until the very last write (one example where it will always be the case are PDF files (I think)).
So the whole question is at which point should be performing an async scan. Close seems like a natural point which should be ideal for majority of applications, I don't see how any time-based lumping/delaying scheme can be better than close?
Quoted 8 lines
Quoted 4 lines
for the
open()case, I would argue that you don't need synchronous behavior as long as theread()case is synchronous. I can imagine thatopen()kicks off an async scan, and if it's done by the time the firstread()happens, no blocking at all happens.An interesting addition. Trying to keep these queues of events gets much more complex, but if people really think the open to read race is that important I've always said it wasn't impossible to close.
This really sounds pretty interesting. Not necessariliy so much as a performance optimisation, because I am not sure there are so many programs where first read comes long after the first open, but as closing the open-read race.
Could the implementation be not so complicated after all? If we generated the same (roughly) event on reads and pass it for scanning if cache has been invalidated in the mean time? The only thing is this could be a big performance hit so some benchmarking might be in order depending on which the read hook could be made run-time optional.
--
Tvrtko A. Ursulin
Senior Software Engineer, Sophos
"Views and opinions expressed in this email are strictly those of the author.
The contents has not been reviewed or approved by Sophos."
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Quoted 4 lines · On Thu, 14 Aug 2008 10:46:55 +0100 [email protected] wrote:
So the whole question is at which point should be performing an async scan. Close seems like a natural point which should be ideal for majority of applications, I don't see how any time-based lumping/delaying scheme can be better than close?
close isn't the answer just because you can write content to the file after that (and that's not just theoretical, glibc stdio supports mmap etc); "dirty" *has* to be the event anyway. It's not impossible to solve; even say a 1 second rearming delay would avoid 99.9% of the useless rescans while still making sure everything gets scanned at some point. Anyway this kind of policy can be done in userspace (and you can get really fancy there and offer the admin various policies)
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
for the
open()case, I would argue that you don't need synchronous behavior as long as theread()case is synchronous. I can imagine thatopen()kicks off an async scan, and if it's done by the time the firstread()happens, no blocking at all happens.
Quoted 8 lines
Quoted 3 lines
An interesting addition. Trying to keep these queues of events gets much more complex, but if people really think the open to read race is that important I've always said it wasn't impossible to close.
This really sounds pretty interesting. Not necessariliy so much as a performance optimisation, because I am not sure there are so many programs where first read comes long after the first open, but as closing the open-read race.
there's quite a few programs that open() but never read. open+fstat is not uncommon as programming pattern for example; with async-in-open (and then wait or sync in read) we wouldn't have the big hit caused by the latency for the sync scan.
(I realize this pattern is much more likely to happen on posixy systems than it is on windows.. one of those differences ;-)
Signature · 3 lines
If you want to reach me at my work email, use [email protected] For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 31 lines · On Thu, 14 Aug 2008, [email protected] wrote:
Eric Paris wrote on 13/08/2008 19:57:44:
Quoted 2 lines
Quoted 2 lines
It's clear from the protection model that you described that on 'read' you want to wait until the scan is done before you give the data to
the
Quoted 9 lines
Quoted 9 lines
process asking for it... and that's totally reasonable: "Do not give out bad data" is a very clear line in terms of security.
for the "dirty" case it gets muddy. You clearly want to scan "some time" after the write, from the principle of getting rid of malware that's on the disk, but it's unclear if this HAS to be synchronous. (obviously, synchronous behavior hurts performance bigtime so lets do as little as we can of that without hurting the protection). One advantage of doing the dirty case async (and a little time
delayed)
Quoted 9 lines
Quoted 8 lines
is that repeated writes will get lumped up into one scan in practice, saving a ton of performance.
(scan-on-close is just another way of implementing "delay the dirty scan").
Based on Alans comments, to me this sounds like we should have an efficient mechanism to notify userspace of "dirty events"; this is not virus scan specific in any way or form. And this mechanism likely will need to allow multiple subscribers.Quoted 3 lines
I'm certainly willing to go down the inotify'ish path for async notification of 'dirty' inodes instead of implement my own async mechanism if I can find a way to do it.
Do I understand correctly that everyone agrees scanning whenever an inode gets dirty would be a terrible thing for performance?
Quoted 8 lines
Another thing we have here is that malware could not be neccessariliy identified until the very last write (one example where it will always be the case are PDF files (I think)).
So the whole question is at which point should be performing an async scan. Close seems like a natural point which should be ideal for majority of applications, I don't see how any time-based lumping/delaying scheme can be better than close?
all you need is the ability to mark a file as 'dirty', and some way for programs that are interested in dirty files learning about it later and decideing to do a scan. if the file gets dirtied again after they do they scan they will need to do another one (this is a classic trade-off between the 'security' of looking for things quickly or 'efficiancy' of only looking when you don't think they will change again. in other words, policy -> userspace)
David Lang
Quoted 8 lines · Arjan van de Ven
2) We very likely should have a mechanism for a userspace app to request a scan on a file, both sync or async (
O_SYNCflag?). This is useful regardless because it allows the source of many things to do the right thing.
3) we need a mechanism in the kernel to track "scanned with generation X of signatures" that invalidates on any dirty operation. The syscall from 2) will use this as a cache to be quick.
Quoted 5 lines
I think few people will disagree about this.
Open questions now are
4) do we have the kernel kick off an async scan inopen()or do we have glibc do this
the kernel should not kick off a scan, instead it should check to see an open/read should not kick off a scan, instead it should check to see if the scan generation tag(s) are current should be enough (remember, you may have more then one type of scanner running on the system)
Quoted 2 lines
5) do we have the kernel do the sync scan on read/mmap/.. or do we have glibc do this
definantly not the kernel. the intent of this is to keep linux from being a storage repository for malware used by other systems. there is no need to penalize linux-only apps by making them wait for a scan to take place. If it lives in glibc there should be a way for linux apps that know that they will not be exporting files to other systems to tell the library not to do a scan.
for example, why should a log analysis program looking at apache logs be forced to wait while multiple 'virii scanners' go through several gigs of logs before it can start looking at them.
you are going to need some way to bypass the checks anyway so that you can avoid the recursive case of the scanners triggering scans on files that they open.
by keeping the scans all in userspace it also simplifies things greatly. All the kernel should do is to maintain the tags with the file (posix attributes??) and have a mechanism to clear them when the file is dirtied.
Quoted 4 lines
I think this is where the whole debate is about now.
And a few hard ones
6) how do we deal with multiple scanning agents in parallel
not a problem, in fact multiple agents scanning in parallel is a good thing, it lets them all see the data with one pass through the disk.
they will all need to set different tags anyway (the fact that agent1 blessed the data doesn't mean that it's safe if agent2 hasn't done so)
Quoted 1 line
7) how do we prevent malware from pretending to be a virus scanner
this is not part of the threat model.
David Lang
Hi!
Quoted 4 lines
Now this to me we have a few basic building blocks:
1) We need an efficient mechanism to notify userspace of files that get dirtied. Virus scanners will subscribe to this for the async dirty scanning; indexing agents also will subscribe to this.
ACK.
Quoted 7 lines
I think few people will disagree about this.
Open questions now are
4) do we have the kernel kick off an async scan inopen()or do we have glibc do this
5) do we have the kernel do the sync scan on read/mmap/.. or do we have glibc do this
How does it work? Memory can still change after mmap; scanning at the mmap time is _NOT_ enough.
You could do 'when app attempts to dirty memory, synchronously unmap it from all apps that have it mapped' and then do sync scan on pagefault time; but that sounds impractical.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 16 lines · Pavel Machek
Quoted 4 lines
Now this to me we have a few basic building blocks:
1) We need an efficient mechanism to notify userspace of files that get dirtied. Virus scanners will subscribe to this for the async dirty scanning; indexing agents also will subscribe to this.ACK.
Quoted 7 lines
I think few people will disagree about this.
Open questions now are
4) do we have the kernel kick off an async scan inopen()or do we have glibc do this
5) do we have the kernel do the sync scan on read/mmap/.. or do we have glibc do this
Quoted 6 lines
How does it work? Memory can still change after mmap; scanning at the mmap time is _NOT_ enough.
You could do 'when app attempts to dirty memory, synchronously unmap it from all apps that have it mapped' and then do sync scan on pagefault time; but that sounds impractical.
what is the threat you are trying to defend against?
for some threats you are right, for others the scan at mmap time is enough.
David Lang
Hi!
Quoted 10 lines
Quoted 9 lines
How does it work? Memory can still change after mmap; scanning at the
mmap time is _NOT_ enough.You could do 'when app attempts to dirty memory, synchronously unmap
it from all apps that have it mapped' and then do sync scan on
pagefault time; but that sounds impractical.
Quoted 4 lines
what is the threat you are trying to defend against?
for some threats you are right, for others the scan at mmap time is enough.
I don't see any threats when check at mmap time is okay.
As soon as file servers use mmap, this race can bite you even in very simple 'make sure Linux fileserver does not pass on windows malwar' threat model.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 14 lines · Pavel Machek
Hi!
Quoted 12 lines
Quoted 9 lines
How does it work? Memory can still change after mmap; scanning at the
mmap time is _NOT_ enough.You could do 'when app attempts to dirty memory, synchronously unmap
it from all apps that have it mapped' and then do sync scan on
pagefault time; but that sounds impractical.what is the threat you are trying to defend against?
Quoted 8 lines
Quoted 2 lines
for some threats you are right, for others the scan at mmap time is enough.
I don't see any threats when check at mmap time is okay.
As soon as file servers use mmap, this race can bite you even in very simple 'make sure Linux fileserver does not pass on windows malwar' threat model.
if you check the file at the time you mmap it and don't expect other programs to change the file under you while you are reading it, there's nothing to worry about.
shared mmap with one program being able to write to it is a expecially hard case of the general problem that if someone writes to a file as you are serving it, you don't know what goes out, it may be part of the old file and part of the new file. it's just easier to see this problem with mmap and harder to catch it.
David Lang
On Wed, Aug 13, 2008 at 12:36:15PM -0400, Eric Paris wrote:
I miss a clear answer to the question: is this supposed to protect against malware injected as root or not?
Assuming it wants to protect against root:
Quoted 8 lines
think we hear claim more grandious things. But from what I've seen they aren't the real deal. Most of the security model descriptions that people on list want actually are framed under the belief that these products need to or attempt to completely block some class of attacks. They don't. If you think they do you need to fix your frame of reference. The only class of attacks this interface is supposed to address is those that can be found by scanning files. This is NOT an LSM.
But you need some LSM like protections to be able to protect the file scanner? Like the block device or kernel memory protection.
Quoted 8 lines
The real goal is to get files into to some userspace scanner and let them do whatever they want. Remember, this isn't a new LSM. The goal isn't to provide perfect security. The goal isn't to stop already running malicious programs. The one and only goal is to scan files. We should not be considering timing attacks, we should not be considering processes actively trying to get around the system for small periods of time. We should certainly not be considering root processes being able to sneak things by.
This means you need significant LSM components simply to protect the integrity of the file scanner against root. It's even unclear it's possible in the general case (e.g. X server doing arbitary DMA and no IOMMU -- how do you protect the file scanner?)
Quoted 2 lines
The idea is that a file exists on disk and we want
some userspace program to give a best effort at scanning it. Yes, we
Ok so you're implying it's ok to not protect against root?
In the later case that means that you don't have to scan anything that only root can touch and you can trust file permissions, which makes a lot of things easier.
I would suggest again to clarify this important point first. It has significant impact on the whole design.
Personally I would think not protecting against root would be quite limiting (e.g. it would mean that e.g. if some worm trojans rpms people download then they wouldn't be caught because rpms are installed as root), but on the other hand if you protect against root you need most of selinux/aa/other lsm functionality simply to guarantee the integrity of the scanner. Also it has impact on some apps, e.g. X server running as root would be usually out due to the arbitary DMA issue. Also protect block devices could theoretically have significant impact on some sysadmin tasks.
-Andi
Quoted 3 lines · On Wed, 13 Aug 2008 20:17:14 +0200 Andi Kleen <[email protected]> wrote:
I would suggest again to clarify this important point first. It has significant impact on the whole design.
agreed.
Quoted 5 lines
Personally I would think not protecting against root would be quite limiting (e.g. it would mean that e.g. if some worm trojans rpms people download then they wouldn't be caught because rpms are installed as root),
on argument could be that root apps like that could/should do explicit scanning regardless.
(if we have an explicit interface to scan a file that's not too hard)
Signature · 3 lines
If you want to reach me at my work email, use [email protected] For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 5 lines · Andi Kleen
This means you need significant LSM components simply to protect
the integrity of the file scanner against root. It's even unclear it's possible in the general case (e.g. X server doing arbitary DMA and no IOMMU -- how do you protect the file scanner?)
Without Treacherous Computing, it isn't possible, even in kernel space.
In Treacherous Computing you can put it in the hypervisor, which of course just means the hypervisor is now much bigger and likely to contain security holes.
-hpa
Quoted 4 lines · Andi Kleen
On Wed, Aug 13, 2008 at 12:36:15PM -0400, Eric Paris wrote:
I miss a clear answer to the question: is this supposed to protect against malware injected as root or not?
I thought I had summed that up. We are not interested in providing protections against maliciously attacking programs be they root or not. We are interested in scanning files read and written by root. We are especially interested in programs run by root.
yum (as root) downloads trojan.rpm from youareanidiot.repo. We aren't worried about yum maliciously attacking the system. What's going to happen is that the scanner is going to scan the trojan.rpm when yum calls rpm and rpm is going to be denied access to that file. Doesn't matter how you download it, its going to get scanned when you try to exec it.
Stop thinking this is an LSM or as a new security model. It's a file scanner and "it ain't perfect security." But its very useful and practical. If some malicious root application wants to turn it off it can and I make no claims otherwise. This isn't supposed to help once root has been subverted, you've already lost as you admit, its supposed to help keep root form getting subverted.
-Eric
Quoted 50 lines
Quoted 9 lines
think we hear claim more grandious things. But from what I've seen they aren't the real deal. Most of the security model descriptions that people on list want actually are framed under the belief that these products need to or attempt to completely block some class of attacks. They don't. If you think they do you need to fix your frame of reference. The only class of attacks this interface is supposed to address is those that can be found by scanning files. This is NOT an LSM.
But you need some LSM like protections to be able to protect the file scanner? Like the block device or kernel memory protection.
Quoted 8 lines
The real goal is to get files into to some userspace scanner and let them do whatever they want. Remember, this isn't a new LSM. The goal isn't to provide perfect security. The goal isn't to stop already running malicious programs. The one and only goal is to scan files. We should not be considering timing attacks, we should not be considering processes actively trying to get around the system for small periods of time. We should certainly not be considering root processes being able to sneak things by.
This means you need significant LSM components simply to protect the integrity of the file scanner against root. It's even unclear it's possible in the general case (e.g. X server doing arbitary DMA and no IOMMU -- how do you protect the file scanner?)
Quoted 2 lines
The idea is that a file exists on disk and we want
some userspace program to give a best effort at scanning it. Yes, weOk so you're implying it's ok to not protect against root?
In the later case that means that you don't have to scan anything that only root can touch and you can trust file permissions, which makes a lot of things easier.
I would suggest again to clarify this important point first. It has significant impact on the whole design.
Personally I would think not protecting against root would be quite limiting (e.g. it would mean that e.g. if some worm trojans rpms people download then they wouldn't be caught because rpms are installed as root), but on the other hand if you protect against root you need most of selinux/aa/other lsm functionality simply to guarantee the integrity of the scanner. Also it has impact on some apps, e.g. X server running as root would be usually out due to the arbitary DMA issue. Also protect block devices could theoretically have significant impact on some sysadmin tasks.
-Andi
Quoted 4 lines · Andi Kleen
On Wed, Aug 13, 2008 at 12:36:15PM -0400, Eric Paris wrote:
I miss a clear answer to the question: is this supposed to protect against malware injected as root or not?
I honestly don't think we should worry about root. Sure, if the AV scanner happens to catch something (as a consequence of it's implementation), then very well. But designing an antimalware solution which assumes the root is compromised will throw us into security talks for years and I don't think we'll live to hear the end of them.
We should focus on the regular users and fix (if needed) the current userland apps (ie. the ones that need root access to do their job). For anymore than that we'll need a super user that supervises root. And then another one.
Quoted 36 lines
Assuming it wants to protect against root:
Quoted 8 lines
think we hear claim more grandious things. But from what I've seen they aren't the real deal. Most of the security model descriptions that people on list want actually are framed under the belief that these products need to or attempt to completely block some class of attacks. They don't. If you think they do you need to fix your frame of reference. The only class of attacks this interface is supposed to address is those that can be found by scanning files. This is NOT an LSM.
But you need some LSM like protections to be able to protect the file scanner? Like the block device or kernel memory protection.
Quoted 8 lines
The real goal is to get files into to some userspace scanner and let them do whatever they want. Remember, this isn't a new LSM. The goal isn't to provide perfect security. The goal isn't to stop already running malicious programs. The one and only goal is to scan files. We should not be considering timing attacks, we should not be considering processes actively trying to get around the system for small periods of time. We should certainly not be considering root processes being able to sneak things by.
This means you need significant LSM components simply to protect the integrity of the file scanner against root. It's even unclear it's possible in the general case (e.g. X server doing arbitary DMA and no IOMMU -- how do you protect the file scanner?)
Quoted 2 lines
The idea is that a file exists on disk and we want
some userspace program to give a best effort at scanning it. Yes, weOk so you're implying it's ok to not protect against root?
In the later case that means that you don't have to scan anything that only root can touch and you can trust file permissions, which makes a lot of things easier.
Quoted 7 lines
I would suggest again to clarify this important point first. It has significant impact on the whole design.
Personally I would think not protecting against root would be quite limiting (e.g. it would mean that e.g. if some worm trojans rpms people download then they wouldn't be caught because rpms are installed as root)
If GPG signatures don't work, then please fix the rpm design and if the user willingly installs a .rpm which is not signed (not from a known trusted host) and somehow doges the basic antimalware scanner, then too bad. We've done all we could.
Quoted 6 lines
, but on the other hand if you protect against
root you need most of selinux/aa/other lsm functionality simply to guarantee the integrity of the scanner. Also it has impact on some apps, e.g. X server running as root would be usually out due to the arbitary DMA issue. Also protect block devices could theoretically have significant impact on some sysadmin tasks.
I think we need to define the 'desktop user' and provide a decent protection mechanism for his common activities (edit documents, listen music, navigate the web, see movies, run scripts which change the IM status etc). For the rest, there are two possibilities:
1. education (_extremely_ important);
2. SELinux (or similar);
I don't think there will ever be an AV product using the marketing line: "it allows you to run your favorite rootkit and enjoy the pretty text it shows, with no worries".
In conclusion: everything AV related should stop at the user root. Popular distro-s already provide a way to do your daily office tasks without super user rights, which _is_ the correct thing to do.
Signature · 1 line
Mihai Donțu
Quoted 22 lines
-----Original Message-----
From: [email protected] [mailto:malware-list- [email protected]] On Behalf Of Mihai Don?u
Sent: Wednesday, August 13, 2008 8:18 PM
To: Andi Kleen
Cc: [email protected]; [email protected]; malware- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [malware-list] TALPA - a threat model? well sorta.Quoted 4 lines · Andi Kleen
On Wed, Aug 13, 2008 at 12:36:15PM -0400, Eric Paris wrote:
I miss a clear answer to the question: is this supposed to protect against malware injected as root or not?
I honestly don't think we should worry about root. Sure, if the AV scanner happens to catch something (as a consequence of it's implementation), then very well. But designing an antimalware solution which assumes the root is compromised will throw us into security talks for years and I don't think we'll live to hear the end of them.
Quoted 3 lines
We should focus on the regular users and fix (if needed) the current userland apps (ie. the ones that need root access to do their job). For anymore than that we'll need a super user that supervises root. And then another one.
I think that some people are missing the important point of Eric's recent original statement of the "threat model". Whether we move further in the direction of other security protections or not, we are currently talking about providing a mechanism for basic AV product to do their job, and the job we are talking about is scanning files when they are about to be used and might cause harm, or have just been created and we want to make sure they are OK. That is, the AV products that we are talking about in this context don't do anything else other than scan files.
With that in mind, there is no difference between scanning files being accessed/executed/created by root and the same for any other users. And in fact, to the extent that we claim at all to have a somewhat complete protection in that realm, excluding root will completely blow that protect out of the water and make it essentially useless.
Quoted 6 lines
I think we need to define the 'desktop user' and provide a decent protection mechanism for his common activities (edit documents, listen music, navigate the web, see movies, run scripts which change the IM status etc). For the rest, there are two possibilities:
1. education (_extremely_ important);
It's like abstinence education...it sounds good, at least to some, but it doesn't work. In a way, that's the whole point. There are millions of users. It doesn't take many who missed the class to create an outbreak that does real damage. It goes back to the medical analogy. Do you spray the swamps for the mosquitoes that carry Eastern Equine Encephalitis, or do you knock on everyone's door and tell them not to go near the swamps, and hope that everyone's home when you're in their neighborhood?
Quoted 3 lines
I don't think there will ever be an AV product using the marketing line: "it allows you to run your favorite rootkit and enjoy the pretty text it shows, with no worries".
You are right... Complete rootkit protection is a whole other area not fundamentally addressed by a scan. So let's not create a straw man about the things we don't claim to do and then knock the products because we don't do them.
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Quoted 17 lines · Jonathan
Quoted 12 lines · Andi Kleen
Quoted 4 lines · Eric Paris
I miss a clear answer to the question: is this supposed to protect against malware injected as root or not?
I honestly don't think we should worry about root. Sure, if the AV scanner happens to catch something (as a consequence of it's implementation), then very well. But designing an antimalware solution which assumes the root is compromised will throw us into security talks for years and I don't think we'll live to hear the end of them.
Quoted 4 lines
We should focus on the regular users and fix (if needed) the current userland apps (ie. the ones that need root access to do their job). For anymore than that we'll need a super user that supervises root. And then another one.
Quoted 8 lines
I think that some people are missing the important point of Eric's recent original statement of the "threat model". Whether we move further in the direction of other security protections or not, we are currently talking about providing a mechanism for basic AV product to do their job, and the job we are talking about is scanning files when they are about to be used and might cause harm, or have just been created and we want to make sure they are OK. That is, the AV products that we are talking about in this context don't do anything else other than scan files.
I see. Well, as long as everyone sticks to _just_ the file scan. To be honest, the only immediate use of the patch that is/was in question, is a "natural" scanner for file servers (Samba, NFS etc). 7v5w7go9ub0o, however, might have some more ideas. :)
I admit and I apologize, I got pretty worked up when people started asking questions like: "how do we protect the file scanner", when the answer should have been obvious: the way we protect any other daemon (service) today, by means of chmod/chown.
Quoted 29 lines
With that in mind, there is no difference between scanning files being accessed/executed/created by root and the same for any other users. And in fact, to the extent that we claim at all to have a somewhat complete protection in that realm, excluding root will completely blow that protect out of the water and make it essentially useless.
Quoted 6 lines
I think we need to define the 'desktop user' and provide a decent protection mechanism for his common activities (edit documents, listen music, navigate the web, see movies, run scripts which change the IM status etc). For the rest, there are two possibilities:
1. education (_extremely_ important);
It's like abstinence education...it sounds good, at least to some, but it doesn't work. In a way, that's the whole point. There are millions of users. It doesn't take many who missed the class to create an outbreak that does real damage. It goes back to the medical analogy. Do you spray the swamps for the mosquitoes that carry Eastern Equine Encephalitis, or do you knock on everyone's door and tell them not to go near the swamps, and hope that everyone's home when you're in their neighborhood?
Quoted 3 lines
I don't think there will ever be an AV product using the marketing line:
"it allows you to run your favorite rootkit and enjoy the pretty text it shows, with no worries".You are right... Complete rootkit protection is a whole other area not fundamentally addressed by a scan. So let's not create a straw man about the things we don't claim to do and then knock the products because we don't do them.
Signature · 2 lines
Mihai Donțu Again, this mail == my own opinion
(this was posted in linux.kernel, before I realized there was a linux.kernel.malware. Hope it helps your discussion)
(FYI. Dazuko may have trailblazed some of the issues now under discussion re: libmalware.so. It has worked well for me. It used to be an LKM, it is now a source patch. It is used in a number of commercial products)
<http://dazuko.dnsalias.org/wiki/index.php/Main_Page>
"A Virtual Device Driver to Allow Online File Access Control
A common interface is needed, which allows userland applications to perform online file access control. Dazuko aims to provide that interface."
FWIW, I'm not associated with Dazuko or Antivir; I've been happily using Dazuko with AntiVir for a year or so.
1. AntiVir includes numerous Linux signatures as well as Windows. So I scan both 'ix downloads, as well as the process of compiling new software.
2. Other AntiMalwares are using Dazuko, though many are scanning for Windows malware only.
3. The AntiVir/Dazuko combination with full heuristics has blocked access to clearly dangerous JS scripts in my browser cache.
4. IMHO, what is needed is a Dazuko or libmalware/Integrity database link. If an md5 of an executable or script is new or has changed, access is blocked 'til a response to a popup is given. Access can be blocked; one-time allowed; or permanently allowed, in which case the md5 is updated.
Hope This Helps.
<next msg>
Quoted 9 lines · Andi Kleen
Quoted 2 lines · 7v5w7go9ub0o <[email protected]> writes:
(FYI. Dazuko may have trailblazed some of the issues now under discussion re: libmalware.so. It has worked well for me.
Against what exactly did it protect you? Please give a concrete example.
-Andi
1. This came in a few minutes ago:
Aug 13 14:56:31 tux antivir[6381]: AntiVir ALERT: [EML/FakeLink.F]
/jail/tbird/root/.thunderbird/0r2957kg.default/Mail/L
ocal Folders/Junk.XXX <<< Contains detection pattern of EML/FakeLink.F in EML form
2. I have not retained the logs of "suspicious scripts" in my browser, but have come across perhaps 4 blocked scripts within the last month. Admittedly at dodgy sites.
XSS attacks are platform independent, and are a significant concern.
Please note that when I say it has worked well for me, I am not saying that it has saved my bacon! :-)
1. I am referring to the mechanics of having the Kernel/userland app stop processing when it finds a malware signature or heuristic detection.
2. Am also referring to the totally manageable (IMHO) overhead.
I've mentioned my experience with Dazuko/antivir only because it may be useful to the ongoing discussion about the nature of libmalware.so.
3. I am frankly waiting for a bug to get into my upstream distribution chain - through a hijacking or some wonderful DNS prank - at which point I ..hope.. a signature or heuristic will block my root-enabled make install.
4. Again, my hope for libmalware.so/dazuko is a realtime integrity-management link.
<end posts>
HTH
p.s. The question has developed, should this monitor root activities. IMHO, the answer is a definite YES! We are most vulnerable during software updating; AntiMailware signatures may stop the compilation or installation of a Trojan - by root.
Quoted 10 lines · 7v5w7go9ub0o wrote:
4. Again, my hope for libmalware.so/dazuko is a realtime integrity-management link.
<end posts>
HTH
p.s. The question has developed, should this monitor root activities. IMHO, the answer is a definite YES! We are most vulnerable during software updating; AntiMailware signatures may stop the compilation or installation of a Trojan - by root.
I just noticed a separate discussion about integrity-checking LKMs and LSMs.
Obviously, a libmalware.so or Dazuko based integrity-checker would block a kernel from loading in a Trojaned LKM - noting that the MD5 had changed, and asking you to block, temporarily allow, or permanently allow the changed module.
Another security benefit of your pursuit.
HTH