Out of tree module using LSM
Hi Linus, all,
During one recent LKML discussion
(http://marc.info/?l=linux-kernel&m=119267398722085&w=2) about LSM going static you called for LSM users to speak up.
We here at Sophos (the fourth largest endpoint security vendor in the world) have such a module called Talpa which is a part of our main endpoint security product for Linux that protects from viruses and malware hosted on Linux, including those targetting Windows or other connected devices,
(http://www.sophos.com/products/enterprise/endpoint/security-and-control/linux/index.html)
which is GPL code and has been in the field for almost three years now. It's source code has been shipping with the product from the start. We also have a SourceForge project at http://sourceforge.net/projects/talpa/ to host it.
In essence, what our module does is it intercepts file accesses and allows userspace daemons to vet them. One of the means we implemented that is through LSM and although it is not a perfect match for such use we prefer to use an official interface. Unfortunately, with time it became impossible to use LSM on some distributions (SELinux) so we had to implement other intercept methods which are significantly less nice, and which may also become unworkable over time.
The main point here is that making LSM not available for modules makes our life even more difficult and we would like to ask you to revert that change until another acceptable, secure solution is found.
At this point I would not like this to turn into a discussion on technical or code merits of our current approach. It is just that usage scenarios of (dis-)allowing access to files based on their content, regardless of the actual details, do sound like a reasonable idea. What we would like to do is to start discussion about possible solutions in which other interested parties will hopefully join and in future create some progress on this front.
Signature · 11 lines
Tvrtko August Ursulin Senior Software Engineer, Sophos Tel: 01235 559933 Web: www.sophos.com Protecting business against viruses, spyware, spam and policy abuse Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom. Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Quoted 7 lines · Tvrtko A. Ursulin
Hi Linus, all,
During one recent LKML discussion
(http://marc.info/?l=linux-kernel&m=119267398722085&w=2) about LSM going static you called for LSM users to speak up.
Quoted 8 lines
We here at Sophos (the fourth largest endpoint security vendor in the world) have such a module called Talpa which is a part of our main endpoint security product for Linux that protects from viruses and malware hosted on Linux, including those targetting Windows or other connected devices,
(http://www.sophos.com/products/enterprise/endpoint/security-and-control/linux/index.html)
which is GPL code and has been in the field for almost three years now. It's source code has been shipping with the product from the start. We also have a SourceForge project at http://sourceforge.net/projects/talpa/ to host it.
Thanks for showing a wonderful source code example of why lsm shouldn't be exported :)
Quoted 4 lines · --- Christoph Hellwig
Quoted 3 lines · Tvrtko A. Ursulin
Hi Linus, all,
Quoted 12 lines
Quoted 5 lines
During one recent LKML discussion
(http://marc.info/?l=linux-kernel&m=119267398722085&w=2) about LSM going static you called for LSM users to speak up.We here at Sophos (the fourth largest endpoint security vendor in the
world)
Quoted 1 line
have such a module called Talpa which is a part of our main endpoint
security
Quoted 3 lines
product for Linux that protects from viruses and malware hosted on Linux, including those targetting Windows or other connected devices,
(http://www.sophos.com/products/enterprise/endpoint/security-and-control/linux/index.html)
Quoted 9 lines
Quoted 2 lines
which is GPL code and has been in the field for almost three years now.
It's
Quoted 1 line
source code has been shipping with the product from the start. We also
have
Quoted 1 line
a SourceForge project at http://sourceforge.net/projects/talpa/ to host it.
Thanks for showing a wonderful source code example of why lsm shouldn't be exported :)
Would you like to expound on that, or do you feel your claws are sharp enough already?
Casey Schaufler
[email protected]
Quoted 2 lines · Casey Schaufler
Would you like to expound on that, or do you feel your claws are sharp enough already?
Just take a look at code.
Quoted 6 lines · On Wed, 28 Nov 2007 16:46:13 +0000 Christoph Hellwig <[email protected]> wrote:
Quoted 2 lines · Casey Schaufler
Would you like to expound on that, or do you feel your claws are sharp enough already?
Just take a look at code.
The module in question hooks to the syscall table which is not acceptable. Including dumpster diving through symbol table to find the syscall table since it is no longer exported.
Signature · 1 line
Stephen Hemminger <[email protected]>
[email protected] wrote on 28/11/2007 17:39:56:
Quoted 7 lines
Quoted 4 lines · On Wed, 28 Nov 2007 16:46:13 +0000 Christoph Hellwig <[email protected]> wrote:
Quoted 2 lines · Casey Schaufler
Would you like to expound on that, or do you feel your claws are sharp enough already?
Quoted 6 lines
Quoted 2 lines
Just take a look at code.
The module in question hooks to the syscall table which is not acceptable. Including dumpster diving through symbol table to find the syscall table since it is no longer exported.
That is not completely true.
As I said in the second part of my original email (that part Christoph deleted so please read it whole) I would prefer not to discuss all the ugly things some parts of our code do because they are irrelevant to the LSM discussion. But I still feel some extra explanation might be needed.
Talpa is modular itself being composed of a set of kernel modules of which not all are loaded simultaneously. Where possible LSM can be used and _no_ messing with syscall table will take place. Unfortunately where another LSM user is present that won't work and another solution had to be found. Also that drags history from 2.4 where that was the only solution.
So as there is no question the current code does some ugly things it is even more true that we would be even more happy to use an official API. LSM was that and we were happily using it which we won't be able to do if it abruptly goes away. Yes it is not a perfect match but until it is modified to be better, or until something appropriate is designed and implemented, it would be very nice if it could stay.
Signature · 12 lines
Tvrtko August Ursulin Senior Software Engineer, Sophos Tel: 01235 559933 Web: www.sophos.com Protecting business against viruses, spyware, spam and policy abuse Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom. Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Quoted 6 lines
So as there is no question the current code does some ugly things it is even more true that we would be even more happy to use an official API. LSM was that and we were happily using it which we won't be able to do if it abruptly goes away. Yes it is not a perfect match but until it is modified to be better, or until something appropriate is designed and implemented, it would be very nice if it could stay.
So for an SELinux based system what you are saying is you want to be able to stack your module with the SELinux module and after SELinux has considered policy rules still be able to veto them on the grounds that you are say about to serve a virus to a windows box ?
Alan
Alan Cox <[email protected]> wrote on 28/11/2007 19:50:42:
Quoted 1 line
Quoted 1 line
So as there is no question the current code does some ugly things it
is
Quoted 1 line
Quoted 1 line
even more true that we would be even more happy to use an official
API.
Quoted 1 line
Quoted 1 line
LSM was that and we were happily using it which we won't be able to do
if
Quoted 5 lines
Quoted 3 lines
it abruptly goes away. Yes it is not a perfect match but until it is modified to be better, or until something appropriate is designed and implemented, it would be very nice if it could stay.
So for an SELinux based system what you are saying is you want to be
able
Quoted 3 lines
to stack your module with the SELinux module and after SELinux has considered policy rules still be able to veto them on the grounds that you are say about to serve a virus to a windows box ?
Basically yes but the effective scenario is a bit wider. Local actions like disallowing execution of rootkits, exploits and other similar malware are also interesting. Another example would be enforcing a corporate policy on which IM clients shouldn't be used so it is not just fileserver scenario in which Linux machines can be compromised.
But really I am not the best person to know all current attack vectors. Overall set of requirements and ideas is something we are working on with other vendors and hopefully with the community. This is one of the two main things my original post was about.
--
Tvrtko August Ursulin
Senior Software Engineer, Sophos
"Views and opinions expressed in this email are strictly those of the author.
The contents has not been reviewed or approved by Sophos."
Tel: 01235 559933
Web: www.sophos.com
Protecting business against viruses, spyware, spam and policy abuse
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Quoted 2 lines · On Wed, 28 Nov 2007, [email protected] wrote:
So as there is no question the current code does some ugly things it is even more true that we would be even more happy to use an official API.
How about becoming involved in creating that official API ?
"A person will stand on the top of a hill for a very long time with their mouth open before a roast duck will fly in"
Quoted 4 lines
LSM was that and we were happily using it which we won't be able to do if it abruptly goes away. Yes it is not a perfect match but until it is modified to be better, or until something appropriate is designed and implemented, it would be very nice if it could stay.
So, what you're asking for is for us to maintain infrastructure in the kernel, for your out of tree code, which we could not have even known about until it was dumped on sourceforge a couple of days ago ?
And you're expecting us to modify it to be "better", for you, but without us possibly knowing what you want ?
How is this sustainable? Every time we make a change in the kernel code, do we have to wait for months while all of the (unknown) out of tree users discover their code is broken and then complain to Linus ?
Do you really thing we should be supporting developers who make zero effort to engage with the kernel community, and instead abuse kernel APIs and ship out of tree code to customers?
You are essentially demanding that we provide you with a stable kernel API. I suggest you review Greg KH's excellent document on the issue:
<http://os.cqu.edu.au/docs/kernel-doc/Documentation/stable_api_nonsense.txt>
Also, your users are likely unaware of the risks associated with these techniques, such as, that they are not actually running "linux" any more, and that their kernel is made unsupportable by both the community and their OS vendors when code like this is being used:
/*
* hidden vfsmnt_lock handling
*/
#ifdef TALPA_USE_VFSMOUNT_LOCK
static inline void talpa_vfsmount_lock(void)
{
spinlock_t* talpa_vfsmount_lock_addr = (spinlock_t *)TALPA_VFSMOUNT_LOCK_ADDR;
spin_lock(talpa_vfsmount_lock_addr);
}Code quality and correctness issues are definitely relevant, and in fact are directly linked to the fact that out of tree code receives no community peer review, and is not able to be considered in the general kernel design & development process.
Hacks like this effectively void support from the community, which is another way of saying that the user's kernel becomes proprietary once the module containing it is loaded.
Actually, it's worse than that, as not only is the kernel no longer "open source", but also lacks even the level of support provided by proprietary OS vendors (who provide stable interfaces for such purposes).
If you want to resolve this properly, you'll need to do more than complain to Linus when the upstream APIs (which you are abusing) change.
You'll need to become productively engaged in designing, developing and maintaining appropriate APIs, which suit not only your specific needs, but likely those of others, and submit your code for review and upstream inclusion.
What I'd suggest, is that you create a public mailing list, get the other AV projects involved, and then work on a set of requirements so that the general problem can be understood. Then, propose a solution to the problem and have it reviewed by core kernel folk (cc it to lkml), so that it can be evaluated in terms of e.g. VFS correctness, maintainability etc.
That would be at least a useful first step in taking this issue seriously.
Thanks,
- James
Signature · 2 lines
James Morris <[email protected]>
Quoted 6 lines · James Morris
Quoted 2 lines · On Wed, 28 Nov 2007, [email protected] wrote:
So as there is no question the current code does some ugly things it is even more true that we would be even more happy to use an official API.
How about becoming involved in creating that official API ?
Sophos are interested in doing so, and we have spoken about this several times recently over the phone. This is why they sent the email in question yesterday, to kickstart debate. And that's awesome. I am trying to bring a few of these folks together at the moment, so that we can get a solution that is acceptable to upstream at some point in the future.
So, rather than criticise their current code, or their intentions, or blanketly dismiss the virus protection market, perhaps we can focus instead on the fact that there is a known third party who wishes to perform a task that is not well supportable at this moment. We can all agree the syscall table hacking isn't such a good idea - but these guys are *very* open to listening to useful alternative suggestions.
They (virus protection folks) generally think they want to intercept various system calls, such as open() and block until they have performed a scan operation on the file. I explained the mmap issue to several of these companies recently, in quite some detail, and I know they are interested in listening this time around :-) At the end of the day, what I have been lead to believe is that they don't care whether they intercept syscall entries, or use a better method, they just want to scan files and take some action if a file is "bad". That's it really.
I have been trying to put together an exact feature set that is needed from these different vendors, so we can discuss it further here, and hopefully actually get somewhere, too. There have been a few delays after I pointed out the mmap issues at some length.
Jon.
Quoted 10 lines · Jon Masters
Quoted 6 lines · James Morris
Quoted 2 lines · On Wed, 28 Nov 2007, [email protected] wrote:
So as there is no question the current code does some ugly things it is even more true that we would be even more happy to use an official API.
How about becoming involved in creating that official API ?
Sophos are interested in doing so, and we have spoken about this several times recently over the phone.
Great, but this email did not show that they were interested in doing so.
Quoted 6 lines
So, rather than criticise their current code, or their intentions, or blanketly dismiss the virus protection market, perhaps we can focus instead on the fact that there is a known third party who wishes to perform a task that is not well supportable at this moment. We can all agree the syscall table hacking isn't such a good idea - but these guys are *very* open to listening to useful alternative suggestions.
It's not only not a good idea, it's insecure and is trivial to circumvent, so it's a non-issue here.
Quoted 8 lines
They (virus protection folks) generally think they want to intercept various system calls, such as
open()and block until they have performed a scan operation on the file. I explained the mmap issue to several of these companies recently, in quite some detail, and I know they are interested in listening this time around :-) At the end of the day, what I have been lead to believe is that they don't care whether they intercept syscall entries, or use a better method, they just want to scan files and take some action if a file is "bad". That's it really.
Yes, I too have talked with people doing this kind of work, see my other messages in this thread for how it can be done all in userspace, no kernel changes needed at all.
thanks,
greg k-h
Quoted 4 lines · Jon Masters
They (virus protection folks) generally think they want to intercept various system calls, such as
open()and block until they have performed a scan operation on the file. I explained the mmap issue [...]
If open and close was everything, then that would be wonderful. You could only wonder how many false positives scanners could bring up if they checked every write() for signatures - not to mention performance bogdown.
Quoted 3 lines
they just want to scan files and take some action if a file is "bad". That's it really.
struct security->dentry_open sounds like the candidate, together with relayfs with submits filenames to userspace.
Quoted 30 lines · On Thu, 29 Nov 2007 11:27:45 -0500 Jon Masters <[email protected]> wrote:
Quoted 6 lines · James Morris
Quoted 2 lines · On Wed, 28 Nov 2007, [email protected] wrote:
So as there is no question the current code does some ugly things it is even more true that we would be even more happy to use an official API.
How about becoming involved in creating that official API ?
Sophos are interested in doing so, and we have spoken about this several times recently over the phone. This is why they sent the email in question yesterday, to kickstart debate. And that's awesome. I am trying to bring a few of these folks together at the moment, so that we can get a solution that is acceptable to upstream at some point in the future.
So, rather than criticise their current code, or their intentions, or blanketly dismiss the virus protection market, perhaps we can focus instead on the fact that there is a known third party who wishes to perform a task that is not well supportable at this moment. We can all agree the syscall table hacking isn't such a good idea - but these guys are *very* open to listening to useful alternative suggestions.
They (virus protection folks) generally think they want to intercept various system calls, such as
open()and block until they have performed a scan operation on the file. I explained the mmap issue to several of these companies recently, in quite some detail, and I know they are interested in listening this time around :-) At the end of the day, what I have been lead to believe is that they don't care whether they intercept syscall entries, or use a better method, they just want to scan files and take some action if a file is "bad". That's it really.
Quoted 5 lines
I have been trying to put together an exact feature set that is needed from these different vendors, so we can discuss it further here, and hopefully actually get somewhere, too. There have been a few delays after I pointed out the mmap issues at some length.
Perhaps this kind of scanning belongs in the application. Couldn't an apache or samba have a plugin to do it?
Signature · 1 line
Stephen Hemminger <[email protected]>
Quoted 5 lines · On Nov 28 2007 18:22, [email protected] wrote:
Talpa is modular itself being composed of a set of kernel modules of which not all are loaded simultaneously. Where possible LSM can be used and _no_ messing with syscall table will take place. Unfortunately where another LSM user is present that won't work
SELinux supports chaining, so if talpa is loaded as a secondary to selinux, where is the problem? For those LSMs which do not support chaining (*cough* apparmor *cough* be one, mtadm another), fix them.
Quoted 11 lines · --- Jan Engelhardt
Quoted 5 lines · On Nov 28 2007 18:22, [email protected] wrote:
Talpa is modular itself being composed of a set of kernel modules of which not all are loaded simultaneously. Where possible LSM can be used and _no_ messing with syscall table will take place. Unfortunately where another LSM user is present that won't work
SELinux supports chaining, so if talpa is loaded as a secondary to selinux, where is the problem? For those LSMs which do not support chaining (*cough* apparmor *cough* be one, mtadm another), fix them.
Um, cough cough (I ready do have a nasty cold) SELinux supports a very limited bit of chaining. I don't think you're going to be chaining security_secid_to_secctx() or security_secctx_to_secid() with the current SELinux code, but you could prove me wrong there.
Chaining is a red herring. If you want talpa it seems that you have a use case that isn't going to require the presence of another LSM. You may have other issues, but at this point I say throw caution to the wind, clean it up based on the suggestions you've seen here, and put the patch up as an RFC on the LSM list.
What's the worst that could happen?
Casey Schaufler
[email protected]
Quoted 5 lines · Christoph Hellwig
Quoted 2 lines · Casey Schaufler
Would you like to expound on that, or do you feel your claws are sharp enough already?
Just take a look at code.
Just to clarify - you're OK with the *concept* (a security model that determines whether you can do an I/O based on the content), it's just their code that's ugly?
(Note that the concept has interesting implications in the other direction as well - rather than stopping you from reading a file that has malware, you could in theory write an anti-export package that would let you write onto external memory or outbound e-mail, but prevent the write if it was corporate-sensitive data, or whatever. Yes, I *know* a smart attacker can bypass it by simply crypting/compressing it first - but the vast majority of attackers aren't smart, and will just use 'cp' or the GUI equivalent to move the secret design documents onto the USB key... )
Quoted 5 lines · On Wed, Nov 28, 2007 at 01:15:05PM -0500, [email protected] wrote:
(Note that the concept has interesting implications in the other direction as well - rather than stopping you from reading a file that has malware, you could in theory write an anti-export package that would let you write onto external memory or outbound e-mail, but prevent the write if it was corporate-sensitive data, or whatever.
You _can_ _not_ do that. If shared mapping gets dirtied, you have no way to intercept that. At all. Especially since the page stays mapped while it is written out, so the next modification can come when hardware had already started outbound DMA and there's no way to abort it, no matter what your external scanner would do.
Folks, really, that doesn't work. At all. You can intercept all system calls you want and it will not be enough to prevent the "bad" contents from hitting the disk.
And if we are talking about the situation when files are written to in controlled way (i.e. we are not concerned with malware running on the box in question and just want to stop it from passing through mailsewer, etc.), then there's no damn need to play with LSM - just have e.g. coda with its commit-on-close and run the scanner on commit. End of story. Mind you, in such setups one would be much better off just having the mail server run the tests explicitly in the userland, along with the rest of anti-spam, etc. filters.
Quoted 17 lines · Al Viro
Quoted 5 lines · On Wed, Nov 28, 2007 at 01:15:05PM -0500, [email protected] wrote:
(Note that the concept has interesting implications in the other direction as well - rather than stopping you from reading a file that has malware, you could in theory write an anti-export package that would let you write onto external memory or outbound e-mail, but prevent the write if it was corporate-sensitive data, or whatever.
You _can_ _not_ do that. If shared mapping gets dirtied, you have no way to intercept that. At all. Especially since the page stays mapped while it is written out, so the next modification can come when hardware had already started outbound DMA and there's no way to abort it, no matter what your external scanner would do.
Folks, really, that doesn't work. At all. You can intercept all system calls you want and it will not be enough to prevent the "bad" contents from hitting the disk.
Quoted 8 lines
And if we are talking about the situation when files are written to in controlled way (i.e. we are not concerned with malware running on the box in question and just want to stop it from passing through mailsewer, etc.), then there's no damn need to play with LSM - just have e.g. coda with its commit-on-close and run the scanner on commit. End of story. Mind you, in such setups one would be much better off just having the mail server run the tests explicitly in the userland, along with the rest of anti-spam, etc. filters.
I've repeated the above statements so many times to a number of the anti-virus companies, and other people that really should know better, that I'm really sick of it. For some reason, they keep trying to do things like this in the kernel, despite it being trivial to do in userspace properly.
In the end, I even got one company to agree that it should be done in userspace (McAfee), but they ignored this and went off to update their kernel code again :(
Just because other operating systems require you to do things like this within the kernel, doesn't mean that you have to do the same thing on Linux...
so sad,
greg k-h
Quoted 10 lines · Greg KH
Quoted 9 lines
And if we are talking about the situation when files are written to in controlled way (i.e. we are not concerned with malware running on the box in question and just want to stop it from passing through mailsewer, etc.), then there's no damn need to play with LSM - just have e.g. coda with its commit-on-close and run the scanner on commit. End of story. Mind you, in such setups one would be much better off just having the mail server run the tests explicitly in the userland, along with the rest of anti-spam, etc. filters.
Quoted 6 lines
I've repeated the above statements so many times to a number of the anti-virus companies, and other people that really should know better, that I'm really sick of it. For some reason, they keep trying to do things like this in the kernel, despite it being trivial to do in userspace properly.
Do you mean something along the lines of FUSE?
Quoted 12 lines · Jan Engelhardt
Quoted 10 lines · Greg KH
Quoted 9 lines
And if we are talking about the situation when files are written to in controlled way (i.e. we are not concerned with malware running on the box in question and just want to stop it from passing through mailsewer, etc.), then there's no damn need to play with LSM - just have e.g. coda with its commit-on-close and run the scanner on commit. End of story. Mind you, in such setups one would be much better off just having the mail server run the tests explicitly in the userland, along with the rest of anti-spam, etc. filters.
Quoted 7 lines
Quoted 6 lines
I've repeated the above statements so many times to a number of the anti-virus companies, and other people that really should know better, that I'm really sick of it. For some reason, they keep trying to do things like this in the kernel, despite it being trivial to do in userspace properly.
Do you mean something along the lines of FUSE?
That is one way, but not the simplest or nicest (people don't want to run their whole fs on FUSE just yet).
The easiest way is as Al described above, just have the userspace program that wrote the file to disk, check it then.
There are some nice SAMBA plugins that do just that already out there...
thanks,
greg k-h
Quoted 2 lines · Greg KH
The easiest way is as Al described above, just have the userspace program that wrote the file to disk, check it then.
But the problem is that this isn't just Samba, this is a countless myriad of different applications. And if one of them doesn't support on-access scanning, then the whole solution isn't worth using.
Quoted 1 line
There are some nice SAMBA plugins that do just that already out there...
That's really not the problem :-)
Jon.
Quoted 8 lines · Jon Masters
Quoted 2 lines · Greg KH
The easiest way is as Al described above, just have the userspace program that wrote the file to disk, check it then.
But the problem is that this isn't just Samba, this is a countless myriad of different applications. And if one of them doesn't support on-access scanning, then the whole solution isn't worth using.
Ok, which specific applications do they care about? Last time I asked it was still limited to a very small handful, all of which would be trivial to add such a hook to.
Quoted 3 lines
Quoted 1 line
There are some nice SAMBA plugins that do just that already out there...
That's really not the problem :-)
Yes it is. That's all you want to catch, when a Windows machine wants to access a file on a SAMBA server. Do the check then, in userspace.
Believe me, I've been over and over and over and over this before...
thanks,
greg k-h
Quoted 6 lines · Greg KH
Quoted 5 lines · Jon Masters
Quoted 2 lines · Greg KH
The easiest way is as Al described above, just have the userspace program that wrote the file to disk, check it then.
Quoted 8 lines
Quoted 3 lines
But the problem is that this isn't just Samba, this is a countless myriad of different applications. And if one of them doesn't support on-access scanning, then the whole solution isn't worth using.
Ok, which specific applications do they care about? Last time I asked it was still limited to a very small handful, all of which would be trivial to add such a hook to.
Well, think bash, syscalls. While you can add a plugin to samba "easily", it seems overkill to do the same for rm, mv, cp, bash.
Quoted 10 lines · Jan Engelhardt
Quoted 8 lines
Quoted 3 lines
But the problem is that this isn't just Samba, this is a countless myriad of different applications. And if one of them doesn't support on-access scanning, then the whole solution isn't worth using.
Ok, which specific applications do they care about? Last time I asked it was still limited to a very small handful, all of which would be trivial to add such a hook to.
Well, think bash, syscalls. While you can add a plugin to samba "easily", it seems overkill to do the same for rm, mv, cp, bash.
Can we please stop this useless discussion? Trying to check the content of files to see whether they might be malicious is inherently braindead, and no amounts of plugins in random places will fix this.
Quoted 3 lines
Can we please stop this useless discussion? Trying to check the content of files to see whether they might be malicious is inherently braindead, and no amounts of plugins in random places will fix this.
Actually it is quite effective especially for files whose content is expected not to be executable material - there are some very distinct mathematical signatures.
For the purposes of figuring out what is needed you can consider a random simple user case such as a system which protects you against the works of Eric S Raymond. Replace the mathematical analysis and heuristics with a user space tool which spots the various ESR papers and design it for that if it makes you happier.
SELinux seems to be able to do most of the lifting around the problem as it can relabel a file into eric_t and constrain further access to it.
The big problem to me is that this essentially boils down to revoke() on ordinary files, which as we know is _hard_.
The simple case is
open
write cathedral and bazaar in some order
close
<trap close -> process -> label eric_t>
open (eric_t) - SELinux "no"
Anyone smart will then write it out of order and keep the file open, or make sure someone else has it open. At that point close ceases to be a point you can check the labelling. Open doesn't seem too helpful either as the target may have the file open before it is modified into an ESR work.
Even if you can spot the surrepticious assembly of an ESR work on your computer you then have to implement revocation in order to take the file away from those with it open.
Alan
Quoted 11 lines · Alan Cox
The simple case is
open
write cathedral and bazaar in some order
close
<trap close -> process -> labeleric_t>open (
eric_t) - SELinux "no"Anyone smart will then write it out of order and keep the file open, or
That would assume Eric already has a program running on your system optimized to inject his works in a obfuscated way. And if he has a program running he can do nearly everything already. You already lost the game.
The normal case Tvrtko et.al. are trying to handle would be more the work getting downloaded from somewhere or read from a usb stick using normal programs like web browsers or file managers who don't do any out of order writing tricks and other obfuscation.
Important exception might be things like BitTorrent who write out of order or parallel downloaders to cheat TCP congestion control. Or simply tar+gzip with automatic depacking in desktops. There are probably more and it's probably tricky but it is not a "need to handle arbitary nastiness by a determined attacker" situation.
Anyways I'm not saying that pattern matching is a useful security measure (just the interaction with compression and encryption makes it very dubious), but if you're talking hypothetically you should at least look closely at the hypothetical use cases @)
-Andi
Quoted 18 lines · Andi Kleen
Quoted 11 lines · Alan Cox
The simple case is
open
write cathedral and bazaar in some order
close
<trap close -> process -> labeleric_t>open (
eric_t) - SELinux "no"Anyone smart will then write it out of order and keep the file open, or
That would assume Eric already has a program running on your system optimized to inject his works in a obfuscated way. And if he has a program running he can do nearly everything already. You already lost the game.
Quoted 4 lines
The normal case Tvrtko et.al. are trying to handle would be more the work getting downloaded from somewhere or read from a usb stick using normal programs like web browsers or file managers who don't do any out of order writing tricks and other obfuscation.
Fortunately normal programs tend to be dynamically linked, so LD_PRELOAD is fine to handle them. And we know we can't handle nasty programs. Seems like LD_PRELOAD is the way to go.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 12 lines · Jan Engelhardt
Quoted 10 lines · Greg KH
Quoted 8 lines · Jon Masters
Quoted 2 lines · Greg KH
The easiest way is as Al described above, just have the userspace program that wrote the file to disk, check it then.
But the problem is that this isn't just Samba, this is a countless myriad of different applications. And if one of them doesn't support on-access scanning, then the whole solution isn't worth using.
Quoted 6 lines
Quoted 4 lines
Ok, which specific applications do they care about? Last time I asked it was still limited to a very small handful, all of which would be trivial to add such a hook to.
Well, think bash, syscalls. While you can add a plugin to samba "easily", it seems overkill to do the same for rm, mv, cp, bash.
Again, these are not things that these companies care about.
Please, let's stop arguing about this, until someone actually produces some code they want to see help resolve this "issue".
thanks,
greg k-h
Quoted 13 lines · Greg KH
Quoted 12 lines · Jan Engelhardt
Quoted 10 lines · Greg KH
Quoted 8 lines · Jon Masters
Quoted 2 lines · Greg KH
The easiest way is as Al described above, just have the userspace program that wrote the file to disk, check it then.
But the problem is that this isn't just Samba, this is a countless myriad of different applications. And if one of them doesn't support on-access scanning, then the whole solution isn't worth using.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Ok, which specific applications do they care about? Last time I asked it was still limited to a very small handful, all of which would be trivial to add such a hook to.
Well, think bash, syscalls. While you can add a plugin to samba "easily", it seems overkill to do the same for rm, mv, cp, bash.
Again, these are not things that these companies care about.
Perhaps if you looked at this outside of a file-server scenario, the problem would be clearer? Anti-malware companies want to check anything written to disk on a system, either at write time or blocking the open/mmap. That means proactively protecting email programs with known vulnerabilities that have yet to be patched, web browsers writing and reading their caches, an Apache instance running WebDAV, the list goes on. And these are on desktop systems, with no attached file/network server.
Yes, each and every one of these programs could have a malware scanning engine slapped inside of them. But that proves what? That's like saying each an every program on a system should have the SELinux policies built into them, and yet we have that in-kernel instead.
Quoted 23 lines · Ray Lee
Quoted 13 lines · Greg KH
Quoted 12 lines · Jan Engelhardt
Quoted 10 lines · Greg KH
Quoted 8 lines · Jon Masters
Quoted 2 lines · Greg KH
The easiest way is as Al described above, just have the userspace program that wrote the file to disk, check it then.
But the problem is that this isn't just Samba, this is a countless myriad of different applications. And if one of them doesn't support on-access scanning, then the whole solution isn't worth using.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Ok, which specific applications do they care about? Last time I asked it was still limited to a very small handful, all of which would be trivial to add such a hook to.
Well, think bash, syscalls. While you can add a plugin to samba "easily", it seems overkill to do the same for rm, mv, cp, bash.
Again, these are not things that these companies care about.
Quoted 8 lines
Perhaps if you looked at this outside of a file-server scenario, the problem would be clearer? Anti-malware companies want to check anything written to disk on a system, either at write time or blocking the open/mmap. That means proactively protecting email programs with known vulnerabilities that have yet to be patched, web browsers writing and reading their caches, an Apache instance running WebDAV, the list goes on. And these are on desktop systems, with no attached file/network server.
Ok, if they want to check on every open/mmap then just hook in glibc to do this. Especially as they want to run userspace code at this point in time.
Again, let's wait for some code to show up before discussing this further.
thanks,
greg k-h
Quoted 9 lines · Greg KH
Quoted 8 lines
Perhaps if you looked at this outside of a file-server scenario, the problem would be clearer? Anti-malware companies want to check anything written to disk on a system, either at write time or blocking the open/mmap. That means proactively protecting email programs with known vulnerabilities that have yet to be patched, web browsers writing and reading their caches, an Apache instance running WebDAV, the list goes on. And these are on desktop systems, with no attached file/network server.
Quoted 3 lines
Ok, if they want to check on every open/mmap then just hook in glibc to do this. Especially as they want to run userspace code at this point in time.
Doesn't help statically linked binaries, or anything else that bypases glibc.
But yes, I'll let them argue their point from here.
Ray Lee wrote
Quoted 10 lines · Greg KH
Quoted 9 lines
Quoted 8 lines
Perhaps if you looked at this outside of a file-server scenario, the problem would be clearer? Anti-malware companies want to check anything written to disk on a system, either at write time or blocking the open/mmap. That means proactively protecting email programs with known vulnerabilities that have yet to be patched, web browsers writing and reading their caches, an Apache instance running WebDAV, the list goes on. And these are on desktop systems, with no attached file/network server.
Quoted 5 lines
Quoted 3 lines
Ok, if they want to check on every open/mmap then just hook in glibc to do this. Especially as they want to run userspace code at this point in time.
Doesn't help statically linked binaries, or anything else that bypases glibc.
Or NFS servers for that matter, either.
-justinb
Signature · 3 lines
Justin Banks BakBone Software [email protected]
Quoted 11 lines · Justin Banks
Ray Lee wrote
Quoted 10 lines · Greg KH
Quoted 9 lines
Quoted 8 lines
Perhaps if you looked at this outside of a file-server scenario, the problem would be clearer? Anti-malware companies want to check anything written to disk on a system, either at write time or blocking the open/mmap. That means proactively protecting email programs with known vulnerabilities that have yet to be patched, web browsers writing and reading their caches, an Apache instance running WebDAV, the list goes on. And these are on desktop systems, with no attached file/network server.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Ok, if they want to check on every open/mmap then just hook in glibc to do this. Especially as they want to run userspace code at this point in time.
Doesn't help statically linked binaries, or anything else that bypases glibc.
Or NFS servers for that matter, either.
Right. I threw all of these examples out there and was given the impression that a one-size fits all "solution" is needed. And to be honest, I can understand that desire.
I don't want to fall into any "snake oil" traps, because I know the reality is that there are holes in any solution. The only way to /really/ be sure of file integrity is to mark every user-visible pagecache page read only, trap on every single write, and re-scan the entirety of the file at that point. And even more silliness.
A compromise is therefore called for. I will go shut up and go talk to these guys and see if I can hook up a public discussion around a set of requirements. Preferably some code too, but at least a requirements doc.
Thanks,
Jon.
Quoted 8 lines · Ray Lee
Perhaps if you looked at this outside of a file-server scenario, the problem would be clearer? Anti-malware companies want to check anything written to disk on a system, either at write time or blocking the open/mmap. That means proactively protecting email programs with known vulnerabilities that have yet to be patched, web browsers writing and reading their caches, an Apache instance running WebDAV, the list goes on. And these are on desktop systems, with no attached file/network server.
... and anything promising that is a grade-A snake oil.
Quoted 6 lines · Greg KH
Quoted 5 lines · Jon Masters
Quoted 2 lines · Greg KH
The easiest way is as Al described above, just have the userspace program that wrote the file to disk, check it then.
Quoted 7 lines
Quoted 3 lines
But the problem is that this isn't just Samba, this is a countless myriad of different applications. And if one of them doesn't support on-access scanning, then the whole solution isn't worth using.
Ok, which specific applications do they care about? Last time I asked it was still limited to a very small handful, all of which would be trivial to add such a hook to.
Like I said, I'm trying to put together a set of "feature requirements" that we can publish to LKML and get feedback. I am lead to believe that they basically want to trap every file operation, of every potential userspace program, so I don't think it's a handful at this point.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
There are some nice SAMBA plugins that do just that already out there...
That's really not the problem :-)
Yes it is. That's all you want to catch, when a Windows machine wants to access a file on a SAMBA server. Do the check then, in userspace.
Oh, I meant that if it was just Samba we'd all be home and dry by now.
Quoted 1 line
Believe me, I've been over and over and over and over this before...
Yeah. I know, I'm trying not to flog a dead horse here, but I think there's real genuine interest here this time around. So, maybe I'm just trying because everyone else already did, but it's worth a shot :-)
Jon.
Quoted 12 lines · Jon Masters
Quoted 10 lines · Greg KH
Quoted 8 lines · Jon Masters
Quoted 2 lines · Greg KH
The easiest way is as Al described above, just have the userspace program that wrote the file to disk, check it then.
But the problem is that this isn't just Samba, this is a countless myriad of different applications. And if one of them doesn't support on-access scanning, then the whole solution isn't worth using.
Quoted 6 lines
Quoted 3 lines
Ok, which specific applications do they care about? Last time I asked it was still limited to a very small handful, all of which would be trivial to add such a hook to.
Like I said, I'm trying to put together a set of "feature requirements" that we can publish to LKML and get feedback.
Heh, ok, I did this a while ago, but I'd be interested in seeing this done in public, on lkml this time around so that we have a thread to point people at in the future when it comes up again :)
good luck,
greg k-h
Al Viro <[email protected]> wrote on 28/11/2007 18:30:40:
Quoted 12 lines · On Wed, Nov 28, 2007 at 01:15:05PM -0500, [email protected] wrote:
Quoted 1 line
(Note that the concept has interesting implications in the other
direction as
Quoted 1 line
well - rather than stopping you from reading a file that has
malware, you could
Quoted 1 line
in theory write an anti-export package that would let you write
onto external
Quoted 1 line
memory or outbound e-mail, but prevent the write if it was
corporate-sensitive
Quoted 1 line
data, or whatever.
You _can_ _not_ do that. If shared mapping gets dirtied, you have no
way to
Quoted 1 line
intercept that. At all. Especially since the page stays mapped while
it is
Quoted 7 lines
written out, so the next modification can come when hardware had already started outbound DMA and there's no way to abort it, no matter what your external scanner would do.
Folks, really, that doesn't work. At all. You can intercept all system calls you want and it will not be enough to prevent the "bad" contents from hitting the disk.
On this level we are not talking about stopping bad stuff from hitting the disk. There is a lot simpler scenario how that can happen and that is via a dropper at which point it can be detected and reported when the file is closed. But more importantly further access to it can be blocked until appropriate actions are taken which also applies with your example, no? Is it possible to open for execute and have dirty mappings (or open for write) on a file at the same time?
Quoted 2 lines
And if we are talking about the situation when files are written to in controlled way (i.e. we are not concerned with malware running on the
box
Quoted 1 line
in question and just want to stop it from passing through mailsewer,
etc.),
Quoted 1 line
then there's no damn need to play with LSM - just have e.g. coda with
its
Quoted 2 lines
commit-on-close and run the scanner on commit. End of story. Mind you, in such setups one would be much better off just having the mail server
run
Quoted 1 line
the tests explicitly in the userland, along with the rest of anti-spam,
etc.
Quoted 1 line
filters.
That is true for that scenario.
--
Tvrtko August Ursulin
Senior Software Engineer, Sophos
"Views and opinions expressed in this email are strictly those of the author.
The contents has not been reviewed or approved by Sophos."
Tel: 01235 559933
Web: www.sophos.com
Protecting business against viruses, spyware, spam and policy abuse
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Quoted 2 lines
closed. But more importantly further access to it can be blocked until appropriate actions are taken which also applies with your example, no? Is
That bit is hard- very hard.
Quoted 2 lines
it possible to open for execute and have dirty mappings (or open for write) on a file at the same time?
If I want I can have 16 threads executing code in a shared object being written to by ten other programs at once and shared over a network while we are at it. Its probably not a good idea but I can do it if I have reason to.
Unix and Linux likewise take the basic view that locking is a problem for the user, while file permissions prevent it being made a problem for other users.
Quoted 4 lines · Alan Cox
Quoted 2 lines
closed. But more importantly further access to it can be blocked until appropriate actions are taken which also applies with your example, no? Is
That bit is hard- very hard.
In some sense it seems like the same problem faced by dynamic translators such as Qemu. They really want to vet a dirtied or faulted page before allowing the app to run unhindered. It's be nice to have some way to do that without virtualizing the whole of userspace.
Quoted 10 lines · Ray Lee
Quoted 4 lines · Alan Cox
Quoted 2 lines
closed. But more importantly further access to it can be blocked until appropriate actions are taken which also applies with your example, no? Is
That bit is hard- very hard.
In some sense it seems like the same problem faced by dynamic translators such as Qemu. They really want to vet a dirtied or faulted page before allowing the app to run unhindered. It's be nice to have some way to do that without virtualizing the whole of userspace.
Like I hinted at, you can't just "vet a page". Because a page alone is meaningless garbage, unless it happens to be an extremely small program, with headers, all nicely aligned. Most likely you don't know if a random page of data is code from a COFF file, ELF file, or some random crap I typed in at a terminal after having too much coffee.
So. You'd need to scan *all the pages* of *the entire file*, every time that you performed any type of operation. Hence, that idea is completely out right away. The on-access scanning isn't perfect, but it's probably about as good as you can get and still have a reasonably usable system.
Jon.
Quoted 12 lines · Jon Masters
Quoted 10 lines · Ray Lee
Quoted 4 lines · Alan Cox
Quoted 2 lines
closed. But more importantly further access to it can be blocked until appropriate actions are taken which also applies with your example, no? Is
That bit is hard- very hard.
In some sense it seems like the same problem faced by dynamic translators such as Qemu. They really want to vet a dirtied or faulted page before allowing the app to run unhindered. It's be nice to have some way to do that without virtualizing the whole of userspace.
Quoted 8 lines
Like I hinted at, you can't just "vet a page". Because a page alone is meaningless garbage, unless it happens to be an extremely small program, with headers, all nicely aligned. Most likely you don't know if a random page of data is code from a COFF file, ELF file, or some random crap I typed in at a terminal after having too much coffee.
So. You'd need to scan *all the pages* of *the entire file*, every time that you performed any type of operation.
*blink* Really? To lift Alan's example, a naive first implementation would be to create a suffix tree of all of ESR's works, then scan each page on fault to see if there are any partial matches in the tree. If there's a partial match that ends at the page boundary, mark the page as questionable, but let execution/reading continue. On the next linear page, if the match finishes, mark that one as bad, and disallow access. That wouldn't be very expensive (in terms of scanning effort).
But I'm by no means an anti-malware guy, so perhaps I'm spouting crap, especially given that I've thought about this a total of five minutes.
Regardless, thanks for doing the coordination work with them, and for interfacing them with the kernel community. I'm really going to shut up now.
Quoted 19 lines · Ray Lee
Quoted 18 lines · Jon Masters
Quoted 10 lines · Ray Lee
Quoted 4 lines · Alan Cox
Quoted 2 lines
closed. But more importantly further access to it can be blocked until appropriate actions are taken which also applies with your example, no? Is
That bit is hard- very hard.
In some sense it seems like the same problem faced by dynamic translators such as Qemu. They really want to vet a dirtied or faulted page before allowing the app to run unhindered. It's be nice to have some way to do that without virtualizing the whole of userspace.
Like I hinted at, you can't just "vet a page". Because a page alone is meaningless garbage, unless it happens to be an extremely small program, with headers, all nicely aligned. Most likely you don't know if a random page of data is code from a COFF file, ELF file, or some random crap I typed in at a terminal after having too much coffee.
Quoted 4 lines
Quoted 2 lines
So. You'd need to scan *all the pages* of *the entire file*, every time that you performed any type of operation.
*blink* Really?
Yeah, really.
Quoted 3 lines
To lift Alan's example, a naive first implementation
would be to create a suffix tree of all of ESR's works, then scan each page on fault to see if there are any partial matches in the tree.
Ah, but I could write a sequence of pages that on their own looked garbage, but in reality, when executed would print out a copy of the Jargon File in all its glory. And if you still think you could look for patterns, how about executable code that self-modifies in random ways but when executed as a whole actually has the functionality of fetchmail embedded within it? How would you guard against that?
Jon.
Quoted 6 lines · Jon Masters
Ah, but I could write a sequence of pages that on their own looked garbage, but in reality, when executed would print out a copy of the Jargon File in all its glory. And if you still think you could look for patterns, how about executable code that self-modifies in random ways but when executed as a whole actually has the functionality of fetchmail embedded within it? How would you guard against that?
So, just because Fred Cohen showed in his PhD thesis that *perfect* virus/malware scanning is equivalent to the Turing Halting Problem, we should abandon efforts to make a 99.9998% workable system?
Yes, most of these schemes *can* be bypassed because some malicious code does a mmap() or similar trick. But what is being overlooked here is that in most cases, what is *desired* is a way to filter things being handled by *non* malicious code. Yeah, sure, a shar archive can contain a binary that does evil things - but if we stop /bin/cp from copying the file that has the evil in it, it's a non-issue.
Let's get real here guys - trying to do *absolutely perfect* security is pointless. You want to do security that reduces your *total* cost - and in most cases this means "pretty good security" that stops "almost all issues".
As Linus reminds us once in a while - the perfect is the enemy of the good. In this case, we don't *need* to be perfect - we only need to be noticably better than another well-known operating system that isn't even very good at it.
Quoted 6 lines · On Thu, Nov 29, 2007 at 03:56:28PM -0500, [email protected] wrote:
Yes, most of these schemes *can* be bypassed because some malicious code does a
mmap()or similar trick. But what is being overlooked here is that in most cases, what is *desired* is a way to filter things being handled by *non* malicious code. Yeah, sure, a shar archive can contain a binary that does evil things - but if we stop/bin/cpfrom copying the file that has the evil in it, it's a non-issue.
Then all you need is scan-on-commit with commit-on-close. Which requires no changes of anything that would run in kernel mode and no bothering with LSM whatsoever.
Incidentally, I would really love to see the threat profile we are talking about. I have some impressions in that area, but I would rather keep them to myself for now - I really want to see the answers.
Quoted 2 lines · Al Viro
Incidentally, I would really love to see the threat profile we are talking about.
Exactly.
Please come up with a set of requirements that can be reviewed by the core kernel folk, and perhaps then focus on how to meet those requirements once they have been accepted.
- James
Signature · 2 lines
James Morris <[email protected]>
Quoted 11 lines · On Thu, 2007-11-29 at 15:56 -0500, [email protected] wrote:
Quoted 6 lines · Jon Masters
Ah, but I could write a sequence of pages that on their own looked garbage, but in reality, when executed would print out a copy of the Jargon File in all its glory. And if you still think you could look for patterns, how about executable code that self-modifies in random ways but when executed as a whole actually has the functionality of fetchmail embedded within it? How would you guard against that?
So, just because Fred Cohen showed in his PhD thesis that *perfect* virus/malware scanning is equivalent to the Turing Halting Problem, we should abandon efforts to make a 99.9998% workable system?
I think you misread what I said. I implied the exact opposite :-)
I'm trying to show that I understand the problem by saying the above, that doing this perfectly is impossible, but I also happen to believe that there are those who have solutions that provide a level of protection to their users, who ask for such things. Hence my point is that it's not really our place to debate whether virus scanning is good/bad but more how to provide a sane API. I'll get a spec.
Jon.
Quoted 4 lines
Jargon File in all its glory. And if you still think you could look for patterns, how about executable code that self-modifies in random ways but when executed as a whole actually has the functionality of fetchmail embedded within it? How would you guard against that?
Thats a problem for whoever writes the ESR detection tool and to what level it works. The question for the kernel is how do we provide a mechanism to allow (to some extent at least) this kind of tool to run.
Alan
Alan Cox wrote
Quoted 8 lines
Quoted 4 lines
Jargon File in all its glory. And if you still think you could look for patterns, how about executable code that self-modifies in random ways but when executed as a whole actually has the functionality of fetchmail embedded within it? How would you guard against that?
Thats a problem for whoever writes the ESR detection tool and to what level it works. The question for the kernel is how do we provide a mechanism to allow (to some extent at least) this kind of tool to run.
In a general way, what about providing a kind of {file,inode}_operations xchange functionality given a super_block such that filesystems could essentially be stacked, e.g. a
int op_xchange(struct super_block *sb, struct inode_operations *new_iop,
struct inode_operations *old_iop, struct file_operations *new_fop, struct file_operations *old_fop);
The caller could/would/should be vetted, but replacing the static ops with new ones of the caller's specification would allow them to intercept the i/o, do whatever checking they wanted to do, and then call the proper underlying operation (even modifying the parameters, I guess, but that may be too weird). Perhaps separate get_* and set_* so that the caller can retrieve the current operations vector and only "override" the functions they really care about?
It's not perfect, but as was recently pointed out, if you can only get 98% of the way there rather than 100% is that a reason for not trying to make it possible? It'd obviously not work for mmap, but as near as I can tell the whole point is to get the "normal" malware cases without having to do icky things like mangle the syscall table.
-justinb
Signature · 3 lines
Justin Banks BakBone Software [email protected]
Quoted 3 lines · Justin Banks
It's not perfect, but as was recently pointed out, if you can only get 98% of the way there rather than 100% is that a reason for not trying to make it possible?
BTW, that's a fine example of a common fallacy: "$FOO is 98% of the way to $TARGET" does not allow to interpolate the properties of $TARGET to those of $FOO.
Telling that a condom is a 98% approximation to platonic ideal of such is not particulary useful, especially if it turns out that what this number really means is that there's a hole on its tip covering 2% of surface...
Al Viro wrote
Quoted 6 lines · Justin Banks
Quoted 4 lines
It's not perfect, but as was recently pointed out, if you can only get 98% of the way there rather than 100% is that a reason for not trying to make it possible?
Quoted 7 lines
BTW, that's a fine example of a common fallacy: "$FOO is 98% of the way to $TARGET" does not allow to interpolate the properties of $TARGET to those of $FOO.
Telling that a condom is a 98% approximation to platonic ideal of such is not particulary useful, especially if it turns out that what this number really means is that there's a hole on its tip covering 2% of surface...
Understood, but for this use case (and indexing, probably replication, IDS, and others) hooking at the "normal" VFS layer would provide enough flexibility to do what's needed. I'm not saying it should be done, only that it'd work, provide more use cases for linux, and not be totally unsupportable.
-justinb
Signature · 3 lines
Justin Banks BakBone Software [email protected]
Quoted 8 lines · Alan Cox
Quoted 4 lines
Jargon File in all its glory. And if you still think you could look for patterns, how about executable code that self-modifies in random ways but when executed as a whole actually has the functionality of fetchmail embedded within it? How would you guard against that?
Thats a problem for whoever writes the ESR detection tool and to what level it works. The question for the kernel is how do we provide a mechanism to allow (to some extent at least) this kind of tool to run.
Right. I'm just saying reading a single page out of context (no pun intended) is not going to be very useful. They need to scan the entire file, which means that there are limited ways this is practical - it's not practical to do that on every write into a shared mapping, hence a solution that scans on open, etc. is probably the best there is.
(I know you know this)
Jon.
Quoted 7 lines · Jon Masters
Quoted 5 lines · Alan Cox
Quoted 4 lines
Jargon File in all its glory. And if you still think you could look for patterns, how about executable code that self-modifies in random ways but when executed as a whole actually has the functionality of fetchmail embedded within it? How would you guard against that?
Quoted 6 lines
Quoted 3 lines
Thats a problem for whoever writes the ESR detection tool and to what level it works. The question for the kernel is how do we provide a mechanism to allow (to some extent at least) this kind of tool to run.
Right. I'm just saying reading a single page out of context (no pun intended) is not going to be very useful.
Fortunately for all concerned, although Alan's self-modifying code is indeed a possibility, it's much less of an issue than the sort of malware that can be found with a simple "find this 27-byte sequence, which will be found in either block 36 or 37 of the file".
And I'll make the prediction that we won't see anything doing the sorts of things that Alan's program does, until that's the *easiest* way to get into a system. Until that time, they're either going to be sending simpler stuff that a scanner can easily template and find, or using other means of attacks that are outside the scope of a scanner.
Remember guys - we want to think about *realistic* threat models. The e-mail virus scanners we use catch hundreds to thousands of known viruses *every day*. But I can count on the fingers of both hands the number of times I've had to deal with a *real* "0-day" in a quarter century. The scanner doesn't have to be perfect - it just has to make it hard enough to bypass to render it economically infeasible. If you're targeted by a military/govt/political/ religious group that doesn't *care* if it's economically viable, you have other, bigger problems to deal with...
Quoted 4 lines
Fortunately for all concerned, although Alan's self-modifying code is indeed a possibility, it's much less of an issue than the sort of malware that can be found with a simple "find this 27-byte sequence, which will be found in either block 36 or 37 of the file"
Thats a very old model of detection ;)
There is another usage case for this stuff as well - which is on the fly indexing of material. An indexer has the same basic needs as a virus checker (without the blocking aspect) and doesn't have to be robust against sneaky material.
Alan
Quoted 4 lines · Alan Cox
If I want I can have 16 threads executing code in a shared object being written to by ten other programs at once and shared over a network while we are at it. Its probably not a good idea but I can do it if I have reason to.
Actually the kernel prevents writing to files which are executed currently and you can also not execute what is currently open for writing. That is what VM_DENYWRITE is good for.
Does not work for shared libraries though -- you cannot set VM_DENYWRITE in user mmap aka ld.so Man page says
MAP_DENYWRITE
This flag is ignored. (Long ago, it signalled that attempts to write to the underlying file should fail with ETXTBUSY. But this was a source of denial-of-service attacks.)
-Andi
Quoted 3 lines · Tvrtko A. Ursulin
We here at Sophos (the fourth largest endpoint security vendor in the world) have such a module called Talpa which is a part of our main endpoint security product
What is a "endpoint security product" exactly? A gateway that scans files passing through it?
Quoted 2 lines
In essence, what our module does is it intercepts file accesses and allows userspace daemons to vet them.
I suspect the only good way forward would be to collect the requirements of your user space and then define a proper official interface to do what they want to do. Ideally would be a shared proposal from multiple groups who are doing this, e.g. if you could talk to some others doing similar things that would be best.
Personally I admit I never quite saw the point of intercepting all file accesses for everything. That will just always be slow as often demonstrated on other operating systems and racey and unreliable too. And at least the internal daemons should be already reasonably well protected by standard (or beyond standard, like AA/SELinux etc.) security measures, so e.g. it does not really make sense to intercept all of your /etc file accesses and similar.
It might be better to identify the services (gateway, samba, file server whatever) that are actually dealing with possible infected "external" files and then define some generic interface that would allow you to check those as the data appears.
I would expect such an approach to perform better in the end and be more reliable too.
Note such a interface might well be user space only.
-Andi
Quoted 4 lines
It might be better to identify the services (gateway, samba, file server whatever) that are actually dealing with possible infected "external" files and then define some generic interface that would allow you to check those as the data appears.
I am wondering if the right interface is actually more related to the existing audit interfaces ?
Quoted 7 lines · Alan Cox
Quoted 4 lines
It might be better to identify the services (gateway, samba, file server whatever) that are actually dealing with possible infected "external" files and then define some generic interface that would allow you to check those as the data appears.
I am wondering if the right interface is actually more related to the existing audit interfaces ?
The problem there is that the audit interface just *records* - it doesn't have the ability to say "No, I don't *think* so.." that the LSM interface has.
[email protected] wrote on 28/11/2007 19:20:26:
Quoted 10 lines
Quoted 1 line · Tvrtko A. Ursulin
We here at Sophos (the fourth largest endpoint security vendor in
the world)
Quoted 1 line
have such a module called Talpa which is a part of our main
endpoint security
Quoted 1 line
product
What is a "endpoint security product" exactly? A gateway that scans files passing through it?
No, it would be more like a desktop/workstation thingy which protects against different kinds of malware and can also enforce policies, all centraly managed. I am afraid to use to more marketing words not to upset the LKML filters because originaly my posdt was blocked on.. hah! I almost wrote it now! :) It was "security vendor" but with another term for vendor.
Quoted 1 line
Quoted 1 line
In essence, what our module does is it intercepts file accesses and
allows
Quoted 7 lines
Quoted 1 line
userspace daemons to vet them.
I suspect the only good way forward would be to collect the requirements of your user space and then define a proper official interface to do what they want to do. Ideally would be a shared proposal from multiple groups who are doing this, e.g. if you could talk to some others doing similar things that would be best.
I completely agree and effort to do that is under way. I am not the one coordinating that but hopefully those who are will post something soon.
Quoted 7 lines
Personally I admit I never quite saw the point of intercepting all file accesses for everything. That will just always be slow as often demonstrated on other operating systems and racey and unreliable too. And at least the internal daemons should be already reasonably well protected by standard (or beyond standard, like AA/SELinux etc.) security measures, so e.g. it does not really make sense to intercept all of your /etc file accesses and similar.
Personally I don't know. I am often surprised when I hear what techniques the bad guys use and what is possible.
Solution that we currently have is not so slow (btw you are free to try it out). And by working on a proper interface race and reliability issues should be solvable.
Quoted 9 lines
It might be better to identify the services (gateway, samba, file server whatever) that are actually dealing with possible infected "external" files and then define some generic interface that would allow you to check those as the data appears.
I would expect such an approach to perform better in the end and be more reliable too.
Note such a interface might well be user space only.
That is fine for some classes of servers. But for some other and desktop more could be needed.
--
Tvrtko August Ursulin
Senior Software Engineer, Sophos
"Views and opinions expressed in this email are strictly those of the author.
The contents has not been reviewed or approved by Sophos."
Tel: 01235 559933
Web: www.sophos.com
Protecting business against viruses, spyware, spam and policy abuse
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.
Hi!
Quoted 8 lines
Quoted 7 lines
Personally I admit I never quite saw the point of intercepting all file accesses for everything. That will just always be slow as often demonstrated on other operating systems and racey and unreliable too. And at least the internal daemons should be already reasonably well protected by standard (or beyond standard, like AA/SELinux etc.) security measures, so e.g. it does not really make sense to intercept all of your /etc file accesses and similar.
Quoted 6 lines
Personally I don't know. I am often surprised when I hear what techniques the bad guys use and what is possible.
Solution that we currently have is not so slow (btw you are free to try it out). And by working on a proper interface race and reliability issues should be solvable.
Really?
So what you are trying to do is 'application may never read bad sequence of bits from disk', right?
Now, how do you propose to solve mmap(MAP_SHARED)? The app on the other cpu may see the bad bits before kernel has chance to see them.
Quoted 5 lines
Quoted 5 lines
It might be better to identify the services (gateway, samba, file server whatever) that are actually dealing with possible infected "external" files and then define some generic interface that would allow you to check those as the data appears.
Quoted 7 lines
Quoted 4 lines
I would expect such an approach to perform better in the end and be more reliable too.
Note such a interface might well be user space only.
That is fine for some classes of servers. But for some other and desktop more could be needed.
Andi's solution seems the only one that has chance to work _reliably_.
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 2 lines · Pavel Machek
So what you are trying to do is 'application may never read bad sequence of bits from disk', right?
No, in many of the use cases, we're trying to do "if application reads certain specified sequences of bits from disk we know about it", which is subtly different. Often, *absolute* prevention isn't required, as long as we can generate audit trails and/or alerts...
Quoted 2 lines
Now, how do you propose to solve
mmap(MAP_SHARED)? The app on the other cpu may see the bad bits before kernel has chance to see them.
For many usage cases (such as virus scanners), mmap() isn't really an issue, because if another process is *already* trying to mmap() the file before it's even finished downloading from the network interface, you have other problems. In that case, what you *really* want to be hooking is read() from a network socket, not a disk file - and I don't think you can mmap() a socket anyhow, right?
Similarly, if you're using these exit points to do *indexing*, you really don't care if it got mmap()'ed - at worst, it isn't indexed until the other CPU gets around to it.
Quoted 9 lines · On Sun, 02 Dec 2007 14:44:48 -0500 [email protected] wrote:
Quoted 2 lines · Pavel Machek
So what you are trying to do is 'application may never read bad sequence of bits from disk', right?
No, in many of the use cases, we're trying to do "if application reads certain specified sequences of bits from disk we know about it", which is subtly different. Often, *absolute* prevention isn't required, as long as we can generate audit trails and/or alerts...
.. which breaks down if/when glibc uses mmap() to implement the fopen/fread etc interface
(note: it already does, just not quite yet by default)
Signature · 3 lines
If you want to reach me at my work email, use [email protected] For development, discussion and tips for power savings, visit http://www.lesswatts.org
Quoted 2 lines
and I don't think you can
mmap()a socket anyhow,
right?
You can mmap packet sockets.
-Andi
Hi!
Quoted 8 lines
Quoted 2 lines
So what you are trying to do is 'application may never read bad sequence of bits from disk', right?
No, in many of the use cases, we're trying to do "if application reads certain specified sequences of bits from disk we know about it", which is subtly different. Often, *absolute* prevention isn't required, as long as we can generate audit trails and/or alerts...
Quoted 7 lines
Quoted 2 lines
Now, how do you propose to solve
mmap(MAP_SHARED)? The app on the other cpu may see the bad bits before kernel has chance to see them.For many usage cases (such as virus scanners),
mmap()isn't really an issue, because if another process is *already* trying tommap()the file before it's even finished downloading from the network interface, you have other problems.
Well, if you only want to detect viruses _sometimes_, you can just LD_PRELOAD your scanner.
I guess the A/V people should describe what they are trying to do, as in
"forbidden sequences of bits should never hit disk" or "forbidden sequences of bits should be never read from disk" or something...
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 2 lines · Pavel Machek
Well, if you only want to detect viruses _sometimes_, you can just
LD_PRELOADyour scanner.
And for some use cases, that probably *is* the best answer..
Quoted 5 lines
I guess the A/V people should describe what they are trying to do, as in
"forbidden sequences of bits should never hit disk" or "forbidden sequences of bits should be never read from disk" or something...
We probably want to hear related usages as well - what *besides* A/V would be interested? Indexing services? Software that tries to limit the distribution of sensitive info off the machine - for instance, imagine a rule that said "Data that comes from a file that contains SSNs or the string 'Corporate Secret' data isn't allowed to leave the computer" and a Perl-like 'taint' concept. I'm not saying its *doable*, but it's certainly a goal that somebody would like to achieve...
Quoted 5 lines · On Sun 2007-12-02 16:09:55, [email protected] wrote:
Quoted 2 lines · Pavel Machek
Well, if you only want to detect viruses _sometimes_, you can just
LD_PRELOADyour scanner.And for some use cases, that probably *is* the best answer..
I'd say so.
Quoted 8 lines
Quoted 5 lines
I guess the A/V people should describe what they are trying to do, as in
"forbidden sequences of bits should never hit disk" or "forbidden sequences of bits should be never read from disk" or something...
We probably want to hear related usages as well - what *besides* A/V would be interested? Indexing services?
Well... I'd really like to know what A/V people are trying to do.
Indexing services are really different, and doable with recursive m-time Jan is preparing...
Quoted 7 lines
Software that tries to limit the
distribution
of sensitive info off the machine - for instance, imagine a rule that said "Data that comes from a file that contains SSNs or the string 'Corporate Secret' data isn't allowed to leave the computer" and a Perl-like 'taint' concept. I'm not saying its *doable*, but it's certainly a goal that somebody would like to achieve...
That sounds like a task for SELinux, no?
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 4 lines · Pavel Machek
Quoted 3 lines
We probably want to hear related usages as well - what *besides* A/V would be interested? Indexing services?
Indexing services would probably benefit much more from a recursive-aware inotify, though that has its own sort of problems to solve first.
Quoted 5 lines
Well... I'd really like to know what A/V people are trying to do.
Indexing services are really different, and doable with recursive m-time Jan is preparing...
m-time <=> modification time?
What am I preparing?
I am actually on a freeze, because I really do not know what to make of the situation with the static LSM interface.
There is a grave problem with chaining, because you cannot specify the activation order of one or more LSMs with compiled-in code!
Some kernel Makefiles even contain hints "this depends on link order" (e.g. net/ipv6/netfilter/Makefile) - and I bet for sure that this will also be the case for LSM. No thanks.
While we are at it, consider the hypothethical case of a production server, and the boss tells you to switch to $ThatLSM, with no downtime. After all, it worked when $Company switched to $ThisLSM with Linux 2.6.x ∀ x<24.
Hi!
Quoted 6 lines
Quoted 5 lines
Well... I'd really like to know what A/V people are trying to do.
Indexing services are really different, and doable with recursive m-time Jan is preparing...
m-time <=> modification time?
Yep.
Quoted 1 line
What am I preparing?
Not you, Jan Kara. Sorry.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 16 lines · Tvrtko A. Ursulin
Hi Linus, all,
During one recent LKML discussion
(http://marc.info/?l=linux-kernel&m=119267398722085&w=2) about LSM going static you called for LSM users to speak up.We here at Sophos (the fourth largest endpoint security vendor in the world) have such a module called Talpa which is a part of our main endpoint security product for Linux that protects from viruses and malware hosted on Linux, including those targetting Windows or other connected devices,
(http://www.sophos.com/products/enterprise/endpoint/security-and-control/linux/index.html)
which is GPL code and has been in the field for almost three years now. It's source code has been shipping with the product from the start. We also have a SourceForge project at http://sourceforge.net/projects/talpa/ to host it.
Quoted 7 lines
In essence, what our module does is it intercepts file accesses and allows userspace daemons to vet them. One of the means we implemented that is through LSM and although it is not a perfect match for such use we prefer to use an official interface. Unfortunately, with time it became impossible to use LSM on some distributions (SELinux) so we had to implement other intercept methods which are significantly less nice, and which may also become unworkable over time.
Do you have a patch that shows the type of interface you would like to see? Like James stated, if you do not participate in the development process, we have no way of knowing what you even want from the kernel.
What has kept you from submitting your code for inclusion in the main kernel source tree?
Right now, your customers void their support warranties if they run your software, as it can not be supported by the distros as an out-of-tree kernel module. I'm sure your customers would like to not have this problem.
thanks,
greg k-h