Sabotaged PaXtest (was: Re: Patch 4/6 randomize the stack pointer)
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi!
Quoted 4 lines
I'm not entirely happy yet (it shows a bug in mmap randomisation) but it's way better than what you get in your tests (this is the desabotaged
0.9.6 version fwiw)
As you may or may not know, I am the author of PaXtest. Please tell me what a ``desabotaged'' version of PaXtest exactly is. I've never seen a ``sabotaged'' PaXtest and I'm interested in finding out who sabotaged it and for what purpose.
Come to think of it, sabotaging a test program seems to be a rather stupid concept. I mean, if you don't like the results, then don't run it. For me the integrity of PaXtest is very important. I mean, who would trust a test-suite if it produces manipulated results? Right, noone! Why would I want to write a test-suite noone uses? Right, I wouldn't, it is a waste of time! It is boring to write a test-suite. So it better not be a waste of time! I'm sincerely flattered by the fact that someone as famous as you uses PaXtest.
Well, PaXtest was designed to dig up hard facts about how well a system protects process integrity. So, if someone sabotages PaXtest and releases a changed version, that means that person clearly has the opposite intention, which is producing false information.
The whole concept of sabotaging PaXtest and rigging results doesn't make sense to me. People who do that must not be happy with the results and gain something by rigging them. But to think that it would go unnoticed...? First of all, if you don't like the results, don't run it! Second, run kiddie mode if you want to use PaXtest to feel warm and cozy. Third, the code is out there, anyone can download, compile, run and dissect it. Fourth, it is only a few lines of code per test. In other words, it is trivial to understand for anyone who is worth his salt. Therefore anyone who would want to sabotage it, can easily be publicly exposed and dealt with accordingly. Only stupid people would not be able to figure this out right away or be stupid enough to do it anyway. I honestly can't think of anyone that stupid... But then, you never know, I've been surprised before.
Anyways, I know there are some problems with PaXtest on Fedora, caused by PT_GNU_STACK stuff AFAIK. Unfortunately I don't have access to a Fedora machine and therefore am not able to fix and test it. But I would love to have PaXtest working on Fedora. So if you have it working on Fedora, feel free to send in a patch. There are probably more people who use Fedora who want to check their system's security, so you would really help those people.
I'm sorry to have to bother the people on this list, as you have much more important things to do. But for me personally, the integrity of PaXtest (and related to that, my personal integrity) matters a great deal. So I'd like to get to the bottom of this, even if that means bothering lkml. I hope Arjan can provide facts soon, so I can take action against this sabotage. If anyone else on this mailing list has information on this sabotaged PaXtest matter, then please speak up.
As a side note, I am really glad that this code goes into the kernel. It is good to finally see some security being added to the Linux kernel. Big thumbs up for the good work!
Groetjes,
Peter.
Quoted 12 lines · Peter Busser
Hi!
Quoted 4 lines
I'm not entirely happy yet (it shows a bug in mmap randomisation) but it's way better than what you get in your tests (this is the desabotaged
0.9.6 version fwiw)As you may or may not know, I am the author of PaXtest. Please tell me what a ``desabotaged'' version of PaXtest exactly is. I've never seen a ``sabotaged'' PaXtest and I'm interested in finding out who sabotaged it and for what purpose.
Quoted 7 lines
Come to thin
I'm sorry to have to bother the people on this list, as you have much more important things to do. But for me personally, the integrity of PaXtest (and related to that, my personal integrity) matters a great deal. So I'd like to get to the bottom of this, even if that means bothering lkml. I hope Arjan can provide facts soon, so I can take action against this sabotage.
ok the paxtest 0.9.5 I downloaded from a security site (not yours) had this gem in:
body.c 2005-01-18 17:30:11.000000000 +0100 renamed from body.c +1 -0
@@ -29,7 +29,6 @@ fflush( stdout ); if( fork() == 0 ) { + do_mprotect((unsigned long)argv & ~4095U, 4096,
PROT_READ|PROT_WRITE|PROT_EXEC);
doit();
} else {
wait( &status );which is clearly there to sabotage any segmentation based approach (eg execshield and openwall etc); it cannot have any other possible use or meaning.
the paxtest 0.9.6 that John Moser mailed to this list had this gem in it:
@@ -39,8 +42,6 @@
*/
int paxtest_mode = 1;+ /* Dummy nested function */
+ void dummy(void) {}
mode = getenv( "PAXTEST_MODE" );
if( mode == NULL ) {which is clearly there with the only possible function of sabotaging the automatic PT_GNU_STACK setting by the toolchain (which btw is not fedora specific but happens by all new enough (3.3 or later) gcc compilers on all distros) since that requires an executable stack.
Now I know you're a honest and well meaning guy and didn't put those sabotages in, and I did indeed not get paxtests from your site directly, so they obviously must have been tampered with, hence me calling it de- sabotaging when I fixed this issue (by moving the function to not be nested).
Greetings,
Arjan van de Ven
Quoted 9 lines · you
Quoted 6 lines · Peter Busser
Hi!
Quoted 4 lines
I'm not entirely happy yet (it shows a bug in mmap randomisation) but it's way better than what you get in your tests (this is the desabotaged
0.9.6 version fwiw)ok the paxtest 0.9.5 I downloaded from a security site (not yours) had this gem in:
So what does 0.9.5 have to do with 0.9.6?
Quoted 12 lines
body.c 2005-01-18 17:30:11.000000000 +0100 renamed from body.c +1
-0@@ -29,7 +29,6 @@ fflush( stdout ); if( fork() == 0 ) { + do_mprotect((unsigned long)argv & ~4095U, 4096, PROT_READ|PROT_WRITE|PROT_EXEC); doit(); } else { wait( &status );
Quoted 3 lines
which is clearly there to sabotage any segmentation based approach (eg execshield and openwall etc); it cannot have any other possible use or meaning.
Ah, so you are saying that I sabotaged PaXtest? Sorry to burst your bubble, but the PaXtest tests are no real attacks. They are *simulated* attacks. The do_mprotect() is there to *simulate* behaviour people found in GLIBC under certain circumstances. In other words: This is how certain applications behave when run on exec-shield. They complained that PaXtest showed inaccurate results on exec-shield. Since the purpose of PaXtest is to show accurate results, the lack thereof has been fixed.
Quoted 13 lines
the paxtest 0.9.6 that John Moser mailed to this list had this gem in it:
body.c 2005-01-18 17:30:11.000000000 +0100 renamed from body.c +2
-0@@ -39,8 +42,6 @@ */ int paxtest_mode = 1; + /* Dummy nested function */ + void dummy(void) {} mode = getenv( "PAXTEST_MODE" ); if( mode == NULL ) {
Quoted 4 lines
which is clearly there with the only possible function of sabotaging the automatic
PT_GNU_STACKsetting by the toolchain (which btw is not fedora specific but happens by all new enough (3.3 or later) gcc compilers on all distros) since that requires an executable stack.
Again, this is a *simulation* of the way real-life applications could interact with the underlying system. Again people complained that the results shown were not accurate. And that has been fixed.
I am well aware of complaints by some people about this behaviour. That is why there is a separated kiddie and blackhat mode in the latest PaXtest version. The kiddie mode is for those people who prefer to feel warm and cozy and the blackhat mode is for those who want to find out what the worst-case behaviour is. So if you don't like the blackhat results, don't run that test!
Quoted 5 lines
Now I know you're a honest and well meaning guy and didn't put those sabotages in, and I did indeed not get paxtests from your site directly, so they obviously must have been tampered with, hence me calling it de- sabotaging when I fixed this issue (by moving the function to not be nested).
No, these things are also in the officially released sources. I put them in myself in fact.
Interesting. So you are saying that even though applications sometimes use mprotect(), either directly or indirectly through GLIBC (such as multithreaded applications), and there are applications in the wild which use nested functions, that PaXtest should not use these to simulate those kinds of applications. Well, that is an opinion. A strange opinion. Does that mean you ``desabotage'' all other applications on this planet as well? Or is PaXtest perhaps special, because it tells people what really goes on?
My opinion is that PaXtest is correctly showing the consequences of the design decisions you and other people who hacked exec-shield made. These consequences are nowhere documented. So PaXtest is the only source for people to find out about them. It seems to me that you would rather not have people find out about it. It looks like you would rather prefer to cowardly kill the messenger than to stand up for the design decisions you made, like a real man would.
If that is the kind of person who you really are, then you have just managed to disappoint me. If you'd knew me, you'd know that that is quite an accomplishment. And frankly speaking, I don't think I would be happy with this if I were your employer. Since this kind of negative behaviour also reflects on the company you represent.
Anyways, I now know enough about this and I'm done wasting my time on this childish discussion. If you, or anyone else for that matter, has complaints about PaXtest or patches to improve it, please let me know. Otherwise I suggest to leave me alone and to try to pick someone half your size to bully around next time.
Groetjes,
Peter.
Quoted 19 lines · Peter Busser
Quoted 19 lines
ok the paxtest 0.9.5 I downloaded from a security site (not yours) had this gem in:
+
do_mprotect((unsigned long)argv & ~4095U, 4096, PROT_READ|PROT_WRITE|PROT_EXEC);which is clearly there to sabotage any segmentation based approach (eg execshield and openwall etc); it cannot have any other possible use or meaning.
the paxtest 0.9.6 that John Moser mailed to this list had this gem in it:
(quoted diff) +2
-0+ /* Dummy nested function */ + void dummy(void) {}which is clearly there with the only possible function of sabotaging the automatic
PT_GNU_STACKsetting by the toolchain (which btw is not fedora specific but happens by all new enough (3.3 or later) gcc compilers on all distros) since that requires an executable stack.
[...]
Quoted 2 lines
No, these things are also in the officially released sources. I put them in myself in fact.
*PLONK*
Ingo
Quoted 22 lines · you
Quoted 13 lines · Peter Busser
Quoted 13 lines
ok the paxtest 0.9.5 I downloaded from a security site (not yours) had this gem in:
+ do_mprotect((unsigned long)argv & ~4095U, 4096,
PROT_READ|PROT_WRITE|PROT_EXEC);which is clearly there to sabotage any segmentation based approach (eg execshield and openwall etc); it cannot have any other possible use or meaning.
the paxtest 0.9.6 that John Moser mailed to this list had this gem in it:
Quoted 7 lines
Quoted 7 lines
(quoted diff) +2
-0+ /* Dummy nested function */ + void dummy(void) {}which is clearly there with the only possible function of sabotaging the automatic
PT_GNU_STACKsetting by the toolchain (which btw is not fedora specific but happens by all new enough (3.3 or later) gcc compilers on all distros) since that requires an executable stack.
Quoted 6 lines
[...]
Quoted 2 lines
No, these things are also in the officially released sources. I put them in myself in fact.
*PLONK*
You still don't get it, do you?
Groetjes,
Peter.
Quoted 11 lines · El Tue, 1 Feb 2005 10:44:39 +0100 Peter Busser <[email protected]> escribió:
Quoted 3 lines
which is clearly there to sabotage any segmentation based approach (eg execshield and openwall etc); it cannot have any other possible use or meaning.
Ah, so you are saying that I sabotaged PaXtest? Sorry to burst your bubble, but the PaXtest tests are no real attacks. They are *simulated* attacks. The
do_mprotect()is there to *simulate* behaviour people found in GLIBC under certain circumstances. In other words: This is how certain applications behave when run on exec-shield. They complained that PaXtest showed inaccurate results on exec-shield. Since the purpose of PaXtest is to show accurate results, the lack thereof has been fixed.
And people complains that nobody uses pax....
Quoted 9 lines · Peter Busser
Again, this is a *simulation* of the way real-life applications could interact with the underlying system. Again people complained that the results shown were not accurate. And that has been fixed.
I am well aware of complaints by some people about this behaviour. That is why there is a separated kiddie and blackhat mode in the latest PaXtest version. The kiddie mode is for those people who prefer to feel warm and cozy and the blackhat mode is for those who want to find out what the worst-case behaviour is. So if you don't like the blackhat results, don't run that test!
Umm, so exactly how many applications use multithreading (or otherwise trigger the GLIBC mprotect call), and how many applications use nested functions (which is not ANSI C compliant, and as a result, very rare)?
Do the tests both ways, and document when the dummy() re-entrant function might actually be hit in real life, and then maybe people won't feel that you are deliberately and unfairly overstating things to try to root for one security approach versus another. Of course, with name like "paxtest", maybe its only goal was propganda for the PaX way of doing things, in which case, that's fine. But if you want it to be viewed as an honest, fair, and unbaised, then make it very clear in the test results how programs with and without nested functions, and with and without multithreading, would actually behave.
Or are you afraid that someone might then say --- oh, so PaX's extra complexity is only needed if we care about programs that use nested functions --- yawn, I think we'll pass on the complexity. Is that a tradeoff that you're afraid to allow people to make with full knowledge?
- Ted
Quoted 2 lines · Theodore Ts'o
Umm, so exactly how many applications use multithreading (or otherwise trigger the GLIBC mprotect call),
For the record, I've been informed that the glibc mprotect() call doesn't happen in any modern glibc's; there may have been one buggy glibc that was released very briefly before it was fixed in the next release. But if that's what the paxtest developers are hanging their hat on, it seems awfully lame to me.....
"desabotaged" seems like the correct description from my vantage point.
- Ted
Quoted 4 lines · Theodore Ts'o
Quoted 2 lines · Theodore Ts'o
Umm, so exactly how many applications use multithreading (or otherwise trigger the GLIBC mprotect call),
Quoted 8 lines
For the record, I've been informed that the glibc
mprotect()call doesn't happen in any modern glibc's; there may have been one buggy glibc that was released very briefly before it was fixed in the next release. But if that's what the paxtest developers are hanging their hat on, it seems awfully lame to me....."desabotaged" seems like the correct description from my vantage point.
Well, great! One problem eliminated.
Anyways, for me it is not important whether what GLIBC exactly does or doesn't do. There are tons of different libraries and applications which do all kinds of stuff. You can only guess what exactly goes on. And not all compilers generate PT_GNU_STACK stuff either. And so on and so forth.
What is important to me is the question whether the PaXtest results are accurately reflecting the underlying system or not. Therefore I would like to see proof that exec-shield does NOT open up in situations where PaXtest says it does. So far I have seen ``sabotage'' FUD, opinions and excuses. But no proof. Nor any reasonable evidence. That doesn't surprise me, because PaXtest is accurate in what it does.
Groetjes,
Peter.
Hi!
Quoted 8 lines
Umm, so exactly how many applications use multithreading (or otherwise trigger the GLIBC mprotect call), and how many applications use nested functions (which is not ANSI C compliant, and as a result, very rare)?
Do the tests both ways, and document when the
dummy()re-entrant function might actually be hit in real life, and then maybe people won't feel that you are deliberately and unfairly overstating things to try to root for one security approach versus another.
Well, you can already do the test both ways. There is a kiddie mode, which doesn't do this test. And a blackhat mode, which does it. Basically removing the mprotect and nested function is demoting blackhat mode into kiddie mode.
Quoted 3 lines
Of course,
with name like "paxtest", maybe its only goal was propganda for the PaX way of doing things, in which case, that's fine.
Well, I can understand that perception. That is, if I would have made any contributions to PaX. Fact is, I haven't contributed as much as a single ASCII character to PaX. I'm just a user of PaX, just as I am a user of the Linux kernel. Sure, I recommend other people to use PaX. Why not? It works well for me. I recommend Linux to other people. I guess that's a bad thing too, right?
When I started Trusted Debian (which is what Adamantix used to be called), I looked around for security related stuff and found out about PaX. So I decided to give it a try and also the ET_DYN executable to maximise the use of ASLR (Address Space Layout Randomisation). After compiling a bunch of programs this way, I was kind of feeling uneasy. One would expect that such an intrusive kernel patch would break stuff. And PaX didn't. In fact, I didn't have any proof whether it worked or not.
Since my purpose was to distribute this stuff and I am not smart enough to fully understand the kernel code, I would not be able to truthfully tell people wether this PaX stuff worked or not. So I decided to write a test-suite. Well, it tests PaX functionality, not GCC functionality or POSIX compliance, so I named it PaXtest. That is how PaXtest began.
I don't understand where this ``propaganda'' mindframe comes from, but I don't have any money I make off of PaXtest, I don't have a stock price to worry about, no trademarks to defend and stuff like that. And in fact, I don't care whether you use PaXtest or not and what you think about the results. It is sufficient for me to know that they are accurately reflecting the underlying system.
Quoted 4 lines
But if you want
it to be viewed as an honest, fair, and unbaised, then make it very clear in the test results how programs with and without nested functions, and with and without multithreading, would actually behave.
Ok, so how exactly do you propose to do this?
Quoted 5 lines
Or are you afraid that someone might then say --- oh, so PaX's extra complexity is only needed if we care about programs that use nested functions --- yawn, I think we'll pass on the complexity. Is that a tradeoff that you're afraid to allow people to make with full knowledge?
That's looking at this from the bottom up. I'm looking at this from a different perspective. In civil engineering, it is normal to explore worst case scenarios. Suppose you don't do that and a bridge collapses and people die. Or a plane breaks in two while flying and people die. I'm sure you agree that that is a Really Bad Thing<tm>, right?
In the computer industry however, we mostly insist in building fragile systems. I'm just stating a fact, and not trying to imply that I'm any better than anyone else. There is a whole list of perfectly good excuses for building fragile computer systems. These systems are a risk. That's ok, as long as the risk can be managed. That is basically what most of security in the real-world (and I'm not talking about the technical level real-world here) is about: Risk management.
But you can't do risk management properly, unless you know what could happen in the worst case. Otherwise you get situations which Linus described regarding the OpenWall patch, where people might get a false sense of security (note that this is also reflected in the PaXtest results when run on an OpenWall kernel). There are clear differences between how PaX and exec-shield behave in worst case situations. PaXtest shows these when you run blackhat mode but not when you run kiddie mode. And that's all there is to it.
Now, what any person does with the PaXtest results, that is up to that person and not for you or me to decide. And that is why I want to stop the FUD which is being spread and to stop people from abusing their reputation as kernel developer to influence other people to cripple their copies of PaXtest, thereby removing their ability to explore the worst-case scenario. Crippling PaXtest effectively works as a form of censorship. Personally, I think the Linux community is more served by an open discussion than by censorship. But I seems some people have a different opinion on that.
Groetjes,
Peter.
Quoted 6 lines · Peter Busser
Hi!
Quoted 4 lines
Umm, so exactly how many applications use multithreading (or otherwise trigger the GLIBC mprotect call), and how many applications use nested functions (which is not ANSI C compliant, and as a result, very rare)?
Quoted 8 lines
Quoted 4 lines
Do the tests both ways, and document when the
dummy()re-entrant function might actually be hit in real life, and then maybe people won't feel that you are deliberately and unfairly overstating things to try to root for one security approach versus another.Well, you can already do the test both ways. There is a kiddie mode, which doesn't do this test. And a blackhat mode, which does it. Basically removing the mprotect and nested function is demoting blackhat mode into kiddie mode.
actually you don't. The presence of the nested function (technically, the taking of the address of a nested function) marks the PT_GNU_STACK field in the binary, not the runtime behavior. As such, paxtest does not offer any real such choice in behavior. The binary needs stack trampolines, just that in one case you don't use them.
Quoted 3 lines
Umm, so exactly how many applications use multithreading (or otherwise trigger the GLIBC mprotect call), and how many applications use nested functions (which is not ANSI C compliant, and as a result, very rare)?
i think you're missing the whole point of paxtest. it's not about what glibc et al. do or don't do. it's about what an exploit can do (by virtue of forcing the exploited application to do something). if your line of thinking was correct, then why didn't you also object to the fact that the paxtest applications overflow their own stack/heap/etc? or that they put 'shellcode' onto the stack/heap/etc and invoke it by abusing a memory corrupting bug? surely no sane real-life application does any of this.
so once again, let me explain what paxtest does. it tests PaX for its claims. since PaX claims to be an intrusion prevention system, paxtest tries to simulate said intrusions (in particular, we're talking about exploiting memory corruption bugs). the stress is on 'simulate'. none of the paxtest applications are full blown exploits. nor do they need to be. knowing what PaX does (or claims to do), it's very easy to design and write small applications that perform the core steps of an exploit that may be able to break the protection mechanisms. i.e., any 'real' exploit would eventually have to perform these core steps, so by ensuring that they don't work (at least when the PaX claims stand) we can ensure that no real life exploit would work either.
now let's get back to mprotect(), nested functions, etc. when an exploit writer knows that the only barrier that prevents him from executing his shellcode is a circumventible memory protection, then guess what, he'll first try to circumvent said memory protection then execute his shellcode as usual. since memory protection is controlled by mmap/mmprotect, his goal will be to somehow force the exploited application to end up calling these functions to get an executable region holding his shellcode.
your concerns would be valid if this was impossible to achieve by an exploit, sadly, you'd be wrong too, it's possible to force an exploited application to call something like dl_make_stack_executable() and then execute the shellcode. there're many ways of doing this, the simplest (in terms of lines of code) was chosen for paxtest. or put another way, would you still argue that the use of the nested function trampoline is sabotage whereas an exploit forcing a call to dl_make_stack_executable() isn't? because the two achieve the exact same thing, they open up the address space (or part of it, depending on the intrusion prevention system) for shellcode execution.
one thing that paxtest didn't get right in the 'kiddie' mode is that it still ran with an executable stack, that was not the intention but rather an oversight, it'll be fixed in the next release. still, this shouldn't leave you with a warm and fuzzy feeling about the security of intrusion prevention systems that 'pass' the 'kiddie' mode but fail the 'blackhat' mode, in the real life out there, only the latter matters (if for no other reason, then for natural evolution/adaptation of exploit writers).
Hi!
Quoted 8 lines
one thing that paxtest didn't get right in the 'kiddie' mode is that it still ran with an executable stack, that was not the intention but rather an oversight, it'll be fixed in the next release. still, this shouldn't leave you with a warm and fuzzy feeling about the security of intrusion prevention systems that 'pass' the 'kiddie' mode but fail the 'blackhat' mode, in the real life out there, only the latter matters (if for no other reason, then for natural evolution/adaptation of exploit writers).
I apologise for this bug. If someone had pointed this out in a clear and to-the-point kind of way, then this would have been fixed a long time ago.
Anyways, if anyone else has any suggestions, fixes, or special wishes for PaXtest (some exec-shield specific tests perhaps?), then please speak up now. I'd rather not bother this list again about PaXtest related issues.
Groetjes,
Peter.
* [email protected] <[email protected]> wrote:
Quoted 4 lines · On Wed, Feb 02, 2005 at 10:18:27PM +1000, [email protected] wrote:
your concerns would be valid if this was impossible to achieve by an exploit, sadly, you'd be wrong too, it's possible to force an exploited application to call something like
dl_make_stack_executable()and then execute the shellcode.
If you can call mprotect() with a protected environment to unprotect it, you can as easily call exec.
OG.
Re: Sabotaged PaXtest
In reply to Arjan van de Ven-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 3 lines · Arjan van de Ven
Quoted 1 line · Peter Busser
Hi!
[...]
Quoted 13 lines
the paxtest 0.9.6 that John Moser mailed to this list had this gem in it:
(quoted diff) +2
-0@@ -39,8 +42,6 @@ */ int paxtest_mode = 1; + /* Dummy nested function */ + void dummy(void) {} mode = getenv( "PAXTEST_MODE" ); if( mode == NULL ) {
Quoted 4 lines
which is clearly there with the only possible function of sabotaging the automatic
PT_GNU_STACKsetting by the toolchain (which btw is not fedora specific but happens by all new enough (3.3 or later) gcc compilers on all distros) since that requires an executable stack.
I'll say this AGAIN: I execstack -c'd the directory after the first borked up test and stuff randomly failed. After upgrading FC3's kernel though I got predictable results.
- --
All content of all messages exchanged herein are left in the Public Domain, unless otherwise explicitly stated.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFCB7T0hDd4aOud5P8RAk7XAJ4oYgr2ySTdg+y80f6pBasO+x8ttACfdjYx ++dQRSTGzTGP/Vsp2KZ6YHU=
=jp9U
-----END PGP SIGNATURE-----