Linux 2.4.27 SECURITY BUG - TCP Local and REMOTE(verified) Denial of Service Attack
Quoted 11 lines · Wolfpaw - Dale Corse
Quoted 9 lines
No problem :) I run the following, against SSH as the target, and I
can also kill it. (using telnet as the other side of the attack)root@maximus:/home/admin# telnet XXXXXXXXXXXXXXX 22 Trying XXXXXXXXXXXXXX... Connected to XXXXXXXXXXXXXXXXX. Escape character is '^]'. Connection closed by foreign host. root@maximus:/home/admin#
Quoted 4 lines
Quoted 1 line
Now.. Do you really want me to post the source code for it?
With default
sshd_configyou can DOS sshd trivially by opening ten connections using ten times "telnet XXXXXXXXXXXXXXX 22".
A fair comment :) But look at it this way:
- The TCP RFC was last updated when?
- What is the average time for a tcp packet to fly even across the world these days? Maybe 300 ms? 1 second? 5?
- It is not a secret that the TCP protocol has flaws, take for example the RST bug, which required among other things, BGP4 to use MD5 encryption to avoid being potentially attacked.
So this brings me to:
A) Why are the timeouts so long?
B) CLOSE_WAIT having _no timeout at all_ is still using the assumption the other side is honest, and will actually reply. This is a very bad assumption.
C) Socket still re-uses an FD before it is actually completely closed. This is bad, because by calling a second close in the case of mysql, you can get the connection to go away, but in that case, it closes whatever else is on that FD too. (A more likely analysis is that it closes the current connection, and then cleans the CLOSE_WAIT on that FD out of the other pool)
All I am trying to point out is that the Internet in general, and The Open Source movement has survived, and evolved because of innovation, and the ability to meet upcoming threats quickly. TCP has some issues which are blindingly obvious, and they are issues that, in my view (flawed as it may be) can be at least somewhat minimized by a few simple changes.
I realize daemons have connection queues, and timeouts for a reason, but really, if a daemon wishes to close a connection, for whatever reason, sending something to the other side is required, but I can't see why having the other side send something back is part of the protocol. This could be implemented with KEEPALIVE much easier, and would avoid the flaws.. No reply from the host in say 10 seconds, then drop the connection. You could still clog queues, to which I would say the application needs to cope with one client
filling the whole queue as best it can, and it wouldn't stop a DDOS (not much does), but it might help some at least.
Anyway.. That's my 2 cents. I will continue my conversation with Peter from mysql in regards to mysql_close, which was really, the entire point. It is sad however to see a maintainer come across in the manner which David has during the course of this discussion. It doesn't bode well for the future of open source to tell someone off, whom likely has a valid point.. whether or not it is a repairable fault.
Regards,
Dale.
--------------------------------
Dale Corse
System Administrator
Wolfpaw Services Inc.
http://www.wolfpaw.net
(780) 474-4095
Quoted 3 lines · Wolfpaw - Dale Corse
A fair comment :) But look at it this way:
- The TCP RFC was last updated when?
About 2 months ago. The 793 RFC isn't updated instead new ones are added for the additional features/discoveries.
Quoted 5 lines
- What is the average time for a tcp packet to fly even across the world these days? Maybe 300 ms? 1 second? 5?
- It is not a secret that the TCP protocol has flaws, take for example the RST bug, which required among other things, BGP4 to use MD5 encryption to avoid being potentially attacked.
This is not a TCP flaw, its a combination of poor design by certain vendors, poor BGP implementation and a lack of understanding of what TCP does and does not do. See IPSec. TCP gets stuff from A to B in order and knowing to a resonable degree what arrived. TCP does not proide a security service.
(The core of this problem arises because certain people treat TCP connection down on the peering session as link down)
Quoted 3 lines
So this brings me to:
A) Why are the timeouts so long?
So you don't get random corruption
Quoted 1 line
C) Socket still re-uses an FD before it is actually completely
Pardon ?
Quoted 2 lines
sending something to the other side is required, but I can't see why having the other side send something back is part of the protocol. This could be
Because packet sizes are finite and not doing so requires an infinite sequence space and thus infinite packet sizes. Reread the TCP specifications more carefully, also look at RFC1337 which discusses some of the real world cases of getting this wrong.
Quoted 9 lines · Alan Cox
This is not a TCP flaw, its a combination of poor design by certain vendors, poor BGP implementation and a lack of understanding of what TCP does and does not do. See IPSec. TCP gets stuff from A to B in order and knowing to a resonable degree what arrived. TCP does not proide a security service.
(The core of this problem arises because certain people treat TCP connection down on the peering session as link down)
Alan, could you please elaborate on this last statement? I don't understand what you mean, and am very interested.
Thanks,
Toon.
Signature · 3 lines
"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian W. Kernighan
Quoted 8 lines · Toon van der Pas
Quoted 5 lines
knowing to a resonable degree what arrived. TCP does not proide a security service.
(The core of this problem arises because certain people treat TCP connection down on the peering session as link down)
Alan, could you please elaborate on this last statement? I don't understand what you mean, and am very interested.
I think he means that BGP treating TCP connections as if they could reliably and securely indicate link/path status (ie connection reset/timeout == link down) status was, in retrospect, a very dumb idea on the part of BGP.
Quoted 2 lines
Thanks,
Toon.
regards,
Signature · 5 lines
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A Fortune: This restaurant was advertising breakfast any time. So I ordered french toast in the renaissance. - Steven Wright, comedian
Quoted 1 line · Paul Jakma
I think he means that BGP treating TCP connections as if they could reliably and securely indicate link/path status (ie connection reset/timeout == link down) status was, in retrospect, a very dumb idea on the part of BGP.
More specifically, BGP should have treated TCP resets as a transient error, to be expected (indeed, they /cant/ be a sign that a link is down - if you can receive a RST the link or path is patently quite ok). The BGP state machine should instead, in normal operation, have only treated Hold time expired as the definitive sign of "peer is down" and allowed reconnects.
Quoted 1 line
regards,
regards,
Signature · 3 lines
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A Fortune: You will attract cultured and artistic people to your home.
Quoted 4 lines · Paul Jakma
More specifically, BGP should have treated TCP resets as a transient error, to be expected (indeed, they /cant/ be a sign that a link is down - if you can receive a RST the link or path is patently quite ok).
The application level does not always distinguish between TCP RST and error generated by the local system because of a "network unreachable" due to a link down and a lost route.
Quoted 3 lines
The BGP state machine should instead, in normal operation, have only treated Hold time expired as the definitive sign of "peer is down" and allowed reconnects.
It should not necessarily wait for the time-out, but at least wait for a few reconnect errors.
Regards,
willy
Quoted 2 lines · Willy Tarreau
It should not necessarily wait for the time-out, but at least wait for a few reconnect errors.
No, it should wait for the timeout. (how many reconnects? maybe use a time for that? well, you already have one, so use that. if you want to timeout quicker, lower it.)
Alas though, it wouldnt be BGP.
Quoted 2 lines
Regards,
willy
regards,
Signature · 3 lines
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A Fortune: You will be called upon to help a friend in trouble.
Salut,
Quoted 6 lines · Willy Tarreau
Quoted 3 lines
The BGP state machine should instead, in normal operation, have only treated Hold time expired as the definitive sign of "peer is down" and allowed reconnects.
It should not necessarily wait for the time-out, but at least wait for a few reconnect errors.
Problem there: you can fake connection errors almost as easily as sending an RST packet, so the DoS might reappear, might it not?
Tonnerre
Quoted 7 lines · Tonnerre
Salut,
Quoted 4 lines · Willy Tarreau
Quoted 3 lines
The BGP state machine should instead, in normal operation, have only treated Hold time expired as the definitive sign of "peer is down" and allowed reconnects.
Quoted 6 lines
Quoted 2 lines
It should not necessarily wait for the time-out, but at least wait for a few reconnect errors.
Problem there: you can fake connection errors almost as easily as sending an RST packet, so the DoS might reappear, might it not?
No, as long as you don't keep the routes from the old session until the new one establishes and fills up (or you reach the timeout). And when I spoke about "connection errors", I really spoke about connection establishment. I bet you'll have more difficulties trying to send the right RST just after a SYN (or an ICMP unreachable with the right payload) than sending them once the session is already established. It does make a big difference.
Willy
Quoted 1 line · Tonnerre
Problem there: you can fake connection errors almost as easily as sending an RST packet, so the DoS might reappear, might it not?
Sure, but TCP just isnt going to solve this for you.
Quoted 1 line
Tonnerre
regards,
Signature · 5 lines
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A Fortune: All progress is based upon a universal innate desire of every organism to live beyond its income. -- Samuel Butler, "Notebooks"
Quoted 2 lines · Paul Jakma
More specifically, BGP should have treated TCP resets as a transient error, to be expected (indeed, they /cant/ be a sign that a link is
Actually you can treat TCP session failure as transient error. Just use BGP graceful restart (witch basically allows re-opening TCP connection without losing routing tables).
http://www.ietf.org/internet-drafts/draft-ietf-idr-restart-10.txt
Signature · 4 lines
[Ville Hallivuori][[email protected]][http://www.iki.fi/vph/] [ID 8E1AD461][FP16=C9 50 E2 DF 48 F6 33 62 5D 87 47 9D 3F 2B 07 5D] [ID 58543419][FP20=8731 941D 15AB D4A0 88A0 FC8F B55C F4C4 5854 3419] [ID 8061C24E][FP20=C722 12DA 841E D811 DBFE 2FB3 174C E291 8061 C24E]
Quoted 3 lines · Ville Hallivuori
Actually you can treat TCP session failure as transient error. Just use BGP graceful restart (witch basically allows re-opening TCP connection without losing routing tables).
http://www.ietf.org/internet-drafts/draft-ietf-idr-restart-10.txt
Hmm, yes, I hadnt thought of the attack-mitigating aspects of graceful restart. Though, without other measures, the session is still is open to abuse (send RST every second).
regards,
Signature · 6 lines
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A Fortune: Wit, n.: The salt with which the American Humorist spoils his cookery ... by leaving it out. -- Ambrose Bierce, "The Devil's Dictionary"
Quoted 3 lines · Paul Jakma
Hmm, yes, I hadnt thought of the attack-mitigating aspects of graceful restart. Though, without other measures, the session is still is open to abuse (send RST every second).
Its more than that given port randomization, quite a lot more. Of course its much easier to just send "must fragment, size 68" icmp replies and guess them that way. This is spectacularly more effective and various vendors highly invalid rst acking crap won't save you.
Quoted 1 line · Alan Cox
guess them that way. This is spectacularly more effective and various vendors highly invalid rst acking crap won't save you.
Ah, well, I dont care about various vendors. I only care about Linux, BSD and SunOS kernel behaviour ;)
That said, TCP-MD5 signature renders this mostly moot, and deployment of TCP-MD5 has increased a lot since the last round of "BGP TCP is insecure!" non-issues came up. Many IXes and peers now require TCP-MD5.
The rights and wrongs of TCP-MD5 notwithstanding, it'd be nice if Linux could support this. Anyone running BGP on Linux at moment must patch their kernel - or else just switch to Free/Open BSD.
regards,
Signature · 5 lines
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A Fortune: It looks like it's up to me to save our skins. Get into that garbage chute, flyboy! -- Princess Leia Organa
Quoted 4 lines · Paul Jakma
That said, TCP-MD5 signature renders this mostly moot, and deployment of TCP-MD5 has increased a lot since the last round of "BGP TCP is insecure!" non-issues came up. Many IXes and peers now require TCP-MD5.
TCP-MD5 has no effect on ICMP based attacks.,
Quoted 1 line · Alan Cox
TCP-MD5 has no effect on ICMP based attacks.,
Hmm, good point. Which attacks, and what could be done about them? (other than IPsec protect all traffic between peers).
regards,
Signature · 4 lines
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A Fortune: "You can't get very far in this world without your dossier being there first." -- Arthur Miller
Hi Alan,
Quoted 8 lines · Alan Cox
Quoted 3 lines · Paul Jakma
Hmm, yes, I hadnt thought of the attack-mitigating aspects of graceful restart. Though, without other measures, the session is still is open to abuse (send RST every second).
Of course its much easier to just send "must fragment, size 68" icmp replies and guess them that way. This is spectacularly more effective and various vendors highly invalid rst acking crap won't save you.
Just wondering, I have not checked. Isn't the "must fragment" message supposed to embed part of the packet it couldn't send in return ? If this is the case (and if the victim processes it correctly), it would need to guess a recent valid content. If it's not the case, I suspect it would simply update the path mtu in the route cache, thus giving spectacular effects :-)
Cheers,
Willy
Quoted 2 lines · Willy Tarreau
Just wondering, I have not checked. Isn't the "must fragment" message supposed to embed part of the packet it couldn't send in return ?
You need to guess no more than for an RST attack, and furthermore in some cases (buggy stacks) IPsec doesn't save you because the error is from an untrusted midpoint. The proper response to such messages is to turn off DF usage but not all stacks get it right
* Paul Jakma:
Quoted 4 lines
Quoted 1 line
TCP-MD5 has no effect on ICMP based attacks.,
Hmm, good point. Which attacks, and what could be done about them? (other than IPsec protect all traffic between peers).
You just filter ICMP packets, in the way RST packets are already filtered (i.e. rate limit).
The only TCP desynchronization attack that has a chance of working practice is the SYN-based one. The rate limit for RST processing on Cisco routers is far too low.
(Mixed Cisco/Quagga environments are a different matter, but rather unusual and easily DoSed anyway, most of the time.)
* Alan Cox:
Quoted 8 lines · Paul Jakma
Quoted 3 lines
Hmm, yes, I hadnt thought of the attack-mitigating aspects of graceful restart. Though, without other measures, the session is still is open to abuse (send RST every second).
Its more than that given port randomization, quite a lot more. Of course its much easier to just send "must fragment, size 68" icmp replies and guess them that way.
Is this attack documented anywhere?
Quoted 5 lines · Florian Weimer
ndomization, quite a lot more. Of course
Quoted 2 lines
its much easier to just send "must fragment, size 68" icmp replies and guess them that way.
Is this attack documented anywhere?
Bugtraq years ago and also in the discussions of the IP sec protocol design flaws when it was being specified
Quoted 8 lines · Florian Weimer
Quoted 5 lines
Quoted 2 lines
TCP-MD5 has no effect on ICMP based attacks.,
Hmm, good point. Which attacks, and what could be done about them? (other than IPsec protect all traffic between peers).
You just filter ICMP packets, in the way RST packets are already filtered (i.e. rate limit).
Rate-limiting has no effect on ICMP attacks unless your limit is such that you're effectively dropping them all. But then you get PMTU problems...
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt* Herbert Xu:
Quoted 4 lines
Quoted 3 lines · Florian Weimer
Quoted 3 lines
Quoted 2 lines
TCP-MD5 has no effect on ICMP based attacks.,
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Hmm, good point. Which attacks, and what could be done about them? (other than IPsec protect all traffic between peers).
You just filter ICMP packets, in the way RST packets are already filtered (i.e. rate limit).
Rate-limiting has no effect on ICMP attacks unless your limit is such that you're effectively dropping them all.
Yes, that's the idea. Keep in mind that all this is about traffic destined to a router interface address, not about forwarded traffic.
Quoted 1 line
But then you get PMTU problems...
PMTU discovery is not an issue because it's turned off anyway, at least by default.
* David S. Miller:
Quoted 10 lines
Quoted 4 lines · On Tue, 21 Sep 2004 20:32:12 +0200 Florian Weimer <[email protected]> wrote:
Quoted 1 line
But then you get PMTU problems...
PMTU discovery is not an issue because it's turned off anyway, at least by default.
It's on by default, for both TCP and UDP in the kernel, and has been so for a long time.
Linux is not the reference TCP/IP stack for routers. 8-)
Quoted 1 line
Why would it be off by default?
Probably because PMTUD is just a DRAFT STANDARD, and these router folks are usually extremely conservative. Switching the default is dangerous because it's likely to break existing setups, as Herbert noted.
Quoted 1 line
If you disable PMTU discovery, say goodbye to TCP performance.
Indeed. On those platforms, the CPU impact is also significant, and the overall increase in BGP convergence time is measurable.
Quoted 4 lines · On Tue, 21 Sep 2004 20:32:12 +0200 Florian Weimer <[email protected]> wrote:
Quoted 1 line
But then you get PMTU problems...
PMTU discovery is not an issue because it's turned off anyway, at least by default.
It's on by default, for both TCP and UDP in the kernel, and has been so for a long time.
Why would it be off by default?
If you disable PMTU discovery, say goodbye to TCP
performance.
Quoted 3 lines · On Tue, 21 Sep 2004 22:04:41 +0200 Florian Weimer <[email protected]> wrote:
Quoted 1 line
Why would it be off by default?
Probably because PMTUD is just a DRAFT STANDARD,
RFC1191 doesn't look like a draft to me.
* David S. Miller:
Quoted 8 lines
Quoted 3 lines · On Tue, 21 Sep 2004 22:04:41 +0200 Florian Weimer <[email protected]> wrote:
Quoted 1 line
Why would it be off by default?
Probably because PMTUD is just a DRAFT STANDARD,
RFC1191 doesn't look like a draft to me.
It's not a draft document, but it's still a DRAFT STANDRD in the IETF standards track, see RFC 3700. (I wasn't shouting, I was using IETF keywords. 8-)