Skip to contentSkip to conversation

Linux 2.4.27 SECURITY BUG - TCP Local andREMOTE(verified)Denial of Service Attack

DISCUSSION Wolfpaw - Dale Corse 2 participants 2 messages Sep 12, 2004 – Sep 13, 2004 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore
#1 lore raw
In reply to a message not in the archive
Quoted 12 lines · Wolfpaw - Dale Corse
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

A) Why are the timeouts so long?

So you don't get random corruption

Hmm. I'll take your word for it, I'm not quite grasping it, but you know quite a bit more about it then I do :) I would have

thought once

Quoted 1 line

a close is sent, the data has all been received and/or sent

anyway, so

Quoted 1 line

what would corrupt?

Quoted 7 lines

It has all arrived at least once. However you don't know if a retransmitted packet took a long trip round the world and popped out later. When that happens you need to be sure you can tell old and new apart. The TIME_WAIT state is basically "don't use this identical set of port/address data for long enough to be sure any prior use has exited the internet in its entirety.

That makes sense :)

Quoted 10 lines
Quoted 2 lines

It _appears_ that when we close a socket (ie with

mysql_close) on the

Quoted 1 line

client side, the client side closes the FD properly (though Mysql

socket != fd. If you close an fd and open you may get the same fd but its a different socket. If its getting stuck closing could you have another copy of the fd left open somewhere or have passed it to a process you forked (thats a classic nasty to track down error) ?

The way it generally works, is the daemon opens quite a few sql connections for different reasons. It stores all of them in a linked list (a somewhat implementation of persistent connections, since the c API doesn't provide for it)

Every 60 seconds, the list is scanned for "unused" sql connections, and calls mysql_close on them. The instance of the actual list entry is not deleted until the next run to give things 60 seconds to clear up. The issue we were having is this..

Mysql connection: descriptor 3 (from mysql.net.fd) Mysql connection closed (desc 3) (goes into CLOSE_WAIT now) New connection (outbound) for regular proxy on Desc 3
(this was created by a call to socket, and then connect)

Now the only way to get mysql to actually CLOSE the connection, is go call close(mysql.net.fd) before destroying the structure. This would have the effect of clearing the CLOSE_WAIT entry from the netstat table, but it would also close the active connection that was made with socket/connect (they have the same FD). This is why I said they are being reused .. Our software (being the client) has requested the connection closed, and mysql hasn't closed it (unsure why.. Maybe data from the db query still sitting on the SQL side?), so it then reuses what it thinks is a clean FD, and it ends up needing to be closed AGAIN to stop the other bug..

The other bug being, if I simply leave them, in a short time, MySQL is saying "too many connections", and we can't query any data from it. This also occurs with FIFO sockets (such as /tmp/mysql.sock) wherein the connection simply sits as "ESTABLISHED".

I have been unable to find any sort of "flush" command to issue to the SQL server on the socket, to tell it that I am done with whatever data it has pulled out of the DB. This is obviously an SQL bug.

I assumed it an OS bug, mostly because the "socket" that used to belong to the FD is still open at least on one side, I was thinking this needs to be checked for somehow. What do you think?

Quoted 12 lines
Quoted 4 lines

doesn't), and then if we call connect (which it does a lot, being a proxy
server) it will reuse that FD. At this point, the Mysql

side still hasn't

Quoted 1 line

closed it, and it is sitting in CLOSE_WAIT, where it

remains forever,
since

Quoted 1 line

it is in use by the client side elsewhere already. Should

connect be

Quoted 1 line

checking the "list of not connected, but state other then

CLOSED" list

Quoted 8 lines
Quoted 2 lines

before it decides to use a particular FD? Or is this behavior intentional?

Sockets are two way at the TCP layer. "close" really means "have finished sending". When both ends have finished sending the connection is complete but not before. Thus if Mysql daemon says "I am done" but you have open references to the handle then it will stay open one way still.

So something needs to trigger a "flush" of the left over data on the SQL side before closing the connection - yes?

Again - thank you for your input :)
D.

#2 lore raw
Quoted 4 lines · Wolfpaw - Dale Corse

Mysql connection: descriptor 3 (from mysql.net.fd) Mysql connection closed (desc 3) (goes into CLOSE_WAIT now) New connection (outbound) for regular proxy on Desc 3
(this was created by a call to socket, and then connect)

fd != socket. Thats really important to realise. What you get on fd 3 from the new connection isn't the same as you had before. The one you closed has been handed off to the kernel to clean up as and when everyone who has a copy has finished using it. Thats why I asked about fork() - because you can end up giving handles by mistake to other processes you create which don't close them

Quoted 4 lines

The other bug being, if I simply leave them, in a short time, MySQL is saying "too many connections", and we can't query any data from it. This also occurs with FIFO sockets (such as /tmp/mysql.sock) wherein the connection simply sits as "ESTABLISHED".

I'd say your code is buggy then

Quoted 2 lines

So something needs to trigger a "flush" of the left over data on the SQL side before closing the connection - yes?

That will occur anyway for you. I think you need to find out where the other copies of the same fd went and how mysql manages them