SOCK_STREAM TCP: send() returns success even when other side responded with RST packet
Subsystems (2)
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi.
I was advised to forward my networking Kernel related Bug here
so it can get fixed much faster. Here is the Bug description
copied from Red Hat Bugzilla (https://bugzilla.redhat.com/show_bug.cgi?id=912382):
Description of problem:
I'm using SOCK_STREAM TCP sockets. Imagine server-client application.
- Server will accept connection from client.
- Client sends some data to server -> server receives data.
- Client then half-closes the TCP connection using shutdown(fd, SHUT_WR).
- Client waits for some data from the server.
- Server sends some data to client -> client receives data.
- Client exits and closes the socket.
- After some time server tries to send some data to the client.
- The first send() call returns success even the Client's response is RST.
Version-Release number of selected component (if applicable):
3.7.7-201.fc18.x86_64
3.7.8-202.fc18.x86_64
How reproducible:
always
Steps to Reproduce:
1. download attached client/server sources.
2. compile server: # gcc tcp_server.c -o server
3. compile client: # gcc tcp_client.c -o client
4. run the server: # ./server
5. run the client: # ./client
Actual results:
Server started...
Created SOCK_STREAM socket
Bound to localhost:6666
waiting for connection...
Client connected!
Waiting for some data from client
Received: "MSG from client"
Received EOF
Sending msg to client: "MSG from server"
sleep(5)
Sending 1 msg to client: "MSG from server"
try #1 send() result: (0) Success
Sending 2 msg to client: "MSG from server"
try #2 send() result: (32) Broken pipe
Server exiting...
Expected results:
Server should fail to send() data right after sleep().
Server started...
Created SOCK_STREAM socket
Bound to localhost:6666
waiting for connection...
Client connected!
Waiting for some data from client
Received: "MSG from client"
Received EOF
Sending msg to client: "MSG from server"
sleep(5)
Sending 1 msg to client: "MSG from server"
try #1 send() result: (32) Broken pipe
Server exiting...
Additional info:
RST answer means that data were NOT delivered. Therefore the very first send() call after server sleep() should fail!
Regards,
Tomas Hozza
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <signal.h>
int main(int argc, char **argv)
{
int ssock, csock, i;
static char buff[256];
ssize_t nbytes;
struct sockaddr_in saddr;
memset(&saddr, 0, sizeof(saddr));
saddr.sin_family = AF_INET;
saddr.sin_port = htons(6666);
signal(SIGPIPE, SIG_IGN);
printf("Server started...\n");
if ((ssock = socket(AF_INET, SOCK_STREAM, 0)) < 0) {
printf("socket() error. (%d) %s\n", errno, strerror(errno));
exit(1);
}
printf("Created SOCK_STREAM socket\n");
if (bind(ssock, (struct sockaddr*)&saddr, sizeof(saddr)) == -1) {
printf("bind() error. (%d) %s\n", errno, strerror(errno));
close(ssock);
exit(1);
}
printf("Bound to localhost:6666\n");
if (listen(ssock, 1) == -1) {
printf("listen() error. (%d) %s\n", errno, strerror(errno));
close(ssock);
exit(1);
}
printf("waiting for connection...\n");
csock = accept(ssock, NULL, NULL);
if (csock == -1) {
printf("accept() error. (%d) %s\n", errno, strerror(errno));
close(ssock);
exit(1);
}
printf("Client connected!\n");
printf("Waiting for some data from client\n");
while ((nbytes = recv(csock, buff, sizeof(buff), 0)) > 0) {
printf("Received: \"%s\"\n", buff);
}
if (nbytes < 0) {
printf("recv() error. (%d) %s\n", errno, strerror(errno));
close(ssock);
close(csock);
exit(1);
} else if (nbytes == 0) {
printf("Received EOF\n");
printf("Sending msg to client: \"MSG from server\"\n");
if ((nbytes = send(csock, "MSG from server", sizeof("MSG from server"), 0)) <= 0) {
printf("send() error. (%d) %s\n", errno, strerror(errno));
close(ssock);
close(csock);
exit(1);
}
}
printf("sleep(5)\n");
sleep(5);
i = 1;
while (1) {
printf("Sending %d msg to client: \"MSG from server\"\n", i);
nbytes = send(csock, "MSG from server", sizeof("MSG from server"), 0);
printf("try #%d send() result: (%d) %s\n",i ,errno, strerror(errno));
if (nbytes <= 0)
break;
++i;
}
printf("Server exiting...\n");
close(ssock);
close(csock);
return 0;
}#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
int main(int argc, char **argv)
{
int ssock, i;
static char buff[256];
ssize_t nbytes;
struct sockaddr_in saddr;
memset(&saddr, 0, sizeof(saddr));
saddr.sin_family = AF_INET;
saddr.sin_port = htons(6666);
printf("Client started...\n");
if ((ssock = socket(AF_INET, SOCK_STREAM, 0)) < 0) {
printf("socket() error. (%d) %s\n", errno, strerror(errno));
exit(1);
}
printf("Created SOCK_STREAM socket\n");
if (connect(ssock, (struct sockaddr *)&saddr, sizeof(saddr)) == -1) {
printf("connect() error. (%d) %s\n", errno, strerror(errno));
close(ssock);
exit(1);
}
printf("Connected to the server!\n");
printf("Sending some data to server\n");
if ((nbytes = send(ssock, "MSG from client", sizeof("MSG from client"), 0)) <= 0) {
printf("send() error. (%d) %s\n", errno, strerror(errno));
close(ssock);
exit(1);
}
printf("Half-closing connection with shutdown(ssock, SHUT_WR)\n");
if (shutdown(ssock, SHUT_WR) == -1) {
printf("shutdown() error. (%d) %s\n", errno, strerror(errno));
close(ssock);
exit(1);
}
printf("Receiving data from server\n");
if ((nbytes = recv(ssock, buff, sizeof(buff), 0)) > 0) {
printf("Received: \"%s\"\n", buff);
} else {
printf("recv() error. (%d) %s\n", errno, strerror(errno));
close(ssock);
exit(1);
}
printf("Client exiting...\n");
close(ssock);
return 0;
}Quoted 71 lines · Tomas Hozza
Hi.
I was advised to forward my networking Kernel related Bug here
so it can get fixed much faster. Here is the Bug description
copied from Red Hat Bugzilla (https://bugzilla.redhat.com/show_bug.cgi?id=912382):Description of problem:
I'm usingSOCK_STREAMTCP sockets. Imagine server-client application.
- Server will accept connection from client.
- Client sends some data to server -> server receives data.
- Client then half-closes the TCP connection usingshutdown(fd, SHUT_WR).
- Client waits for some data from the server.
- Server sends some data to client -> client receives data.
- Client exits and closes the socket.
- After some time server tries to send some data to the client.
- The firstsend()call returns success even the Client's response is RST.Version-Release number of selected component (if applicable):
3.7.7-201.fc18.x86_64
3.7.8-202.fc18.x86_64How reproducible:
alwaysSteps to Reproduce:
1. download attached client/server sources.
2. compile server: # gcctcp_server.c-o server
3. compile client: # gcctcp_client.c-o client
4. run the server: # ./server
5. run the client: # ./clientActual results:
Server started...
CreatedSOCK_STREAMsocket
Bound to localhost:6666
waiting for connection...
Client connected!
Waiting for some data from client
Received: "MSG from client"
Received EOF
Sending msg to client: "MSG from server"
sleep(5)
Sending 1 msg to client: "MSG from server"
try #1send()result: (0) Success
Sending 2 msg to client: "MSG from server"
try #2send()result: (32) Broken pipe
Server exiting...Expected results:
Server should fail tosend()data right aftersleep().Server started...
CreatedSOCK_STREAMsocket
Bound to localhost:6666
waiting for connection...
Client connected!
Waiting for some data from client
Received: "MSG from client"
Received EOF
Sending msg to client: "MSG from server"
sleep(5)
Sending 1 msg to client: "MSG from server"
try #1send()result: (32) Broken pipe
Server exiting...
Quoted 8 lines
Additional info:
RST answer means that data were NOT delivered. Therefore the very firstsend()call after serversleep()should fail!Regards,
Tomas Hozza
Hi Tomas
I read your bug report, but cant understand why you believe there is a bug.
send() is not synchronous.
It only queues bytes to be transmitted either right now, either later when its possible.
So a RST message can be received _after_ send() has accepted the request and copy data from user land to kernel socket write queue, but TCP stack did not yet delivered the data.
To make sure data is delivered, you need additional logic.
----- Original Message -----
Quoted 83 lines · Tomas Hozza
Quoted 82 lines
Hi.
I was advised to forward my networking Kernel related Bug here so it can get fixed much faster. Here is the Bug description copied from Red Hat Bugzilla
(https://bugzilla.redhat.com/show_bug.cgi?id=912382):Description of problem:
I'm usingSOCK_STREAMTCP sockets. Imagine server-client application.
- Server will accept connection from client.
- Client sends some data to server -> server receives data.
- Client then half-closes the TCP connection usingshutdown(fd, SHUT_WR).
- Client waits for some data from the server.
- Server sends some data to client -> client receives data.
- Client exits and closes the socket.
- After some time server tries to send some data to the client.
- The firstsend()call returns success even the Client's response is RST.Version-Release number of selected component (if applicable):
3.7.7-201.fc18.x86_64
3.7.8-202.fc18.x86_64How reproducible:
alwaysSteps to Reproduce:
1. download attached client/server sources.
2. compile server: # gcctcp_server.c-o server
3. compile client: # gcctcp_client.c-o client
4. run the server: # ./server
5. run the client: # ./clientActual results:
Server started...
CreatedSOCK_STREAMsocket
Bound to localhost:6666
waiting for connection...
Client connected!
Waiting for some data from client
Received: "MSG from client"
Received EOF
Sending msg to client: "MSG from server"
sleep(5)
Sending 1 msg to client: "MSG from server"
try #1send()result: (0) Success
Sending 2 msg to client: "MSG from server"
try #2send()result: (32) Broken pipe
Server exiting...Expected results:
Server should fail tosend()data right aftersleep().Server started...
CreatedSOCK_STREAMsocket
Bound to localhost:6666
waiting for connection...
Client connected!
Waiting for some data from client
Received: "MSG from client"
Received EOF
Sending msg to client: "MSG from server"
sleep(5)
Sending 1 msg to client: "MSG from server"
try #1send()result: (32) Broken pipe
Server exiting...Additional info:
RST answer means that data were NOT delivered. Therefore the very firstsend()call after serversleep()should fail!Regards,
Quoted 8 lines
Quoted 1 line
Tomas Hozza
Hi Tomas
I read your bug report, but cant understand why you believe there is a bug.
send()is not synchronous.
OK, my bad. In man page there is: "Successful completion of a call to send() does not guarantee delivery of the message."
Quoted 7 lines
It only queues bytes to be transmitted either right now, either later when its possible.
So a RST message can be received _after_
send()has accepted the request and copy data from user land to kernel socket write queue, but TCP stack did not yet delivered the data.
I'm using TCP because it should guarantee that my data were delivered or let me know there was some problem. If this is not a bug, then it is at least confusing for TCP.
Quoted 1 line
To make sure data is delivered, you need additional logic.
To be honest I didn't find any way how to get notified there was a RST packet sent as a reply to my previously sent data.
Thank you for your reply.
Regards,
Tomas Hozza
Quoted 10 lines
stack did not yet delivered the data.
I'm using TCP because it should guarantee that my data were delivered or let me know there was some problem. If this is not a bug, then it is at least confusing for TCP.
Quoted 1 line
To make sure data is delivered, you need additional logic.
To be honest I didn't find any way how to get notified there was a RST packet sent as a reply to my previously sent data.
TCP makes no end-to-end (application to application) guarantees. Applications that need assurance need to do a request-reply at end of transaction. You should get a good textbook like Stevens for more detail
Quoted 3 lines · Tomas Hozza
I'm using TCP because it should guarantee that my data were delivered or let me know there was some problem. If this is not a bug, then it is at least confusing for TCP.
Note that a write() on a regular file descriptor has same semantic :
By default, there is no guarantee data is written on stable storage.
Quoted 5 lines
Quoted 2 lines
To make sure data is delivered, you need additional logic.
To be honest I didn't find any way how to get notified there was a RST packet sent as a reply to my previously sent data.
Well, I suggest you read the man pages and some books, as this is well explained, you are not the first guy wanting to exchange data using TCP.
man 7 socket
SO_LINGER
Sets or gets the SO_LINGER option. The argument is a linger structure.
struct linger {
int l_onoff; /* linger active */
int l_linger; /* how many seconds to linger for */
};
When enabled, a close(2) or shutdown(2) will not return until all queued messages for the socket have been successfully sent or the linger timeout has been reached. Otherwise, the call returns immediately and the closing is done in the
background. When the socket is closed as part of exit(2), it always lingers in
the background.
man 2 shutdown
...
----- Original Message -----
Quoted 45 lines · Tomas Hozza
Quoted 6 lines
I'm using TCP because it should guarantee that my data were delivered or
let me know there was some problem. If this is not a bug, then it is at least
confusing for TCP.Note that a
write()on a regular file descriptor has same semantic :
By default, there is no guarantee data is written on stable storage.Quoted 6 lines
Quoted 2 lines
To make sure data is delivered, you need additional logic.
To be honest I didn't find any way how to get notified there was a RST packet
sent as a reply to my previously sent data.Well, I suggest you read the man pages and some books, as this is well
explained, you are not the first guy wanting to exchange data using TCP.man 7 socket
SO_LINGER
Sets or gets theSO_LINGERoption. The argument is a linger structure.struct linger {
intl_onoff; /* linger active */
intl_linger; /* how many seconds to linger
for */
};When enabled, a close(2) or shutdown(2) will not return until all queued messages
for the socket have been successfully sent or
the linger timeout has been
reached. Otherwise, the call returns immediately and the closing is done in the
background. When the socket is closed as part of
exit(2), it always lingers in
the background.
Quoted 1 line
man 2 shutdown
I don't think you understood what I was asking for and this is not the right place to discuss how to do things. I have read Unix Network Programming from Richard Stevens but did not find answer for this either. Thanks anyway.