Fix piggybacked ACKs
Subsystems (2)
PATCH 1/1 Fix piggybacked ACKs
This patch corrects the conditions under which a SACK will be piggybacked on a DATA packet. The previous condition was incorrect due to a misinterpretation of RFC 4960 and/or RFC 2960. Specifically, the following paragraph from section 6.2 had not been implemented correctly:
Before an endpoint transmits a DATA chunk, if any received DATA chunks have not been acknowledged (e.g., due to delayed ack), the sender should create a SACK and bundle it with the outbound DATA chunk, as long as the size of the final SCTP packet does not exceed the current MTU. See Section 6.2.
When about to send a DATA chunk, the code now checks to see if the SACK timer is running. If it is, we know we have a SACK to send to the peer, so we append the SACK (assuming available space in the packet) and turn off the timer. For a simple request-response scenario, this will result in the SACK being bundled with the response, meaning the the SACK is received quickly by the client, and also meaning that no separate SACK packet needs to be sent by the server to acknowledge the request. Prior to this patch, a separate SACK packet would have been sent by the server SCTP only after its delayed-ACK timer had expired (usually 200ms). This is wasteful of bandwidth, and can also have a major negative impact on performance due the interaction of delayed ACKs with the Nagle algorithm.
---
net/sctp/output.c +5 -4
@@ -237,18 +237,19 @@ static sctp_xmit_t sctp_packet_bundle_saif (sctp_chunk_is_data (chunk) && !pkt->has_sack && !pkt->has_cookie_echo) {struct sctp_association *asoc; +struct timer_list *timer; asoc = pkt->transport->asoc; + timer = &asoc->timers[SCTP_EVENT_TIMEOUT_SACK ]; -if (asoc->a_rwnd > asoc->rwnd) { +/* If the SACK timer is running, we have a pending SACK */ +if (timer_pending (timer)) {struct sctp_chunk *sack; asoc->a_rwnd = asoc->rwnd; sack =sctp_make_sack (asoc);if (sack) { -struct timer_list *timer; retval =sctp_packet_append_chunk (pkt, sack); asoc->peer.sack_needed =0 ; - timer = &asoc->timers[SCTP_EVENT_TIMEOUT_SACK ]; -if (timer_pending (timer) &&del_timer (timer)) +if (del_timer (timer))sctp_association_put (asoc); } }
Quoted 11 lines · Doug Graham
This patch corrects the conditions under which a SACK will be piggybacked on a DATA packet. The previous condition was incorrect due to a misinterpretation of RFC 4960 and/or RFC 2960. Specifically, the following paragraph from section 6.2 had not been implemented correctly:
Before an endpoint transmits a DATA chunk, if any received DATA chunks have not been acknowledged (e.g., due to delayed ack), the sender should create a SACK and bundle it with the outbound DATA chunk, as long as the size of the final SCTP packet does not exceed the current MTU. See Section 6.2.
The above text said that SACK is create when the size of the final SCTP packet
does not exceed the current MTU. With the patch, what will happend? If the packet is too large for bundle with SACK, the packet sequence will like this:
Endpoint A Endpoint B
DATA ------------->
<------------- SACK
<------------- DATA (size52)
SACK ------------->
DATA (size52)------------->
<------------- SACK
<------------- DATA (size52)The behavior is the same as no delayed ack support. So I think you also need to check the packet size before append the SACK.
Quoted 51 lines
When about to send a DATA chunk, the code now checks to see if the SACK timer is running. If it is, we know we have a SACK to send to the peer, so we append the SACK (assuming available space in the packet) and turn off the timer. For a simple request-response scenario, this will result in the SACK being bundled with the response, meaning the the SACK is received quickly by the client, and also meaning that no separate SACK packet needs to be sent by the server to acknowledge the request. Prior to this patch, a separate SACK packet would have been sent by the server SCTP only after its delayed-ACK timer had expired (usually 200ms). This is wasteful of bandwidth, and can also have a major negative impact on performance due the interaction of delayed ACKs with the Nagle algorithm.
Signed-off-by Doug Graham <[email protected]>---
net/sctp/output.c +5
-5@@ -237,18 +237,19 @@ static sctp_xmit_t sctp_packet_bundle_saif (sctp_chunk_is_data (chunk) && !pkt->has_sack && !pkt->has_cookie_echo) {struct sctp_association *asoc; +struct timer_list *timer; asoc = pkt->transport->asoc; + timer = &asoc->timers[SCTP_EVENT_TIMEOUT_SACK ]; -if (asoc->a_rwnd > asoc->rwnd) { +/* If the SACK timer is running, we have a pending SACK */ +if (timer_pending (timer)) {struct sctp_chunk *sack; asoc->a_rwnd = asoc->rwnd; sack =sctp_make_sack (asoc);if (sack) { -struct timer_list *timer; retval =sctp_packet_append_chunk (pkt, sack); asoc->peer.sack_needed =0 ; - timer = &asoc->timers[SCTP_EVENT_TIMEOUT_SACK ]; -if (timer_pending (timer) &&del_timer (timer)) +if (del_timer (timer))sctp_association_put (asoc); } } --To unsubscribe from this list: send the line "unsubscribe linux-sctp" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 29 lines · Doug Graham
Quoted 5 lines
This patch corrects the conditions under which a SACK will be piggybacked on a DATA packet. The previous condition was incorrect due to a misinterpretation of RFC 4960 and/or RFC 2960. Specifically, the following paragraph from section 6.2 had not been implemented correctly:
Quoted 7 lines
Before an endpoint transmits a DATA chunk, if any received DATA chunks have not been acknowledged (e.g., due to delayed ack), the sender should create a SACK and bundle it with the outbound DATA chunk, as long as the size of the final SCTP packet does not exceed the current MTU. See Section 6.2.
The above text said that SACK is create when the size of the final SCTP packet
does not exceed the current MTU. With the patch, what will happend? If the packet is too large for bundle with SACK, the packet sequence will like this:Endpoint A Endpoint B
DATA ------------->
<------------- SACK <------------- DATA (size52) SACK -------------> DATA (size52)-------------> <------------- SACK <------------- DATA (size52)
Quoted 2 lines
The behavior is the same as no delayed ack support. So I think you also need to check the packet size before append the SACK.
The patch should be like this:
[PATCH] sctp: Do not create SACK chunk if the final packet size exceed current MTU
The sender should create a SACK only if the size of the final SCTP packet does not exceed the current MTU. Base on RFC 4960:
6.1. Transmission of DATA Chunks
Before an endpoint transmits a DATA chunk, if any received DATA chunks have not been acknowledged (e.g., due to delayed ack), the sender should create a SACK and bundle it with the outbound DATA chunk, as long as the size of the final SCTP packet does not exceed the current MTU.
--- net/sctp/output.c | 13 ++++++++----- 1 files changed, 8 insertions(+), 5 deletions(-)
net/sctp/output.c +8 -5
@@ -274,16 +274,19 @@ sctp_xmit_t sctp_packet_append_chunk(struct sctp_packet *packet,if (retval !=SCTP_XMIT_OK ) goto finish; -/* Try to bundle SACK chunk */ - retval =sctp_packet_bundle_sack (packet, chunk); -if (retval !=SCTP_XMIT_OK ) - goto finish; - psize = packet->size; pmtu = ((packet->transport->asoc) ? (packet->transport->asoc->pathmtu) : (packet->transport->pathmtu)); +/* Try to bundle SACK chunk if not exceed the current MTU */ +if (psize + chunk_len +sizeof (struct sctp_sack_chunk ) <= pmtu) { + retval =sctp_packet_bundle_sack (packet, chunk); +if (retval !=SCTP_XMIT_OK ) + goto finish; + psize = packet->size; + } + too_big = (psize + chunk_len > pmtu);/* Decide if we need to fragment or resubmit later. */
Signature · 1 line
1.6.2.2
Quoted 10 lines · Wei Yongjun
The sender should create a SACK only if the size of the final SCTP packet does not exceed the current MTU. Base on RFC 4960:
6.1. Transmission of DATA Chunks
Before an endpoint transmits a DATA chunk, if any received DATA chunks have not been acknowledged (e.g., due to delayed ack), the sender should create a SACK and bundle it with the outbound DATA chunk, as long as the size of the final SCTP packet does not exceed the current MTU.
[patch deleted]
I think you're right that there's a real problem here, and that a patch similar to yours is needed, but this is not a new problem introduced with my patch. I only changed the conditions under which a SACK chunk was bundled with a DATA chunk, but the same bundling would have been happening before under different conditions.
I'd really need to study the code more than I have to be able to say whether or not your fix is correct (and who cares what I think anyway :-)), but I've just spent all of about 15 minutes looking at parts of the code that I'd never looked at before, and I see something that off the top of my head looks a bit scary. That is that sctp_packet_bundle_sack() calls sctp_packet_append_chunk(), which calls sctp_packet_bundle_sack(). Aside from the possibility of infinite recursion (presumably this must be prevented somehow, because it doesn't seem to happen), the logic of this seems strangely circular to me. If bundle_sack() is going to call append_chunk(), I'd have guessed that append_chunk() would be a lower level routine that just appends the chunk you give it, not one that itself tries to bundle other chunks in.
Anyway, you've got me curious now. I'll have a go at better understanding the code when I get some time.
--Doug.
Quoted 20 lines · Doug Graham
Quoted 10 lines · Wei Yongjun
The sender should create a SACK only if the size of the final SCTP packet does not exceed the current MTU. Base on RFC 4960:
6.1. Transmission of DATA Chunks
Before an endpoint transmits a DATA chunk, if any received DATA chunks have not been acknowledged (e.g., due to delayed ack), the sender should create a SACK and bundle it with the outbound DATA chunk, as long as the size of the final SCTP packet does not exceed the current MTU.
[patch deleted]
I think you're right that there's a real problem here, and that a patch similar to yours is needed, but this is not a new problem introduced with my patch. I only changed the conditions under which a SACK chunk was bundled with a DATA chunk, but the same bundling would have been happening before under different conditions.
Quoted 12 lines
I'd really need to study the code more than I have to be able to say whether or not your fix is correct (and who cares what I think anyway :-)), but I've just spent all of about 15 minutes looking at parts of the code that I'd never looked at before, and I see something that off the top of my head looks a bit scary. That is that
sctp_packet_bundle_sack()callssctp_packet_append_chunk(), which callssctp_packet_bundle_sack(). Aside from the possibility of infinite recursion (presumably this must be prevented somehow, because it doesn't seem to happen), the logic of this seems strangely circular to me. Ifbundle_sack()is going to callappend_chunk(), I'd have guessed thatappend_chunk()would be a lower level routine that just appends the chunk you give it, not one that itself tries to bundle other chunks in.
sctp_append_chunk() tries to be very smart. It's a generic routine that you call that also then tries to figure out if any special bundle can also be done.
The circular bundling doesn't happen because the chunk one passes tosctp_packet_bundle_sack() changes. The first time in, the chunk is a data
chunk that makes us go into the whole added the SACK in. At this time, append_chunk() is called with a SACK chunk and a seemingly recursive call to bundle_sack() bails since the chunk is not data.
I think the Wei's patch for mtu checking should probably move into bundle_sack().
-vlad
Quoted 6 lines
Anyway, you've got me curious now. I'll have a go at better understanding the code when I get some time.
--Doug.
Quoted 12 lines · Doug Graham
Quoted 10 lines · Wei Yongjun
The sender should create a SACK only if the size of the final SCTP packet does not exceed the current MTU. Base on RFC 4960:
6.1. Transmission of DATA Chunks
Before an endpoint transmits a DATA chunk, if any received DATA chunks have not been acknowledged (e.g., due to delayed ack), the sender should create a SACK and bundle it with the outbound DATA chunk, as long as the size of the final SCTP packet does not exceed the current MTU.
Quoted 7 lines
[patch deleted]
I think you're right that there's a real problem here, and that a patch similar to yours is needed, but this is not a new problem introduced with my patch. I only changed the conditions under which a SACK chunk was bundled with a DATA chunk, but the same bundling would have been happening before under different conditions.
Doug
If you still have BSD setup, can you try increasing you message size to say 1442 and see what happens.
I'd expect bundles SACKs at 1440 bytes, but then probably a separate SACK and DATA.
-vlad
Quoted 23 lines
I'd really need to study the code more than I have to be able to say whether or not your fix is correct (and who cares what I think anyway :-)), but I've just spent all of about 15 minutes looking at parts of the code that I'd never looked at before, and I see something that off the top of my head looks a bit scary. That is that
sctp_packet_bundle_sack()callssctp_packet_append_chunk(), which callssctp_packet_bundle_sack(). Aside from the possibility of infinite recursion (presumably this must be prevented somehow, because it doesn't seem to happen), the logic of this seems strangely circular to me. Ifbundle_sack()is going to callappend_chunk(), I'd have guessed thatappend_chunk()would be a lower level routine that just appends the chunk you give it, not one that itself tries to bundle other chunks in.Anyway, you've got me curious now. I'll have a go at better understanding the code when I get some time.
--Doug.
--
To unsubscribe from this list: send the line "unsubscribe linux-sctp" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 21 lines · Vlad Yasevich
Quoted 12 lines · Doug Graham
Quoted 10 lines · Wei Yongjun
The sender should create a SACK only if the size of the final SCTP packet does not exceed the current MTU. Base on RFC 4960:
6.1. Transmission of DATA Chunks
Before an endpoint transmits a DATA chunk, if any received DATA chunks have not been acknowledged (e.g., due to delayed ack), the sender should create a SACK and bundle it with the outbound DATA chunk, as long as the size of the final SCTP packet does not exceed the current MTU.
Quoted 7 lines
[patch deleted]
I think you're right that there's a real problem here, and that a patch similar to yours is needed, but this is not a new problem introduced with my patch. I only changed the conditions under which a SACK chunk was bundled with a DATA chunk, but the same bundling would have been happening before under different conditions.
Quoted 6 lines
Doug
If you still have BSD setup, can you try increasing you message size to say 1442 and see what happens.
I'd expect bundles SACKs at 1440 bytes, but then probably a separate SACK and DATA.
The largest amount of data I can send and still have the BSD server bundle a SACK with the response is 1436 bytes. The total ethernet frame size at that point is 1514 bytes, so this seems correct. I've attached wireshark captures with data sizes of 1436 bytes and 1438 bytes. It's interesting to note that if BSD decides not to bundle a SACK, it instead sends a separate SACK packet immediately; it does not wait for the SACK timer to timeout. It first sends the SACK, then the DATA immediately follows. I don't think Wei's patch would do this; I think that if his patch determined that bundling a SACK would cause the packet to exceed the MTU, then the behaviour will revert to what it was before my patch is applied: ie the SACK will not be sent for 200ms.
So I guess the logic when about to send a DATA chunk should go something like:
if (sack_timer_running) {
/* We know we owe the peer a SACK */
if (SACK + DATA fits in the MTU)
bundle SACK with DATA and send that
else {
send SACK in a separate packet
send DATA in a separate packet
}
turn_off_sack_timer
}
I don't think the RFC was explicit on what to do if the SACK+DATA exceeds the MTU, but this makes sense to me.
--Doug.
Ôò¡ ÿÿ C+rJ–! Z Z ¦ø ň. E L @ @„&