SCTP Multihoming Heartbeat ACK Behavior

TO: Mr. Borkmann

Good day!

Thank you very much for your quick reply.

I need to confirm if your environment there with 2 machines with 5 interfaces each have the same RHEL and LKSCTP versions with the ones we are using?

If you are using the latest RHEL and LKSCTP versions and HB-ACKS are doing fine there, we might think that our older LKSCTP version has something to do with the abnormal HB-ACK behavior.

Thanks,

W. Tizon

My mistake. If in case you will encounter the same Multihoming Heartbeat ACK Behavior using RHEL 6, any information from you would be of great help to us.

Again, thank you for the replies.

Thanks for the reply Neil.

We've just performed test to check if Multihoming Heartbeat ACK Behavior I've mentioned is related to routing table configuration or
SCTP Primary IP address setting. Please see below test information and results.

"Test case1" (Set the primary in "172.168.39.2")

[client]

# sctp_darn -H 172.168.39.2 -B 172.168.39.3 -P 9099 -h 172.168.39.4 -p 
9099 -s

[server]

# sctp_darn -H 172.168.39.4 -B 172.168.39.5 -P 9099 -l

[result]
172.168.39.2(client) 172.168.39.4(server) INIT 172.168.39.4(server) 172.168.39.2(client) INIT_ACK 172.168.39.2(client) 172.168.39.4(server) COOKIE_ECHO 172.168.39.4(server) 172.168.39.2(client) COOKIE_ACK 172.168.39.4(server) 172.168.39.2(client) HB 172.168.39.2(client) 172.168.39.4(server) HB_ACK 172.168.39.4(server) 172.168.39.3(client) HB
172.168.39.2(client) 172.168.39.4(server) HB_ACK (***)
172.168.39.2(client) 172.168.39.4(server) DATA 172.168.39.4(server) 172.168.39.2(client) SACK

"Test case2" (Set the primary in "172.168.39.3")

[client]

# sctp_darn -H 172.168.39.3 -B 172.168.39.2 -P 9099 -h 172.168.39.4 -p 
9099 -s

[server]

# sctp_darn -H 172.168.39.4 -B 172.168.39.5 -P 9099 -l

[result]
172.168.39.3(client) 172.168.39.4(server) INIT 172.168.39.4(server) 172.168.39.3(client) INIT_ACK 172.168.39.3(client) 172.168.39.4(server) COOKIE_ECHO 172.168.39.4(server) 172.168.39.3(client) COOKIE_ACK 172.168.39.4(server) 172.168.39.3(client) HB 172.168.39.3(client) 172.168.39.4(server) HB_ACK 172.168.39.4(server) 172.168.39.2(client) HB
172.168.39.3(client) 172.168.39.4(server) HB_ACK (***)
172.168.39.3(client) 172.168.39.4(server) DATA 172.168.39.4(server) 172.168.39.3(client) SACK

Based on results, SCTP kernel always choose Primary IP address so I think this is not related to routing table configuration problems.

(***) -> Based on SCTP RFC4960, expected behavior is secondary IP address
should be used as path in sending the HB_ACK.

Wins

TO: Neil, Daniel, Michael

Thank you very much for your answers. I misunderstood the SCTP RFC statement regarding reply Chunks in a multi-homed association. I thought reply chunks should be sent always to the same path from where DATA or Control chunks were received.

I'm sorry if I haven't explained yet the other details of our problem using the following "Environment Setup" below.

CLIENT SERVER

 172.168.39.91                172.168.40.93  
 +-----------+    +------+    +-----------+  
 |       eth0|----|  L2  |----|eth0       |  
 |           |    +------+    |           |  
 |           |       |        |           |  
 |           |  +----------+  |           |  
 |           |  |    L3    |  |           |  
 |           |  +----------+  |           |  
 |           |       |        |           |  
 |           |    +------+    |           |  
 |       eth1|----|  L2  |----|eth1       |  
 +-----------+    +------+    +-----------+  
 172.168.39.92                172.168.40.94  

Our problem goes like this. Normal sequence is shown in "[NORMAL]" sequence. While our problem started when we've performed "[ABNORMAL]"sequence below.

[NORMAL]

SERVER L2 and L3 CLIENT

Secondary Primary           |      Primary Secondary 
    |        |              |          |       |     
    |        |              |          |       |     
    |        |INIT-INIT_ACK |          |       |     
    |        |<-------------|--------->|       |     
    |        |COOKIE_ECHO-COOKIE_ACK   |       |     
    |        |              |          |       |     
    |        |<-------------|--------->|       |     
    |        |HB/HB_ACK     |          |       |     
    |        |              |          |       |     
    |<-------|--------------|----------|       |     
    |        |              |       HB |       |     
    |        |--------------|--------->|       |     
    |        |HB_ACK        |          |       |     
    |        |        :     |          |       |     
    |        |        :     |          |       |     

[ABNORMAL]

SERVER L2 and L3 CLIENT

Secondary Primary           |      Primary Secondary 
    |        |              |          |       |     
    |        |              |          |       |     
    |        |INIT-INIT_ACK |          |       |     
    |        |<-------------|--------->|       |     
    |        |COOKIE_ECHO-COOKIE_ACK   |       |     
    |        |              |          |       |     
    |        |<-------------|--------->|       |     
    |        |HB/HB_ACK     |          |       |     
    |        |              |          |       |     
    |<-------|--------------|----------|       |     
    |        |              |       HB |       |     
    |        |---->X        |          |       |     
    |        |HB_ACK        |          |       |     
    |        |        :     |          |       |     
    |<-------|--------------|----------|       |     

ABORT

X -> LAN cable unplug, cannot send HB_ACK to Client

Based on what happened in our environment, the Client sent ABORT chunk to Secondary IP address of Server causing this path to be closed.

We don't want this to happen in our environment.

It would be of great help to us if you have any idea on how we could solve question #3 below.

[Question]
3. Is there a solution to force Server to use its Secondary path in sending the HEARTBEAT_ACK chunk to Client's primary?

Wins