Skip to contentSkip to conversation

New: Incorrect SCTP DATA dispatching

Bug 205753 Stephen Hemminger 3 participants 3 messages Dec 3, 2019 – Dec 5, 2019 SCTP protocol

lore
Share & tools
lore
#0 lore raw

All networking bugzilla bugs go to me. This one is new and SCTP related

Begin forwarded message:

Date: Tue, 03 Dec 2019 13:32:39 +0000
From: [email protected]
To: [email protected]
Subject: [Bug 205753] New: Incorrect SCTP DATA dispatching

https://bugzilla.kernel.org/show_bug.cgi?id 5753

            Bug ID: 205753
           Summary: Incorrect SCTP DATA dispatching
           Product: Networking
           Version: 2.5
    Kernel Version: 4.19.67
          Hardware: Intel
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: Other
          Assignee: [email protected]
          Reporter: [email protected]
        Regression: No

I think that root cause in skipped messages.

For example, frame 56 in your debian10_sctp_test_to_sctp_test.pcapng shows that the data message with sctp.data_tsn = 919935042 was received on server interface but it was acked in frame 78 only.

It happens periodically with high-load traffic. The server can capture DATA on network interface but SCTP level does not receive it.

Signature · 2 lines
You are receiving this mail because:
You are the assignee for the bug.
#1 lore raw

Hi Evgeni,

Quoted 28 lines · Stephen Hemminger

All networking bugzilla bugs go to me. This one is new and SCTP related

Begin forwarded message:

Date: Tue, 03 Dec 2019 13:32:39 +0000
From: [email protected]
To: [email protected]
Subject: [Bug 205753] New: Incorrect SCTP DATA dispatching

https://bugzilla.kernel.org/show_bug.cgi?id 5753

            Bug ID: 205753
           Summary: Incorrect SCTP DATA dispatching
           Product: Networking
           Version: 2.5
    Kernel Version: 4.19.67
          Hardware: Intel
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: Other
          Assignee: [email protected]
          Reporter: [email protected]
        Regression: No
Quoted 8 lines

I think that root cause in skipped messages.

For example, frame 56 in your debian10_sctp_test_to_sctp_test.pcapng shows that the data message with sctp.data_tsn = 919935042 was received on server interface but it was acked in frame 78 only.

It happens periodically with high-load traffic. The server can capture DATA on network interface but SCTP level does not receive it.

In my opinion the reason that a SACK for frame 56 is not returned is that the receive buffer is full

$ tshark -r debian10_sctp_test_to_sctp_test.pcapng -Y "sctp.sack_a_rwnd" -T fields -e frame.number -e sctp.sack_a_rwnd
4       106495
7       106423
...
49      105662
53      105591
55      0
57      0
58      13500
Quoted 3 lines

Reproduction scenario:
system1: /usr/bin/sctp_test -H local-addr -P local-port -l
system2: /usr/bin/sctp_test -H local-addr -P local-port -h remote-addr -p remote-port -s -x 1000

I'm not sure whether the sctp_test is the best tool for stress/load tests.

You may try increasing the buffer size (-m 212992) on the server side, see also https://github.com/sctp/lksctp-tools/blob/master/src/apps/sctp_test.c#L1117-L1123

Monitor the buffers sizes and sctp's snmp stats when testing

cat /proc/net/sctp/assocs
cat /proc/net/sctp/snmp

Regards,
Vasil

#2 lore raw
In reply to Stephen Hemminger

Hello Linux SCTP team,

Unfortunately some of you have not received the previous email. I've got response from [email protected]:
550 5.7.1 Content-Policy reject msg: The message contains HTML subpart, therefore we consider it SPAM or Outlook Virus. TEXT/PLAIN is accepted.! BF:<U 0.5>; S1727912AbfLDQJ4
I do not know where HTML subpart is. I wrote email with gmal WEB service. Maybe the problem was with attached file.

The file debian9_sctp_test_to_sctp_test.pcapng can be found here:
https://bugzilla.kernel.org/show_bug.cgi?id 5753

If you know how to prevent this sending error in future please let me know.

I am trying to resend the original message to [email protected]. The original message:

Quoted 84 lines

Hi Vasil,

The issue when I have 1 second without traffic can periodically reproduces under Debian 10. I tried to reproduce it with Debian 9 but everything works fine with the same equipment.

Ok. DATA in frame 56 may be dropped because of buffer is full.
In this case I want to understand why a_rwnd decries rapidly between frames 53-55.

Also I try to understand why a_rwnd is stable enough when the same server side works under Debian 9. Please look at debian9_sctp_test_to_sctp_test.pcapng.

Best Regards,
Evgeny

Quoted 68 lines · Vasil Velichkov

Hi Evgeni,

Quoted 28 lines · Stephen Hemminger

All networking bugzilla bugs go to me. This one is new and SCTP related

Begin forwarded message:

Date: Tue, 03 Dec 2019 13:32:39 +0000
From: [email protected]
To: [email protected]
Subject: [Bug 205753] New: Incorrect SCTP DATA dispatching

https://bugzilla.kernel.org/show_bug.cgi?id 5753

            Bug ID: 205753
           Summary: Incorrect SCTP DATA dispatching
           Product: Networking
           Version: 2.5
    Kernel Version: 4.19.67
          Hardware: Intel
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: Other
          Assignee: [email protected]
          Reporter: [email protected]
        Regression: No
Quoted 8 lines

I think that root cause in skipped messages.

For example, frame 56 in your debian10_sctp_test_to_sctp_test.pcapng shows that the data message with sctp.data_tsn = 919935042 was received on server interface but it was acked in frame 78 only.

It happens periodically with high-load traffic. The server can capture DATA on network interface but SCTP level does not receive it.

In my opinion the reason that a SACK for frame 56 is not returned is that the receive buffer is full

$ tshark -r debian10_sctp_test_to_sctp_test.pcapng -Y "sctp.sack_a_rwnd" -T fields -e frame.number -e sctp.sack_a_rwnd
4       106495
7       106423
...
49      105662
53      105591
55      0
57      0
58      13500
Quoted 3 lines

Reproduction scenario:
system1: /usr/bin/sctp_test -H local-addr -P local-port -l
system2: /usr/bin/sctp_test -H local-addr -P local-port -h remote-addr -p remote-port -s -x 1000

I'm not sure whether the sctp_test is the best tool for stress/load tests.

You may try increasing the buffer size (-m 212992) on the server side, see also https://github.com/sctp/lksctp-tools/blob/master/src/apps/sctp_test.c#L1117-L1123

Monitor the buffers sizes and sctp's snmp stats when testing

cat /proc/net/sctp/assocs
cat /proc/net/sctp/snmp

Regards,
Vasil