sctp: add another two stream schedulers
Subsystems (3)
Xin Long
maintainercover sctp: add another two stream schedulers
All SCTP stream schedulers are defined in rfc8260#section-3, First-Come First-Served, Round-Robin and Priority-Based Schedulers are already added in kernel.
This patchset adds another two schedulers: Fair Capacity Scheduler and Weighted Fair Queueing Scheduler.
Note that the left one "Round-Robin Scheduler per Packet" Scheduler is not implemented by this patch, as it's still intrusive to be added in the current SCTP kernel code.
Xin Long (2):
sctp: add fair capacity stream scheduler
sctp: add weighted fair queueing stream scheduler
include/net/sctp/stream_sched.h | 2 + include/net/sctp/structs.h | 8 ++ include/uapi/linux/sctp.h | 4 +- net/sctp/Makefile | 3 +- net/sctp/stream_sched.c | 2 + net/sctp/stream_sched_fc.c | 225 ++++++++++++++++++++++++++++++++ 6 files changed, 242 insertions(+), 2 deletions(-) create mode 100644 net/sctp/stream_sched_fc.c
Signature · 1 line
2.39.1
Xin Long
maintainerPATCH 1/2 sctp: add fair capacity stream scheduler
As it says in rfc8260#section-3.5 about the fair capacity scheduler:
A fair capacity distribution between the streams is used. This scheduler considers the lengths of the messages of each stream and schedules them in a specific way to maintain an equal capacity for all streams. The details are implementation dependent. interleaving user messages allows for a better realization of the fair capacity usage.
This patch adds Fair Capacity Scheduler based on the foundations added by commit 5bbbbe32a431 ("sctp: introduce stream scheduler foundations"):
A fc_list and a fc_length are added into struct sctp_stream_out_ext and a fc_list is added into struct sctp_stream. In .enqueue, when there are chunks enqueued into a stream, this stream will be linked into stream-> fc_list by its fc_list ordered by its fc_length. In .dequeue, it always picks up the 1st skb from stream->fc_list. In .dequeue_done, fc_length is increased by chunk's len and update its location in stream->fc_list according to the its new fc_length.
Note that when the new fc_length overflows in .dequeue_done, instead of resetting all fc_lengths to 0, we only reduced them by U32_MAX / 4 to avoid a moment of imbalance in the scheduling, as Marcelo suggested.
--- include/net/sctp/stream_sched.h | 1 + include/net/sctp/structs.h | 7 ++ include/uapi/linux/sctp.h | 3 +- net/sctp/Makefile | 3 +- net/sctp/stream_sched.c | 1 + net/sctp/stream_sched_fc.c | 183 ++++++++++++++++++++++++++++++++ 6 files changed, 196 insertions(+), 2 deletions(-) create mode 100644 net/sctp/stream_sched_fc.c
include/net/sctp/stream_sched.h +1 -0
@@ -58,5 +58,6 @@ void sctp_sched_ops_register(enum sctp_sched_type sched, struct sctp_sched_ops *sched_ops);void sctp_sched_ops_prio_init (void );void sctp_sched_ops_rr_init (void ); +void sctp_sched_ops_fc_init (void );#endif /* __sctp_stream_sched_h__ */
include/net/sctp/structs.h +7 -0
@@ -1429,6 +1429,10 @@ struct sctp_stream_out_ext {struct {struct list_head rr_list; }; +struct { +struct list_head fc_list; +__u32 fc_length; + }; }; };
@@ -1475,6 +1479,9 @@ struct sctp_stream {/* The next stream in line */ struct sctp_stream_out_ext *rr_next; }; +struct { +struct list_head fc_list; + }; };struct sctp_stream_interleave *si; };
include/uapi/linux/sctp.h +2 -1
@@ -1211,7 +1211,8 @@ enum sctp_sched_type {SCTP_SS_DEFAULT =SCTP_SS_FCFS ,SCTP_SS_PRIO ,SCTP_SS_RR , - SCTP_SS_MAX = SCTP_SS_RR +SCTP_SS_FC , + SCTP_SS_MAX = SCTP_SS_FC };/* Probe Interval socket option */
net/sctp/Makefile +2 -1
@@ -13,7 +13,8 @@ sctp-y := sm_statetable.o sm_statefuns.o sm_sideeffect.o \ tsnmap.o bind_addr.o socket.o primitive.o \ output.o input.o debug.o stream.o auth.o \ offload.o stream_sched.o stream_sched_prio.o \ - stream_sched_rr.o stream_interleave.o + stream_sched_rr.o stream_sched_fc.o \ + stream_interleave.o sctp_diag-y := diag.o
net/sctp/stream_sched.c +1 -0
@@ -124,6 +124,7 @@ void sctp_sched_ops_init(void)sctp_sched_ops_fcfs_init ();sctp_sched_ops_prio_init ();sctp_sched_ops_rr_init (); +sctp_sched_ops_fc_init (); }static void sctp_sched_free_sched (struct sctp_stream *stream)
net/sctp/stream_sched_fc.c new file +183 -0
183 lines not on this page · raw message
Signature · 1 line
2.39.1
Re: [PATCH net-next 1/2] sctp: add fair capacity stream scheduler
Quoted 20 lines · Xin Long
As it says in rfc8260#section-3.5 about the fair capacity scheduler:
A fair capacity distribution between the streams is used. This scheduler considers the lengths of the messages of each stream and schedules them in a specific way to maintain an equal capacity for all streams. The details are implementation dependent. interleaving user messages allows for a better realization of the fair capacity usage.
This patch adds Fair Capacity Scheduler based on the foundations added by commit
5bbbbe32a431("sctp: introduce stream scheduler foundations"):A
fc_listand afc_lengthare added into structsctp_stream_out_extand afc_listis added into structsctp_stream. In .enqueue, when there are chunks enqueued into a stream, this stream will be linked into stream->fc_listby itsfc_listordered by itsfc_length. In .dequeue, it always picks up the 1st skb fromstream->fc_list. In .dequeue_done,fc_lengthis increased by chunk's len and update its location instream->fc_listaccording to the its newfc_length.
Quoted 5 lines
Note that when the new
fc_lengthoverflows in .dequeue_done, instead of resetting allfc_lengthsto 0, we only reduced them byU32_MAX/ 4 to avoid a moment of imbalance in the scheduling, as Marcelo suggested.Signed-off-by Xin Long <[email protected]>
Paolo Abeni
maintainerRe: [PATCH net-next 1/2] sctp: add fair capacity stream scheduler
In reply to Xin LongQuoted 14 lines · Xin Long
net/sctp/stream_sched_fc.c new file +8
-0@@ -0,0 +1,183 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* SCTP kernel implementation +* (C) Copyright Red Hat Inc. 2022 +* +* This file is part of the SCTP kernel implementation +* +* These functions manipulate sctp stream queue/scheduling. +*
net/sctp/stream_sched_fc.c line 18
Quoted 10 lines
net/sctp/stream_sched_fc.c new file +10
-0+* Please send any bug reports or fixes you make to the +* email addresched(es): +* lksctp developers <[email protected]> +* +* Written or modified by: +* Xin Long <[email protected]> +*/ + +#include <linux/list.h> +#include <net/sctp/sctp.h>
Note that the above introduces a new compile warning:
../net/sctp/stream_sched_fc.c: note: in included file (through ../include/net/sctp/sctp.h):../include/net/sctp/structs.h:335:41: warning: array of flexible structures
that is not really a fault of the new code, it's due to:
struct sctp_init_chunk peer_init[];
struct sctp_init_chunk {
struct sctp_chunkhdr chunk_hdr;
struct sctp_inithdr init_hdr;
};struct sctp_inithdr {
__be32 init_tag;
__be32 a_rwnd;
__be16 num_outbound_streams;
__be16 num_inbound_streams;
__be32 initial_tsn;
__u8 params[]; // <- this!
};Any chance a future patch could remove the 'params' field from the struct included by sctp_init_chunk?
e.g.
struct sctp_inithdr_base {
__be32 init_tag;
__be32 a_rwnd;
__be16 num_outbound_streams;
__be16 num_inbound_streams;
__be32 initial_tsn;
};struct sctp_init_chunk {
struct sctp_chunkhdr chunk_hdr;
struct sctp_inithdr_base init_hdr;
};and then cast 'init_hdr' to 'struct sctp_inithdr' if/where needed.
In any case, the above is not blocking this series.
Cheers,
Paolo
Xin Long
maintainerRe: [PATCH net-next 1/2] sctp: add fair capacity stream scheduler
Quoted 63 lines · Paolo Abeni
Quoted 14 lines · Xin Long
net/sctp/stream_sched_fc.c new file +8
-0@@ -0,0 +1,183 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* SCTP kernel implementation +* (C) Copyright Red Hat Inc. 2022 +* +* This file is part of the SCTP kernel implementation +* +* These functions manipulate sctp stream queue/scheduling. +* net/sctp/stream_sched_fc.c line 18
Quoted 10 lines
net/sctp/stream_sched_fc.c new file +10
-0+* Please send any bug reports or fixes you make to the +* email addresched(es): +* lksctp developers <[email protected]> +* +* Written or modified by: +* Xin Long <[email protected]> +*/ + +#include <linux/list.h> +#include <net/sctp/sctp.h> Note that the above introduces a new compile warning:
../net/sctp/stream_sched_fc.c: note: in included file (through../include/net/sctp/sctp.h):../include/net/sctp/structs.h:335:41: warning: array of flexible structuresthat is not really a fault of the new code, it's due to:
struct sctp_init_chunk peer_init[];struct sctp_init_chunk { struct sctp_chunkhdr chunk_hdr; struct sctp_inithdr init_hdr; };struct sctp_inithdr { __be32 init_tag; __be32 a_rwnd; __be16 num_outbound_streams; __be16 num_inbound_streams; __be32 initial_tsn; __u8 params[]; // <- this! };Any chance a future patch could remove the 'params' field from the struct included by
sctp_init_chunk?e.g.
struct sctp_inithdr_base { __be32 init_tag; __be32 a_rwnd; __be16 num_outbound_streams; __be16 num_inbound_streams; __be32 initial_tsn; };
Quoted 9 lines
struct sctp_init_chunk { struct sctp_chunkhdr chunk_hdr; struct sctp_inithdr_base init_hdr; };and then cast '
init_hdr' to 'structsctp_inithdr' if/where needed.In any case, the above is not blocking this series.
Hi, Paolo,
There are actually quite some warnings like this in SCTP:
# make C=1 CF="-Wflexible-array-nested" M=./net/sctp/ 2> /tmp/warnings # grep "nested flexible array" /tmp/warnings |grep sctp |sort |uniq
./include/linux/sctp.h:230:29: warning: nested flexible array ./include/linux/sctp.h:278:29: warning: nested flexible array ./include/linux/sctp.h:348:29: warning: nested flexible array ./include/linux/sctp.h:393:29: warning: nested flexible array ./include/linux/sctp.h:451:28: warning: nested flexible array ./include/linux/sctp.h:611:32: warning: nested flexible array ./include/linux/sctp.h:628:33: warning: nested flexible array ./include/linux/sctp.h:675:30: warning: nested flexible array ./include/linux/sctp.h:735:29: warning: nested flexible array ./include/net/sctp/structs.h:1145:41: warning: nested flexible array ./include/net/sctp/structs.h:1588:28: warning: nested flexible array ./include/net/sctp/structs.h:343:28: warning: nested flexible array ./include/uapi/linux/sctp.h:641:34: warning: nested flexible array ./include/uapi/linux/sctp.h:643:34: warning: nested flexible array ./include/uapi/linux/sctp.h:644:33: warning: nested flexible array ./include/uapi/linux/sctp.h:650:40: warning: nested flexible array ./include/uapi/linux/sctp.h:653:39: warning: nested flexible array
Other than the uapis, I can try to give others a cleanup by deleting these flexible array members and casting it by (struct xxx *) (hdr + 1) when accessing it.
This warning is reported if a structure having a flexible array member is included by other structures, and I also noticed there is the same problem on some core structures like:
struct ip_options
struct nft_set_ext
which are included in many other structures, and can also cause such warnings. I guess we'll just leave it as it is, right?
Thanks.
Xin Long
maintainerPATCH 2/2 sctp: add weighted fair queueing stream scheduler
In reply to Xin LongAs it says in rfc8260#section-3.6 about the weighted fair queueing scheduler:
A Weighted Fair Queueing scheduler between the streams is used. The weight is configurable per outgoing SCTP stream. This scheduler considers the lengths of the messages of each stream and schedules them in a specific way to use the capacity according to the given weights. If the weight of stream S1 is n times the weight of stream S2, the scheduler should assign to stream S1 n times the capacity it assigns to stream S2. The details are implementation dependent. Interleaving user messages allows for a better realization of the capacity usage according to the given weights.
This patch adds Weighted Fair Queueing Scheduler actually based on the code of Fair Capacity Scheduler by adding fc_weight into struct sctp_stream_out_ext and taking it into account when sorting stream-> fc_list in sctp_sched_fc_sched() and sctp_sched_fc_dequeue_done().
--- include/net/sctp/stream_sched.h | 1 + include/net/sctp/structs.h | 1 + include/uapi/linux/sctp.h | 3 +- net/sctp/stream_sched.c | 1 + net/sctp/stream_sched_fc.c | 50 ++++++++++++++++++++++++++++++--- 5 files changed, 51 insertions(+), 5 deletions(-)
include/net/sctp/stream_sched.h +1 -0
@@ -59,5 +59,6 @@ void sctp_sched_ops_register(enum sctp_sched_type sched,void sctp_sched_ops_prio_init (void );void sctp_sched_ops_rr_init (void );void sctp_sched_ops_fc_init (void ); +void sctp_sched_ops_wfq_init (void );#endif /* __sctp_stream_sched_h__ */
include/net/sctp/structs.h +1 -0
@@ -1432,6 +1432,7 @@ struct sctp_stream_out_ext {struct {struct list_head fc_list;__u32 fc_length; +__u16 fc_weight; }; }; };
include/uapi/linux/sctp.h +2 -1
@@ -1212,7 +1212,8 @@ enum sctp_sched_type {SCTP_SS_PRIO ,SCTP_SS_RR ,SCTP_SS_FC , - SCTP_SS_MAX = SCTP_SS_FC +SCTP_SS_WFQ , + SCTP_SS_MAX = SCTP_SS_WFQ };/* Probe Interval socket option */
net/sctp/stream_sched.c +1 -0
@@ -125,6 +125,7 @@ void sctp_sched_ops_init(void)sctp_sched_ops_prio_init ();sctp_sched_ops_rr_init ();sctp_sched_ops_fc_init (); +sctp_sched_ops_wfq_init (); }static void sctp_sched_free_sched (struct sctp_stream *stream)
net/sctp/stream_sched_fc.c +46 -4
80 lines not on this page · raw message
Signature · 1 line
2.39.1
Re: [PATCH net-next 2/2] sctp: add weighted fair queueing stream scheduler
Quoted 13 lines · Xin Long
As it says in rfc8260#section-3.6 about the weighted fair queueing scheduler:
A Weighted Fair Queueing scheduler between the streams is used. The weight is configurable per outgoing SCTP stream. This scheduler considers the lengths of the messages of each stream and schedules them in a specific way to use the capacity according to the given weights. If the weight of stream S1 is n times the weight of stream S2, the scheduler should assign to stream S1 n times the capacity it assigns to stream S2. The details are implementation dependent. Interleaving user messages allows for a better realization of the capacity usage according to the given weights.
Quoted 6 lines
This patch adds Weighted Fair Queueing Scheduler actually based on the code of Fair Capacity Scheduler by adding
fc_weightinto structsctp_stream_out_extand taking it into account when sorting stream->fc_listinsctp_sched_fc_sched()andsctp_sched_fc_dequeue_done().Signed-off-by Xin Long <[email protected]>
This series was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:
Quoted 8 lines · On Tue, 7 Mar 2023 16:23:25 -0500 you wrote:
All SCTP stream schedulers are defined in rfc8260#section-3, First-Come First-Served, Round-Robin and Priority-Based Schedulers are already added in kernel.
This patchset adds another two schedulers: Fair Capacity Scheduler and Weighted Fair Queueing Scheduler.
[...]
Here is the summary with links:
- [net-next,1/2] sctp: add fair capacity stream scheduler
https://git.kernel.org/netdev/net-next/c/4821a076eb60 - [net-next,2/2] sctp: add weighted fair queueing stream scheduler https://git.kernel.org/netdev/net-next/c/42d452e7709f
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html