Using do_tcp_sendpages() for MPTCP (?)
Christoph Paasch
Hello,
I am trying to rework our handling of the skb-building in tcp_sendmsg (avoiding custom function-pointers for select_size,...) to reduce our impact on regular TCP.
So, I started looking into what it takes to use do_tcp_sendpages when pushing from the MPTCP-level to the subflow-level.
A problem is that do_tcp_sendpages() only works on subflows that have SG-support (see the check in tcp_sendpage_locked()).
So, that would be kind of problematic, no?
I'm also not 100% sure how that is handled for the in-kernel TLS :-/
Thanks,
Christoph
Mat Martineau
Quoted 9 lines · Christoph Paasch
Hello,
I am trying to rework our handling of the skb-building in
tcp_sendmsg(avoiding custom function-pointers forselect_size,...) to reduce our impact on regular TCP.So, I started looking into what it takes to use
do_tcp_sendpageswhen pushing from the MPTCP-level to the subflow-level.
Quoted 6 lines
A problem is that
do_tcp_sendpages()only works on subflows that have SG-support (see the check intcp_sendpage_locked()).So, that would be kind of problematic, no?
I'm also not 100% sure how that is handled for the in-kernel TLS :-/
Hi Christoph -
I'm not quite sure why tcp_sendpage_locked() has that check. It may even force extra copying by GSO. If the net device does not have NETIF_F_SG when an skb is to be sent to the device, the skb will get linearized just before getting pushed to the device (see __dev_queue_xmit() and validate_xmit_skb())
What's more, NETIF_F_SG is not necessarily fixed for a given device over time (see commit message for 113f99c3358564a0647d444c2ae34e8b1abfd5b9), so the stack must be able to handle scatter/gather skbs from do_tcp_sendpages even if the device turns off that feature while those skbs are queued above the device layer.
GSO being always enabled now may also be a factor in not requiring linearized skbs.
This is what I've pieced together this afternoon, anyway - if anyone has better information, please share!
Regards,
--
Mat Martineau
Intel OTC
Christoph Paasch
Hello,
Quoted 33 lines · Mat Martineau
Quoted 9 lines · Christoph Paasch
Hello,
I am trying to rework our handling of the skb-building in
tcp_sendmsg(avoiding custom function-pointers forselect_size,...) to reduce our impact on regular TCP.So, I started looking into what it takes to use
do_tcp_sendpageswhen pushing from the MPTCP-level to the subflow-level.Quoted 6 lines
A problem is that
do_tcp_sendpages()only works on subflows that have SG-support (see the check intcp_sendpage_locked()).So, that would be kind of problematic, no?
I'm also not 100% sure how that is handled for the in-kernel TLS :-/
Hi Christoph -
I'm not quite sure why
tcp_sendpage_locked()has that check. It may even force extra copying by GSO. If the net device does not haveNETIF_F_SGwhen an skb is to be sent to the device, the skb will get linearized just before getting pushed to the device (see__dev_queue_xmit()andvalidate_xmit_skb())What's more,
NETIF_F_SGis not necessarily fixed for a given device over time (see commit message for113f99c3358564a0647d444c2ae34e8b1abfd5b9), so the stack must be able to handle scatter/gather skbs fromdo_tcp_sendpageseven if the device turns off that feature while those skbs are queued above the device layer.
Quoted 5 lines
GSO being always enabled now may also be a factor in not requiring linearized skbs.
This is what I've pieced together this afternoon, anyway - if anyone has better information, please share!
yes you are probably right in the sense that it does not matter now with GSO always-on. I just got confused because tcp_sendmsg is still checking for SG to decide whether or not to do zero-copy.
Thanks,
Christoph