Skip to contentSkip to conversation

Using do_tcp_sendpages() for MPTCP (?)

MPTCP Christoph Paasch 2 participants 3 messages Sep 27, 2018 – Sep 28, 2018 Networking MPTCP

lore
Share & tools
lore

Christoph Paasch

author
#0 lore raw

Hello,

I am trying to rework our handling of the skb-building in tcp_sendmsg (avoiding custom function-pointers for select_size,...) to reduce our impact on regular TCP.

So, I started looking into what it takes to use do_tcp_sendpages when pushing from the MPTCP-level to the subflow-level.

A problem is that do_tcp_sendpages() only works on subflows that have SG-support (see the check in tcp_sendpage_locked()).

So, that would be kind of problematic, no?

I'm also not 100% sure how that is handled for the in-kernel TLS :-/

Thanks,
Christoph

#1 lore raw
Quoted 9 lines · Christoph Paasch

Hello,

I am trying to rework our handling of the skb-building in tcp_sendmsg (avoiding custom function-pointers for select_size,...) to reduce our impact on regular TCP.

So, I started looking into what it takes to use do_tcp_sendpages when pushing from the MPTCP-level to the subflow-level.

Quoted 6 lines

A problem is that do_tcp_sendpages() only works on subflows that have SG-support (see the check in tcp_sendpage_locked()).

So, that would be kind of problematic, no?

I'm also not 100% sure how that is handled for the in-kernel TLS :-/

Hi Christoph -

I'm not quite sure why tcp_sendpage_locked() has that check. It may even force extra copying by GSO. If the net device does not have NETIF_F_SG when an skb is to be sent to the device, the skb will get linearized just before getting pushed to the device (see __dev_queue_xmit() and validate_xmit_skb())

What's more, NETIF_F_SG is not necessarily fixed for a given device over time (see commit message for 113f99c3358564a0647d444c2ae34e8b1abfd5b9), so the stack must be able to handle scatter/gather skbs from do_tcp_sendpages even if the device turns off that feature while those skbs are queued above the device layer.

GSO being always enabled now may also be a factor in not requiring linearized skbs.

This is what I've pieced together this afternoon, anyway - if anyone has better information, please share!

Regards,

--
Mat Martineau
Intel OTC

author
#2 lore raw

Hello,

Quoted 33 lines · Mat Martineau
Quoted 9 lines · Christoph Paasch

Hello,

I am trying to rework our handling of the skb-building in tcp_sendmsg (avoiding custom function-pointers for select_size,...) to reduce our impact on regular TCP.

So, I started looking into what it takes to use do_tcp_sendpages when pushing from the MPTCP-level to the subflow-level.

Quoted 6 lines

A problem is that do_tcp_sendpages() only works on subflows that have SG-support (see the check in tcp_sendpage_locked()).

So, that would be kind of problematic, no?

I'm also not 100% sure how that is handled for the in-kernel TLS :-/

Hi Christoph -

I'm not quite sure why tcp_sendpage_locked() has that check. It may even force extra copying by GSO. If the net device does not have NETIF_F_SG when an skb is to be sent to the device, the skb will get linearized just before getting pushed to the device (see __dev_queue_xmit() and validate_xmit_skb())

What's more, NETIF_F_SG is not necessarily fixed for a given device over time (see commit message for 113f99c3358564a0647d444c2ae34e8b1abfd5b9), so the stack must be able to handle scatter/gather skbs from do_tcp_sendpages even if the device turns off that feature while those skbs are queued above the device layer.

Quoted 5 lines

GSO being always enabled now may also be a factor in not requiring linearized skbs.

This is what I've pieced together this afternoon, anyway - if anyone has better information, please share!

yes you are probably right in the sense that it does not matter now with GSO always-on. I just got confused because tcp_sendmsg is still checking for SG to decide whether or not to do zero-copy.

Thanks,
Christoph