net: Fix IP_UNICAST_IF option behavior for connected sockets

PATCH Superseded Richard Gobert 4 participants 9 messages Jun 27, 2022 – Jul 13, 2022

Subsystems (2)

Networking Network drivers

lore

Share & tools

lore

Richard Gobert

author

2022-06-27 08:54 UTC #0 lore raw

PATCH 1/1 net: Fix IP_UNICAST_IF option behavior for connected sockets

The IP_UNICAST_IF socket option is used to set the outgoing interface for outbound packets.
The IP_UNICAST_IF socket option was added as it was needed by the Wine project, since no other existing option (SO_BINDTODEVICE socket option, IP_PKTINFO socket option or the bind function) provided the needed characteristics needed by the IP_UNICAST_IF socket option. [1] The IP_UNICAST_IF socket option works well for unconnected sockets, that is, the interface specified by the IP_UNICAST_IF socket option is taken into consideration in the route lookup process when a packet is being sent.
However, for connected sockets, the outbound interface is chosen when connecting the socket, and in the route lookup process which is done when a packet is being sent, the interface specified by the IP_UNICAST_IF socket option is being ignored.

This inconsistent behavior was reported and discussed in an issue opened on systemd's GitHub project [2]. Also, a bug report was submitted in the kernel's bugzilla [3].

To understand the problem in more detail, we can look at what happens for UDP packets over IPv4 (The same analysis was done separately in the referenced systemd issue).
When a UDP packet is sent the udp_sendmsg function gets called and the following happens:

1. The oif member of the struct ipcm_cookie ipc (which stores the output interface of the packet) is initialized by the ipcm_init_sk function to inet->sk.sk_bound_dev_if (the device set by the SO_BINDTODEVICE socket option).

2. If the IP_PKTINFO socket option was set, the oif member gets overridden by the call to the ip_cmsg_send function.

3. If no output interface was selected yet, the interface specified by the IP_UNICAST_IF socket option is used.

4. If the socket is connected and no destination address is specified in the send function, the struct ipcm_cookie ipc is not taken into consideration and the cached route, that was calculated in the connect function is being used.

Thus, for a connected socket, the IP_UNICAST_IF sockopt isn't taken into consideration.

This patch corrects the behavior of the IP_UNICAST_IF socket option for connect()ed sockets by taking into consideration the IP_UNICAST_IF sockopt when connecting the socket.

In order to avoid reconnecting the socket, this option is still ignored when applied on an already connected socket until connect() is called again by the user.

Change the __ip4_datagram_connect function, which is called during socket connection, to take into consideration the interface set by the IP_UNICAST_IF socket option, in a similar way to what is done in the udp_sendmsg function.

[1] https://lore.kernel.org/netdev/1328685717.4736.4.camel@edumazet-laptop/T/
[2] https://github.com/systemd/systemd/issues/11935#issuecomment-618691018
[3] https://bugzilla.kernel.org/show_bug.cgi?id=210255

Signed-off-by Richard Gobert <[email protected]>

---
 net/ipv4/datagram.c | 2 ++
 1 file changed, 2 insertions(+)

net/ipv4/datagram.c +2 -0

index ffd57523331f..405a8c2aea64 100644
@@ -42,6 +42,8 @@ int __ip4_datagram_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len
 			oif = inet->mc_index;
 		if (!saddr)
 			saddr = inet->mc_addr;
+	} else if (!oif) {
+		oif = inet->uc_index;
 	}
 	fl4 = &inet->cork.fl.u.ip4;
 	rt = ip_route_connect(fl4, usin->sin_addr.s_addr, saddr,

Signature · 1 line

2.36.1

Paolo Abeni

maintainer

2022-06-28 13:09 UTC #1 lore raw

Quoted 41 lines · Richard Gobert

The IP_UNICAST_IF socket option is used to set the outgoing interface for outbound packets.
The IP_UNICAST_IF socket option was added as it was needed by the Wine project, since no other existing option (SO_BINDTODEVICE socket option, IP_PKTINFO socket option or the bind function) provided the needed characteristics needed by the IP_UNICAST_IF socket option. [1] The IP_UNICAST_IF socket option works well for unconnected sockets, that is, the interface specified by the IP_UNICAST_IF socket option is taken into consideration in the route lookup process when a packet is being sent.
However, for connected sockets, the outbound interface is chosen when connecting the socket, and in the route lookup process which is done when a packet is being sent, the interface specified by the IP_UNICAST_IF socket option is being ignored.
This inconsistent behavior was reported and discussed in an issue opened on systemd's GitHub project [2]. Also, a bug report was submitted in the kernel's bugzilla [3].
To understand the problem in more detail, we can look at what happens for UDP packets over IPv4 (The same analysis was done separately in the referenced systemd issue).
When a UDP packet is sent the udp_sendmsg function gets called and the following happens:
1. The oif member of the struct ipcm_cookie ipc (which stores the output interface of the packet) is initialized by the ipcm_init_sk function to inet->sk.sk_bound_dev_if (the device set by the SO_BINDTODEVICE socket option).
2. If the IP_PKTINFO socket option was set, the oif member gets overridden by the call to the ip_cmsg_send function.
3. If no output interface was selected yet, the interface specified by the IP_UNICAST_IF socket option is used.
4. If the socket is connected and no destination address is specified in the send function, the struct ipcm_cookie ipc is not taken into consideration and the cached route, that was calculated in the connect function is being used.

Quoted 6 lines

Thus, for a connected socket, the IP_UNICAST_IF sockopt isn't taken into consideration.
This patch corrects the behavior of the IP_UNICAST_IF socket option for connect()ed sockets by taking into consideration the IP_UNICAST_IF sockopt when connecting the socket.

This also changes a long-established behavior for such socket option. It can break existing application assuming connect() is not affected by IP_UNICAST_IF. I'm unsure we can accept it.

Cheers,

Paolo

Richard Gobert

author

2022-07-05 15:53 UTC #2 lore raw

Quoted 3 lines · Paolo Abeni

This also changes a long-established behavior for such socket option. It can break existing application assuming connect() is not affected by IP_UNICAST_IF. I'm unsure we can accept it.

The IP_UNICAST_IF option was initially introduced for better compatibility with the matching Windows socket-option. Its goal was better support for wine applications.
This patch improves the compatibility even further since Windows behaves this way for connect()ed sockets.

Also, I have not been able to find any examples of Linux applications that use IP_UNICAST_IF with connect(). It would be quite confusing to use this sockopt and expect that it would not affect your socket. I think that unless someone finds an example of such a use case, then it is better to accept this patch to improve compatibility for applications that run with wine.

What are your thoughts on this?

Regards,
Richard

David Ahern

2022-07-06 15:14 UTC #3 lore raw

Quoted 11 lines · Richard Gobert

Quoted 3 lines · Paolo Abeni
This also changes a long-established behavior for such socket option. It can break existing application assuming connect() is not affected by IP_UNICAST_IF. I'm unsure we can accept it.
The IP_UNICAST_IF option was initially introduced for better compatibility with the matching Windows socket-option. Its goal was better support for wine applications.
This patch improves the compatibility even further since Windows behaves this way for connect()ed sockets.

Quoted 9 lines

Also, I have not been able to find any examples of Linux applications that use IP_UNICAST_IF with connect(). It would be quite confusing to use this sockopt and expect that it would not affect your socket. I think that unless someone finds an example of such a use case, then it is better to accept this patch to improve compatibility for applications that run with wine.
What are your thoughts on this?

I can't imagine how a 'connected' socket would propelry work if connect path does not consider oif and then per message does. i.e, i think the patch has some risk but is the right thing to do.

David Ahern

Reviewed-by on v1 1/1

2022-07-06 15:14 UTC #4 lore raw

In reply to Richard Gobert

Quoted 57 lines · Richard Gobert

The IP_UNICAST_IF socket option is used to set the outgoing interface for outbound packets.
The IP_UNICAST_IF socket option was added as it was needed by the Wine project, since no other existing option (SO_BINDTODEVICE socket option, IP_PKTINFO socket option or the bind function) provided the needed characteristics needed by the IP_UNICAST_IF socket option. [1] The IP_UNICAST_IF socket option works well for unconnected sockets, that is, the interface specified by the IP_UNICAST_IF socket option is taken into consideration in the route lookup process when a packet is being sent.
However, for connected sockets, the outbound interface is chosen when connecting the socket, and in the route lookup process which is done when a packet is being sent, the interface specified by the IP_UNICAST_IF socket option is being ignored.
This inconsistent behavior was reported and discussed in an issue opened on systemd's GitHub project [2]. Also, a bug report was submitted in the kernel's bugzilla [3].
To understand the problem in more detail, we can look at what happens for UDP packets over IPv4 (The same analysis was done separately in the referenced systemd issue).
When a UDP packet is sent the udp_sendmsg function gets called and the following happens:
1. The oif member of the struct ipcm_cookie ipc (which stores the output interface of the packet) is initialized by the ipcm_init_sk function to inet->sk.sk_bound_dev_if (the device set by the SO_BINDTODEVICE socket option).
2. If the IP_PKTINFO socket option was set, the oif member gets overridden by the call to the ip_cmsg_send function.
3. If no output interface was selected yet, the interface specified by the IP_UNICAST_IF socket option is used.
4. If the socket is connected and no destination address is specified in the send function, the struct ipcm_cookie ipc is not taken into consideration and the cached route, that was calculated in the connect function is being used.
Thus, for a connected socket, the IP_UNICAST_IF sockopt isn't taken into consideration.
This patch corrects the behavior of the IP_UNICAST_IF socket option for connect()ed sockets by taking into consideration the IP_UNICAST_IF sockopt when connecting the socket.
In order to avoid reconnecting the socket, this option is still ignored when applied on an already connected socket until connect() is called again by the user.
Change the __ip4_datagram_connect function, which is called during socket connection, to take into consideration the interface set by the IP_UNICAST_IF socket option, in a similar way to what is done in the udp_sendmsg function.

Quoted 9 lines

[1] https://lore.kernel.org/netdev/1328685717.4736.4.camel@edumazet-laptop/T/
[2] https://github.com/systemd/systemd/issues/11935#issuecomment-618691018
[3] https://bugzilla.kernel.org/show_bug.cgi?id=210255
Signed-off-by Richard Gobert <[email protected]>
---
 net/ipv4/datagram.c | 2 ++
 1 file changed, 2 insertions(+)

Reviewed-by David Ahern <[email protected]>

if the maintainers decide to pick it up.

Paolo Abeni

maintainer

2022-07-06 16:21 UTC #5 lore raw

Quoted 68 lines · David Ahern

Quoted 57 lines · Richard Gobert
The IP_UNICAST_IF socket option is used to set the outgoing interface for outbound packets.
The IP_UNICAST_IF socket option was added as it was needed by the Wine project, since no other existing option (SO_BINDTODEVICE socket option, IP_PKTINFO socket option or the bind function) provided the needed characteristics needed by the IP_UNICAST_IF socket option. [1] The IP_UNICAST_IF socket option works well for unconnected sockets, that is, the interface specified by the IP_UNICAST_IF socket option is taken into consideration in the route lookup process when a packet is being sent.
However, for connected sockets, the outbound interface is chosen when connecting the socket, and in the route lookup process which is done when a packet is being sent, the interface specified by the IP_UNICAST_IF socket option is being ignored.
This inconsistent behavior was reported and discussed in an issue opened on systemd's GitHub project [2]. Also, a bug report was submitted in the kernel's bugzilla [3].
To understand the problem in more detail, we can look at what happens for UDP packets over IPv4 (The same analysis was done separately in the referenced systemd issue).
When a UDP packet is sent the udp_sendmsg function gets called and the following happens:
1. The oif member of the struct ipcm_cookie ipc (which stores the output interface of the packet) is initialized by the ipcm_init_sk function to inet->sk.sk_bound_dev_if (the device set by the SO_BINDTODEVICE socket option).
2. If the IP_PKTINFO socket option was set, the oif member gets overridden by the call to the ip_cmsg_send function.
3. If no output interface was selected yet, the interface specified by the IP_UNICAST_IF socket option is used.
4. If the socket is connected and no destination address is specified in the send function, the struct ipcm_cookie ipc is not taken into consideration and the cached route, that was calculated in the connect function is being used.
Thus, for a connected socket, the IP_UNICAST_IF sockopt isn't taken into consideration.
This patch corrects the behavior of the IP_UNICAST_IF socket option for connect()ed sockets by taking into consideration the IP_UNICAST_IF sockopt when connecting the socket.
In order to avoid reconnecting the socket, this option is still ignored when applied on an already connected socket until connect() is called again by the user.
Change the __ip4_datagram_connect function, which is called during socket connection, to take into consideration the interface set by the IP_UNICAST_IF socket option, in a similar way to what is done in the udp_sendmsg function.
Quoted 9 lines
[1] https://lore.kernel.org/netdev/1328685717.4736.4.camel@edumazet-laptop/T/
[2] https://github.com/systemd/systemd/issues/11935#issuecomment-618691018
[3] https://bugzilla.kernel.org/show_bug.cgi?id=210255
Signed-off-by Richard Gobert <[email protected]>
---
 net/ipv4/datagram.c | 2 ++
 1 file changed, 2 insertions(+)

Quoted 3 lines

Reviewed-by David Ahern <[email protected]>
if the maintainers decide to pick it up.

I think your reasoning is correct, and I'm now ok with the patch. Jakub noted it does not apply cleanly, so a repost will be needed. Additionally it would be great to include some self-tests.

It looks like the feature (even the original one, I mean) is IPv4 specific, don't you need an IPv6 counter-part?

Thanks!

Paolo

David Ahern

2022-07-06 17:55 UTC #6 lore raw

Quoted 69 lines · Paolo Abeni

Quoted 68 lines · David Ahern
Quoted 66 lines · Richard Gobert
The IP_UNICAST_IF socket option is used to set the outgoing interface for outbound packets.
The IP_UNICAST_IF socket option was added as it was needed by the Wine project, since no other existing option (SO_BINDTODEVICE socket option, IP_PKTINFO socket option or the bind function) provided the needed characteristics needed by the IP_UNICAST_IF socket option. [1] The IP_UNICAST_IF socket option works well for unconnected sockets, that is, the interface specified by the IP_UNICAST_IF socket option is taken into consideration in the route lookup process when a packet is being sent.
However, for connected sockets, the outbound interface is chosen when connecting the socket, and in the route lookup process which is done when a packet is being sent, the interface specified by the IP_UNICAST_IF socket option is being ignored.
This inconsistent behavior was reported and discussed in an issue opened on systemd's GitHub project [2]. Also, a bug report was submitted in the kernel's bugzilla [3].
To understand the problem in more detail, we can look at what happens for UDP packets over IPv4 (The same analysis was done separately in the referenced systemd issue).
When a UDP packet is sent the udp_sendmsg function gets called and the following happens:
1. The oif member of the struct ipcm_cookie ipc (which stores the output interface of the packet) is initialized by the ipcm_init_sk function to inet->sk.sk_bound_dev_if (the device set by the SO_BINDTODEVICE socket option).
2. If the IP_PKTINFO socket option was set, the oif member gets overridden by the call to the ip_cmsg_send function.
3. If no output interface was selected yet, the interface specified by the IP_UNICAST_IF socket option is used.
4. If the socket is connected and no destination address is specified in the send function, the struct ipcm_cookie ipc is not taken into consideration and the cached route, that was calculated in the connect function is being used.
Thus, for a connected socket, the IP_UNICAST_IF sockopt isn't taken into consideration.
This patch corrects the behavior of the IP_UNICAST_IF socket option for connect()ed sockets by taking into consideration the IP_UNICAST_IF sockopt when connecting the socket.
In order to avoid reconnecting the socket, this option is still ignored when applied on an already connected socket until connect() is called again by the user.
Change the __ip4_datagram_connect function, which is called during socket connection, to take into consideration the interface set by the IP_UNICAST_IF socket option, in a similar way to what is done in the udp_sendmsg function.
[1] https://lore.kernel.org/netdev/1328685717.4736.4.camel@edumazet-laptop/T/
[2] https://github.com/systemd/systemd/issues/11935#issuecomment-618691018
[3] https://bugzilla.kernel.org/show_bug.cgi?id=210255
Signed-off-by Richard Gobert <[email protected]>
---
 net/ipv4/datagram.c | 2 ++
 1 file changed, 2 insertions(+)

Quoted 7 lines

Quoted 3 lines
Reviewed-by David Ahern <[email protected]>
if the maintainers decide to pick it up.
I think your reasoning is correct, and I'm now ok with the patch. Jakub noted it does not apply cleanly, so a repost will be needed. Additionally it would be great to include some self-tests.

Agreed. nettest.c has '-S' option that sets IP_UNICAST_IF after the socket() call and before connect() so it is already doing what is wanted by this patch. Just need positive and negative test cases added to tools/testing/selftests/net.

Quoted 8 lines

It looks like the feature (even the original one, I mean) is IPv4 specific, don't you need an IPv6 counter-part?
Thanks!
Paolo

Richard Gobert

author

2022-07-13 12:47 UTC #7 lore raw

In reply to Paolo Abeni

Quoted 3 lines · Paolo Abeni

I think your reasoning is correct, and I'm now ok with the patch. Jakub noted it does not apply cleanly, so a repost will be needed. Additionally it would be great to include some self-tests.

Will include self-tests and submit V2.
The patch applies cleanly in my local setup. Do you have an idea as to why this might happen? I missed the email where Jakub mentioned this.

Thanks,
Richard

Jakub Kicinski

maintainer

2022-07-13 18:39 UTC #8 lore raw

Quoted 8 lines · On Wed, 13 Jul 2022 14:45:11 +0200 Richard Gobert wrote:

Quoted 3 lines · Paolo Abeni
I think your reasoning is correct, and I'm now ok with the patch. Jakub noted it does not apply cleanly, so a repost will be needed. Additionally it would be great to include some self-tests.
Will include self-tests and submit V2.
The patch applies cleanly in my local setup. Do you have an idea as to why this might happen? I missed the email where Jakub mentioned this.

Jakub noted it in a private conversation with Paolo :)

If it does indeed apply cleanly to net-next [1] please repost with the tree name explicitly stated in the subject line, ie. [PATCH net-next] to make sure our bots don't make any mistakes in tree selection for testing.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/