xfrm: Clear sk_dst_cache when applying per-socket policy.
Subsystems (3)
PATCH 1/1 xfrm: Clear sk_dst_cache when applying per-socket policy.
If an IPv6 socket has a valid dst cache, then xfrm_lookup_route will get skipped. However, the cache is not invalidated when applying policy to a socket (i.e. IPV6_XFRM_POLICY). The result is that new policies are sometimes ignored on those sockets.
This can be demonstrated like so,
1. Create UDPv6 socket.
2. connect() the socket.
3. Apply an outbound XFRM policy to the socket.
4. send() data on the socket.
Packets will continue to be sent in the clear instead of matching an xfrm or returning a no-match error (EAGAIN). This affects calls to send() and not sendto().
Note: Creating normal XFRM policies should have a similar effect on sk_dst_cache entries that match the policy, but that is not fixed in this patch.
Tested: https://android-review.googlesource.com/418659
--- net/xfrm/xfrm_state.c | 1 + 1 file changed, 1 insertion(+)
net/xfrm/xfrm_state.c +1 -0
@@ -2028,33 +2028,34 @@ EXPORT_SYMBOL(km_is_alive);int xfrm_user_policy (struct sock *sk,int optname,u8 __user *optval,int optlen) {int err;u8 *data;struct xfrm_mgr *km;struct xfrm_policy *pol =NULL ;if (optlen <=0 || optlen >PAGE_SIZE )return -EMSGSIZE ; data =memdup_user (optval, optlen);if (IS_ERR (data))return PTR_ERR (data); err = -EINVAL ;rcu_read_lock ();list_for_each_entry_rcu (km, &xfrm_km_list, list) { pol = km->compile_policy (sk, optname, data, optlen, &err);if (err >=0 )break ; }rcu_read_unlock ();if (err >=0 ) {xfrm_sk_policy_insert (sk, err, pol);xfrm_pol_put (pol); +sk_dst_reset (sk); err =0 ; }kfree (data);return err; }
Signature · 1 line
2.14.1.480.gb18f417b89-goog
Quoted 1 line · Jonathan Basseri
If an IPv6 socket has a valid dst cache
Did you look into why IPv4 does not suffer from this problem?
That said, clearing the dst cache entry does seem prudent in general.
Quoted 15 lines · On Tue, 15 Aug 2017 15:25:10 -0700 Jonathan Basseri <[email protected]>…
If an IPv6 socket has a valid dst cache, then
xfrm_lookup_routewill get skipped. However, the cache is not invalidated when applying policy to a socket (i.e.IPV6_XFRM_POLICY). The result is that new policies are sometimes ignored on those sockets.This can be demonstrated like so,
1. Create UDPv6 socket.
2.connect()the socket.
3. Apply an outbound XFRM policy to the socket.
4.send()data on the socket.Packets will continue to be sent in the clear instead of matching an xfrm or returning a no-match error (EAGAIN). This affects calls to
send()and notsendto().
Quoted 8 lines
Note: Creating normal XFRM policies should have a similar effect on
sk_dst_cacheentries that match the policy, but that is not fixed in this patch.Fixes 00bc0ef5880d ("ipv6: Skip XFRM lookup if dst_entry in socket cache is valid")Tested: https://android-review.googlesource.com/418659
Signed-off-by Jonathan Basseri <[email protected]>---
Thank you for the fix.
Quoted 22 lines · Jakub Sitnicki
Quoted 19 lines · On Tue, 15 Aug 2017 15:25:10 -0700 Jonathan Basseri <[email protected]>…
If an IPv6 socket has a valid dst cache, then
xfrm_lookup_routewill get skipped. However, the cache is not invalidated when applying policy to a socket (i.e.IPV6_XFRM_POLICY). The result is that new policies are sometimes ignored on those sockets.This can be demonstrated like so,
1. Create UDPv6 socket.
2.connect()the socket.
3. Apply an outbound XFRM policy to the socket.
4.send()data on the socket.Packets will continue to be sent in the clear instead of matching an xfrm or returning a no-match error (EAGAIN). This affects calls to
send()and notsendto().Note: Creating normal XFRM policies should have a similar effect on
sk_dst_cacheentries that match the policy, but that is not fixed in this patch.
Quoted 8 lines
Quoted 4 lines
Fixes 00bc0ef5880d ("ipv6: Skip XFRM lookup if dst_entry in socket cache is valid")Tested: https://android-review.googlesource.com/418659
Signed-off-by Jonathan Basseri <[email protected]>---
Thank you for the fix.
Acked-by Jakub Sitnicki <[email protected]>
I do not believe this fix is correct.
What happens if the socket is TCP ?
sk_dst_reset(sk) is not safe for them.
This might add use-after-free, and eventually crash.
Quoted 32 lines · On Wed, 16 Aug 2017 03:43:54 -0700 Eric Dumazet <[email protected]> wrote:
Quoted 22 lines · Jakub Sitnicki
Quoted 19 lines · On Tue, 15 Aug 2017 15:25:10 -0700 Jonathan Basseri <[email protected]>…
If an IPv6 socket has a valid dst cache, then
xfrm_lookup_routewill get skipped. However, the cache is not invalidated when applying policy to a socket (i.e.IPV6_XFRM_POLICY). The result is that new policies are sometimes ignored on those sockets.This can be demonstrated like so,
1. Create UDPv6 socket.
2.connect()the socket.
3. Apply an outbound XFRM policy to the socket.
4.send()data on the socket.Packets will continue to be sent in the clear instead of matching an xfrm or returning a no-match error (EAGAIN). This affects calls to
send()and notsendto().Note: Creating normal XFRM policies should have a similar effect on
sk_dst_cacheentries that match the policy, but that is not fixed in this patch.Quoted 8 lines
Quoted 4 lines
Fixes 00bc0ef5880d ("ipv6: Skip XFRM lookup if dst_entry in socket cache is valid")Tested: https://android-review.googlesource.com/418659
Signed-off-by Jonathan Basseri <[email protected]>---
Thank you for the fix.
Acked-by Jakub Sitnicki <[email protected]>
Quoted 7 lines
I do not believe this fix is correct.
What happens if the socket is TCP ?
sk_dst_reset(sk)is not safe for them.This might add use-after-free, and eventually crash.
You are right. I see that RCU-variant __sk_dst_reset() is used throughout TCP stack. Thank you for pointing it out.
Please disregard my earlier ACK.
-Jakub
PATCH 1/1 xfrm: Clear sk_dst_cache when applying per-socket policy.
In reply to Jonathan BasseriIf a socket has a valid dst cache, then xfrm_lookup_route will get skipped. However, the cache is not invalidated when applying policy to a socket (i.e. IPV6_XFRM_POLICY). The result is that new policies are sometimes ignored on those sockets. (Note: This was broken for IPv4 and IPv6 at different times.)
This can be demonstrated like so,
1. Create UDP socket.
2. connect() the socket.
3. Apply an outbound XFRM policy to the socket.
4. send() data on the socket.
Packets will continue to be sent in the clear instead of matching an xfrm or returning a no-match error (EAGAIN). This affects calls to send() and not sendto().
Invalidating the sk_dst_cache is necessary to correctly apply xfrm policies. Since we do this in xfrm_user_policy(), the sk_lock was already acquired in either do_ip_setsockopt() or do_ipv6_setsockopt(), and we may call __sk_dst_reset().
Performance impact should be negligible, since this code is only called when changing xfrm policy, and only affects the socket in question.
Note: Creating normal XFRM policies should have a similar effect on sk_dst_cache entries that match the policy, but that is not fixed in this patch.
Tested: https://android-review.googlesource.com/517555
Tested: https://android-review.googlesource.com/418659
net/xfrm/xfrm_state.c +1 -0
@@ -2045,33 +2045,34 @@ EXPORT_SYMBOL(km_is_alive);int xfrm_user_policy (struct sock *sk,int optname,u8 __user *optval,int optlen) {int err;u8 *data;struct xfrm_mgr *km;struct xfrm_policy *pol =NULL ;if (optlen <=0 || optlen >PAGE_SIZE )return -EMSGSIZE ; data =memdup_user (optval, optlen);if (IS_ERR (data))return PTR_ERR (data); err = -EINVAL ;rcu_read_lock ();list_for_each_entry_rcu (km, &xfrm_km_list, list) { pol = km->compile_policy (sk, optname, data, optlen, &err);if (err >=0 )break ; }rcu_read_unlock ();if (err >=0 ) {xfrm_sk_policy_insert (sk, err, pol);xfrm_pol_put (pol); +__sk_dst_reset (sk); err =0 ; }kfree (data);return err; }
Signature · 1 line
2.15.0.rc0.271.g36b669edcc-goog
Steffen Klassert
maintainerQuoted 21 lines · Jonathan Basseri
If a socket has a valid dst cache, then
xfrm_lookup_routewill get skipped. However, the cache is not invalidated when applying policy to a socket (i.e.IPV6_XFRM_POLICY). The result is that new policies are sometimes ignored on those sockets. (Note: This was broken for IPv4 and IPv6 at different times.)This can be demonstrated like so,
1. Create UDP socket.
2.connect()the socket.
3. Apply an outbound XFRM policy to the socket.
4.send()data on the socket.Packets will continue to be sent in the clear instead of matching an xfrm or returning a no-match error (EAGAIN). This affects calls to
send()and notsendto().Invalidating the
sk_dst_cacheis necessary to correctly apply xfrm policies. Since we do this inxfrm_user_policy(), thesk_lockwas already acquired in eitherdo_ip_setsockopt()ordo_ipv6_setsockopt(), and we may call__sk_dst_reset().
Quoted 6 lines
Performance impact should be negligible, since this code is only called when changing xfrm policy, and only affects the socket in question.
Note: Creating normal XFRM policies should have a similar effect on
sk_dst_cacheentries that match the policy, but that is not fixed in this patch.
I think we don't have this problem with 'normal' policies. When inserting such a policy, we bump the IPv4/IPv6 genid. This should invalidate all cached dst entries, no?
Quoted 26 lines · Steffen Klassert
Quoted 24 lines · Jonathan Basseri
If a socket has a valid dst cache, then
xfrm_lookup_routewill get skipped. However, the cache is not invalidated when applying policy to a socket (i.e.IPV6_XFRM_POLICY). The result is that new policies are sometimes ignored on those sockets. (Note: This was broken for IPv4 and IPv6 at different times.)This can be demonstrated like so,
1. Create UDP socket.
2.connect()the socket.
3. Apply an outbound XFRM policy to the socket.
4.send()data on the socket.Packets will continue to be sent in the clear instead of matching an xfrm or returning a no-match error (EAGAIN). This affects calls to
send()and notsendto().Invalidating the
sk_dst_cacheis necessary to correctly apply xfrm policies. Since we do this inxfrm_user_policy(), thesk_lockwas already acquired in eitherdo_ip_setsockopt()ordo_ipv6_setsockopt(), and we may call__sk_dst_reset().Performance impact should be negligible, since this code is only called when changing xfrm policy, and only affects the socket in question.
Quoted 8 lines
Quoted 3 lines
Note: Creating normal XFRM policies should have a similar effect on
sk_dst_cacheentries that match the policy, but that is not fixed in this patch.I think we don't have this problem with 'normal' policies. When inserting such a policy, we bump the IPv4/IPv6 genid. This should invalidate all cached dst entries, no?
That sounds reasonable to me. I had not confirmed the behavior for normal policies, so I was trying to point out that this fix is only for socket policies. Should I modify the commit message?
Steffen Klassert
maintainerQuoted 32 lines · Jonathan Basseri 😶
Quoted 30 lines · Steffen Klassert
Quoted 27 lines · Jonathan Basseri
If a socket has a valid dst cache, then
xfrm_lookup_routewill get skipped. However, the cache is not invalidated when applying policy to a socket (i.e.IPV6_XFRM_POLICY). The result is that new policies are sometimes ignored on those sockets. (Note: This was broken for IPv4 and IPv6 at different times.)This can be demonstrated like so,
1. Create UDP socket.
2.connect()the socket.
3. Apply an outbound XFRM policy to the socket.
4.send()data on the socket.Packets will continue to be sent in the clear instead of matching an xfrm or returning a no-match error (EAGAIN). This affects calls to
send()and notsendto().Invalidating the
sk_dst_cacheis necessary to correctly apply xfrm policies. Since we do this inxfrm_user_policy(), thesk_lockwas already acquired in eitherdo_ip_setsockopt()ordo_ipv6_setsockopt(), and we may call__sk_dst_reset().Performance impact should be negligible, since this code is only called when changing xfrm policy, and only affects the socket in question.
Note: Creating normal XFRM policies should have a similar effect on
sk_dst_cacheentries that match the policy, but that is not fixed in this patch.
Quoted 7 lines
Quoted 4 lines
I think we don't have this problem with 'normal' policies. When inserting such a policy, we bump the IPv4/IPv6 genid. This should invalidate all cached dst entries, no?
That sounds reasonable to me. I had not confirmed the behavior for normal policies, so I was trying to point out that this fix is only for socket policies. Should I modify the commit message?
Yes, please do so. This comment may lead people to the wrong direction.
Thanks!
Quoted 37 lines · Steffen Klassert
Quoted 36 lines · Jonathan Basseri 😶
Quoted 34 lines · Steffen Klassert
Quoted 21 lines · Jonathan Basseri
If a socket has a valid dst cache, then
xfrm_lookup_routewill get skipped. However, the cache is not invalidated when applying policy to a socket (i.e.IPV6_XFRM_POLICY). The result is that new policies are sometimes ignored on those sockets. (Note: This was broken for IPv4 and IPv6 at different times.)This can be demonstrated like so,
1. Create UDP socket.
2.connect()the socket.
3. Apply an outbound XFRM policy to the socket.
4.send()data on the socket.Packets will continue to be sent in the clear instead of matching an xfrm or returning a no-match error (EAGAIN). This affects calls to
send()and notsendto().Invalidating the
sk_dst_cacheis necessary to correctly apply xfrm policies. Since we do this inxfrm_user_policy(), thesk_lockwas already acquired in eitherdo_ip_setsockopt()ordo_ipv6_setsockopt(), and we may call__sk_dst_reset().Quoted 6 lines
Performance impact should be negligible, since this code is only called when changing xfrm policy, and only affects the socket in question.
Note: Creating normal XFRM policies should have a similar effect on
sk_dst_cacheentries that match the policy, but that is not fixed in this patch.I think we don't have this problem with 'normal' policies. When inserting such a policy, we bump the IPv4/IPv6 genid. This should invalidate all cached dst entries, no?
Quoted 7 lines
Quoted 3 lines
That sounds reasonable to me. I had not confirmed the behavior for normal policies, so I was trying to point out that this fix is only for socket policies. Should I modify the commit message?
Yes, please do so. This comment may lead people to the wrong direction.
Thanks!
Thank you for the feedback. Sending a v2 patch with updated message.
PATCH v2 1/1 xfrm: Clear sk_dst_cache when applying per-socket policy.
In reply to Jonathan BasseriIf a socket has a valid dst cache, then xfrm_lookup_route will get skipped. However, the cache is not invalidated when applying policy to a socket (i.e. IPV6_XFRM_POLICY). The result is that new policies are sometimes ignored on those sockets. (Note: This was broken for IPv4 and IPv6 at different times.)
This can be demonstrated like so,
1. Create UDP socket.
2. connect() the socket.
3. Apply an outbound XFRM policy to the socket. (setsockopt)
4. send() data on the socket.
Packets will continue to be sent in the clear instead of matching an xfrm or returning a no-match error (EAGAIN). This affects calls to send() and not sendto().
Invalidating the sk_dst_cache is necessary to correctly apply xfrm policies. Since we do this in xfrm_user_policy(), the sk_lock was already acquired in either do_ip_setsockopt() or do_ipv6_setsockopt(), and we may call __sk_dst_reset().
Performance impact should be negligible, since this code is only called when changing xfrm policy, and only affects the socket in question.
Tested: https://android-review.googlesource.com/517555
Tested: https://android-review.googlesource.com/418659
--- net/xfrm/xfrm_state.c | 1 + 1 file changed, 1 insertion(+)
net/xfrm/xfrm_state.c +1 -0
@@ -2045,33 +2045,34 @@ EXPORT_SYMBOL(km_is_alive);int xfrm_user_policy (struct sock *sk,int optname,u8 __user *optval,int optlen) {int err;u8 *data;struct xfrm_mgr *km;struct xfrm_policy *pol =NULL ;if (optlen <=0 || optlen >PAGE_SIZE )return -EMSGSIZE ; data =memdup_user (optval, optlen);if (IS_ERR (data))return PTR_ERR (data); err = -EINVAL ;rcu_read_lock ();list_for_each_entry_rcu (km, &xfrm_km_list, list) { pol = km->compile_policy (sk, optname, data, optlen, &err);if (err >=0 )break ; }rcu_read_unlock ();if (err >=0 ) {xfrm_sk_policy_insert (sk, err, pol);xfrm_pol_put (pol); +__sk_dst_reset (sk); err =0 ; }kfree (data);return err; }
Signature · 1 line
2.15.0.rc2.357.g7e34df9404-goog
Quoted 21 lines · Jonathan Basseri
If a socket has a valid dst cache, then
xfrm_lookup_routewill get skipped. However, the cache is not invalidated when applying policy to a socket (i.e.IPV6_XFRM_POLICY). The result is that new policies are sometimes ignored on those sockets. (Note: This was broken for IPv4 and IPv6 at different times.)This can be demonstrated like so,
1. Create UDP socket.
2.connect()the socket.
3. Apply an outbound XFRM policy to the socket. (setsockopt)
4.send()data on the socket.Packets will continue to be sent in the clear instead of matching an xfrm or returning a no-match error (EAGAIN). This affects calls to
send()and notsendto().Invalidating the
sk_dst_cacheis necessary to correctly apply xfrm policies. Since we do this inxfrm_user_policy(), thesk_lockwas already acquired in eitherdo_ip_setsockopt()ordo_ipv6_setsockopt(), and we may call__sk_dst_reset().
Quoted 7 lines
Performance impact should be negligible, since this code is only called when changing xfrm policy, and only affects the socket in question.
Fixes 00bc0ef5880d ("ipv6: Skip XFRM lookup if dst_entry in socket cache is valid")Tested: https://android-review.googlesource.com/517555
Tested: https://android-review.googlesource.com/418659Signed-off-by Jonathan Basseri <[email protected]>
Applied, thanks for your work Jonathan!