"xfrm: Fix the gc threshold value for ipv4" broke my IPSec connections
Subsystems (2)
I've recently upgraded from 3.4.x to 3.10.x and this broke my IPSec setup in transport mode. The simplest test case is to setup few such connections with few boxes like this:
spdadd 192.168.1.100 192.168.2.100 any -P out ipsec
esp/transport//require
ah/transport//require;spdadd 192.168.2.100 192.168.1.100 any -P in ipsec
esp/transport//require
ah/transport//require;Then set up an HTTP server on one box and run ab on the other box to create come TCP connections:
ab -n 10000 -c 50 http://192.168.1.100/
Then the connect() call will very quickly start returning ENOBUFS. I haven't seen anything wrong with my simple setup (just copy of ipsec-howto.org in transport mode and pre shared keys) and started bisecting. That way I found this commit to break my case:
703fb94ec58e0e8769380c2877a8a34aeb5b6c97
xfrm: Fix the gc threshold value for ipv4
Reverting it on 3.10.15 fixes my issue. This seems to be there from 3.7 and I don't really believe such simple case stayed broken for so long. Em I missing something or there is really a bug?
If smeone is interested in details of this configuration and commands I'm running, just let me know. This was reproduced with few VMs under XEN.
Signature · 1 line
Damian Pietras
Quoted 25 lines · Damian Pietras
I've recently upgraded from 3.4.x to 3.10.x and this broke my IPSec setup in transport mode. The simplest test case is to setup few such connections with few boxes like this:
spdadd 192.168.1.100 192.168.2.100 any -P out ipsec
esp/transport//require ah/transport//require;spdadd 192.168.2.100 192.168.1.100 any -P in ipsec
esp/transport//require ah/transport//require;Then set up an HTTP server on one box and run ab on the other box to create come TCP connections:
ab -n 10000 -c 50 http://192.168.1.100/
Then the
connect()call will very quickly start returning ENOBUFS. I haven't seen anything wrong with my simple setup (just copy of ipsec-howto.org in transport mode and pre shared keys) and started bisecting. That way I found this commit to break my case:
703fb94ec58e0e8769380c2877a8a34aeb5b6c97
xfrm: Fix the gc threshold value for ipv4
Quoted 7 lines
Reverting it on 3.10.15 fixes my issue. This seems to be there from 3.7 and I don't really believe such simple case stayed broken for so long. Em I missing something or there is really a bug?
If smeone is interested in details of this configuration and commands I'm running, just let me know. This was reproduced with few VMs under XEN.
It looks like you need to tune /proc/sys/net/ipv4/xfrm4_gc_thresh to a sensible value given your workload.
try :
echo 65536 >/proc/sys/net/ipv4/xfrm4_gc_thresh
Presumably the 1024 default is really too small...
Quoted 11 lines · Eric Dumazet
Quoted 10 lines
703fb94ec58e0e8769380c2877a8a34aeb5b6c97
xfrm: Fix the gc threshold value for ipv4Reverting it on 3.10.15 fixes my issue. This seems to be there from 3.7 and I don't really believe such simple case stayed broken for so long. Em I missing something or there is really a bug?
If smeone is interested in details of this configuration and commands I'm running, just let me know. This was reproduced with few VMs under XEN.
Quoted 8 lines
It looks like you need to tune
/proc/sys/net/ipv4/xfrm4_gc_threshto a sensible value given your workload.try :
echo 65536 >/proc/sys/net/ipv4/xfrm4_gc_thresh
Presumably the 1024 default is really too small...
Now it's working in my test setup, I'm changing it on the production boxes, thanks!
Signature · 1 line
Damian Pietras
Quoted 15 lines · Damian Pietras
Quoted 14 lines · Eric Dumazet
Quoted 10 lines
703fb94ec58e0e8769380c2877a8a34aeb5b6c97
xfrm: Fix the gc threshold value for ipv4Reverting it on 3.10.15 fixes my issue. This seems to be there from 3.7 and I don't really believe such simple case stayed broken for so long. Em I missing something or there is really a bug?
If smeone is interested in details of this configuration and commands I'm running, just let me know. This was reproduced with few VMs under XEN.
It looks like you need to tune
/proc/sys/net/ipv4/xfrm4_gc_threshto a sensible value given your workload.
Quoted 10 lines
Quoted 5 lines
try :
echo 65536 >/proc/sys/net/ipv4/xfrm4_gc_thresh
Presumably the 1024 default is really too small...
Now it's working in my test setup, I'm changing it on the production boxes, thanks!
Steffen, what do you think ?
1024 seems really small, given we had much higher values.
(256 K on a 1GB host)
This sysctl also needs an entry inDocumentation/networking/ip-sysctl.txt
Quoted 22 lines · Eric Dumazet
Quoted 21 lines · Damian Pietras
Quoted 19 lines · Eric Dumazet
Quoted 10 lines
703fb94ec58e0e8769380c2877a8a34aeb5b6c97
xfrm: Fix the gc threshold value for ipv4Reverting it on 3.10.15 fixes my issue. This seems to be there from 3.7 and I don't really believe such simple case stayed broken for so long. Em I missing something or there is really a bug?
If smeone is interested in details of this configuration and commands I'm running, just let me know. This was reproduced with few VMs under XEN.
It looks like you need to tune
/proc/sys/net/ipv4/xfrm4_gc_threshto a sensible value given your workload.try :
echo 65536 >/proc/sys/net/ipv4/xfrm4_gc_thresh
Presumably the 1024 default is really too small...
Quoted 8 lines
Quoted 4 lines
Now it's working in my test setup, I'm changing it on the production boxes, thanks!
Steffen, what do you think ?
1024 seems really small, given we had much higher values.
Sure, we can increase the default value, maybe along with the ipv6 side. Any recomendation on a good default for both?
Quoted 6 lines
(256 K on a 1GB host)
This sysctl also needs an entry in
Documentation/networking/ip-sysctl.txt
Yes, it's undocumented. I'll take care of it.