xfrm: policy: remove pcpu policy cache
Subsystems (3)
PATCH v2 1/1 xfrm: policy: remove pcpu policy cache
Kristian Evensen says:
In a project I am involved in, we are running ipsec (Strongswan) on different mt7621-based routers. Each router is configured as an initiator and has around ~30 tunnels to different responders (running on misc. devices). Before the flow cache was removed (kernel 4.9), we got a combined throughput of around 70Mbit/s for all tunnels on one router. However, we recently switched to kernel 4.14 (4.14.48), and the total throughput is somewhere around 57Mbit/s (best-case). I.e., a drop of around 20%. Reverting the flow cache removal restores, as expected, performance levels to that of kernel 4.9.
When pcpu xdst exists, it has to be validated first before it can be used.
A negative hit thus increases cost vs. no-cache.
As number of tunnels increases, hit rate decreases so this pcpu caching isn't a viable strategy.
Furthermore, the xdst cache also needs to run with BH off, so when removing this the bh disable/enable pairs can be removed too.
Kristian tested a 4.14.y backport of this change and reported increased performance:
In our tests, the throughput reduction has been reduced from around -20% to -5%. We also see that the overall throughput is independent of the number of tunnels, while before the throughput was reduced as the number of tunnels increased.
---
v2: rebase on top of current ipsec-next
include/net/xfrm.h | 1 - net/xfrm/xfrm_device.c | 10 ---- net/xfrm/xfrm_policy.c | 139 +------------------------------------------------ net/xfrm/xfrm_state.c | 5 +- 4 files changed, 3 insertions(+), 152 deletions(-)
include/net/xfrm.h +0 -1
@@ -332,7 +332,6 @@ int xfrm_policy_register_afinfo(const struct xfrm_policy_afinfo *afinfo, int famvoid xfrm_policy_unregister_afinfo (const struct xfrm_policy_afinfo *afinfo);void km_policy_notify (struct xfrm_policy *xp,int dir,const struct km_event *c); -void xfrm_policy_cache_flush (void );void km_state_notify (struct xfrm_state *x,const struct km_event *c);struct xfrm_tmpl ;
net/xfrm/xfrm_device.c +0 -10
@@ -307,12 +307,6 @@ static int xfrm_dev_register(struct net_device *dev)return xfrm_api_check (dev); } -static int xfrm_dev_unregister (struct net_device *dev) -{ -xfrm_policy_cache_flush (); -return NOTIFY_DONE ; -} -static int xfrm_dev_feat_change (struct net_device *dev) {return xfrm_api_check (dev);
@@ -323,7 +317,6 @@ static int xfrm_dev_down(struct net_device *dev)if (dev->features &NETIF_F_HW_ESP )xfrm_dev_state_flush (dev_net (dev), dev, true); -xfrm_policy_cache_flush ();return NOTIFY_DONE ; }
@@ -335,9 +328,6 @@ static int xfrm_dev_event(struct notifier_block *this, unsigned long event, voidcase NETDEV_REGISTER :return xfrm_dev_register (dev); -case NETDEV_UNREGISTER : -return xfrm_dev_unregister (dev); -case NETDEV_FEAT_CHANGE :return xfrm_dev_feat_change (dev);
net/xfrm/xfrm_policy.c +1 -138
192 lines not on this page · raw message
net/xfrm/xfrm_state.c +2 -3
@@ -735,10 +735,9 @@ int xfrm_state_flush(struct net *net, u8 proto, bool task_valid) } out:spin_unlock_bh (&net->xfrm.xfrm_state_lock); -if (cnt) { +if (cnt) err =0 ; -xfrm_policy_cache_flush (); - } +return err; }EXPORT_SYMBOL (xfrm_state_flush);
Signature · 1 line
2.16.4
Quoted 25 lines · Florian Westphal
Kristian Evensen says:
In a project I am involved in, we are running ipsec (Strongswan) on different mt7621-based routers. Each router is configured as an initiator and has around ~30 tunnels to different responders (running on misc. devices). Before the flow cache was removed (kernel 4.9), we got a combined throughput of around 70Mbit/s for all tunnels on one router. However, we recently switched to kernel 4.14 (4.14.48), and the total throughput is somewhere around 57Mbit/s (best-case). I.e., a drop of around 20%. Reverting the flow cache removal restores, as expected, performance levels to that of kernel 4.9.When pcpu xdst exists, it has to be validated first before it can be used.
A negative hit thus increases cost vs. no-cache.
As number of tunnels increases, hit rate decreases so this pcpu caching isn't a viable strategy.
Furthermore, the xdst cache also needs to run with BH off, so when removing this the bh disable/enable pairs can be removed too.
Kristian tested a 4.14.y backport of this change and reported increased performance:
Quoted 7 lines
In our tests, the throughput reduction has been reduced from around -20% to -5%. We also see that the overall throughput is independent of the number of tunnels, while before the throughput was reduced as the number of tunnels increased.
Reported-by Kristian Evensen <[email protected]>Signed-off-by Florian Westphal <[email protected]>
Applied to ipsec-next, thanks a lot!