Skip to contentSkip to conversation

perf regression from ipv6: use net->rt_genid to check dst validity

DISCUSSION Chris Mason 1 participant 1 message Jun 16, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi everyone,

I tracked down a perf regression last week in our 3.10-stable based kernel. fib6_lookup_1 was at the top of the profiles, being called very frequently during sends. The 2.6.38 kernel we were comparing against only called fib6_lookup_1 during recv.

The call chain was doing the lookups because we were always tossing the destination cache. A little trial and error led me to this commit:

	commit 6f3118b571b8a4c06c7985dc3172c3526cb86253
	Author: Nicolas Dichtel <[email protected]>
	Date:   Mon Sep 10 22:09:46 2012 +0000

	    ipv6: use net->rt_genid to check dst validity

The workload was our in memory database, and dropping this commit gave us a 10% boost to overall queries per second.

Moving up to mainline, it looks like we're still failing the validity check most of the time. A few printks show when we do fail, it's always this line:

       if (rt->rt6i_genid != rt_genid_ipv6(dev_net(rt->dst.dev)))

The cached dst had a genid of 2 and the dev_net version was 3.

What I haven't done yet is fully reproduce the 10% hit on mainline. I have a few patches to port in and I'll get a workload running on 3.15. But it doesn't look like this part has changed. Somehow we're hanging onto a destination cache with an old genid and we're hammering on lookups because of it.

Any ideas before I shower things with printk?

-chris