pre-fetching skb for delayed send
Subsystems (2)
Suppose one has a bridge-like thing that may store up packets for a bit (100+ms) before sending them....
We notice that performance is very good if we can run with (near) zero delay, but at higher delay, performance goes down significantly when attempting 10G speeds. I assume this is because we are sending skbs that are no longer in the cache...
So, is there an easy way to pre-fetch an skb and it's related pages?
Thanks,
Ben
Signature · 2 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Quoted 1 line · Ben Greear
So, is there an easy way to pre-fetch an skb and it's related pages?
One trick would be to store the pointers to the skbs in an array rather than relying on a list of skbs. That would enable you to issue prefetches for multiple skbs from a single cache line of the array, rather than relying upon the slow chasing of pointers guaranteed to cache miss in a list.
-ben
Signature · 1 line
"Thought is the essence of where you are now."
Quoted 7 lines · Benjamin LaHaise
Quoted 1 line · Ben Greear
So, is there an easy way to pre-fetch an skb and it's related pages?
One trick would be to store the pointers to the skbs in an array rather than relying on a list of skbs. That would enable you to issue prefetches for multiple skbs from a single cache line of the array, rather than relying upon the slow chasing of pointers guaranteed to cache miss in a list.
Well, to start with..I at least know the next skb to transmit, so I figured I'd prefetch it before starting tx of the current skb.
My question is more basic though: Given an skb, how do you prefetch it...do you just prefetch the skb pointer, or do you need to dig into the guts of the skb?
Thanks,
Ben
Quoted 3 lines
-ben
Signature · 2 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Quoted 3 lines · Ben Greear
Well, to start with..I at least know the next skb to transmit, so I figured I'd prefetch it before starting tx of the current skb.
Prefetching data you're just about to immediately access doesn't actually help improve performance -- it's better to just access the data. Prefetching subsequent skbs should be of more benefit.
Quoted 3 lines
My question is more basic though: Given an skb, how do you prefetch it...do you just prefetch the skb pointer, or do you need to dig into the guts of the skb?
See prefetch.h for details. Just pass the pointer to the cacheline you want to trigger prefetch on to prefetch() or prefetchw(), or use prefetch_range() (probably useful for skbs given that they're larger than one cacheline). For an skb, you may have to prefetch the frag list as well.
-ben
Signature · 1 line
"Thought is the essence of where you are now."