Skip to contentSkip to conversation

net: tx scalability works : tx_packets/tx_bytes/tx_dropped

PATCH In review Eric Dumazet 2 participants 2 messages May 18, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover net: tx scalability works : tx_packets/tx_bytes/tx_dropped

offsetof(struct net_device, features)=0x44 offsetof(struct net_device, stats.tx_packets)=0x54 offsetof(struct net_device, stats.tx_bytes)=0x5c offsetof(struct net_device, stats.tx_dropped)=0x6c

This means drivers that touch stats.tx_packets/stats.tx_bytes in their tx path can slow down SMP operations, since they dirty a cache line that should stay shared (dev->features is needed in rx and tx paths)

We could move away stats field in net_device but it wont help that much. (Two cache lines dirtied in tx path, we can do one only)

Better solution is to add tx_packets/tx_bytes/tx_dropped in struct netdev_queue because this structure is already touched in tx path and counters updates will then be free.

First patch adds the infrastructure, and second one uses this infrastructure for VLAN devices, for which I spoted the performance problem on Gigabit links.

rx path has same problem of course, but less prevalent as less cpus are involved in rx path if IRQ affinities not too dumb.

Thank you

maintainer
#1 lore raw

From: Eric Dumazet <[email protected]>
Date: Mon, 18 May 2009 12:33:17 +0200

Quoted 16 lines

offsetof(struct net_device, features)=0x44 offsetof(struct net_device, stats.tx_packets)=0x54 offsetof(struct net_device, stats.tx_bytes)=0x5c offsetof(struct net_device, stats.tx_dropped)=0x6c

This means drivers that touch stats.tx_packets/stats.tx_bytes in their tx path can slow down SMP operations, since they dirty a cache line that should stay shared (dev->features is needed in rx and tx paths)

We could move away stats field in net_device but it wont help that much. (Two cache lines dirtied in tx path, we can do one only)

Better solution is to add tx_packets/tx_bytes/tx_dropped in struct netdev_queue because this structure is already touched in tx path and counters updates will then be free.

Quoted 6 lines

First patch adds the infrastructure, and second one uses this infrastructure for VLAN devices, for which I spoted the performance problem on Gigabit links.

rx path has same problem of course, but less prevalent as less cpus are involved in rx path if IRQ affinities not too dumb.

Looks great, both patches applied, thanks!