net-sysfs+forcedeth: stats & debug enhancements
v5 cover net-sysfs+forcedeth: stats & debug enhancements
These changes provide a new tx_timeout sysfs attribute and implement the ndo_get_stats64 API. They also add a few more stats and debugging features for forcedeth. They ensure that stats updates are correct in SMP systems, 32 or 64-bits.
Note: patch 1 is the cherry-pick of 898bdf2cb43e ("forcedeth: fix stats on hardware without extended stats support")
Changes since v4:
- tx_timeout counter now a generic sysfs attribute. Credits to Stephen Hemminger for initial implementation
- revert get_stats64 to using atomic variables: see
http://patchwork.ozlabs.org/patch/125861/ for motivations
- dropped patch "expose module parameters in /sys/module" for now: I will work on this later, following Stephen's recommendations
(http://patchwork.ozlabs.org/patch/125862/).
Changes since v3:
- updated get_stats64 + rx_dropped patches to use u64_stats_sync.h
- dropped indentation "whitespace/indentation fixes" (included in get_stats64 api patch)
Changes since v2:
- patch 1/9 is the cherry-pick of 898bdf2cb43e ("forcedeth: fix stats on hardware without extended stats support")
- removed patch 5/10 "stats for rx_packets based on hardware registers" because packets&bytes stats are updated in software only (898bdf2cb43e)
Changes since v1:
- patch 1/10 is the same as
http://patchwork.ozlabs.org/patch/125017/ (targetting net)
- other patches updated to take patch 1/10 into account
- various commit message updates
Tested:
~150Mbps incoming TCP, ethtool -S in a loop, x86_64 16-way:
tx_bytes: 5441989419
rx_packets: 5439224
tx_timeout: 0
tx_packets: 5456705
rx_bytes: 5566763850Tested:
pktgen + loopback report same RX/TX packets and bytes stats
Tested:
tests above with Kconfig DEBUG_PAGEALLOC DEBUG_MUTEXES DEBUG_SPINLOCK LOCKUP_DETECTOR DEBUG_RT_MUTEXES DEBUG_LOCK_ALLOC PROVE_LOCKING DEBUG_ATOMIC_SLEEP DEBUG_STACK_USAGE DEBUG_KOBJECT DEBUG_VM DEBUG_LIST DEBUG_SG DEBUG_NOTIFIERS TEST_KSTRTOX STRICT_DEVMEM DEBUG_STACKOVERFLOW
############################################
# Patch Set Summary:
David Decotigny (7):
net-sysfs: fixed minor sparse warning
kbuild: document RPS/XPS network Kconfig options
net: provide counter for tx_timeout errors in sysfs
forcedeth: implement ndo_get_stats64() API
forcedeth: account for dropped RX frames
forcedeth: stats updated with a deferrable timer
forcedeth: whitespace/indentation fixes
Mike Ditto (1):
forcedeth: Add messages to indicate using MSI or MSI-X
Sameer Nanda (1):
forcedeth: allow to silence "TX timeout" debug messages
david decotigny (1):
forcedeth: fix stats on hardware without extended stats support
drivers/net/ethernet/nvidia/forcedeth.c | 320 ++++++++++++++++++++++--------- include/linux/netdevice.h | 12 +- net/Kconfig | 16 ++- net/core/net-sysfs.c | 49 ++++-- net/sched/sch_generic.c | 1 + 5 files changed, 293 insertions(+), 105 deletions(-)
Signature · 1 line
1.7.3.1
PATCH v5 1/10 forcedeth: fix stats on hardware without extended stats support
From: david decotigny <[email protected]>
This change makes sure that tx_packets/rx_bytes ifconfig counters are updated even on NICs that don't provide hardware support for these
stats: they are now updated in software. For the sake of consistency, we also now have tx_bytes updated in software (hardware counters include ethernet CRC, and software doesn't account for it).
This reverts parts of:
- "forcedeth: statistics optimization" (21828163b2)
- "forcedeth: Improve stats counters" (0bdfea8ba8)
- "forcedeth: remove unneeded stats updates" (4687f3f364)
Tested:
pktgen + loopback (http://patchwork.ozlabs.org/patch/124698/)
reports identical tx_packets/rx_packets and tx_bytes/rx_bytes.
--- drivers/net/ethernet/nvidia/forcedeth.c | 36 +++++++++++++++++++++++------- 1 files changed, 27 insertions(+), 9 deletions(-)
drivers/net/ethernet/nvidia/forcedeth.c +27 -9
@@ -609,7 +609,7 @@ struct nv_ethtool_str { };static const struct nv_ethtool_str nv_estats_str[] = { - {"tx_bytes" }, + {"tx_bytes" },/* includes Ethernet FCS CRC */ {"tx_zero_rexmt" }, {"tx_one_rexmt" }, {"tx_many_rexmt" },
@@ -637,7 +637,7 @@ static const struct nv_ethtool_str nv_estats_str[] = {/* version 2 stats */ { "tx_deferral" }, { "tx_packets" }, - { "rx_bytes" }, + { "rx_bytes" }, /* includes Ethernet FCS CRC */ { "tx_pause" }, { "rx_pause" }, { "rx_drop_frame" },
@@ -649,7 +649,7 @@ static const struct nv_ethtool_str nv_estats_str[] = { };struct nv_ethtool_stats { -u64 tx_bytes; +u64 tx_bytes;/* should be ifconfig->tx_bytes + 4*tx_packets */ u64 tx_zero_rexmt;u64 tx_one_rexmt;u64 tx_many_rexmt;
@@ -670,14 +670,14 @@ struct nv_ethtool_stats {u64 rx_unicast;u64 rx_multicast;u64 rx_broadcast; -u64 rx_packets; +u64 rx_packets;/* should be ifconfig->rx_packets */ u64 rx_errors_total;u64 tx_errors_total;/* version 2 stats */ u64 tx_deferral; -u64 tx_packets; -u64 rx_bytes; +u64 tx_packets;/* should be ifconfig->tx_packets */ +u64 rx_bytes;/* should be ifconfig->rx_bytes + 4*rx_packets */ u64 tx_pause;u64 rx_pause;u64 rx_drop_frame;
@@ -1706,10 +1706,17 @@ static struct net_device_stats *nv_get_stats(struct net_device *dev)if (np->driver_data & (DEV_HAS_STATISTICS_V1 |DEV_HAS_STATISTICS_V2 |DEV_HAS_STATISTICS_V3 )) {nv_get_hw_stats (dev); +/* +* Note: because HW stats are not always available and +* for consistency reasons, the following ifconfig +* stats are managed by software: rx_bytes, tx_bytes, +* rx_packets and tx_packets. The related hardware +* stats reported by ethtool should be equivalent to +* these ifconfig stats, with 4 additional bytes per +* packet (Ethernet FCS CRC). +*/ +/* copy to net_device stats */ - dev->stats.tx_packets = np->estats.tx_packets; - dev->stats.rx_bytes = np->estats.rx_bytes; - dev->stats.tx_bytes = np->estats.tx_bytes; dev->stats.tx_fifo_errors = np->estats.tx_fifo_errors; dev->stats.tx_carrier_errors = np->estats.tx_carrier_errors; dev->stats.rx_crc_errors = np->estats.rx_crc_errors;
@@ -2380,6 +2387,9 @@ static int nv_tx_done(struct net_device *dev, int limit)if (flags &NV_TX_ERROR ) {if ((flags &NV_TX_RETRYERROR ) && !(flags &NV_TX_RETRYCOUNT_MASK ))nv_legacybackoff_reseed (dev); + }else { + dev->stats.tx_packets++; + dev->stats.tx_bytes += np->get_tx_ctx->skb->len; }dev_kfree_skb_any (np->get_tx_ctx->skb); np->get_tx_ctx->skb =NULL ;
@@ -2390,6 +2400,9 @@ static int nv_tx_done(struct net_device *dev, int limit)if (flags &NV_TX2_ERROR ) {if ((flags &NV_TX2_RETRYERROR ) && !(flags &NV_TX2_RETRYCOUNT_MASK ))nv_legacybackoff_reseed (dev); + }else { + dev->stats.tx_packets++; + dev->stats.tx_bytes += np->get_tx_ctx->skb->len; }dev_kfree_skb_any (np->get_tx_ctx->skb); np->get_tx_ctx->skb =NULL ;
@@ -2429,6 +2442,9 @@ static int nv_tx_done_optimized(struct net_device *dev, int limit)else nv_legacybackoff_reseed (dev); } + }else { + dev->stats.tx_packets++; + dev->stats.tx_bytes += np->get_tx_ctx->skb->len; }dev_kfree_skb_any (np->get_tx_ctx->skb);
@@ -2678,6 +2694,7 @@ static int nv_rx_process(struct net_device *dev, int limit) skb->protocol =eth_type_trans (skb, dev);napi_gro_receive (&np->napi, skb); dev->stats.rx_packets++; + dev->stats.rx_bytes += len; next_pkt:if (unlikely (np->get_rx.orig++ == np->last_rx.orig)) np->get_rx.orig = np->first_rx.orig;
@@ -2761,6 +2778,7 @@ static int nv_rx_process_optimized(struct net_device *dev, int limit) }napi_gro_receive (&np->napi, skb); dev->stats.rx_packets++; + dev->stats.rx_bytes += len; }else {dev_kfree_skb (skb); }
Signature · 1 line
1.7.3.1
PATCH v5 2/10 net-sysfs: fixed minor sparse warning
In reply to David DecotignyThis commit fixes following warning:net/core/net-sysfs.c:921:6: warning: symbol 'numa_node' shadows an earlier one include/linux/topology.h:222:1: originally declared here
--- net/core/net-sysfs.c | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-)
net/core/net-sysfs.c +6 -6
@@ -901,7 +901,7 @@ static ssize_t store_xps_map(struct netdev_queue *queue,struct xps_map *map, *new_map;struct xps_dev_maps *dev_maps, *new_dev_maps;int nonempty =0 ; -int numa_node =-2 ; +int numa_node_id =-2 ;if (!capable (CAP_NET_ADMIN ))return -EPERM ;
@@ -944,10 +944,10 @@ static ssize_t store_xps_map(struct netdev_queue *queue, need_set =cpumask_test_cpu (cpu, mask) &&cpu_online (cpu);#ifdef CONFIG_NUMA if (need_set) { -if (numa_node ==-2 ) - numa_node =cpu_to_node (cpu); -else if (numa_node !=cpu_to_node (cpu)) - numa_node =-1 ; +if (numa_node_id ==-2 ) + numa_node_id =cpu_to_node (cpu); +else if (numa_node_id !=cpu_to_node (cpu)) + numa_node_id =-1 ; }#endif if (need_set && pos >= map_len) {
@@ -997,7 +997,7 @@ static ssize_t store_xps_map(struct netdev_queue *queue,if (dev_maps)kfree_rcu (dev_maps, rcu); -netdev_queue_numa_node_write (queue, (numa_node >=0 ) ? numa_node : +netdev_queue_numa_node_write (queue, (numa_node_id >=0 ) ? numa_node_id :NUMA_NO_NODE );mutex_unlock (&xps_map_mutex);
Signature · 1 line
1.7.3.1
PATCH v5 3/10 kbuild: document RPS/XPS network Kconfig options
In reply to David DecotignyThis adds a description of RPS/XPS options and allow them to be changed at make menuconfig time.
--- net/Kconfig | 16 +++++++++++++--- 1 files changed, 13 insertions(+), 3 deletions(-)
net/Kconfig +13 -3
@@ -217,20 +217,30 @@ source "net/dns_resolver/Kconfig" source "net/batman-adv/Kconfig" config RPS - boolean + boolean "Enable Receive Packet Steering" depends on SMP && SYSFS && USE_GENERIC_SMP_HELPERS default y + RPS distributes the load of received packet processing + across multiple CPUs. If unsure, say Y. config RFS_ACCEL - boolean + boolean "Enable Hardware Acceleration of RFS" depends on RPS && GENERIC_HARDIRQS select CPU_RMAP default y + This is the hardware version of RPS. On multi-queue network + devices, this configures the hardware to distribute the + received packets across multiple CPUs. If unsure, say Y. config XPS - boolean + boolean "Enable Transmit Packet Steering" depends on SMP && SYSFS && USE_GENERIC_SMP_HELPERS default y + For multiqueue devices, XPS selects a transmit queue during + packet transmission based on configuration. This is done by + mapping the CPU transmitting the packet to a queue. XPS can + reduce transmit network latency on SMP systems. If unsure, + say Y. config HAVE_BPF_JIT bool
Signature · 1 line
1.7.3.1
PATCH v5 4/10 net: provide counter for tx_timeout errors in sysfs
In reply to David DecotignyThis adds the /sys/class/net/DEV/queues/Q/tx_timeout attribute containing the total number of timeout events on the given queue. It is always available, independently of CONFIG_RPS/XPS.
Credits to Stephen Hemminger for a preliminary version of this patch.
Tested:
without CONFIG_SYSFS (compilation only)
with sysfs and without CONFIG_RPS & CONFIG_XPS
with sysfs and without CONFIG_RPS
with sysfs and without CONFIG_XPS
with defaults
--- include/linux/netdevice.h | 12 ++++++++++-- net/core/net-sysfs.c | 37 +++++++++++++++++++++++++++++++------ net/sched/sch_generic.c | 1 + 3 files changed, 42 insertions(+), 8 deletions(-)
include/linux/netdevice.h +10 -2
@@ -530,7 +530,7 @@ struct netdev_queue {struct Qdisc *qdisc;unsigned long state;struct Qdisc *qdisc_sleeping; -#if defined(CONFIG_RPS ) || defined(CONFIG_XPS ) +#ifdef CONFIG_SYSFS struct kobject kobj;#endif #if defined(CONFIG_XPS ) && defined(CONFIG_NUMA )
@@ -545,6 +545,12 @@ struct netdev_queue {* please use this field instead of dev->trans_start */ unsigned long trans_start; + +/* +* Number of TX timeouts for this queue +* (/sys/class/net/DEV/Q/trans_timeout) +*/ +unsigned long trans_timeout; }____cacheline_aligned_in_smp ;static inline int netdev_queue_numa_node_read (const struct netdev_queue *q)
@@ -1184,9 +1190,11 @@ struct net_device {unsigned char broadcast[MAX_ADDR_LEN ];/* hw bcast add */ -#if defined(CONFIG_RPS ) || defined(CONFIG_XPS ) +#ifdef CONFIG_SYSFS struct kset *queues_kset; +#endif +#ifdef CONFIG_RPS struct netdev_rx_queue *_rx;/* Number of RX queues allocated at register_netdev() time */
net/core/net-sysfs.c +31 -6
@@ -780,7 +780,7 @@ net_rx_queue_update_kobjects(struct net_device *net, int old_num, int new_num)#endif } -#ifdef CONFIG_XPS +#ifdef CONFIG_SYSFS /* * netdev_queue sysfs structures and functions. */
@@ -826,6 +826,23 @@ static const struct sysfs_ops netdev_queue_sysfs_ops = { .store = netdev_queue_attr_store, }; +static ssize_t show_trans_timeout (struct netdev_queue *queue, +struct netdev_queue_attribute *attribute, +char *buf) +{ +unsigned long trans_timeout; + +spin_lock_irq (&queue->_xmit_lock); + trans_timeout = queue->trans_timeout; +spin_unlock_irq (&queue->_xmit_lock); + +return sprintf (buf,"%lu" , trans_timeout); +} + +static struct netdev_queue_attribute queue_trans_timeout = +__ATTR (tx_timeout,S_IRUGO , show_trans_timeout,NULL ); + +#ifdef CONFIG_XPS static inline unsigned int get_netdev_queue_index (struct netdev_queue *queue) {struct net_device *dev = queue->dev;
@@ -1020,12 +1037,17 @@ error:static struct netdev_queue_attribute xps_cpus_attribute =__ATTR (xps_cpus,S_IRUGO |S_IWUSR , show_xps_map, store_xps_map); +#endif /* CONFIG_XPS */ static struct attribute *netdev_queue_default_attrs[] = { + &queue_trans_timeout.attr, +#ifdef CONFIG_XPS &xps_cpus_attribute.attr, +#endifNULL }; +#ifdef CONFIG_XPS static void netdev_queue_release (struct kobject *kobj) {struct netdev_queue *queue =to_netdev_queue (kobj);
@@ -1076,10 +1098,13 @@ static void netdev_queue_release(struct kobject *kobj)memset (kobj,0 ,sizeof (*kobj));dev_put (queue->dev); } +#endif /* CONFIG_XPS */ static struct kobj_type netdev_queue_ktype = { .sysfs_ops = &netdev_queue_sysfs_ops, +#ifdef CONFIG_XPS .release = netdev_queue_release, +#endif .default_attrs = netdev_queue_default_attrs, };
@@ -1102,12 +1127,12 @@ static int netdev_queue_add_kobject(struct net_device *net, int index) return error; } -#endif /* CONFIG_XPS */ +#endif /* CONFIG_SYSFS */ int netdev_queue_update_kobjects (struct net_device *net,int old_num,int new_num) { -#ifdef CONFIG_XPS +#ifdef CONFIG_SYSFS int i;int error =0 ;
@@ -1125,14 +1150,14 @@ netdev_queue_update_kobjects(struct net_device *net, int old_num, int new_num)return error;#else return 0 ; -#endif +#endif /* CONFIG_SYSFS */ }static int register_queue_kobjects (struct net_device *net) {int error =0 , txq =0 , rxq =0 , real_rx =0 , real_tx =0 ; -#if defined(CONFIG_RPS ) || defined(CONFIG_XPS ) +#ifdef CONFIG_SYSFS net->queues_kset =kset_create_and_add ("queues" ,NULL , &net->dev.kobj);if (!net->queues_kset)
@@ -1173,7 +1198,7 @@ static void remove_queue_kobjects(struct net_device *net)net_rx_queue_update_kobjects (net, real_rx,0 );netdev_queue_update_kobjects (net, real_tx,0 ); -#if defined(CONFIG_RPS ) || defined(CONFIG_XPS ) +#ifdef CONFIG_SYSFS kset_unregister (net->queues_kset);#endif }
net/sched/sch_generic.c +1 -0
@@ -246,6 +246,7 @@ static void dev_watchdog(unsigned long arg) time_after(jiffies, (trans_start + dev->watchdog_timeo))) { some_queue_timedout =1 ; + txq->trans_timeout++;break ; } }
Signature · 1 line
1.7.3.1
Re: [PATCH net-next v5 04/10] net: provide counter for tx_timeout errors in sysfs
Quoted 5 lines · On Tue, 15 Nov 2011 21:15:52 -0800 David Decotigny…
This adds the
/sys/class/net/DEV/queues/Q/tx_timeoutattribute containing the total number of timeout events on the given queue. It is always available, independently of CONFIG_RPS/XPS.Credits to Stephen Hemminger for a preliminary version of this patch.
Cool I like it.
PATCH v5 5/10 forcedeth: Add messages to indicate using MSI or MSI-X
In reply to David DecotignyFrom: Mike Ditto <[email protected]>
This adds a few kernel messages to indicate whether PCIe interrupts are signaled with MSI or MSI-X.
--- drivers/net/ethernet/nvidia/forcedeth.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
drivers/net/ethernet/nvidia/forcedeth.c +2 -0
@@ -3810,6 +3810,7 @@ static int nv_request_irq(struct net_device *dev, int intr_test)writel (0 , base + NvRegMSIXMap0);writel (0 , base + NvRegMSIXMap1); } +netdev_info (dev,"MSI-X enabled\n" ); } }if (ret !=0 && np->msi_flags &NV_MSI_CAPABLE ) {
@@ -3831,6 +3832,7 @@ static int nv_request_irq(struct net_device *dev, int intr_test)writel (0 , base + NvRegMSIMap1);/* enable msi vector 0 */ writel (NVREG_MSI_VECTOR_0_ENABLED , base + NvRegMSIIrqMask); +netdev_info (dev,"MSI enabled\n" ); } }if (ret !=0 ) {
Signature · 1 line
1.7.3.1
PATCH v5 6/10 forcedeth: allow to silence "TX timeout" debug messages
In reply to David DecotignyFrom: Sameer Nanda <[email protected]>
This adds a new module parameter "debug_tx_timeout" to silence most debug messages in case of TX timeout. These messages don't provide a signal/noise ratio high enough for production systems and, with ~30kB logged each time, they tend to add to a cascade effect if the system is already under stress (memory pressure, disk, etc.).
By default, the parameter is clear, meaning that only a single warning will be reported.
--- drivers/net/ethernet/nvidia/forcedeth.c | 98 ++++++++++++++++++------------- 1 files changed, 57 insertions(+), 41 deletions(-)
drivers/net/ethernet/nvidia/forcedeth.c +57 -41
@@ -892,6 +892,11 @@ enum {static int dma_64bit =NV_DMA_64BIT_ENABLED ;/* +* Debug output control for tx_timeout +*/ +static bool debug_tx_timeout = false; + +/* * Crossover Detection * Realtek 8201 phy + some OEM boards do not work properly. */
@@ -2477,56 +2482,64 @@ static void nv_tx_timeout(struct net_device *dev)u32 status;union ring_type put_tx;int saved_tx_limit; -int i;if (np->msi_flags &NV_MSI_X_ENABLED ) status =readl (base + NvRegMSIXIrqStatus) &NVREG_IRQSTAT_MASK ;else status =readl (base + NvRegIrqStatus) &NVREG_IRQSTAT_MASK ; -netdev_info (dev,"Got tx_timeout. irq: %08x\n" , status); +netdev_warn (dev,"Got tx_timeout. irq status: %08x\n" , status); -netdev_info (dev,"Ring at %lx\n" , (unsigned long )np->ring_addr); -netdev_info (dev,"Dumping tx registers\n" ); -for (i =0 ; i <= np->register_size; i +=32 ) { -netdev_info (dev, -"%3x: %08x %08x %08x %08x %08x %08x %08x %08x\n" , - i, -readl (base + i +0 ),readl (base + i +4 ), -readl (base + i +8 ),readl (base + i +12 ), -readl (base + i +16 ),readl (base + i +20 ), -readl (base + i +24 ),readl (base + i +28 )); - } -netdev_info (dev,"Dumping tx ring\n" ); -for (i =0 ; i < np->tx_ring_size; i +=4 ) { -if (!nv_optimized (np)) { -netdev_info (dev, -"%03x: %08x %08x // %08x %08x // %08x %08x // %08x %08x\n" , - i, -le32_to_cpu (np->tx_ring.orig[i].buf), -le32_to_cpu (np->tx_ring.orig[i].flaglen), -le32_to_cpu (np->tx_ring.orig[i+1 ].buf), -le32_to_cpu (np->tx_ring.orig[i+1 ].flaglen), -le32_to_cpu (np->tx_ring.orig[i+2 ].buf), -le32_to_cpu (np->tx_ring.orig[i+2 ].flaglen), -le32_to_cpu (np->tx_ring.orig[i+3 ].buf), -le32_to_cpu (np->tx_ring.orig[i+3 ].flaglen)); - }else { +if (unlikely (debug_tx_timeout)) { +int i; + +netdev_info (dev,"Ring at %lx\n" , (unsigned long )np->ring_addr); +netdev_info (dev,"Dumping tx registers\n" ); +for (i =0 ; i <= np->register_size; i +=32 ) {netdev_info (dev, -"%03x: %08x %08x %08x // %08x %08x %08x // %08x %08x %08x // %08x %08x %08x\n" , +"%3x: %08x %08x %08x %08x " +"%08x %08x %08x %08x\n" , i, -le32_to_cpu (np->tx_ring.ex[i].bufhigh), -le32_to_cpu (np->tx_ring.ex[i].buflow), -le32_to_cpu (np->tx_ring.ex[i].flaglen), -le32_to_cpu (np->tx_ring.ex[i+1 ].bufhigh), -le32_to_cpu (np->tx_ring.ex[i+1 ].buflow), -le32_to_cpu (np->tx_ring.ex[i+1 ].flaglen), -le32_to_cpu (np->tx_ring.ex[i+2 ].bufhigh), -le32_to_cpu (np->tx_ring.ex[i+2 ].buflow), -le32_to_cpu (np->tx_ring.ex[i+2 ].flaglen), -le32_to_cpu (np->tx_ring.ex[i+3 ].bufhigh), -le32_to_cpu (np->tx_ring.ex[i+3 ].buflow), -le32_to_cpu (np->tx_ring.ex[i+3 ].flaglen)); +readl (base + i +0 ),readl (base + i +4 ), +readl (base + i +8 ),readl (base + i +12 ), +readl (base + i +16 ),readl (base + i +20 ), +readl (base + i +24 ),readl (base + i +28 )); + } +netdev_info (dev,"Dumping tx ring\n" ); +for (i =0 ; i < np->tx_ring_size; i +=4 ) { +if (!nv_optimized (np)) { +netdev_info (dev, +"%03x: %08x %08x // %08x %08x " +"// %08x %08x // %08x %08x\n" , + i, +le32_to_cpu (np->tx_ring.orig[i].buf), +le32_to_cpu (np->tx_ring.orig[i].flaglen), +le32_to_cpu (np->tx_ring.orig[i+1 ].buf), +le32_to_cpu (np->tx_ring.orig[i+1 ].flaglen), +le32_to_cpu (np->tx_ring.orig[i+2 ].buf), +le32_to_cpu (np->tx_ring.orig[i+2 ].flaglen), +le32_to_cpu (np->tx_ring.orig[i+3 ].buf), +le32_to_cpu (np->tx_ring.orig[i+3 ].flaglen)); + }else { +netdev_info (dev, +"%03x: %08x %08x %08x " +"// %08x %08x %08x " +"// %08x %08x %08x " +"// %08x %08x %08x\n" , + i, +le32_to_cpu (np->tx_ring.ex[i].bufhigh), +le32_to_cpu (np->tx_ring.ex[i].buflow), +le32_to_cpu (np->tx_ring.ex[i].flaglen), +le32_to_cpu (np->tx_ring.ex[i+1 ].bufhigh), +le32_to_cpu (np->tx_ring.ex[i+1 ].buflow), +le32_to_cpu (np->tx_ring.ex[i+1 ].flaglen), +le32_to_cpu (np->tx_ring.ex[i+2 ].bufhigh), +le32_to_cpu (np->tx_ring.ex[i+2 ].buflow), +le32_to_cpu (np->tx_ring.ex[i+2 ].flaglen), +le32_to_cpu (np->tx_ring.ex[i+3 ].bufhigh), +le32_to_cpu (np->tx_ring.ex[i+3 ].buflow), +le32_to_cpu (np->tx_ring.ex[i+3 ].flaglen)); + } } }
@@ -6156,6 +6169,9 @@ module_param(phy_cross, int, 0);MODULE_PARM_DESC (phy_cross,"Phy crossover detection for Realtek 8201 phy is enabled by setting to 1 and disabled by setting to 0." );module_param (phy_power_down, int,0 );MODULE_PARM_DESC (phy_power_down,"Power down phy and disable link when interface is down (1), or leave phy powered up (0)." ); +module_param (debug_tx_timeout, bool,0 ); +MODULE_PARM_DESC (debug_tx_timeout, +"Dump tx related registers and ring when tx_timeout happens" );MODULE_AUTHOR ("Manfred Spraul <[email protected]>" );MODULE_DESCRIPTION ("Reverse Engineered nForce ethernet driver" );
Signature · 1 line
1.7.3.1
PATCH v5 7/10 forcedeth: implement ndo_get_stats64() API
In reply to David DecotignyThis commit implements the ndo_get_stats64() API for forcedeth. Since these stats are being updated from different contexts (process and timer), this commit adds protection (locking + atomic variables).
Tested:
- 16-way SMP x86_64 ->
RX bytes:7244556582 (7.2 GB) TX bytes:181904254 (181.9 MB)
- pktgen + loopback: identical rx_bytes/tx_bytes and rx_packets/tx_packets
--- drivers/net/ethernet/nvidia/forcedeth.c | 182 ++++++++++++++++++++++++------- 1 files changed, 141 insertions(+), 41 deletions(-)
drivers/net/ethernet/nvidia/forcedeth.c +141 -41
@@ -692,6 +692,21 @@ struct nv_ethtool_stats {#define NV_DEV_STATISTICS_V2_COUNT (NV_DEV_STATISTICS_V3_COUNT - 3)#define NV_DEV_STATISTICS_V1_COUNT (NV_DEV_STATISTICS_V2_COUNT - 6) +/* driver statistics */ +struct nv_driver_stat { +atomic_t delta;/* increase since last nv_update_stats() */ +u64 total;/* cumulative, requires netdev_priv(dev)->stats_lock */ +}; + +#define NV_DRIVER_STAT_ATOMIC_INC (ptr_stat)/* atomic */ \ + ({ atomic_inc(&(ptr_stat)->delta); }) +#define NV_DRIVER_STAT_ATOMIC_ADD (ptr_stat,increment)/* atomic */ \ + ({ atomic_add((increment), &(ptr_stat)->delta); }) +#define NV_DRIVER_STAT_UPDATE_TOTAL (ptr_stat)/* requires stats_lock */ \ + ({ (ptr_stat)->total += atomic_xchg(&(ptr_stat)->delta, 0); }) +#define NV_DRIVER_STAT_GET_TOTAL (ptr_stat)/* requires stats_lock */ \ + ((ptr_stat)->total) +/* diagnostics */ #define NV_TEST_COUNT_BASE 3#define NV_TEST_COUNT_EXTENDED 4
@@ -736,6 +751,12 @@ struct nv_skb_map {* - tx setup is lockless: it relies on netif_tx_lock. Actual submission * needs netdev_priv(dev)->lock :-( * - set_multicast_list: preparation lockless, relies on netif_tx_lock. +* +* Stats are protected with stats_lock: +* - updated by nv_do_stats_poll (timer). This is meant to avoid +* integer wraparound in the NIC stats registers, at low frequency +* (0.1 Hz) +* - updated by nv_get_ethtool_stats + nv_get_stats64 */ /* in dev: base, irq */
@@ -745,9 +766,10 @@ struct fe_priv {struct net_device *dev;struct napi_struct napi; -/* General data: -* Locking: spin_lock(&np->lock); */ +/* stats are updated in syscall and timer */ +spinlock_t stats_lock;struct nv_ethtool_stats estats; +int in_shutdown;u32 linkspeed;int duplex;
@@ -798,6 +820,11 @@ struct fe_priv {u32 nic_poll_irq;int rx_ring_size; +/* RX software stats */ +struct nv_driver_stat stat_rx_packets; +struct nv_driver_stat stat_rx_bytes;/* not always available in HW */ +struct nv_driver_stat stat_rx_missed_errors; +/* media detection workaround. * Locking: Within irq hander or disable_irq+spin_lock(&np->lock); */
@@ -820,6 +847,11 @@ struct fe_priv {struct nv_skb_map *tx_end_flip;int tx_stop; +/* TX software stats */ +struct nv_driver_stat stat_tx_packets;/* not always available in HW */ +struct nv_driver_stat stat_tx_bytes; +struct nv_driver_stat stat_tx_dropped; +/* msi/msi-x fields */ u32 msi_flags;struct msix_entry msi_x_entry[NV_MSI_X_MAX_VECTORS ];
@@ -1635,11 +1667,19 @@ static void nv_mac_reset(struct net_device *dev)pci_push (base); } -static void nv_get_hw_stats (struct net_device *dev) +/* Caller must appropriately lock netdev_priv(dev)->stats_lock */ +static void nv_update_stats (struct net_device *dev) {struct fe_priv *np =netdev_priv (dev);u8 __iomem *base =get_hwbase (dev); +/* If it happens that this is run in top-half context, then +* replace the spin_lock of stats_lock with +* spin_lock_irqsave() in calling functions. */ +WARN_ONCE (in_irq (),"forcedeth: estats spin_lock(_bh) from top-half" ); +assert_spin_locked (&np->stats_lock); + +/* query hardware */ np->estats.tx_bytes +=readl (base + NvRegTxCnt); np->estats.tx_zero_rexmt +=readl (base + NvRegTxZeroReXmt); np->estats.tx_one_rexmt +=readl (base + NvRegTxOneReXmt);
@@ -1695,21 +1735,35 @@ static void nv_get_hw_stats(struct net_device *dev) np->estats.tx_multicast +=readl (base + NvRegTxMulticast); np->estats.tx_broadcast +=readl (base + NvRegTxBroadcast); } + +/* update software stats */ +NV_DRIVER_STAT_UPDATE_TOTAL (&np->stat_rx_packets); +NV_DRIVER_STAT_UPDATE_TOTAL (&np->stat_rx_bytes); +NV_DRIVER_STAT_UPDATE_TOTAL (&np->stat_rx_missed_errors); + +NV_DRIVER_STAT_UPDATE_TOTAL (&np->stat_tx_packets); +NV_DRIVER_STAT_UPDATE_TOTAL (&np->stat_tx_bytes); +NV_DRIVER_STAT_UPDATE_TOTAL (&np->stat_tx_dropped); }/* -* nv_get_stats: dev->get_stats function +* nv_get_stats64: dev->ndo_get_stats64 function * Get latest stats value from the nic. * Called with read_lock(&dev_base_lock) held for read - * only synchronized against unregister_netdevice. */ -static struct net_device_stats *nv_get_stats (struct net_device *dev) +static struct rtnl_link_stats64 * +nv_get_stats64 (struct net_device *dev,struct rtnl_link_stats64 *storage) + __acquires(&netdev_priv(dev)->stats_lock) + __releases(&netdev_priv(dev)->stats_lock) {struct fe_priv *np =netdev_priv (dev);/* If the nic supports hw counters then retrieve latest values */ -if (np->driver_data & (DEV_HAS_STATISTICS_V1 |DEV_HAS_STATISTICS_V2 |DEV_HAS_STATISTICS_V3 )) { -nv_get_hw_stats (dev); +if (np->driver_data &DEV_HAS_STATISTICS_V123 ) { +spin_lock_bh (&np->stats_lock); + +nv_update_stats (dev);/* * Note: because HW stats are not always available and
@@ -1721,17 +1775,40 @@ static struct net_device_stats *nv_get_stats(struct net_device *dev)* packet (Ethernet FCS CRC). */ -/* copy to net_device stats */ - dev->stats.tx_fifo_errors = np->estats.tx_fifo_errors; - dev->stats.tx_carrier_errors = np->estats.tx_carrier_errors; - dev->stats.rx_crc_errors = np->estats.rx_crc_errors; - dev->stats.rx_over_errors = np->estats.rx_over_errors; - dev->stats.rx_fifo_errors = np->estats.rx_drop_frame; - dev->stats.rx_errors = np->estats.rx_errors_total; - dev->stats.tx_errors = np->estats.tx_errors_total; - } - -return &dev->stats; +/* generic stats */ + storage->rx_packets =NV_DRIVER_STAT_GET_TOTAL ( + &np->stat_rx_packets); + storage->tx_packets =NV_DRIVER_STAT_GET_TOTAL ( + &np->stat_tx_packets); + storage->rx_bytes =NV_DRIVER_STAT_GET_TOTAL ( + &np->stat_rx_bytes); + storage->tx_bytes =NV_DRIVER_STAT_GET_TOTAL ( + &np->stat_tx_bytes); + storage->rx_errors = np->estats.rx_errors_total; + storage->tx_errors = np->estats.tx_errors_total; + storage->tx_dropped =NV_DRIVER_STAT_GET_TOTAL ( + &np->stat_tx_dropped); + +/* meaningful only when NIC supports stats v3 */ + storage->multicast = np->estats.rx_multicast; + +/* detailed rx_errors */ + storage->rx_length_errors = np->estats.rx_length_error; + storage->rx_over_errors = np->estats.rx_over_errors; + storage->rx_crc_errors = np->estats.rx_crc_errors; + storage->rx_frame_errors = np->estats.rx_frame_align_error; + storage->rx_fifo_errors = np->estats.rx_drop_frame; + storage->rx_missed_errors =NV_DRIVER_STAT_GET_TOTAL ( + &np->stat_rx_missed_errors); + +/* detailed tx_errors */ + storage->tx_carrier_errors = np->estats.tx_carrier_errors; + storage->tx_fifo_errors = np->estats.tx_fifo_errors; + +spin_unlock_bh (&np->stats_lock); + } + +return storage; }/*
@@ -1933,7 +2010,7 @@ static void nv_drain_tx(struct net_device *dev) np->tx_ring.ex[i].buflow =0 ; }if (nv_release_txskb (np, &np->tx_skb[i])) - dev->stats.tx_dropped++; +NV_DRIVER_STAT_ATOMIC_INC (&np->stat_tx_dropped); np->tx_skb[i].dma =0 ; np->tx_skb[i].dma_len =0 ; np->tx_skb[i].dma_single =0 ;
@@ -2390,11 +2467,15 @@ static int nv_tx_done(struct net_device *dev, int limit)if (np->desc_ver ==DESC_VER_1 ) {if (flags &NV_TX_LASTPACKET ) {if (flags &NV_TX_ERROR ) { -if ((flags &NV_TX_RETRYERROR ) && !(flags &NV_TX_RETRYCOUNT_MASK )) +if ((flags &NV_TX_RETRYERROR ) + && !(flags &NV_TX_RETRYCOUNT_MASK ))nv_legacybackoff_reseed (dev); }else { - dev->stats.tx_packets++; - dev->stats.tx_bytes += np->get_tx_ctx->skb->len; +NV_DRIVER_STAT_ATOMIC_INC ( + &np->stat_tx_packets); +NV_DRIVER_STAT_ATOMIC_ADD ( + &np->stat_tx_bytes, + np->get_tx_ctx->skb->len); }dev_kfree_skb_any (np->get_tx_ctx->skb); np->get_tx_ctx->skb =NULL ;
@@ -2403,11 +2484,15 @@ static int nv_tx_done(struct net_device *dev, int limit) }else {if (flags &NV_TX2_LASTPACKET ) {if (flags &NV_TX2_ERROR ) { -if ((flags &NV_TX2_RETRYERROR ) && !(flags &NV_TX2_RETRYCOUNT_MASK )) +if ((flags &NV_TX2_RETRYERROR ) + && !(flags &NV_TX2_RETRYCOUNT_MASK ))nv_legacybackoff_reseed (dev); }else { - dev->stats.tx_packets++; - dev->stats.tx_bytes += np->get_tx_ctx->skb->len; +NV_DRIVER_STAT_ATOMIC_INC ( + &np->stat_tx_packets); +NV_DRIVER_STAT_ATOMIC_ADD ( + &np->stat_tx_bytes, + np->get_tx_ctx->skb->len); }dev_kfree_skb_any (np->get_tx_ctx->skb); np->get_tx_ctx->skb =NULL ;
@@ -2441,15 +2526,18 @@ static int nv_tx_done_optimized(struct net_device *dev, int limit)if (flags &NV_TX2_LASTPACKET ) {if (flags &NV_TX2_ERROR ) { -if ((flags &NV_TX2_RETRYERROR ) && !(flags &NV_TX2_RETRYCOUNT_MASK )) { +if ((flags &NV_TX2_RETRYERROR ) + && !(flags &NV_TX2_RETRYCOUNT_MASK )) {if (np->driver_data &DEV_HAS_GEAR_MODE )nv_gear_backoff_reseed (dev);else nv_legacybackoff_reseed (dev); } }else { - dev->stats.tx_packets++; - dev->stats.tx_bytes += np->get_tx_ctx->skb->len; +NV_DRIVER_STAT_ATOMIC_INC (&np->stat_tx_packets); +NV_DRIVER_STAT_ATOMIC_ADD ( + &np->stat_tx_bytes, + np->get_tx_ctx->skb->len); }dev_kfree_skb_any (np->get_tx_ctx->skb);
@@ -2663,7 +2751,7 @@ static int nv_rx_process(struct net_device *dev, int limit)/* the rest are hard errors */ else {if (flags &NV_RX_MISSEDFRAME ) - dev->stats.rx_missed_errors++; +NV_DRIVER_STAT_ATOMIC_INC (&np->stat_rx_missed_errors);dev_kfree_skb (skb); goto next_pkt; }
@@ -2706,8 +2794,8 @@ static int nv_rx_process(struct net_device *dev, int limit)skb_put (skb, len); skb->protocol =eth_type_trans (skb, dev);napi_gro_receive (&np->napi, skb); - dev->stats.rx_packets++; - dev->stats.rx_bytes += len; +NV_DRIVER_STAT_ATOMIC_INC (&np->stat_rx_packets); +NV_DRIVER_STAT_ATOMIC_ADD (&np->stat_rx_bytes, len); next_pkt:if (unlikely (np->get_rx.orig++ == np->last_rx.orig)) np->get_rx.orig = np->first_rx.orig;
@@ -2790,8 +2878,8 @@ static int nv_rx_process_optimized(struct net_device *dev, int limit)__vlan_hwaccel_put_tag (skb, vid); }napi_gro_receive (&np->napi, skb); - dev->stats.rx_packets++; - dev->stats.rx_bytes += len; +NV_DRIVER_STAT_ATOMIC_INC (&np->stat_rx_packets); +NV_DRIVER_STAT_ATOMIC_ADD (&np->stat_rx_bytes, len); }else {dev_kfree_skb (skb); }
@@ -4000,11 +4088,18 @@ static void nv_poll_controller(struct net_device *dev)#endif static void nv_do_stats_poll (unsigned long data) + __acquires(&netdev_priv(dev)->stats_lock) + __releases(&netdev_priv(dev)->stats_lock) {struct net_device *dev = (struct net_device *) data;struct fe_priv *np =netdev_priv (dev); -nv_get_hw_stats (dev); +/* If lock is currently taken, the stats are being refreshed +* and hence fresh enough */ +if (spin_trylock (&np->stats_lock)) { +nv_update_stats (dev); +spin_unlock (&np->stats_lock); + }if (!np->in_shutdown) mod_timer(&np->stats_poll,
@@ -4711,14 +4806,18 @@ static int nv_get_sset_count(struct net_device *dev, int sset) } } -static void nv_get_ethtool_stats (struct net_device *dev,struct ethtool_stats *estats,u64 *buffer) +static void nv_get_ethtool_stats (struct net_device *dev, +struct ethtool_stats *estats,u64 *buffer) + __acquires(&netdev_priv(dev)->stats_lock) + __releases(&netdev_priv(dev)->stats_lock) {struct fe_priv *np =netdev_priv (dev); -/* update stats */ -nv_get_hw_stats (dev); - -memcpy (buffer, &np->estats,nv_get_sset_count (dev,ETH_SS_STATS )*sizeof (u64)); +spin_lock_bh (&np->stats_lock); +nv_update_stats (dev); +memcpy (buffer, &np->estats, +nv_get_sset_count (dev,ETH_SS_STATS )*sizeof (u64)); +spin_unlock_bh (&np->stats_lock); }static int nv_link_test (struct net_device *dev)
@@ -5362,7 +5461,7 @@ static int nv_close(struct net_device *dev)static const struct net_device_ops nv_netdev_ops = { .ndo_open = nv_open, .ndo_stop = nv_close, - .ndo_get_stats = nv_get_stats, + .ndo_get_stats64 = nv_get_stats64, .ndo_start_xmit = nv_start_xmit, .ndo_tx_timeout = nv_tx_timeout, .ndo_change_mtu = nv_change_mtu,
@@ -5379,7 +5478,7 @@ static const struct net_device_ops nv_netdev_ops = {static const struct net_device_ops nv_netdev_ops_optimized = { .ndo_open = nv_open, .ndo_stop = nv_close, - .ndo_get_stats = nv_get_stats, + .ndo_get_stats64 = nv_get_stats64, .ndo_start_xmit = nv_start_xmit_optimized, .ndo_tx_timeout = nv_tx_timeout, .ndo_change_mtu = nv_change_mtu,
@@ -5418,6 +5517,7 @@ static int __devinit nv_probe(struct pci_dev *pci_dev, const struct pci_device_i np->dev = dev; np->pci_dev = pci_dev;spin_lock_init (&np->lock); +spin_lock_init (&np->stats_lock);SET_NETDEV_DEV (dev, &pci_dev->dev);init_timer (&np->oom_kick);
Signature · 1 line
1.7.3.1
Re: [PATCH net-next v5 07/10] forcedeth: implement ndo_get_stats64() API
Quoted 8 lines · On Tue, 15 Nov 2011 21:15:55 -0800 David Decotigny…
This commit implements the
ndo_get_stats64()API for forcedeth. Since these stats are being updated from different contexts (process and timer), this commit adds protection (locking + atomic variables).Tested:
- 16-way SMPx86_64->RX bytes:7244556582 (7.2 GB) TX bytes:181904254 (181.9 MB)- pktgen + loopback: identical rx_bytes/tx_bytes and rx_packets/tx_packets
Rather than adding additional locking why not use a percpu structure to hold statistics. See loopback, macvlan, veth, xen-netfront.
Or realize that the Tx side is single threaded by dev->xmit_lock already and the Rx side is single threaded by NAPI.
Re: [PATCH net-next v5 07/10] forcedeth: implement ndo_get_stats64() API
Quoted 7 lines · Stephen Hemminger
Quoted 4 lines · On Tue, 15 Nov 2011 21:15:55 -0800 David Decotigny…
This commit implements the
ndo_get_stats64()API for forcedeth. Since these stats are being updated from different contexts (process and timer), this commit adds protection (locking + atomic variables).
Quoted 7 lines
Quoted 4 lines
Tested:
- 16-way SMPx86_64->RX bytes:7244556582 (7.2 GB) TX bytes:181904254 (181.9 MB)- pktgen + loopback: identical rx_bytes/tx_bytes and rx_packets/tx_packets
Rather than adding additional locking why not use a percpu structure to hold statistics. See loopback, macvlan, veth, xen-netfront.
percpu is stupid for a single-queue driver.
Quoted 2 lines
Or realize that the Tx side is single threaded by
dev->xmit_lockalready and the Rx side is single threaded by NAPI.
Right.
Ben.
Signature · 3 lines
Ben Hutchings, Staff Engineer, Solarflare Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Re: [PATCH net-next v5 07/10] forcedeth: implement ndo_get_stats64() API
Thanks for your feedback on these concerns.
Quoted 5 lines · Ben Hutchings
Quoted 2 lines · Stephen Hemminger
Or realize that the Tx side is single threaded by
dev->xmit_lockalready and the Rx side is single threaded by NAPI.Right.
Yes, that's what I meant by "Each software stat field is updated by one single writer." in a previous email. However, I think that TX and RX paths are not always synchronized. So I'm afraid that if I'm using a single seqcount, I might run into trouble in the absence of a lock around each update.... and I'd really prefer to avoid such a lock. Are you suggesting I should use 2 independent seqcounts? One for RX path, the other for TX path, all this without a lock around writers?
Side-note: what are the bad implications of using atomic_t (in the fast paths)?
Re: [PATCH net-next v5 07/10] forcedeth: implement ndo_get_stats64() API
Quoted 10 lines · David Decotigny
Thanks for your feedback on these concerns.
Quoted 5 lines · Ben Hutchings
Quoted 2 lines · Stephen Hemminger
Or realize that the Tx side is single threaded by
dev->xmit_lockalready and the Rx side is single threaded by NAPI.Right.
Quoted 7 lines
Yes, that's what I meant by "Each software stat field is updated by one single writer." in a previous email. However, I think that TX and RX paths are not always synchronized. So I'm afraid that if I'm using a single seqcount, I might run into trouble in the absence of a lock around each update.... and I'd really prefer to avoid such a lock. Are you suggesting I should use 2 independent seqcounts? One for RX path, the other for TX path, all this without a lock around writers?
Yes, there should be 1 for each path.
Ben.
Signature · 3 lines
Ben Hutchings, Staff Engineer, Solarflare Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Re: [PATCH net-next v5 07/10] forcedeth: implement ndo_get_stats64() API
Quoted 19 lines · Ben Hutchings
Quoted 10 lines · David Decotigny
Thanks for your feedback on these concerns.
Quoted 5 lines · Ben Hutchings
Quoted 2 lines · Stephen Hemminger
Or realize that the Tx side is single threaded by
dev->xmit_lockalready and the Rx side is single threaded by NAPI.Right.
Quoted 7 lines
Yes, that's what I meant by "Each software stat field is updated by one single writer." in a previous email. However, I think that TX and RX paths are not always synchronized. So I'm afraid that if I'm using a single seqcount, I might run into trouble in the absence of a lock around each update.... and I'd really prefer to avoid such a lock. Are you suggesting I should use 2 independent seqcounts? One for RX path, the other for TX path, all this without a lock around writers?
Quoted 1 line
Yes, there should be 1 for each path.
David, drivers/net/ethernet/marvell/sky2.c contains code like that (different syncpfor rx/tx)
TX path:
u64_stats_update_begin(&sky2->tx_stats.syncp);
++sky2->tx_stats.packets;
sky2->tx_stats.bytes += skb->len;
u64_stats_update_end(&sky2->tx_stats.syncp);RX path:
u64_stats_update_begin(&sky2->rx_stats.syncp);
sky2->rx_stats.packets += packets;
sky2->rx_stats.bytes += bytes;
u64_stats_update_end(&sky2->rx_stats.syncp);Re: [PATCH net-next v5 07/10] forcedeth: implement ndo_get_stats64() API
Thanks to you all!
I have an implementation that's similar to sky2's. I will run the usual set of tests and send the updated patch series here.
Regards,
--
David Decotigny
Quoted 41 lines · Eric Dumazet
Quoted 19 lines · Ben Hutchings
Quoted 10 lines · David Decotigny
Thanks for your feedback on these concerns.
Quoted 5 lines · Ben Hutchings
Quoted 2 lines · Stephen Hemminger
Or realize that the Tx side is single threaded by
dev->xmit_lockalready and the Rx side is single threaded by NAPI.Right.
Quoted 7 lines
Yes, that's what I meant by "Each software stat field is updated by one single writer." in a previous email. However, I think that TX and RX paths are not always synchronized. So I'm afraid that if I'm using a single seqcount, I might run into trouble in the absence of a lock around each update.... and I'd really prefer to avoid such a lock. Are you suggesting I should use 2 independent seqcounts? One for RX path, the other for TX path, all this without a lock around writers?
Quoted 1 line
Yes, there should be 1 for each path.
David,
drivers/net/ethernet/marvell/sky2.ccontains code like that (different syncpfor rx/tx)TX path:
u64_stats_update_begin(&sky2->tx_stats.syncp); ++sky2->tx_stats.packets;sky2->tx_stats.bytes+=skb->len;u64_stats_update_end(&sky2->tx_stats.syncp);RX path:
u64_stats_update_begin(&sky2->rx_stats.syncp);
sky2->rx_stats.packets+= packets;sky2->rx_stats.bytes+= bytes;u64_stats_update_end(&sky2->rx_stats.syncp);
PATCH v5 8/10 forcedeth: account for dropped RX frames
In reply to David DecotignyThis adds code to update the stats counter for dropped RX frames.
--- drivers/net/ethernet/nvidia/forcedeth.c | 12 ++++++++++-- 1 files changed, 10 insertions(+), 2 deletions(-)
drivers/net/ethernet/nvidia/forcedeth.c +10 -2
@@ -824,6 +824,7 @@ struct fe_priv {struct nv_driver_stat stat_rx_packets;struct nv_driver_stat stat_rx_bytes;/* not always available in HW */ struct nv_driver_stat stat_rx_missed_errors; +struct nv_driver_stat stat_rx_dropped;/* media detection workaround. * Locking: Within irq hander or disable_irq+spin_lock(&np->lock);
@@ -1739,6 +1740,7 @@ static void nv_update_stats(struct net_device *dev)/* update software stats */ NV_DRIVER_STAT_UPDATE_TOTAL (&np->stat_rx_packets);NV_DRIVER_STAT_UPDATE_TOTAL (&np->stat_rx_bytes); +NV_DRIVER_STAT_UPDATE_TOTAL (&np->stat_rx_dropped);NV_DRIVER_STAT_UPDATE_TOTAL (&np->stat_rx_missed_errors);NV_DRIVER_STAT_UPDATE_TOTAL (&np->stat_tx_packets);
@@ -1786,6 +1788,8 @@ nv_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *storage) &np->stat_tx_bytes); storage->rx_errors = np->estats.rx_errors_total; storage->tx_errors = np->estats.tx_errors_total; + storage->rx_dropped =NV_DRIVER_STAT_GET_TOTAL ( + &np->stat_rx_dropped); storage->tx_dropped =NV_DRIVER_STAT_GET_TOTAL ( &np->stat_tx_dropped);
@@ -1841,8 +1845,10 @@ static int nv_alloc_rx(struct net_device *dev) np->put_rx.orig = np->first_rx.orig;if (unlikely (np->put_rx_ctx++ == np->last_rx_ctx)) np->put_rx_ctx = np->first_rx_ctx; - }else + }else { +NV_DRIVER_STAT_ATOMIC_INC (&np->stat_rx_dropped);return 1 ; + } }return 0 ; }
@@ -1873,8 +1879,10 @@ static int nv_alloc_rx_optimized(struct net_device *dev) np->put_rx.ex = np->first_rx.ex;if (unlikely (np->put_rx_ctx++ == np->last_rx_ctx)) np->put_rx_ctx = np->first_rx_ctx; - }else + }else { +NV_DRIVER_STAT_ATOMIC_INC (&np->stat_rx_dropped);return 1 ; + } }return 0 ; }
Signature · 1 line
1.7.3.1
PATCH v5 9/10 forcedeth: stats updated with a deferrable timer
In reply to David DecotignyMark stats timer as deferrable: punctuality in waking the stats timer callback doesn't matter much, as it is responsible only to avoid integer wraparound.
We need at least 1 other timer to fire within 17s (fully loaded 1Gbps) to avoid wrap-arounds. Desired period is still 10s.
--- drivers/net/ethernet/nvidia/forcedeth.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/net/ethernet/nvidia/forcedeth.c +1 -1
@@ -5534,7 +5534,7 @@ static int __devinit nv_probe(struct pci_dev *pci_dev, const struct pci_device_iinit_timer (&np->nic_poll); np->nic_poll.data = (unsigned long ) dev; np->nic_poll.function = nv_do_nic_poll;/* timer handler */ -init_timer (&np->stats_poll); +init_timer_deferrable (&np->stats_poll); np->stats_poll.data = (unsigned long ) dev; np->stats_poll.function = nv_do_stats_poll;/* timer handler */
Signature · 1 line
1.7.3.1
PATCH v5 10/10 forcedeth: whitespace/indentation fixes
In reply to David Decotigny--- drivers/net/ethernet/nvidia/forcedeth.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/net/ethernet/nvidia/forcedeth.c +1 -1
@@ -65,7 +65,7 @@#include <linux/slab.h> #include <linux/uaccess.h> #include <linux/prefetch.h> -#include <linux/io.h> +#include <linux/io.h> #include <asm/irq.h> #include <asm/system.h>
Signature · 1 line
1.7.3.1