igb: Record hardware RX overruns in net_stats
Subsystems (2)
PATCH 1/1 igb: Record hardware RX overruns in net_stats
Hardware RX fifo overruns for the 82576 is stored in the RNBC (Receive No Buffers Count) register.
I choose the store the RNBC value in net_stats.rx_fifo_errors.
Saving the stats in dev->net_stats makes it visible via /proc/net/dev as "fifo", and thus viewable to ifconfig as "overruns" and 'netstat -i' as "RX-OVR".
The Receive No Buffers Count (RNBC) can already be queried by ethtool -S as "rx_no_buffer_count".
---
drivers/net/igb/igb_main.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
drivers/net/igb/igb_main.c +1 -0
@@ -3596,6 +3596,7 @@ void igb_update_stats(struct igb_adapter *adapter) adapter->net_stats.rx_crc_errors = adapter->stats.crcerrs; adapter->net_stats.rx_frame_errors = adapter->stats.algnerrc; adapter->net_stats.rx_missed_errors = adapter->stats.mpc; + adapter->net_stats.rx_fifo_errors = adapter->stats.rnbc;/* Tx Errors */ adapter->net_stats.tx_errors = adapter->stats.ecol +
Quoted 10 lines · Jesper Dangaard Brouer
Hardware RX fifo overruns for the 82576 is stored in the RNBC (Receive No Buffers Count) register.
I choose the store the RNBC value in net_stats.rx_fifo_errors.
Saving the stats in
dev->net_statsmakes it visible via/proc/net/devas "fifo", and thus viewable to ifconfig as "overruns" and 'netstat -i' as "RX-OVR".
Quoted 6 lines
The Receive No Buffers Count (RNBC) can already be queried by ethtool -S as "
rx_no_buffer_count".Signed-off-by Jesper Dangaard Brouer <[email protected]>---
NAK. RNBC is not a counter for buffer overruns, and so should not be counted as such.
Signature · 2 lines
Cheers, Jeff
David Miller
maintainerFrom: Jeff Kirsher <[email protected]>
Date: Tue, 5 May 2009 11:47:09 -0700
Quoted 2 lines
NAK. RNBC is not a counter for buffer overruns, and so should not be counted as such.
I'd say technically it is, it indicates that more packets arrived than the available receive buffers could handle.
If anything, this is the closest this device has for this kind of situation, and it's useful for diagnosing problems.
Quoted 8 lines · David Miller
From: Jeff Kirsher <[email protected]>
Date: Tue, 5 May 2009 11:47:09 -0700Quoted 2 lines
NAK. RNBC is not a counter for buffer overruns, and so should not be counted as such.
I'd say technically it is, it indicates that more packets arrived than the available receive buffers could handle.
I agree with DaveM. Technically it _is_ a buffer overflow, but in the host memory not the NIC. I'm sort of pushing the system into a situation where it cannot empty the receive buffers fast enough.
I can fairly easily provoke this situation by adding too many iptables rules, which (intentionally) cause high CPU load and causes ksoftirqd to run (I'm Oprofiling netfilter modules).
Quoted 2 lines
If anything, this is the closest this device has for this kind of situation, and it's useful for diagnosing problems.
Its really useful for diagnosing problems, and I'm betting that this is a real-life situation which people is going to experience. We might as well help our self to more easily identify this issue when people report drop problems.
Notice that I'm seeing:
rx_no_buffer_count: 136955
rx_missed_errors: 0
Thus, the rx_missed_errors is zero, which according to the datasheet is the "real" fifo drop (the MPC register, Missed Packets Count) and PCI bandwidth problem indications.
If we really should nitpick, then:
adapter->net_stats.rx_missed_errors = adapter->stats.mpc
Should then have been stored in the rx_fifo_errors. Notice that rx_missed_errors is presented to userspace as drops (see net/core/dev.c:2624).
I think that both MPC and RNBC should be stored in rx_fifo_errors (and of cause still keeping them seperate to ethtool -S).
I'll post two patches with these changes tomorrow, for you evaluation.
Please reconsider you NAK.
Greetings,
Jesper Brouer
--
-------------------------------------------------------------------
MSc. Master of Computer Science
Dept. of Computer Science, University of Copenhagen Author of http://www.adsl-optimizer.dk
-------------------------------------------------------------------
Quoted 48 lines · Jesper Dangaard Brouer
Quoted 8 lines · David Miller
From: Jeff Kirsher <[email protected]>
Date: Tue, 5 May 2009 11:47:09 -0700Quoted 2 lines
NAK. RNBC is not a counter for buffer overruns, and so should not be counted as such.
I'd say technically it is, it indicates that more packets arrived than the available receive buffers could handle.
I agree with DaveM. Technically it _is_ a buffer overflow, but in the host memory not the NIC. I'm sort of pushing the system into a situation where it cannot empty the receive buffers fast enough.
I can fairly easily provoke this situation by adding too many iptables rules, which (intentionally) cause high CPU load and causes ksoftirqd to run (I'm Oprofiling netfilter modules).
Quoted 2 lines
If anything, this is the closest this device has for this kind of situation, and it's useful for diagnosing problems.
Its really useful for diagnosing problems, and I'm betting that this is a real-life situation which people is going to experience. We might as well help our self to more easily identify this issue when people report drop problems.
Notice that I'm seeing:
rx_no_buffer_count: 136955
rx_missed_errors: 0Thus, the
rx_missed_errorsis zero, which according to the datasheet is the "real" fifo drop (the MPC register, Missed Packets Count) and PCI bandwidth problem indications.If we really should nitpick, then:
adapter->net_stats.rx_missed_errors =
adapter->stats.mpcShould then have been stored in the
rx_fifo_errors. Notice thatrx_missed_errorsis presented to userspace as drops (seenet/core/dev.c:2624).I think that both MPC and RNBC should be stored in
rx_fifo_errors(and of cause still keeping them seperate to ethtool -S).
Quoted 8 lines
I'll post two patches with these changes tomorrow, for you evaluation.
Please reconsider you NAK.
Greetings,
Jesper Brouer--
the manual[1] for the hardware says:
RNBC:
This register counts the number of times that frames were received when there were no available buffers in host memory to store those frames (receive descriptor head and tail pointers were equal). The packet is still received if there is space in the FIFO. This register only increments if receives are enabled. This register does not increment when flow control packets are received.
The critical bit "The packet is still received if there is space in the FIFO" (AND a host memory buffer becomes available) So the reason we don't want to put it in the net_stats stats for drops is that the packet
*wasn't* necessarily dropped.
The rx_missed errors is for packets that were definitely dropped, and is already stored in the net_stats structure.
Signature · 13 lines
Cheers, Jeff ------------------------------------------------------------------------------ The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel
David Miller
maintainerFrom: Jeff Kirsher <[email protected]>
Date: Tue, 5 May 2009 14:32:04 -0700
Quoted 9 lines
the manual[1] for the hardware says:
RNBC:
This register counts the number of times that frames were received when there were no available buffers in host memory to store those frames (receive descriptor head and tail pointers were equal). The packet is still received if there is space in the FIFO. This register only increments if receives are enabled. This register does not increment when flow control packets are received.
Quoted 8 lines
The critical bit "The packet is still received if there is space in the FIFO" (AND a host memory buffer becomes available) So the reason we don't want to put it in the
net_statsstats for drops is that the packet
*wasn't* necessarily dropped.The
rx_missederrors is for packets that were definitely dropped, and is already stored in thenet_statsstructure.
While not an "rx_missed" because we do eventually take the packet, conceptually it is a "fifo overflow" in the sense that we exceeded available receive resources at the time that the packet arrived.
Quoted 18 lines · David Miller
From: Jeff Kirsher <[email protected]>
Date: Tue, 5 May 2009 14:32:04 -0700Quoted 15 lines
the manual[1] for the hardware says:
RNBC:
This register counts the number of times that frames were received when there were no available buffers in host memory to store those frames (receive descriptor head and tail pointers were equal). The packet is still received if there is space in the FIFO. This register only increments if receives are enabled. This register does not increment when flow control packets are received.The critical bit "The packet is still received if there is space in the FIFO" (AND a host memory buffer becomes available) So the reason we don't want to put it in the
net_statsstats for drops is that the packet
*wasn't* necessarily dropped.
Quoted 7 lines
Quoted 2 lines
The
rx_missederrors is for packets that were definitely dropped, and is already stored in thenet_statsstructure.While not an "
rx_missed" because we do eventually take the packet, conceptually it is a "fifo overflow" in the sense that we exceeded available receive resources at the time that the packet arrived.
Yes, with this argumentation, the MPC should then be kept as "rx_missed" packets. And the RNBC stored as "rx_fifo_errors" as its an overflow indication, not a number of packets dropped.
Signature · 7 lines
Med venlig hilsen / Best regards Jesper Brouer ComX Networks A/S Linux Network developer Cand. Scient Datalog / MSc. Author of http://adsl-optimizer.dk LinkedIn: http://www.linkedin.com/in/brouer
Quoted 22 lines · Jesper Dangaard Brouer
Quoted 21 lines · David Miller
From: Jeff Kirsher <[email protected]>
Date: Tue, 5 May 2009 14:32:04 -0700Quoted 17 lines
the manual[1] for the hardware says:
RNBC:
This register counts the number of times that frames were received when there were no available buffers in host memory to store those frames (receive descriptor head and tail pointers were equal). The packet is still received if there is space in the FIFO. This register only increments if receives are enabled. This register does not increment when flow control packets are received.The critical bit "The packet is still received if there is space in the FIFO" (AND a host memory buffer becomes available) So the reason we don't want to put it in the
net_statsstats for drops is that the packet
*wasn't* necessarily dropped.The
rx_missederrors is for packets that were definitely dropped, and is already stored in thenet_statsstructure.
Quoted 8 lines
Quoted 4 lines
While not an "
rx_missed" because we do eventually take the packet, conceptually it is a "fifo overflow" in the sense that we exceeded available receive resources at the time that the packet arrived.Yes, with this argumentation, the MPC should then be kept as "
rx_missed" packets. And the RNBC stored as "rx_fifo_errors" as its an overflow indication, not a number of packets dropped.
The way RNBC works depends on how the queues themselves are configured. Specifically, if you have packet drop enabled per queue or not will affect RNBC.
In the SRRCTL registers, there is a DROP_EN bit, bit 31. If this bit is set to 1b for the queue in question, then the packet will be dropped when there are no buffers in the packet buffer. This does not mean the FIFO is full or has been overrun, it just means there's no more descriptors available in the Rx ring for that queue. In this case, RNBC is incremented, MPC is not.
If bit 31 in SRRCTL is 0b, then if there's no room in the packet buffer (no more descriptors available), the device tries to store the packet in the FIFO. RNBC will *not* be incremented in this case. If there's no space in the FIFO, then the packet is dropped. RNBC still is not incremented in this case, rather MPC will be incremented, since the packet was dropped due to the FIFO being full.
In 82576, according to the manual, SRRCTL bit 31 is 0b for queue 0 by default, and is 1b for all other queues by default.
I hope this helps explain what the hardware is doing, and how these two counters get used in overrun cases.
Cheers,
-PJ Waskiewicz
------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com
Quoted 29 lines · Peter P
Quoted 27 lines · Jesper Dangaard Brouer
Quoted 25 lines · David Miller
From: Jeff Kirsher <[email protected]>
Date: Tue, 5 May 2009 14:32:04 -0700Quoted 9 lines
the manual[1] for the hardware says:
RNBC:
This register counts the number of times that frames were received when there were no available buffers in host memory to store those frames (receive descriptor head and tail pointers were equal). The packet is still received if there is space in the FIFO. This register only increments if receives are enabled. This register does not increment when flow control packets are received.Quoted 8 lines
The critical bit "The packet is still received if there is space in the FIFO" (AND a host memory buffer becomes available) So the reason we don't want to put it in the
net_statsstats for drops is that the packet
*wasn't* necessarily dropped.The
rx_missederrors is for packets that were definitely dropped, and is already stored in thenet_statsstructure.While not an "
rx_missed" because we do eventually take the packet, conceptually it is a "fifo overflow" in the sense that we exceeded available receive resources at the time that the packet arrived.
Quoted 7 lines
Quoted 3 lines
Yes, with this argumentation, the MPC should then be kept as "
rx_missed" packets. And the RNBC stored as "rx_fifo_errors" as its an overflow indication, not a number of packets dropped.The way RNBC works depends on how the queues themselves are configured. Specifically, if you have packet drop enabled per queue or not will affect RNBC.
Very good description, thank you Peter.
But I could not resist to actually verify/test it, and my observations differ some! ;-) (patch in bottom indicate where I set it in the code)
Quoted 6 lines
In the SRRCTL registers, there is a
DROP_ENbit, bit 31. If this bit is set to 1b for the queue in question, then the packet will be dropped when there are no buffers in the packet buffer. This does not mean the FIFO is full or has been overrun, it just means there's no more descriptors available in the Rx ring for that queue. In this case, RNBC is incremented, MPC is not.
My experience is that if DROP_EN bit is *set*. Then I cannot find the drop count anywhere... not in RNBC and not in MPC ... and I can still see the drops with my netfilter module mp2t.
ethtool -S eth21 | egrep 'rx_no_buffer_count|rx_miss'
rx_no_buffer_count: 0
rx_missed_errors: 0
I'm guessing that the drop counters are now in the per queue RQDPC register (Receive Queue Drop Packet Count), but reading that is not implemented in the driver.
(kernel: [438792.665028] Hawk hack -- Register: srrctl:[0x82000002])
Quoted 6 lines
If bit 31 in SRRCTL is 0b, then if there's no room in the packet buffer (no more descriptors available), the device tries to store the packet in the FIFO. RNBC will *not* be incremented in this case. If there's no space in the FIFO, then the packet is dropped. RNBC still is not incremented in this case, rather MPC will be incremented, since the packet was dropped due to the FIFO being full.
My experience is that if DROP_EN bit is *NOT* set. Then the RNBC *is* incremented...
ethtool -S eth21 | egrep 'rx_no_buffer_count|rx_miss'
rx_no_buffer_count: 26436
rx_missed_errors: 0(kernel: [439261.463628] Hawk hack -- Register: srrctl:[0x2000002])
Quoted 2 lines
In 82576, according to the manual, SRRCTL bit 31 is 0b for queue 0 by default, and is 1b for all other queues by default.
Funny default...
Quoted 2 lines
I hope this helps explain what the hardware is doing, and how these two counters get used in overrun cases.
Signature · 16 lines
Med venlig hilsen / Best regards Jesper Brouer ComX Networks A/S Linux Network developer Cand. Scient Datalog / MSc. Author of http://adsl-optimizer.dk LinkedIn: http://www.linkedin.com/in/brouer Testing the SRRCTL_DROP_EN bit behavior. From: Jesper Dangaard Brouer <[email protected]> --- drivers/net/igb/igb_main.c | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-)
drivers/net/igb/igb_main.c +6 -1
@@ -49,7 +49,7 @@#endif #include "igb.h" -#define DRV_VERSION "1.3.16-k2" +#define DRV_VERSION "1.3.16-k2-test-drop-bit"char igb_driver_name[] ="igb" ;char igb_driver_version[] =DRV_VERSION ; static const char igb_driver_string[] =
@@ -2091,6 +2091,11 @@ static void igb_setup_rctl(struct igb_adapter *adapter)wr32 (E1000_VMOLR (j), vmolr); } +/* Hawk: Hack to test the SRRCTL_DROP_EN bit behavior */ + srrctl &= ~E1000_SRRCTL_DROP_EN ;/* Unset bit */ +//srrctl |= E1000_SRRCTL_DROP_EN; /* Set bit */ +printk (KERN_INFO "Hawk hack -- Register: srrctl:[0x%X]\n" , srrctl); +for (i =0 ; i < adapter->num_rx_queues; i++) { j = adapter->rx_ring[i].reg_idx;wr32 (E1000_SRRCTL (j), srrctl);
------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com
Quoted 14 lines · Jesper Dangaard Brouer
Quoted 3 lines
In the SRRCTL registers, there is a
DROP_ENbit, bit 31. If this bit is set to 1b for the queue in question, then the packet will be dropped when there are no buffers in the packet buffer. This doesnot
Quoted 1 line
mean the FIFO is full or has been overrun, it just means there's no
more
Quoted 1 line
descriptors available in the Rx ring for that queue. In this case,
RNBC
Quoted 1 line
is incremented, MPC is not.
My experience is that if
DROP_ENbit is *set*. Then I cannot find the drop count anywhere... not in RNBC and not in MPC ... and I can still see the drops with my netfilter module mp2t.
Quoted 7 lines
ethtool -S eth21 | egrep 'rx_no_buffer_count|rx_miss'
rx_no_buffer_count: 0
rx_missed_errors: 0I'm guessing that the drop counters are now in the per queue RQDPC register (Receive Queue Drop Packet Count), but reading that is not implemented in the driver.
Just did a quick implemenation in the driver, see draft patch below (not ready for inclusion yet!).
An my guess was correct, the drops count is stored per queue in RQDPC. The only really anoying thing is that this counter is only 12 bit, thus overruns can occur quickly. (This reminds me of my drop count implemenation for Sun niu driver. This counter has less bits that the Sun counter, but at least it automatically clears on reads)
I'll post some more clean patches tomorrow...
Signature · 18 lines
Med venlig hilsen / Best regards Jesper Brouer ComX Networks A/S Linux Network developer Cand. Scient Datalog / MSc. Author of http://adsl-optimizer.dk LinkedIn: http://www.linkedin.com/in/brouer Quick implemenation of register: Receive Queue Drop Packet Count - RQDPC (0xC030 + 0x40*n [n=0...15]; RC). --- drivers/net/igb/e1000_regs.h | 2 ++ drivers/net/igb/igb.h | 1 + drivers/net/igb/igb_ethtool.c | 4 ++++ drivers/net/igb/igb_main.c | 6 ++++++ 4 files changed, 13 insertions(+), 0 deletions(-)
drivers/net/igb/e1000_regs.h +2 -0
@@ -165,6 +165,8 @@ enum { : (0x0C018 + ((_n) * 0x40)))#define E1000_RXDCTL (_n) ((_n) < 4 ? (0x02828 + ((_n) * 0x100)) \ : (0x0C028 + ((_n) * 0x40))) +#define E1000_RQDPC (_n) ((_n) < 4 ? (0x02830 + ((_n) * 0x100)) \ + : (0x0C030 + ((_n) * 0x40)))#define E1000_TDBAL (_n) ((_n) < 4 ? (0x03800 + ((_n) * 0x100)) \ : (0x0E000 + ((_n) * 0x40)))#define E1000_TDBAH (_n) ((_n) < 4 ? (0x03804 + ((_n) * 0x100)) \
drivers/net/igb/igb.h +1 -0
@@ -140,6 +140,7 @@ struct igb_buffer {struct igb_queue_stats {u64 packets;u64 bytes; +u64 drops; };struct igb_ring {
drivers/net/igb/igb_ethtool.c +4 -0
@@ -1998,12 +1998,16 @@ static void igb_get_strings(struct net_device *netdev, u32 stringset, u8 *data) p +=ETH_GSTRING_LEN ;sprintf (p,"tx_queue_%u_bytes" , i); p +=ETH_GSTRING_LEN ; +sprintf (p,"tx_queue_%u_drops" , i); + p +=ETH_GSTRING_LEN ; }for (i =0 ; i < adapter->num_rx_queues; i++) {sprintf (p,"rx_queue_%u_packets" , i); p +=ETH_GSTRING_LEN ;sprintf (p,"rx_queue_%u_bytes" , i); p +=ETH_GSTRING_LEN ; +sprintf (p,"rx_queue_%u_drops" , i); + p +=ETH_GSTRING_LEN ; }/* BUG_ON(p - data != IGB_STATS_LEN * ETH_GSTRING_LEN); */ break ;
drivers/net/igb/igb_main.c +6 -0
@@ -3494,6 +3494,7 @@ void igb_update_stats(struct igb_adapter *adapter)struct e1000_hw *hw = &adapter->hw;struct pci_dev *pdev = adapter->pdev;u16 phy_tmp; +int i;#define PHY_IDLE_ERROR_COUNT_MASK 0x00FF
@@ -3583,6 +3584,11 @@ void igb_update_stats(struct igb_adapter *adapter) adapter->net_stats.multicast = adapter->stats.mprc; adapter->net_stats.collisions = adapter->stats.colc; +/* Read out drops stats per RX queue */ +for (i =0 ; i < adapter->num_rx_queues; i++) { + adapter->rx_ring[i].rx_stats.drops +=rd32 (E1000_RQDPC (i)); + } +/* Rx Errors */ /* RLEC on some newer hardware can be incorrect so build
Quoted 26 lines · Jesper Dangaard Brouer
Quoted 14 lines · Jesper Dangaard Brouer
Quoted 3 lines
In the SRRCTL registers, there is a
DROP_ENbit, bit 31. If this bit is set to 1b for the queue in question, then the packet will be dropped when there are no buffers in the packet buffer. This doesnot
Quoted 1 line
mean the FIFO is full or has been overrun, it just means there's no
more
Quoted 1 line
descriptors available in the Rx ring for that queue. In this case,
RNBC
Quoted 1 line
is incremented, MPC is not.
My experience is that if
DROP_ENbit is *set*. Then I cannot find the drop count anywhere... not in RNBC and not in MPC ... and I can still see the drops with my netfilter module mp2t.Quoted 7 lines
ethtool -S eth21 | egrep 'rx_no_buffer_count|rx_miss'
rx_no_buffer_count: 0
rx_missed_errors: 0I'm guessing that the drop counters are now in the per queue RQDPC register (Receive Queue Drop Packet Count), but reading that is not implemented in the driver.
Just did a quick implemenation in the driver, see draft patch below (not ready for inclusion yet!).
Quoted 7 lines
An my guess was correct, the drops count is stored per queue in RQDPC. The only really anoying thing is that this counter is only 12 bit, thus overruns can occur quickly. (This reminds me of my drop count implemenation for Sun niu driver. This counter has less bits that the Sun counter, but at least it automatically clears on reads)
I'll post some more clean patches tomorrow...
Ok, so 82576 is much more like 82599 with the queue counters, but different enough to be annoying. Yes, I'm seeing this in the data manual for 82576. Honestly, we have too many damn counters for packet accounting. I'm very happy you're getting what you wanted now. Your patch looks reasonable to me; please let us know if you have any other questions about the part that we might be able to help with.
Cheers,
-PJ Waskiewicz
------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com
Quoted 6 lines
Its really useful for diagnosing problems, and I'm betting that this is a
real-life situation which people is going to experience. We might as well
help our self to more easily identify this issue when people report drop problems.
The problem is that the RNBC aren't dropped packets as the numbers you have show. While we can agree that the MPC are the actual dropped packets and could eaily be be used in the fifo overflow count since the packets were really dropped.
Quoted 3 lines
I think that both MPC and RNBC should be stored in
rx_fifo_errors(and of
cause still keeping them seperate to ethtool -S).
This would count RNBC packets as packets that the stack did not process, which it did. The MPC packets were not processed by the stack and should be counted as dropped. As you point out, both counts are available via ethtool -S.
Quoted 1 line
I'll post two patches with these changes tomorrow, for you evaluation.
Thanks, we look forward to see them.
Cheers,
John
-----------------------------------------------------------
"...that your people will judge you on what you can build, not what you destroy.", B. Obama, 2009
Quoted 7 lines · John
Quoted 4 lines
Its really useful for diagnosing problems, and I'm betting that this is a real-life situation which people is going to experience. We might as well help our self to more easily identify this issue when people report drop problems.
The problem is that the RNBC aren't dropped packets as the numbers you have show.
Just to make it clear, I am experiencing dropped packets. The reason I positivly know this, is that I'm writing a MPEG2-TS drop detection netfilter module. Which were the only reason I discovered the packet drops and the "hidden" RNBC counter via ethtool.
I have read the datasheeet, and with Jeffrey's detailed explaination, I do know that this number might be higher than the actually drops I'm experiencing.
Quoted 3 lines
While we can agree that the MPC are the actual dropped packets and could eaily be be used in the fifo overflow count since the packets were really dropped.
Well, then I think we should keep MPC as drops, and use the fifo_errors as an fifo overflow indication containing RNBC count.
Quoted 11 lines
Quoted 2 lines
I think that both MPC and RNBC should be stored in
rx_fifo_errors(and of cause still keeping them seperate to ethtool -S).This would count RNBC packets as packets that the stack did not process, which it did. The MPC packets were not processed by the stack and should be counted as dropped. As you point out, both counts are available via ethtool -S.
Quoted 1 line
I'll post two patches with these changes tomorrow, for you evaluation.
Thanks, we look forward to see them.
I'll keep it to one patch (with an extra comment reflecting this disscussion), as you have convinced me that the MPC should stay as "rx_missed", as this is presented to userspace as a positive drop.
Signature · 15 lines
Med venlig hilsen / Best regards Jesper Brouer ComX Networks A/S Linux Network developer Cand. Scient Datalog / MSc. Author of http://adsl-optimizer.dk LinkedIn: http://www.linkedin.com/in/brouer ------------------------------------------------------------------------------ The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com
PATCH v2 1/1 igb: Record host memory receive overflow in net_stats
The RNBC (Receive No Buffers Count) register for the 82576, indicate that frames were received when there were no available buffers in host memory to store those frames (receive descriptor head and tail pointers were equal). The packet is still received by the NIC if there is space in the FIFO
I have shown that this situation can arise when the kernel is too busy else where.
As the the RNBC value is not necessary a packet drop, I choose to store the RNBC value in net_stats.rx_fifo_errors.
Saving the stats in dev->net_stats makes it visible via /proc/net/dev as "fifo", and thus viewable to ifconfig as "overruns" and 'netstat -i' as "RX-OVR".
The Receive No Buffers Count (RNBC) can already be queried by ethtool -S as "rx_no_buffer_count".
---
drivers/net/igb/igb_main.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-)
drivers/net/igb/igb_main.c +4 -0
@@ -3597,6 +3597,10 @@ void igb_update_stats(struct igb_adapter *adapter) adapter->net_stats.rx_frame_errors = adapter->stats.algnerrc; adapter->net_stats.rx_missed_errors = adapter->stats.mpc; +/* Note RNBC (Receive No Buffers Count) is an overflow +* indication, thus not necessary a dropped packet. */ + adapter->net_stats.rx_fifo_errors = adapter->stats.rnbc; +/* Tx Errors */ adapter->net_stats.tx_errors = adapter->stats.ecol + adapter->stats.latecol;