igb: enable RX network flow classification
Subsystems (2)
v5 cover igb: enable RX network flow classification
After apply this series of patches, igb driver will supports advanced RX filter that direct receive packets by flows to different hardware queue. Enables tight control on routing a flow in the platform.
In our product, we use the the RX traffic classification to gurantee the PTP(ethertype is 0x88F7) packets won't be flooded by best effort packet. PTP packets is always be processed by Linux network stack.
step 1.
use the ethertype filter to filter most of packet(0x0800) to HW queue;
step 2.
Add a rule to forward the PTP packets to another HW queue
The ethtool commands and options:
-n --show-nfc
Retrieves the receive network flow classification configurations
-N --config-nfc
Configures the receive network flow classification classification
Change history of this patches:
Version 1:
Split the patch to 3 patches for review;
Save the filters and restore them after a reset;
In ethtool command, use key word "proto" to replace "vlan-etype:
Fix bugs
Version 2:
Rebase the patches on the the latest dev-queue;
Version 3:
Report a meaningful error code for misoperation;
Version 4:
Rebase the patches on latest dev-queue;
Update the patch description and code comments;
Version 5:
Rebase the patches on latest dev-queue;
Update the patch description;
Gangfeng Huang (4):
igb: add support of RX network flow classification
igb: support RX flow classification by ethertype
igb: support RX flow classification by VLAN priority
igb: fix error code in igb_add_ethtool_nfc_entry()
drivers/net/ethernet/intel/igb/e1000_82575.h | 5 + drivers/net/ethernet/intel/igb/e1000_defines.h | 4 + drivers/net/ethernet/intel/igb/e1000_regs.h | 1 + drivers/net/ethernet/intel/igb/igb.h | 53 ++++ drivers/net/ethernet/intel/igb/igb_ethtool.c | 345 +++++++++++++++++++++++++ drivers/net/ethernet/intel/igb/igb_main.c | 44 ++++ drivers/net/ethernet/intel/igb/igb_ptp.c | 4 +- 7 files changed, 454 insertions(+), 2 deletions(-)
Signature · 1 line
2.7.2
PATCH v5 1/4 igb: add support of RX network flow classification
From: Gangfeng Huang <[email protected]>
This patch is meant to allow for RX network flow classification to insert and remove Rx filter by ethtool. Ethtool interface has it's own rules manager
Show all filters:
$ ethtool -n eth0 4 RX rings available Total 2 rules
--- drivers/net/ethernet/intel/igb/igb.h | 32 +++++ drivers/net/ethernet/intel/igb/igb_ethtool.c | 193 +++++++++++++++++++++++++++ drivers/net/ethernet/intel/igb/igb_main.c | 44 ++++++ 3 files changed, 269 insertions(+)
drivers/net/ethernet/intel/igb/igb.h +32 -0
@@ -355,6 +355,27 @@ struct hwmon_buff {#define IGB_N_SDP 4#define IGB_RETA_SIZE 128 +enum igb_filter_match_flags { +IGB_FILTER_FLAG_NONE =0x0 , +}; + +#define IGB_MAX_RXNFC_FILTERS 16 + +/* RX network flow classification data structure */ +struct igb_nfc_input { +/* Byte layout in order, all values with MSB first: +* match_flags - 1 byte +*/ +u8 match_flags; +}; + +struct igb_nfc_filter { +struct hlist_node nfc_node; +struct igb_nfc_input filter; +u16 sw_idx; +u16 action; +}; +/* board specific private data structure */ struct igb_adapter {unsigned long active_vlans[BITS_TO_LONGS (VLAN_N_VID )];
@@ -472,6 +493,12 @@ struct igb_adapter {int copper_tries;struct e1000_info ei;u16 eee_advert; + +/* RX network flow classification support */ +struct hlist_head nfc_filter_list; +unsigned int nfc_filter_count; +/* lock for RX network flow classification filter */ +spinlock_t nfc_lock; };#define IGB_FLAG_HAS_MSI BIT(0)
@@ -594,4 +621,9 @@ static inline struct netdev_queue *txring_txq(const struct igb_ring *tx_ring) return netdev_get_tx_queue(tx_ring->netdev, tx_ring->queue_index); } +int igb_add_filter (struct igb_adapter *adapter, +struct igb_nfc_filter *input); +int igb_erase_filter (struct igb_adapter *adapter, +struct igb_nfc_filter *input); +#endif /* _IGB_H_ */
drivers/net/ethernet/intel/igb/igb_ethtool.c +193 -0
217 lines not on this page · raw message
drivers/net/ethernet/intel/igb/igb_main.c +44 -0
72 lines not on this page · raw message
Signature · 1 line
2.7.2
Brown
PATCH v5 1/4 igb: add support of RX network flow classification
Quoted 20 lines
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Gangfeng
Sent: Monday, May 9, 2016 2:28 AM
To: intel-wired-lan at lists.osuosl.org
Cc: Gangfeng Huang <[email protected]>; Ruhao Gao <[email protected]>
Subject: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classificationFrom: Gangfeng Huang <[email protected]>
This patch is meant to allow for RX network flow classification to insert and remove Rx filter by ethtool. Ethtool interface has it's own rules manager
Show all filters:
$ ethtool -n eth0 4 RX rings available Total 2 rules
Quoted 8 lines
Signed-off-by Ruhao Gao <[email protected]>Signed-off-by Gangfeng Huang <[email protected]>--- drivers/net/ethernet/intel/igb/igb.h | 32 +++++ drivers/net/ethernet/intel/igb/igb_ethtool.c | 193 +++++++++++++++++++++++++++ drivers/net/ethernet/intel/igb/igb_main.c | 44 ++++++ 3 files changed, 269 insertions(+)
This patch is causing 3/4 of my regression systems to fail. Driver load seems normal, but applying an IP address via ifconfig causes the following splat in dmesg and /var/log/messages:
----------------------------------------------
May 16 14:37:50 u1486 kernel: Hardware name: Supermicro A1SAi/A1SRi, BIOS 1.0b 11/06/2013 May 16 14:37:50 u1486 kernel: 0000000000000000 ffff880849ad3938 ffffffff813373d7 0000000000000007 May 16 14:37:50 u1486 kernel: 0000000000000006 0000000000000000 ffff88085c2f6770 ffff880849ad3a58 May 16 14:37:50 u1486 kernel: ffffffff810c4e13 ffff880849ad39f8 0000000000000005 0000000000000000 May 16 14:37:50 u1486 kernel: Call Trace: May 16 14:37:50 u1486 kernel: [<ffffffff813373d7>] dump_stack+0x6b/0xa4 May 16 14:37:50 u1486 kernel: [<ffffffff810c4e13>] register_lock_class+0x523/0x5c0 May 16 14:37:50 u1486 kernel: [<ffffffff8136644b>] ? check_preemption_disabled+0x1b/0x110 May 16 14:37:50 u1486 kernel: [<ffffffff811f5655>] ? kfree+0x1a5/0x3a0 May 16 14:37:50 u1486 kernel: [<ffffffff81366553>] ? __this_cpu_preempt_check+0x13/0x20 May 16 14:37:50 u1486 kernel: [<ffffffff810c7ae0>] __lock_acquire+0x80/0x5d0 May 16 14:37:50 u1486 kernel: [<ffffffff811f83f5>] ? __kmalloc+0x265/0x3a0 May 16 14:37:50 u1486 kernel: [<ffffffffa051864f>] ? kzalloc+0xf/0x20 [igb] May 16 14:37:50 u1486 kernel: [<ffffffff810c80fa>] lock_acquire+0xca/0x240 May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>] ? igb_configure+0xaf/0x1d0 [igb] May 16 14:37:50 u1486 kernel: [<ffffffff815b958b>] ? netdev_rss_key_fill+0x5b/0xa0 May 16 14:37:50 u1486 kernel: [<ffffffffa052dfb9>] ? igb_vfta_set+0x189/0x1f0 [igb] May 16 14:37:50 u1486 kernel: [<ffffffff816a8930>] _raw_spin_lock+0x40/0x80 May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>] ? igb_configure+0xaf/0x1d0 [igb] May 16 14:37:50 u1486 kernel: [<ffffffffa051bf62>] ? igb_setup_rctl+0x22/0xb0 [igb] May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>] igb_configure+0xaf/0x1d0 [igb] May 16 14:37:50 u1486 kernel: [<ffffffffa052408d>] __igb_open+0xfd/0x300 [igb] May 16 14:37:50 u1486 kernel: [<ffffffff815ab260>] ? call_netdevice_notifiers_info+0x40/0x70 May 16 14:37:50 u1486 kernel: [<ffffffffa0524420>] igb_open+0x10/0x20 [igb] May 16 14:37:50 u1486 kernel: [<ffffffff815ac7f8>] __dev_open+0xb8/0x110 May 16 14:37:50 u1486 kernel: [<ffffffff815ac5fc>] __dev_change_flags+0xac/0x180 May 16 14:37:50 u1486 kernel: [<ffffffff815ac700>] dev_change_flags+0x30/0x70 May 16 14:37:50 u1486 kernel: [<ffffffff815c6685>] ? lockdep_rtnl_is_held+0x15/0x20 May 16 14:37:50 u1486 kernel: [<ffffffff816403a5>] devinet_ioctl+0x5b5/0x620 May 16 14:37:50 u1486 kernel: [<ffffffff81156660>] ? trace_buffer_unlock_commit+0x60/0x80 May 16 14:37:50 u1486 kernel: [<ffffffff81643033>] inet_ioctl+0x63/0x80 May 16 14:37:50 u1486 kernel: [<ffffffff8158fd60>] sock_do_ioctl+0x30/0x70 May 16 14:37:50 u1486 kernel: [<ffffffff815901b3>] sock_ioctl+0x73/0x280 May 16 14:37:50 u1486 kernel: [<ffffffff8121f678>] vfs_ioctl+0x18/0x30 May 16 14:37:50 u1486 kernel: [<ffffffff81220057>] do_vfs_ioctl+0x87/0x430 May 16 14:37:50 u1486 kernel: [<ffffffff8100297e>] ? syscall_trace_enter_phase2+0x6e/0x280 May 16 14:37:50 u1486 kernel: [<ffffffff81220492>] SyS_ioctl+0x92/0xa0 May 16 14:37:50 u1486 kernel: [<ffffffff81002fd3>] do_syscall_64+0x63/0x130 May 16 14:37:50 u1486 kernel: [<ffffffff8100201b>] ? trace_hardirqs_on_thunk+0x1b/0x1d May 16 14:37:50 u1486 kernel: [<ffffffff816a981a>] entry_SYSCALL64_slow_path+0x25/0x25
----------------------------------------------
The system will then seem to function normally for a few minutes more then, even if left completely idle, will spew out another splat and the console freezes. A few more minutes and another splat hits the screen, but he console is otherwise still locked up. These later splats do not get captured to the logs.
I am seeing this on systems with a pair i350s and 82576s, a quad i354 and an i211 and pair of 82580s. A system with a pair of 82580 ports and an i210 seems fine.
Brown
PATCH v5 1/4 igb: add support of RX network flow classification
Quoted 39 lines
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Brown, Aaron F
Sent: Monday, May 16, 2016 3:09 PM
To: Gangfeng <[email protected]>; intel-wired-lan at lists.osuosl.org
Cc: Ruhao Gao <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classificationQuoted 20 lines
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Gangfeng
Sent: Monday, May 9, 2016 2:28 AM
To: intel-wired-lan at lists.osuosl.org
Cc: Gangfeng Huang <[email protected]>; Ruhao Gao <[email protected]>
Subject: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classificationFrom: Gangfeng Huang <[email protected]>
This patch is meant to allow for RX network flow classification to insert and remove Rx filter by ethtool. Ethtool interface has it's own rules manager
Show all filters:
$ ethtool -n eth0 4 RX rings available Total 2 rulesQuoted 8 lines
Signed-off-by Ruhao Gao <[email protected]>Signed-off-by Gangfeng Huang <[email protected]>--- drivers/net/ethernet/intel/igb/igb.h | 32 +++++ drivers/net/ethernet/intel/igb/igb_ethtool.c | 193 +++++++++++++++++++++++++++ drivers/net/ethernet/intel/igb/igb_main.c | 44 ++++++ 3 files changed, 269 insertions(+)This patch is causing 3/4 of my regression systems to fail. Driver load seems normal, but applying an IP address via ifconfig causes the following splat in
<snip>
Quoted 5 lines
The system will then seem to function normally for a few minutes more then, even if left completely idle, will spew out another splat and the console freezes. A few more minutes and another splat hits the screen, but he console is otherwise still locked up. These later splats do not get captured to the logs.
This second set of splats is not related to this patch or the series. Only the splat that follows address binding seems to be caused by this patch. A later series of patches for igb ptp seems to be the culprit for the delayed freeze trace.
PATCH v5 1/4 igb: add support of RX network flow classification
In reply to BrownQuoted 29 lines · Aaron F
Quoted 20 lines
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Gangfeng
Sent: Monday, May 9, 2016 2:28 AM
To: intel-wired-lan at lists.osuosl.org
Cc: Gangfeng Huang <[email protected]>; Ruhao Gao <[email protected]>
Subject: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classificationFrom: Gangfeng Huang <[email protected]>
This patch is meant to allow for RX network flow classification to insert and remove Rx filter by ethtool. Ethtool interface has it's own rules manager
Show all filters:
$ ethtool -n eth0 4 RX rings available Total 2 rulesQuoted 8 lines
Signed-off-by Ruhao Gao <[email protected]>Signed-off-by Gangfeng Huang <[email protected]>--- drivers/net/ethernet/intel/igb/igb.h | 32 +++++ drivers/net/ethernet/intel/igb/igb_ethtool.c | 193 +++++++++++++++++++++++++++ drivers/net/ethernet/intel/igb/igb_main.c | 44 ++++++ 3 files changed, 269 insertions(+)
Quoted 1 line
This patch is causing 3/4 of my regression systems to fail. Driver load seems normal, but applying an IP address via ifconfig causes the following splat in dmesg and
/var/log/messages:
Hi Aaron,
I'm looking at this series on current net-next and am wondering if you saw this issue with just patch 1 applied or you meant the entire series?
I've been working with this on an i210 and haven't reproduced your results yet either with just the (non-functional) first patch applied or the entire series. However, I noticed you had no problems on your system with an i210.
Quoted 41 lines
----------------------------------------------
May 16 14:37:50 u1486 kernel: Hardware name: Supermicro A1SAi/A1SRi, BIOS 1.0b 11/06/2013 May 16 14:37:50 u1486 kernel: 0000000000000000 ffff880849ad3938 ffffffff813373d7 0000000000000007 May 16 14:37:50 u1486 kernel: 0000000000000006 0000000000000000 ffff88085c2f6770 ffff880849ad3a58 May 16 14:37:50 u1486 kernel: ffffffff810c4e13 ffff880849ad39f8 0000000000000005 0000000000000000 May 16 14:37:50 u1486 kernel: Call Trace: May 16 14:37:50 u1486 kernel: [<ffffffff813373d7>] dump_stack+0x6b/0xa4 May 16 14:37:50 u1486 kernel: [<ffffffff810c4e13>] register_lock_class+0x523/0x5c0 May 16 14:37:50 u1486 kernel: [<ffffffff8136644b>] ? check_preemption_disabled+0x1b/0x110 May 16 14:37:50 u1486 kernel: [<ffffffff811f5655>] ? kfree+0x1a5/0x3a0 May 16 14:37:50 u1486 kernel: [<ffffffff81366553>] ? __this_cpu_preempt_check+0x13/0x20 May 16 14:37:50 u1486 kernel: [<ffffffff810c7ae0>] __lock_acquire+0x80/0x5d0 May 16 14:37:50 u1486 kernel: [<ffffffff811f83f5>] ? __kmalloc+0x265/0x3a0 May 16 14:37:50 u1486 kernel: [<ffffffffa051864f>] ? kzalloc+0xf/0x20 [igb] May 16 14:37:50 u1486 kernel: [<ffffffff810c80fa>] lock_acquire+0xca/0x240 May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>] ? igb_configure+0xaf/0x1d0 [igb] May 16 14:37:50 u1486 kernel: [<ffffffff815b958b>] ? netdev_rss_key_fill+0x5b/0xa0 May 16 14:37:50 u1486 kernel: [<ffffffffa052dfb9>] ? igb_vfta_set+0x189/0x1f0 [igb] May 16 14:37:50 u1486 kernel: [<ffffffff816a8930>] _raw_spin_lock+0x40/0x80 May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>] ? igb_configure+0xaf/0x1d0 [igb] May 16 14:37:50 u1486 kernel: [<ffffffffa051bf62>] ? igb_setup_rctl+0x22/0xb0 [igb] May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>] igb_configure+0xaf/0x1d0 [igb] May 16 14:37:50 u1486 kernel: [<ffffffffa052408d>] __igb_open+0xfd/0x300 [igb] May 16 14:37:50 u1486 kernel: [<ffffffff815ab260>] ? call_netdevice_notifiers_info+0x40/0x70 May 16 14:37:50 u1486 kernel: [<ffffffffa0524420>] igb_open+0x10/0x20 [igb] May 16 14:37:50 u1486 kernel: [<ffffffff815ac7f8>] __dev_open+0xb8/0x110 May 16 14:37:50 u1486 kernel: [<ffffffff815ac5fc>] __dev_change_flags+0xac/0x180 May 16 14:37:50 u1486 kernel: [<ffffffff815ac700>] dev_change_flags+0x30/0x70 May 16 14:37:50 u1486 kernel: [<ffffffff815c6685>] ? lockdep_rtnl_is_held+0x15/0x20 May 16 14:37:50 u1486 kernel: [<ffffffff816403a5>] devinet_ioctl+0x5b5/0x620 May 16 14:37:50 u1486 kernel: [<ffffffff81156660>] ? trace_buffer_unlock_commit+0x60/0x80 May 16 14:37:50 u1486 kernel: [<ffffffff81643033>] inet_ioctl+0x63/0x80 May 16 14:37:50 u1486 kernel: [<ffffffff8158fd60>] sock_do_ioctl+0x30/0x70 May 16 14:37:50 u1486 kernel: [<ffffffff815901b3>] sock_ioctl+0x73/0x280 May 16 14:37:50 u1486 kernel: [<ffffffff8121f678>] vfs_ioctl+0x18/0x30 May 16 14:37:50 u1486 kernel: [<ffffffff81220057>] do_vfs_ioctl+0x87/0x430 May 16 14:37:50 u1486 kernel: [<ffffffff8100297e>] ? syscall_trace_enter_phase2+0x6e/0x280 May 16 14:37:50 u1486 kernel: [<ffffffff81220492>] SyS_ioctl+0x92/0xa0 May 16 14:37:50 u1486 kernel: [<ffffffff81002fd3>] do_syscall_64+0x63/0x130 May 16 14:37:50 u1486 kernel: [<ffffffff8100201b>] ? trace_hardirqs_on_thunk+0x1b/0x1d May 16 14:37:50 u1486 kernel: [<ffffffff816a981a>] entry_SYSCALL64_slow_path+0x25/0x25
Quoted 1 line
----------------------------------------------
Since it's doing dump_stack in register_lock_class it appears some of the error has been truncated before this stack trace. Can you confirm that this is the complete output logged? By inspection, I would expect to see one of the contextual messages from register_lock_class when it calls dump_stack.
Also, any chance of seeing a .config for this run or a freshly reproduced run? By inspection at least there's no obvious locking or otherwise issues in the open path (only *filter_restore() is executed on open and it's a mostly a NOP if this is just patch 1 applied) so I think we need some more detailed output since you have the only system that seems to produce this issue.
Any other details you can provide would be appreciated. I'm happy to dig into the root cause.
Thanks,
Matt
Brown
PATCH v5 1/4 igb: add support of RX network flow classification
Quoted 39 lines
From: Matt Porter [mailto:mporter at konsulko.com]
Sent: Wednesday, June 29, 2016 12:13 PM
To: Brown, Aaron F <[email protected]>
Cc: Gangfeng <[email protected]>; intel-wired-lan at lists.osuosl.org; Ruhao Gao <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classificationQuoted 1 line · Aaron F
Quoted 1 line
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]
On
Quoted 28 lines
Quoted 27 lines
Behalf Of Gangfeng
Sent: Monday, May 9, 2016 2:28 AM
To: intel-wired-lan at lists.osuosl.org
Cc: Gangfeng Huang <[email protected]>; Ruhao Gao <[email protected]>
Subject: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classificationFrom: Gangfeng Huang <[email protected]>
This patch is meant to allow for RX network flow classification to insert and remove Rx filter by ethtool. Ethtool interface has it's own rules manager
Show all filters:
$ ethtool -n eth0 4 RX rings available Total 2 rulesSigned-off-by Ruhao Gao <[email protected]>Signed-off-by Gangfeng Huang <[email protected]>--- drivers/net/ethernet/intel/igb/igb.h | 32 +++++ drivers/net/ethernet/intel/igb/igb_ethtool.c | 193 +++++++++++++++++++++++++++ drivers/net/ethernet/intel/igb/igb_main.c | 44 ++++++ 3 files changed, 269 insertions(+)
Quoted 8 lines
Quoted 1 line
This patch is causing 3/4 of my regression systems to fail. Driver load
seems normal, but applying an IP address via ifconfig causes the following splat in dmesg and
/var/log/messages:Hi Aaron,
I'm looking at this series on current net-next and am wondering if you saw this issue with just patch 1 applied or you meant the entire series?
Hi Matt,
My recollection is that I saw it with just patch 1 applied. And my procedure when I see an issue with a series is to try and isolate it to the individual patch and reply to the one in the series that triggers the issue, so I am pretty sure it was with this patch applied and the rest of the series not applied.
Quoted 5 lines
I've been working with this on an i210 and haven't reproduced your results yet either with just the (non-functional) first patch applied or the entire series. However, I noticed you had no problems on your system with an i210.
Correct, the system with an i210 included was one of the ones not affected by this. I'm not sure if that is due to it not being a problem with the i210 or something more elusive like the system's chipset of a variation in the .config.
Quoted 73 lines
Quoted 3 lines
----------------------------------------------
May 16 14:37:50 u1486 kernel: Hardware name: Supermicro A1SAi/A1SRi,BIOS 1.0b 11/06/2013
Quoted 1 line
May 16 14:37:50 u1486 kernel: 0000000000000000 ffff880849ad3938
ffffffff813373d70000000000000007Quoted 1 line
May 16 14:37:50 u1486 kernel: 0000000000000006 0000000000000000
ffff88085c2f6770ffff880849ad3a58Quoted 1 line
May 16 14:37:50 u1486 kernel: ffffffff810c4e13 ffff880849ad39f80000000000000005 0000000000000000
Quoted 3 lines
May 16 14:37:50 u1486 kernel: Call Trace: May 16 14:37:50 u1486 kernel: [<ffffffff813373d7>] dump_stack+0x6b/0xa4 May 16 14:37:50 u1486 kernel: [<ffffffff810c4e13>]
register_lock_class+0x523/0x5c0Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff8136644b>] ?check_preemption_disabled+0x1b/0x110 > May 16 14:37:50 u1486 kernel: [<ffffffff811f5655>] ? kfree+0x1a5/0x3a0Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff81366553>] ?
__this_cpu_preempt_check+0x13/0x20Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff810c7ae0>]
__lock_acquire+0x80/0x5d0Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff811f83f5>] ?__kmalloc+0x265/0x3a0 > May 16 14:37:50 u1486 kernel: [<ffffffffa051864f>] ? kzalloc+0xf/0x20 [igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff810c80fa>]
lock_acquire+0xca/0x240Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>] ?
igb_configure+0xaf/0x1d0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815b958b>] ?
netdev_rss_key_fill+0x5b/0xa0Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa052dfb9>] ?
igb_vfta_set+0x189/0x1f0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff816a8930>]
_raw_spin_lock+0x40/0x80Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>] ?
igb_configure+0xaf/0x1d0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa051bf62>] ?
igb_setup_rctl+0x22/0xb0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>]
igb_configure+0xaf/0x1d0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa052408d>]
__igb_open+0xfd/0x300[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815ab260>] ?call_netdevice_notifiers_info+0x40/0x70 > May 16 14:37:50 u1486 kernel: [<ffffffffa0524420>] igb_open+0x10/0x20[igb]
Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815ac7f8>]
__dev_open+0xb8/0x110Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815ac5fc>]
__dev_change_flags+0xac/0x180Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815ac700>]
dev_change_flags+0x30/0x70Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815c6685>] ?
lockdep_rtnl_is_held+0x15/0x20Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff816403a5>]
devinet_ioctl+0x5b5/0x620Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff81156660>] ?trace_buffer_unlock_commit+0x60/0x80 > May 16 14:37:50 u1486 kernel: [<ffffffff81643033>] inet_ioctl+0x63/0x80Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff8158fd60>]sock_do_ioctl+0x30/0x70 > May 16 14:37:50 u1486 kernel: [<ffffffff815901b3>] sock_ioctl+0x73/0x280 > May 16 14:37:50 u1486 kernel: [<ffffffff8121f678>] vfs_ioctl+0x18/0x30Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff81220057>]
do_vfs_ioctl+0x87/0x430Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff8100297e>] ?syscall_trace_enter_phase2+0x6e/0x280 > May 16 14:37:50 u1486 kernel: [<ffffffff81220492>] SyS_ioctl+0x92/0xa0Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff81002fd3>]
do_syscall_64+0x63/0x130Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff8100201b>] ?
trace_hardirqs_on_thunk+0x1b/0x1dQuoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff816a981a>]
Quoted 8 lines
entry_SYSCALL64_slow_path+0x25/0x25Quoted 1 line
----------------------------------------------
Since it's doing
dump_stackinregister_lock_classit appears some of the error has been truncated before this stack trace. Can you confirm that this is the complete output logged? By inspection, I would expect to see one of the contextual messages fromregister_lock_classwhen it callsdump_stack.
I will see if I still have a copy of, or can reproduce the trace along with more of the log messages leading up to it.
Quoted 8 lines
Also, any chance of seeing a .config for this run or a freshly reproduced run? By inspection at least there's no obvious locking or otherwise issues in the open path (only
*filter_restore()is executed on open and it's a mostly a NOP if this is just patch 1 applied) so I think we need some more detailed output since you have the only system that seems
to produce this issue.
I can certainly get you a copy of the .config file on the affected systems, however, they will have changed some as the kernel gets updated frequently for tests, along with a make oldconfig pushing occasional changes in. Assuming I can re-apply the patch to the current tree I'll try and reproduce the issue and get you copies of a .config known to be current when the issue strikes.
Quoted 3 lines
Any other details you can provide would be appreciated. I'm happy to dig into the root cause.
Only thing that immediately comes to mind is that my test systems all have multiple ports to minimize the lab space needed to get a sampling of the different parts. I will see if I can reproduce the issue in a system with a single port. If I remember correctly, it was a consistent issue, always appearing relatively quickly on the affected systems.
Quoted 3 lines
Thanks,
Matt
Brown
PATCH v5 1/4 igb: add support of RX network flow classification
Quoted 57 lines
From: Brown, Aaron F
Sent: Wednesday, June 29, 2016 1:06 PM
To: Matt Porter <[email protected]>
Cc: Gangfeng <[email protected]>; intel-wired-lan at lists.osuosl.org; Ruhao Gao <[email protected]>
Subject: RE: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classificationQuoted 6 lines
From: Matt Porter [mailto:mporter at konsulko.com]
Sent: Wednesday, June 29, 2016 12:13 PM
To: Brown, Aaron F <[email protected]>
Cc: Gangfeng <[email protected]>; intel-wired-lan at lists.osuosl.org; Ruhao Gao <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support ofRX
Quoted 33 lines
network flow classification
Quoted 1 line · Aaron F
Quoted 1 line
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]
On
Quoted 28 lines
Quoted 27 lines
Behalf Of Gangfeng
Sent: Monday, May 9, 2016 2:28 AM
To: intel-wired-lan at lists.osuosl.org
Cc: Gangfeng Huang <[email protected]>; Ruhao Gao <[email protected]>
Subject: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classificationFrom: Gangfeng Huang <[email protected]>
This patch is meant to allow for RX network flow classification to insert and remove Rx filter by ethtool. Ethtool interface has it's own rules manager
Show all filters:
$ ethtool -n eth0 4 RX rings available Total 2 rulesSigned-off-by Ruhao Gao <[email protected]>Signed-off-by Gangfeng Huang <[email protected]>--- drivers/net/ethernet/intel/igb/igb.h | 32 +++++ drivers/net/ethernet/intel/igb/igb_ethtool.c | 193 +++++++++++++++++++++++++++ drivers/net/ethernet/intel/igb/igb_main.c | 44 ++++++ 3 files changed, 269 insertions(+)Quoted 8 lines
Quoted 1 line
This patch is causing 3/4 of my regression systems to fail. Driver load
seems normal, but applying an IP address via ifconfig causes the following splat in dmesg and
/var/log/messages:Hi Aaron,
I'm looking at this series on current net-next and am wondering if you saw this issue with just patch 1 applied or you meant the entire series?
Quoted 6 lines
Hi Matt,
My recollection is that I saw it with just patch 1 applied. And my procedure when I see an issue with a series is to try and isolate it to the individual patch and reply to the one in the series that triggers the issue, so I am pretty sure it was with this patch applied and the rest of the series not applied.
I was able to apply this (v5 1/4) patch to a recent version of Jeff's next-queue / dev-branch (with a little fuzz) and reproduce the problem on one of the systems that was previously triggering it without any difficulty. It does occur with just this one patch applied. This was with the system that has an i211 and a pair of 82580 ports. I will still try to sort out if it happens with just i211 or the pair of 82580s (leaning towards the i211 as a system with a pair of 82580s and an i210 worked fine) but have been searching for another i211 as the system in question is at the bottom of a rack with a bunch of systems stacked on top of it, making the card cage rather difficult to get to.
Quoted 90 lines
Quoted 6 lines
I've been working with this on an i210 and haven't reproduced your results yet either with just the (non-functional) first patch applied or the entire series. However, I noticed you had no problems on your system with an i210.
Correct, the system with an i210 included was one of the ones not affected by this. I'm not sure if that is due to it not being a problem with the i210 or something more elusive like the system's chipset of a variation in the .config.
Quoted 12 lines
Quoted 3 lines
----------------------------------------------
May 16 14:37:50 u1486 kernel: Hardware name: Supermicro A1SAi/A1SRi,BIOS 1.0b 11/06/2013
Quoted 1 line
May 16 14:37:50 u1486 kernel: 0000000000000000 ffff880849ad3938
ffffffff813373d70000000000000007Quoted 1 line
May 16 14:37:50 u1486 kernel: 0000000000000006 0000000000000000
ffff88085c2f6770ffff880849ad3a58Quoted 1 line
May 16 14:37:50 u1486 kernel: ffffffff810c4e13 ffff880849ad39f80000000000000005 0000000000000000
Quoted 2 lines
May 16 14:37:50 u1486 kernel: Call Trace: May 16 14:37:50 u1486 kernel: [<ffffffff813373d7>]
dump_stack+0x6b/0xa4Quoted 5 lines
Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff810c4e13>]
register_lock_class+0x523/0x5c0Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff8136644b>] ?check_preemption_disabled+0x1b/0x110 > May 16 14:37:50 u1486 kernel: [<ffffffff811f5655>] ? kfree+0x1a5/0x3a0Quoted 7 lines
Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff81366553>] ?
__this_cpu_preempt_check+0x13/0x20Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff810c7ae0>]
__lock_acquire+0x80/0x5d0Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff811f83f5>] ?__kmalloc+0x265/0x3a0 > May 16 14:37:50 u1486 kernel: [<ffffffffa051864f>] ? kzalloc+0xf/0x20[igb]
Quoted 29 lines
Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff810c80fa>]
lock_acquire+0xca/0x240Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>] ?
igb_configure+0xaf/0x1d0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815b958b>] ?
netdev_rss_key_fill+0x5b/0xa0Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa052dfb9>] ?
igb_vfta_set+0x189/0x1f0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff816a8930>]
_raw_spin_lock+0x40/0x80Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>] ?
igb_configure+0xaf/0x1d0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa051bf62>] ?
igb_setup_rctl+0x22/0xb0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>]
igb_configure+0xaf/0x1d0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa052408d>]
__igb_open+0xfd/0x300[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815ab260>] ?call_netdevice_notifiers_info+0x40/0x70 > May 16 14:37:50 u1486 kernel: [<ffffffffa0524420>] igb_open+0x10/0x20[igb]
Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815ac7f8>]
__dev_open+0xb8/0x110Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815ac5fc>]
__dev_change_flags+0xac/0x180Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815ac700>]
dev_change_flags+0x30/0x70Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815c6685>] ?Quoted 9 lines
lockdep_rtnl_is_held+0x15/0x20Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff816403a5>]
devinet_ioctl+0x5b5/0x620Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff81156660>] ?trace_buffer_unlock_commit+0x60/0x80 > May 16 14:37:50 u1486 kernel: [<ffffffff81643033>] inet_ioctl+0x63/0x80Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff8158fd60>]
sock_do_ioctl+0x30/0x70Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815901b3>]sock_ioctl+0x73/0x280 > > May 16 14:37:50 u1486 kernel: [<ffffffff8121f678>] vfs_ioctl+0x18/0x30Quoted 13 lines
Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff81220057>]
do_vfs_ioctl+0x87/0x430Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff8100297e>] ?syscall_trace_enter_phase2+0x6e/0x280 > May 16 14:37:50 u1486 kernel: [<ffffffff81220492>] SyS_ioctl+0x92/0xa0Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff81002fd3>]
do_syscall_64+0x63/0x130Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff8100201b>] ?
trace_hardirqs_on_thunk+0x1b/0x1dQuoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff816a981a>]
entry_SYSCALL64_slow_path+0x25/0x25Quoted 1 line
----------------------------------------------
Quoted 8 lines
Quoted 5 lines
Since it's doing
dump_stackinregister_lock_classit appears some of the error has been truncated before this stack trace. Can you confirm that this is the complete output logged? By inspection, I would expect to see one of the contextual messages fromregister_lock_classwhen it callsdump_stack.I will see if I still have a copy of, or can reproduce the trace along with more of the log messages leading up to it.
Yes, a tiny bit of more info before the trace. I cleared the dmesg queue before running ifconfig and here is what I got, I also checked /var/log/messages and did not see any extra information there:
------------------------------------------------------------------------------------------------------------------------pps_core: LinuxPPS API ver. 1 registeredpps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]>
PTP clock support registered
dca service started, version 1.12.1
igb: Intel(R) Gigabit Ethernet Network Driver - version 5.3.0-k
igb: Copyright (c) 2007-2014 Intel Corporation.
pps pps0: new PPS source ptp0
igb 0000:08:00.0: added PHC on eth1
igb 0000:08:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:08:00.0: eth1: (PCIe:2.5Gb/s:Width x1) a0:36:9f:0c:76:de
igb 0000:08:00.0: eth1: PBA No: FFFFFF-0FF
igb 0000:08:00.0: Using MSI-X interrupts. 2 rx queue(s), 2 tx queue(s)
igb 0000:09:00.0: added PHC on eth2
igb 0000:09:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:09:00.0: eth2: (PCIe:2.5Gb/s:Width x4) 00:1b:21:56:25:64
igb 0000:09:00.0: eth2: PBA No: Unknown
igb 0000:09:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
igb 0000:09:00.1: added PHC on eth3
igb 0000:09:00.1: Intel(R) Gigabit Ethernet Network Connection
igb 0000:09:00.1: eth3: (PCIe:2.5Gb/s:Width x4) 00:1b:21:56:25:65
igb 0000:09:00.1: eth3: PBA No: Unknown
igb 0000:09:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
BUG: spinlock bad magic on CPU#6, ifconfig/5569 lock: 0xffff88007cbca080, .magic: 00000000, .owner: <none>/-1, .owner_cpu: 0 CPU: 6 PID: 5569 Comm: ifconfig Not tainted 4.7.0-rc2_next_dev_rx_flow-00941-gb9e6e3f-dirty #2 Hardware name: Supermicro X7DBX/X7DBX, BIOS 2.1 06/23/2008 0000000000000000 ffff880074393b08 ffffffff811e2937 0000000000000000 ffffffff81717dbc ffff88007cbca080 0000000000000000 ffff880074393b28 ffffffff81070724 ffff88007cbca080 ffff88007cbc8db0 ffff880074393b48 Call Trace: [<ffffffff811e2937>] dump_stack+0x53/0x74 [<ffffffff81070724>] spin_dump+0x86/0x8b [<ffffffff8107074f>] spin_bug+0x26/0x28 [<ffffffff810708d5>] do_raw_spin_lock+0x29/0x12c [<ffffffff811f55ad>] ? __do_once_done+0x71/0x78 [<ffffffff814b488b>] _raw_spin_lock+0x1e/0x22 [<ffffffffa00bd0d7>] igb_configure+0x2c8/0x3b1 [igb] [<ffffffffa00c0295>] __igb_open+0xb3/0x509 [igb] [<ffffffff81404799>] ? call_netdevice_notifiers_info+0x51/0x5a [<ffffffffa00c083c>] igb_open+0xb/0xd [igb] [<ffffffff81406542>] __dev_open+0xa7/0xf5 [<ffffffff814063aa>] __dev_change_flags+0xb5/0x14d [<ffffffff81406465>] dev_change_flags+0x23/0x59 [<ffffffff814b2b4f>] ? mutex_lock+0x27/0x38 [<ffffffff8145cc49>] devinet_ioctl+0x296/0x576 [<ffffffff8145ecba>] inet_ioctl+0x92/0xaa [<ffffffff813f1566>] sock_ioctl+0x204/0x229 [<ffffffff811092cf>] vfs_ioctl+0x13/0x23 [<ffffffff811099e2>] do_vfs_ioctl+0x5e8/0x621 [<ffffffff811fe00c>] ? __percpu_counter_add+0x8c/0xa8 [<ffffffff810dfca7>] ? do_munmap+0x2e7/0x301 [<ffffffff81109a5e>] SyS_ioctl+0x43/0x62 [<ffffffff814b499b>] entry_SYSCALL_64_fastpath+0x13/0x8f
IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
igb 0000:08:00.0 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
------------------------------------------------------------------------------------------------------------------------
Quoted 4 lines
Quoted 4 lines
Also, any chance of seeing a .config for this run or a freshly reproduced run?
Sure, I've attached the current .config from the system that I just reproduced it on. I know we should not use attachments on the lists, but a .config file is rather long for pasting into a message. If it gets stripped let me know and I'll figure out how to get it to you.
Quoted 15 lines
Quoted 6 lines
By inspection at least there's no obvious locking or otherwise issues in the open path (only
*filter_restore()is executed on open and it's a mostly a NOP if this is just patch 1 applied) so I think we need some more detailed output since you have the only system that seems
to produce this issue.I can certainly get you a copy of the .config file on the affected systems, however, they will have changed some as the kernel gets updated frequently for tests, along with a make oldconfig pushing occasional changes in. Assuming I can re-apply the patch to the current tree I'll try and reproduce the issue and get you copies of a .config known to be current when the issue strikes.
Quoted 1 line
Quoted 12 lines
Quoted 2 lines
Any other details you can provide would be appreciated. I'm happy to dig into the root cause.
Only thing that immediately comes to mind is that my test systems all have multiple ports to minimize the lab space needed to get a sampling of the different parts. I will see if I can reproduce the issue in a system with a single port. If I remember correctly, it was a consistent issue, always appearing relatively quickly on the affected systems.
Quoted 3 lines
Thanks,
Matt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ifconfig-up-bug-dotconfig
Type: application/octet-stream
Size: 78378 bytes
Desc: ifconfig-up-bug-dotconfig
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20160630/eccc7c8c/attachment-0001.obj>
PATCH v5 1/4 igb: add support of RX network flow classification
Quoted 57 lines · Aaron F
Quoted 57 lines
From: Brown, Aaron F
Sent: Wednesday, June 29, 2016 1:06 PM
To: Matt Porter <[email protected]>
Cc: Gangfeng <[email protected]>; intel-wired-lan at lists.osuosl.org; Ruhao Gao <[email protected]>
Subject: RE: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classificationQuoted 6 lines
From: Matt Porter [mailto:mporter at konsulko.com]
Sent: Wednesday, June 29, 2016 12:13 PM
To: Brown, Aaron F <[email protected]>
Cc: Gangfeng <[email protected]>; intel-wired-lan at lists.osuosl.org; Ruhao Gao <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support ofRX
Quoted 41 lines
network flow classification
Quoted 1 line · Aaron F
Quoted 1 line
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]
On
Quoted 28 lines
Quoted 19 lines
Behalf Of Gangfeng
Sent: Monday, May 9, 2016 2:28 AM
To: intel-wired-lan at lists.osuosl.org
Cc: Gangfeng Huang <[email protected]>; Ruhao Gao <[email protected]>
Subject: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classificationFrom: Gangfeng Huang <[email protected]>
This patch is meant to allow for RX network flow classification to insert and remove Rx filter by ethtool. Ethtool interface has it's own rules manager
Show all filters:
$ ethtool -n eth0 4 RX rings available Total 2 rulesQuoted 8 lines
Signed-off-by Ruhao Gao <[email protected]>Signed-off-by Gangfeng Huang <[email protected]>--- drivers/net/ethernet/intel/igb/igb.h | 32 +++++ drivers/net/ethernet/intel/igb/igb_ethtool.c | 193 +++++++++++++++++++++++++++ drivers/net/ethernet/intel/igb/igb_main.c | 44 ++++++ 3 files changed, 269 insertions(+)Quoted 1 line
This patch is causing 3/4 of my regression systems to fail. Driver load
seems normal, but applying an IP address via ifconfig causes the following splat in dmesg and
/var/log/messages:Hi Aaron,
I'm looking at this series on current net-next and am wondering if you saw this issue with just patch 1 applied or you meant the entire series?
Quoted 8 lines
Quoted 6 lines
Hi Matt,
My recollection is that I saw it with just patch 1 applied. And my procedure when I see an issue with a series is to try and isolate it to the individual patch and reply to the one in the series that triggers the issue, so I am pretty sure it was with this patch applied and the rest of the series not applied.
I was able to apply this (v5 1/4) patch to a recent version of Jeff's next-queue / dev-branch (with a little fuzz) and reproduce the problem on one of the systems that was previously triggering it without any difficulty. It does occur with just this one patch applied. This was with the system that has an i211 and a pair of 82580 ports. I will still try to sort out if it happens with just i211 or the pair of 82580s (leaning towards the i211 as a system with a pair of 82580s and an i210 worked fine) but have been searching for another i211 as the system in question is at the bottom of a rack with a bunch of systems stacked on top of it, making the card cage rather difficult to get to.
Ok, thanks for trying this again. I'm switching to that branch and trying to get a multi-port system together to try to reproduce here though it sounds like I may need to pick up more than the i210's I have handy.
-Matt
Quoted 94 lines
Quoted 94 lines
Quoted 4 lines
I've been working with this on an i210 and haven't reproduced your results yet either with just the (non-functional) first patch applied or the entire series. However, I noticed you had no problems on your system with an i210.
Correct, the system with an i210 included was one of the ones not affected by this. I'm not sure if that is due to it not being a problem with the i210 or something more elusive like the system's chipset of a variation in the .config.
Quoted 12 lines
Quoted 3 lines
----------------------------------------------
May 16 14:37:50 u1486 kernel: Hardware name: Supermicro A1SAi/A1SRi,BIOS 1.0b 11/06/2013
Quoted 1 line
May 16 14:37:50 u1486 kernel: 0000000000000000 ffff880849ad3938
ffffffff813373d70000000000000007Quoted 1 line
May 16 14:37:50 u1486 kernel: 0000000000000006 0000000000000000
ffff88085c2f6770ffff880849ad3a58Quoted 1 line
May 16 14:37:50 u1486 kernel: ffffffff810c4e13 ffff880849ad39f80000000000000005 0000000000000000
Quoted 2 lines
May 16 14:37:50 u1486 kernel: Call Trace: May 16 14:37:50 u1486 kernel: [<ffffffff813373d7>]
dump_stack+0x6b/0xa4Quoted 5 lines
Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff810c4e13>]
register_lock_class+0x523/0x5c0Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff8136644b>] ?check_preemption_disabled+0x1b/0x110 > May 16 14:37:50 u1486 kernel: [<ffffffff811f5655>] ? kfree+0x1a5/0x3a0Quoted 7 lines
Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff81366553>] ?
__this_cpu_preempt_check+0x13/0x20Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff810c7ae0>]
__lock_acquire+0x80/0x5d0Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff811f83f5>] ?__kmalloc+0x265/0x3a0 > May 16 14:37:50 u1486 kernel: [<ffffffffa051864f>] ? kzalloc+0xf/0x20[igb]
Quoted 29 lines
Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff810c80fa>]
lock_acquire+0xca/0x240Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>] ?
igb_configure+0xaf/0x1d0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815b958b>] ?
netdev_rss_key_fill+0x5b/0xa0Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa052dfb9>] ?
igb_vfta_set+0x189/0x1f0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff816a8930>]
_raw_spin_lock+0x40/0x80Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>] ?
igb_configure+0xaf/0x1d0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa051bf62>] ?
igb_setup_rctl+0x22/0xb0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa0520c3f>]
igb_configure+0xaf/0x1d0[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffffa052408d>]
__igb_open+0xfd/0x300[igb]Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815ab260>] ?call_netdevice_notifiers_info+0x40/0x70 > May 16 14:37:50 u1486 kernel: [<ffffffffa0524420>] igb_open+0x10/0x20[igb]
Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815ac7f8>]
__dev_open+0xb8/0x110Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815ac5fc>]
__dev_change_flags+0xac/0x180Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815ac700>]
dev_change_flags+0x30/0x70Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815c6685>] ?Quoted 9 lines
lockdep_rtnl_is_held+0x15/0x20Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff816403a5>]
devinet_ioctl+0x5b5/0x620Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff81156660>] ?trace_buffer_unlock_commit+0x60/0x80 > May 16 14:37:50 u1486 kernel: [<ffffffff81643033>] inet_ioctl+0x63/0x80Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff8158fd60>]
sock_do_ioctl+0x30/0x70Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff815901b3>]sock_ioctl+0x73/0x280 > > May 16 14:37:50 u1486 kernel: [<ffffffff8121f678>] vfs_ioctl+0x18/0x30Quoted 18 lines
Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff81220057>]
do_vfs_ioctl+0x87/0x430Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff8100297e>] ?syscall_trace_enter_phase2+0x6e/0x280 > May 16 14:37:50 u1486 kernel: [<ffffffff81220492>] SyS_ioctl+0x92/0xa0Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff81002fd3>]
do_syscall_64+0x63/0x130Quoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff8100201b>] ?
trace_hardirqs_on_thunk+0x1b/0x1dQuoted 1 line
May 16 14:37:50 u1486 kernel: [<ffffffff816a981a>]
entry_SYSCALL64_slow_path+0x25/0x25Quoted 1 line
----------------------------------------------
Since it's doing
dump_stackinregister_lock_classit appears some of the error has been truncated before this stack trace. Can you confirm that this is the complete output logged? By inspection, I would expect to see one of the contextual messages fromregister_lock_classwhen it callsdump_stack.
Quoted 5 lines
Quoted 2 lines
I will see if I still have a copy of, or can reproduce the trace along with more of the log messages leading up to it.
Yes, a tiny bit of more info before the trace. I cleared the dmesg queue before running ifconfig and here is what I got, I also checked
/var/log/messagesand did not see any extra information there:
------------------------------------------------------------------------------------------------------------------------
Aha, yes, that helps isolate it a bit further.
Quoted 59 lines
pps_core: LinuxPPS API ver. 1 registeredpps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]> PTP clock support registered
dca service started, version 1.12.1
igb: Intel(R) Gigabit Ethernet Network Driver - version 5.3.0-k
igb: Copyright (c) 2007-2014 Intel Corporation.
pps pps0: new PPS source ptp0
igb 0000:08:00.0: added PHC on eth1
igb 0000:08:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:08:00.0: eth1: (PCIe:2.5Gb/s:Width x1) a0:36:9f:0c:76:de
igb 0000:08:00.0: eth1: PBA No: FFFFFF-0FF
igb 0000:08:00.0: Using MSI-X interrupts. 2 rx queue(s), 2 tx queue(s)
igb 0000:09:00.0: added PHC on eth2
igb 0000:09:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:09:00.0: eth2: (PCIe:2.5Gb/s:Width x4) 00:1b:21:56:25:64
igb 0000:09:00.0: eth2: PBA No: Unknown
igb 0000:09:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
igb 0000:09:00.1: added PHC on eth3
igb 0000:09:00.1: Intel(R) Gigabit Ethernet Network Connection
igb 0000:09:00.1: eth3: (PCIe:2.5Gb/s:Width x4) 00:1b:21:56:25:65
igb 0000:09:00.1: eth3: PBA No: Unknown
igb 0000:09:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)BUG: spinlock bad magic on CPU#6, ifconfig/5569 lock: 0xffff88007cbca080, .magic: 00000000, .owner: <none>/-1, .owner_cpu: 0 CPU: 6 PID: 5569 Comm: ifconfig Not tainted 4.7.0-rc2_next_dev_rx_flow-00941-gb9e6e3f-dirty #2 Hardware name: Supermicro X7DBX/X7DBX, BIOS 2.1 06/23/2008 0000000000000000 ffff880074393b08 ffffffff811e2937 0000000000000000 ffffffff81717dbc ffff88007cbca080 0000000000000000 ffff880074393b28 ffffffff81070724 ffff88007cbca080 ffff88007cbc8db0 ffff880074393b48 Call Trace: [<ffffffff811e2937>] dump_stack+0x53/0x74 [<ffffffff81070724>] spin_dump+0x86/0x8b [<ffffffff8107074f>] spin_bug+0x26/0x28 [<ffffffff810708d5>] do_raw_spin_lock+0x29/0x12c [<ffffffff811f55ad>] ? __do_once_done+0x71/0x78 [<ffffffff814b488b>] _raw_spin_lock+0x1e/0x22 [<ffffffffa00bd0d7>] igb_configure+0x2c8/0x3b1 [igb] [<ffffffffa00c0295>] __igb_open+0xb3/0x509 [igb] [<ffffffff81404799>] ? call_netdevice_notifiers_info+0x51/0x5a [<ffffffffa00c083c>] igb_open+0xb/0xd [igb] [<ffffffff81406542>] __dev_open+0xa7/0xf5 [<ffffffff814063aa>] __dev_change_flags+0xb5/0x14d [<ffffffff81406465>] dev_change_flags+0x23/0x59 [<ffffffff814b2b4f>] ? mutex_lock+0x27/0x38 [<ffffffff8145cc49>] devinet_ioctl+0x296/0x576 [<ffffffff8145ecba>] inet_ioctl+0x92/0xaa [<ffffffff813f1566>] sock_ioctl+0x204/0x229 [<ffffffff811092cf>] vfs_ioctl+0x13/0x23 [<ffffffff811099e2>] do_vfs_ioctl+0x5e8/0x621 [<ffffffff811fe00c>] ? __percpu_counter_add+0x8c/0xa8 [<ffffffff810dfca7>] ? do_munmap+0x2e7/0x301 [<ffffffff81109a5e>] SyS_ioctl+0x43/0x62 [<ffffffff814b499b>] entry_SYSCALL_64_fastpath+0x13/0x8fIPv6:
ADDRCONF(NETDEV_UP): eth1: link is not ready
igb 0000:08:00.0 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
IPv6:ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
------------------------------------------------------------------------------------------------------------------------Quoted 2 lines
Quoted 2 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Also, any chance of seeing a .config for this run or a freshly reproduced run?
Sure, I've attached the current .config from the system that I just reproduced it on. I know we should not use attachments on the lists, but a .config file is rather long for pasting into a message. If it gets stripped let me know and I'll figure out how to get it to you.
Got it. Thanks for everything so far, this gives me quite a bit more to go on for debugging.
-Matt
Quoted 28 lines
Quoted 28 lines
Quoted 7 lines
By inspection at least there's no obvious locking or otherwise issues in the open path (only
*filter_restore()is executed on open and it's a mostly a NOP if this is just patch 1 applied) so I think we need some more detailed output since you have the only system that seems
to produce this issue.I can certainly get you a copy of the .config file on the affected systems, however, they will have changed some as the kernel gets updated frequently for tests, along with a make oldconfig pushing occasional changes in. Assuming I can re-apply the patch to the current tree I'll try and reproduce the issue and get you copies of a .config known to be current when the issue strikes.
Quoted 3 lines
Any other details you can provide would be appreciated. I'm happy to dig into the root cause.
Only thing that immediately comes to mind is that my test systems all have multiple ports to minimize the lab space needed to get a sampling of the different parts. I will see if I can reproduce the issue in a system with a single port. If I remember correctly, it was a consistent issue, always appearing relatively quickly on the affected systems.
Quoted 3 lines
Thanks,
Matt
PATCH v5 1/4 igb: add support of RX network flow classification
In reply to BrownQuoted 57 lines · Aaron F
Quoted 57 lines
From: Brown, Aaron F
Sent: Wednesday, June 29, 2016 1:06 PM
To: Matt Porter <[email protected]>
Cc: Gangfeng <[email protected]>; intel-wired-lan at lists.osuosl.org; Ruhao Gao <[email protected]>
Subject: RE: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classificationQuoted 6 lines
From: Matt Porter [mailto:mporter at konsulko.com]
Sent: Wednesday, June 29, 2016 12:13 PM
To: Brown, Aaron F <[email protected]>
Cc: Gangfeng <[email protected]>; intel-wired-lan at lists.osuosl.org; Ruhao Gao <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support ofRX
Quoted 41 lines
network flow classification
Quoted 1 line · Aaron F
Quoted 1 line
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]
On
Quoted 28 lines
Quoted 19 lines
Behalf Of Gangfeng
Sent: Monday, May 9, 2016 2:28 AM
To: intel-wired-lan at lists.osuosl.org
Cc: Gangfeng Huang <[email protected]>; Ruhao Gao <[email protected]>
Subject: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classificationFrom: Gangfeng Huang <[email protected]>
This patch is meant to allow for RX network flow classification to insert and remove Rx filter by ethtool. Ethtool interface has it's own rules manager
Show all filters:
$ ethtool -n eth0 4 RX rings available Total 2 rulesQuoted 8 lines
Signed-off-by Ruhao Gao <[email protected]>Signed-off-by Gangfeng Huang <[email protected]>--- drivers/net/ethernet/intel/igb/igb.h | 32 +++++ drivers/net/ethernet/intel/igb/igb_ethtool.c | 193 +++++++++++++++++++++++++++ drivers/net/ethernet/intel/igb/igb_main.c | 44 ++++++ 3 files changed, 269 insertions(+)Quoted 1 line
This patch is causing 3/4 of my regression systems to fail. Driver load
seems normal, but applying an IP address via ifconfig causes the following splat in dmesg and
/var/log/messages:Hi Aaron,
I'm looking at this series on current net-next and am wondering if you saw this issue with just patch 1 applied or you meant the entire series?
Quoted 8 lines
Quoted 6 lines
Hi Matt,
My recollection is that I saw it with just patch 1 applied. And my procedure when I see an issue with a series is to try and isolate it to the individual patch and reply to the one in the series that triggers the issue, so I am pretty sure it was with this patch applied and the rest of the series not applied.
I was able to apply this (v5 1/4) patch to a recent version of Jeff's next-queue / dev-branch (with a little fuzz) and reproduce the problem on one of the systems that was previously triggering it without any difficulty. It does occur with just this one patch applied. This was with the system that has an i211 and a pair of 82580 ports. I will still try to sort out if it happens with just i211 or the pair of 82580s (leaning towards the i211 as a system with a pair of 82580s and an i210 worked fine) but have been searching for another i211 as the system in question is at the bottom of a rack with a bunch of systems stacked on top of it, making the card cage rather difficult to get to.
Ok, the full oops output made me notice the problem. Can you try the following untested fix on your failing system? It'll apply against patch 1/4 and is build tested against the next-queue dev-queue branch.
-Matt
From 8773d0f09741d7a318cdd96cf5aad2ddb79096e7 Mon Sep 17 00:00:00 2001
From: Matt Porter <[email protected]>
Date: Thu, 30 Jun 2016 11:47:50 -0400
Subject: [PATCH] igb: Fix missing lock init in rx network flow classification
support
"[net-next,v5,1/4] igb: add support of RX network flow classification" adds a new nfc_lock to each adapter to protect the rx filter hash list. This lock is not initialized before use which results in intermittent oopses. Initialize this lock during probe to fix the issue.
--- drivers/net/ethernet/intel/igb/igb_main.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/ethernet/intel/igb/igb_main.c +1 -0
@@ -3075,6 +3075,7 @@ static int igb_sw_init(struct igb_adapter *adapter)VLAN_HLEN ; adapter->min_frame_size =ETH_ZLEN +ETH_FCS_LEN ; +spin_lock_init (&adapter->nfc_lock);spin_lock_init (&adapter->stats64_lock);#ifdef CONFIG_PCI_IOV switch (hw->mac.type) {
Signature · 1 line
2.7.0
Brown
PATCH v5 1/4 igb: add support of RX network flow classification
Quoted 11 lines
-----Original Message-----
From: 'Matt Porter' [mailto:mporter at konsulko.com]
Sent: Thursday, June 30, 2016 9:16 AM
To: Brown, Aaron F <[email protected]>
Cc: 'Gangfeng' <[email protected]>; 'intel-wired-lan at lists.osuosl.org' <[email protected]>; 'Ruhao Gao' <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classification
<snip>
Quoted 38 lines
Ok, the full oops output made me notice the problem. Can you try the following untested fix on your failing system? It'll apply against patch 1/4 and is build tested against the next-queue dev-queue branch.
-Matt
From
8773d0f09741d7a318cdd96cf5aad2ddb79096e7Mon Sep 17 00:00:00 2001
From: Matt Porter <[email protected]>
Date: Thu, 30 Jun 2016 11:47:50 -0400
Subject: [PATCH] igb: Fix missing lock init in rx network flow classification
support"[net-next,v5,1/4] igb: add support of RX network flow classification" adds a new
nfc_lockto each adapter to protect the rx filter hash list. This lock is not initialized before use which results in intermittent oopses. Initialize this lock during probe to fix the issue.Signed-off-by Matt Porter <[email protected]>--- drivers/net/ethernet/intel/igb/igb_main.c | 1 + 1 file changed, 1 insertion(+)drivers/net/ethernet/intel/igb/igb_main.c +1
-1@@ -3075,6 +3075,7 @@ static int igb_sw_init(struct igb_adapter *adapter)VLAN_HLEN ; adapter->min_frame_size =ETH_ZLEN +ETH_FCS_LEN ; +spin_lock_init (&adapter->nfc_lock);spin_lock_init (&adapter->stats64_lock);#ifdef CONFIG_PCI_IOV switch (hw->mac.type) { --
Quoted 1 line
2.7.0
Bingo! That resolved it. I can now load and bind (with ifconfig) on all the systems that were previously giving me the splat without seeing anything out of the ordinary.
Thanks,
Aaron
PATCH v5 1/4 igb: add support of RX network flow classification
Quoted 45 lines · Aaron F
Quoted 11 lines
-----Original Message-----
From: 'Matt Porter' [mailto:mporter at konsulko.com]
Sent: Thursday, June 30, 2016 9:16 AM
To: Brown, Aaron F <[email protected]>
Cc: 'Gangfeng' <[email protected]>; 'intel-wired-lan at lists.osuosl.org' <[email protected]>; 'Ruhao Gao' <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classification<snip>
Quoted 33 lines
Ok, the full oops output made me notice the problem. Can you try the following untested fix on your failing system? It'll apply against patch 1/4 and is build tested against the next-queue dev-queue branch.
-Matt
From
8773d0f09741d7a318cdd96cf5aad2ddb79096e7Mon Sep 17 00:00:00 2001
From: Matt Porter <[email protected]>
Date: Thu, 30 Jun 2016 11:47:50 -0400
Subject: [PATCH] igb: Fix missing lock init in rx network flow classification
support"[net-next,v5,1/4] igb: add support of RX network flow classification" adds a new
nfc_lockto each adapter to protect the rx filter hash list. This lock is not initialized before use which results in intermittent oopses. Initialize this lock during probe to fix the issue.Signed-off-by Matt Porter <[email protected]>--- drivers/net/ethernet/intel/igb/igb_main.c | 1 + 1 file changed, 1 insertion(+)drivers/net/ethernet/intel/igb/igb_main.c +0
-0@@ -3075,6 +3075,7 @@ static int igb_sw_init(struct igb_adapter *adapter)VLAN_HLEN ; adapter->min_frame_size =ETH_ZLEN +ETH_FCS_LEN ;
Quoted 8 lines
Quoted 6 lines
drivers/net/ethernet/intel/igb/igb_main.c +1
-1+spin_lock_init (&adapter->nfc_lock);spin_lock_init (&adapter->stats64_lock);#ifdef CONFIG_PCI_IOV switch (hw->mac.type) { --2.7.0
Bingo! That resolved it. I can now load and bind (with ifconfig) on all the systems that were previously giving me the splat without seeing anything out of the ordinary.
Excellent! Thanks for doing the grunt testing work on it. ;) Looks like this was the only remaining issue from the v5 submission.
Gangfeng: can you fold this fix into a v6 submission? If this isn't something you are still working on then let me know and I can resubmit.
Regards,
Matt
PATCH v5 1/4 igb: add support of RX network flow classification
Hi Matt,
Thank you very much for the fix and I will be very appreciate if you can help to resubmit it. Because I caught pneumonia and do not have time to submit it in this week.
If you are too busy, I can submit the v6 in next week.
Gangfeng Huang
-----Original Message-----
From: 'Matt Porter' [mailto:mporter at konsulko.com]
Sent: 2016?7?1? 4:11
To: Brown, Aaron F <[email protected]>
Cc: Gangfeng Huang <[email protected]>; 'intel-wired-lan at lists.osuosl.org' <[email protected]>; Ruhao Gao <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classification
Quoted 45 lines · Aaron F
Quoted 11 lines
-----Original Message-----
From: 'Matt Porter' [mailto:mporter at konsulko.com]
Sent: Thursday, June 30, 2016 9:16 AM
To: Brown, Aaron F <[email protected]>
Cc: 'Gangfeng' <[email protected]>; 'intel-wired-lan at lists.osuosl.org' <[email protected]>; 'Ruhao Gao' <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classification<snip>
Quoted 33 lines
Ok, the full oops output made me notice the problem. Can you try the following untested fix on your failing system? It'll apply against patch 1/4 and is build tested against the next-queue dev-queue branch.
-Matt
From
8773d0f09741d7a318cdd96cf5aad2ddb79096e7Mon Sep 17 00:00:00 2001
From: Matt Porter <[email protected]>
Date: Thu, 30 Jun 2016 11:47:50 -0400
Subject: [PATCH] igb: Fix missing lock init in rx network flow classification support"[net-next,v5,1/4] igb: add support of RX network flow classification" adds a new
nfc_lockto each adapter to protect the rx filter hash list. This lock is not initialized before use which results in intermittent oopses. Initialize this lock during probe to fix the issue.Signed-off-by Matt Porter <[email protected]>--- drivers/net/ethernet/intel/igb/igb_main.c | 1 + 1 file changed, 1 insertion(+)drivers/net/ethernet/intel/igb/igb_main.c +0
-0@@ -3075,6 +3075,7 @@ static int igb_sw_init(struct igb_adapter *adapter)VLAN_HLEN ; adapter->min_frame_size =ETH_ZLEN +ETH_FCS_LEN ;
Quoted 8 lines
Quoted 6 lines
drivers/net/ethernet/intel/igb/igb_main.c +1
-1+spin_lock_init (&adapter->nfc_lock);spin_lock_init (&adapter->stats64_lock);#ifdef CONFIG_PCI_IOV switch (hw->mac.type) { --2.7.0
Bingo! That resolved it. I can now load and bind (with ifconfig) on all the systems that were previously giving me the splat without seeing anything out of the ordinary.
Excellent! Thanks for doing the grunt testing work on it. ;) Looks like this was the only remaining issue from the v5 submission.
Gangfeng: can you fold this fix into a v6 submission? If this isn't something you are still working on then let me know and I can resubmit.
Regards,
Matt
PATCH v5 1/4 igb: add support of RX network flow classification
In reply to 'Matt Porter'Hi matt,
I have add your fix to the v6 patches and resubmit them, please help to test it.
Thanks
Gangfeng Huang
-----Original Message-----
From: 'Matt Porter' [mailto:mporter at konsulko.com]
Sent: 2016?7?1? 4:11
To: Brown, Aaron F <[email protected]>
Cc: Gangfeng Huang <[email protected]>; 'intel-wired-lan at lists.osuosl.org' <[email protected]>; Ruhao Gao <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classification
Quoted 45 lines · Aaron F
Quoted 11 lines
-----Original Message-----
From: 'Matt Porter' [mailto:mporter at konsulko.com]
Sent: Thursday, June 30, 2016 9:16 AM
To: Brown, Aaron F <[email protected]>
Cc: 'Gangfeng' <[email protected]>; 'intel-wired-lan at lists.osuosl.org' <[email protected]>; 'Ruhao Gao' <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classification<snip>
Quoted 33 lines
Ok, the full oops output made me notice the problem. Can you try the following untested fix on your failing system? It'll apply against patch 1/4 and is build tested against the next-queue dev-queue branch.
-Matt
From
8773d0f09741d7a318cdd96cf5aad2ddb79096e7Mon Sep 17 00:00:00 2001
From: Matt Porter <[email protected]>
Date: Thu, 30 Jun 2016 11:47:50 -0400
Subject: [PATCH] igb: Fix missing lock init in rx network flow classification support"[net-next,v5,1/4] igb: add support of RX network flow classification" adds a new
nfc_lockto each adapter to protect the rx filter hash list. This lock is not initialized before use which results in intermittent oopses. Initialize this lock during probe to fix the issue.Signed-off-by Matt Porter <[email protected]>--- drivers/net/ethernet/intel/igb/igb_main.c | 1 + 1 file changed, 1 insertion(+)drivers/net/ethernet/intel/igb/igb_main.c +0
-0@@ -3075,6 +3075,7 @@ static int igb_sw_init(struct igb_adapter *adapter)VLAN_HLEN ; adapter->min_frame_size =ETH_ZLEN +ETH_FCS_LEN ;
Quoted 8 lines
Quoted 6 lines
drivers/net/ethernet/intel/igb/igb_main.c +1
-1+spin_lock_init (&adapter->nfc_lock);spin_lock_init (&adapter->stats64_lock);#ifdef CONFIG_PCI_IOV switch (hw->mac.type) { --2.7.0
Bingo! That resolved it. I can now load and bind (with ifconfig) on all the systems that were previously giving me the splat without seeing anything out of the ordinary.
Excellent! Thanks for doing the grunt testing work on it. ;) Looks like this was the only remaining issue from the v5 submission.
Gangfeng: can you fold this fix into a v6 submission? If this isn't something you are still working on then let me know and I can resubmit.
Regards,
Matt
PATCH v5 1/4 igb: add support of RX network flow classification
Quoted 3 lines · Gangfeng Huang
Hi matt,
I have add your fix to the v6 patches and resubmit them, please help to test it.
Ok, thanks. I'll retest with v6 and reply back to that thread. I hope you are feeling better.
-Matt
Quoted 72 lines
-----Original Message-----
From: 'Matt Porter' [mailto:mporter at konsulko.com]
Sent: 2016?7?1? 4:11
To: Brown, Aaron F <[email protected]>
Cc: Gangfeng Huang <[email protected]>; 'intel-wired-lan at lists.osuosl.org' <[email protected]>; Ruhao Gao <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classificationQuoted 45 lines · Aaron F
Quoted 11 lines
-----Original Message-----
From: 'Matt Porter' [mailto:mporter at konsulko.com]
Sent: Thursday, June 30, 2016 9:16 AM
To: Brown, Aaron F <[email protected]>
Cc: 'Gangfeng' <[email protected]>; 'intel-wired-lan at lists.osuosl.org' <[email protected]>; 'Ruhao Gao' <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH net-next v5 1/4] igb: add support of RX network flow classification<snip>
Quoted 33 lines
Ok, the full oops output made me notice the problem. Can you try the following untested fix on your failing system? It'll apply against patch 1/4 and is build tested against the next-queue dev-queue branch.
-Matt
From
8773d0f09741d7a318cdd96cf5aad2ddb79096e7Mon Sep 17 00:00:00 2001
From: Matt Porter <[email protected]>
Date: Thu, 30 Jun 2016 11:47:50 -0400
Subject: [PATCH] igb: Fix missing lock init in rx network flow classification support"[net-next,v5,1/4] igb: add support of RX network flow classification" adds a new
nfc_lockto each adapter to protect the rx filter hash list. This lock is not initialized before use which results in intermittent oopses. Initialize this lock during probe to fix the issue.Signed-off-by Matt Porter <[email protected]>--- drivers/net/ethernet/intel/igb/igb_main.c | 1 + 1 file changed, 1 insertion(+)drivers/net/ethernet/intel/igb/igb_main.c +0
-0@@ -3075,6 +3075,7 @@ static int igb_sw_init(struct igb_adapter *adapter)VLAN_HLEN ; adapter->min_frame_size =ETH_ZLEN +ETH_FCS_LEN ;Quoted 8 lines
Quoted 6 lines
drivers/net/ethernet/intel/igb/igb_main.c +1
-1+spin_lock_init (&adapter->nfc_lock);spin_lock_init (&adapter->stats64_lock);#ifdef CONFIG_PCI_IOV switch (hw->mac.type) { --2.7.0
Bingo! That resolved it. I can now load and bind (with ifconfig) on all the systems that were previously giving me the splat without seeing anything out of the ordinary.
Excellent! Thanks for doing the grunt testing work on it. ;) Looks like this was the only remaining issue from the v5 submission.
Gangfeng: can you fold this fix into a v6 submission? If this isn't something you are still working on then let me know and I can resubmit.
Regards,
Matt
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan at lists.osuosl.org
http://lists.osuosl.org/mailman/listinfo/intel-wired-lan
PATCH v5 2/4 igb: support RX flow classification by ethertype
In reply to GangfengFrom: Gangfeng Huang <[email protected]>
This patch is meant to allow for RX network flow classification to insert and remove ethertype filter by ethtool
Example:
Add an ethertype filter:
$ ethtool -N eth0 flow-type ether proto 0x88F8 action 2
Show all filters:
$ ethtool -n eth0 4 RX rings available Total 1 rules
Filter: 15
Flow Type: Raw Ethernet Src MAC addr: 00:00:00:00:00:00 mask: FF:FF:FF:FF:FF:FF Dest MAC addr: 00:00:00:00:00:00 mask: FF:FF:FF:FF:FF:FF Ethertype: 0x88F8 mask: 0x0 Action: Direct to queue 2
Delete the filter by location:
$ ethtool -N delete 15
--- drivers/net/ethernet/intel/igb/e1000_82575.h | 5 ++ drivers/net/ethernet/intel/igb/igb.h | 20 +++++++- drivers/net/ethernet/intel/igb/igb_ethtool.c | 77 +++++++++++++++++++++++++++- drivers/net/ethernet/intel/igb/igb_ptp.c | 4 +- 4 files changed, 102 insertions(+), 4 deletions(-)
drivers/net/ethernet/intel/igb/e1000_82575.h +5 -0
@@ -188,6 +188,11 @@ struct e1000_adv_tx_context_desc {/* ETQF register bit definitions */ #define E1000_ETQF_FILTER_ENABLE BIT(26)#define E1000_ETQF_1588 BIT(30) +#define E1000_ETQF_IMM_INT BIT(29) +#define E1000_ETQF_QUEUE_ENABLE BIT(31) +#define E1000_ETQF_QUEUE_SHIFT 16 +#define E1000_ETQF_QUEUE_MASK 0x00070000 +#define E1000_ETQF_ETYPE_MASK 0x0000FFFF/* FTQF register bit definitions */ #define E1000_FTQF_VF_BP 0x00008000
drivers/net/ethernet/intel/igb/igb.h +19 -1
@@ -350,13 +350,27 @@ struct hwmon_buff { };#endif +/* The number of L2 ether-type filter registers, Index 3 is reserved +* for PTP 1588 timestamp +*/ +#define MAX_ETYPE_FILTER (4 - 1) + +/* ETQF filter list: one static filter per filter consumer. This is +* to avoid filter collisions later. Add new filters +* here!! +* +* Current filters: +* 1588 (0x88f7): Filter 3 +*/ +#define IGB_ETQF_FILTER_1588 3 +#define IGB_N_EXTTS 2#define IGB_N_PEROUT 2#define IGB_N_SDP 4#define IGB_RETA_SIZE 128enum igb_filter_match_flags { -IGB_FILTER_FLAG_NONE =0x0 , +IGB_FILTER_FLAG_ETHER_TYPE =0x1 , };#define IGB_MAX_RXNFC_FILTERS 16
@@ -365,13 +379,16 @@ enum igb_filter_match_flags {struct igb_nfc_input {/* Byte layout in order, all values with MSB first: * match_flags - 1 byte +* etype - 2 bytes */ u8 match_flags; +__be16 etype; };struct igb_nfc_filter {struct hlist_node nfc_node;struct igb_nfc_input filter; +u16 etype_reg_index;u16 sw_idx;u16 action; };
@@ -499,6 +516,7 @@ struct igb_adapter {unsigned int nfc_filter_count;/* lock for RX network flow classification filter */ spinlock_t nfc_lock; +bool etype_bitmap[MAX_ETYPE_FILTER ]; };#define IGB_FLAG_HAS_MSI BIT(0)
drivers/net/ethernet/intel/igb/igb_ethtool.c +76 -1
111 lines not on this page · raw message
drivers/net/ethernet/intel/igb/igb_ptp.c +2 -2
@@ -977,12 +977,12 @@ static int igb_ptp_set_timestamp_mode(struct igb_adapter *adapter,/* define ethertype filter for timestamped packets */ if (is_l2) -wr32 (E1000_ETQF (3 ), +wr32 (E1000_ETQF (IGB_ETQF_FILTER_1588 ), (E1000_ETQF_FILTER_ENABLE |/* enable filter */ E1000_ETQF_1588 |/* enable timestamping */ ETH_P_1588 ));/* 1588 eth protocol type */ else -wr32 (E1000_ETQF (3 ),0 ); +wr32 (E1000_ETQF (IGB_ETQF_FILTER_1588 ),0 );/* L4 Queue Filter[3]: filter by destination port and protocol */ if (is_l4) {
Signature · 1 line
2.7.2
PATCH v5 3/4 igb: support RX flow classification by VLAN priority
In reply to GangfengFrom: Gangfeng Huang <[email protected]>
This patch is meant to allow for RX network flow classification to insert and remove VLAN priority filter by ethtool
Example:
Add an VLAN priority filter:
$ ethtool -N eth0 flow-type ether vlan 0x6000 vlan-mask 0x1FFF action 2 loc 1
Show all filters:
$ ethtool -n eth0 4 RX rings available Total 1 rules
Filter: 1
Flow Type: Raw Ethernet Src MAC addr: 00:00:00:00:00:00 mask: FF:FF:FF:FF:FF:FF Dest MAC addr: 00:00:00:00:00:00 mask: FF:FF:FF:FF:FF:FF Ethertype: 0x0 mask: 0xFFFF VLAN EtherType: 0x0 mask: 0xffff VLAN: 0x6000 mask: 0x1fff User-defined: 0x0 mask: 0xffffffffffffffff Action: Direct to queue 2
Delete the filter by location:
$ ethtool -N delete 1
--- drivers/net/ethernet/intel/igb/e1000_defines.h | 4 ++ drivers/net/ethernet/intel/igb/e1000_regs.h | 1 + drivers/net/ethernet/intel/igb/igb.h | 3 + drivers/net/ethernet/intel/igb/igb_ethtool.c | 91 ++++++++++++++++++++++++-- 4 files changed, 92 insertions(+), 7 deletions(-)
drivers/net/ethernet/intel/igb/e1000_defines.h +4 -0
@@ -1024,4 +1024,8 @@#define E1000_RTTBCNRC_RF_INT_MASK \ (E1000_RTTBCNRC_RF_DEC_MASK << E1000_RTTBCNRC_RF_INT_SHIFT) +#define E1000_VLAPQF_QUEUE_SEL (_n, q_idx) (q_idx << ((_n) * 4)) +#define E1000_VLAPQF_P_VALID (_n) (0x1 << (3 + (_n) * 4)) +#define E1000_VLAPQF_QUEUE_MASK 0x03 +#endif
drivers/net/ethernet/intel/igb/e1000_regs.h +1 -0
@@ -309,6 +309,7 @@ (0x054E0 + ((_i - 16) * 8)))#define E1000_RAH (_i) (((_i) <= 15) ? (0x05404 + ((_i) * 8)) : \ (0x054E4 + ((_i - 16) * 8))) +#define E1000_VLAPQF 0x055B0/* VLAN Priority Queue Filter VLAPQF */ #define E1000_IP4AT_REG (_i) (0x05840 + ((_i) * 8))#define E1000_IP6AT_REG (_i) (0x05880 + ((_i) * 4))#define E1000_WUPM_REG (_i) (0x05A00 + ((_i) * 4))
drivers/net/ethernet/intel/igb/igb.h +3 -0
@@ -371,6 +371,7 @@ struct hwmon_buff {enum igb_filter_match_flags {IGB_FILTER_FLAG_ETHER_TYPE =0x1 , +IGB_FILTER_FLAG_VLAN_TCI =0x2 , };#define IGB_MAX_RXNFC_FILTERS 16
@@ -380,9 +381,11 @@ struct igb_nfc_input {/* Byte layout in order, all values with MSB first: * match_flags - 1 byte * etype - 2 bytes +* vlan_tci - 2 bytes */ u8 match_flags;__be16 etype; +__be16 vlan_tci; };struct igb_nfc_filter {
drivers/net/ethernet/intel/igb/igb_ethtool.c +84 -7
139 lines not on this page · raw message
Signature · 1 line
2.7.2
PATCH v5 4/4 igb: fix error code in igb_add_ethtool_nfc_entry()
In reply to GangfengFrom: Gangfeng Huang <[email protected]>
Use error "rmgr: Cannot insert RX class rule: Operation not supported" is more meaningful than "rmgr: Cannot insert RX class rule: Unknown error 524"
--- drivers/net/ethernet/intel/igb/igb_ethtool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/intel/igb/igb_ethtool.c +1 -1
@@ -2851,7 +2851,7 @@ static int igb_add_ethtool_nfc_entry(struct igb_adapter *adapter,int err =0 ;if (!(netdev->hw_features &NETIF_F_NTUPLE )) -return -ENOTSUPP ; +return -EOPNOTSUPP ;/* Don't allow programming if the action is a queue greater than * the number of online Rx queues.
Signature · 1 line
2.7.2
v5 cover igb: enable RX network flow classification
In reply to GangfengQuoted 21 lines · Gangfeng
After apply this series of patches, igb driver will supports advanced RX filter that direct receive packets by flows to different hardware queue. Enables tight control on routing a flow in the platform.
In our product, we use the the RX traffic classification to gurantee the PTP(ethertype is
0x88F7) packets won't be flooded by best effort packet. PTP packets is always be processed by Linux network stack.step 1.?
? use the ethertype filter to filter most of packet(0x0800) to HW queue; step 2.
? Add a rule to forward the PTP packets to another HW queueThe ethtool commands and options:
-n --show-nfc
?? Retrieves the receive network flow classification configurations-N --config-nfc
?? Configures the receive network flow classification classification
Quoted 16 lines
Change history of this patches:
Version 1:
? Split the patch to 3 patches for review;
? Save the filters and restore them after a reset;
? In ethtool command, use key word "proto" to replace "vlan-etype:
? Fix bugs
Version 2:
? Rebase the patches on the the latest dev-queue; Version 3:
? Report a meaningful error code for misoperation; Version 4:
? Rebase the patches on latest dev-queue;
? Update the patch description and code comments;? Version 5:
? Rebase the patches on latest dev-queue;
? Update the patch description;
Based on the issues validation has found with your first patch, I am
dropping your series of patches. ?Please address the issues Aaron found
during validation.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20160516/6b7d9e6b/attachment-0001.asc>