Fixes in igbvf driver
Subsystems (3)
cover Fixes in igbvf driver
The first patch fixes a bug manifested manifested in the igbvf driver when interrupting handling for the igb driver delays.
The second is just a cleanup for igbvf.
Wander Lairson Costa (2):
igb: Disable threaded IRQ for igb_msix_other
igbvf: remove unused spinlock
drivers/net/ethernet/intel/igb/igb_main.c | 2 +- drivers/net/ethernet/intel/igbvf/igbvf.h | 3 --- drivers/net/ethernet/intel/igbvf/netdev.c | 3 --- 3 files changed, 1 insertion(+), 7 deletions(-)
Signature · 1 line
2.46.1
PATCH 1/2 igb: Disable threaded IRQ for igb_msix_other
During testing of SR-IOV, Red Hat QE encountered an issue where the ip link up command intermittently fails for the igbvf interfaces when using the PREEMPT_RT variant. Investigation revealed that e1000_write_posted_mbx returns an error due to the lack of an ACK from e1000_poll_for_ack.
The underlying issue arises from the fact that IRQs are threaded by default under PREEMPT_RT. While the exact hardware details are not available, it appears that the IRQ handled by igb_msix_other must be processed before e1000_poll_for_ack times out. However, e1000_write_posted_mbx is called with preemption disabled, leading to a scenario where the IRQ is serviced only after the failure of e1000_write_posted_mbx.
To resolve this, we set IRQF_NO_THREAD for the affected interrupt, ensuring that the kernel handles it immediately, thereby preventing the aforementioned error.
Reproducer:
#!/bin/bash
# echo 2 > /sys/class/net/ens14f0/device/sriov_numvfs
ipaddr_vlan=3
nic_test=ens14f0
vf=${nic_test}v0
while true; do
ip link set ${nic_test} mtu 1500
ip link set ${vf} mtu 1500
ip link set $vf up
ip link set ${nic_test} vf 0 vlan ${ipaddr_vlan}
ip addr add 172.30.${ipaddr_vlan}.1/24 dev ${vf}
ip addr add 2021:db8:${ipaddr_vlan}::1/64 dev ${vf}
if ! ip link show $vf | grep 'state UP'; then
echo 'Error found'
break
fi
ip link set $vf down
done--- drivers/net/ethernet/intel/igb/igb_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/intel/igb/igb_main.c +1 -1
@@ -907,7 +907,7 @@ static int igb_request_msix(struct igb_adapter *adapter)int i, err =0 , vector =0 , free_vector =0 ; err =request_irq (adapter->msix_entries[vector].vector, - igb_msix_other,0 , netdev->name, adapter); + igb_msix_other,IRQF_NO_THREAD , netdev->name, adapter);if (err) goto err_out;
Signature · 1 line
2.46.1
Re: [PATCH 1/2] igb: Disable threaded IRQ for igb_msix_other
Quoted 52 lines · Wander Lairson Costa
During testing of SR-IOV, Red Hat QE encountered an issue where the ip link up command intermittently fails for the igbvf interfaces when using the
PREEMPT_RTvariant. Investigation revealed thate1000_write_posted_mbxreturns an error due to the lack of an ACK frome1000_poll_for_ack.The underlying issue arises from the fact that IRQs are threaded by default under
PREEMPT_RT. While the exact hardware details are not available, it appears that the IRQ handled byigb_msix_othermust be processed beforee1000_poll_for_acktimes out. However,e1000_write_posted_mbxis called with preemption disabled, leading to a scenario where the IRQ is serviced only after the failure ofe1000_write_posted_mbx.To resolve this, we set
IRQF_NO_THREADfor the affected interrupt, ensuring that the kernel handles it immediately, thereby preventing the aforementioned error.Reproducer:
#!/bin/bash # echo 2 > /sys/class/net/ens14f0/device/sriov_numvfs ipaddr_vlan=3 nic_test=ens14f0 vf=${nic_test}v0 while true; do ip link set ${nic_test} mtu 1500 ip link set ${vf} mtu 1500 ip link set $vf up ip link set ${nic_test} vf 0 vlan ${ipaddr_vlan} ip addr add 172.30.${ipaddr_vlan}.1/24 dev ${vf} ip addr add 2021:db8:${ipaddr_vlan}::1/64 dev ${vf} if ! ip link show $vf | grep 'state UP'; then echo 'Error found' break fi ip link set $vf down doneSigned-off-by Wander Lairson Costa <[email protected]>Reported-by Yuying Ma <[email protected]>--- drivers/net/ethernet/intel/igb/igb_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/net/ethernet/intel/igb/igb_main.c +0
-0
drivers/net/ethernet/intel/igb/igb_main.c static int igb_request_msix(struct igb_adapter *adapter) · line 913
Quoted 8 lines
drivers/net/ethernet/intel/igb/igb_main.c +1
-1@@ -907,7 +907,7 @@ static int igb_request_msix(struct igb_adapter *adapter)int i, err =0 , vector =0 , free_vector =0 ; err =request_irq (adapter->msix_entries[vector].vector, - igb_msix_other,0 , netdev->name, adapter); + igb_msix_other,IRQF_NO_THREAD , netdev->name, adapter);if (err) goto err_out;
Thank you for small, localized fix with a good description. Our VAL will check it also on non-RT OS.
PS: for future intel ethernet submissions please split out fixes and refactors, and tag each commit with the [iwl-net] or [iwl-next] tags
Romanowski, Rafal
Tested-by Rafal RomanowskiRe: [Intel-wired-lan] [PATCH 1/2] igb: Disable threaded IRQ for igb_msix_other
From: Intel-wired-lan <[email protected]> on behalf of Przemek Kitszel <[email protected]>
Sent: Monday, September 23, 2024 11:07 AM
To: Wander Lairson Costa <[email protected]>
Cc: Yuying Ma <[email protected]>; moderated list:INTEL ETHERNET DRIVERS <[email protected]>; open list <[email protected]>; Eric Dumazet <[email protected]>; Nguyen, Anthony L <[email protected]>; open list:NETWORKING DRIVERS <[email protected]>; Jakub Kicinski <[email protected]>; Paolo Abeni <[email protected]>; David S. Miller <[email protected]>
Subject: Re: [Intel-wired-lan] [PATCH 1/2] igb: Disable threaded IRQ for igb_msix_other
Quoted 52 lines · Wander Lairson Costa
During testing of SR-IOV, Red Hat QE encountered an issue where the ip link up command intermittently fails for the igbvf interfaces when using the
PREEMPT_RTvariant. Investigation revealed thate1000_write_posted_mbxreturns an error due to the lack of an ACK frome1000_poll_for_ack.The underlying issue arises from the fact that IRQs are threaded by default under
PREEMPT_RT. While the exact hardware details are not available, it appears that the IRQ handled byigb_msix_othermust be processed beforee1000_poll_for_acktimes out. However,e1000_write_posted_mbxis called with preemption disabled, leading to a scenario where the IRQ is serviced only after the failure ofe1000_write_posted_mbx.To resolve this, we set
IRQF_NO_THREADfor the affected interrupt, ensuring that the kernel handles it immediately, thereby preventing the aforementioned error.Reproducer:
#!/bin/bash # echo 2 > /sys/class/net/ens14f0/device/sriov_numvfs ipaddr_vlan=3 nic_test=ens14f0 vf=${nic_test}v0 while true; do ip link set ${nic_test} mtu 1500 ip link set ${vf} mtu 1500 ip link set $vf up ip link set ${nic_test} vf 0 vlan ${ipaddr_vlan} ip addr add 172.30.${ipaddr_vlan}.1/24 dev ${vf} ip addr add 2021:db8:${ipaddr_vlan}::1/64 dev ${vf} if ! ip link show $vf | grep 'state UP'; then echo 'Error found' break fi ip link set $vf down doneSigned-off-by Wander Lairson Costa <[email protected]>Reported-by Yuying Ma <[email protected]>--- drivers/net/ethernet/intel/igb/igb_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/net/ethernet/intel/igb/igb_main.c +0
-0
drivers/net/ethernet/intel/igb/igb_main.c static int igb_request_msix(struct igb_adapter *adapter) · line 913
Quoted 8 lines
drivers/net/ethernet/intel/igb/igb_main.c +1
-1@@ -907,7 +907,7 @@ static int igb_request_msix(struct igb_adapter *adapter)int i, err =0 , vector =0 , free_vector =0 ; err =request_irq (adapter->msix_entries[vector].vector, - igb_msix_other,0 , netdev->name, adapter); + igb_msix_other,IRQF_NO_THREAD , netdev->name, adapter);if (err) goto err_out;
PATCH 2/2 igbvf: remove unused spinlock
In reply to Wander Lairson Costatx_queue_lock and stats_lock are declared and initialized, but never used. Remove them.
--- drivers/net/ethernet/intel/igbvf/igbvf.h | 3 --- drivers/net/ethernet/intel/igbvf/netdev.c | 3 --- 2 files changed, 6 deletions(-)
drivers/net/ethernet/intel/igbvf/igbvf.h +0 -3
@@ -169,8 +169,6 @@ struct igbvf_adapter {u16 link_speed;u16 link_duplex; -spinlock_t tx_queue_lock;/* prevent concurrent tail updates */ -/* track device up/down/testing state */ unsigned long state;
@@ -220,7 +218,6 @@ struct igbvf_adapter {/* OS defined structs */ struct net_device *netdev;struct pci_dev *pdev; -spinlock_t stats_lock;/* prevent concurrent stats updates */ /* structs defined in e1000_hw.h */ struct e1000_hw hw;
drivers/net/ethernet/intel/igbvf/netdev.c +0 -3
@@ -1656,12 +1656,9 @@ static int igbvf_sw_init(struct igbvf_adapter *adapter)if (igbvf_alloc_queues (adapter))return -ENOMEM ; -spin_lock_init (&adapter->tx_queue_lock); -/* Explicitly disable IRQ since the NIC can be in any state. */ igbvf_irq_disable (adapter); -spin_lock_init (&adapter->stats_lock);spin_lock_init (&adapter->hw.mbx_lock);set_bit (__IGBVF_DOWN, &adapter->state);
Signature · 1 line
2.46.1
Re: [Intel-wired-lan] [PATCH 2/2] igbvf: remove unused spinlock
Dear Wander,
Thank you for your patch.
Am 20.09.24 um 20:59 schrieb Wander Lairson Costa:
Quoted 4 lines
tx_queue_lockandstats_lockare declared and initialized, but never used. Remove them.Signed-off-by Wander Lairson Costa <[email protected]>
It’d be great if you added a Fixes: tag.
Quoted 33 lines
--- drivers/net/ethernet/intel/igbvf/igbvf.h | 3 --- drivers/net/ethernet/intel/igbvf/netdev.c | 3 --- 2 files changed, 6 deletions(-)drivers/net/ethernet/intel/igbvf/igbvf.h +0
-3@@ -169,8 +169,6 @@ struct igbvf_adapter {u16 link_speed;u16 link_duplex; -spinlock_t tx_queue_lock;/* prevent concurrent tail updates */ -/* track device up/down/testing state */ unsigned long state;@@ -220,7 +218,6 @@ struct igbvf_adapter {/* OS defined structs */ struct net_device *netdev;struct pci_dev *pdev; -spinlock_t stats_lock;/* prevent concurrent stats updates */ /* structs defined in e1000_hw.h */ struct e1000_hw hw;drivers/net/ethernet/intel/igbvf/netdev.c +0
-0@@ -1656,12 +1656,9 @@ static int igbvf_sw_init(struct igbvf_adapter *adapter)if (igbvf_alloc_queues (adapter))return -ENOMEM ;
drivers/net/ethernet/intel/igbvf/netdev.c static int igbvf_sw_init(struct igbvf_adapter *adapter) · line 1664
Quoted 10 lines
drivers/net/ethernet/intel/igbvf/netdev.c +0
-3-spin_lock_init (&adapter->tx_queue_lock); -/* Explicitly disable IRQ since the NIC can be in any state. */ igbvf_irq_disable (adapter); -spin_lock_init (&adapter->stats_lock);spin_lock_init (&adapter->hw.mbx_lock);set_bit (__IGBVF_DOWN, &adapter->state);
With that addressed:
Kind regards,
Paul
Przemek Kitszel
maintainerRe: [Intel-wired-lan] [PATCH 2/2] igbvf: remove unused spinlock
Quoted 5 lines · Paul Menzel
Dear Wander,
Thank you for your patch.
Quoted 7 lines
Am 20.09.24 um 20:59 schrieb Wander Lairson Costa:
Quoted 4 lines
tx_queue_lockandstats_lockare declared and initialized, but never used. Remove them.Signed-off-by Wander Lairson Costa <[email protected]>It’d be great if you added a Fixes: tag.
Alternatively you could split this series into two, and send this patch to iwl-next tree, without the fixes tag. For me this patch is just a cleanup, not a fix.
Quoted 1 line
[...]
Quoted 9 lines
With that addressed:
Reviewed-by Paul Menzel <[email protected]>Kind regards,
Paul
Re: [Intel-wired-lan] [PATCH 2/2] igbvf: remove unused spinlock
Quoted 11 lines · Przemek Kitszel
Quoted 9 lines · Paul Menzel
Dear Wander,
Thank you for your patch.
Am 20.09.24 um 20:59 schrieb Wander Lairson Costa:
Quoted 3 lines
tx_queue_lockandstats_lockare declared and initialized, but never used. Remove them.
Quoted 10 lines
Quoted 3 lines
Quoted 1 line
Signed-off-by Wander Lairson Costa <[email protected]>It’d be great if you added a Fixes: tag.
Alternatively you could split this series into two, and send this patch to iwl-next tree, without the fixes tag. For me this patch is just a cleanup, not a fix.
Quoted 1 line
Should I send a new version of the patches separately?
Quoted 12 lines
[...]
Quoted 9 lines
With that addressed:
Reviewed-by Paul Menzel <[email protected]>Kind regards,
Paul
Tony Nguyen
maintainerRe: [Intel-wired-lan] [PATCH 2/2] igbvf: remove unused spinlock
Quoted 17 lines · Wander Lairson Costa
Quoted 15 lines · Przemek Kitszel
Quoted 12 lines · Paul Menzel
Dear Wander,
Thank you for your patch.
Am 20.09.24 um 20:59 schrieb Wander Lairson Costa:
Quoted 4 lines
tx_queue_lockandstats_lockare declared and initialized, but never used. Remove them.Signed-off-by Wander Lairson Costa <[email protected]>It’d be great if you added a Fixes: tag.
Quoted 8 lines
Quoted 6 lines
Alternatively you could split this series into two, and send this patch to iwl-next tree, without the fixes tag. For me this patch is just a cleanup, not a fix.
Quoted 1 line
Should I send a new version of the patches separately?
The patches apply to the respective trees when split out so I can take these without a re-send. Patch 1 will need a Fixes: for it though...
I'm seeing it as: 9d5c824399de ("igb: PCI-Express 82575 Gigabit Ethernet driver")?
Thanks,
Tony
Quoted 13 lines
Quoted 12 lines
[...]
Quoted 9 lines
With that addressed:
Reviewed-by Paul Menzel <[email protected]>Kind regards,
Paul
Re: [Intel-wired-lan] [PATCH 2/2] igbvf: remove unused spinlock
Quoted 28 lines · Tony Nguyen
Quoted 24 lines · Wander Lairson Costa
Quoted 21 lines · Przemek Kitszel
Quoted 5 lines · Paul Menzel
Dear Wander,
Thank you for your patch.
Quoted 7 lines
Am 20.09.24 um 20:59 schrieb Wander Lairson Costa:
Quoted 4 lines
tx_queue_lockandstats_lockare declared and initialized, but never used. Remove them.Signed-off-by Wander Lairson Costa <[email protected]>It’d be great if you added a Fixes: tag.
Alternatively you could split this series into two, and send this patch to iwl-next tree, without the fixes tag. For me this patch is just a cleanup, not a fix.
Quoted 1 line
Quoted 8 lines
Quoted 1 line
Should I send a new version of the patches separately?
The patches apply to the respective trees when split out so I can take these without a re-send. Patch 1 will need a Fixes: for it though...
I'm seeing it as:
9d5c824399de("igb: PCI-Express 82575 Gigabit Ethernet driver")?
Can you add the tag when you apply the patch or should I add it?
Quoted 17 lines
Thanks,
TonyQuoted 13 lines
Quoted 12 lines
[...]
Quoted 9 lines
With that addressed:
Reviewed-by Paul Menzel <[email protected]>Kind regards,
Paul
Re: [Intel-wired-lan] [PATCH 2/2] igbvf: remove unused spinlock
Quoted 31 lines · Wander Lairson Costa
Quoted 30 lines · Tony Nguyen
Quoted 25 lines · Wander Lairson Costa
Quoted 11 lines · Przemek Kitszel
Quoted 9 lines · Paul Menzel
Dear Wander,
Thank you for your patch.
Am 20.09.24 um 20:59 schrieb Wander Lairson Costa:
Quoted 3 lines
tx_queue_lockandstats_lockare declared and initialized, but never used. Remove them.Quoted 10 lines
Quoted 3 lines
Quoted 1 line
Signed-off-by Wander Lairson Costa <[email protected]>It’d be great if you added a Fixes: tag.
Alternatively you could split this series into two, and send this patch to iwl-next tree, without the fixes tag. For me this patch is just a cleanup, not a fix.
Quoted 1 line
Should I send a new version of the patches separately?
Quoted 9 lines
Quoted 6 lines
The patches apply to the respective trees when split out so I can take these without a re-send. Patch 1 will need a Fixes: for it though...
I'm seeing it as:
9d5c824399de("igb: PCI-Express 82575 Gigabit Ethernet driver")?Can you add the tag when you apply the patch or should I add it?
I will add the fixes tag when I said it.
Thanks,
Jake