examples/ntb: remove redundant logic for dev close
PATCH 1/1 examples/ntb: remove redundant logic for dev close
During EAL cleanup stage, all bus devices are cleaned up properly. Based on this, there is no need to do extra device close process, which may call the dev ops '*dev->dev_ops->dev_close' twice.
If this dev ops for ntb was called twice, the interrupt handle for EAL will be disabled twice and will lead to error for the seconde time. Like this: "EAL: Error disabling MSI-X interrupts for fd xx"
Thus, this patch just remove the redundant logic for device close. Ports will be closed at rte_eal_cleanup() in the main loop.
--- examples/ntb/ntb_fwd.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
examples/ntb/ntb_fwd.c +3 -5
@@ -177,13 +177,11 @@ cmd_quit_parsed(__rte_unused void *parsed_result,rte_eal_mp_wait_lcore (); in_test =0 ; -/* Stop traffic and Close port. */ +/* Stop traffic only. */ rte_rawdev_stop (dev_id); -rte_rawdev_close (dev_id); -if (eth_port_id <RTE_MAX_ETHPORTS && fwd_mode ==IOFWD ) { +if (eth_port_id <RTE_MAX_ETHPORTS && fwd_mode ==IOFWD )rte_eth_dev_stop (eth_port_id); -rte_eth_dev_close (eth_port_id); - } +/* Ports will be closed at rte_eal_cleanup() in the main loop. */ cmdline_quit (cl); }
Signature · 1 line
2.34.1
Quoted 17 lines
-----Original Message-----
From: Junfeng Guo <[email protected]>
Sent: Wednesday, June 14, 2023 1:14 PM
To: Wu, Jingjing <[email protected]>
Cc: [email protected]; [email protected]; He, Xingguang <[email protected]>; Laatz, Kevin <[email protected]>; Guo, Junfeng <[email protected]>
Subject: [PATCH] examples/ntb: remove redundant logic for dev closeDuring EAL cleanup stage, all bus devices are cleaned up properly. Based on this, there is no need to do extra device close process, which may call the dev ops '
*dev->dev_ops->dev_close' twice.If this dev ops for ntb was called twice, the interrupt handle for EAL will be disabled twice and will lead to error for the seconde time. Like this: "EAL:
Error disabling MSI-X interrupts for fd xx"
Quoted 8 lines
Thus, this patch just remove the redundant logic for device close. Ports will be closed at
rte_eal_cleanup()in the main loop.Fixes 1cab1a40ea9b ("bus: cleanup devices on shutdown")Signed-off-by Junfeng Guo <[email protected]>---
14/06/2023 07:14, Junfeng Guo:
Quoted 7 lines
During EAL cleanup stage, all bus devices are cleaned up properly. Based on this, there is no need to do extra device close process, which may call the dev ops '
*dev->dev_ops->dev_close' twice.If this dev ops for ntb was called twice, the interrupt handle for EAL will be disabled twice and will lead to error for the seconde time. Like this: "EAL: Error disabling MSI-X interrupts for fd xx"
You should fix ntb to not disable interrupt twice.
Quoted 17 lines
-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Wednesday, June 28, 2023 08:12
To: Wu, Jingjing <[email protected]>; Guo, Junfeng <[email protected]>
Cc: [email protected]; [email protected]; He, Xingguang <[email protected]>; Laatz, Kevin <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [PATCH] examples/ntb: remove redundant logic for dev close14/06/2023 07:14, Junfeng Guo:
Quoted 4 lines
During EAL cleanup stage, all bus devices are cleaned up properly. Based on this, there is no need to do extra device close process, which may call the dev ops '
*dev->dev_ops->dev_close' twice.
Quoted 5 lines
Quoted 3 lines
If this dev ops for ntb was called twice, the interrupt handle for EAL will be disabled twice and will lead to error for the seconde time. Like this: "EAL: Error disabling MSI-X interrupts for fd xx"
You should fix ntb to not disable interrupt twice.
Thanks for the comment!
Yes, it is exactly what this patch did.
The interrupt disabling process is part of 'dev_close' ops, and will be called twice (one in EAL cleanup, another in ntb example cleanup).
Now that fixed & covered by the EAL cleanup, there is no need to run dev_close at ntb example. Just remove that redundant logic.
Regards,
Junfeng
Quoted 1 line
Quoted 34 lines
-----Original Message-----
From: Guo, Junfeng
Sent: Wednesday, June 28, 2023 10:19
To: Thomas Monjalon <[email protected]>; Wu, Jingjing <[email protected]>
Cc: [email protected]; [email protected]; He, Xingguang <[email protected]>; Laatz, Kevin <[email protected]>; Richardson, Bruce <[email protected]>
Subject: RE: [PATCH] examples/ntb: remove redundant logic for dev closeQuoted 9 lines
-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Wednesday, June 28, 2023 08:12
To: Wu, Jingjing <[email protected]>; Guo, Junfeng <[email protected]>
Cc: [email protected]; [email protected]; He, Xingguang <[email protected]>; Laatz, Kevin <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [PATCH] examples/ntb: remove redundant logic for devclose
Quoted 10 lines
14/06/2023 07:14, Junfeng Guo:
Quoted 7 lines
During EAL cleanup stage, all bus devices are cleaned up properly. Based on this, there is no need to do extra device close process, which may call the dev ops '
*dev->dev_ops->dev_close' twice.If this dev ops for ntb was called twice, the interrupt handle for EAL will be disabled twice and will lead to error for the seconde time. Like this: "EAL: Error disabling MSI-X interrupts for fd xx"
Quoted 5 lines
Quoted 1 line
You should fix ntb to not disable interrupt twice.
Thanks for the comment!
Yes, it is exactly what this patch did.
Another solution is to add extra logic in dev_close, i.e., to do the interrupt disabling process only when the interrupt handler is valid. That is, to disable interrupt only when 'intr_handle->max_intr' or 'intr_handle->nb_efd' is non-zero.
But I'm not sure if this method is a better solution. Please help give some advice, thanks!
Quoted 11 lines
The interrupt disabling process is part of '
dev_close' ops, and will be called twice (one in EAL cleanup, another in ntb example cleanup).Now that fixed & covered by the EAL cleanup, there is no need to run
dev_closeat ntb example. Just remove that redundant logic.Regards,
JunfengQuoted 1 line
28/06/2023 04:37, Guo, Junfeng:
Quoted 17 lines
From: Guo, Junfeng
Quoted 10 lines
From: Thomas Monjalon <[email protected]>
Quoted 9 lines
14/06/2023 07:14, Junfeng Guo:
Quoted 7 lines
During EAL cleanup stage, all bus devices are cleaned up properly. Based on this, there is no need to do extra device close process, which may call the dev ops '
*dev->dev_ops->dev_close' twice.If this dev ops for ntb was called twice, the interrupt handle for EAL will be disabled twice and will lead to error for the seconde time. Like this: "EAL: Error disabling MSI-X interrupts for fd xx"
Quoted 5 lines
Quoted 1 line
You should fix ntb to not disable interrupt twice.
Thanks for the comment!
Yes, it is exactly what this patch did.
Quoted 7 lines
Another solution is to add extra logic in
dev_close, i.e., to do the interrupt disabling process only when the interrupt handler is valid. That is, to disable interrupt only when 'intr_handle->max_intr' or 'intr_handle->nb_efd' is non-zero.But I'm not sure if this method is a better solution. Please help give some advice, thanks!
Yes this is what I suggest.
You cannot fix all applications in the world, but you can fix the driver, so it is a better way :)
Quoted 5 lines
Quoted 5 lines
The interrupt disabling process is part of '
dev_close' ops, and will be called twice (one in EAL cleanup, another in ntb example cleanup).Now that fixed & covered by the EAL cleanup, there is no need to run
dev_closeat ntb example. Just remove that redundant logic.
Quoted 35 lines
-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Wednesday, June 28, 2023 15:53
To: Wu, Jingjing <[email protected]>; Guo, Junfeng <[email protected]>
Cc: [email protected]; [email protected]; He, Xingguang <[email protected]>; Laatz, Kevin <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [PATCH] examples/ntb: remove redundant logic for dev close28/06/2023 04:37, Guo, Junfeng:
Quoted 22 lines
From: Guo, Junfeng
Quoted 10 lines
From: Thomas Monjalon <[email protected]>
Quoted 9 lines
14/06/2023 07:14, Junfeng Guo:
Quoted 7 lines
During EAL cleanup stage, all bus devices are cleaned up properly. Based on this, there is no need to do extra device close process, which may call the dev ops '
*dev->dev_ops->dev_close' twice.If this dev ops for ntb was called twice, the interrupt handle for EAL will be disabled twice and will lead to error for the seconde time. Like this: "EAL: Error disabling MSI-X interrupts for fd xx"
Quoted 5 lines
Quoted 1 line
You should fix ntb to not disable interrupt twice.
Thanks for the comment!
Yes, it is exactly what this patch did.
Another solution is to add extra logic in
dev_close, i.e., to do the interrupt disabling process only when the interrupt handler is valid. That is, to disable interrupt only when 'intr_handle->max_intr' or 'intr_handle->nb_efd' is non-zero.
Quoted 6 lines
Quoted 2 lines
But I'm not sure if this method is a better solution. Please help give some advice, thanks!
Yes this is what I suggest.
You cannot fix all applications in the world, but you can fix the driver, so it is a better way :)
Sure! Thanks for the advice!
Will update with this solution, thanks!
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
The interrupt disabling process is part of '
dev_close' ops, and will be called twice (one in EAL cleanup, another in ntb example cleanup).Now that fixed & covered by the EAL cleanup, there is no need to run
dev_closeat ntb example. Just remove that redundant logic.
PATCH v2 1/1 raw/ntb: add check for disabling interrupt in dev close ops
In reply to Junfeng GuoDuring EAL cleanup stage, all bus devices are cleaned up properly. In the meantime, the ntb example app will also do the device cleanup process, which may call the dev ops '*dev->dev_ops->dev_close' twice.
If this dev ops for ntb was called twice, the interrupt handle for EAL will be disabled twice and will lead to error for the seconde time. Like this: "EAL: Error disabling MSI-X interrupts for fd xx"
Thus, this patch added the check process for disabling interrupt in dev_close ops, to ensure that interrupt only be disabled once.
--- drivers/raw/ntb/ntb.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/raw/ntb/ntb.c +5 -0
@@ -1045,6 +1045,11 @@ ntb_dev_close(struct rte_rawdev *dev) hw->queue_pairs =0 ; intr_handle = hw->pci_dev->intr_handle; +/* Disable interrupt only once */ +if (!rte_intr_nb_efd_get (intr_handle) && + !rte_intr_max_intr_get (intr_handle)) +return 0 ; +/* Clean datapath event and vec mapping */ rte_intr_efd_disable (intr_handle);rte_intr_vec_list_free (intr_handle);
Signature · 1 line
2.34.1
RE: [PATCH v2] raw/ntb: add check for disabling interrupt in dev close ops
Quoted 21 lines
-----Original Message-----
From: Junfeng Guo <[email protected]>
Sent: Wednesday, June 28, 2023 5:12 PM
To: Wu, Jingjing <[email protected]>
Cc: [email protected]; [email protected]; He, Xingguang <[email protected]>; Laatz, Kevin <[email protected]>; Guo, Junfeng <[email protected]>
Subject: [PATCH v2] raw/ntb: add check for disabling interrupt in dev close opsDuring EAL cleanup stage, all bus devices are cleaned up properly. In the meantime, the ntb example app will also do the device cleanup process, which may call the dev ops '
*dev->dev_ops->dev_close' twice.If this dev ops for ntb was called twice, the interrupt handle for EAL will be disabled twice and will lead to error for the seconde time. Like this: "EAL:
Error disabling MSI-X interrupts for fd xx"Thus, this patch added the check process for disabling interrupt in
dev_closeops, to ensure that interrupt only be disabled once.
Quoted 8 lines
Fixes 1cab1a40ea9b ("bus: cleanup devices on shutdown")Signed-off-by Junfeng Guo <[email protected]>--- drivers/raw/ntb/ntb.c | 5 +++++ 1 file changed, 5 insertions(+)
RE: [PATCH v2] raw/ntb: add check for disabling interrupt in dev close ops
In reply to Junfeng GuoQuoted 18 lines
-----Original Message-----
From: Guo, Junfeng <[email protected]>
Sent: Wednesday, June 28, 2023 5:12 PM
To: Wu, Jingjing <[email protected]>
Cc: [email protected]; [email protected]; He, Xingguang <[email protected]>; Laatz, Kevin <[email protected]>; Guo, Junfeng <[email protected]>
Subject: [PATCH v2] raw/ntb: add check for disabling interrupt in dev close opsDuring EAL cleanup stage, all bus devices are cleaned up properly. In the meantime, the ntb example app will also do the device cleanup process, which may call the dev ops '
*dev->dev_ops->dev_close' twice.If this dev ops for ntb was called twice, the interrupt handle for EAL will be disabled twice and will lead to error for the seconde time. Like this: "EAL: Error disabling MSI-X interrupts for fd xx"
Quoted 7 lines
Thus, this patch added the check process for disabling interrupt in
dev_closeops, to ensure that interrupt only be disabled once.Fixes 1cab1a40ea9b ("bus: cleanup devices on shutdown")Signed-off-by Junfeng Guo <[email protected]>
Thomas Monjalon
AppliedRe: [PATCH v2] raw/ntb: add check for disabling interrupt in dev close ops
03/07/2023 08:53, Wu, Jingjing:
Quoted 20 lines
Quoted 20 lines
-----Original Message-----
From: Guo, Junfeng <[email protected]>
Sent: Wednesday, June 28, 2023 5:12 PM
To: Wu, Jingjing <[email protected]>
Cc: [email protected]; [email protected]; He, Xingguang <[email protected]>; Laatz, Kevin <[email protected]>; Guo, Junfeng <[email protected]>
Subject: [PATCH v2] raw/ntb: add check for disabling interrupt in dev close opsDuring EAL cleanup stage, all bus devices are cleaned up properly. In the meantime, the ntb example app will also do the device cleanup process, which may call the dev ops '
*dev->dev_ops->dev_close' twice.If this dev ops for ntb was called twice, the interrupt handle for EAL will be disabled twice and will lead to error for the seconde time. Like this: "EAL: Error disabling MSI-X interrupts for fd xx"
Thus, this patch added the check process for disabling interrupt in
dev_closeops, to ensure that interrupt only be disabled once.
Quoted 6 lines
Quoted 4 lines
Fixes 1cab1a40ea9b ("bus: cleanup devices on shutdown")Signed-off-by Junfeng Guo <[email protected]>Acked-by Jingjing Wu <[email protected]>
Applied, thanks.