net/mlx5: set correct CPU socket ID for mlx5_rxq_ctrl
PATCH 1/1 net/mlx5: set correct CPU socket ID for mlx5_rxq_ctrl
I hit a failure during ports drop queue RQ creation when my adapters
are on CPU socket ID 1 instead of socket ID 0:
....
EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0020:01:00.0 (socket 1)
EAL: set_mempolicy failed: Invalid argumentmlx5_common: Failed to allocate memory for RQ.mlx5_net: Port 0 drop queue RQ creation failed.mlx5_net: Cannot create drop RX queuemlx5_net: probe of PCI device 0020:01:00.0 aborted after encountering an error: Success
EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0020:01:00.1 (socket 1)
EAL: set_mempolicy failed: Invalid argumentmlx5_common: Failed to allocate memory for RQ.mlx5_net: Port 0 drop queue RQ creation failed.mlx5_net: Cannot create drop RX queuemlx5_net: probe of PCI device 0020:01:00.1 aborted after encountering an error: Success
TELEMETRY: No legacy callbacks, legacy socket not created
testpmd: No probed ethernet devices
...
The patch sets the correct CPU socket ID for the mlx5_rxq_ctrl before calling the mlx5_rxq_create_devx_rq_resources() which eventually calls mlx5_devx_rq_create() with correct CPU socket ID.
Result with this patch:
......
EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0020:01:00.0 (socket 1)
EAL: Probe PCI driver: mlx5_pci (15b3:1019) device: 0020:01:00.1 (socket 1)
TELEMETRY: No legacy callbacks, legacy socket not created Interactive-mode selected
......
Configuring Port 0 (socket 1)
Port 0: 0C:42:A1:ED:C1:20
Configuring Port 1 (socket 1)
Port 1: 0C:42:A1:ED:C1:21
Checking link statuses...
Done
--- drivers/net/mlx5/mlx5_devx.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/net/mlx5/mlx5_devx.c +2 -0
@@ -947,6 +947,8 @@ mlx5_rxq_devx_obj_drop_create(struct rte_eth_dev *dev) rte_errno =ENOMEM ; goto error; } +/* set the CPU socket ID where the rxq_ctrl was allocated */ + rxq_ctrl->socket = socket_id; rxq_obj->rxq_ctrl = rxq_ctrl; rxq_ctrl->is_hairpin = false; rxq_ctrl->sh = priv->sh;
Signature · 1 line
2.27.0
From: Thinh Tran <[email protected]>
Quoted 38 lines
I hit a failure during ports drop queue RQ creation when my adapters are on CPU socket ID 1 instead of socket ID 0:
....
EAL: Probe PCI driver:mlx5_pci(15b3:1019) device: 0020:01:00.0 (socket 1)
EAL:set_mempolicyfailed: Invalid argumentmlx5_common: Failed to allocate memory for RQ.mlx5_net: Port 0 drop queue RQ creation failed.mlx5_net: Cannot create drop RX queuemlx5_net: probe of PCI device 0020:01:00.0 aborted after encountering an
error: Success
EAL: Probe PCI driver:mlx5_pci(15b3:1019) device: 0020:01:00.1 (socket 1)
EAL:set_mempolicyfailed: Invalid argumentmlx5_common: Failed to allocate memory for RQ.mlx5_net: Port 0 drop queue RQ creation failed.mlx5_net: Cannot create drop RX queuemlx5_net: probe of PCI device 0020:01:00.1 aborted after encountering an
error: Success
TELEMETRY: No legacy callbacks, legacy socket not created
testpmd: No probed ethernet devices
...The patch sets the correct CPU socket ID for the
mlx5_rxq_ctrlbefore calling themlx5_rxq_create_devx_rq_resources()which eventually callsmlx5_devx_rq_create()with correct CPU socket ID.
Result with this patch:
......
EAL: Probe PCI driver:mlx5_pci(15b3:1019) device: 0020:01:00.0 (socket 1)
EAL: Probe PCI driver:mlx5_pci(15b3:1019) device: 0020:01:00.1 (socket 1)
TELEMETRY: No legacy callbacks, legacy socket not created Interactive-mode selected ......
Configuring Port 0 (socket 1)
Port 0: 0C:42:A1:ED:C1:20
Configuring Port 1 (socket 1)
Port 1: 0C:42:A1:ED:C1:21
Checking link statuses...
Done
Quoted 2 lines
Signed-off-by Thinh Tran <[email protected]>Reviewed-by David Christensen <[email protected]>
Hi Thinh,
Quoted 7 lines
-----Original Message-----
From: Thinh Tran <[email protected]>
Sent: Tuesday, March 8, 2022 12:53 AM
To: [email protected]
Cc: [email protected]; Thinh Tran <[email protected]>
Subject: [PATCH] net/mlx5: set correct CPU socket ID formlx5_rxq_ctrl
This is a fix, so it's better to start commit title with a fix:
Something like this maybe:
"net/mlx5: fix CPU socket ID for mlx5_rxq_ctrl"
Quoted 12 lines
I hit a failure during ports drop queue RQ creation when my adapters are on CPU socket ID 1 instead of socket ID 0:
....
EAL: Probe PCI driver:mlx5_pci(15b3:1019) device: 0020:01:00.0 (socket 1)
EAL:set_mempolicyfailed: Invalid argumentmlx5_common: Failed to allocate memory for RQ.mlx5_net: Port 0 drop queue RQ creation failed.mlx5_net: Cannot create drop RX queuemlx5_net: probe of PCI device 0020:01:00.0 aborted after encountering an
error: Success
EAL: Probe PCI driver:mlx5_pci(15b3:1019) device: 0020:01:00.1 (socket 1)
EAL:set_mempolicyfailed: Invalid argument
Quoted 8 lines
mlx5_common: Failed to allocate memory for RQ.mlx5_net: Port 0 drop queue RQ creation failed.mlx5_net: Cannot create drop RX queuemlx5_net: probe of PCI device 0020:01:00.1 aborted after encountering an
error: Success
TELEMETRY: No legacy callbacks, legacy socket not created
testpmd: No probed ethernet devices
...
It's better to describe the issue rather than showing an example alone, Maybe something like this:
The socket ID is used to determine the socket where to allocate memory for mlx5_rxq_ctrl, currently it's set to 0 by default which mistakenly leads to allocating the memory on socket 0 always.
Quoted 5 lines
The patch sets the correct CPU socket ID for the
mlx5_rxq_ctrlbefore calling themlx5_rxq_create_devx_rq_resources()which eventually callsmlx5_devx_rq_create()with correct CPU socket ID.
Result with this patch:
This sets the correct CPU socket ID before memory allocation to use the correct socket ID.
Quoted 4 lines
......
EAL: Probe PCI driver:mlx5_pci(15b3:1019) device: 0020:01:00.0 (socket 1)
EAL: Probe PCI driver:mlx5_pci(15b3:1019) device: 0020:01:00.1 (socket 1)
TELEMETRY: No legacy callbacks, legacy socket not created
Quoted 9 lines
Interactive-mode selected
......
Configuring Port 0 (socket 1)
Port 0: 0C:42:A1:ED:C1:20
Configuring Port 1 (socket 1)
Port 1: 0C:42:A1:ED:C1:21
Checking link statuses...
Done
Missing:
Fixes tag:
Missing Cc stable for backport.
Quoted 4 lines
Signed-off-by Thinh Tran <[email protected]>Reviewed-by David Christensen <[email protected]>---
Kindest regards,
Raslan Darawsheh
Hi Raslan,
Quoted 5 lines
Missing:
Fixes tag:Fixes 5ceb3a02b000 ("net/mlx5: move Rx queue DevX resource")
I believe the bug originates from my earlier commit, not Xueming's one:
Quoted 14 lines
-----Original Message-----
From: Dmitry Kozlyuk <[email protected]>
Sent: Tuesday, March 8, 2022 2:23 PM
To: Raslan Darawsheh <[email protected]>
Cc: Thinh Tran <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH] net/mlx5: set correct CPU socket ID formlx5_rxq_ctrlHi Raslan,
Quoted 3 lines
Missing:
Fixes tag:
Quoted 6 lines
Quoted 2 lines
Fixes 5ceb3a02b000 ("net/mlx5: move Rx queue DevX resource")I believe the bug originates from my earlier commit, not Xueming's one:
Fixes bc5bee028ebc ("net/mlx5: create drop queue using DevX")
Yes I think you are correct, my mistake 😊
Kindest regards,
Raslan Darawsheh
Hi, Thinh
Thank you for the patch, the code looks OK to me, but commit message is not compliant:
- it should contain "fix" keyword in the title, like this:
"net/mlx5: fix CPU socket ID for Rx queue creation"
- could you, please, make problem description less personal and less wordy? "The default CPU socket ID was used while creating the Rx queue and this caused
creation failure in case if hardware was not resided on the default socket.
The patch sets the correct CPU socket ID for the mlx5_rxq_ctrl before calling the mlx5_rxq_create_devx_rq_resources() which eventually calls mlx5_devx_rq_create() with correct CPU socket ID."
- please add tags:
With best regards,
Slava
Quoted 33 lines
-----Original Message-----
From: Raslan Darawsheh <[email protected]>
Sent: Tuesday, March 8, 2022 14:25
To: Dmitry Kozlyuk <[email protected]>
Cc: Thinh Tran <[email protected]>; [email protected]; [email protected]
Subject: RE: [PATCH] net/mlx5: set correct CPU socket ID formlx5_rxq_ctrlQuoted 14 lines
-----Original Message-----
From: Dmitry Kozlyuk <[email protected]>
Sent: Tuesday, March 8, 2022 2:23 PM
To: Raslan Darawsheh <[email protected]>
Cc: Thinh Tran <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH] net/mlx5: set correct CPU socket ID formlx5_rxq_ctrlHi Raslan,
Quoted 3 lines
Missing:
Fixes tag:Quoted 6 lines
Quoted 2 lines
Fixes 5ceb3a02b000 ("net/mlx5: move Rx queue DevX resource")I believe the bug originates from my earlier commit, not Xueming's one:
Fixes bc5bee028ebc ("net/mlx5: create drop queue using DevX")Yes I think you are correct, my mistake 😊
Kindest regards,
Raslan Darawsheh
Hi,
Quoted 13 lines · Slava Ovsiienko
Hi, Thinh
Thank you for the patch, the code looks OK to me, but commit message is not compliant:
- it should contain "fix" keyword in the title, like this:
"net/mlx5: fix CPU socket ID for Rx queue creation"
- could you, please, make problem description less personal and less wordy? "The default CPU socket ID was used while creating the Rx queue and this caused
creation failure in case if hardware was not resided on the default socket.The patch sets the correct CPU socket ID for the
mlx5_rxq_ctrlbefore calling themlx5_rxq_create_devx_rq_resources()which eventually callsmlx5_devx_rq_create()with correct CPU socket ID."
- please add tags:
Quoted 6 lines
Fixes bc5bee028ebc ("net/mlx5: create drop queue using DevX")With best regards,
Slava
I'll resubmit the patch with suggestions above.
Regards,
Thinh
Quoted 33 lines
Quoted 33 lines
-----Original Message-----
From: Raslan Darawsheh <[email protected]>
Sent: Tuesday, March 8, 2022 14:25
To: Dmitry Kozlyuk <[email protected]>
Cc: Thinh Tran <[email protected]>; [email protected]; [email protected]
Subject: RE: [PATCH] net/mlx5: set correct CPU socket ID formlx5_rxq_ctrlQuoted 14 lines
-----Original Message-----
From: Dmitry Kozlyuk <[email protected]>
Sent: Tuesday, March 8, 2022 2:23 PM
To: Raslan Darawsheh <[email protected]>
Cc: Thinh Tran <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH] net/mlx5: set correct CPU socket ID formlx5_rxq_ctrlHi Raslan,
Quoted 3 lines
Missing:
Fixes tag:Quoted 6 lines
Quoted 2 lines
Fixes 5ceb3a02b000 ("net/mlx5: move Rx queue DevX resource")I believe the bug originates from my earlier commit, not Xueming's one:
Fixes bc5bee028ebc ("net/mlx5: create drop queue using DevX")Yes I think you are correct, my mistake 😊
Kindest regards,
Raslan Darawsheh
PATCH v2 1/1 net/mlx5: fix CPU socket ID for Rx queue creation
In reply to Thinh TranThe default CPU socket ID was used while creating the Rx queue and this caused creation failure in case if hardware was not resided on the default socket.
The patch sets the correct CPU socket ID for the mlx5_rxq_ctrl before calling the mlx5_rxq_create_devx_rq_resources() which eventually calls mlx5_devx_rq_create() with correct CPU socket ID.
v2: change patch description, no code change
--- drivers/net/mlx5/mlx5_devx.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/net/mlx5/mlx5_devx.c +2 -0
@@ -947,6 +947,8 @@ mlx5_rxq_devx_obj_drop_create(struct rte_eth_dev *dev) rte_errno =ENOMEM ; goto error; } +/* set the CPU socket ID where the rxq_ctrl was allocated */ + rxq_ctrl->socket = socket_id; rxq_obj->rxq_ctrl = rxq_ctrl; rxq_ctrl->is_hairpin = false; rxq_ctrl->sh = priv->sh;
Signature · 1 line
2.27.0
RE: [PATCH v2] net/mlx5: fix CPU socket ID for Rx queue creation
Hi, Thinh
the line: "v2: change patch description, no code change" should not be in commit message body, it should be after "---" Besides this:
With best regards,
Slava
Quoted 47 lines
-----Original Message-----
From: Thinh Tran <[email protected]>
Sent: Wednesday, March 9, 2022 21:49
To: [email protected]
Cc: [email protected]; Slava Ovsiienko <[email protected]>; Raslan Darawsheh <[email protected]>; Dmitry Kozlyuk <[email protected]>; Thinh Tran <[email protected]>; [email protected]
Subject: [PATCH v2] net/mlx5: fix CPU socket ID for Rx queue creationThe default CPU socket ID was used while creating the Rx queue and this caused creation failure in case if hardware was not resided on the default socket.
The patch sets the correct CPU socket ID for the
mlx5_rxq_ctrlbefore calling themlx5_rxq_create_devx_rq_resources()which eventually callsmlx5_devx_rq_create()with correct CPU socket ID.v2: change patch description, no code change
Fixes bc5bee028ebc ("net/mlx5: create drop queue using DevX")Signed-off-by Thinh Tran <[email protected]>Reviewed-by David Christensen <[email protected]>--- drivers/net/mlx5/mlx5_devx.c | 2 ++ 1 file changed, 2 insertions(+)drivers/net/mlx5/mlx5_devx.c +2
-0@@ -947,6 +947,8 @@ mlx5_rxq_devx_obj_drop_create(struct rte_eth_dev *dev) rte_errno =ENOMEM ; goto error; } +/* set the CPU socket ID where the rxq_ctrl was allocated */ + rxq_ctrl->socket = socket_id; rxq_obj->rxq_ctrl = rxq_ctrl; rxq_ctrl->is_hairpin = false;
rxq_ctrl->sh=priv->sh;
--
2.27.0
RE: [PATCH v2] net/mlx5: fix CPU socket ID for Rx queue creation
Hi,
Quoted 8 lines
-----Original Message-----
From: Slava Ovsiienko <[email protected]>
Sent: Thursday, March 10, 2022 11:23 AM
To: Thinh Tran <[email protected]>; [email protected]
Cc: [email protected]; Raslan Darawsheh <[email protected]>; Dmitry Kozlyuk <[email protected]>; [email protected]
Subject: RE: [PATCH v2] net/mlx5: fix CPU socket ID for Rx queue creation
Quoted 4 lines
Hi, Thinh
the line: "v2: change patch description, no code change" should not be in commit message body, it should be after "---"
I'll handle this before integration,
It's also missing the Ack from Matan from the previous version, I'll add it as well.
Quoted 6 lines
Besides this:
With best regards,
Slava
Kindest regards
Raslan Darawsheh
RE: [PATCH v2] net/mlx5: fix CPU socket ID for Rx queue creation
In reply to Thinh TranHi,
Quoted 14 lines
-----Original Message-----
From: Thinh Tran <[email protected]>
Sent: Wednesday, March 9, 2022 9:49 PM
To: [email protected]
Cc: [email protected]; Slava Ovsiienko <[email protected]>; Raslan Darawsheh <[email protected]>; Dmitry Kozlyuk <[email protected]>; Thinh Tran <[email protected]>; [email protected]
Subject: [PATCH v2] net/mlx5: fix CPU socket ID for Rx queue creationThe default CPU socket ID was used while creating the Rx queue and this caused creation failure in case if hardware was not resided on the default socket.
Quoted 5 lines
The patch sets the correct CPU socket ID for the
mlx5_rxq_ctrlbefore calling themlx5_rxq_create_devx_rq_resources()which eventually callsmlx5_devx_rq_create()with correct CPU socket ID.v2: change patch description, no code change
Removed this part,
Quoted 7 lines
Fixes bc5bee028ebc ("net/mlx5: create drop queue using DevX")Signed-off-by Thinh Tran <[email protected]>Reviewed-by David Christensen <[email protected]>
Added missing Ack from V1 from Matan,
Patch applied to next-net-mlx,
Kindest regards,
Raslan Darawsheh