drivers/common/mlx5: mlx5_malloc() called on invalid socket ID when global MR cache is full and rte_extmem_* API is used
https://bugs.dpdk.org/show_bug.cgi?id=1256
Bug ID: 1256
Summary: drivers/common/mlx5: mlx5_malloc() called on invalid
socket ID when global MR cache is full and rte_extmem_* API is used
Product: DPDK
Version: 21.11
Hardware: x86
OS: Linux
Status: UNCONFIRMED
Severity: normal
Priority: Normal
Component: other
Assignee: [email protected]
Reporter: [email protected]
Target Milestone: ---
Overview:
Attempt to allocate a new mlx5 MR entry when global Btree cache is full ends up calling mlx5_malloc with the EXTERNAL_HEAP_MIN_SOCKET_ID socket ID, given that no external-memory heap has been created. Instead the rte_extmem_* API is used.
Steps to reproduce:
- start a primary DPDK process, on a NIC compatible with mlx5_core driver;
- use rte_extmem_register() to register >512 pages of 4KB;
- use rte_dev_dma_map() to dma-map each page;
- rte_eth_tx_burst() an mbuf with an external buffer from the last page of the registered memory (or a page above index 512). (a virtual address that will not be found in the global Btree cache);
Actual results:mlx5_malloc in mlx5_mr_create_primary() fails with "Unable to allocate memory for a new MR". From this point forward, packets never reach the other end.
Expected results:
MR entries should be successfully retrieved from backup or created when cache becomes full; calling mlx5_malloc() on external heap socket should not be possible when rte_extmem_* API is used. As it is stated in the DPDK documentation[1], "Memory added this way will not be available for any regular DPDK allocators".
Build Date & Hardware:
20 Jun 2023 on Debian GNU/Linux 4.18.0
[1]: https://doc.dpdk.org/guides-21.11/prog_guide/env_abstraction_layer.html
Signature · 2 lines
You are receiving this mail because: You are the assignee for the bug.
https://bugs.dpdk.org/show_bug.cgi?id=1256
Raslan Darawsheh ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |RESOLVED
Resolution|--- |FIXED--- Comment #2 from Raslan Darawsheh ([email protected]) ---
https://git.dpdk.org/dpdk/commit/?h=releases&id=147f6fb42bd7637b37a9180b0774275531c05f9b
Signature · 2 lines
You are receiving this mail because: You are the assignee for the bug.
https://bugs.dpdk.org/show_bug.cgi?id=1256
Marius-Cristian Baciu ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|FIXED |---
Status|RESOLVED |UNCONFIRMED--- Comment #3 from Marius-Cristian Baciu ([email protected]) --- Hi,
Unfortunately that patch only targets a memory socket issue with the ASO mechanism. However, in my setup ASO is never an issue - I actually do not believe it is enabled.
To give a little more insight, the problem I am describing manifests on the data path:
- rte_eth_tx_burst();
- mlx5_tx_burst_*() is called;
- at some later point, in mr_lookup_caches(), mr_btree_lookup() returns UINT32_MAX because all 256 entries in the cache have been occupied and last memory registration did not catch an empty slot;
- when mr_lookup_caches() fails, mlx5_mr_create() -> mlx5_mr_create_primary() is called;
- mlx5_malloc() at line 723 fails because it is called with an inappropriate socket ID (the socket ID of the memseg list associated with an external buffer (prior with rte_extmem_register()), EXTERNAL_HEAP_MIN_SOCKET_ID, which does not actually have a valid heap associated, from which memory could be allocated.
Signature · 2 lines
You are receiving this mail because: You are the assignee for the bug.
http://bugs.dpdk.org/show_bug.cgi?id=1256
[email protected] ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |RESOLVED
Resolution|--- |WONTFIX--- Comment #5 from [email protected] ([email protected]) --- closing
Signature · 2 lines
You are receiving this mail because: You are the assignee for the bug.