NUMA node/socket
There are some mix between NUMA node and socket IDs in DPDK. Examples:
* rte_eth_dev_socket_id() returns the NUMA node.
* rte_malloc use sockets to allocate the memory
Is it critical?
04/07/2021 03:38, Thomas Monjalon:
Quoted 6 lines
There are some mix between NUMA node and socket IDs in DPDK. Examples:
*rte_eth_dev_socket_id()returns the NUMA node.
*rte_mallocuse sockets to allocate the memoryIs it critical?
There is a function, implemented for Windows only, which distinguishes clearly node and socket but it assumes there is only 1 node per socket:
unsigned inteal_socket_numa_node(unsigned int socket_id)
{
return cpu_map.sockets[socket_id].node_id;
}Reminder: AMD can be configured to have multiple nodes per socket.
Quoted 12 lines · Thomas Monjalon
04/07/2021 03:38, Thomas Monjalon:
Quoted 6 lines
There are some mix between NUMA node and socket IDs in DPDK. Examples:
*rte_eth_dev_socket_id()returns the NUMA node.
*rte_mallocuse sockets to allocate the memoryIs it critical?
There is a function, implemented for Windows only, which distinguishes clearly node and socket but it assumes there is only 1 node per socket:
Quoted 7 lines
unsigned int
eal_socket_numa_node(unsigned int socket_id){ return cpu_map.sockets[socket_id].node_id; }Reminder: AMD can be configured to have multiple nodes per socket.
Taking the reminder into account the topic definitely requires improvements.
I apologize for my ignorance, but
Is socket ID really interesting to anybody in DPDK? If no, we should just clarify terminology and switch to NUMA node everywhere.
04/07/2021 10:27, Andrew Rybchenko:
Quoted 19 lines · Thomas Monjalon
Quoted 18 lines
04/07/2021 03:38, Thomas Monjalon:
Quoted 6 lines
There are some mix between NUMA node and socket IDs in DPDK. Examples:
*rte_eth_dev_socket_id()returns the NUMA node.
*rte_mallocuse sockets to allocate the memoryIs it critical?
There is a function, implemented for Windows only, which distinguishes clearly node and socket but it assumes there is only 1 node per socket:
unsigned int
eal_socket_numa_node(unsigned int socket_id){ return cpu_map.sockets[socket_id].node_id; }
Quoted 7 lines
Quoted 1 line
Reminder: AMD can be configured to have multiple nodes per socket.
Taking the reminder into account the topic definitely requires improvements.
I apologize for my ignorance, but
Is socket ID really interesting to anybody in DPDK?
I think the socket ID might be interesting for the threads, but not for memory or devices.
Quoted 2 lines
If no, we should just clarify terminology and switch to NUMA node everywhere.
I have the same opinion as Andrew.
If socket ID is required, it could be deduced from the NUMA node or from the CPU core.
Quoted 31 lines · On Sun, 04 Jul 2021 11:28:23 +0200 Thomas Monjalon <[email protected]> wrote:
04/07/2021 10:27, Andrew Rybchenko:
Quoted 19 lines · Thomas Monjalon
Quoted 18 lines
04/07/2021 03:38, Thomas Monjalon:
Quoted 6 lines
There are some mix between NUMA node and socket IDs in DPDK. Examples:
*rte_eth_dev_socket_id()returns the NUMA node.
*rte_mallocuse sockets to allocate the memoryIs it critical?
There is a function, implemented for Windows only, which distinguishes clearly node and socket but it assumes there is only 1 node per socket:
unsigned int
eal_socket_numa_node(unsigned int socket_id){ return cpu_map.sockets[socket_id].node_id; }Quoted 7 lines
Quoted 1 line
Reminder: AMD can be configured to have multiple nodes per socket.
Taking the reminder into account the topic definitely requires improvements.
I apologize for my ignorance, but
Is socket ID really interesting to anybody in DPDK?I think the socket ID might be interesting for the threads, but not for memory or devices.
Quoted 8 lines
Quoted 2 lines
If no, we should just clarify terminology and switch to NUMA node everywhere.
I have the same opinion as Andrew.
If socket ID is required, it could be deduced from the NUMA node or from the CPU core.
Agree that NUMA node should be used everywhere.
There can be some confusion because some architectures define NUMA for lcore's that have slower interconnect even if they are on the same chip. Also Dell used to have different terminology for socket number on some machines. The socket id reported by BIOS started with 1!
2021-07-04 11:28 (UTC+0200), Thomas Monjalon:
Quoted 31 lines
04/07/2021 10:27, Andrew Rybchenko:
Quoted 19 lines · Thomas Monjalon
Quoted 18 lines
04/07/2021 03:38, Thomas Monjalon:
Quoted 6 lines
There are some mix between NUMA node and socket IDs in DPDK. Examples:
*rte_eth_dev_socket_id()returns the NUMA node.
*rte_mallocuse sockets to allocate the memoryIs it critical?
There is a function, implemented for Windows only, which distinguishes clearly node and socket but it assumes there is only 1 node per socket:
unsigned int
eal_socket_numa_node(unsigned int socket_id){ return cpu_map.sockets[socket_id].node_id; }Quoted 7 lines
Quoted 1 line
Reminder: AMD can be configured to have multiple nodes per socket.
Taking the reminder into account the topic definitely requires improvements.
I apologize for my ignorance, but
Is socket ID really interesting to anybody in DPDK?I think the socket ID might be interesting for the threads, but not for memory or devices.
Quoted 6 lines
Quoted 2 lines
If no, we should just clarify terminology and switch to NUMA node everywhere.
I have the same opinion as Andrew.
If socket ID is required, it could be deduced from the NUMA node or from the CPU core.
I agree with renaming too.
Everywhere in DPDK "socket ID" really means "NUMA node".
I don't see how exactly socket ID can be deduced from NUMA node or CPU core (assuming rte_socket_id becomes rte_numa_node_id), but I also can't imagine why an app would need it. EAL could use NUMA distance info for better memory
management: currently SOCKET_ID_ANY means "current NUMA node or the first one with enough memory available" while it could be "or the closest one".