drm/amdgpu: Fix topology device creation and proximity domain mappings for sparse and CPU-less NUMA systems
Subsystems (3)
cover drm/amdgpu: Fix topology device creation and proximity domain mappings for sparse and CPU-less NUMA systems
This series fixes topology device creation and proximity domain mappings in AMDKFD when a system does not provide a CRAT table and the driver generates a Virtual CRAT (VCRAT).
The current implementation assumes that CPU NUMA node IDs are contiguous and that every NUMA node contains CPUs. During VCRAT generation, CPU topology entries and proximity domains are created only for NUMA nodes that have CPUs. GPU proximity domains are then allocated immediately after the CPU proximity domains, and the GPU I/O link (proximity_domain_to) is initialized using the NUMA node ID to which the GPU is attached, implicitly assuming that NUMA node IDs and proximity domains have a one-to-one mapping.
These assumptions break on systems with:
Sparse (non-contiguous) NUMA node IDs
CPU-less NUMA nodes
CPU-less and memory-less NUMA nodes
For example:
available: 3 nodes (0,2-3)
node 0: CPUs present
node 2: CPU-less
node 3: CPU-less
In this case, the driver creates a CPU topology device only for node 0 and assigns a single CPU proximity domain (0). However, the GPU VCRAT still references the NUMA node ID to which the GPU is attached (for example, node 3) in the proximity_domain_to field. Since no corresponding CPU proximity domain exists for node 3, the parser cannot find a matching proximity domain during VCRAT parsing, causing topology initialization to fail.
The failure is observed as:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD node
Since every online NUMA node is a valid topology object and can contain CPUs, memory, I/O links, or any combination of these, topology devices and proximity domains should be created for every online NUMA node rather than only for NUMA nodes that contain CPUs.
To address this, this series introduces a new VCRAT subtype that records the mapping between the NUMA node ID and the generated VCRAT proximity domain. When topology devices are created, this information is stored in the corresponding topology device, allowing the driver to translate a NUMA node ID into its associated proximity domain whenever required.
Returning to the previous example, the system contains three online NUMA nodes, so three CPU topology devices and three proximity domains are created, even though only one NUMA node contains CPUs. The NUMA node ID is stored in each topology device together with its generated proximity domain.
Later, when the GPU VCRAT is generated, the driver only knows the NUMA node ID to which the GPU is attached (for example, node 3). Instead of assuming that the NUMA node ID is equal to the proximity domain, the driver walks the existing topology devices to locate the corresponding NUMA node and retrieves its generated proximity domain. In this example, NUMA node 3 maps to proximity domain 2, so proximity_domain_to is populated with the correct value.
Since the GPU I/O link now references a valid proximity domain, VCRAT parsing completes successfully and topology initialization proceeds without errors on systems with sparse NUMA node IDs, CPU-less NUMA nodes, and CPU-less/memory-less NUMA nodes.
This series consists of the following patches:
Patch 1 removes an unused argument from kfd_create_crat_image_virtual() as a preparatory cleanup.
Patch 2 introduces a new VCRAT NUMA affinity subtype that stores the NUMA node ID and its corresponding proximity domain.
Patch 3 populates the NUMA affinity entries during VCRAT generation for every online NUMA node.
Patch 4 parses the NUMA affinity entries from the VCRAT and stores the NUMA node ID and proximity domain in the corresponding topology device.
Patch 5 fixes GPU VCRAT proximity domain mappings by translating the GPU's NUMA node ID to the corresponding CPU proximity domain before programming proximity_domain_to.
Patch 6 creates proximity domains and VCRAT entries for all online NUMA nodes, including CPU-less and memory-less nodes.
Patch 7 adds a numa_node sysfs attribute for CPU and GPU topology devices, exposing the associated NUMA node through the topology sysfs interface.
Please note that the changes in this series are on a best effort basis from our end. Therefore, requesting the amd-gfx community (who have deeper knowledge of the HW & SW stack) to kindly help with the review and provide feedback / comments on these patches
Donet Tom (7):
drm/amdgpu: Remove unused argument from kfd_create_crat_image_virtual drm/amdgpu: Add VCRAT NUMA affinity entry
drm/amdgpu: Populate NUMA affinity entries in VCRAT drm/amdgpu: Parse NUMA affinity entries from VCRAT drm/amdgpu: Fix VCRAT proximity domain mappings for GPU nodes drm/amdgpu: Create proximity domains and VCRAT entries for CPU-less
and memory-less NUMA nodes
drm/amdgpu: Add numa_node in cpu/gpu topology device sysfs entry
drivers/gpu/drm/amd/amdkfd/kfd_crat.c | 152 +++++++++++++++------- drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 19 ++- drivers/gpu/drm/amd/amdkfd/kfd_topology.c | 49 +++++-- drivers/gpu/drm/amd/amdkfd/kfd_topology.h | 4 + 4 files changed, 168 insertions(+), 56 deletions(-)
Signature · 1 line
2.54.0
PATCH 1/7 drm/amdgpu: Remove unused argument from kfd_create_crat_image_virtual
The size argument is neither passed to nor returned from kfd_create_crat_image_virtual(). Since it is unused, remove it from the function and update the callers accordingly.
--- drivers/gpu/drm/amd/amdkfd/kfd_crat.c | 11 +++++------ drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 4 ++-- drivers/gpu/drm/amd/amdkfd/kfd_topology.c | 12 ++++-------- 3 files changed, 11 insertions(+), 16 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_crat.c +5 -6
@@ -2412,8 +2412,8 @@ static int kfd_create_vcrat_image_gpu(void *pcrat_image,* * Return 0 if successful else return -ve value */ -int kfd_create_crat_image_virtual (void **crat_image,size_t *size, -int flags,struct kfd_node *kdev, +int kfd_create_crat_image_virtual (void **crat_image,int flags, +struct kfd_node *kdev,uint32_t proximity_domain) {void *pcrat_image =NULL ;
@@ -2440,9 +2440,8 @@ int kfd_create_crat_image_virtual(void **crat_image, size_t *size, pcrat_image =kvmalloc (dyn_size,GFP_KERNEL );if (!pcrat_image)return -ENOMEM ; - *size = dyn_size;pr_debug ("CRAT size is %ld" , dyn_size); - ret =kfd_create_vcrat_image_cpu (pcrat_image, size); + ret =kfd_create_vcrat_image_cpu (pcrat_image, &dyn_size);break ;case COMPUTE_UNIT_GPU :if (!kdev)
@@ -2450,8 +2449,8 @@ int kfd_create_crat_image_virtual(void **crat_image, size_t *size, pcrat_image =kvmalloc (VCRAT_SIZE_FOR_GPU ,GFP_KERNEL );if (!pcrat_image)return -ENOMEM ; - *size =VCRAT_SIZE_FOR_GPU ; - ret =kfd_create_vcrat_image_gpu (pcrat_image, size, kdev, + dyn_size =VCRAT_SIZE_FOR_GPU ; + ret =kfd_create_vcrat_image_gpu (pcrat_image, &dyn_size, kdev, proximity_domain);break ;case (COMPUTE_UNIT_CPU |COMPUTE_UNIT_GPU ):
drivers/gpu/drm/amd/amdkfd/kfd_crat.h +2 -2
@@ -313,8 +313,8 @@ int kfd_get_gpu_cache_info(struct kfd_node *kdev, struct kfd_gpu_cache_info **pcvoid kfd_destroy_crat_image (void *crat_image);int kfd_parse_crat_table (void *crat_image,struct list_head *device_list,uint32_t proximity_domain); -int kfd_create_crat_image_virtual (void **crat_image,size_t *size, -int flags,struct kfd_node *kdev, +int kfd_create_crat_image_virtual (void **crat_image,int flags, +struct kfd_node *kdev,uint32_t proximity_domain);#endif /* KFD_CRAT_H_INCLUDED */
drivers/gpu/drm/amd/amdkfd/kfd_topology.c +4 -8
@@ -996,7 +996,6 @@ static void kfd_add_non_crat_information(struct kfd_topology_device *kdev)int kfd_topology_init (void ) {void *crat_image =NULL ; -size_t image_size =0 ;int ret;struct list_head temp_topology_device_list;int cpu_only_node =0 ;
@@ -1023,9 +1022,8 @@ int kfd_topology_init(void)*/ proximity_domain =0 ; - ret =kfd_create_crat_image_virtual (&crat_image, &image_size, -COMPUTE_UNIT_CPU ,NULL , - proximity_domain); + ret =kfd_create_crat_image_virtual (&crat_image,COMPUTE_UNIT_CPU , +NULL , proximity_domain); cpu_only_node =1 ;if (ret) {pr_err ("Error creating VCRAT table for CPU\n" );
@@ -1871,12 +1869,10 @@ static int kfd_topology_add_device_locked(struct kfd_node *gpu,int proximity_domain = ++topology_crat_proximity_domain;struct list_head temp_topology_device_list;void *crat_image =NULL ; -size_t image_size =0 ;int res; - res =kfd_create_crat_image_virtual (&crat_image, &image_size, -COMPUTE_UNIT_GPU , gpu, - proximity_domain); + res =kfd_create_crat_image_virtual (&crat_image,COMPUTE_UNIT_GPU , + gpu, proximity_domain);if (res) {dev_err (gpu->adev->dev,"Error creating VCRAT\n" ); topology_crat_proximity_domain--;
Signature · 1 line
2.54.0
PATCH 2/7 drm/amdgpu: Add VCRAT NUMA affinity entry
In reply to Donet TomAdd a new VCRAT subtype to describe NUMA affinity information. The new entry stores the NUMA node ID and proximity domain.
--- drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdkfd/kfd_crat.h +14 -1
@@ -75,7 +75,8 @@ struct crat_header {#define CRAT_SUBTYPE_TLB_AFFINITY 3#define CRAT_SUBTYPE_CCOMPUTE_AFFINITY 4#define CRAT_SUBTYPE_IOLINK_AFFINITY 5 -#define CRAT_SUBTYPE_MAX 6 +#define CRAT_SUBTYPE_NUMA_AFFINITY 6 +#define CRAT_SUBTYPE_MAX 7/* * Do not change the value of CRAT_SIBLINGMAP_SIZE from 32
@@ -280,6 +281,18 @@ struct crat_subtype_iolink {uint8_t weight_xgmi; }; +/* +* HSA NUMA sub-type header +*/ +struct crat_subtype_numa { +uint8_t type; +uint8_t length; +uint16_t reserved; +uint32_t flags; +uint32_t numa_node; +uint32_t proximity_domain; +}; +/* * HSA generic sub-type header */
Signature · 1 line
2.54.0
PATCH 3/7 drm/amdgpu: Populate NUMA affinity entries in VCRAT
In reply to Donet TomPopulate the new VCRAT NUMA affinity subtype with the NUMA node ID and the corresponding proximity domain of each NUMA node.
--- drivers/gpu/drm/amd/amdkfd/kfd_crat.c | 32 +++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)
drivers/gpu/drm/amd/amdkfd/kfd_crat.c +32 -0
@@ -1877,6 +1877,26 @@ static int kfd_fill_iolink_info_for_cpu(int numa_node_id, int *avail_size, }#endif +static int kfd_fill_numa_info_for_cpu (int *avail_size, +struct crat_subtype_numa *sub_type_hdr, +int numa_node,int proximity_domain) +{ + *avail_size -=sizeof (struct crat_subtype_numa ); +if (*avail_size <0 ) +return -ENOMEM ; + +memset (sub_type_hdr,0 ,sizeof (struct crat_subtype_numa )); + +/* Fill in subtype header data */ + sub_type_hdr->type =CRAT_SUBTYPE_NUMA_AFFINITY ; + sub_type_hdr->length =sizeof (struct crat_subtype_numa ); + sub_type_hdr->flags =CRAT_SUBTYPE_FLAGS_ENABLED ; + sub_type_hdr->numa_node = numa_node; + sub_type_hdr->proximity_domain = proximity_domain; + +return 0 ; +} +/* kfd_create_vcrat_image_cpu - Create Virtual CRAT for CPU * * @pcrat_image: Fill in VCRAT for CPU
@@ -1974,6 +1994,17 @@ static int kfd_create_vcrat_image_cpu(void *pcrat_image, size_t *size)pr_info ("IO link not available for non x86 platforms\n" );#endif + ret =kfd_fill_numa_info_for_cpu (&avail_size, + (struct crat_subtype_numa *)sub_type_hdr, + numa_node_id, crat_table->num_domains); +if (ret <0 ) +return ret; + crat_table->length += sub_type_hdr->length; + crat_table->total_entries++; + + sub_type_hdr = (typeof (sub_type_hdr))((char *)sub_type_hdr + + sub_type_hdr->length); + crat_table->num_domains++; }
@@ -2436,6 +2467,7 @@ int kfd_create_crat_image_virtual(void **crat_image, int flags, dyn_size =sizeof (struct crat_header ) + num_nodes * (sizeof (struct crat_subtype_computeunit ) +sizeof (struct crat_subtype_memory ) + +sizeof (struct crat_subtype_numa ) + (num_nodes -1 ) *sizeof (struct crat_subtype_iolink )); pcrat_image =kvmalloc (dyn_size,GFP_KERNEL );if (!pcrat_image)
Signature · 1 line
2.54.0
PATCH 4/7 drm/amdgpu: Parse NUMA affinity entries from VCRAT
In reply to Donet TomParse the NUMA affinity entries from the VCRAT and populate the corresponding NUMA node information in the topology device.
--- drivers/gpu/drm/amd/amdkfd/kfd_crat.c | 23 +++++++++++++++++++++++ drivers/gpu/drm/amd/amdkfd/kfd_topology.h | 1 + 2 files changed, 24 insertions(+)
drivers/gpu/drm/amd/amdkfd/kfd_crat.c +23 -0
@@ -1299,6 +1299,23 @@ static int kfd_parse_subtype_iolink(struct crat_subtype_iolink *iolink,return 0 ; } +static int kfd_parse_subtype_numa (struct crat_subtype_numa *numa, +struct list_head *device_list) +{ +struct kfd_topology_device *dev; + +pr_debug ("Found NUMA entry in CRAT table with proximity_domain=%u numa node=%u\n" , + numa->proximity_domain, numa->numa_node); +list_for_each_entry (dev, device_list, list) { +if (numa->proximity_domain == dev->proximity_domain) { + dev->numa_node = numa->numa_node; +break ; + } + } + +return 0 ; +} +/* kfd_parse_subtype - parse subtypes and attach it to correct topology device * present in the device_list * @sub_type_hdr - subtype section of crat_image
@@ -1311,6 +1328,7 @@ static int kfd_parse_subtype(struct crat_subtype_generic *sub_type_hdr,struct crat_subtype_memory *mem;struct crat_subtype_cache *cache;struct crat_subtype_iolink *iolink; +struct crat_subtype_numa *numa;int ret =0 ;switch (sub_type_hdr->type) {
@@ -1342,6 +1360,10 @@ static int kfd_parse_subtype(struct crat_subtype_generic *sub_type_hdr, iolink = (struct crat_subtype_iolink *)sub_type_hdr; ret =kfd_parse_subtype_iolink (iolink, device_list);break ; +case CRAT_SUBTYPE_NUMA_AFFINITY : + numa = (struct crat_subtype_numa *)sub_type_hdr; + ret =kfd_parse_subtype_numa (numa, device_list); +break ;default :pr_warn ("Unknown subtype %d in CRAT\n" , sub_type_hdr->type);
@@ -1389,6 +1411,7 @@ int kfd_parse_crat_table(void *crat_image, struct list_head *device_list,if (!top_dev)break ; top_dev->proximity_domain = proximity_domain++; + top_dev->numa_node =-1 ; }if (!top_dev) {
drivers/gpu/drm/amd/amdkfd/kfd_topology.h +1 -0
@@ -144,6 +144,7 @@ struct kfd_perf_properties {struct kfd_topology_device {struct list_head list;uint32_t gpu_id; +uint32_t numa_node;uint32_t proximity_domain;struct kfd_node_properties node_props;struct list_head mem_props;
Signature · 1 line
2.54.0
PATCH 5/7 drm/amdgpu: Fix VCRAT proximity domain mappings for GPU nodes
In reply to Donet TomThe VCRAT CPU builder, kfd_create_vcrat_image_cpu(), assigns proximity domains sequentially while iterating over for_each_online_node(). As a result, the first online node is assigned proximity domain 0, the second online node is assigned proximity domain 1, and so on.
The GPU VCRAT builder creates GPU proximity domains after the CPU proximity domains and programs the GPU-to-CPU affinity (proximity_domain_to) using the NUMA node ID of the node to which the GPU is attached. This is only correct when NUMA node IDs are contiguous, because the CPU proximity domains happen to match the NUMA node IDs.
On systems with discontiguous NUMA node IDs, the CPU proximity domains no longer correspond to the NUMA node IDs. For example:
available: 3 nodes (0,2,4)
Node IDs: 0 2 4 CPU proximity: 0 1 2 GPU proximity: 3
In this example, a GPU attached to NUMA node 4 is assigned GPU proximity domain 3. However, its proximity_domain_to field is programmed with the NUMA node ID (4) instead of the corresponding CPU proximity domain (2).
During VCRAT parsing, no CPU proximity domain with value 4 exists, so the GPU affinity cannot be resolved and topology initialization fails:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD node
Fix this by translating the GPU's NUMA node ID to the corresponding CPU proximity domain assigned during GPU VCRAT creation before programming proximity_domain_to. This ensures that GPU affinity entries always reference a valid CPU proximity domain, allowing VCRAT parsing and KFD topology initialization to succeed on systems with discontiguous NUMA node IDs.
--- drivers/gpu/drm/amd/amdkfd/kfd_crat.c | 10 +++++++--- drivers/gpu/drm/amd/amdkfd/kfd_topology.c | 22 ++++++++++++++++++++++ drivers/gpu/drm/amd/amdkfd/kfd_topology.h | 2 ++ 3 files changed, 31 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_crat.c +7 -3
@@ -1267,6 +1267,7 @@ static int kfd_parse_subtype_iolink(struct crat_subtype_iolink *iolink, iolink->recommended_transfer_size; dev->node_props.io_links_count++; + dev->numa_node =kfd_proximity_domain_to_numa_node (id_to);list_add_tail (&props->list, &dev->io_link_props);break ; }
@@ -1888,8 +1889,10 @@ static int kfd_fill_iolink_info_for_cpu(int numa_node_id, int *avail_size, sub_type_hdr->flags =CRAT_SUBTYPE_FLAGS_ENABLED ;/* Fill in IO link data */ - sub_type_hdr->proximity_domain_from = numa_node_id; - sub_type_hdr->proximity_domain_to = nid; + sub_type_hdr->proximity_domain_from = +kfd_numa_node_to_proximity_domain (numa_node_id); + sub_type_hdr->proximity_domain_to = +kfd_numa_node_to_proximity_domain (nid); sub_type_hdr->io_interface_type = link_type; (*num_entries)++;
@@ -2237,7 +2240,8 @@ static int kfd_fill_gpu_direct_io_link_to_cpu(int *avail_size,if (kdev->adev->pdev->dev.numa_node ==NUMA_NO_NODE ) sub_type_hdr->proximity_domain_to =0 ;else - sub_type_hdr->proximity_domain_to = kdev->adev->pdev->dev.numa_node; + sub_type_hdr->proximity_domain_to = +kfd_numa_node_to_proximity_domain (kdev->adev->pdev->dev.numa_node);#else sub_type_hdr->proximity_domain_to =0 ;#endif
drivers/gpu/drm/amd/amdkfd/kfd_topology.c +22 -0
@@ -2463,3 +2463,25 @@ void kfd_update_svm_support_properties(struct amdgpu_device *adev) }up_write (&topology_lock); } + +int kfd_proximity_domain_to_numa_node (uint32_t proximity_domain) +{ +struct kfd_topology_device *dev; + +list_for_each_entry (dev, &topology_device_list, list) { +if (dev->proximity_domain == proximity_domain) +return dev->numa_node; + } +return -1 ; +} + +int kfd_numa_node_to_proximity_domain (int target_node) +{ +struct kfd_topology_device *dev; + +list_for_each_entry (dev, &topology_device_list, list) { +if (dev->numa_node == target_node) +return dev->proximity_domain; + } +return -1 ; +}
drivers/gpu/drm/amd/amdkfd/kfd_topology.h +2 -0
@@ -201,6 +201,8 @@ struct dmi_mem_device {struct kfd_topology_device *kfd_create_topology_device (struct list_head *device_list);void kfd_release_topology_device_list (struct list_head *device_list); +int kfd_numa_node_to_proximity_domain (int target_node); +int kfd_proximity_domain_to_numa_node (uint32_t proximity_domain);#if IS_ENABLED (CONFIG_HSA_AMD )void kfd_update_svm_support_properties (struct amdgpu_device *adev);
Signature · 1 line
2.54.0
PATCH 6/7 drm/amdgpu: Create proximity domains and VCRAT entries for CPU-less and memory-less NUMA nodes
In reply to Donet TomThe current implementation creates proximity domains and the associated VCRAT entries only for NUMA nodes that contain CPUs. However, Linux systems can have CPU-less, memory-less, or CPU-less and memory-less NUMA nodes.
When proximity domains and VCRAT entries are not created for these NUMA nodes, the generated VCRAT is incomplete and KFD topology initialization fails while parsing it.
For example:
available: 3 nodes (0,2-3)
node 0 cpus: 0-39
node 0 size: 519305 MB
node 2 cpus:
node 2 size: 519305 MB
node 3 cpus:
node 3 size:
With the current implementation, only node 0 is assigned a CPU proximity domain (domain 0). A GPU attached to NUMA node 3 is assigned GPU proximity domain 1, and its proximity_domain_to field references the CPU proximity domain corresponding to node 3. Since no CPU proximity domain exists for node 3, the referenced proximity domain is missing, causing VCRAT parsing to fail.
Fix this by creating proximity domains and the corresponding VCRAT entries for all online NUMA nodes, including CPU-less and memory-less nodes. This ensures that every online NUMA node has a corresponding proximity domain, allowing GPU affinity information to be resolved correctly during VCRAT parsing.
--- drivers/gpu/drm/amd/amdkfd/kfd_crat.c | 78 ++++++++++++++------------- 1 file changed, 41 insertions(+), 37 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_crat.c +41 -37
@@ -1974,51 +1974,55 @@ static int kfd_create_vcrat_image_cpu(void *pcrat_image, size_t *size) sub_type_hdr = (struct crat_subtype_generic *)(crat_table+1 );for_each_online_node (numa_node_id) { -if (kfd_numa_node_to_apic_id (numa_node_id) ==-1 ) -continue ; - -/* Fill in Subtype: Compute Unit */ - ret =kfd_fill_cu_for_cpu (numa_node_id, &avail_size, - crat_table->num_domains, - (struct crat_subtype_computeunit *)sub_type_hdr); -if (ret <0 ) -return ret; - crat_table->length += sub_type_hdr->length; - crat_table->total_entries++; +int apic_id =kfd_numa_node_to_apic_id (numa_node_id); +int mem =node_state (numa_node_id,N_MEMORY ); + +if (apic_id !=-1 ) { +/* Fill in Subtype: Compute Unit */ + ret =kfd_fill_cu_for_cpu (numa_node_id, &avail_size, + crat_table->num_domains, + (struct crat_subtype_computeunit *)sub_type_hdr); +if (ret <0 ) +return ret; + crat_table->length += sub_type_hdr->length; + crat_table->total_entries++; - sub_type_hdr = (typeof (sub_type_hdr))((char *)sub_type_hdr + - sub_type_hdr->length); + sub_type_hdr = (typeof (sub_type_hdr))((char *)sub_type_hdr + + sub_type_hdr->length); -/* Fill in Subtype: Memory */ - ret =kfd_fill_mem_info_for_cpu (numa_node_id, &avail_size, - crat_table->num_domains, - (struct crat_subtype_memory *)sub_type_hdr); -if (ret <0 ) -return ret; - crat_table->length += sub_type_hdr->length; - crat_table->total_entries++; +/* Fill in Subtype: IO Link */ +#if defined(CONFIG_X86_64 ) && !defined(CONFIG_UML ) + ret =kfd_fill_iolink_info_for_cpu (numa_node_id, &avail_size, + &entries, + (struct crat_subtype_iolink *)sub_type_hdr); +if (ret <0 ) +return ret; - sub_type_hdr = (typeof (sub_type_hdr))((char *)sub_type_hdr + - sub_type_hdr->length); +if (entries) { + crat_table->length += (sub_type_hdr->length * entries); + crat_table->total_entries += entries; -/* Fill in Subtype: IO Link */ -#if defined(CONFIG_X86_64 ) && !defined(CONFIG_UML ) - ret =kfd_fill_iolink_info_for_cpu (numa_node_id, &avail_size, - &entries, - (struct crat_subtype_iolink *)sub_type_hdr); -if (ret <0 ) -return ret; + sub_type_hdr = (typeof (sub_type_hdr))((char *)sub_type_hdr + + sub_type_hdr->length * entries); + } +#else +pr_info ("IO link not available for non x86 platforms\n" ); +#endif + } -if (entries) { - crat_table->length += (sub_type_hdr->length * entries); - crat_table->total_entries += entries; +if (mem) { +/* Fill in Subtype: Memory */ + ret =kfd_fill_mem_info_for_cpu (numa_node_id, &avail_size, + crat_table->num_domains, + (struct crat_subtype_memory *)sub_type_hdr); +if (ret <0 ) +return ret; + crat_table->length += sub_type_hdr->length; + crat_table->total_entries++; sub_type_hdr = (typeof (sub_type_hdr))((char *)sub_type_hdr + - sub_type_hdr->length * entries); + sub_type_hdr->length); } -#else -pr_info ("IO link not available for non x86 platforms\n" ); -#endif ret = kfd_fill_numa_info_for_cpu(&avail_size, (struct crat_subtype_numa *)sub_type_hdr,
Signature · 1 line
2.54.0
PATCH 7/7 drm/amdgpu: Add numa_node in cpu/gpu topology device sysfs entry
In reply to Donet TomAdded a numa_node sysfs attribute for CPU and GPU topology devices. This attribute exposed the NUMA node associated with the corresponding topology device through the topology sysfs interface.
--- drivers/gpu/drm/amd/amdkfd/kfd_topology.c | 15 +++++++++++++++ drivers/gpu/drm/amd/amdkfd/kfd_topology.h | 1 + 2 files changed, 16 insertions(+)
drivers/gpu/drm/amd/amdkfd/kfd_topology.c +15 -0
@@ -418,6 +418,14 @@ static ssize_t node_show(struct kobject *kobj, struct attribute *attr,return sysfs_show_32bit_val (buffer, offs, dev->gpu_id); } +if (strcmp (attr->name,"numa_node" ) ==0 ) { + dev = container_of(attr, struct kfd_topology_device, + attr_numa_node); +if (dev->gpu &&kfd_devcgroup_check_permission (dev->gpu)) +return -EPERM ; +return sysfs_show_32bit_val (buffer, offs, dev->numa_node); + } +if (strcmp (attr->name,"name" ) ==0 ) { dev = container_of(attr, struct kfd_topology_device, attr_name);
@@ -630,6 +638,7 @@ static void kfd_remove_sysfs_node_entry(struct kfd_topology_device *dev)if (dev->kobj_node) {sysfs_remove_file (dev->kobj_node, &dev->attr_gpuid); +sysfs_remove_file (dev->kobj_node, &dev->attr_numa_node);sysfs_remove_file (dev->kobj_node, &dev->attr_name);sysfs_remove_file (dev->kobj_node, &dev->attr_props);kobject_del (dev->kobj_node);
@@ -693,6 +702,9 @@ static int kfd_build_sysfs_node_entry(struct kfd_topology_device *dev, dev->attr_gpuid.name ="gpu_id" ; dev->attr_gpuid.mode =KFD_SYSFS_FILE_MODE ;sysfs_attr_init (&dev->attr_gpuid); + dev->attr_numa_node.name ="numa_node" ; + dev->attr_numa_node.mode =KFD_SYSFS_FILE_MODE ; +sysfs_attr_init (&dev->attr_numa_node); dev->attr_name.name ="name" ; dev->attr_name.mode =KFD_SYSFS_FILE_MODE ;sysfs_attr_init (&dev->attr_name);
@@ -700,6 +712,9 @@ static int kfd_build_sysfs_node_entry(struct kfd_topology_device *dev, dev->attr_props.mode =KFD_SYSFS_FILE_MODE ;sysfs_attr_init (&dev->attr_props); ret =sysfs_create_file (dev->kobj_node, &dev->attr_gpuid); +if (ret <0 ) +return ret; + ret =sysfs_create_file (dev->kobj_node, &dev->attr_numa_node);if (ret <0 )return ret; ret =sysfs_create_file (dev->kobj_node, &dev->attr_name);
drivers/gpu/drm/amd/amdkfd/kfd_topology.h +1 -0
@@ -160,6 +160,7 @@ struct kfd_topology_device {struct kobject *kobj_p2plink;struct kobject *kobj_perf;struct attribute attr_gpuid; +struct attribute attr_numa_node;struct attribute attr_name;struct attribute attr_props;union {
Signature · 1 line
2.54.0
Felix Kuehling
maintainerQuoted 26 lines · Donet Tom
This series fixes topology device creation and proximity domain mappings in AMDKFD when a system does not provide a CRAT table and the driver generates a Virtual CRAT (VCRAT).
The current implementation assumes that CPU NUMA node IDs are contiguous and that every NUMA node contains CPUs. During VCRAT generation, CPU topology entries and proximity domains are created only for NUMA nodes that have CPUs. GPU proximity domains are then allocated immediately after the CPU proximity domains, and the GPU I/O link (
proximity_domain_to) is initialized using the NUMA node ID to which the GPU is attached, implicitly assuming that NUMA node IDs and proximity domains have a one-to-one mapping.These assumptions break on systems with:
Sparse (non-contiguous) NUMA node IDs
CPU-less NUMA nodes
CPU-less and memory-less NUMA nodesFor example:
available: 3 nodes (0,2-3)
node 0: CPUs present
node 2: CPU-less
node 3: CPU-less
Quoted 6 lines
In this case, the driver creates a CPU topology device only for node 0 and assigns a single CPU proximity domain (0). However, the GPU VCRAT still references the NUMA node ID to which the GPU is attached (for example, node 3) in the
proximity_domain_tofield. Since no corresponding CPU proximity domain exists for node 3, the parser cannot find a matching proximity domain during VCRAT parsing, causing topology initialization to fail.
Hi Tom,
Thank you for the explanation and the patch series. I may have some gaps in my understanding that I would like to clarify. In my mind, I was using "proximity domain" and "NUMA node" interchangeably. You're demonstrating that they are not the same thing. Is that just a different way of labeling the same thing, or are NUMA nodes and proximity domains fundamentally different concepts.
Your code in patch 5 (kfd_proximity_domain_to_numa_node and kfd_numa_node_to_proximity_domain) seems to imply that there is, in fact, a 1:1 mapping, as it assumes that there is a unique translation in both directions. Am I missing something?
If they are just different numbering systems, do we really need to keep track of both proximity domains and NUMA nodes in the KFD topology? Or would it be sufficient to only track NUMA nodes, if that's what we really care about in the uAPI (KFD sysfs)?
Thanks,
Felix
Quoted 87 lines
The failure is observed as:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeSince every online NUMA node is a valid topology object and can contain CPUs, memory, I/O links, or any combination of these, topology devices and proximity domains should be created for every online NUMA node rather than only for NUMA nodes that contain CPUs.
To address this, this series introduces a new VCRAT subtype that records the mapping between the NUMA node ID and the generated VCRAT proximity domain. When topology devices are created, this information is stored in the corresponding topology device, allowing the driver to translate a NUMA node ID into its associated proximity domain whenever required.
Returning to the previous example, the system contains three online NUMA nodes, so three CPU topology devices and three proximity domains are created, even though only one NUMA node contains CPUs. The NUMA node ID is stored in each topology device together with its generated proximity domain.
Later, when the GPU VCRAT is generated, the driver only knows the NUMA node ID to which the GPU is attached (for example, node 3). Instead of assuming that the NUMA node ID is equal to the proximity domain, the driver walks the existing topology devices to locate the corresponding NUMA node and retrieves its generated proximity domain. In this example, NUMA node 3 maps to proximity domain 2, so
proximity_domain_tois populated with the correct value.Since the GPU I/O link now references a valid proximity domain, VCRAT parsing completes successfully and topology initialization proceeds without errors on systems with sparse NUMA node IDs, CPU-less NUMA nodes, and CPU-less/memory-less NUMA nodes.
This series consists of the following patches:
Patch 1 removes an unused argument from
kfd_create_crat_image_virtual()as a preparatory cleanup.Patch 2 introduces a new VCRAT NUMA affinity subtype that stores the NUMA node ID and its corresponding proximity domain.
Patch 3 populates the NUMA affinity entries during VCRAT generation for every online NUMA node.
Patch 4 parses the NUMA affinity entries from the VCRAT and stores the NUMA node ID and proximity domain in the corresponding topology device.
Patch 5 fixes GPU VCRAT proximity domain mappings by translating the GPU's NUMA node ID to the corresponding CPU proximity domain before programming
proximity_domain_to.Patch 6 creates proximity domains and VCRAT entries for all online NUMA nodes, including CPU-less and memory-less nodes.
Patch 7 adds a
numa_nodesysfs attribute for CPU and GPU topology devices, exposing the associated NUMA node through the topology sysfs interface.Please note that the changes in this series are on a best effort basis from our end. Therefore, requesting the amd-gfx community (who have deeper knowledge of the HW & SW stack) to kindly help with the review and provide feedback / comments on these patches
Donet Tom (7):
drm/amdgpu: Remove unused argument fromkfd_create_crat_image_virtualdrm/amdgpu: Add VCRAT NUMA affinity entry
drm/amdgpu: Populate NUMA affinity entries in VCRAT drm/amdgpu: Parse NUMA affinity entries from VCRAT drm/amdgpu: Fix VCRAT proximity domain mappings for GPU nodes drm/amdgpu: Create proximity domains and VCRAT entries for CPU-less
and memory-less NUMA nodes
drm/amdgpu: Addnuma_nodein cpu/gpu topology device sysfs entrydrivers/gpu/drm/amd/amdkfd/kfd_crat.c | 152 +++++++++++++++------- drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 19 ++- drivers/gpu/drm/amd/amdkfd/kfd_topology.c | 49 +++++-- drivers/gpu/drm/amd/amdkfd/kfd_topology.h | 4 + 4 files changed, 168 insertions(+), 56 deletions(-)
Quoted 28 lines · Felix Kuehling
Quoted 26 lines · Donet Tom
This series fixes topology device creation and proximity domain mappings in AMDKFD when a system does not provide a CRAT table and the driver generates a Virtual CRAT (VCRAT).
The current implementation assumes that CPU NUMA node IDs are contiguous and that every NUMA node contains CPUs. During VCRAT generation, CPU topology entries and proximity domains are created only for NUMA nodes that have CPUs. GPU proximity domains are then allocated immediately after the CPU proximity domains, and the GPU I/O link (
proximity_domain_to) is initialized using the NUMA node ID to which the GPU is attached, implicitly assuming that NUMA node IDs and proximity domains have a one-to-one mapping.These assumptions break on systems with:
Sparse (non-contiguous) NUMA node IDs
CPU-less NUMA nodes
CPU-less and memory-less NUMA nodesFor example:
available: 3 nodes (0,2-3)
node 0: CPUs present
node 2: CPU-less
node 3: CPU-less
Quoted 8 lines
Quoted 6 lines
In this case, the driver creates a CPU topology device only for node 0 and assigns a single CPU proximity domain (0). However, the GPU VCRAT still references the NUMA node ID to which the GPU is attached (for example, node 3) in the
proximity_domain_tofield. Since no corresponding CPU proximity domain exists for node 3, the parser cannot find a matching proximity domain during VCRAT parsing, causing topology initialization to fail.Hi Tom,
Hi Felix,
Quoted 4 lines
Thank you for the explanation and the patch series. I may have some gaps in my understanding that I would like to clarify. In my mind, I was using "proximity domain" and "NUMA node" interchangeably. You're demonstrating that they are not the same thing. Is that just a different way of labeling the same thing, or are NUMA nodes and proximity domains fundamentally different concepts.
Your code in patch 5 (
kfd_proximity_domain_to_numa_nodeandkfd_numa_node_to_proximity_domain) seems to imply that there is, in fact, a 1:1 mapping, as it assumes that there is a unique translation in both directions. Am I missing something?
Thanks for the comment.
IIUC, NUMA node IDs and proximity domains are different numbering schemes. We have proximity domains for both CPU and GPU devices. CPU proximity domains start from 0, and GPU proximity domains start after the last CPU proximity domain.
If the NUMA node IDs are contiguous, the CPU proximity domains happen to match the NUMA node IDs. However, if the NUMA node IDs are discontiguous, the CPU proximity domains and NUMA node IDs no longer have a one-to-one mapping.
Quoted 2 lines
If they are just different numbering systems, do we really need to keep track of both proximity domains and NUMA nodes in the KFD topology? Or would it be sufficient to only track NUMA nodes, if that's what we really care about in the uAPI (KFD sysfs)?
Thanks for the suggestion. I also think we don't need to keep track of both the proximity domains and the NUMA node IDs in KFD. Do you think the approach below would be reasonable?
Just to make sure I understand correctly, for CPU devices the proximity domain will be the same as the NUMA node ID, and the GPU proximity domains will start after the last CPU proximity domain.
In that case, the CPU proximity domains and NUMA node IDs will always have a one-to-one mapping, and the GPU proximity domains will follow after them.
For example, if a system has three NUMA nodes and two GPUs:
NUMA node IDs: 0 2 3
CPU proximity domains: 0 2 3
GPU proximity domains: 4 5
Would it be okay to proceed with this approach?
I think this approach should also resolve the driver loading issue.
Thanks
Donet Tom
Quoted 92 lines
Thanks,
FelixQuoted 87 lines
The failure is observed as:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeSince every online NUMA node is a valid topology object and can contain CPUs, memory, I/O links, or any combination of these, topology devices and proximity domains should be created for every online NUMA node rather than only for NUMA nodes that contain CPUs.
To address this, this series introduces a new VCRAT subtype that records the mapping between the NUMA node ID and the generated VCRAT proximity domain. When topology devices are created, this information is stored in the corresponding topology device, allowing the driver to translate a NUMA node ID into its associated proximity domain whenever required.
Returning to the previous example, the system contains three online NUMA nodes, so three CPU topology devices and three proximity domains are created, even though only one NUMA node contains CPUs. The NUMA node ID is stored in each topology device together with its generated proximity domain.
Later, when the GPU VCRAT is generated, the driver only knows the NUMA node ID to which the GPU is attached (for example, node 3). Instead of assuming that the NUMA node ID is equal to the proximity domain, the driver walks the existing topology devices to locate the corresponding NUMA node and retrieves its generated proximity domain. In this example, NUMA node 3 maps to proximity domain 2, so
proximity_domain_tois populated with the correct value.Since the GPU I/O link now references a valid proximity domain, VCRAT parsing completes successfully and topology initialization proceeds without errors on systems with sparse NUMA node IDs, CPU-less NUMA nodes, and CPU-less/memory-less NUMA nodes.
This series consists of the following patches:
Patch 1 removes an unused argument from
kfd_create_crat_image_virtual()as a preparatory cleanup.Patch 2 introduces a new VCRAT NUMA affinity subtype that stores the NUMA node ID and its corresponding proximity domain.
Patch 3 populates the NUMA affinity entries during VCRAT generation for every online NUMA node.
Patch 4 parses the NUMA affinity entries from the VCRAT and stores the NUMA node ID and proximity domain in the corresponding topology device.
Patch 5 fixes GPU VCRAT proximity domain mappings by translating the GPU's NUMA node ID to the corresponding CPU proximity domain before programming
proximity_domain_to.Patch 6 creates proximity domains and VCRAT entries for all online NUMA nodes, including CPU-less and memory-less nodes.
Patch 7 adds a
numa_nodesysfs attribute for CPU and GPU topology devices, exposing the associated NUMA node through the topology sysfs interface.Please note that the changes in this series are on a best effort basis from our end. Therefore, requesting the amd-gfx community (who have deeper knowledge of the HW & SW stack) to kindly help with the review and provide feedback / comments on these patches
Donet Tom (7):
drm/amdgpu: Remove unused argument fromkfd_create_crat_image_virtualdrm/amdgpu: Add VCRAT NUMA affinity entry
drm/amdgpu: Populate NUMA affinity entries in VCRAT drm/amdgpu: Parse NUMA affinity entries from VCRAT drm/amdgpu: Fix VCRAT proximity domain mappings for GPU nodes drm/amdgpu: Create proximity domains and VCRAT entries for CPU-less and memory-less NUMA nodes
drm/amdgpu: Addnuma_nodein cpu/gpu topology device sysfs entrydrivers/gpu/drm/amd/amdkfd/kfd_crat.c | 152 +++++++++++++++------- drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 19 ++-
drivers/gpu/drm/amd/amdkfd/kfd_topology.c| 49 +++++--drivers/gpu/drm/amd/amdkfd/kfd_topology.h| 4 +4 files changed, 168 insertions(+), 56 deletions(-)
Kuehling, Felix
maintainerQuoted 67 lines · Donet Tom
Quoted 28 lines · Felix Kuehling
Quoted 26 lines · Donet Tom
This series fixes topology device creation and proximity domain mappings in AMDKFD when a system does not provide a CRAT table and the driver generates a Virtual CRAT (VCRAT).
The current implementation assumes that CPU NUMA node IDs are contiguous and that every NUMA node contains CPUs. During VCRAT generation, CPU topology entries and proximity domains are created only for NUMA nodes that have CPUs. GPU proximity domains are then allocated immediately after the CPU proximity domains, and the GPU I/O link (
proximity_domain_to) is initialized using the NUMA node ID to which the GPU is attached, implicitly assuming that NUMA node IDs and proximity domains have a one-to-one mapping.These assumptions break on systems with:
Sparse (non-contiguous) NUMA node IDs
CPU-less NUMA nodes
CPU-less and memory-less NUMA nodesFor example:
available: 3 nodes (0,2-3)
node 0: CPUs present
node 2: CPU-less
node 3: CPU-lessQuoted 8 lines
Quoted 6 lines
In this case, the driver creates a CPU topology device only for node 0 and assigns a single CPU proximity domain (0). However, the GPU VCRAT still references the NUMA node ID to which the GPU is attached (for example, node 3) in the
proximity_domain_tofield. Since no corresponding CPU proximity domain exists for node 3, the parser cannot find a matching proximity domain during VCRAT parsing, causing topology initialization to fail.Hi Tom,
Hi Felix,
Quoted 4 lines
Thank you for the explanation and the patch series. I may have some gaps in my understanding that I would like to clarify. In my mind, I was using "proximity domain" and "NUMA node" interchangeably. You're demonstrating that they are not the same thing. Is that just a different way of labeling the same thing, or are NUMA nodes and proximity domains fundamentally different concepts.
Your code in patch 5 (
kfd_proximity_domain_to_numa_nodeandkfd_numa_node_to_proximity_domain) seems to imply that there is, in fact, a 1:1 mapping, as it assumes that there is a unique translation in both directions. Am I missing something?Thanks for the comment.
IIUC, NUMA node IDs and proximity domains are different numbering schemes. We have proximity domains for both CPU and GPU devices. CPU proximity domains start from 0, and GPU proximity domains start after the last CPU proximity domain.
If the NUMA node IDs are contiguous, the CPU proximity domains happen to match the NUMA node IDs. However, if the NUMA node IDs are discontiguous, the CPU proximity domains and NUMA node IDs no longer have a one-to-one mapping.
Quoted 2 lines
If they are just different numbering systems, do we really need to keep track of both proximity domains and NUMA nodes in the KFD topology? Or would it be sufficient to only track NUMA nodes, if that's what we really care about in the uAPI (KFD sysfs)?
Thanks for the suggestion. I also think we don't need to keep track of both the proximity domains and the NUMA node IDs in KFD. Do you think the approach below would be reasonable?
Just to make sure I understand correctly, for CPU devices the proximity domain will be the same as the NUMA node ID, and the GPU proximity domains will start after the last CPU proximity domain.
In that case, the CPU proximity domains and NUMA node IDs will always have a one-to-one mapping, and the GPU proximity domains will follow after them.
Quoted 7 lines
For example, if a system has three NUMA nodes and two GPUs:
NUMA node IDs: 0 2 3
CPU proximity domains: 0 2 3
GPU proximity domains: 4 5Would it be okay to proceed with this approach?
Yes, this looks good to me.
Regards,
Felix
Quoted 101 lines
I think this approach should also resolve the driver loading issue.
Thanks
Donet TomQuoted 93 lines
Thanks,
FelixQuoted 88 lines
The failure is observed as:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeSince every online NUMA node is a valid topology object and can contain CPUs, memory, I/O links, or any combination of these, topology devices and proximity domains should be created for every online NUMA node rather than only for NUMA nodes that contain CPUs.
To address this, this series introduces a new VCRAT subtype that records the mapping between the NUMA node ID and the generated VCRAT proximity domain. When topology devices are created, this information is stored in the corresponding topology device, allowing the driver to translate a NUMA node ID into its associated proximity domain whenever required.
Returning to the previous example, the system contains three online NUMA nodes, so three CPU topology devices and three proximity domains are created, even though only one NUMA node contains CPUs. The NUMA node ID is stored in each topology device together with its generated proximity domain.
Later, when the GPU VCRAT is generated, the driver only knows the NUMA node ID to which the GPU is attached (for example, node 3). Instead of assuming that the NUMA node ID is equal to the proximity domain, the driver walks the existing topology devices to locate the corresponding NUMA node and retrieves its generated proximity domain. In this example, NUMA node 3 maps to proximity domain 2, so
proximity_domain_tois populated with the correct value.Since the GPU I/O link now references a valid proximity domain, VCRAT parsing completes successfully and topology initialization proceeds without errors on systems with sparse NUMA node IDs, CPU-less NUMA nodes, and CPU-less/memory-less NUMA nodes.
This series consists of the following patches:
Patch 1 removes an unused argument from
kfd_create_crat_image_virtual()as a preparatory cleanup.Patch 2 introduces a new VCRAT NUMA affinity subtype that stores the NUMA node ID and its corresponding proximity domain.
Patch 3 populates the NUMA affinity entries during VCRAT generation for every online NUMA node.
Patch 4 parses the NUMA affinity entries from the VCRAT and stores the NUMA node ID and proximity domain in the corresponding topology device.
Patch 5 fixes GPU VCRAT proximity domain mappings by translating the GPU's NUMA node ID to the corresponding CPU proximity domain before programming
proximity_domain_to.Patch 6 creates proximity domains and VCRAT entries for all online NUMA nodes, including CPU-less and memory-less nodes.
Patch 7 adds a
numa_nodesysfs attribute for CPU and GPU topology devices, exposing the associated NUMA node through the topology sysfs interface.Please note that the changes in this series are on a best effort basis from our end. Therefore, requesting the amd-gfx community (who have deeper knowledge of the HW & SW stack) to kindly help with the review and provide feedback / comments on these patches
Donet Tom (7):
drm/amdgpu: Remove unused argument fromkfd_create_crat_image_virtualdrm/amdgpu: Add VCRAT NUMA affinity entry
drm/amdgpu: Populate NUMA affinity entries in VCRAT drm/amdgpu: Parse NUMA affinity entries from VCRAT drm/amdgpu: Fix VCRAT proximity domain mappings for GPU nodes drm/amdgpu: Create proximity domains and VCRAT entries for CPU-less and memory-less NUMA nodes
drm/amdgpu: Addnuma_nodein cpu/gpu topology device sysfs entrydrivers/gpu/drm/amd/amdkfd/kfd_crat.c | 152
+++++++++++++++-------
drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 19 ++-drivers/gpu/drm/amd/amdkfd/kfd_topology.c| 49 +++++--drivers/gpu/drm/amd/amdkfd/kfd_topology.h| 4 +4 files changed, 168 insertions(+), 56 deletions(-)
Quoted 71 lines · Felix
Quoted 69 lines · Donet Tom
Quoted 28 lines · Felix Kuehling
Quoted 26 lines · Donet Tom
This series fixes topology device creation and proximity domain mappings in AMDKFD when a system does not provide a CRAT table and the driver generates a Virtual CRAT (VCRAT).
The current implementation assumes that CPU NUMA node IDs are contiguous and that every NUMA node contains CPUs. During VCRAT generation, CPU topology entries and proximity domains are created only for NUMA nodes that have CPUs. GPU proximity domains are then allocated immediately after the CPU proximity domains, and the GPU I/O link (
proximity_domain_to) is initialized using the NUMA node ID to which the GPU is attached, implicitly assuming that NUMA node IDs and proximity domains have a one-to-one mapping.These assumptions break on systems with:
Sparse (non-contiguous) NUMA node IDs
CPU-less NUMA nodes
CPU-less and memory-less NUMA nodesFor example:
available: 3 nodes (0,2-3)
node 0: CPUs present
node 2: CPU-less
node 3: CPU-lessQuoted 8 lines
Quoted 6 lines
In this case, the driver creates a CPU topology device only for node 0 and assigns a single CPU proximity domain (0). However, the GPU VCRAT still references the NUMA node ID to which the GPU is attached (for example, node 3) in the
proximity_domain_tofield. Since no corresponding CPU proximity domain exists for node 3, the parser cannot find a matching proximity domain during VCRAT parsing, causing topology initialization to fail.Hi Tom,
Hi Felix,
Quoted 4 lines
Thank you for the explanation and the patch series. I may have some gaps in my understanding that I would like to clarify. In my mind, I was using "proximity domain" and "NUMA node" interchangeably. You're demonstrating that they are not the same thing. Is that just a different way of labeling the same thing, or are NUMA nodes and proximity domains fundamentally different concepts.
Your code in patch 5 (
kfd_proximity_domain_to_numa_nodeandkfd_numa_node_to_proximity_domain) seems to imply that there is, in fact, a 1:1 mapping, as it assumes that there is a unique translation in both directions. Am I missing something?Thanks for the comment.
IIUC, NUMA node IDs and proximity domains are different numbering schemes. We have proximity domains for both CPU and GPU devices. CPU proximity domains start from 0, and GPU proximity domains start after the last CPU proximity domain.
If the NUMA node IDs are contiguous, the CPU proximity domains happen to match the NUMA node IDs. However, if the NUMA node IDs are discontiguous, the CPU proximity domains and NUMA node IDs no longer have a one-to-one mapping.
Quoted 2 lines
If they are just different numbering systems, do we really need to keep track of both proximity domains and NUMA nodes in the KFD topology? Or would it be sufficient to only track NUMA nodes, if that's what we really care about in the uAPI (KFD sysfs)?
Thanks for the suggestion. I also think we don't need to keep track of both the proximity domains and the NUMA node IDs in KFD. Do you think the approach below would be reasonable?
Just to make sure I understand correctly, for CPU devices the proximity domain will be the same as the NUMA node ID, and the GPU proximity domains will start after the last CPU proximity domain.
In that case, the CPU proximity domains and NUMA node IDs will always have a one-to-one mapping, and the GPU proximity domains will follow after them.
For example, if a system has three NUMA nodes and two GPUs:
Quoted 7 lines
Quoted 5 lines
NUMA node IDs: 0 2 3
CPU proximity domains: 0 2 3
GPU proximity domains: 4 5Would it be okay to proceed with this approach?
Yes, this looks good to me.
Thanks, Felix. I'll implement this approach and post a new version.
Thanks,
Donet Tom
Quoted 106 lines
Regards,
FelixQuoted 101 lines
I think this approach should also resolve the driver loading issue.
Thanks
Donet TomQuoted 93 lines
Thanks,
FelixQuoted 88 lines
The failure is observed as:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeSince every online NUMA node is a valid topology object and can contain CPUs, memory, I/O links, or any combination of these, topology devices and proximity domains should be created for every online NUMA node rather than only for NUMA nodes that contain CPUs.
To address this, this series introduces a new VCRAT subtype that records the mapping between the NUMA node ID and the generated VCRAT proximity domain. When topology devices are created, this information is stored in the corresponding topology device, allowing the driver to translate a NUMA node ID into its associated proximity domain whenever required.
Returning to the previous example, the system contains three online NUMA nodes, so three CPU topology devices and three proximity domains are created, even though only one NUMA node contains CPUs. The NUMA node ID is stored in each topology device together with its generated proximity domain.
Later, when the GPU VCRAT is generated, the driver only knows the NUMA node ID to which the GPU is attached (for example, node 3). Instead of assuming that the NUMA node ID is equal to the proximity domain, the driver walks the existing topology devices to locate the corresponding NUMA node and retrieves its generated proximity domain. In this example, NUMA node 3 maps to proximity domain 2, so
proximity_domain_tois populated with the correct value.Since the GPU I/O link now references a valid proximity domain, VCRAT parsing completes successfully and topology initialization proceeds without errors on systems with sparse NUMA node IDs, CPU-less NUMA nodes, and CPU-less/memory-less NUMA nodes.
This series consists of the following patches:
Patch 1 removes an unused argument from
kfd_create_crat_image_virtual()as a preparatory cleanup.Patch 2 introduces a new VCRAT NUMA affinity subtype that stores the NUMA node ID and its corresponding proximity domain.
Patch 3 populates the NUMA affinity entries during VCRAT generation for every online NUMA node.
Patch 4 parses the NUMA affinity entries from the VCRAT and stores the NUMA node ID and proximity domain in the corresponding topology device.
Patch 5 fixes GPU VCRAT proximity domain mappings by translating the GPU's NUMA node ID to the corresponding CPU proximity domain before programming
proximity_domain_to.Patch 6 creates proximity domains and VCRAT entries for all online NUMA nodes, including CPU-less and memory-less nodes.
Patch 7 adds a
numa_nodesysfs attribute for CPU and GPU topology devices, exposing the associated NUMA node through the topology sysfs interface.Please note that the changes in this series are on a best effort basis from our end. Therefore, requesting the amd-gfx community (who have deeper knowledge of the HW & SW stack) to kindly help with the review and provide feedback / comments on these patches
Donet Tom (7):
drm/amdgpu: Remove unused argument fromkfd_create_crat_image_virtualdrm/amdgpu: Add VCRAT NUMA affinity entry
drm/amdgpu: Populate NUMA affinity entries in VCRAT drm/amdgpu: Parse NUMA affinity entries from VCRAT drm/amdgpu: Fix VCRAT proximity domain mappings for GPU nodes drm/amdgpu: Create proximity domains and VCRAT entries for CPU-less and memory-less NUMA nodes
drm/amdgpu: Addnuma_nodein cpu/gpu topology device sysfs entrydrivers/gpu/drm/amd/amdkfd/kfd_crat.c | 152
+++++++++++++++-------
drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 19 ++-drivers/gpu/drm/amd/amdkfd/kfd_topology.c| 49 +++++--drivers/gpu/drm/amd/amdkfd/kfd_topology.h| 4 +4 files changed, 168 insertions(+), 56 deletions(-)
Quoted 71 lines · Felix
Quoted 69 lines · Donet Tom
Quoted 28 lines · Felix Kuehling
Quoted 26 lines · Donet Tom
This series fixes topology device creation and proximity domain mappings in AMDKFD when a system does not provide a CRAT table and the driver generates a Virtual CRAT (VCRAT).
The current implementation assumes that CPU NUMA node IDs are contiguous and that every NUMA node contains CPUs. During VCRAT generation, CPU topology entries and proximity domains are created only for NUMA nodes that have CPUs. GPU proximity domains are then allocated immediately after the CPU proximity domains, and the GPU I/O link (
proximity_domain_to) is initialized using the NUMA node ID to which the GPU is attached, implicitly assuming that NUMA node IDs and proximity domains have a one-to-one mapping.These assumptions break on systems with:
Sparse (non-contiguous) NUMA node IDs
CPU-less NUMA nodes
CPU-less and memory-less NUMA nodesFor example:
available: 3 nodes (0,2-3)
node 0: CPUs present
node 2: CPU-less
node 3: CPU-lessQuoted 8 lines
Quoted 6 lines
In this case, the driver creates a CPU topology device only for node 0 and assigns a single CPU proximity domain (0). However, the GPU VCRAT still references the NUMA node ID to which the GPU is attached (for example, node 3) in the
proximity_domain_tofield. Since no corresponding CPU proximity domain exists for node 3, the parser cannot find a matching proximity domain during VCRAT parsing, causing topology initialization to fail.Hi Tom,
Hi Felix,
Quoted 4 lines
Thank you for the explanation and the patch series. I may have some gaps in my understanding that I would like to clarify. In my mind, I was using "proximity domain" and "NUMA node" interchangeably. You're demonstrating that they are not the same thing. Is that just a different way of labeling the same thing, or are NUMA nodes and proximity domains fundamentally different concepts.
Your code in patch 5 (
kfd_proximity_domain_to_numa_nodeandkfd_numa_node_to_proximity_domain) seems to imply that there is, in fact, a 1:1 mapping, as it assumes that there is a unique translation in both directions. Am I missing something?Thanks for the comment.
IIUC, NUMA node IDs and proximity domains are different numbering schemes. We have proximity domains for both CPU and GPU devices. CPU proximity domains start from 0, and GPU proximity domains start after the last CPU proximity domain.
If the NUMA node IDs are contiguous, the CPU proximity domains happen to match the NUMA node IDs. However, if the NUMA node IDs are discontiguous, the CPU proximity domains and NUMA node IDs no longer have a one-to-one mapping.
Quoted 2 lines
If they are just different numbering systems, do we really need to keep track of both proximity domains and NUMA nodes in the KFD topology? Or would it be sufficient to only track NUMA nodes, if that's what we really care about in the uAPI (KFD sysfs)?
Thanks for the suggestion. I also think we don't need to keep track of both the proximity domains and the NUMA node IDs in KFD. Do you think the approach below would be reasonable?
Just to make sure I understand correctly, for CPU devices the proximity domain will be the same as the NUMA node ID, and the GPU proximity domains will start after the last CPU proximity domain.
In that case, the CPU proximity domains and NUMA node IDs will always have a one-to-one mapping, and the GPU proximity domains will follow after them.
For example, if a system has three NUMA nodes and two GPUs:
Quoted 7 lines
Quoted 5 lines
NUMA node IDs: 0 2 3
CPU proximity domains: 0 2 3
GPU proximity domains: 4 5Would it be okay to proceed with this approach?
Yes, this looks good to me.
Hi Felix,
I have been looking into this issue along with Donet.
From my understanding, there are three related but distinct concepts involved here:
* NUMA node: A system can have sparse NUMA node IDs. For example, online nodes can be 0, 2, and 3, with node 1 absent.
* Proximity domain: The proximity domain is the identifier associated with a KFD topology device. In the current implementation, proximity domains for CPU topology devices are assigned using a sequential counter, and GPU proximity domains are allocated after the CPU proximity domains.
* KFD sysfs topology node: KFD exposes each topology device through a node under its sysfs topology interface. Userspace reads these sysfs nodes to discover the CPU/GPU topology and their relationships. In the current implementation, the sysfs node numbering corresponds to the proximity domain numbering.
IIUC, the proximity-domain numbering and KFD sysfs topology-node numbering are expected to correspond, while the Linux NUMA node ID can be a separate identifier that is mapped to the corresponding topology or proximity domain.
As discussed in the thread, we implemented the
'NUMA node ==Proximity Domain' approach. However, after implementing this for sparse nodes, we found that the userspace libraries using the KFD sysfs nodes have an assumption that the topology devices are contiguous.
With this approach, discontiguous NUMA nodes result in discontiguous proximity-domain and sysfs node numbers. This would break userspace libraries that assume the topology devices are contiguous.
To support this approach, we would need to make corresponding changes in the userspace libraries, which would break applications using the existing userspace and introduce backward-compatibility concerns.
With this in mind, we think the current RFC v1 approach would be the better way to proceed, keeping the NUMA node IDs separate from the proximity-domain/sysfs numbering.
Would you agree with this approach? If so, I can rebase the RFC v1 series onto the latest kernel and post a new version for review.
Thanks,
Dhruv B
Quoted 106 lines
Regards,
FelixQuoted 101 lines
I think this approach should also resolve the driver loading issue.
Thanks
Donet TomQuoted 93 lines
Thanks,
FelixQuoted 88 lines
The failure is observed as:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeSince every online NUMA node is a valid topology object and can contain CPUs, memory, I/O links, or any combination of these, topology devices and proximity domains should be created for every online NUMA node rather than only for NUMA nodes that contain CPUs.
To address this, this series introduces a new VCRAT subtype that records the mapping between the NUMA node ID and the generated VCRAT proximity domain. When topology devices are created, this information is stored in the corresponding topology device, allowing the driver to translate a NUMA node ID into its associated proximity domain whenever required.
Returning to the previous example, the system contains three online NUMA nodes, so three CPU topology devices and three proximity domains are created, even though only one NUMA node contains CPUs. The NUMA node ID is stored in each topology device together with its generated proximity domain.
Later, when the GPU VCRAT is generated, the driver only knows the NUMA node ID to which the GPU is attached (for example, node 3). Instead of assuming that the NUMA node ID is equal to the proximity domain, the driver walks the existing topology devices to locate the corresponding NUMA node and retrieves its generated proximity domain. In this example, NUMA node 3 maps to proximity domain 2, so
proximity_domain_tois populated with the correct value.Since the GPU I/O link now references a valid proximity domain, VCRAT parsing completes successfully and topology initialization proceeds without errors on systems with sparse NUMA node IDs, CPU-less NUMA nodes, and CPU-less/memory-less NUMA nodes.
This series consists of the following patches:
Patch 1 removes an unused argument from
kfd_create_crat_image_virtual()as a preparatory cleanup.Patch 2 introduces a new VCRAT NUMA affinity subtype that stores the NUMA node ID and its corresponding proximity domain.
Patch 3 populates the NUMA affinity entries during VCRAT generation for every online NUMA node.
Patch 4 parses the NUMA affinity entries from the VCRAT and stores the NUMA node ID and proximity domain in the corresponding topology device.
Patch 5 fixes GPU VCRAT proximity domain mappings by translating the GPU's NUMA node ID to the corresponding CPU proximity domain before programming
proximity_domain_to.Patch 6 creates proximity domains and VCRAT entries for all online NUMA nodes, including CPU-less and memory-less nodes.
Patch 7 adds a
numa_nodesysfs attribute for CPU and GPU topology devices, exposing the associated NUMA node through the topology sysfs interface.Please note that the changes in this series are on a best effort basis from our end. Therefore, requesting the amd-gfx community (who have deeper knowledge of the HW & SW stack) to kindly help with the review and provide feedback / comments on these patches
Donet Tom (7):
drm/amdgpu: Remove unused argument fromkfd_create_crat_image_virtualdrm/amdgpu: Add VCRAT NUMA affinity entry
drm/amdgpu: Populate NUMA affinity entries in VCRAT drm/amdgpu: Parse NUMA affinity entries from VCRAT drm/amdgpu: Fix VCRAT proximity domain mappings for GPU nodes drm/amdgpu: Create proximity domains and VCRAT entries for CPU-less and memory-less NUMA nodes
drm/amdgpu: Addnuma_nodein cpu/gpu topology device sysfs entrydrivers/gpu/drm/amd/amdkfd/kfd_crat.c | 152
+++++++++++++++-------
drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 19 ++-drivers/gpu/drm/amd/amdkfd/kfd_topology.c| 49 +++++--drivers/gpu/drm/amd/amdkfd/kfd_topology.h| 4 +4 files changed, 168 insertions(+), 56 deletions(-)
Kuehling, Felix
maintainerQuoted 155 lines · Dhruv Bhogaonkar
[Some people who received this message don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Quoted 107 lines · Felix
Quoted 105 lines · Donet Tom
Quoted 48 lines · Felix Kuehling
Quoted 34 lines · Donet Tom
This series fixes topology device creation and proximity domain mappings in
AMDKFD when a system does not provide a CRAT table and the driver generates a
Virtual CRAT (VCRAT).The current implementation assumes that CPU NUMA node IDs are contiguous and
that every NUMA node contains CPUs. During VCRAT generation, CPU topology
entries and proximity domains are created only for NUMA nodes that have CPUs.
GPU proximity domains are then allocated immediately after the CPU proximity
domains, and the GPU I/O link (proximity_domain_to) is initialized using the
NUMA node ID to which the GPU is attached, implicitly assuming that NUMA node
IDs and proximity domains have a one-to-one mapping.These assumptions break on systems with:
Sparse (non-contiguous) NUMA node IDs
CPU-less NUMA nodes
CPU-less and memory-less NUMA nodesFor example:
available: 3 nodes (0,2-3)
node 0: CPUs present
node 2: CPU-less
node 3: CPU-lessQuoted 11 lines
In this case, the driver creates a CPU topology device only for node 0 and
assigns a single CPU proximity domain (0). However, the GPU VCRAT still
references the NUMA node ID to which the GPU is attached (for example, node 3)
in theproximity_domain_tofield. Since no corresponding CPU proximity domain
exists for node 3, the parser cannot find a matching proximity domain during
VCRAT parsing, causing topology initialization to fail.Quoted 1 line
Hi Tom,
Hi Felix,
Quoted 8 lines
Thank you for the explanation and the patch series. I may have some gaps in my understanding that I would like to clarify. In my mind, I was using "proximity domain" and "NUMA node" interchangeably. You're demonstrating that they are not the same thing. Is that just a different way of labeling the same thing, or are NUMA nodes and proximity domains fundamentally different concepts.
Quoted 4 lines
Your code in patch 5 (
kfd_proximity_domain_to_numa_nodeandkfd_numa_node_to_proximity_domain) seems to imply that there is, in fact, a 1:1 mapping, as it assumes that there is a unique translation in both directions. Am I missing something?Thanks for the comment.
IIUC, NUMA node IDs and proximity domains are different numbering schemes. We have proximity domains for both CPU and GPU devices. CPU proximity domains start from 0, and GPU proximity domains start after the last CPU proximity domain.
If the NUMA node IDs are contiguous, the CPU proximity domains happen to match the NUMA node IDs. However, if the NUMA node IDs are discontiguous, the CPU proximity domains and NUMA node IDs no longer have a one-to-one mapping.
Quoted 5 lines
If they are just different numbering systems, do we really need to keep track of both proximity domains and NUMA nodes in the KFD topology? Or would it be sufficient to only track NUMA nodes, if that's what we really care about in the uAPI (KFD sysfs)?
Thanks for the suggestion. I also think we don't need to keep track of both the proximity domains and the NUMA node IDs in KFD. Do you think the approach below would be reasonable?
Just to make sure I understand correctly, for CPU devices the proximity domain will be the same as the NUMA node ID, and the GPU proximity domains will start after the last CPU proximity domain.
In that case, the CPU proximity domains and NUMA node IDs will always have a one-to-one mapping, and the GPU proximity domains will follow after them.
For example, if a system has three NUMA nodes and two GPUs:
Quoted 7 lines
Quoted 5 lines
NUMA node IDs: 0 2 3
CPU proximity domains: 0 2 3
GPU proximity domains: 4 5Would it be okay to proceed with this approach?
Yes, this looks good to me.
Hi Felix,
I have been looking into this issue along with Donet.
From my understanding, there are three related but distinct concepts involved here:
* NUMA node: A system can have sparse NUMA node IDs. For example, online nodes can be 0, 2, and 3, with node 1 absent.
* Proximity domain: The proximity domain is the identifier associated with a KFD topology device. In the current implementation, proximity domains for CPU topology devices are assigned using a sequential counter, and GPU proximity domains are allocated after the CPU proximity domains.
* KFD sysfs topology node: KFD exposes each topology device through a node under its sysfs topology interface. Userspace reads these sysfs nodes to discover the CPU/GPU topology and their relationships. In the current implementation, the sysfs node numbering corresponds to the proximity domain numbering.
IIUC, the proximity-domain numbering and KFD sysfs topology-node numbering are expected to correspond, while the Linux NUMA node ID can be a separate identifier that is mapped to the corresponding topology or proximity domain.
As discussed in the thread, we implemented the
'NUMA node ==Proximity Domain' approach. However, after implementing this for sparse nodes, we found that the userspace libraries using the KFD sysfs nodes have an assumption that the topology devices are contiguous.With this approach, discontiguous NUMA nodes result in discontiguous proximity-domain and sysfs node numbers. This would break userspace libraries that assume the topology devices are contiguous.
Quoted 7 lines
To support this approach, we would need to make corresponding changes in the userspace libraries, which would break applications using the existing userspace and introduce backward-compatibility concerns.
With this in mind, we think the current RFC v1 approach would be the better way to proceed, keeping the NUMA node IDs separate from the proximity-domain/sysfs numbering.
I'm not sure about this. We have code in user mode that uses the KFD proximity domains as CPU NUMA nodes, e.g. in calls to mbind. E.g. https://github.com/ROCm/rocm-systems/blob/f437a7135fa67b96b48da4baeb72d5f75c406052/projects/rocr-runtime/libhsakmt/src/fmm.c#L2058
So if you change the KFD CPU proximity domain numbers to something contiguous, you're probably breaking user mode either way.
Can you point out what specific problems you run into with non-contiguous NUMA nodes topologies?
Regards,
Felix
Quoted 117 lines
Would you agree with this approach? If so, I can rebase the RFC v1 series onto the latest kernel and post a new version for review.
Thanks,
Dhruv BQuoted 111 lines
Regards,
FelixQuoted 106 lines
I think this approach should also resolve the driver loading issue.
Thanks
Donet TomQuoted 98 lines
Thanks,
FelixQuoted 93 lines
The failure is observed as:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeSince every online NUMA node is a valid topology object and can contain
CPUs, memory, I/O links, or any combination of these, topology devices and proximity domains should be created for every online NUMA node rather than only for NUMA nodes that contain CPUs.To address this, this series introduces a new VCRAT subtype that records the mapping between the NUMA node ID and the generated VCRAT proximity domain. When topology devices are created, this information is stored in the corresponding topology device, allowing the driver to translate a NUMA node ID into its associated proximity domain whenever required.
Returning to the previous example, the system contains three online NUMA nodes, so three CPU topology devices and three proximity domains are created, even though only one NUMA node contains CPUs. The NUMA node ID is stored in each topology device together with its generated proximity domain.
Later, when the GPU VCRAT is generated, the driver only knows the NUMA node ID to which the GPU is attached (for example, node 3). Instead of assuming that the NUMA node ID is equal to the proximity domain, the driver walks the existing topology devices to locate the corresponding NUMA node and retrieves its generated proximity domain. In this example, NUMA node 3 maps to proximity domain 2, so
proximity_domain_tois populated with the correct value.Since the GPU I/O link now references a valid proximity domain, VCRAT parsing completes successfully and topology initialization proceeds without errors on systems with sparse NUMA node IDs, CPU-less NUMA nodes, and CPU-less/memory-less NUMA nodes.
This series consists of the following patches:
Patch 1 removes an unused argument from
kfd_create_crat_image_virtual()as a preparatory cleanup.Patch 2 introduces a new VCRAT NUMA affinity subtype that stores the NUMA node ID and its corresponding proximity domain.
Patch 3 populates the NUMA affinity entries during VCRAT generation for every online NUMA node.
Patch 4 parses the NUMA affinity entries from the VCRAT and stores the NUMA node ID and proximity domain in the corresponding topology device.
Patch 5 fixes GPU VCRAT proximity domain mappings by translating the GPU's NUMA node ID to the corresponding CPU proximity domain before programming
proximity_domain_to.Patch 6 creates proximity domains and VCRAT entries for all online NUMA nodes, including CPU-less and memory-less nodes.
Patch 7 adds a
numa_nodesysfs attribute for CPU and GPU topology devices, exposing the associated NUMA node through the topology sysfs interface.Please note that the changes in this series are on a best effort basis from our
end. Therefore, requesting the amd-gfx community (who have deeper knowledge of the
HW & SW stack) to kindly help with the review and provide feedback / comments on
these patchesDonet Tom (7):
drm/amdgpu: Remove unused argument fromkfd_create_crat_image_virtual
drm/amdgpu: Add VCRAT NUMA affinity entry
drm/amdgpu: Populate NUMA affinity entries in VCRAT drm/amdgpu: Parse NUMA affinity entries from VCRAT drm/amdgpu: Fix VCRAT proximity domain mappings for GPU nodes drm/amdgpu: Create proximity domains and VCRAT entries for CPU-less and memory-less NUMA nodes
drm/amdgpu: Addnuma_nodein cpu/gpu topology device sysfs entrydrivers/gpu/drm/amd/amdkfd/kfd_crat.c | 152
+++++++++++++++-------
drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 19 ++-drivers/gpu/drm/amd/amdkfd/kfd_topology.c| 49 +++++--drivers/gpu/drm/amd/amdkfd/kfd_topology.h| 4 +4 files changed, 168 insertions(+), 56 deletions(-)
Felix Kuehling
maintainerQuoted 189 lines · Dhruv Bhogaonkar
Quoted 163 lines · Felix
Quoted 154 lines · Dhruv Bhogaonkar
Quoted 107 lines · Felix
Quoted 105 lines · Donet Tom
Quoted 48 lines · Felix Kuehling
Quoted 34 lines · Donet Tom
This series fixes topology device creation and proximity domain mappings in
AMDKFD when a system does not provide a CRAT table and the driver generates a
Virtual CRAT (VCRAT).The current implementation assumes that CPU NUMA node IDs are contiguous and
that every NUMA node contains CPUs. During VCRAT generation, CPU topology
entries and proximity domains are created only for NUMA nodes that have CPUs.
GPU proximity domains are then allocated immediately after the CPU proximity
domains, and the GPU I/O link (proximity_domain_to) is initialized using the
NUMA node ID to which the GPU is attached, implicitly assuming that NUMA node
IDs and proximity domains have a one-to-one mapping.These assumptions break on systems with:
Sparse (non-contiguous) NUMA node IDs
CPU-less NUMA nodes
CPU-less and memory-less NUMA nodesFor example:
available: 3 nodes (0,2-3)
node 0: CPUs present
node 2: CPU-less
node 3: CPU-lessQuoted 11 lines
In this case, the driver creates a CPU topology device only for node 0 and
assigns a single CPU proximity domain (0). However, the GPU VCRAT still
references the NUMA node ID to which the GPU is attached (for example, node 3)
in theproximity_domain_tofield. Since no corresponding CPU proximity domain
exists for node 3, the parser cannot find a matching proximity domain during
VCRAT parsing, causing topology initialization to fail.Quoted 1 line
Hi Tom,
Hi Felix,
Quoted 8 lines
Thank you for the explanation and the patch series. I may have some gaps in my understanding that I would like to clarify. In my mind, I was using "proximity domain" and "NUMA node" interchangeably. You're demonstrating that they are not the same thing. Is that just a different way of labeling the same thing, or are NUMA nodes and proximity domains fundamentally different concepts.
Quoted 4 lines
Your code in patch 5 (
kfd_proximity_domain_to_numa_nodeandkfd_numa_node_to_proximity_domain) seems to imply that there is, in fact, a 1:1 mapping, as it assumes that there is a unique translation in both directions. Am I missing something?Thanks for the comment.
IIUC, NUMA node IDs and proximity domains are different numbering schemes. We have proximity domains for both CPU and GPU devices. CPU proximity domains start from 0, and GPU proximity domains start after the last CPU proximity domain.
If the NUMA node IDs are contiguous, the CPU proximity domains happen to match the NUMA node IDs. However, if the NUMA node IDs are discontiguous, the CPU proximity domains and NUMA node IDs no longer have a one-to-one mapping.
Quoted 5 lines
If they are just different numbering systems, do we really need to keep track of both proximity domains and NUMA nodes in the KFD topology? Or would it be sufficient to only track NUMA nodes, if that's what we really care about in the uAPI (KFD sysfs)?
Thanks for the suggestion. I also think we don't need to keep track of both the proximity domains and the NUMA node IDs in KFD. Do you think the approach below would be reasonable?
Just to make sure I understand correctly, for CPU devices the proximity domain will be the same as the NUMA node ID, and the GPU proximity domains will start after the last CPU proximity domain.
In that case, the CPU proximity domains and NUMA node IDs will always have a one-to-one mapping, and the GPU proximity domains will follow after them.
For example, if a system has three NUMA nodes and two GPUs:
Quoted 7 lines
Quoted 5 lines
NUMA node IDs: 0 2 3
CPU proximity domains: 0 2 3
GPU proximity domains: 4 5Would it be okay to proceed with this approach?
Yes, this looks good to me.
Hi Felix,
I have been looking into this issue along with Donet.
From my understanding, there are three related but distinct concepts involved here:
* NUMA node: A system can have sparse NUMA node IDs. For example, online nodes can be 0, 2, and 3, with node 1 absent.
* Proximity domain: The proximity domain is the identifier associated with a KFD topology device. In the current implementation, proximity domains for CPU topology devices are assigned using a sequential counter, and GPU proximity domains are allocated after the CPU proximity domains.
* KFD sysfs topology node: KFD exposes each topology device through a node under its sysfs topology interface. Userspace reads these sysfs nodes to discover the CPU/GPU topology and their relationships. In the current implementation, the sysfs node numbering corresponds to the proximity domain numbering.
IIUC, the proximity-domain numbering and KFD sysfs topology-node numbering are expected to correspond, while the Linux NUMA node ID can be a separate identifier that is mapped to the corresponding topology or proximity domain.
As discussed in the thread, we implemented the
'NUMA node ==Proximity Domain' approach. However, after implementing this for sparse nodes, we found that the userspace libraries using the KFD sysfs nodes have an assumption that the topology devices are contiguous.With this approach, discontiguous NUMA nodes result in discontiguous proximity-domain and sysfs node numbers. This would break userspace libraries that assume the topology devices are contiguous.
Quoted 7 lines
To support this approach, we would need to make corresponding changes in the userspace libraries, which would break applications using the existing userspace and introduce backward-compatibility concerns.
With this in mind, we think the current RFC v1 approach would be the better way to proceed, keeping the NUMA node IDs separate from the proximity-domain/sysfs numbering.
Quoted 7 lines
I'm not sure about this. We have code in user mode that uses the KFD proximity domains as CPU NUMA nodes, e.g. in calls to mbind. E.g.
https://github.com/ROCm/rocm-systems/blob/f437a7135fa67b96b48da4baeb72d5f75c406052/projects/rocr-runtime/libhsakmt/src/fmm.c#L2058So if you change the KFD CPU proximity domain numbers to something contiguous, you're probably breaking user mode either way.
Hi Felix,
Thanks for pointing this out.
I think the approach you originally suggested seems to be the better one:
keeping the CPU proximity domain numbers aligned with the NUMA node IDs, including sparse NUMA node IDs.This will require corresponding userspace changes to support non-contiguous NUMA node IDs, but I don't expect this to break any existing applications or introduce backward compatibility issues. For systems with contiguous NUMA node IDs, the existing behavior will work as is, while systems with non-contiguous NUMA node IDs will work after the userspace changes.
Quoted 5 lines
For example, one of the areas that will need to be updated is here:
https://github.com/ROCm/rocm-systems/blob/2d743306cb341b5973cb0d6af7b549569d127724/
projects/rocr-runtime/libhsakmt/src/topology.c#L831
The code here is already designed to handle sysfs nodes that are not accessible because GPUs are not available in the process' cgroup. Maybe we need additional fixes here to handle the case where a directory is completely missing in sysfs. The biggest problem is probably, that user mode tries to remap the node-IDs into a contiguous range here: https://github.com/ROCm/rocm-systems/blob/2d743306cb341b5973cb0d6af7b549569d127724/projects/rocr-runtime/libhsakmt/src/topology.c#L836. We'd need to avoid that for CPU nodes so that we preserve the identity mapping from node IDs to NUMA node IDs. Maybe we can create dummy nodes with no CPU cores and no memory as place-holders.
Quoted 20 lines
If you have any pointers or suggestions regarding this approach, especially for the userspace changes, I would really appreciate your feedback.
Quoted 6 lines
Can you point out what specific problems you run into with non-contiguous NUMA nodes topologies?
The issue we see on a system with discontiguous NUMA nodes is that the GPU VCRAT parsing fails when we load the GPU driver, resulting in the following errors in dmesg:
Quoted 7 lines
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeOn our system, we have 3 NUMA nodes: 0, 2, and 3, with 2 GPUs attached to NUMA node 3.
OK, that's all stuff that would need to be addressed with kernel mode driver patches. I'm hoping it would be a simplified version of the patches that Donet already proposed.
Regards,
Felix
Quoted 173 lines
With the current upstream implementation, CPU proximity domains are assigned sequentially. Thus, NUMA nodes 0, 2, and 3 get CPU proximity domains 0, 1, and 2 respectively. The two GPUs then get proximity domains 3 and 4.
However, when generating the GPU VCRAT,
proximity_domain_tois populated directly with the NUMA node ID. Since both GPUs are attached to NUMA node 3, the VCRAT contains:GPU0 ->
proximity_domain_to= 3
GPU1 ->proximity_domain_to= 3https://elixir.bootlin.com/linux/v7.1/source/drivers/gpu/drm/amd/amdkfd/kfd_crat.c#L2175
Here, the CPU proximity domain corresponding to NUMA node 3 is actually 2, while proximity domain 3 belongs to GPU0.
During VCRAT parsing, the I/O link parser looks up this
id_toproximity domain, which is 3, and returns-ENODEVbecause there is no CPU topology device with proximity domain 3:https://elixir.bootlin.com/linux/v7.1/source/drivers/gpu/drm/amd/amdkfd/kfd_crat.c#L1284
Thus, the GPU I/O link is referencing the NUMA node ID as if it were the CPU proximity domain, which causes the VCRAT parsing to fail.
Since the parsing fails, the driver does not get loaded.
Thanks,
Dhruv BQuoted 140 lines
Regards,
FelixQuoted 130 lines
Would you agree with this approach? If so, I can rebase the RFC v1 series onto the latest kernel and post a new version for review.
Thanks,
Dhruv BQuoted 124 lines
Regards,
FelixQuoted 119 lines
I think this approach should also resolve the driver loading issue.
Thanks
Donet TomQuoted 111 lines
Thanks,
FelixQuoted 106 lines
The failure is observed as:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeSince every online NUMA node is a valid topology object and can contain
CPUs, memory, I/O links, or any combination of these, topology devices
and proximity domains should be created for every online NUMA node rather than only for NUMA nodes that contain CPUs.To address this, this series introduces a new VCRAT subtype that records the mapping between the NUMA node ID and the generated VCRAT proximity domain. When topology devices are created, this information
is stored in the corresponding topology device, allowing the driver to
translate a NUMA node ID into its associated proximity domain whenever
required.Returning to the previous example, the system contains three online NUMA nodes, so three CPU topology devices and three proximity domains
are created, even though only one NUMA node contains CPUs. The NUMA node ID is stored in each topology device together with its generated
proximity domain.Later, when the GPU VCRAT is generated, the driver only knows the NUMA
node ID to which the GPU is attached (for example, node 3). Instead of
assuming that the NUMA node ID is equal to the proximity domain, the driver walks the existing topology devices to locate the corresponding
NUMA node and retrieves its generated proximity domain. In this example, NUMA node 3 maps to proximity domain 2, soproximity_domain_tois populated with the correct value.Since the GPU I/O link now references a valid proximity domain, VCRAT
parsing completes successfully and topology initialization proceeds without errors on systems with sparse NUMA node IDs, CPU-less NUMA nodes, and CPU-less/memory-less NUMA nodes.This series consists of the following patches:
Patch 1 removes an unused argument from
kfd_create_crat_image_virtual()as a preparatory cleanup.Patch 2 introduces a new VCRAT NUMA affinity subtype that stores the NUMA node ID and its corresponding proximity domain.
Patch 3 populates the NUMA affinity entries during VCRAT generation for every online NUMA node.
Patch 4 parses the NUMA affinity entries from the VCRAT and stores the
NUMA node ID and proximity domain in the corresponding topology device.Patch 5 fixes GPU VCRAT proximity domain mappings by translating the GPU's NUMA node ID to the corresponding CPU proximity domain before programming
proximity_domain_to.Patch 6 creates proximity domains and VCRAT entries for all online NUMA nodes, including CPU-less and memory-less nodes.
Patch 7 adds a
numa_nodesysfs attribute for CPU and GPU topology devices, exposing the associated NUMA node through the topology sysfs
interface.Please note that the changes in this series are on a best effort basis from our
end. Therefore, requesting the amd-gfx community (who have deeper knowledge of the
HW & SW stack) to kindly help with the review and provide feedback / comments on
these patchesDonet Tom (7):
drm/amdgpu: Remove unused argument fromkfd_create_crat_image_virtual
drm/amdgpu: Add VCRAT NUMA affinity entry
drm/amdgpu: Populate NUMA affinity entries in VCRAT drm/amdgpu: Parse NUMA affinity entries from VCRAT drm/amdgpu: Fix VCRAT proximity domain mappings for GPU nodes drm/amdgpu: Create proximity domains and VCRAT entries for CPU-less
and memory-less NUMA nodes
drm/amdgpu: Addnuma_nodein cpu/gpu topology device sysfs entrydrivers/gpu/drm/amd/amdkfd/kfd_crat.c | 152
+++++++++++++++-------
drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 19 ++-drivers/gpu/drm/amd/amdkfd/kfd_topology.c| 49 +++++--drivers/gpu/drm/amd/amdkfd/kfd_topology.h| 4 +4 files changed, 168 insertions(+), 56 deletions(-)
Quoted 224 lines · Felix Kuehling
Quoted 189 lines · Dhruv Bhogaonkar
Quoted 163 lines · Felix
Quoted 154 lines · Dhruv Bhogaonkar
Quoted 107 lines · Felix
Quoted 105 lines · Donet Tom
Quoted 48 lines · Felix Kuehling
Quoted 34 lines · Donet Tom
This series fixes topology device creation and proximity domain mappings in
AMDKFD when a system does not provide a CRAT table and the driver generates a
Virtual CRAT (VCRAT).The current implementation assumes that CPU NUMA node IDs are contiguous and
that every NUMA node contains CPUs. During VCRAT generation, CPU topology
entries and proximity domains are created only for NUMA nodes that have CPUs.
GPU proximity domains are then allocated immediately after the CPU proximity
domains, and the GPU I/O link (proximity_domain_to) is initialized using the
NUMA node ID to which the GPU is attached, implicitly assuming that NUMA node
IDs and proximity domains have a one-to-one mapping.These assumptions break on systems with:
Sparse (non-contiguous) NUMA node IDs
CPU-less NUMA nodes
CPU-less and memory-less NUMA nodesFor example:
available: 3 nodes (0,2-3)
node 0: CPUs present
node 2: CPU-less
node 3: CPU-lessQuoted 11 lines
In this case, the driver creates a CPU topology device only for node 0 and
assigns a single CPU proximity domain (0). However, the GPU VCRAT still
references the NUMA node ID to which the GPU is attached (for example, node 3)
in theproximity_domain_tofield. Since no corresponding CPU proximity domain
exists for node 3, the parser cannot find a matching proximity domain during
VCRAT parsing, causing topology initialization to fail.Quoted 1 line
Hi Tom,
Hi Felix,
Quoted 8 lines
Thank you for the explanation and the patch series. I may have some gaps in my understanding that I would like to clarify. In my mind, I was using "proximity domain" and "NUMA node" interchangeably. You're demonstrating that they are not the same thing. Is that just a different way of labeling the same thing, or are NUMA nodes and proximity domains fundamentally different concepts.
Quoted 4 lines
Your code in patch 5 (
kfd_proximity_domain_to_numa_nodeandkfd_numa_node_to_proximity_domain) seems to imply that there is, in fact, a 1:1 mapping, as it assumes that there is a unique translation in both directions. Am I missing something?Thanks for the comment.
IIUC, NUMA node IDs and proximity domains are different numbering schemes. We have proximity domains for both CPU and GPU devices. CPU proximity domains start from 0, and GPU proximity domains start after the last CPU proximity domain.
If the NUMA node IDs are contiguous, the CPU proximity domains happen to match the NUMA node IDs. However, if the NUMA node IDs are discontiguous, the CPU proximity domains and NUMA node IDs no longer have a one-to-one mapping.
Quoted 5 lines
If they are just different numbering systems, do we really need to keep track of both proximity domains and NUMA nodes in the KFD topology? Or would it be sufficient to only track NUMA nodes, if that's what we really care about in the uAPI (KFD sysfs)?
Thanks for the suggestion. I also think we don't need to keep track of both the proximity domains and the NUMA node IDs in KFD. Do you think the approach below would be reasonable?
Just to make sure I understand correctly, for CPU devices the proximity domain will be the same as the NUMA node ID, and the GPU proximity domains will start after the last CPU proximity domain.
In that case, the CPU proximity domains and NUMA node IDs will always have a one-to-one mapping, and the GPU proximity domains will follow after them.
For example, if a system has three NUMA nodes and two GPUs:
Quoted 7 lines
Quoted 5 lines
NUMA node IDs: 0 2 3
CPU proximity domains: 0 2 3
GPU proximity domains: 4 5Would it be okay to proceed with this approach?
Yes, this looks good to me.
Hi Felix,
I have been looking into this issue along with Donet.
From my understanding, there are three related but distinct concepts involved here:
* NUMA node: A system can have sparse NUMA node IDs. For example, online nodes can be 0, 2, and 3, with node 1 absent.
* Proximity domain: The proximity domain is the identifier associated with a KFD topology device. In the current implementation, proximity domains for CPU topology devices are assigned using a sequential counter, and GPU proximity domains are allocated after the CPU proximity domains.
* KFD sysfs topology node: KFD exposes each topology device through a node under its sysfs topology interface. Userspace reads these sysfs nodes to discover the CPU/GPU topology and their relationships. In the current implementation, the sysfs node numbering corresponds to the proximity domain numbering.
IIUC, the proximity-domain numbering and KFD sysfs topology-node numbering are expected to correspond, while the Linux NUMA node ID can be a separate identifier that is mapped to the corresponding topology or proximity domain.
As discussed in the thread, we implemented the
'NUMA node ==Proximity Domain' approach. However, after implementing this for sparse nodes, we found that the userspace libraries using the KFD sysfs nodes have an assumption that the topology devices are contiguous.With this approach, discontiguous NUMA nodes result in discontiguous proximity-domain and sysfs node numbers. This would break userspace libraries that assume the topology devices are contiguous.
Quoted 7 lines
To support this approach, we would need to make corresponding changes in the userspace libraries, which would break applications using the existing userspace and introduce backward-compatibility concerns.
With this in mind, we think the current RFC v1 approach would be the better way to proceed, keeping the NUMA node IDs separate from the proximity-domain/sysfs numbering.
Quoted 7 lines
I'm not sure about this. We have code in user mode that uses the KFD proximity domains as CPU NUMA nodes, e.g. in calls to mbind. E.g.
https://github.com/ROCm/rocm-systems/blob/f437a7135fa67b96b48da4baeb72d5f75c406052/projects/rocr-runtime/libhsakmt/src/fmm.c#L2058So if you change the KFD CPU proximity domain numbers to something contiguous, you're probably breaking user mode either way.
Hi Felix,
Thanks for pointing this out.
I think the approach you originally suggested seems to be the better one:
keeping the CPU proximity domain numbers aligned with the NUMA node IDs, including sparse NUMA node IDs.This will require corresponding userspace changes to support non-contiguous NUMA node IDs, but I don't expect this to break any existing applications or introduce backward compatibility issues. For systems with contiguous NUMA node IDs, the existing behavior will work as is, while systems with non-contiguous NUMA node IDs will work after the userspace changes.
Quoted 5 lines
For example, one of the areas that will need to be updated is here:
https://github.com/ROCm/rocm-systems/blob/2d743306cb341b5973cb0d6af7b549569d127724/
projects/rocr-runtime/libhsakmt/src/topology.c#L831
The code here is already designed to handle sysfs nodes that are not accessible because GPUs are not available in the process' cgroup. Maybe we need additional fixes here to handle the case where a directory is completely missing in sysfs. The biggest problem is probably, that user mode tries to remap the node-IDs into a contiguous range here: https://github.com/ROCm/rocm-systems/blob/2d743306cb341b5973cb0d6af7b549569d127724/projects/rocr-runtime/libhsakmt/src/topology.c#L836. We'd need to avoid that for CPU nodes so that we preserve the identity mapping from node IDs to NUMA node IDs. Maybe we can create dummy nodes with no CPU cores and no memory as place-holders.
Quoted 25 lines
If you have any pointers or suggestions regarding this approach, especially for the userspace changes, I would really appreciate your feedback.
Quoted 6 lines
Can you point out what specific problems you run into with non-contiguous NUMA nodes topologies?
The issue we see on a system with discontiguous NUMA nodes is that the GPU VCRAT parsing fails when we load the GPU driver, resulting in the following errors in dmesg:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD node
Quoted 7 lines
Quoted 2 lines
On our system, we have 3 NUMA nodes: 0, 2, and 3, with 2 GPUs attached to NUMA node 3.
OK, that's all stuff that would need to be addressed with kernel mode driver patches. I'm hoping it would be a simplified version of the patches that Donet already proposed.
Regards,
Felix
Hi Felix,
Thanks for your suggestions.
I'll start working in this direction and post a patch once it's ready.
Thanks,
Dhruv B
Quoted 175 lines
Quoted 175 lines
With the current upstream implementation, CPU proximity domains are assigned sequentially. Thus, NUMA nodes 0, 2, and 3 get CPU proximity domains 0, 1, and 2 respectively. The two GPUs then get proximity domains 3 and 4.
However, when generating the GPU VCRAT,
proximity_domain_tois populated directly with the NUMA node ID. Since both GPUs are attached to NUMA node 3, the VCRAT contains:GPU0 ->
proximity_domain_to= 3
GPU1 ->proximity_domain_to= 3https://elixir.bootlin.com/linux/v7.1/source/drivers/gpu/drm/amd/amdkfd/kfd_crat.c#L2175
Here, the CPU proximity domain corresponding to NUMA node 3 is actually 2, while proximity domain 3 belongs to GPU0.
During VCRAT parsing, the I/O link parser looks up this
id_toproximity domain, which is 3, and returns-ENODEVbecause there is no CPU topology device with proximity domain 3:https://elixir.bootlin.com/linux/v7.1/source/drivers/gpu/drm/amd/amdkfd/kfd_crat.c#L1284
Thus, the GPU I/O link is referencing the NUMA node ID as if it were the CPU proximity domain, which causes the VCRAT parsing to fail.
Since the parsing fails, the driver does not get loaded.
Thanks,
Dhruv BQuoted 140 lines
Regards,
FelixQuoted 130 lines
Would you agree with this approach? If so, I can rebase the RFC v1 series onto the latest kernel and post a new version for review.
Thanks,
Dhruv BQuoted 124 lines
Regards,
FelixQuoted 119 lines
I think this approach should also resolve the driver loading issue.
Thanks
Donet TomQuoted 111 lines
Thanks,
FelixQuoted 106 lines
The failure is observed as:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeSince every online NUMA node is a valid topology object and can contain
CPUs, memory, I/O links, or any combination of these, topology devices
and proximity domains should be created for every online NUMA node rather than only for NUMA nodes that contain CPUs.To address this, this series introduces a new VCRAT subtype that records the mapping between the NUMA node ID and the generated VCRAT proximity domain. When topology devices are created, this information
is stored in the corresponding topology device, allowing the driver to
translate a NUMA node ID into its associated proximity domain whenever
required.Returning to the previous example, the system contains three online NUMA nodes, so three CPU topology devices and three proximity domains
are created, even though only one NUMA node contains CPUs. The NUMA node ID is stored in each topology device together with its generated
proximity domain.Later, when the GPU VCRAT is generated, the driver only knows the NUMA
node ID to which the GPU is attached (for example, node 3). Instead of
assuming that the NUMA node ID is equal to the proximity domain, the driver walks the existing topology devices to locate the corresponding
NUMA node and retrieves its generated proximity domain. In this example, NUMA node 3 maps to proximity domain 2, soproximity_domain_tois populated with the correct value.Since the GPU I/O link now references a valid proximity domain, VCRAT
parsing completes successfully and topology initialization proceeds without errors on systems with sparse NUMA node IDs, CPU-less NUMA nodes, and CPU-less/memory-less NUMA nodes.This series consists of the following patches:
Patch 1 removes an unused argument from
kfd_create_crat_image_virtual()as a preparatory cleanup.Patch 2 introduces a new VCRAT NUMA affinity subtype that stores the NUMA node ID and its corresponding proximity domain.
Patch 3 populates the NUMA affinity entries during VCRAT generation for every online NUMA node.
Patch 4 parses the NUMA affinity entries from the VCRAT and stores the
NUMA node ID and proximity domain in the corresponding topology device.Patch 5 fixes GPU VCRAT proximity domain mappings by translating the GPU's NUMA node ID to the corresponding CPU proximity domain before programming
proximity_domain_to.Patch 6 creates proximity domains and VCRAT entries for all online NUMA nodes, including CPU-less and memory-less nodes.
Patch 7 adds a
numa_nodesysfs attribute for CPU and GPU topology devices, exposing the associated NUMA node through the topology sysfs
interface.Please note that the changes in this series are on a best effort basis from our
end. Therefore, requesting the amd-gfx community (who have deeper knowledge of the
HW & SW stack) to kindly help with the review and provide feedback / comments on
these patchesDonet Tom (7):
drm/amdgpu: Remove unused argument fromkfd_create_crat_image_virtual
drm/amdgpu: Add VCRAT NUMA affinity entry
drm/amdgpu: Populate NUMA affinity entries in VCRAT drm/amdgpu: Parse NUMA affinity entries from VCRAT drm/amdgpu: Fix VCRAT proximity domain mappings for GPU nodes drm/amdgpu: Create proximity domains and VCRAT entries for CPU-less
and memory-less NUMA nodes
drm/amdgpu: Addnuma_nodein cpu/gpu topology device sysfs entrydrivers/gpu/drm/amd/amdkfd/kfd_crat.c | 152
+++++++++++++++-------
drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 19 ++-drivers/gpu/drm/amd/amdkfd/kfd_topology.c| 49 +++++--drivers/gpu/drm/amd/amdkfd/kfd_topology.h| 4 +4 files changed, 168 insertions(+), 56 deletions(-)
Quoted 160 lines · Felix
Quoted 159 lines · Dhruv Bhogaonkar
[Some people who received this message don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Quoted 107 lines · Felix
Quoted 105 lines · Donet Tom
Quoted 48 lines · Felix Kuehling
Quoted 34 lines · Donet Tom
This series fixes topology device creation and proximity domain mappings in
AMDKFD when a system does not provide a CRAT table and the driver generates a
Virtual CRAT (VCRAT).The current implementation assumes that CPU NUMA node IDs are contiguous and
that every NUMA node contains CPUs. During VCRAT generation, CPU topology
entries and proximity domains are created only for NUMA nodes that have CPUs.
GPU proximity domains are then allocated immediately after the CPU proximity
domains, and the GPU I/O link (proximity_domain_to) is initialized using the
NUMA node ID to which the GPU is attached, implicitly assuming that NUMA node
IDs and proximity domains have a one-to-one mapping.These assumptions break on systems with:
Sparse (non-contiguous) NUMA node IDs
CPU-less NUMA nodes
CPU-less and memory-less NUMA nodesFor example:
available: 3 nodes (0,2-3)
node 0: CPUs present
node 2: CPU-less
node 3: CPU-lessQuoted 11 lines
In this case, the driver creates a CPU topology device only for node 0 and
assigns a single CPU proximity domain (0). However, the GPU VCRAT still
references the NUMA node ID to which the GPU is attached (for example, node 3)
in theproximity_domain_tofield. Since no corresponding CPU proximity domain
exists for node 3, the parser cannot find a matching proximity domain during
VCRAT parsing, causing topology initialization to fail.Quoted 1 line
Hi Tom,
Hi Felix,
Quoted 8 lines
Thank you for the explanation and the patch series. I may have some gaps in my understanding that I would like to clarify. In my mind, I was using "proximity domain" and "NUMA node" interchangeably. You're demonstrating that they are not the same thing. Is that just a different way of labeling the same thing, or are NUMA nodes and proximity domains fundamentally different concepts.
Quoted 4 lines
Your code in patch 5 (
kfd_proximity_domain_to_numa_nodeandkfd_numa_node_to_proximity_domain) seems to imply that there is, in fact, a 1:1 mapping, as it assumes that there is a unique translation in both directions. Am I missing something?Thanks for the comment.
IIUC, NUMA node IDs and proximity domains are different numbering schemes. We have proximity domains for both CPU and GPU devices. CPU proximity domains start from 0, and GPU proximity domains start after the last CPU proximity domain.
If the NUMA node IDs are contiguous, the CPU proximity domains happen to match the NUMA node IDs. However, if the NUMA node IDs are discontiguous, the CPU proximity domains and NUMA node IDs no longer have a one-to-one mapping.
Quoted 5 lines
If they are just different numbering systems, do we really need to keep track of both proximity domains and NUMA nodes in the KFD topology? Or would it be sufficient to only track NUMA nodes, if that's what we really care about in the uAPI (KFD sysfs)?
Thanks for the suggestion. I also think we don't need to keep track of both the proximity domains and the NUMA node IDs in KFD. Do you think the approach below would be reasonable?
Just to make sure I understand correctly, for CPU devices the proximity domain will be the same as the NUMA node ID, and the GPU proximity domains will start after the last CPU proximity domain.
In that case, the CPU proximity domains and NUMA node IDs will always have a one-to-one mapping, and the GPU proximity domains will follow after them.
For example, if a system has three NUMA nodes and two GPUs:
Quoted 7 lines
Quoted 5 lines
NUMA node IDs: 0 2 3
CPU proximity domains: 0 2 3
GPU proximity domains: 4 5Would it be okay to proceed with this approach?
Yes, this looks good to me.
Hi Felix,
I have been looking into this issue along with Donet.
From my understanding, there are three related but distinct concepts involved here:
* NUMA node: A system can have sparse NUMA node IDs. For example, online nodes can be 0, 2, and 3, with node 1 absent.
* Proximity domain: The proximity domain is the identifier associated with a KFD topology device. In the current implementation, proximity domains for CPU topology devices are assigned using a sequential counter, and GPU proximity domains are allocated after the CPU proximity domains.
* KFD sysfs topology node: KFD exposes each topology device through a node under its sysfs topology interface. Userspace reads these sysfs nodes to discover the CPU/GPU topology and their relationships. In the current implementation, the sysfs node numbering corresponds to the proximity domain numbering.
IIUC, the proximity-domain numbering and KFD sysfs topology-node numbering are expected to correspond, while the Linux NUMA node ID can be a separate identifier that is mapped to the corresponding topology or proximity domain.
As discussed in the thread, we implemented the
'NUMA node ==Proximity Domain' approach. However, after implementing this for sparse nodes, we found that the userspace libraries using the KFD sysfs nodes have an assumption that the topology devices are contiguous.With this approach, discontiguous NUMA nodes result in discontiguous proximity-domain and sysfs node numbers. This would break userspace libraries that assume the topology devices are contiguous.
To support this approach, we would need to make corresponding changes in the userspace libraries, which would break applications using the existing userspace and introduce backward-compatibility concerns.
Quoted 7 lines
Quoted 3 lines
With this in mind, we think the current RFC v1 approach would be the better way to proceed, keeping the NUMA node IDs separate from the proximity-domain/sysfs numbering.
I'm not sure about this. We have code in user mode that uses the KFD proximity domains as CPU NUMA nodes, e.g. in calls to mbind. E.g. https://github.com/ROCm/rocm-systems/blob/f437a7135fa67b96b48da4baeb72d5f75c406052/projects/rocr-runtime/libhsakmt/src/fmm.c#L2058
So if you change the KFD CPU proximity domain numbers to something contiguous, you're probably breaking user mode either way.
Hi Felix,
Thanks for pointing this out.
I think the approach you originally suggested seems to be the better one:
keeping the CPU proximity domain numbers aligned with the NUMA node IDs, including sparse NUMA node IDs.
This will require corresponding userspace changes to support non-contiguous NUMA node IDs, but I don't expect this to break any existing applications or introduce backward compatibility issues. For systems with contiguous NUMA node IDs, the existing behavior will work as is, while systems with non-contiguous NUMA node IDs will work after the userspace changes.
For example, one of the areas that will need to be updated is here:
https://github.com/ROCm/rocm-systems/blob/2d743306cb341b5973cb0d6af7b549569d127724/
projects/rocr-runtime/libhsakmt/src/topology.c#L831
If you have any pointers or suggestions regarding this approach, especially for the userspace changes, I would really appreciate your feedback.
Quoted 5 lines
Can you point out what specific problems you run into with non-contiguous NUMA nodes topologies?
The issue we see on a system with discontiguous NUMA nodes is that the GPU VCRAT parsing fails when we load the GPU driver, resulting in the following errors in dmesg:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD node
On our system, we have 3 NUMA nodes: 0, 2, and 3, with 2 GPUs attached to NUMA node 3.
With the current upstream implementation, CPU proximity domains are assigned sequentially. Thus, NUMA nodes 0, 2, and 3 get CPU proximity domains 0, 1, and 2 respectively. The two GPUs then get proximity domains 3 and 4.
However, when generating the GPU VCRAT, proximity_domain_to is populated directly with the NUMA node ID. Since both GPUs are attached to NUMA node 3, the VCRAT contains:
GPU0 -> proximity_domain_to = 3
GPU1 -> proximity_domain_to = 3
https://elixir.bootlin.com/linux/v7.1/source/drivers/gpu/drm/amd/amdkfd/kfd_crat.c#L2175
Here, the CPU proximity domain corresponding to NUMA node 3 is actually 2, while proximity domain 3 belongs to GPU0.
During VCRAT parsing, the I/O link parser looks up this id_to proximity domain, which is 3, and returns -ENODEV because there is no CPU topology device with proximity domain 3:
https://elixir.bootlin.com/linux/v7.1/source/drivers/gpu/drm/amd/amdkfd/kfd_crat.c#L1284
Thus, the GPU I/O link is referencing the NUMA node ID as if it were the CPU proximity domain, which causes the VCRAT parsing to fail.
Since the parsing fails, the driver does not get loaded.
Thanks,
Dhruv B
Quoted 127 lines
Regards,
FelixQuoted 117 lines
Would you agree with this approach? If so, I can rebase the RFC v1 series onto the latest kernel and post a new version for review.
Thanks,
Dhruv BQuoted 111 lines
Regards,
FelixQuoted 106 lines
I think this approach should also resolve the driver loading issue.
Thanks
Donet TomQuoted 98 lines
Thanks,
FelixQuoted 93 lines
The failure is observed as:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeSince every online NUMA node is a valid topology object and can contain
CPUs, memory, I/O links, or any combination of these, topology devices and proximity domains should be created for every online NUMA node rather than only for NUMA nodes that contain CPUs.To address this, this series introduces a new VCRAT subtype that records the mapping between the NUMA node ID and the generated VCRAT proximity domain. When topology devices are created, this information is stored in the corresponding topology device, allowing the driver to translate a NUMA node ID into its associated proximity domain whenever required.
Returning to the previous example, the system contains three online NUMA nodes, so three CPU topology devices and three proximity domains are created, even though only one NUMA node contains CPUs. The NUMA node ID is stored in each topology device together with its generated proximity domain.
Later, when the GPU VCRAT is generated, the driver only knows the NUMA node ID to which the GPU is attached (for example, node 3). Instead of assuming that the NUMA node ID is equal to the proximity domain, the driver walks the existing topology devices to locate the corresponding NUMA node and retrieves its generated proximity domain. In this example, NUMA node 3 maps to proximity domain 2, so
proximity_domain_tois populated with the correct value.Since the GPU I/O link now references a valid proximity domain, VCRAT parsing completes successfully and topology initialization proceeds without errors on systems with sparse NUMA node IDs, CPU-less NUMA nodes, and CPU-less/memory-less NUMA nodes.
This series consists of the following patches:
Patch 1 removes an unused argument from
kfd_create_crat_image_virtual()as a preparatory cleanup.Patch 2 introduces a new VCRAT NUMA affinity subtype that stores the NUMA node ID and its corresponding proximity domain.
Patch 3 populates the NUMA affinity entries during VCRAT generation for every online NUMA node.
Patch 4 parses the NUMA affinity entries from the VCRAT and stores the NUMA node ID and proximity domain in the corresponding topology device.
Patch 5 fixes GPU VCRAT proximity domain mappings by translating the GPU's NUMA node ID to the corresponding CPU proximity domain before programming
proximity_domain_to.Patch 6 creates proximity domains and VCRAT entries for all online NUMA nodes, including CPU-less and memory-less nodes.
Patch 7 adds a
numa_nodesysfs attribute for CPU and GPU topology devices, exposing the associated NUMA node through the topology sysfs interface.Please note that the changes in this series are on a best effort basis from our
end. Therefore, requesting the amd-gfx community (who have deeper knowledge of the
HW & SW stack) to kindly help with the review and provide feedback / comments on
these patchesDonet Tom (7):
drm/amdgpu: Remove unused argument fromkfd_create_crat_image_virtual
drm/amdgpu: Add VCRAT NUMA affinity entry
drm/amdgpu: Populate NUMA affinity entries in VCRAT drm/amdgpu: Parse NUMA affinity entries from VCRAT drm/amdgpu: Fix VCRAT proximity domain mappings for GPU nodes drm/amdgpu: Create proximity domains and VCRAT entries for CPU-less and memory-less NUMA nodes
drm/amdgpu: Addnuma_nodein cpu/gpu topology device sysfs entrydrivers/gpu/drm/amd/amdkfd/kfd_crat.c | 152
+++++++++++++++-------
drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 19 ++-drivers/gpu/drm/amd/amdkfd/kfd_topology.c| 49 +++++--drivers/gpu/drm/amd/amdkfd/kfd_topology.h| 4 +4 files changed, 168 insertions(+), 56 deletions(-)
Quoted 192 lines · Dhruv Bhogaonkar
Quoted 190 lines · Felix Kuehling
Quoted 189 lines · Dhruv Bhogaonkar
Quoted 163 lines · Felix
Quoted 154 lines · Dhruv Bhogaonkar
Quoted 107 lines · Felix
Quoted 105 lines · Donet Tom
Quoted 48 lines · Felix Kuehling
Quoted 34 lines · Donet Tom
This series fixes topology device creation and proximity domain mappings in
AMDKFD when a system does not provide a CRAT table and the driver generates a
Virtual CRAT (VCRAT).The current implementation assumes that CPU NUMA node IDs are contiguous and
that every NUMA node contains CPUs. During VCRAT generation, CPU topology
entries and proximity domains are created only for NUMA nodes that have CPUs.
GPU proximity domains are then allocated immediately after the CPU proximity
domains, and the GPU I/O link (proximity_domain_to) is initialized using the
NUMA node ID to which the GPU is attached, implicitly assuming that NUMA node
IDs and proximity domains have a one-to-one mapping.These assumptions break on systems with:
Sparse (non-contiguous) NUMA node IDs
CPU-less NUMA nodes
CPU-less and memory-less NUMA nodesFor example:
available: 3 nodes (0,2-3)
node 0: CPUs present
node 2: CPU-less
node 3: CPU-lessQuoted 11 lines
In this case, the driver creates a CPU topology device only for node 0 and
assigns a single CPU proximity domain (0). However, the GPU VCRAT still
references the NUMA node ID to which the GPU is attached (for example, node 3)
in theproximity_domain_tofield. Since no corresponding CPU proximity domain
exists for node 3, the parser cannot find a matching proximity domain during
VCRAT parsing, causing topology initialization to fail.Quoted 1 line
Hi Tom,
Hi Felix,
Quoted 8 lines
Thank you for the explanation and the patch series. I may have some gaps in my understanding that I would like to clarify. In my mind, I was using "proximity domain" and "NUMA node" interchangeably. You're demonstrating that they are not the same thing. Is that just a different way of labeling the same thing, or are NUMA nodes and proximity domains fundamentally different concepts.
Quoted 4 lines
Your code in patch 5 (
kfd_proximity_domain_to_numa_nodeandkfd_numa_node_to_proximity_domain) seems to imply that there is, in fact, a 1:1 mapping, as it assumes that there is a unique translation in both directions. Am I missing something?Thanks for the comment.
IIUC, NUMA node IDs and proximity domains are different numbering schemes. We have proximity domains for both CPU and GPU devices. CPU proximity domains start from 0, and GPU proximity domains start after the last CPU proximity domain.
If the NUMA node IDs are contiguous, the CPU proximity domains happen to match the NUMA node IDs. However, if the NUMA node IDs are discontiguous, the CPU proximity domains and NUMA node IDs no longer have a one-to-one mapping.
Quoted 5 lines
If they are just different numbering systems, do we really need to keep track of both proximity domains and NUMA nodes in the KFD topology? Or would it be sufficient to only track NUMA nodes, if that's what we really care about in the uAPI (KFD sysfs)?
Thanks for the suggestion. I also think we don't need to keep track of both the proximity domains and the NUMA node IDs in KFD. Do you think the approach below would be reasonable?
Just to make sure I understand correctly, for CPU devices the proximity domain will be the same as the NUMA node ID, and the GPU proximity domains will start after the last CPU proximity domain.
In that case, the CPU proximity domains and NUMA node IDs will always have a one-to-one mapping, and the GPU proximity domains will follow after them.
For example, if a system has three NUMA nodes and two GPUs:
Quoted 7 lines
Quoted 5 lines
NUMA node IDs: 0 2 3
CPU proximity domains: 0 2 3
GPU proximity domains: 4 5Would it be okay to proceed with this approach?
Yes, this looks good to me.
Hi Felix,
I have been looking into this issue along with Donet.
From my understanding, there are three related but distinct concepts involved here:
* NUMA node: A system can have sparse NUMA node IDs. For example, online nodes can be 0, 2, and 3, with node 1 absent.
* Proximity domain: The proximity domain is the identifier associated with a KFD topology device. In the current implementation, proximity domains for CPU topology devices are assigned using a sequential counter, and GPU proximity domains are allocated after the CPU proximity domains.
* KFD sysfs topology node: KFD exposes each topology device through a node under its sysfs topology interface. Userspace reads these sysfs nodes to discover the CPU/GPU topology and their relationships. In the current implementation, the sysfs node numbering corresponds to the proximity domain numbering.
IIUC, the proximity-domain numbering and KFD sysfs topology-node numbering are expected to correspond, while the Linux NUMA node ID can be a separate identifier that is mapped to the corresponding topology or proximity domain.
As discussed in the thread, we implemented the
'NUMA node ==Proximity Domain' approach. However, after implementing this for sparse nodes, we found that the userspace libraries using the KFD sysfs nodes have an assumption that the topology devices are contiguous.With this approach, discontiguous NUMA nodes result in discontiguous proximity-domain and sysfs node numbers. This would break userspace libraries that assume the topology devices are contiguous.
Quoted 7 lines
To support this approach, we would need to make corresponding changes in the userspace libraries, which would break applications using the existing userspace and introduce backward-compatibility concerns.
With this in mind, we think the current RFC v1 approach would be the better way to proceed, keeping the NUMA node IDs separate from the proximity-domain/sysfs numbering.
Quoted 7 lines
I'm not sure about this. We have code in user mode that uses the KFD proximity domains as CPU NUMA nodes, e.g. in calls to mbind. E.g.
https://github.com/ROCm/rocm-systems/blob/f437a7135fa67b96b48da4baeb72d5f75c406052/projects/rocr-runtime/libhsakmt/src/fmm.c#L2058So if you change the KFD CPU proximity domain numbers to something contiguous, you're probably breaking user mode either way.
Hi Felix,
Thanks for pointing this out.
I think the approach you originally suggested seems to be the better one:
keeping the CPU proximity domain numbers aligned with the NUMA node IDs, including sparse NUMA node IDs.This will require corresponding userspace changes to support non-contiguous NUMA node IDs, but I don't expect this to break any existing applications or introduce backward compatibility issues. For systems with contiguous NUMA node IDs, the existing behavior will work as is, while systems with non-contiguous NUMA node IDs will work after the userspace changes.
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
For example, one of the areas that will need to be updated is here:
https://github.com/ROCm/rocm-systems/blob/2d743306cb341b5973cb0d6af7b549569d127724/
projects/rocr-runtime/libhsakmt/src/topology.c#L831
The code here is already designed to handle sysfs nodes that are not accessible because GPUs are not available in the process' cgroup. Maybe we need additional fixes here to handle the case where a directory is completely missing in sysfs. The biggest problem is probably, that user mode tries to remap the node-IDs into a contiguous range here: https://github.com/ROCm/rocm-systems/blob/2d743306cb341b5973cb0d6af7b549569d127724/projects/rocr-runtime/libhsakmt/src/topology.c#L836. We'd need to avoid that for CPU nodes so that we preserve the identity mapping from node IDs to NUMA node IDs. Maybe we can create dummy nodes with no CPU cores and no memory as place-holders.
Hi Felix,
As we discussed, I tried an approach where dummy topology devices and corresponding sysfs entries are created for absent NUMA nodes. With this approach, the KFD sysfs entries remain contiguous even when the underlying NUMA node IDs are discontiguous. Therefore, no userspace changes are required.
The core change is:
drivers/gpu/drm/amd/amdkfd/kfd_crat.c +0 -0
@@ -1879,7 +1879,7 @@ static int kfd_create_vcrat_image_cpu(
acpi_status status;
struct crat_subtype_generic *sub_type_hdr;
int avail_size = *size;
- int numa_node_id;
+ int numa_node_id, last_node;
#ifdef CONFIG_X86_64
uint32_t entries = 0;
#endif
@@ -1916,8 +1916,14 @@ static int kfd_create_vcrat_image_cpu( sub_type_hdr = (struct crat_subtype_generic *)(crat_table+1); - for_each_online_node(numa_node_id) {
- if (kfd_numa_node_to_apic_id(numa_node_id) == -1) + /*
+ * Iterate 0..last_online_node so proximity_domain == NUMA node ID. + * Absent (gap) node IDs get a domain slot but no subtypes. + */
+ last_node = find_last_bit(node_online_map.bits, MAX_NUMNODES); + for (numa_node_id = 0; numa_node_id <= last_node; numa_node_id++) { + crat_table->num_domains++;
+
+ if (!node_online(numa_node_id))
continue;
Here, VCRAT entries were not previously created for NUMA nodes that did not have CPUs. I made changes to also support CPU-less, memory-less, and CPU-less/memory-less NUMA nodes.
For missing NUMA node IDs, we reserve a proximity-domain slot without creating any CPU or memory subtypes. Similarly, for online nodes without CPUs or memory, the domain slot is still reserved. This results in a placeholder topology device while keeping the NUMA node IDs and KFD proximity domains aligned.
For example, if we have online NUMA nodes {0, 2, 3}, we reserve:
proximity domain sysfs ID
0 0
1 1 (dummy)
2 2
3 3
This allows the existing kernel and userspace code to handle the topology without remapping the actual NUMA node IDs.
I implemented this with only kernel-side changes; no userspace changes were required. With these changes, the driver loads successfully on our system, and all RCCL unit tests are passing.
Could you please share your guidance on whether this approach looks reasonable? If so, I can prepare and post a proper patch as soon as possible.
Thanks,
Dhruv B
Quoted 220 lines
Quoted 27 lines
Quoted 27 lines
If you have any pointers or suggestions regarding this approach, especially for the userspace changes, I would really appreciate your feedback.
Quoted 6 lines
Can you point out what specific problems you run into with non-contiguous NUMA nodes topologies?
The issue we see on a system with discontiguous NUMA nodes is that the GPU VCRAT parsing fails when we load the GPU driver, resulting in the following errors in dmesg:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeQuoted 7 lines
Quoted 2 lines
On our system, we have 3 NUMA nodes: 0, 2, and 3, with 2 GPUs attached to NUMA node 3.
OK, that's all stuff that would need to be addressed with kernel mode driver patches. I'm hoping it would be a simplified version of the patches that Donet already proposed.
Regards,
FelixHi Felix,
Thanks for your suggestions.
I'll start working in this direction and post a patch once it's ready.
Thanks,
Dhruv BQuoted 175 lines
Quoted 175 lines
With the current upstream implementation, CPU proximity domains are assigned sequentially. Thus, NUMA nodes 0, 2, and 3 get CPU proximity domains 0, 1, and 2 respectively. The two GPUs then get proximity domains 3 and 4.
However, when generating the GPU VCRAT,
proximity_domain_tois populated directly with the NUMA node ID. Since both GPUs are attached to NUMA node 3, the VCRAT contains:GPU0 ->
proximity_domain_to= 3
GPU1 ->proximity_domain_to= 3https://elixir.bootlin.com/linux/v7.1/source/drivers/gpu/drm/amd/amdkfd/kfd_crat.c#L2175
Here, the CPU proximity domain corresponding to NUMA node 3 is actually 2, while proximity domain 3 belongs to GPU0.
During VCRAT parsing, the I/O link parser looks up this
id_toproximity domain, which is 3, and returns-ENODEVbecause there is no CPU topology device with proximity domain 3:https://elixir.bootlin.com/linux/v7.1/source/drivers/gpu/drm/amd/amdkfd/kfd_crat.c#L1284
Thus, the GPU I/O link is referencing the NUMA node ID as if it were the CPU proximity domain, which causes the VCRAT parsing to fail.
Since the parsing fails, the driver does not get loaded.
Thanks,
Dhruv BQuoted 140 lines
Regards,
FelixQuoted 130 lines
Would you agree with this approach? If so, I can rebase the RFC v1 series onto the latest kernel and post a new version for review.
Thanks,
Dhruv BQuoted 124 lines
Regards,
FelixQuoted 119 lines
I think this approach should also resolve the driver loading issue.
Thanks
Donet TomQuoted 111 lines
Thanks,
FelixQuoted 106 lines
The failure is observed as:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeSince every online NUMA node is a valid topology object and can contain
CPUs, memory, I/O links, or any combination of these, topology devices
and proximity domains should be created for every online NUMA node rather than only for NUMA nodes that contain CPUs.To address this, this series introduces a new VCRAT subtype that records the mapping between the NUMA node ID and the generated VCRAT proximity domain. When topology devices are created, this information
is stored in the corresponding topology device, allowing the driver to
translate a NUMA node ID into its associated proximity domain whenever
required.Returning to the previous example, the system contains three online NUMA nodes, so three CPU topology devices and three proximity domains
are created, even though only one NUMA node contains CPUs. The NUMA node ID is stored in each topology device together with its generated
proximity domain.Later, when the GPU VCRAT is generated, the driver only knows the NUMA
node ID to which the GPU is attached (for example, node 3). Instead of
assuming that the NUMA node ID is equal to the proximity domain, the driver walks the existing topology devices to locate the corresponding
NUMA node and retrieves its generated proximity domain. In this example, NUMA node 3 maps to proximity domain 2, soproximity_domain_tois populated with the correct value.Since the GPU I/O link now references a valid proximity domain, VCRAT
parsing completes successfully and topology initialization proceeds without errors on systems with sparse NUMA node IDs, CPU-less NUMA nodes, and CPU-less/memory-less NUMA nodes.This series consists of the following patches:
Patch 1 removes an unused argument from
kfd_create_crat_image_virtual()as a preparatory cleanup.Patch 2 introduces a new VCRAT NUMA affinity subtype that stores the NUMA node ID and its corresponding proximity domain.
Patch 3 populates the NUMA affinity entries during VCRAT generation for every online NUMA node.
Patch 4 parses the NUMA affinity entries from the VCRAT and stores the
NUMA node ID and proximity domain in the corresponding topology device.Patch 5 fixes GPU VCRAT proximity domain mappings by translating the GPU's NUMA node ID to the corresponding CPU proximity domain before programming
proximity_domain_to.Patch 6 creates proximity domains and VCRAT entries for all online NUMA nodes, including CPU-less and memory-less nodes.
Patch 7 adds a
numa_nodesysfs attribute for CPU and GPU topology devices, exposing the associated NUMA node through the topology sysfs
interface.Please note that the changes in this series are on a best effort basis from our
end. Therefore, requesting the amd-gfx community (who have deeper knowledge of the
HW & SW stack) to kindly help with the review and provide feedback / comments on
these patchesDonet Tom (7):
drm/amdgpu: Remove unused argument fromkfd_create_crat_image_virtual
drm/amdgpu: Add VCRAT NUMA affinity entry
drm/amdgpu: Populate NUMA affinity entries in VCRAT drm/amdgpu: Parse NUMA affinity entries from VCRAT drm/amdgpu: Fix VCRAT proximity domain mappings for GPU nodes drm/amdgpu: Create proximity domains and VCRAT entries for CPU-less
and memory-less NUMA nodes
drm/amdgpu: Addnuma_nodein cpu/gpu topology device sysfs entrydrivers/gpu/drm/amd/amdkfd/kfd_crat.c | 152
+++++++++++++++-------
drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 19 ++-drivers/gpu/drm/amd/amdkfd/kfd_topology.c| 49 +++++--drivers/gpu/drm/amd/amdkfd/kfd_topology.h| 4 +4 files changed, 168 insertions(+), 56 deletions(-)
Felix Kuehling
maintainerQuoted 264 lines · Dhruv Bhogaonkar
Quoted 192 lines · Dhruv Bhogaonkar
Quoted 190 lines · Felix Kuehling
Quoted 189 lines · Dhruv Bhogaonkar
Quoted 163 lines · Felix
Quoted 154 lines · Dhruv Bhogaonkar
Quoted 107 lines · Felix
Quoted 105 lines · Donet Tom
Quoted 48 lines · Felix Kuehling
Quoted 34 lines · Donet Tom
This series fixes topology device creation and proximity domain mappings in
AMDKFD when a system does not provide a CRAT table and the driver generates a
Virtual CRAT (VCRAT).The current implementation assumes that CPU NUMA node IDs are contiguous and
that every NUMA node contains CPUs. During VCRAT generation, CPU topology
entries and proximity domains are created only for NUMA nodes that have CPUs.
GPU proximity domains are then allocated immediately after the CPU proximity
domains, and the GPU I/O link (proximity_domain_to) is initialized using the
NUMA node ID to which the GPU is attached, implicitly assuming that NUMA node
IDs and proximity domains have a one-to-one mapping.These assumptions break on systems with:
Sparse (non-contiguous) NUMA node IDs
CPU-less NUMA nodes
CPU-less and memory-less NUMA nodesFor example:
available: 3 nodes (0,2-3)
node 0: CPUs present
node 2: CPU-less
node 3: CPU-lessQuoted 11 lines
In this case, the driver creates a CPU topology device only for node 0 and
assigns a single CPU proximity domain (0). However, the GPU VCRAT still
references the NUMA node ID to which the GPU is attached (for example, node 3)
in theproximity_domain_tofield. Since no corresponding CPU proximity domain
exists for node 3, the parser cannot find a matching proximity domain during
VCRAT parsing, causing topology initialization to fail.Quoted 1 line
Hi Tom,
Hi Felix,
Quoted 8 lines
Thank you for the explanation and the patch series. I may have some gaps in my understanding that I would like to clarify. In my mind, I was using "proximity domain" and "NUMA node" interchangeably. You're demonstrating that they are not the same thing. Is that just a different way of labeling the same thing, or are NUMA nodes and proximity domains fundamentally different concepts.
Quoted 4 lines
Your code in patch 5 (
kfd_proximity_domain_to_numa_nodeandkfd_numa_node_to_proximity_domain) seems to imply that there is, in fact, a 1:1 mapping, as it assumes that there is a unique translation in both directions. Am I missing something?Thanks for the comment.
IIUC, NUMA node IDs and proximity domains are different numbering schemes. We have proximity domains for both CPU and GPU devices. CPU proximity domains start from 0, and GPU proximity domains start after the last CPU proximity domain.
If the NUMA node IDs are contiguous, the CPU proximity domains happen to match the NUMA node IDs. However, if the NUMA node IDs are discontiguous, the CPU proximity domains and NUMA node IDs no longer have a one-to-one mapping.
Quoted 5 lines
If they are just different numbering systems, do we really need to keep track of both proximity domains and NUMA nodes in the KFD topology? Or would it be sufficient to only track NUMA nodes, if that's what we really care about in the uAPI (KFD sysfs)?
Thanks for the suggestion. I also think we don't need to keep track of both the proximity domains and the NUMA node IDs in KFD. Do you think the approach below would be reasonable?
Just to make sure I understand correctly, for CPU devices the proximity domain will be the same as the NUMA node ID, and the GPU proximity domains will start after the last CPU proximity domain.
In that case, the CPU proximity domains and NUMA node IDs will always have a one-to-one mapping, and the GPU proximity domains will follow after them.
For example, if a system has three NUMA nodes and two GPUs:
Quoted 7 lines
Quoted 5 lines
NUMA node IDs: 0 2 3
CPU proximity domains: 0 2 3
GPU proximity domains: 4 5Would it be okay to proceed with this approach?
Yes, this looks good to me.
Hi Felix,
I have been looking into this issue along with Donet.
From my understanding, there are three related but distinct concepts involved here:
* NUMA node: A system can have sparse NUMA node IDs. For example, online nodes can be 0, 2, and 3, with node 1 absent.
* Proximity domain: The proximity domain is the identifier associated with a KFD topology device. In the current implementation, proximity domains for CPU topology devices are assigned using a sequential counter, and GPU proximity domains are allocated after the CPU proximity domains.
* KFD sysfs topology node: KFD exposes each topology device through a node under its sysfs topology interface. Userspace reads these sysfs nodes to discover the CPU/GPU topology and their relationships. In the current implementation, the sysfs node numbering corresponds to the proximity domain numbering.
IIUC, the proximity-domain numbering and KFD sysfs topology-node numbering are expected to correspond, while the Linux NUMA node ID can be a separate identifier that is mapped to the corresponding topology or proximity domain.
As discussed in the thread, we implemented the
'NUMA node ==Proximity Domain' approach. However, after implementing this for sparse nodes, we found that the userspace libraries using the KFD sysfs nodes have an assumption that the topology devices are contiguous.With this approach, discontiguous NUMA nodes result in discontiguous proximity-domain and sysfs node numbers. This would break userspace libraries that assume the topology devices are contiguous.
Quoted 7 lines
To support this approach, we would need to make corresponding changes in the userspace libraries, which would break applications using the existing userspace and introduce backward-compatibility concerns.
With this in mind, we think the current RFC v1 approach would be the better way to proceed, keeping the NUMA node IDs separate from the proximity-domain/sysfs numbering.
Quoted 7 lines
I'm not sure about this. We have code in user mode that uses the KFD proximity domains as CPU NUMA nodes, e.g. in calls to mbind. E.g.
https://github.com/ROCm/rocm-systems/blob/f437a7135fa67b96b48da4baeb72d5f75c406052/projects/rocr-runtime/libhsakmt/src/fmm.c#L2058So if you change the KFD CPU proximity domain numbers to something contiguous, you're probably breaking user mode either way.
Hi Felix,
Thanks for pointing this out.
I think the approach you originally suggested seems to be the better one:
keeping the CPU proximity domain numbers aligned with the NUMA node IDs, including sparse NUMA node IDs.This will require corresponding userspace changes to support non-contiguous NUMA node IDs, but I don't expect this to break any existing applications or introduce backward compatibility issues. For systems with contiguous NUMA node IDs, the existing behavior will work as is, while systems with non-contiguous NUMA node IDs will work after the userspace changes.
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
For example, one of the areas that will need to be updated is here:
https://github.com/ROCm/rocm-systems/blob/2d743306cb341b5973cb0d6af7b549569d127724/
projects/rocr-runtime/libhsakmt/src/topology.c#L831
The code here is already designed to handle sysfs nodes that are not accessible because GPUs are not available in the process' cgroup. Maybe we need additional fixes here to handle the case where a directory is completely missing in sysfs. The biggest problem is probably, that user mode tries to remap the node-IDs into a contiguous range here: https://github.com/ROCm/rocm-systems/blob/2d743306cb341b5973cb0d6af7b549569d127724/projects/rocr-runtime/libhsakmt/src/topology.c#L836. We'd need to avoid that for CPU nodes so that we preserve the identity mapping from node IDs to NUMA node IDs. Maybe we can create dummy nodes with no CPU cores and no memory as place-holders.
Hi Felix,
As we discussed, I tried an approach where dummy topology devices and corresponding sysfs entries are created for absent NUMA nodes. With this approach, the KFD sysfs entries remain contiguous even when the underlying NUMA node IDs are discontiguous. Therefore, no userspace changes are required.
The core change is:
drivers/gpu/drm/amd/amdkfd/kfd_crat.c +3
-3@@ -1879,7 +1879,7 @@ static int kfd_create_vcrat_image_cpu( acpi_status status; struct crat_subtype_generic *sub_type_hdr; int avail_size = *size; - int numa_node_id; + int numa_node_id, last_node; #ifdef CONFIG_X86_64 uint32_t entries = 0; #endif@@ -1916,8 +1916,14 @@ static int kfd_create_vcrat_image_cpu( sub_type_hdr = (struct crat_subtype_generic *)(crat_table+1); - for_each_online_node(numa_node_id) { - if (kfd_numa_node_to_apic_id(numa_node_id) == -1) + /* +* Iterate 0..last_online_node so proximity_domain == NUMA node ID.
+ * Absent (gap) node IDs get a domain slot but no subtypes. + */
+ last_node =find_last_bit(node_online_map.bits, MAX_NUMNODES); + for (numa_node_id= 0;numa_node_id<=last_node; numa_node_id++) {
+ crat_table->num_domains++;
+
+ if (!node_online(numa_node_id))
continue;Here, VCRAT entries were not previously created for NUMA nodes that did not have CPUs. I made changes to also support CPU-less, memory-less, and CPU-less/memory-less NUMA nodes.
For missing NUMA node IDs, we reserve a proximity-domain slot without creating any CPU or memory subtypes. Similarly, for online nodes without CPUs or memory, the domain slot is still reserved. This results in a placeholder topology device while keeping the NUMA node IDs and KFD proximity domains aligned.
For example, if we have online NUMA nodes {0, 2, 3}, we reserve:
proximity domain sysfs ID
0 0
1 1 (dummy)
2 2
3 3This allows the existing kernel and userspace code to handle the topology without remapping the actual NUMA node IDs.
Quoted 7 lines
I implemented this with only kernel-side changes; no userspace changes were required. With these changes, the driver loads successfully on our system, and all RCCL unit tests are passing.
Could you please share your guidance on whether this approach looks reasonable? If so, I can prepare and post a proper patch as soon as possible.
Thanks, this looks good. Sorry for the late response, I'm catching up with a backlog after I was out for a few days.
I'd be happy to review the patch that implements this idea.
Regards,
Felix
Quoted 223 lines
Thanks,
Dhruv BQuoted 220 lines
Quoted 27 lines
Quoted 27 lines
If you have any pointers or suggestions regarding this approach, especially for the userspace changes, I would really appreciate your feedback.
Quoted 6 lines
Can you point out what specific problems you run into with non-contiguous NUMA nodes topologies?
The issue we see on a system with discontiguous NUMA nodes is that the GPU VCRAT parsing fails when we load the GPU driver, resulting in the following errors in dmesg:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeQuoted 7 lines
Quoted 2 lines
On our system, we have 3 NUMA nodes: 0, 2, and 3, with 2 GPUs attached to NUMA node 3.
OK, that's all stuff that would need to be addressed with kernel mode driver patches. I'm hoping it would be a simplified version of the patches that Donet already proposed.
Regards,
FelixHi Felix,
Thanks for your suggestions.
I'll start working in this direction and post a patch once it's ready.
Thanks,
Dhruv BQuoted 175 lines
Quoted 175 lines
With the current upstream implementation, CPU proximity domains are assigned sequentially. Thus, NUMA nodes 0, 2, and 3 get CPU proximity domains 0, 1, and 2 respectively. The two GPUs then get proximity domains 3 and 4.
However, when generating the GPU VCRAT,
proximity_domain_tois populated directly with the NUMA node ID. Since both GPUs are attached to NUMA node 3, the VCRAT contains:GPU0 ->
proximity_domain_to= 3
GPU1 ->proximity_domain_to= 3https://elixir.bootlin.com/linux/v7.1/source/drivers/gpu/drm/amd/amdkfd/kfd_crat.c#L2175
Here, the CPU proximity domain corresponding to NUMA node 3 is actually 2, while proximity domain 3 belongs to GPU0.
During VCRAT parsing, the I/O link parser looks up this
id_toproximity domain, which is 3, and returns-ENODEVbecause there is no CPU topology device with proximity domain 3:https://elixir.bootlin.com/linux/v7.1/source/drivers/gpu/drm/amd/amdkfd/kfd_crat.c#L1284
Thus, the GPU I/O link is referencing the NUMA node ID as if it were the CPU proximity domain, which causes the VCRAT parsing to fail.
Since the parsing fails, the driver does not get loaded.
Thanks,
Dhruv BQuoted 140 lines
Regards,
FelixQuoted 130 lines
Would you agree with this approach? If so, I can rebase the RFC v1 series onto the latest kernel and post a new version for review.
Thanks,
Dhruv BQuoted 124 lines
Regards,
FelixQuoted 119 lines
I think this approach should also resolve the driver loading issue.
Thanks
Donet TomQuoted 111 lines
Thanks,
FelixQuoted 106 lines
The failure is observed as:
amdgpu: Virtual CRAT table created for GPU
amdgpu: Error parsing VCRAT
kfd: amdgpu: Error adding device to topology
kfd: amdgpu: Error initializing KFD nodeSince every online NUMA node is a valid topology object and can contain
CPUs, memory, I/O links, or any combination of these, topology devices
and proximity domains should be created for every online NUMA node rather than only for NUMA nodes that contain CPUs.To address this, this series introduces a new VCRAT subtype that records the mapping between the NUMA node ID and the generated VCRAT proximity domain. When topology devices are created, this information
is stored in the corresponding topology device, allowing the driver to
translate a NUMA node ID into its associated proximity domain whenever
required.Returning to the previous example, the system contains three online NUMA nodes, so three CPU topology devices and three proximity domains
are created, even though only one NUMA node contains CPUs. The NUMA node ID is stored in each topology device together with its generated
proximity domain.Later, when the GPU VCRAT is generated, the driver only knows the NUMA
node ID to which the GPU is attached (for example, node 3). Instead of
assuming that the NUMA node ID is equal to the proximity domain, the driver walks the existing topology devices to locate the corresponding
NUMA node and retrieves its generated proximity domain. In this example, NUMA node 3 maps to proximity domain 2, soproximity_domain_tois populated with the correct value.Since the GPU I/O link now references a valid proximity domain, VCRAT
parsing completes successfully and topology initialization proceeds without errors on systems with sparse NUMA node IDs, CPU-less NUMA nodes, and CPU-less/memory-less NUMA nodes.This series consists of the following patches:
Patch 1 removes an unused argument from
kfd_create_crat_image_virtual()as a preparatory cleanup.Patch 2 introduces a new VCRAT NUMA affinity subtype that stores the NUMA node ID and its corresponding proximity domain.
Patch 3 populates the NUMA affinity entries during VCRAT generation for every online NUMA node.
Patch 4 parses the NUMA affinity entries from the VCRAT and stores the
NUMA node ID and proximity domain in the corresponding topology device.Patch 5 fixes GPU VCRAT proximity domain mappings by translating the GPU's NUMA node ID to the corresponding CPU proximity domain before programming
proximity_domain_to.Patch 6 creates proximity domains and VCRAT entries for all online NUMA nodes, including CPU-less and memory-less nodes.
Patch 7 adds a
numa_nodesysfs attribute for CPU and GPU topology devices, exposing the associated NUMA node through the topology sysfs
interface.Please note that the changes in this series are on a best effort basis from our
end. Therefore, requesting the amd-gfx community (who have deeper knowledge of the
HW & SW stack) to kindly help with the review and provide feedback / comments on
these patchesDonet Tom (7):
drm/amdgpu: Remove unused argument fromkfd_create_crat_image_virtual
drm/amdgpu: Add VCRAT NUMA affinity entry
drm/amdgpu: Populate NUMA affinity entries in VCRAT drm/amdgpu: Parse NUMA affinity entries from VCRAT drm/amdgpu: Fix VCRAT proximity domain mappings for GPU nodes drm/amdgpu: Create proximity domains and VCRAT entries for CPU-less
and memory-less NUMA nodes
drm/amdgpu: Addnuma_nodein cpu/gpu topology device sysfs entrydrivers/gpu/drm/amd/amdkfd/kfd_crat.c | 152
+++++++++++++++-------
drivers/gpu/drm/amd/amdkfd/kfd_crat.h | 19 ++-drivers/gpu/drm/amd/amdkfd/kfd_topology.c| 49 +++++--drivers/gpu/drm/amd/amdkfd/kfd_topology.h| 4 +4 files changed, 168 insertions(+), 56 deletions(-)