Skip to contentSkip to conversation

RDMA/irdma: Use kvzalloc for lvl2 leafmem allocation

PATCH Applied Jeremy Bongio 4 participants 4 messages Sep 16 – Sep 22

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 RDMA/irdma: Use kvzalloc for lvl2 leafmem allocation

During VM live migration or when registering large memory regions (e.g., guest memory >= 256 GB backed by 4 KiB pages), get_lvl2_pble() calculates total leaves exceeding ~131,072. The memory required for lvl2->leafmem.va exceeds 4 MiB (sizeof(*leaf) * total > 5.24 MiB).

Because kzalloc() requires physically contiguous pages, requests larger than MAX_PAGE_ORDER (4 MiB on x86_64) fail and trigger a warning:

  WARNING: CPU: 123 PID: 119906 at mm/page_alloc.c:5997 __alloc_frozen_pages_noprof+0x50c/0x1050
  Call Trace:
   <TASK>
   alloc_frozen_pages_noprof+0xc0/0x240
   ___kmalloc_large_node+0x3d/0xd0
   __kmalloc_large_node_noprof+0x1b/0x90
   __kmalloc_noprof+0x254/0x660
   get_lvl1_lvl2_pble+0xbc/0x270 [irdma]
   irdma_get_pble+0x44/0xc0 [irdma]
   irdma_setup_pbles+0x46/0x1d0 [irdma]
   irdma_reg_user_mr_type_mem+0x41/0x230 [irdma]
   irdma_reg_user_mr+0x194/0x1c0 [irdma]
   ib_uverbs_reg_mr+0x1b0/0x2c0 [ib_uverbs]

The leafmem array is purely host software tracking structures (struct irdma_pble_info) and does not require physically contiguous memory.

Switch the allocation from kzalloc() to kvzalloc() and free it with kvfree() so that requests exceeding MAX_PAGE_ORDER fall back to virtually contiguous memory via vmalloc.

Fixes e8c4dbc2fcac ("RDMA/irdma: Add PBLE resource manager")
Assisted-by LLM
---
 drivers/infiniband/hw/irdma/pble.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/infiniband/hw/irdma/pble.c +3 -3
index 28dfad7f940c..5380cfbfec21 100644
@@ -330,7 +330,7 @@ static void free_lvl2(struct irdma_hmc_pble_rsrc *pble_rsrc,
 	if (root->addr)
 		irdma_prm_return_pbles(&pble_rsrc->pinfo, &root->chunkinfo);
 
-	kfree(lvl2->leafmem.va);
+	kvfree(lvl2->leafmem.va);
 	lvl2->leaf = NULL;
 }
 
@@ -358,7 +358,7 @@ static int get_lvl2_pble(struct irdma_hmc_pble_rsrc *pble_rsrc,
 	lvl2->leaf_cnt = total;
 
 	lvl2->leafmem.size = (sizeof(*leaf) * total);
-	lvl2->leafmem.va = kzalloc(lvl2->leafmem.size, GFP_KERNEL);
+	lvl2->leafmem.va = kvzalloc(lvl2->leafmem.size, GFP_KERNEL);
 	if (!lvl2->leafmem.va)
 		return -ENOMEM;
 
@@ -367,7 +367,7 @@ static int get_lvl2_pble(struct irdma_hmc_pble_rsrc *pble_rsrc,
 	ret_code = irdma_prm_get_pbles(&pble_rsrc->pinfo, &root->chunkinfo,
 				       total << 3, &root->addr, &fpm_addr);
 	if (ret_code) {
-		kfree(lvl2->leafmem.va);
+		kvfree(lvl2->leafmem.va);
 		lvl2->leaf = NULL;
 		return -ENOMEM;
 	}
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
#1 lore raw
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 15 lines

During VM live migration or when registering large memory regions (e.g., guest memory >= 256 GB backed by 4 KiB pages), get_lvl2_pble() calculates total leaves exceeding ~131,072. The memory required for lvl2->leafmem.va exceeds 4 MiB (sizeof(*leaf) * total > 5.24 MiB).

Because kzalloc() requires physically contiguous pages, requests larger than MAX_PAGE_ORDER (4 MiB on x86_64) fail and trigger a warning:

  WARNING: CPU: 123 PID: 119906 at mm/page_alloc.c:5997 __alloc_frozen_pages_noprof+0x50c/0x1050
  Call Trace:
   <TASK>
   alloc_frozen_pages_noprof+0xc0/0x240
   ___kmalloc_large_node+0x3d/0xd0
   __kmalloc_large_node_noprof+0x1b/0x90
   __kmalloc_noprof+0x254/0x660
Quoted 6 lines

[ ... ]
virtually contiguous memory via vmalloc.

Fixes e8c4dbc2fcac ("RDMA/irdma: Add PBLE resource manager")
Assisted-by LLM

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw
In reply to Jeremy Bongio
Quoted 9 lines · Jeremy Bongio

During VM live migration or when registering large memory regions (e.g., guest memory >= 256 GB backed by 4 KiB pages), get_lvl2_pble() calculates total leaves exceeding ~131,072. The memory required for lvl2->leafmem.va exceeds 4 MiB (sizeof(*leaf) * total > 5.24 MiB).

Because kzalloc() requires physically contiguous pages, requests larger than MAX_PAGE_ORDER (4 MiB on x86_64) fail and trigger a warning:

[...]

Applied, thanks!

[1/1] RDMA/irdma: Use kvzalloc for lvl2 leafmem allocation

      https://git.kernel.org/rdma/rdma/c/9c40f13a63a4b8

Best regards,

Signature · 1 line
Leon Romanovsky <[email protected]>