mm/page_alloc: fix MEMF_no_dma allocations for single NUMA
PATCH 1/1 mm/page_alloc: fix MEMF_no_dma allocations for single NUMA
Currently dma_bitsize is zero by default on single NUMA node machines. This makes all alloc_domheap_pages() calls with MEMF_no_dma return NULL.
There is only 1 user of MEMF_no_dma: dom0_memflags, which are used during memory allocation for Dom0. Failing allocation with default dom0_memflags is especially severe for the PV Dom0 case: it makes alloc_chunk() to use suboptimal 2MB allocation algorithm with a search for higher memory addresses.
This can lead to the NMI watchdog timeout during PV Dom0 construction on some machines, which can be worked around by specifying "dma_bits" in Xen's cmdline manually.
Fix the issue by initialising dma_bitsize even on single NUMA machines.
---
CC: Andrew Cooper <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Julien Grall <[email protected]>
CC: Wei Liu <[email protected]>
CC: Boris Ostrovsky <[email protected]>
CC: George Dunlap <[email protected]>
CC: Roger Pau Monné <[email protected]>
--- xen/common/page_alloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
xen/common/page_alloc.c +1 -1
@@ -1863,7 +1863,7 @@ void __init end_boot_allocator(void) nr_bootmem_regions =0 ;init_heap_pages (virt_to_page (bootmem_region_list),1 ); -if ( !dma_bitsize && (num_online_nodes () >1 ) ) +if ( !dma_bitsize ) dma_bitsize =arch_get_dma_bitsize ();printk ("Domain heap initialised" );
Signature · 7 lines
2.17.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 07.01.19 at 12:27, <[email protected]> wrote:
Currently
dma_bitsizeis zero by default on single NUMA node machines. This makes allalloc_domheap_pages()calls withMEMF_no_dmareturn NULL.There is only 1 user of
MEMF_no_dma:dom0_memflags, which are used during memory allocation for Dom0. Failing allocation with defaultdom0_memflagsis especially severe for the PV Dom0 case: it makesalloc_chunk()to use suboptimal 2MB allocation algorithm with a search for higher memory addresses.
Quoted 5 lines
This can lead to the NMI watchdog timeout during PV Dom0 construction on some machines, which can be worked around by specifying "
dma_bits" in Xen's cmdline manually.Fix the issue by initialising
dma_bitsizeeven on single NUMA machines.
I've not yet looked at why exactly this was done for multi-node systems only, but in any event this change renders somewhat stale the comment next to the dma_bitsize definition.
xen/common/page_alloc.c void __init end_boot_allocator(void) · line 1867
Quoted 9 lines
xen/common/page_alloc.c +1
-1@@ -1863,7 +1863,7 @@ void __init end_boot_allocator(void) nr_bootmem_regions =0 ;init_heap_pages (virt_to_page (bootmem_region_list),1 ); -if ( !dma_bitsize && (num_online_nodes () >1 ) ) +if ( !dma_bitsize ) dma_bitsize =arch_get_dma_bitsize ();
Did you consider the alternative of leaving this code alone and instead doing
if ( !dma_bitsize )
memflags &= ~MEMF_no_dma;
else if ( (dma_zone = bits_to_zone(dma_bitsize)) < zone_hi )
pg = alloc_heap_pages(dma_zone + 1, zone_hi, order, memflags, d);
in alloc_domheap_pages(), which would also address the same issue in the case of arch_get_dma_bitsize() returning zero?
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 33 lines · Jan Beulich
Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 07.01.19 at 12:27, <[email protected]> wrote:
Currently
dma_bitsizeis zero by default on single NUMA node machines. This makes allalloc_domheap_pages()calls withMEMF_no_dmareturn NULL.There is only 1 user of
MEMF_no_dma:dom0_memflags, which are used during memory allocation for Dom0. Failing allocation with defaultdom0_memflagsis especially severe for the PV Dom0 case: it makesalloc_chunk()to use suboptimal 2MB allocation algorithm with a search for higher memory addresses.Quoted 5 lines
This can lead to the NMI watchdog timeout during PV Dom0 construction on some machines, which can be worked around by specifying "
dma_bits" in Xen's cmdline manually.Fix the issue by initialising
dma_bitsizeeven on single NUMA machines.I've not yet looked at why exactly this was done for multi-node systems only, but in any event this change renders somewhat stale the comment next to the
dma_bitsizedefinition.xen/common/page_alloc.c void __init end_boot_allocator(void) · line 1867
Quoted 9 lines
xen/common/page_alloc.c +1
-1@@ -1863,7 +1863,7 @@ void __init end_boot_allocator(void) nr_bootmem_regions =0 ;init_heap_pages (virt_to_page (bootmem_region_list),1 ); -if ( !dma_bitsize && (num_online_nodes () >1 ) ) +if ( !dma_bitsize ) dma_bitsize =arch_get_dma_bitsize ();Did you consider the alternative of leaving this code alone and instead doing
Quoted 7 lines
if ( !dma_bitsize )
memflags &= ~MEMF_no_dma;
else if ( (dma_zone=bits_to_zone(dma_bitsize)) <zone_hi)
pg =alloc_heap_pages(dma_zone + 1, zone_hi, order, memflags, d);in
alloc_domheap_pages(), which would also address the same issue in the case ofarch_get_dma_bitsize()returning zero?
I like this suggestion. There is no point in MEMF_no_dma if dma_zone doesn't exist, so the flag can be ignored. The allocator should first use higher memory addresses for memory allocation requests, and DMA (<4G) memory shouldn't end up allocated for Dom0 RAM (given there is enough mem >4G).
I'll prepare a v2 soon.
Thanks,
Sergey
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel