ring_buffer: NUMA aware page allocations
Subsystems (2)
PATCH 1/1 ring_buffer: NUMA aware page allocations
Hi Robert & Steven
While browsing oprofile code in current tree, I discovered drivers/oprofile/cpu_buffer.c was using new ring_buffer code.
Apparently we lost in the conversion NUMA allocations, unless I am mistaken, since rb_allocate_pages() uses plain __get_free_page(GFP_KERNEL) calls to allocate pages.
All "buffer_page" structs are allocated with kzalloc_node(), but not the pages themselves.
Thank you
[PATCH] ring_buffer: NUMA aware page allocations
rb_allocate_pages() & ring_buffer_resize() should use alloc_pages_node() instead of __get_free_page() to allocate pages.
kernel/trace/ring_buffer.c +13 -10
@@ -333,21 +333,22 @@ static int rb_allocate_pages(struct ring_buffer_per_cpu *cpu_buffer, {struct list_head *head = &cpu_buffer->pages;struct buffer_page *bpage, *tmp; -unsigned long addr; +struct page *page;LIST_HEAD (pages);unsigned i; +int node =cpu_to_node (cpu_buffer->cpu);for (i =0 ; i < nr_pages; i++) { bpage =kzalloc_node (ALIGN (sizeof (*bpage),cache_line_size ()), -GFP_KERNEL ,cpu_to_node (cpu_buffer->cpu)); +GFP_KERNEL , node);if (!bpage) goto free_pages;list_add (&bpage->list, &pages); - addr =__get_free_page (GFP_KERNEL ); -if (!addr) + page =alloc_pages_node (node,GFP_KERNEL ,0 ); +if (!page) goto free_pages; - bpage->page = (void *)addr; + bpage->page =page_address (page);rb_init_page (bpage->page); }
@@ -605,7 +606,7 @@ int ring_buffer_resize(struct ring_buffer *buffer, unsigned long size)unsigned nr_pages, rm_pages, new_pages;struct buffer_page *bpage, *tmp;unsigned long buffer_size; -unsigned long addr; +struct page *page;LIST_HEAD (pages);int i, cpu;
@@ -663,17 +664,19 @@ int ring_buffer_resize(struct ring_buffer *buffer, unsigned long size) new_pages = nr_pages - buffer->pages;for_each_buffer_cpu (buffer, cpu) { +int node =cpu_to_node (cpu); +for (i =0 ; i < new_pages; i++) { bpage =kzalloc_node (ALIGN (sizeof (*bpage),cache_line_size ()), -GFP_KERNEL ,cpu_to_node (cpu)); +GFP_KERNEL , node);if (!bpage) goto free_pages;list_add (&bpage->list, &pages); - addr =__get_free_page (GFP_KERNEL ); -if (!addr) + page =alloc_pages_node (node,GFP_KERNEL ,0 ); +if (!page) goto free_pages; - bpage->page = (void *)addr; + bpage->page =page_address (page);rb_init_page (bpage->page); } }
Quoted 12 lines · Eric Dumazet
Hi Robert & Steven
While browsing oprofile code in current tree, I discovered
drivers/oprofile/cpu_buffer.cwas using newring_buffercode.Apparently we lost in the conversion NUMA allocations, unless I am mistaken, since
rb_allocate_pages()uses plain__get_free_page(GFP_KERNEL)calls to allocate pages.All "
buffer_page" structs are allocated withkzalloc_node(), but not the pages themselves.
Quoted 6 lines
Thank you
[PATCH]
ring_buffer: NUMA aware page allocations
rb_allocate_pages()&ring_buffer_resize()should usealloc_pages_node()instead of__get_free_page()to allocate pages.
nice improvement!
Ingo