mm: support multi_freearea to the reduction of external fragmentation
Subsystems (7)
RFC 1/1 mm: support multi_freearea to the reduction of external fragmentation
From: lipeifeng <[email protected]>
This patch would "sort" the free-pages in buddy by pages-PFN to concentrate low-order-pages allocation in the front area of memory and high-order-pages allcation on the contrary so that few memory-pollution in the back area of memory and the probablity of high-order-pages allocation would be increased significantly.
-----------------------------------------------------------------------
1) Divide memory into several segments by pages-PFN "Multi_freearea" would divide memory into FREE_AREA_COUNTS segments by pages-PFN,each memory-segment corresponds to a free_area.
Example: machine(4G of physical memery) and FREE_AREA_COUNTS(4):
page-PFN:0x0 0x40000(1G) 0x80000(2G) 0xc0000(3G) 0xFFFFF(4G)
|------------|--------------|--------------|-------------|
free_area: [0][] [1][] [2][] [3][]
NOTE: Selecting the corresponding freearea when pages are freed back
to buddy:
- pages-PFN[0, free_area_segment[0].max_pfn] -> free_area[0][]
- pages-PFN[free_area_segment[flc - 1].max_pfn, free_area_segment[flc].max_pfn] -> free_area[flc][] (flc > 0)
By this way, all pages in the same segment/free_area is within a certain range of pages-PFN.
2) Select the corresponding freearea to alloc-pages "Multi_freearea" would select the corresponding free_area by the allocation-order when alloc-pages.
- order < HIGH_ORDER_TO_FLC:
free_area[0] -> ... -> free_area[FREE_AREA_COUNTS - 1]
- order >= HIGH_ORDER_TO_FLC:
free_area[FREE_AREA_COUNTS - 1] -> ... -> free_area[0]
Example:
The machine(4G of physical memery) and FREE_AREA_COUNTS(4) and HIGH_ORDER_TO_FLC(3).
If user allocs page(order = 0),it would take page from free_area[0][] first, if that fails,try free_area[1][] and so on. If user allocs page(order = 4),it would take page from free_area[3][] first, if that fails,try free_area[2][] and so on.
By this way,low-order pages will be concentrated in the front area of memory.Because of few memory-pollution in the back area of memory, the sussessful probablity of high-order allocation would be improved.
3) Adjust the location of free-pages in the free_list "Multi_freearea" would place free-pages in the head of free_list if pages-PFN is smaller than free_area_segment[flc]->median_pfn and in the tail of free_list on the contrary.
Example:
page-PFN: free_area_segment[flc]->median_pfn
|
free_list: page->page->page->...|...page->page->page
pages-PFN:| < median_pfn | >= median_pfn |
Because it would take pages from the head of the freelist first in buddy system,the free-pages in the tail are more likely to keep in the buddy system.The closer the PFN of pages kept in buddy system, the greater the probablity of merging that into high-order pages.
--------------------------------------------------------------------------
We tested the patch on linux-4.4/linux-4.9/linux-4.14/linux-4.19 and the results shows that the patch is effective in reducing COMPACTSTALL.
Example:the machine with 4 gigabytes of physical memery and in linux-4.19.
- monkey for 12 hours.
- record COMPACTSTALL after test.
Test-result: reduced COMPACTSTALL by 95.6% with the patch.
---------------------------------
| COMPACTSTALL
---------------------------------
ori | 2189
---------------------------------
optimization | 95
---------------------------------
Note: it is a patch in linux-4.19 to consult experts:
Is it possible to merge the optimization approach to the baseline? I would develop a formal patch based on the lastest linux if there is a chance.
Thank you very much indeed for your help to check the patch.
--- arch/s390/mm/page-states.c | 7 +- include/linux/mmzone.h | 48 +++++++++++- mm/Makefile | 2 +- mm/compaction.c | 6 +- mm/memory_hotplug.c | 4 + mm/multi_freearea.c | 177 +++++++++++++++++++++++++++++++++++++++++++++ mm/multi_freearea.h | 18 +++++ mm/page_alloc.c | 119 +++++++++++++++++++++--------- mm/vmstat.c | 21 +++++- 9 files changed, 357 insertions(+), 45 deletions(-) create mode 100644 mm/multi_freearea.c create mode 100644 mm/multi_freearea.h
arch/s390/mm/page-states.c +6 -1
@@ -260,6 +260,7 @@ void arch_set_page_states(int make_stable)struct list_head *l;struct page *page;struct zone *zone; +int flc =0 ;if (!cmma_flag)return ;
@@ -267,8 +268,10 @@ void arch_set_page_states(int make_stable)drain_local_pages (NULL );for_each_populated_zone (zone) {spin_lock_irqsave (&zone->lock, flags); +traversal_multi_freearea:for_each_migratetype_order (order, t) { -list_for_each (l, &zone->free_area[order].free_list[t]) { +list_for_each (l, + &zone->free_area[flc][order].free_list[t]) { page = list_entry(l, struct page, lru);if (make_stable)set_page_stable_dat (page, order);
@@ -276,6 +279,8 @@ void arch_set_page_states(int make_stable)set_page_unused (page, order); } } +if (flc++ <FREE_AREA_COUNTS ) + goto traversal_multi_freearea;spin_unlock_irqrestore (&zone->lock, flags); } }
include/linux/mmzone.h +44 -4
78 lines not on this page · raw message
mm/Makefile +1 -1
@@ -39,7 +39,7 @@ obj-y := filemap.o mempool.o oom_kill.o fadvise.o \ mm_init.o mmu_context.o percpu.o slab_common.o \ compaction.o vmacache.o \ interval_tree.o list_lru.o workingset.o \ - debug.o $(mmu-y) + debug.o multi_freearea.o $(mmu-y) obj-y += init-mm.o
mm/compaction.c +5 -1
@@ -1314,6 +1314,7 @@ static enum compact_result __compact_finished(struct zone *zone, {unsigned int order;const int migratetype = cc->migratetype; +int flc =0 ;if (cc->contended ||fatal_signal_pending (current))return COMPACT_CONTENDED ;
@@ -1353,8 +1354,9 @@ static enum compact_result __compact_finished(struct zone *zone, }/* Direct compactor: Is a suitable page free? */ +traversal_multi_freearea:for (order = cc->order; order <MAX_ORDER ; order++) { -struct free_area *area = &zone->free_area[order]; +struct free_area *area = &zone->free_area[flc][order];bool can_steal;/* Job done if page is free of the right migratetype */
@@ -1396,6 +1398,8 @@ static enum compact_result __compact_finished(struct zone *zone,return COMPACT_CONTINUE ; } } +if (flc++ <FREE_AREA_COUNTS ) + goto traversal_multi_freearea;return COMPACT_NO_SUITABLE_PAGE ; }
mm/memory_hotplug.c +4 -0
@@ -41,6 +41,7 @@#include "internal.h" +#include "multi_freearea.h" /* * online_page_callback contains pointer to current page onlining function. * Initially it is generic_online_page(). If it is required it could be
@@ -391,6 +392,7 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, zone->spanned_pages = pfn - zone_start_pfn +1 ; } +adjust_freearea_segment (zone);/* * The section is not biggest or smallest mem_section in the zone, it * only creates a hole in the zone. So in this case, we need not
@@ -417,6 +419,7 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,/* The zone has no valid section */ zone->zone_start_pfn =0 ; zone->spanned_pages =0 ; +adjust_freearea_segment (zone);zone_span_writeunlock (zone); }
@@ -693,6 +696,7 @@ static void __meminit resize_zone_range(struct zone *zone, unsigned long start_p zone->zone_start_pfn = start_pfn; zone->spanned_pages =max (start_pfn + nr_pages, old_end_pfn) - zone->zone_start_pfn; +adjust_freearea_segment (zone); } static void __meminit resize_pgdat_range(struct pglist_data *pgdat, unsigned long start_pfn,
mm/multi_freearea.c new file +177 -0
177 lines not on this page · raw message
mm/multi_freearea.h new file +18 -0
@@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* Copyright (C) 2018-2020 Oplus. All rights reserved. +* Written by Peifeng Li ([email protected]) +*/ + +#ifndef __MULTI_FREEAREA_H__ +#define __MULTI_FREEAREA_H__ + +#define HIGH_ORDER_TO_FLC 3 + +extern void list_sort_add (struct page *page,struct zone *zone, +unsigned int order,int mt); +extern int page_to_flc (struct page *page); +extern void adjust_freearea_segment (struct zone *zone); +extern unsigned int adjust_flc (unsigned int current_flc,unsigned int order); + +#endif //__MULTI_FREEAREA_H__
mm/page_alloc.c +85 -34
358 lines not on this page · raw message
mm/vmstat.c +17 -4
@@ -30,6 +30,8 @@#include "internal.h" +#include "multi_freearea.h" +#define NUMA_STATS_THRESHOLD (U16_MAX - 2)#ifdef CONFIG_NUMA
@@ -1021,16 +1023,18 @@ static void fill_contig_page_info(struct zone *zone, struct contig_page_info *info) {unsigned int order; +int flc =0 ; info->free_pages =0 ; info->free_blocks_total =0 ; info->free_blocks_suitable =0 ; +traversal_multi_freearea:for (order =0 ; order <MAX_ORDER ; order++) {unsigned long blocks;/* Count number of free blocks */ - blocks = zone->free_area[order].nr_free; + blocks = zone->free_area[flc][order].nr_free; info->free_blocks_total += blocks;/* Count free base pages */
@@ -1041,6 +1045,8 @@ static void fill_contig_page_info(struct zone *zone, info->free_blocks_suitable += blocks << (order - suitable_order); } +if (flc++ <FREE_AREA_COUNTS ) + goto traversal_multi_freearea; }/*
@@ -1347,10 +1353,13 @@ static void frag_show_print(struct seq_file *m, pg_data_t *pgdat, struct zone *zone) {int order; +int flc =0 ;seq_printf (m,"Node %d, zone %8s " , pgdat->node_id, zone->name); -for (order =0 ; order <MAX_ORDER ; ++order) -seq_printf (m,"%6lu " , zone->free_area[order].nr_free); +for (flc =0 ; flc <FREE_AREA_COUNTS ; flc++) +for (order =0 ; order <MAX_ORDER ; ++order) +seq_printf (m,"%6lu " , + zone->free_area[flc][order].nr_free);seq_putc (m,'\n' ); }
@@ -1368,7 +1377,9 @@ static void pagetypeinfo_showfree_print(struct seq_file *m, pg_data_t *pgdat, struct zone *zone) { int order, mtype; +int flc =0 ; +traversal_multi_freearea:for (mtype =0 ; mtype <MIGRATE_TYPES ; mtype++) { seq_printf(m, "Node %4d, zone %8s, type %12s ", pgdat->node_id,
@@ -1379,7 +1390,7 @@ static void pagetypeinfo_showfree_print(struct seq_file *m,struct free_area *area;struct list_head *curr; - area = &(zone->free_area[order]); + area = &(zone->free_area[flc][order]);list_for_each (curr, &area->free_list[mtype]) freecount++;
@@ -1387,6 +1398,8 @@ static void pagetypeinfo_showfree_print(struct seq_file *m, }seq_putc (m,'\n' ); } +if (flc++ <FREE_AREA_COUNTS ) + goto traversal_multi_freearea; }/* Print out the free pages at each order for each migatetype */
Signature · 1 line
2.7.4
Quoted 59 lines · On 4/14/21 4:38 AM, [email protected] wrote:
From: lipeifeng <[email protected]>
This patch would "sort" the free-pages in buddy by pages-PFN to concentrate low-order-pages allocation in the front area of memory and high-order-pages allcation on the contrary so that few memory-pollution in the back area of memory and the probablity of high-order-pages allocation would be increased significantly.
-----------------------------------------------------------------------1) Divide memory into several segments by pages-PFN "
Multi_freearea" would divide memory intoFREE_AREA_COUNTSsegments by pages-PFN,each memory-segment corresponds to afree_area.Example: machine(4G of physical memery) and
FREE_AREA_COUNTS(4):
page-PFN:0x0 0x40000(1G) 0x80000(2G) 0xc0000(3G) 0xFFFFF(4G)
|------------|--------------|--------------|-------------|
free_area: [0][] [1][] [2][] [3][]NOTE: Selecting the corresponding freearea when pages are freed back
to buddy:
- pages-PFN[0, free_area_segment[0].max_pfn] -> free_area[0][]
- pages-PFN[free_area_segment[flc - 1].max_pfn, free_area_segment[flc].max_pfn] -> free_area[flc][] (flc > 0)By this way, all pages in the same segment/free_area is within a certain range of pages-PFN.
2) Select the corresponding freearea to alloc-pages "
Multi_freearea" would select the correspondingfree_areaby the allocation-order when alloc-pages.
- order <HIGH_ORDER_TO_FLC:
free_area[0] -> ... -> free_area[FREE_AREA_COUNTS - 1]
- order >=HIGH_ORDER_TO_FLC:
free_area[FREE_AREA_COUNTS - 1] -> ... -> free_area[0]Example:
The machine(4G of physical memery) andFREE_AREA_COUNTS(4)andHIGH_ORDER_TO_FLC(3).
If user allocs page(order = 0),it would take page from free_area[0][] first, if that fails,try free_area[1][] and so on. If user allocs page(order = 4),it would take page from free_area[3][] first, if that fails,try free_area[2][] and so on.By this way,low-order pages will be concentrated in the front area of memory.Because of few memory-pollution in the back area of memory, the sussessful probablity of high-order allocation would be improved.
3) Adjust the location of free-pages in the
free_list"Multi_freearea" would place free-pages in the head offree_listif pages-PFN is smaller than free_area_segment[flc]->median_pfn and in the tail offree_liston the contrary.Example:
page-PFN: free_area_segment[flc]->median_pfn
|
free_list: page->page->page->...|...page->page->page
pages-PFN:| <median_pfn| >=median_pfn|
Quoted 4 lines
Because it would take pages from the head of the freelist first in buddy system,the free-pages in the tail are more likely to keep in the buddy system.The closer the PFN of pages kept in buddy system, the greater the probablity of merging that into high-order pages.
I think this part 3) would be worth to be tried separately first, as it's not a big change compared to the other ones.
Hi Vlastimil Babka:
Thank you very much indeed for your advice.
Hi Vlastimil Babka, schwidefsky,heiko.carstens:
It is a temporary patch to consult experts:
Is it possible to merge the optimization idea and the implementation method to the baseline?
If there is a chance,i would develop a formal patch based on the lastest linux and it will split into serveral pacthes.
I hope you can give us your opinions, thank you very much.
From: Vlastimil Babka
Date: 2021-04-16 19:06
To: lipeifeng; peifengl55; schwidefsky; heiko.carstens
CC: linux-s390; linux-kernel; linux-mm
Subject: Re: [RFC] mm: support multi_freearea to the reduction of external fragmentation
Quoted 59 lines · On 4/14/21 4:38 AM, [email protected] wrote:
From: lipeifeng <[email protected]>
This patch would "sort" the free-pages in buddy by pages-PFN to concentrate low-order-pages allocation in the front area of memory and high-order-pages allcation on the contrary so that few memory-pollution in the back area of memory and the probablity of high-order-pages allocation would be increased significantly.
-----------------------------------------------------------------------1) Divide memory into several segments by pages-PFN "
Multi_freearea" would divide memory intoFREE_AREA_COUNTSsegments by pages-PFN,each memory-segment corresponds to afree_area.Example: machine(4G of physical memery) and
FREE_AREA_COUNTS(4):
page-PFN:0x0 0x40000(1G) 0x80000(2G) 0xc0000(3G) 0xFFFFF(4G)
|------------|--------------|--------------|-------------|
free_area: [0][] [1][] [2][] [3][]NOTE: Selecting the corresponding freearea when pages are freed back
to buddy:
- pages-PFN[0, free_area_segment[0].max_pfn] -> free_area[0][]
- pages-PFN[free_area_segment[flc - 1].max_pfn, free_area_segment[flc].max_pfn] -> free_area[flc][] (flc > 0)By this way, all pages in the same segment/free_area is within a certain range of pages-PFN.
2) Select the corresponding freearea to alloc-pages "
Multi_freearea" would select the correspondingfree_areaby the allocation-order when alloc-pages.
- order <HIGH_ORDER_TO_FLC:
free_area[0] -> ... -> free_area[FREE_AREA_COUNTS - 1]
- order >=HIGH_ORDER_TO_FLC:
free_area[FREE_AREA_COUNTS - 1] -> ... -> free_area[0]Example:
The machine(4G of physical memery) andFREE_AREA_COUNTS(4)andHIGH_ORDER_TO_FLC(3).
If user allocs page(order = 0),it would take page from free_area[0][] first, if that fails,try free_area[1][] and so on. If user allocs page(order = 4),it would take page from free_area[3][] first, if that fails,try free_area[2][] and so on.By this way,low-order pages will be concentrated in the front area of memory.Because of few memory-pollution in the back area of memory, the sussessful probablity of high-order allocation would be improved.
3) Adjust the location of free-pages in the
free_list"Multi_freearea" would place free-pages in the head offree_listif pages-PFN is smaller than free_area_segment[flc]->median_pfn and in the tail offree_liston the contrary.Example:
page-PFN: free_area_segment[flc]->median_pfn
|
free_list: page->page->page->...|...page->page->page
pages-PFN:| <median_pfn| >=median_pfn|
Quoted 4 lines
Because it would take pages from the head of the freelist first in buddy system,the free-pages in the tail are more likely to keep in the buddy system.The closer the PFN of pages kept in buddy system, the greater the probablity of merging that into high-order pages.
I think this part 3) would be worth to be tried separately first, as it's not a big change compared to the other ones.
Quoted 9 lines · On 19.04.21 04:37, [email protected] wrote:
Hi Vlastimil Babka:
Thank you very much indeed for your advice.Hi Vlastimil Babka, schwidefsky,heiko.carstens:
It is a temporary patch to consult experts:
Is it possible to merge the optimization idea and the implementation method to the baseline?
Well, we cannot really say that :)
History taught that merging large and invasive buddy changes is a tedious task, can take a long time, and can fail even after a lot of discussions and patch series.
Further, usually there has to be a very compelling reason to merge large, invasive buddy changes (read: not only optimize very specific scenarios); otherwise there will just push back because the invasive changes might introduce additional problems or degrade other special cases or even the general case.
Last but not least, there have to be more benchmarks and test cases that proof that other workload won't be degraded to a degree that people care; as one example, this includes runtime overhead when allocating/freeing pages.
What usually works best is improving the code in small steps, doing minor adjustments but moving into the desired direction.
Signature · 3 lines
Thanks, David / dhildenb
Quoted 60 lines · Vlastimil Babka
Quoted 59 lines · On 4/14/21 4:38 AM, [email protected] wrote:
From: lipeifeng <[email protected]>
This patch would "sort" the free-pages in buddy by pages-PFN to concentrate low-order-pages allocation in the front area of memory and high-order-pages allcation on the contrary so that few memory-pollution in the back area of memory and the probablity of high-order-pages allocation would be increased significantly.
-----------------------------------------------------------------------1) Divide memory into several segments by pages-PFN "
Multi_freearea" would divide memory intoFREE_AREA_COUNTSsegments by pages-PFN,each memory-segment corresponds to afree_area.Example: machine(4G of physical memery) and
FREE_AREA_COUNTS(4):
page-PFN:0x0 0x40000(1G) 0x80000(2G) 0xc0000(3G) 0xFFFFF(4G)
|------------|--------------|--------------|-------------|
free_area: [0][] [1][] [2][] [3][]NOTE: Selecting the corresponding freearea when pages are freed back
to buddy:
- pages-PFN[0, free_area_segment[0].max_pfn] -> free_area[0][]
- pages-PFN[free_area_segment[flc - 1].max_pfn, free_area_segment[flc].max_pfn] -> free_area[flc][] (flc > 0)By this way, all pages in the same segment/free_area is within a certain range of pages-PFN.
2) Select the corresponding freearea to alloc-pages "
Multi_freearea" would select the correspondingfree_areaby the allocation-order when alloc-pages.
- order <HIGH_ORDER_TO_FLC:
free_area[0] -> ... -> free_area[FREE_AREA_COUNTS - 1]
- order >=HIGH_ORDER_TO_FLC:
free_area[FREE_AREA_COUNTS - 1] -> ... -> free_area[0]Example:
The machine(4G of physical memery) andFREE_AREA_COUNTS(4)andHIGH_ORDER_TO_FLC(3).
If user allocs page(order = 0),it would take page from free_area[0][] first, if that fails,try free_area[1][] and so on. If user allocs page(order = 4),it would take page from free_area[3][] first, if that fails,try free_area[2][] and so on.By this way,low-order pages will be concentrated in the front area of memory.Because of few memory-pollution in the back area of memory, the sussessful probablity of high-order allocation would be improved.
3) Adjust the location of free-pages in the
free_list"Multi_freearea" would place free-pages in the head of free_list if pages-PFN is smaller than free_area_segment[flc]->median_pfn and in the tail of free_list on the contrary. Example: page-PFN: free_area_segment[flc]->median_pfn | free_list: page->page->page->...|...page->page->page pages-PFN:| < median_pfn | >= median_pfn |
Quoted 8 lines
Quoted 4 lines
Because it would take pages from the head of the freelist first in buddy system,the free-pages in the tail are more likely to keep in the buddy system.The closer the PFN of pages kept in buddy system, the greater the probablity of merging that into high-order pages.
I think this part 3) would be worth to be tried separately first, as it's not a big change compared to the other ones.
Let's consider part 3 only and ignore the 1) multi freearea (which might be problematic with sparcity) and 2) the modified allocation scheme (which doesn't yet quite sense to me yet, e.g., because we group by mobility and have compaction in place; I assume this really only helps in some special cases -- like the test case you are giving; I might be wrong)
Right now, we decide whether to but to head or tail based on how likely it is that we might merge to a higher-order page (buddy_merge_likely()) in the future. So we only consider the current "neighborhood" of the page we're freeing. As we restrict our neighborhood to MAX_ORDER - 1 pages (what we can actually merge). Of course, we can easily be wrong here. Grouping by movability and compaction only helps to some degree I guess.
AFAIK, what you propose is basing the decisions where to place a page (in addition?) on a median_pfn. Without 1) and 2) I cannot completely understand if 3) itself would help at all (and how to set the median_pfn). But it would certainly be interesting if we can tweak the current logic to better identify merge targets simply by tweaking buddy_merge_likely() or the assumptions it is making.
Signature · 3 lines
Thanks, David / dhildenb
Quoted 16 lines
Quoted 15 lines
Let's consider part 3 only and ignore the 1) multi freearea (which might be problematic with sparcity) and 2) the modified allocation scheme (which doesn't yet quite sense to me yet, e.g., because we group by mobility and have compaction in place; I assume this really only helps in some special cases -- like the test case you are giving; I might be wrong)
Right now, we decide whether to but to head or tail based on how likely it is that we might merge to a higher-order page (
buddy_merge_likely()) in the future. So we only consider the current "neighborhood" of the page we're freeing. As we restrict our neighborhood toMAX_ORDER- 1 pages (what we can actually merge). Of course, we can easily be wrong here. Grouping by movability and compaction only helps to some degree I guess.
Quoted 6 lines
Quoted 6 lines
AFAIK, what you propose is basing the decisions where to place a page (in addition?) on a
median_pfn. Without 1) and 2) I cannot completely understand if 3) itself would help at all (and how to set themedian_pfn). But it would certainly be interesting if we can tweak the current logic to better identify merge targets simply by tweakingbuddy_merge_likely()or the assumptions it is making.
Hi David Hildenbrand, Vlastimil Babka:
Thank you very much indeed for advices.
Quoted 4 lines
Quoted 4 lines
2) the modified allocation scheme
(which doesn't yet quite sense to me yet, e.g., because we group by mobility and have compaction in place; I assume this really only helps in some special cases -- like the test case you are giving;
---------------------------------------------------------------------------------
1) Divide memory into several segments by pages-PFN
2) Select the corresponding freearea to alloc-pages
These two parts art for the same purpose:
low-order-pages allocation will be concentrated in the front area of physical memory so that few memory-pollution in the back area of memory, the sussessful probablity of high-order allocation would be improved.
I think that it would help in almost all cases of high-oder-pages allocation, instead of special case, because it can let more high-order free-pages in buddy, example:
when user alloc 64K bytes, if the unit is page(4K bytes) and it needs to 16 times.
if the unit is 64Kbytes, it only takes once.
if there are more free-high-order-pages in buddy that few compact-stall in
alloction-process, the allocstall-time would be shortened.
We tested the speed of the high-orders-pages(order=4 and order = 8) allocation after monkey and found that it increased by more than 18%.
3) Adjust the location of free-pages in the free_list
Quoted 2 lines
Quoted 2 lines
Without 1) and 2) I cannot completely
understand if 3) itself would help at all (and how to set themedian_pfn)
-----------------------------------------------------------------------------------------------------
Median_pfn is set by the range of pages-PFN of free_area. if part 3) would be tried separately without 1) and 2), the simple setting is the median of the entire memory. But i think it will play the better role in optimization based on the 1) and 2).
Quoted 4 lines
Quoted 4 lines
Last but not least, there have to be more benchmarks and test cases that proof that other workload won't be degraded to a degree that people care; as one example, this includes runtime overhead when allocating/freeing pages.
---------------------------------------------
1. For modification of buddy: the modified allocation scheme 1)+2)
Is thers any standard detailed test-list of the modified allocation in the community? like benchmarks or any other tests? if i pass the test required by communiry that can proof the patch will not degraded to a degree that people care and can merge it in the baseline?
Thanks.
From: David Hildenbrand
Date: 2021-04-22 17:29
To: Vlastimil Babka; lipeifeng; peifengl55; schwidefsky; heiko.carstens
CC: linux-s390; linux-kernel; linux-mm
Subject: Re: [RFC] mm: support multi_freearea to the reduction of external fragmentation
Quoted 60 lines · Vlastimil Babka
Quoted 59 lines · On 4/14/21 4:38 AM, [email protected] wrote:
From: lipeifeng <[email protected]>
This patch would "sort" the free-pages in buddy by pages-PFN to concentrate low-order-pages allocation in the front area of memory and high-order-pages allcation on the contrary so that few memory-pollution in the back area of memory and the probablity of high-order-pages allocation would be increased significantly.
-----------------------------------------------------------------------1) Divide memory into several segments by pages-PFN "
Multi_freearea" would divide memory intoFREE_AREA_COUNTSsegments by pages-PFN,each memory-segment corresponds to afree_area.Example: machine(4G of physical memery) and
FREE_AREA_COUNTS(4):
page-PFN:0x0 0x40000(1G) 0x80000(2G) 0xc0000(3G) 0xFFFFF(4G)
|------------|--------------|--------------|-------------|
free_area: [0][] [1][] [2][] [3][]NOTE: Selecting the corresponding freearea when pages are freed back
to buddy:
- pages-PFN[0, free_area_segment[0].max_pfn] -> free_area[0][]
- pages-PFN[free_area_segment[flc - 1].max_pfn, free_area_segment[flc].max_pfn] -> free_area[flc][] (flc > 0)By this way, all pages in the same segment/free_area is within a certain range of pages-PFN.
2) Select the corresponding freearea to alloc-pages "
Multi_freearea" would select the correspondingfree_areaby the allocation-order when alloc-pages.
- order <HIGH_ORDER_TO_FLC:
free_area[0] -> ... -> free_area[FREE_AREA_COUNTS - 1]
- order >=HIGH_ORDER_TO_FLC:
free_area[FREE_AREA_COUNTS - 1] -> ... -> free_area[0]Example:
The machine(4G of physical memery) andFREE_AREA_COUNTS(4)andHIGH_ORDER_TO_FLC(3).
If user allocs page(order = 0),it would take page from free_area[0][] first, if that fails,try free_area[1][] and so on. If user allocs page(order = 4),it would take page from free_area[3][] first, if that fails,try free_area[2][] and so on.By this way,low-order pages will be concentrated in the front area of memory.Because of few memory-pollution in the back area of memory, the sussessful probablity of high-order allocation would be improved.
3) Adjust the location of free-pages in the
free_list"Multi_freearea" would place free-pages in the head of free_list if pages-PFN is smaller than free_area_segment[flc]->median_pfn and in the tail of free_list on the contrary. Example: page-PFN: free_area_segment[flc]->median_pfn | free_list: page->page->page->...|...page->page->page pages-PFN:| < median_pfn | >= median_pfn |
Quoted 8 lines
Quoted 4 lines
Because it would take pages from the head of the freelist first in buddy system,the free-pages in the tail are more likely to keep in the buddy system.The closer the PFN of pages kept in buddy system, the greater the probablity of merging that into high-order pages.
I think this part 3) would be worth to be tried separately first, as it's not a big change compared to the other ones.
Let's consider part 3 only and ignore the 1) multi freearea (which might be problematic with sparcity) and 2) the modified allocation scheme (which doesn't yet quite sense to me yet, e.g., because we group by mobility and have compaction in place; I assume this really only helps in some special cases -- like the test case you are giving; I might be wrong)
Right now, we decide whether to but to head or tail based on how likely it is that we might merge to a higher-order page (buddy_merge_likely()) in the future. So we only consider the current "neighborhood" of the page we're freeing. As we restrict our neighborhood to MAX_ORDER - 1 pages (what we can actually merge). Of course, we can easily be wrong here. Grouping by movability and compaction only helps to some degree I guess.
AFAIK, what you propose is basing the decisions where to place a page (in addition?) on a median_pfn. Without 1) and 2) I cannot completely understand if 3) itself would help at all (and how to set the median_pfn). But it would certainly be interesting if we can tweak the current logic to better identify merge targets simply by tweaking buddy_merge_likely() or the assumptions it is making.
Signature · 3 lines
Thanks, David / dhildenb
Quoted 38 lines · On 26.04.21 05:19, [email protected] wrote:
Quoted 12 lines
Quoted 12 lines
Let's consider part 3 only and ignore the 1) multi freearea (which might be problematic with sparcity) and 2) the modified allocation scheme (which doesn't yet quite sense to me yet, e.g., because we group by mobility and have compaction in place; I assume this really only helps in some special cases -- like the test case you are giving; I might be wrong)
Right now, we decide whether to but to head or tail based on how likely it is that we might merge to a higher-order page (buddy_merge_likely()) in the future. So we only consider the current "neighborhood" of the page we're freeing. As we restrict our neighborhood toMAX_ORDER- 1 pages (what we can actually merge). Of course, we can easily be wrongQuoted 8 lines
Quoted 8 lines
here. Grouping by movability and compaction only helps to some degree I guess.
AFAIK, what you propose is basing the decisions where to place a page (in addition?) on amedian_pfn. Without 1) and 2) I cannot completely understand if 3) itself would help at all (and how to set themedian_pfn). But it would certainly be interesting if we can tweak the current logic to better identify merge targets simply by tweakingbuddy_merge_likely()or the assumptions it is making.Hi David Hildenbrand,Vlastimil Babka:
Thank you very much indeed for advices.Quoted 4 lines
Quoted 4 lines
2) the modified allocation scheme
(which doesn't yet quite sense to me yet, e.g., because we group by mobility and have compaction in place; I assume this really only helps in some special cases -- like the test case you are giving;---------------------------------------------------------------------------------
1) Divide memory into several segments by pages-PFN
2) Select the corresponding freearea to alloc-pages
These two parts art for the same purpose:
low-order-pages allocation will be concentrated in the front area of physical memory so that few memory-pollution in the back area of memory, the sussessful probablity of high-order allocation would be improved.
Quoted 4 lines
I think that it would help in almost all cases of high-oder-pages
allocation, instead
of special case, because it can let more high-order free-pages in
buddy, example:
See, and I am not convinced that this is the case, because you really only report one example (Monkey) and I have to assume it is a special case then.
Quoted 6 lines
* when user alloc 64K bytes, if the unit is page(4K bytes) and it needs to 16 times.
if the unit is 64Kbytes, it only takes once.
Quoted 9 lines
* if there are more free-high-order-pages in buddy that few compact-stall in
alloction-process, the allocstall-time would be shortened.
We tested the speed of the high-orders-pages(order=4 and order = 8)
allocation
after monkey and found that it increased by more than 18%.
And you don't mention what the baseline configuration was. For example, how was compaction configured?
Just to clarify, what is monkey?
Monkey HTTP server? MonkeyTest disk benchmark? UI/Application Exerciser Monkey?
Quoted 16 lines
3) Adjust the location of free-pages in the
free_listQuoted 2 lines
Quoted 2 lines
Without 1) and 2) I cannot completely
understand if 3) itself would help at all (and how to set themedian_pfn)-----------------------------------------------------------------------------------------------------
Median_pfnis set by the range of pages-PFN offree_area. if part
3) would be tried separately
without 1) and 2), the simple setting is the median of the entire
memory. But i think it will play the
better role in optimization based on the 1) and 2).Quoted 4 lines
Quoted 4 lines
Last but not least, there have to be more benchmarks and test cases that proof that other workload won't be degraded to a degree that people care; as one example, this includes runtime overhead when allocating/freeing pages.
Quoted 6 lines
---------------------------------------------
1. For modification of buddy: the modified allocation scheme 1)+2)
Is thers any standard detailed test-list of the modified
allocation in the community? like benchmarks
or any other tests? if i pass the test required by communiry that can proof the patch will not degraded to a degree that people care and can merge it in the baseline?
IIRC, there are plenty. One example is will-it-scale.
Have a look at https://github.com/intel/lkp-tests.git
Signature · 3 lines
Thanks, David / dhildenb
Hi David Hildenbrand:
Quoted 7 lines
Quoted 7 lines
And you don't mention what the baseline configuration was. For example, how was compaction configured?
Just to clarify, what is monkey?
Monkey HTTP server? MonkeyTest disk benchmark? UI/Application Exerciser Monkey?
-------------------------------------------------------------------------------------
I am sorry that i didn't give a clear explanation about Monkey. It meant "UI/Application Exerciser Monkey" from google.
Excuse me, let me introduce our test:
1. record COMPACT_STALL
We tested the patch on linux-4.4/linux-4.9/linux-4.14/linux-4.19 and the results shows that the patch is effective in reducing COMPACTSTALL.
- monkey for 12 hours.
- record COMPACTSTALL after test.
Test-result: reduced COMPACTSTALL by 95.6% with the patch. (the machine with 4 gigabytes of physical memery and in linux-4.19.)
---------------------------------
| COMPACTSTALL
---------------------------------
ori | 2189
---------------------------------
optimization | 95
---------------------------------
I fully agree with the value of compaction, but compaction also bring cpu consumption and will increase the time of alloc_stall. So if we can let more free high-orders-pages in buddy instead of signal pages, it will decrease COMPACT_STALL and speed up memory allocation.
2. record the speed of the high-orders-pages allocation(order=4 and order = 8) Before and after optimization, we tested the speed of the high-orders-pages allocation after 120-hours-Monkey in 10 Android mobile phones. and the result show that the speed has been increased by more than 18%.
Also, we do some test designed by us:
(the machine with 4 gigabytes of physical memery and in linux-4.19.) model the usage of users, and constantly start and
operate the diffrent application for 120h, and we record COMPACT_STALL is decreased by 90+% and speed of the high-orders-pages is increaed by 15+%.
and I have some question, i hope you can guide me if when you are free.
1) What is the compaction configured?
Dost it meant the members in zone? like as follows:
unsigned int compact_considered;
unsigned int compact_defer_shift;
int compact_order_failed;
bool compact_blockskip_failed;
Or the some Macro variable? like as follows:
PAGE_ALLOC_COSTLY_ORDER = 3
MIN_COMPACT_PRIORITY = 1
MAX_COMPACT_RETRIES = 16
Quoted 2 lines
Quoted 2 lines
1) multi freearea (which might
be problematic with sparcity)
2) Can you pls tell me what is soarcity and what is the impact of this? and whether there are some documents about it?
IIRC, there are plenty. One example is will-it-scale.
Thanks you indeed, we will do the test and see if there's any revenue.
From: David Hildenbrand
Date: 2021-04-26 16:37
To: [email protected]; Vlastimil Babka; peifengl55; schwidefsky; heiko.carstens; zhangshiming; zhouhuacai; guoweichao; guojian
CC: linux-s390; linux-kernel; linux-mm
Subject: Re: [RFC] mm: support multi_freearea to the reduction of external fragmentation
Quoted 40 lines · On 26.04.21 05:19, [email protected] wrote:
Quoted 12 lines
Quoted 12 lines
Let's consider part 3 only and ignore the 1) multi freearea (which might be problematic with sparcity) and 2) the modified allocation scheme (which doesn't yet quite sense to me yet, e.g., because we group by mobility and have compaction in place; I assume this really only helps in some special cases -- like the test case you are giving; I might be wrong)
Right now, we decide whether to but to head or tail based on how likely it is that we might merge to a higher-order page (buddy_merge_likely()) in the future. So we only consider the current "neighborhood" of the page we're freeing. As we restrict our neighborhood toMAX_ORDER- 1 pages (what we can actually merge). Of course, we can easily be wrongQuoted 8 lines
Quoted 8 lines
here. Grouping by movability and compaction only helps to some degree I guess.
AFAIK, what you propose is basing the decisions where to place a page (in addition?) on amedian_pfn. Without 1) and 2) I cannot completely understand if 3) itself would help at all (and how to set themedian_pfn). But it would certainly be interesting if we can tweak the current logic to better identify merge targets simply by tweakingbuddy_merge_likely()or the assumptions it is making.Hi David Hildenbrand,Vlastimil Babka:
Thank you very much indeed for advices.Quoted 4 lines
Quoted 4 lines
2) the modified allocation scheme
(which doesn't yet quite sense to me yet, e.g., because we group by mobility and have compaction in place; I assume this really only helps in some special cases -- like the test case you are giving;---------------------------------------------------------------------------------
1) Divide memory into several segments by pages-PFN
2) Select the corresponding freearea to alloc-pages
These two parts art for the same purpose:
low-order-pages allocation will be concentrated in the front area of physical memory
so that few memory-pollution in the back area of memory, the sussessful probablity
of high-order allocation would be improved.
Quoted 4 lines
I think that it would help in almost all cases of high-oder-pages
allocation, instead
of special case, because it can let more high-order free-pages in
buddy, example:
See, and I am not convinced that this is the case, because you really only report one example (Monkey) and I have to assume it is a special case then.
Quoted 6 lines
* when user alloc 64K bytes, if the unit is page(4K bytes) and it needs to 16 times.
if the unit is 64Kbytes, it only takes once.
Quoted 9 lines
* if there are more free-high-order-pages in buddy that few compact-stall in
alloction-process, the allocstall-time would be shortened.
We tested the speed of the high-orders-pages(order=4 and order = 8)
allocation
after monkey and found that it increased by more than 18%.
And you don't mention what the baseline configuration was. For example, how was compaction configured?
Just to clarify, what is monkey?
Monkey HTTP server? MonkeyTest disk benchmark? UI/Application Exerciser Monkey?
Quoted 16 lines
3) Adjust the location of free-pages in the
free_listQuoted 2 lines
Quoted 2 lines
Without 1) and 2) I cannot completely
understand if 3) itself would help at all (and how to set themedian_pfn)-----------------------------------------------------------------------------------------------------
Median_pfnis set by the range of pages-PFN offree_area. if part
3) would be tried separately
without 1) and 2), the simple setting is the median of the entire
memory. But i think it will play the
better role in optimization based on the 1) and 2).Quoted 4 lines
Quoted 4 lines
Last but not least, there have to be more benchmarks and test cases that proof that other workload won't be degraded to a degree that people care; as one example, this includes runtime overhead when allocating/freeing pages.
Quoted 7 lines
---------------------------------------------
1. For modification of buddy: the modified allocation scheme 1)+2) Is thers any standard detailed test-list of the modified
allocation in the community? like benchmarks
or any other tests? if i pass the test required by communiry that can proof the patch will not degraded
to a degree that people care and can merge it in the baseline?
IIRC, there are plenty. One example is will-it-scale.
Signature · 3 lines
Thanks, David / dhildenb
Quoted 7 lines · On 26.04.21 12:19, [email protected] wrote:
Hi David Hildenbrand <mailto:[email protected]>:
Quoted 5 lines
Quoted 5 lines
And you don't mention what the baseline configuration was. For example, how was compaction configured?
Just to clarify, what is monkey?
Monkey HTTP server? MonkeyTest disk benchmark? UI/Application Exerciser Monkey?
Quoted 6 lines
-------------------------------------------------------------------------------------
I am sorry that i didn't give a clear explanation about Monkey. It meant "UI/Application Exerciser Monkey" from google.Excuse me, let me introduce our test:
Thanks for more details on the test.
Quoted 12 lines
1. record
COMPACT_STALL
We tested the patch on linux-4.4/linux-4.9/linux-4.14/linux-4.19 and the results shows that the patch is effective in reducing COMPACTSTALL.
- monkey for 12 hours.
- record COMPACTSTALL after test.Test-result: reduced COMPACTSTALL by 95.6% with the patch. (the machine with 4 gigabytes of physical memery and in linux-4.19.)
---------------------------------
| COMPACTSTALL
---------------------------------ori | 2189
Quoted 8 lines
---------------------------------
optimization | 95
---------------------------------I fully agree with the value of compaction, but compaction also bring cpu consumption and will increase the time of
alloc_stall. So if we can let more free high-orders-pages in buddy instead of signal pages, it will decreaseCOMPACT_STALLand speed up memory allocation.
Okay, but then I assume the target goal of your patch set is to minimize CPU consumption/allocation stall time when allocating larger order pages.
Currently you state "the probablity of high-order-pages allocation would be increased significantly", but I assume that's then not 100% correct. What you measure is the stall time to allocate higher order pages, not that you can allocate them.
Quoted 11 lines
2. record the speed of the high-orders-pages allocation(order=4 and order = 8) Before and after optimization, we tested the speed of the high-orders-pages allocation after 120-hours-Monkey in 10 Android mobile phones. and the result show that the speed has been increased by more than 18%.
Also, we do some test designed by us:
(the machine with 4 gigabytes of physical memery and in linux-4.19.) model the usage of users, and constantly start and
operate the diffrent application for 120h, and we recordCOMPACT_STALLis decreased by 90+% and speed of the high-orders-pages is increaed by 15+%.
Okay, again, this is then some optimization for allocation speed; which makes it less attractive IMHO (at least for more invasive changes), because I suspect this mostly helps in corner cases (Monkey benchmarks corner cases AFAIU).
Quoted 13 lines
and I have some question, i hope you can guide me if when you are free.
1) What is the compaction configured?
Dost it meant the members in zone? like as follows:
unsigned intcompact_considered;
unsigned intcompact_defer_shift;
intcompact_order_failed;
boolcompact_blockskip_failed;
Or the some Macro variable? like as follows:
PAGE_ALLOC_COSTLY_ORDER= 3
MIN_COMPACT_PRIORITY= 1
MAX_COMPACT_RETRIES= 16
Rather if you have proactive compaction (/proc/sys/vm/compaction_proactiveness). But I assume because you're messing with older kernels, that you didn't compare against that yet. Would be worth a comparison.
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
1) multi freearea (which might
be problematic with sparcity)2) Can you pls tell me what is soarcity and what is the impact of this? and whether there are some documents about it?
Essentially CONFIG_SPARSEMEM, whereby we can have huge holes in physical memory layout and memory areas coming/going with memory hot(un)plug. Usually we manage all metadata per section. For example, pageblocks are allocated per section. We avoid arrays that depend on the initial/maximum physical memory size.
Signature · 3 lines
Thanks, David / dhildenb
Hi David Hildenbrand:
Quoted 7 lines
Quoted 7 lines
Okay, but then I assume the target goal of your patch set is to minimize CPU consumption/allocation stall time when allocating larger order pages.
Currently you state "the probablity of high-order-pages allocation would be increased significantly", but I assume that's then not 100% correct. What you measure is the stall time to allocate higher order pages, not that you can allocate them.
You are right that multi_freearea is to to speed the rate of large-order-pages allocation instead of increasing the probality of it if CONFIG_COMPACTION is open.
Quoted 4 lines
Quoted 4 lines
Okay, again, this is then some optimization for allocation speed; which makes it less attractive IMHO (at least for more invasive changes), because I suspect this mostly helps in corner cases (Monkey benchmarks corner cases AFAIU).
I really agree with the preciseness of the community that if we want to merge the patch in baseline, we would do more test to proof that the patch will not degraded to a degree that people care. But, maybe there are the detailed test-list for us to test, or we really don't know how to proof there are no any side-effects on all situation of memory-opt.
It is a question that has puzzled me for a long time.
Quoted 5 lines
Quoted 5 lines
Essentially
CONFIG_SPARSEMEM, whereby we can have huge holes in physical memory layout and memory areas coming/going with memory hot(un)plug. Usually we manage all metadata per section. For example, pageblocks are allocated per section. We avoid arrays that depend on the initial/maximum physical memory size.
CONFIG_SPRSEMEM has been opened in some of our product with Qcom-platform and MTK platform. AFAIK, multi_freearea would not bring problem to it?because the patch just manage the physical memory of zone to serveral section(free_area) and adjust the the range of pages-PFN for buddy-alloc-pages by the alloction-order. With memory hot(un)plug, we would initialize the members of "multi_freearea" in zone.
The patch has been merged in the baseline of our product that has been sold all over the world with Linux-4.4/4.9/4.19 so that i don't think there will be too much risk. Of course, i might be wrong.
From: David Hildenbrand
Date: 2021-04-27 20:46
To: [email protected]; Vlastimil Babka; peifengl55; schwidefsky; heiko.carstens; zhangshiming; zhouhuacai; guoweichao; guojian
CC: linux-s390; linux-kernel; linux-mm
Subject: Re: [RFC] mm: support multi_freearea to the reduction of external fragmentation
Quoted 7 lines · On 26.04.21 12:19, [email protected] wrote:
Hi David Hildenbrand <mailto:[email protected]>:
Quoted 5 lines
Quoted 5 lines
And you don't mention what the baseline configuration was. For example, how was compaction configured?
Just to clarify, what is monkey?
Monkey HTTP server? MonkeyTest disk benchmark? UI/Application Exerciser Monkey?
Quoted 6 lines
-------------------------------------------------------------------------------------
I am sorry that i didn't give a clear explanation about Monkey. It meant "UI/Application Exerciser Monkey" from google.Excuse me, let me introduce our test:
Thanks for more details on the test.
Quoted 12 lines
1. record
COMPACT_STALL
We tested the patch on linux-4.4/linux-4.9/linux-4.14/linux-4.19 and the results shows that the patch is effective in reducing COMPACTSTALL.
- monkey for 12 hours.
- record COMPACTSTALL after test.Test-result: reduced COMPACTSTALL by 95.6% with the patch. (the machine with 4 gigabytes of physical memery and in linux-4.19.)
---------------------------------
| COMPACTSTALL
---------------------------------ori | 2189
Quoted 8 lines
---------------------------------
optimization | 95
---------------------------------I fully agree with the value of compaction, but compaction also bring cpu consumption and will increase the time of
alloc_stall. So if we can let more free high-orders-pages in buddy instead of signal pages, it will decreaseCOMPACT_STALLand speed up memory allocation.
Okay, but then I assume the target goal of your patch set is to minimize CPU consumption/allocation stall time when allocating larger order pages.
Currently you state "the probablity of high-order-pages allocation would be increased significantly", but I assume that's then not 100% correct. What you measure is the stall time to allocate higher order pages, not that you can allocate them.
Quoted 8 lines
2. record the speed of the high-orders-pages allocation(order=4 and order = 8)
Before and after optimization, we tested the speed of the high-orders-pages allocation
after 120-hours-Monkey in 10 Android mobile phones. and the result show that the speed has been increased by more than 18%.
Quoted 6 lines
Also, we do some test designed by us:
(the machine with 4 gigabytes of physical memery and in linux-4.19.) model the usage of users, and constantly start and
operate the diffrent application for 120h, and we recordCOMPACT_STALLis decreased by
90+% and speed of the high-orders-pages is increaed by 15+%.
Okay, again, this is then some optimization for allocation speed; which makes it less attractive IMHO (at least for more invasive changes), because I suspect this mostly helps in corner cases (Monkey benchmarks corner cases AFAIU).
Quoted 13 lines
and I have some question, i hope you can guide me if when you are free.
1) What is the compaction configured?
Dost it meant the members in zone? like as follows:
unsigned intcompact_considered;
unsigned intcompact_defer_shift;
intcompact_order_failed;
boolcompact_blockskip_failed;
Or the some Macro variable? like as follows:
PAGE_ALLOC_COSTLY_ORDER= 3
MIN_COMPACT_PRIORITY= 1
MAX_COMPACT_RETRIES= 16
Rather if you have proactive compaction (/proc/sys/vm/compaction_proactiveness). But I assume because you're messing with older kernels, that you didn't compare against that yet. Would be worth a comparison.
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
1) multi freearea (which might
be problematic with sparcity)2) Can you pls tell me what is soarcity and what is the impact of this? and whether there are some documents about it?
Essentially CONFIG_SPARSEMEM, whereby we can have huge holes in physical memory layout and memory areas coming/going with memory hot(un)plug. Usually we manage all metadata per section. For example, pageblocks are allocated per section. We avoid arrays that depend on the initial/maximum physical memory size.
Signature · 3 lines
Thanks, David / dhildenb
Quoted 11 lines
Quoted 5 lines
Quoted 5 lines
Essentially
CONFIG_SPARSEMEM, whereby we can have huge holes in physical memory layout and memory areas coming/going with memory hot(un)plug. Usually we manage all metadata per section. For example, pageblocks are allocated per section. We avoid arrays that depend on the initial/maximum physical memory size.
CONFIG_SPRSEMEMhas been opened in some of our product with Qcom-platform and MTK platform. AFAIK,multi_freeareawould not bring problem to it?because the patch just manage the physical memory of zone to serveralsection(free_area)and adjust the the range of pages-PFN for buddy-alloc-pages by the alloction-order. With memory hot(un)plug, we would initialize the members of "multi_freearea" in zone.
From your description only I cannot tell how that would really work. Your description of 1) indicated that we are dealing with an array to manage memory segments, and arrays are a bad data structure when it comes to sparsity.
Quoted 4 lines
The patch has been merged in the baseline of our product that has been sold all over the world with Linux-4.4/4.9/4.19 so that i don't think there will be too much risk. Of course, i might be wrong.
Just always keep in mind that upstream Linux has a very broad community. What might be "good enough" for smartphones might not be well suited for servers, VMs, embedded devices, other archs ... just imagine the RAM size differences, sparse layout, dynamic memory changes, ...
Adding additional complexity to the buddy has to have a compelling benefit; keep in mind that any complexity we introduce has to be maintained in the long term.
Having that said, starting with small steps is IMHO the right approach.
Signature · 3 lines
Thanks, David / dhildenb
Hi David Hildenbrand:
Quoted 4 lines
Quoted 4 lines
From your description only I cannot tell how that would really work. Your description of 1) indicated that we are dealing with an array to manage memory segments, and arrays are a bad data structure when it comes to sparsity.
In the baseline, it also manage memory by array(zone->free_area) in linux, as follows:
struct zone {
......
struct free_area free_area[MAX_ORDER]; }
"Multi_free_area" would devide physical memory into serveral parts in zone by page-PFN, and it also uses free_area to manage each parts of memory:
struct zone {
......
struct free_area free_area[flc][MAX_ORDER]; // flc def to 4
}
All the logic of memory-opt is unchange experts for that: select the corresponding freearea to alloc-pages by allocation-order which is to concentrate most of low-order-pages(almost signal-page) allocation in the front area of physical memory and few memory-pollution in the back area of the memory. Theoretically, there will not be too much risk i think.
Because i am not a expert so that maybe there are some case i have not considered and maybe there are something wrong displeaed you in my email, pls don't take it ill.
Quoted 10 lines
Quoted 10 lines
Just always keep in mind that upstream Linux has a very broad community. What might be "good enough" for smartphones might not be well suited for servers, VMs, embedded devices, other archs ... just imagine the RAM size differences, sparse layout, dynamic memory changes, ...
Adding additional complexity to the buddy has to have a compelling benefit; keep in mind that any complexity we introduce has to be maintained in the long term.
Having that said, starting with small steps is IMHO the right approach.
I will make the patch-series and try to make the patch smaller and readable and thanks your advices indeed.
From: David Hildenbrand
Date: 2021-04-28 17:04
To: [email protected]; Vlastimil Babka; peifengl55; schwidefsky; heiko.carstens; zhangshiming; zhouhuacai; guoweichao; guojian
CC: linux-s390; linux-kernel; linux-mm
Subject: Re: [RFC] mm: support multi_freearea to the reduction of external fragmentation
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
Essentially
CONFIG_SPARSEMEM, whereby we can have huge holes in physical memory layout and memory areas coming/going with memory hot(un)plug. Usually we manage all metadata per section. For example, pageblocks are allocated per section. We avoid arrays that depend on the initial/maximum physical memory size.
Quoted 9 lines
CONFIG_SPRSEMEMhas been opened in some of our product with Qcom-platform and
MTK platform. AFAIK,multi_freeareawould not bring problem to it?because the patch
just manage the physical memory of zone to serveralsection(free_area)and adjust the
the range of pages-PFN for buddy-alloc-pages by the alloction-order. With memory
hot(un)plug, we would initialize the members of "multi_freearea" in zone.
From your description only I cannot tell how that would really work. Your description of 1) indicated that we are dealing with an array to manage memory segments, and arrays are a bad data structure when it comes to sparsity.
Quoted 6 lines
The patch has been merged in the baseline of our product that has been sold all over the
world with Linux-4.4/4.9/4.19 so that i don't think there will be too much risk. Of course,
i might be wrong.
Just always keep in mind that upstream Linux has a very broad community. What might be "good enough" for smartphones might not be well suited for servers, VMs, embedded devices, other archs ... just imagine the RAM size differences, sparse layout, dynamic memory changes, ...
Adding additional complexity to the buddy has to have a compelling benefit; keep in mind that any complexity we introduce has to be maintained in the long term.
Having that said, starting with small steps is IMHO the right approach.
Signature · 3 lines
Thanks, David / dhildenb