change the implementation of the PageHighMem()
Subsystems (15)
v2 cover change the implementation of the PageHighMem()
From: Joonsoo Kim <[email protected]>
Changes on v2
- add "acked-by", "reviewed-by" tags
- replace PageHighMem() with use open-code, instead of using new PageHighMemZone() macro. Related file is "include/linux/migrate.h"
Hello,
This patchset separates two use cases of PageHighMem() by introducing PageHighMemZone() macro. And, it changes the implementation of PageHighMem() to reflect the actual meaning of this macro. This patchset is a preparation step for the patchset,
"mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE" [1].
PageHighMem() is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.
Until now, both the cases are the perfectly same thing. So, implementation of the PageHighMem() uses the one case that checks if the zone of the page is the highmem type zone or not.
"#define PageHighMem(__p) is_highmem_idx(page_zonenum(__p))"
ZONE_MOVABLE is special. It is considered as normal type zone on !CONFIG_HIGHMEM, but, it is considered as highmem type zone on CONFIG_HIGHMEM. Let's focus on later case. In later case, all pages on the ZONE_MOVABLE has no direct mapping until now.
However, following patchset
"mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE" , which is once merged and reverted, will be tried again and will break this assumption that all pages on the ZONE_MOVABLE has no direct mapping. Hence, the ZONE_MOVABLE which is considered as highmem type zone could have the both types of pages, direct mapped and not. Since the ZONE_MOVABLE could have both type of pages, __GFP_HIGHMEM is still required to allocate the memory from it. And, we conservatively need to consider the ZONE_MOVABLE as highmem type zone.
Even in this situation, PageHighMem() for the pages on the ZONE_MOVABLE when it is called for checking the direct mapping should return correct result. Current implementation of PageHighMem() just returns TRUE if the zone of the page is on a highmem type zone. So, it could be wrong if the page on the MOVABLE_ZONE is actually direct mapped.
To solve this potential problem, this patch introduces a new PageHighMemZone() macro. In following patches, two use cases of PageHighMem() are separated by calling proper macro, PageHighMem() and PageHighMemZone(). Then, implementation of PageHighMem() will be changed as just checking if the direct mapping exists or not, regardless of the zone of the page.
Note that there are some rules to determine the proper macro.
1. If PageHighMem() is called for checking if the direct mapping exists or not, use PageHighMem().
2. If PageHighMem() is used to predict the previous gfp_flags for this page, use PageHighMemZone(). The zone of the page is related to the gfp_flags.
3. If purpose of calling PageHighMem() is to count highmem page and to interact with the system by using this count, use PageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, use PageHighMemZone(). It's safe since it's implementation is just copy of the previous PageHighMem() implementation and won't be changed.
My final plan is to change the name, PageHighMem() to PageNoDirectMapped() or something else in order to represent proper meaning.
This patchset is based on next-20200428 and you can find the full patchset on the following link.
https://github.com/JoonsooKim/linux/tree/page_highmem-cleanup-v2.00-next-20200428
Thanks.
[1]: https://lore.kernel.org/linux-mm/[email protected]
Joonsoo Kim (10):
mm/page-flags: introduce PageHighMemZone()
drm/ttm: separate PageHighMem() and PageHighMemZone() use case
kexec: separate PageHighMem() and PageHighMemZone() use case
power: separate PageHighMem() and PageHighMemZone() use case mm/gup: separate PageHighMem() and PageHighMemZone() use case mm/hugetlb: separate PageHighMem() and PageHighMemZone() use case
mm: separate PageHighMem() and PageHighMemZone() use case mm/page_alloc: correct the use of is_highmem_idx() mm/migrate: replace PageHighMem() with open-code mm/page-flags: change the implementation of the PageHighMem()
drivers/gpu/drm/ttm/ttm_memory.c | 4 ++-- drivers/gpu/drm/ttm/ttm_page_alloc.c | 2 +- drivers/gpu/drm/ttm/ttm_page_alloc_dma.c | 2 +- drivers/gpu/drm/ttm/ttm_tt.c | 2 +- include/linux/migrate.h | 4 +++- include/linux/page-flags.h | 10 +++++++++- kernel/kexec_core.c | 2 +- kernel/power/snapshot.c | 12 ++++++------ mm/gup.c | 2 +- mm/hugetlb.c | 2 +- mm/memory_hotplug.c | 2 +- mm/page_alloc.c | 4 ++-- 12 files changed, 29 insertions(+), 19 deletions(-)
Signature · 1 line
2.7.4
PATCH v2 1/10 mm/page-flags: introduce PageHighMemZone()
From: Joonsoo Kim <[email protected]>
PageHighMem() is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.
Until now, both the cases are the perfectly same thing. So, implementation of the PageHighMem() uses the one case that checks if the zone of the page is the highmem type zone or not.
"#define PageHighMem(__p) is_highmem_idx(page_zonenum(__p))"
ZONE_MOVABLE is special. It is considered as normal type zone on !CONFIG_HIGHMEM, but, it is considered as highmem type zone on CONFIG_HIGHMEM. Let's focus on later case. In later case, all pages on the ZONE_MOVABLE has no direct mapping until now.
However, following patchset
"mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE" , which is once merged and reverted, will be tried again and will break this assumption that all pages on the ZONE_MOVABLE has no direct mapping. Hence, the ZONE_MOVABLE which is considered as highmem type zone could have the both types of pages, direct mapped and not. Since the ZONE_MOVABLE could have both type of pages, __GFP_HIGHMEM is still required to allocate the memory from it. And, we conservatively need to consider the ZONE_MOVABLE as highmem type zone.
Even in this situation, PageHighMem() for the pages on the ZONE_MOVABLE when it is called for checking the direct mapping should return correct result. Current implementation of PageHighMem() just returns TRUE if the zone of the page is on a highmem type zone. So, it could be wrong if the page on the MOVABLE_ZONE is actually direct mapped.
To solve this potential problem, this patch introduces a new PageHighMemZone() macro. In following patches, two use cases of PageHighMem() are separated by calling proper macro, PageHighMem() and PageHighMemZone(). Then, implementation of PageHighMem() will be changed as just checking if the direct mapping exists or not, regardless of the zone of the page.
Note that there are some rules to determine the proper macro.
1. If PageHighMem() is called for checking if the direct mapping exists or not, use PageHighMem().
2. If PageHighMem() is used to predict the previous gfp_flags for this page, use PageHighMemZone(). The zone of the page is related to the gfp_flags.
3. If purpose of calling PageHighMem() is to count highmem page and to interact with the system by using this count, use PageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, use PageHighMemZone(). It's safe since it's implementation is just copy of the previous PageHighMem() implementation and won't be changed.
My final plan is to change the name, PageHighMem() to PageNoDirectMapped() or something else in order to represent proper meaning.
--- include/linux/page-flags.h | 6 ++++++ 1 file changed, 6 insertions(+)
include/linux/page-flags.h +6 -0
@@ -378,10 +378,16 @@ PAGEFLAG(Readahead, reclaim, PF_NO_COMPOUND)/* * Must use a macro here due to header dependency issues. page_zone() is not * available at this point. +* PageHighMem() is for checking if the direct mapping exists or not. +* PageHighMemZone() is for checking the zone, where the page is belong to, +* in order to predict previous gfp_flags or to count something for system +* memory management. */ #define PageHighMem (__p) is_highmem_idx(page_zonenum(__p)) +#define PageHighMemZone (__p) is_highmem_idx(page_zonenum(__p))#else PAGEFLAG_FALSE(HighMem) +PAGEFLAG_FALSE(HighMemZone)#endif #ifdef CONFIG_SWAP
Signature · 1 line
2.7.4
PATCH v2 2/10 drm/ttm: separate PageHighMem() and PageHighMemZone() use case
In reply to [email protected]From: Joonsoo Kim <[email protected]>
Until now, PageHighMem() is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.
Now, we have separate functions, PageHighMem() and PageHighMemZone() for each cases. Use appropriate one.
Note that there are some rules to determine the proper macro.
1. If PageHighMem() is called for checking if the direct mapping exists or not, use PageHighMem().
2. If PageHighMem() is used to predict the previous gfp_flags for this page, use PageHighMemZone(). The zone of the page is related to the gfp_flags.
3. If purpose of calling PageHighMem() is to count highmem page and to interact with the system by using this count, use PageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, use PageHighMemZone(). It's safe since it's implementation is just copy of the previous PageHighMem() implementation and won't be changed.
I apply the rule #4 for this patch.
--- drivers/gpu/drm/ttm/ttm_memory.c | 4 ++-- drivers/gpu/drm/ttm/ttm_page_alloc.c | 2 +- drivers/gpu/drm/ttm/ttm_page_alloc_dma.c | 2 +- drivers/gpu/drm/ttm/ttm_tt.c | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-)
drivers/gpu/drm/ttm/ttm_memory.c +2 -2
@@ -641,7 +641,7 @@ int ttm_mem_global_alloc_page(struct ttm_mem_global *glob,*/ #ifdef CONFIG_HIGHMEM -if (PageHighMem (page) && glob->zone_highmem !=NULL ) +if (PageHighMemZone (page) && glob->zone_highmem !=NULL ) zone = glob->zone_highmem;#else if (glob->zone_dma32 &&page_to_pfn (page) >0x00100000UL )
@@ -656,7 +656,7 @@ void ttm_mem_global_free_page(struct ttm_mem_global *glob, struct page *page,struct ttm_mem_zone *zone =NULL ;#ifdef CONFIG_HIGHMEM -if (PageHighMem (page) && glob->zone_highmem !=NULL ) +if (PageHighMemZone (page) && glob->zone_highmem !=NULL ) zone = glob->zone_highmem;#else if (glob->zone_dma32 &&page_to_pfn (page) >0x00100000UL )
drivers/gpu/drm/ttm/ttm_page_alloc.c +1 -1
@@ -530,7 +530,7 @@ static int ttm_alloc_new_pages(struct list_head *pages, gfp_t gfp_flags,/* gfp flags of highmem page should never be dma32 so we * we should be fine in such case */ -if (PageHighMem (p)) +if (PageHighMemZone (p))continue ;#endif
drivers/gpu/drm/ttm/ttm_page_alloc_dma.c +1 -1
@@ -747,7 +747,7 @@ static int ttm_dma_pool_alloc_new_pages(struct dma_pool *pool,/* gfp flags of highmem page should never be dma32 so we * we should be fine in such case */ -if (PageHighMem (p)) +if (PageHighMemZone (p))continue ;#endif
drivers/gpu/drm/ttm/ttm_tt.c +1 -1
@@ -119,7 +119,7 @@ static int ttm_tt_set_page_caching(struct page *p, {int ret =0 ; -if (PageHighMem (p)) +if (PageHighMemZone (p))return 0 ;if (c_old != tt_cached) {
Signature · 1 line
2.7.4
PATCH v2 3/10 kexec: separate PageHighMem() and PageHighMemZone() use case
In reply to [email protected]From: Joonsoo Kim <[email protected]>
Until now, PageHighMem() is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.
Now, we have separate functions, PageHighMem() and PageHighMemZone() for each cases. Use appropriate one.
Note that there are some rules to determine the proper macro.
1. If PageHighMem() is called for checking if the direct mapping exists or not, use PageHighMem().
2. If PageHighMem() is used to predict the previous gfp_flags for this page, use PageHighMemZone(). The zone of the page is related to the gfp_flags.
3. If purpose of calling PageHighMem() is to count highmem page and to interact with the system by using this count, use PageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, use PageHighMemZone(). It's safe since it's implementation is just copy of the previous PageHighMem() implementation and won't be changed.
I apply the rule #2 for this patch.
--- kernel/kexec_core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/kexec_core.c +1 -1
@@ -766,7 +766,7 @@ static struct page *kimage_alloc_page(struct kimage *image,* gfp_flags honor the ones passed in. */ if (!(gfp_mask & __GFP_HIGHMEM) && -PageHighMem (old_page)) { +PageHighMemZone (old_page)) {kimage_free_pages (old_page);continue ; }
Signature · 1 line
2.7.4
Re: [PATCH v2 03/10] kexec: separate PageHighMem() and PageHighMemZone() use case
Quoted 25 lines · [email protected] writes:
From: Joonsoo Kim <[email protected]>
Until now,
PageHighMem()is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.Now, we have separate functions,
PageHighMem()andPageHighMemZone()for each cases. Use appropriate one.Note that there are some rules to determine the proper macro.
1. If
PageHighMem()is called for checking if the direct mapping exists or not, usePageHighMem().
2. IfPageHighMem()is used to predict the previousgfp_flagsfor this page, usePageHighMemZone(). The zone of the page is related to thegfp_flags.
3. If purpose of callingPageHighMem()is to count highmem page and to interact with the system by using this count, usePageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, usePageHighMemZone(). It's safe since it's implementation is just copy of the previousPageHighMem()implementation and won't be changed.
Quoted 1 line
I apply the rule #2 for this patch.
Hmm.
What happened to the notion of deprecating and reducing the usage of highmem? I know that we have some embedded architectures where it is still important but this feels like it flies in the face of that.
This part of kexec would be much more maintainable if it had a proper mm layer helper that tested to see if the page matched the passed in gfp flags. That way the mm layer could keep changing and doing weird gyrations and this code would not care.
What would be really helpful is if there was a straight forward way to allocate memory whose physical address fits in the native word size.
All I know for certain about this patch is that it takes a piece of code that looked like it made sense, and transfroms it into something I can not easily verify, and can not maintain.
As it makes the code unmaintainable.
Not to say that the code isn't questionable as it is, but this change just pushes it over the edge into gobbledy gook.
Eric
Quoted 19 lines
Acked-by Roman Gushchin <[email protected]>Signed-off-by Joonsoo Kim <[email protected]>--- kernel/kexec_core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)kernel/kexec_core.c +1
-1@@ -766,7 +766,7 @@ static struct page *kimage_alloc_page(struct kimage *image,* gfp_flags honor the ones passed in. */ if (!(gfp_mask & __GFP_HIGHMEM) && -PageHighMem (old_page)) { +PageHighMemZone (old_page)) {kimage_free_pages (old_page);continue ; }
Re: [PATCH v2 03/10] kexec: separate PageHighMem() and PageHighMemZone() use case
2020년 5월 1일 (금) 오후 11:06, Eric W. Biederman <[email protected]>님이 작성:
Quoted 28 lines
Quoted 25 lines · [email protected] writes:
From: Joonsoo Kim <[email protected]>
Until now,
PageHighMem()is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.Now, we have separate functions,
PageHighMem()andPageHighMemZone()for each cases. Use appropriate one.Note that there are some rules to determine the proper macro.
1. If
PageHighMem()is called for checking if the direct mapping exists or not, usePageHighMem().
2. IfPageHighMem()is used to predict the previousgfp_flagsfor this page, usePageHighMemZone(). The zone of the page is related to thegfp_flags.
3. If purpose of callingPageHighMem()is to count highmem page and to interact with the system by using this count, usePageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, usePageHighMemZone(). It's safe since it's implementation is just copy of the previousPageHighMem()implementation and won't be changed.
Quoted 7 lines
Quoted 1 line
I apply the rule #2 for this patch.
Hmm.
What happened to the notion of deprecating and reducing the usage of highmem? I know that we have some embedded architectures where it is still important but this feels like it flies in the face of that.
AFAIK, deprecating highmem requires some more time and, before then, we need to support it.
Quoted 5 lines
This part of kexec would be much more maintainable if it had a proper mm layer helper that tested to see if the page matched the passed in gfp flags. That way the mm layer could keep changing and doing weird gyrations and this code would not care.
Good idea! I will do it.
Quoted 8 lines
What would be really helpful is if there was a straight forward way to allocate memory whose physical address fits in the native word size.
All I know for certain about this patch is that it takes a piece of code that looked like it made sense, and transfroms it into something I can not easily verify, and can not maintain.
Although I decide to make a helper as you described above, I don't understand why you think that a new code isn't maintainable. It is just the same thing with different name. Could you elaborate more why do you think so?
Thanks.
Re: [PATCH v2 03/10] kexec: separate PageHighMem() and PageHighMemZone() use case
I have added in the kexec mailling list.
Looking at the patch we are discussing it appears that the kexec code could be doing much better in highmem situations today but is not.
Quoted 31 lines · Joonsoo Kim
2020년 5월 1일 (금) 오후 11:06, Eric W. Biederman <[email protected]>님이 작성:
Quoted 30 lines
Quoted 26 lines · [email protected] writes:
From: Joonsoo Kim <[email protected]>
Until now,
PageHighMem()is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.Now, we have separate functions,
PageHighMem()andPageHighMemZone()for each cases. Use appropriate one.Note that there are some rules to determine the proper macro.
1. If
PageHighMem()is called for checking if the direct mapping exists or not, usePageHighMem().
2. IfPageHighMem()is used to predict the previousgfp_flagsfor this page, usePageHighMemZone(). The zone of the page is related to thegfp_flags.
3. If purpose of callingPageHighMem()is to count highmem page and to interact with the system by using this count, usePageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, usePageHighMemZone(). It's safe since it's implementation is just copy of the previousPageHighMem()implementation and won't be changed.I apply the rule #2 for this patch.
Quoted 8 lines
Quoted 5 lines
Hmm.
What happened to the notion of deprecating and reducing the usage of highmem? I know that we have some embedded architectures where it is still important but this feels like it flies in the face of that.
AFAIK, deprecating highmem requires some more time and, before then, we need to support it.
But it at least makes sense to look at what we are doing with highmem and ask if it makes sense.
Quoted 12 lines
Quoted 4 lines
This part of kexec would be much more maintainable if it had a proper mm layer helper that tested to see if the page matched the passed in gfp flags. That way the mm layer could keep changing and doing weird gyrations and this code would not care.
Good idea! I will do it.
Quoted 5 lines
What would be really helpful is if there was a straight forward way to allocate memory whose physical address fits in the native word size.
Quoted 8 lines
Quoted 3 lines
All I know for certain about this patch is that it takes a piece of code that looked like it made sense, and transfroms it into something I can not easily verify, and can not maintain.
Although I decide to make a helper as you described above, I don't understand why you think that a new code isn't maintainable. It is just the same thing with different name. Could you elaborate more why do you think so?
Because the current code is already wrong. It does not handle the general case of what it claims to handle. When the only distinction that needs to be drawn is highmem or not highmem that is likely fine. But now you are making it possible to draw more distinctions. At which point I have no idea which distinction needs to be drawn.
The code and the logic is about 20 years old. When it was written I don't recally taking numa seriously and the kernel only had 3 zones as I recall (DMA aka the now deprecated GFP_DMA, NORMAL, and HIGH).
The code attempts to work around limitations of those old zones amd play nice in a highmem world by allocating memory HIGH memory and not using it if the memory was above 4G ( on 32bit ).
Looking the kernel now has GFP_DMA32 so on 32bit with highmem we should probably be using that, when allocating memory.
Further in dealing with this memory management situation we only have two situations we call kimage_alloc_page.
For an indirect page which must have a valid page_address(page). We could probably relax that if we cared to.
For a general kexec page to store the next kernel in until we switch. The general pages can be in high memory.
In a highmem world all of those pages should be below 32bit.
Given that we fundamentally have two situations my sense is that we should just refactor the code so that we never have to deal with:
/* The old page I have found cannot be a
* destination page, so return it if it's
* gfp_flags honor the ones passed in.
*/
if (!(gfp_mask & __GFP_HIGHMEM) &&
PageHighMem(old_page)) {
kimage_free_pages(old_page);
continue;
}Either we teach kimage_add_entry how to work with high memory pages (still 32bit accessible) or we teach kimage_alloc_page to notice it is an indirect page allocation and to always skip trying to reuse the page it found in that case.
That way the code does not need to know about forever changing mm internals.
We should probably investigate GFP_DMA32 at the same time, and switch to that for 32bit rather than continuing to use GFP_HIGHUSER.
Eric
RFC PATCH 1/1 kexec: Teach indirect pages how to live in high memory
Recently a patch was proposed to kimage_alloc_page to slightly alter the logic of how pages allocated with incompatible flags were detected. The logic was being altered because the semantics of the page alloctor were changing yet again.
Looking at that case I realized that there is no reason for it to even exist. Either the indirect page allocations and the source page allocations could be separated out, or I could do as I am doing now and simply teach the indirect pages to live in high memory.
This patch replaced pointers of type kimage_entry_t * with a new type kimage_entry_pos_t. This new type holds the physical address of the indirect page and the offset within that page of the next indirect entry to write. A special constant KIMAGE_ENTRY_POS_INVALID is added that kimage_image_pos_t variables that don't currently have a valid may be set to.
Two new functions kimage_read_entry and kimage_write_entry have been provided to write entries in way that works if they live in high memory.
The now unnecessary checks to see if a destination entry is non-zero and to increment it if so have been removed. For safety new indrect pages are now cleared so we have a guarantee everything that has not been used yet is zero. Along with this writing an extra trailing 0 entry has been removed, as it is known all trailing entries are now 0.
With highmem support implemented for indirect pages kimage_image_alloc_page has been updated to always allocate GFP_HIGHUSER pages, and handling of pages with different gfp flags has been removed.
---
I have not done more than compile test this but I think this will remove that tricky case in the kexec highmem support.
Any comments? Does anyone have a 32bit highmem system where they can test this code? I can probably do something with a 32bit x86 kernel but it has been a few days.
Does anyone know how we can more effectively allocate memory below whatever the maximum limit that kexec supports? Typically below 4G on 32bit and below 2^64 on 64bits.
Eric
include/linux/kexec.h | 5 +- kernel/kexec_core.c | 119 +++++++++++++++++++++++++----------------- 2 files changed, 73 insertions(+), 51 deletions(-)
include/linux/kexec.h +3 -2
@@ -69,6 +69,8 @@*/ typedef unsigned long kimage_entry_t ; +typedef unsigned long kimage_entry_pos_t ; +#define KIMAGE_ENTRY_POS_INVALID ((kimage_entry_pos_t)-2)struct kexec_segment {/*
@@ -243,8 +245,7 @@ int kexec_elf_probe(const char *buf, unsigned long len);#endif struct kimage {kimage_entry_t head; -kimage_entry_t *entry; -kimage_entry_t *last_entry; +kimage_entry_pos_t entry_pos;unsigned long start;struct page *control_code_page;
kernel/kexec_core.c +70 -49
214 lines not on this page · raw message
Signature · 1 line
2.25.0
Re: [RFC][PATCH] kexec: Teach indirect pages how to live in high memory
Quoted 28 lines · Eric W. Biederman
Recently a patch was proposed to
kimage_alloc_pageto slightly alter the logic of how pages allocated with incompatible flags were detected. The logic was being altered because the semantics of the page alloctor were changing yet again.Looking at that case I realized that there is no reason for it to even exist. Either the indirect page allocations and the source page allocations could be separated out, or I could do as I am doing now and simply teach the indirect pages to live in high memory.
This patch replaced pointers of type
kimage_entry_t* with a new typekimage_entry_pos_t. This new type holds the physical address of the indirect page and the offset within that page of the next indirect entry to write. A special constantKIMAGE_ENTRY_POS_INVALIDis added thatkimage_image_pos_tvariables that don't currently have a valid may be set to.Two new functions
kimage_read_entryandkimage_write_entryhave been provided to write entries in way that works if they live in high memory.The now unnecessary checks to see if a destination entry is non-zero and to increment it if so have been removed. For safety new indrect pages are now cleared so we have a guarantee everything that has not been used yet is zero. Along with this writing an extra trailing 0 entry has been removed, as it is known all trailing entries are now 0.
Quoted 6 lines
With highmem support implemented for indirect pages
kimage_image_alloc_pagehas been updated to always allocateGFP_HIGHUSERpages, and handling of pages with different gfp flags has been removed.Signed-off-by "Eric W. Biederman" <[email protected]>
Eric, the patch failed with data access exception on ppc64. Using the below patch on top got me going...
kernel/kexec_core.c +11 -2
@@ -570,7 +570,12 @@ static int kimage_add_entry(struct kimage *image, kimage_entry_t entry)return -ENOMEM ; ind_addr =page_to_boot_pfn (page) <<PAGE_SHIFT ; -kimage_write_entry (image->entry_pos, ind_addr |IND_INDIRECTION ); + +/* If it is the first entry, handle it here */ +if (!image->head) + image->head = ind_addr |IND_INDIRECTION ; +else +kimage_write_entry (image->entry_pos, ind_addr |IND_INDIRECTION );clear_highpage (page);
@@ -623,7 +628,11 @@ int __weak machine_kexec_post_load(struct kimage *image)void kimage_terminate (struct kimage *image) { -kimage_write_entry (image->entry_pos,IND_DONE ); +/* This could be the only entry in case of kdump */ +if (!image->head) + image->head =IND_DONE ; +else +kimage_write_entry (image->entry_pos,IND_DONE ); }#define for_each_kimage_entry (image, pos, entry) \
Thanks
Hari
Re: [RFC][PATCH] kexec: Teach indirect pages how to live in high memory
Quoted 34 lines · Hari Bathini
Quoted 33 lines · Eric W. Biederman
Recently a patch was proposed to
kimage_alloc_pageto slightly alter the logic of how pages allocated with incompatible flags were detected. The logic was being altered because the semantics of the page alloctor were changing yet again.Looking at that case I realized that there is no reason for it to even exist. Either the indirect page allocations and the source page allocations could be separated out, or I could do as I am doing now and simply teach the indirect pages to live in high memory.
This patch replaced pointers of type
kimage_entry_t* with a new typekimage_entry_pos_t. This new type holds the physical address of the indirect page and the offset within that page of the next indirect entry to write. A special constantKIMAGE_ENTRY_POS_INVALIDis added thatkimage_image_pos_tvariables that don't currently have a valid may be set to.Two new functions
kimage_read_entryandkimage_write_entryhave been provided to write entries in way that works if they live in high memory.The now unnecessary checks to see if a destination entry is non-zero and to increment it if so have been removed. For safety new indrect pages are now cleared so we have a guarantee everything that has not been used yet is zero. Along with this writing an extra trailing 0 entry has been removed, as it is known all trailing entries are now 0.
With highmem support implemented for indirect pages
kimage_image_alloc_pagehas been updated to always allocateGFP_HIGHUSERpages, and handling of pages with different gfp flags has been removed.
Quoted 4 lines
Quoted 1 line
Signed-off-by "Eric W. Biederman" <[email protected]>Eric, the patch failed with data access exception on ppc64. Using the below patch on top got me going...
Doh! Somehow I thought I had put that logic or something equivalent into kimage_write_entry and it appears I did not. I will see if I can respin the patch.
Thank you very much for testing.
Eric
Quoted 35 lines
kernel/kexec_core.c +11
-2@@ -570,7 +570,12 @@ static int kimage_add_entry(struct kimage *image, kimage_entry_t entry)return -ENOMEM ; ind_addr =page_to_boot_pfn (page) <<PAGE_SHIFT ; -kimage_write_entry (image->entry_pos, ind_addr |IND_INDIRECTION ); + +/* If it is the first entry, handle it here */ +if (!image->head) + image->head = ind_addr |IND_INDIRECTION ; +else +kimage_write_entry (image->entry_pos, ind_addr |IND_INDIRECTION );clear_highpage (page);@@ -623,7 +628,11 @@ int __weak machine_kexec_post_load(struct kimage *image)void kimage_terminate (struct kimage *image) { -kimage_write_entry (image->entry_pos,IND_DONE ); +/* This could be the only entry in case of kdump */ +if (!image->head) + image->head =IND_DONE ; +else +kimage_write_entry (image->entry_pos,IND_DONE ); }#define for_each_kimage_entry (image, pos, entry) \Thanks
Hari
Re: [RFC][PATCH] kexec: Teach indirect pages how to live in high memory
Quoted 38 lines · Eric W. Biederman
Quoted 36 lines · Hari Bathini
Quoted 34 lines · Eric W. Biederman
Recently a patch was proposed to
kimage_alloc_pageto slightly alter the logic of how pages allocated with incompatible flags were detected. The logic was being altered because the semantics of the page alloctor were changing yet again.Looking at that case I realized that there is no reason for it to even exist. Either the indirect page allocations and the source page allocations could be separated out, or I could do as I am doing now and simply teach the indirect pages to live in high memory.
This patch replaced pointers of type
kimage_entry_t* with a new typekimage_entry_pos_t. This new type holds the physical address of the indirect page and the offset within that page of the next indirect entry to write. A special constantKIMAGE_ENTRY_POS_INVALIDis added thatkimage_image_pos_tvariables that don't currently have a valid may be set to.Two new functions
kimage_read_entryandkimage_write_entryhave been provided to write entries in way that works if they live in high memory.The now unnecessary checks to see if a destination entry is non-zero and to increment it if so have been removed. For safety new indrect pages are now cleared so we have a guarantee everything that has not been used yet is zero. Along with this writing an extra trailing 0 entry has been removed, as it is known all trailing entries are now 0.
With highmem support implemented for indirect pages
kimage_image_alloc_pagehas been updated to always allocateGFP_HIGHUSERpages, and handling of pages with different gfp flags has been removed.Signed-off-by "Eric W. Biederman" <[email protected]>
Quoted 8 lines
Quoted 2 lines
Eric, the patch failed with data access exception on ppc64. Using the below patch on top got me going...
Doh! Somehow I thought I had put that logic or something equivalent into
kimage_write_entryand it appears I did not. I will see if I can respin the patch.Thank you very much for testing.
Hello, Eric.
It seems that this patch isn't upstreamed.
Could you respin the patch?
I've tested this one on x86_32 (highmem enabled) and it works well.
Thanks.
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH v2 03/10] kexec: separate PageHighMem() and PageHighMemZone() use case
In reply to Eric W. BiedermanQuoted 5 lines · Eric W. Biederman
I have added in the kexec mailling list.
Looking at the patch we are discussing it appears that the kexec code could be doing much better in highmem situations today but is not.
Sound great!
Quoted 79 lines
Quoted 31 lines · Joonsoo Kim
2020년 5월 1일 (금) 오후 11:06, Eric W. Biederman <[email protected]>님이 작성:
Quoted 30 lines
Quoted 26 lines · [email protected] writes:
From: Joonsoo Kim <[email protected]>
Until now,
PageHighMem()is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.Now, we have separate functions,
PageHighMem()andPageHighMemZone()for each cases. Use appropriate one.Note that there are some rules to determine the proper macro.
1. If
PageHighMem()is called for checking if the direct mapping exists or not, usePageHighMem().
2. IfPageHighMem()is used to predict the previousgfp_flagsfor this page, usePageHighMemZone(). The zone of the page is related to thegfp_flags.
3. If purpose of callingPageHighMem()is to count highmem page and to interact with the system by using this count, usePageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, usePageHighMemZone(). It's safe since it's implementation is just copy of the previousPageHighMem()implementation and won't be changed.I apply the rule #2 for this patch.
Quoted 8 lines
Quoted 5 lines
Hmm.
What happened to the notion of deprecating and reducing the usage of highmem? I know that we have some embedded architectures where it is still important but this feels like it flies in the face of that.
AFAIK, deprecating highmem requires some more time and, before then, we need to support it.
But it at least makes sense to look at what we are doing with highmem and ask if it makes sense.
Quoted 12 lines
Quoted 4 lines
This part of kexec would be much more maintainable if it had a proper mm layer helper that tested to see if the page matched the passed in gfp flags. That way the mm layer could keep changing and doing weird gyrations and this code would not care.
Good idea! I will do it.
Quoted 5 lines
What would be really helpful is if there was a straight forward way to allocate memory whose physical address fits in the native word size.
Quoted 8 lines
Quoted 3 lines
All I know for certain about this patch is that it takes a piece of code that looked like it made sense, and transfroms it into something I can not easily verify, and can not maintain.
Although I decide to make a helper as you described above, I don't understand why you think that a new code isn't maintainable. It is just the same thing with different name. Could you elaborate more why do you think so?
Because the current code is already wrong. It does not handle the general case of what it claims to handle. When the only distinction that needs to be drawn is highmem or not highmem that is likely fine. But now you are making it possible to draw more distinctions. At which point I have no idea which distinction needs to be drawn.
The code and the logic is about 20 years old. When it was written I don't recally taking numa seriously and the kernel only had 3 zones as I recall (DMA aka the now deprecated
GFP_DMA, NORMAL, and HIGH).
Quoted 7 lines
The code attempts to work around limitations of those old zones amd play nice in a highmem world by allocating memory HIGH memory and not using it if the memory was above 4G ( on 32bit ).
Looking the kernel now has
GFP_DMA32so on 32bit with highmem we should probably be using that, when allocating memory.
From quick investigation, unfortunately, ZONE_DMA32 isn't available on x86 32bit now so using GFP_DMA32 to allocate memory below 4G would not work. Enabling ZONE_DMA32 on x86 32bit would be not simple, so, IMHO, it would be better to leave the code as it is.
Quoted 29 lines
Further in dealing with this memory management situation we only have two situations we call
kimage_alloc_page.For an indirect page which must have a valid
page_address(page). We could probably relax that if we cared to.For a general kexec page to store the next kernel in until we switch. The general pages can be in high memory.
In a highmem world all of those pages should be below 32bit.
Given that we fundamentally have two situations my sense is that we should just refactor the code so that we never have to deal with:
/* The old page I have found cannot be a * destination page, so return it if it's * gfp_flags honor the ones passed in. */ if (!(gfp_mask & __GFP_HIGHMEM) && PageHighMem(old_page)) { kimage_free_pages(old_page); continue; }
Quoted 6 lines
Either we teach
kimage_add_entryhow to work with high memory pages (still 32bit accessible) or we teachkimage_alloc_pageto notice it is an indirect page allocation and to always skip trying to reuse the page it found in that case.That way the code does not need to know about forever changing mm internals.
Nice! I already have seen your patch and found that above two lines related to HIGHMEM are removed. Thanks for your help.
Thanks.
PATCH v2 4/10 power: separate PageHighMem() and PageHighMemZone() use case
In reply to [email protected]From: Joonsoo Kim <[email protected]>
Until now, PageHighMem() is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.
Now, we have separate functions, PageHighMem() and PageHighMemZone() for each cases. Use appropriate one.
Note that there are some rules to determine the proper macro.
1. If PageHighMem() is called for checking if the direct mapping exists or not, use PageHighMem().
2. If PageHighMem() is used to predict the previous gfp_flags for this page, use PageHighMemZone(). The zone of the page is related to the gfp_flags.
3. If purpose of calling PageHighMem() is to count highmem page and to interact with the system by using this count, use PageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, use PageHighMemZone(). It's safe since it's implementation is just copy of the previous PageHighMem() implementation and won't be changed.
I apply the rule #3 for this patch.
--- kernel/power/snapshot.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
kernel/power/snapshot.c +6 -6
@@ -1227,7 +1227,7 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)if (!page ||page_zone (page) != zone)return NULL ; -BUG_ON (!PageHighMem (page)); +BUG_ON (!PageHighMemZone (page));if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page))return NULL ;
@@ -1291,7 +1291,7 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)if (!page ||page_zone (page) != zone)return NULL ; -BUG_ON (PageHighMem (page)); +BUG_ON (PageHighMemZone (page));if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page))return NULL ;
@@ -1529,7 +1529,7 @@ static unsigned long preallocate_image_pages(unsigned long nr_pages, gfp_t mask)if (!page)break ;memory_bm_set_bit (©_bm,page_to_pfn (page)); -if (PageHighMem (page)) +if (PageHighMemZone (page)) alloc_highmem++;else alloc_normal++;
@@ -1625,7 +1625,7 @@ static unsigned long free_unnecessary_pages(void)unsigned long pfn =memory_bm_next_pfn (©_bm);struct page *page =pfn_to_page (pfn); -if (PageHighMem (page)) { +if (PageHighMemZone (page)) {if (!to_free_highmem)continue ; to_free_highmem--;
@@ -2264,7 +2264,7 @@ static unsigned int count_highmem_image_pages(struct memory_bitmap *bm)memory_bm_position_reset (bm); pfn =memory_bm_next_pfn (bm);while (pfn !=BM_END_OF_MAP ) { -if (PageHighMem (pfn_to_page (pfn))) +if (PageHighMemZone (pfn_to_page (pfn))) cnt++; pfn =memory_bm_next_pfn (bm);
@@ -2541,7 +2541,7 @@ static void *get_buffer(struct memory_bitmap *bm, struct chain_allocator *ca)return ERR_PTR (-EFAULT ); page =pfn_to_page (pfn); -if (PageHighMem (page)) +if (PageHighMemZone (page))return get_highmem_page_buffer (page, ca);if (swsusp_page_is_forbidden (page) &&swsusp_page_is_free (page))
Signature · 1 line
2.7.4
Re: [PATCH v2 04/10] power: separate PageHighMem() and PageHighMemZone() use case
kernel/power/snapshot.c static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn) · line 1230
Quoted 9 lines · On Wed, Apr 29, 2020 at 12:26:37PM +0900, [email protected] wrote:
index 6598001..be759a6 100644
kernel/power/snapshot.c +1
-1@@ -1227,7 +1227,7 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)if (!page ||page_zone (page) != zone)return NULL ; -BUG_ON (!PageHighMem (page)); +BUG_ON (!PageHighMemZone (page));
The above check already checks for the highmem zone. So if we want to keep the BUG_ON it needs stay PageHighMem to make sense. That being said I'd rather remove it entirelẏ.
kernel/power/snapshot.c static struct page *saveable_page(struct zone *zone, unsigned long pfn) · line 1294
Quoted 2 lines
kernel/power/snapshot.c +1
-1-BUG_ON (PageHighMem (page)); +BUG_ON (PageHighMemZone (page));
Same here.
kernel/power/snapshot.c static unsigned long preallocate_image_pages(unsigned long nr_pages, gfp_t mask) · line 1535
Quoted 9 lines
kernel/power/snapshot.c +1
-1@@ -1529,7 +1529,7 @@ static unsigned long preallocate_image_pages(unsigned long nr_pages, gfp_t mask)if (!page)break ;memory_bm_set_bit (©_bm,page_to_pfn (page)); -if (PageHighMem (page)) +if (PageHighMemZone (page)) alloc_highmem++;else alloc_normal++;
I don't fully understand the log here. Can Pavel or Rafael clarify why swsupst would care about the exact zone?
Re: [PATCH v2 04/10] power: separate PageHighMem() and PageHighMemZone() use case
2020년 5월 1일 (금) 오후 9:22, Christoph Hellwig <[email protected]>님이 작성:
Quoted 9 lines · On Wed, Apr 29, 2020 at 12:26:37PM +0900, [email protected] wrote:
Quoted 7 lines
index 6598001..be759a6 100644
kernel/power/snapshot.c +0
-0@@ -1227,7 +1227,7 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)if (!page ||page_zone (page) != zone)return NULL ;
Quoted 6 lines
kernel/power/snapshot.c static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn) · line 1230
Quoted 2 lines
kernel/power/snapshot.c +1
-1-BUG_ON (!PageHighMem (page)); +BUG_ON (!PageHighMemZone (page));The above check already checks for the highmem zone. So if we want to keep the
BUG_ONit needs stay PageHighMem to make sense. That being said I'd rather remove it entirelẏ.
Okay.
Quoted 4 lines
kernel/power/snapshot.c static struct page *saveable_page(struct zone *zone, unsigned long pfn) · line 1294
Quoted 2 lines
kernel/power/snapshot.c +1
-1-BUG_ON (PageHighMem (page)); +BUG_ON (PageHighMemZone (page));Same here.
Okay.
Thanks.
PATCH v2 5/10 mm/gup: separate PageHighMem() and PageHighMemZone() use case
In reply to [email protected]From: Joonsoo Kim <[email protected]>
Until now, PageHighMem() is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.
Now, we have separate functions, PageHighMem() and PageHighMemZone() for each cases. Use appropriate one.
Note that there are some rules to determine the proper macro.
1. If PageHighMem() is called for checking if the direct mapping exists or not, use PageHighMem().
2. If PageHighMem() is used to predict the previous gfp_flags for this page, use PageHighMemZone(). The zone of the page is related to the gfp_flags.
3. If purpose of calling PageHighMem() is to count highmem page and to interact with the system by using this count, use PageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, use PageHighMemZone(). It's safe since it's implementation is just copy of the previous PageHighMem() implementation and won't be changed.
I apply the rule #2 for this patch.
--- mm/gup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/gup.c +1 -1
@@ -1608,7 +1608,7 @@ static struct page *new_non_cma_page(struct page *page, unsigned long private)*/ gfp_t gfp_mask =GFP_USER | __GFP_NOWARN; -if (PageHighMem (page)) +if (PageHighMemZone (page)) gfp_mask |= __GFP_HIGHMEM;#ifdef CONFIG_HUGETLB_PAGE
Signature · 1 line
2.7.4
Re: [PATCH v2 05/10] mm/gup: separate PageHighMem() and PageHighMemZone() use case
Quoted 38 lines · On Wed, Apr 29, 2020 at 12:26:38PM +0900, [email protected] wrote:
From: Joonsoo Kim <[email protected]>
Until now,
PageHighMem()is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.Now, we have separate functions,
PageHighMem()andPageHighMemZone()for each cases. Use appropriate one.Note that there are some rules to determine the proper macro.
1. If
PageHighMem()is called for checking if the direct mapping exists or not, usePageHighMem().
2. IfPageHighMem()is used to predict the previousgfp_flagsfor this page, usePageHighMemZone(). The zone of the page is related to thegfp_flags.
3. If purpose of callingPageHighMem()is to count highmem page and to interact with the system by using this count, usePageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, usePageHighMemZone(). It's safe since it's implementation is just copy of the previousPageHighMem()implementation and won't be changed.I apply the rule #2 for this patch.
Acked-by Roman Gushchin <[email protected]>Signed-off-by Joonsoo Kim <[email protected]>--- mm/gup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/gup.c +0
-0
mm/gup.c static struct page *new_non_cma_page(struct page *page, unsigned long private) · line 1612
Quoted 6 lines
mm/gup.c +1
-1@@ -1608,7 +1608,7 @@ static struct page *new_non_cma_page(struct page *page, unsigned long private)*/ gfp_t gfp_mask =GFP_USER | __GFP_NOWARN; -if (PageHighMem (page)) +if (PageHighMemZone (page)) gfp_mask |= __GFP_HIGHMEM;
I think this wants to stay PageHighMem. This migrates CMA pages to other places before doing a long term pin. Anything that didn't have a direct mapping before won't need one for the new page, which could also include non-highmem zones without a highmem mapping.
Re: [PATCH v2 05/10] mm/gup: separate PageHighMem() and PageHighMemZone() use case
2020년 5월 1일 (금) 오후 9:24, Christoph Hellwig <[email protected]>님이 작성:
Quoted 43 lines · On Wed, Apr 29, 2020 at 12:26:38PM +0900, [email protected] wrote:
Quoted 41 lines
From: Joonsoo Kim <[email protected]>
Until now,
PageHighMem()is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.Now, we have separate functions,
PageHighMem()andPageHighMemZone()for each cases. Use appropriate one.Note that there are some rules to determine the proper macro.
1. If
PageHighMem()is called for checking if the direct mapping exists or not, usePageHighMem().
2. IfPageHighMem()is used to predict the previousgfp_flagsfor this page, usePageHighMemZone(). The zone of the page is related to thegfp_flags.
3. If purpose of callingPageHighMem()is to count highmem page and to interact with the system by using this count, usePageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, usePageHighMemZone(). It's safe since it's implementation is just copy of the previousPageHighMem()implementation and won't be changed.I apply the rule #2 for this patch.
Acked-by Roman Gushchin <[email protected]>Signed-off-by Joonsoo Kim <[email protected]>--- mm/gup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/gup.c +0
-0@@ -1608,7 +1608,7 @@ static struct page *new_non_cma_page(struct page *page, unsigned long private)*/ gfp_t gfp_mask =GFP_USER | __GFP_NOWARN;
Quoted 8 lines
mm/gup.c static struct page *new_non_cma_page(struct page *page, unsigned long private) · line 1612
Quoted 3 lines
mm/gup.c +1
-1-if (PageHighMem (page)) +if (PageHighMemZone (page)) gfp_mask |= __GFP_HIGHMEM;I think this wants to stay PageHighMem. This migrates CMA pages to other places before doing a long term pin. Anything that didn't have a direct mapping before won't need one for the new page, which could also include non-highmem zones without a highmem mapping.
What we want to do here is to guess allocation gfp flags of original page in order to allocate
a new page with most relaxed gfp flag. It is depend on the zone where the page belong to
rather than existence of direct mapping. Until now, existence of direct mapping implies
the type of zone so there is no problem.
After my future CMA patchset, direct mapped CMA page will be on the ZONE_MOVABLE.
And, a page on ZONE_MOVABLE should be allocated with __GFP_HIGHMEM | __GFP_MOVABLE.
So, most relaxed gfp flag for this CMA page would include __GFP_HIGHMEM. If PageHighMem()
is used here, __GFP_HIGHMEM would be lost since this CMA page has a direct mapping.
Therefore, PageHighMemZone() is right one here.
Anyway, I saw Eric's comment in other e-mail that abstraction is needed to guess gfp flags of
original page and I agree with it. This site can also get benefit of such a change.
Thanks.
PATCH v2 6/10 mm/hugetlb: separate PageHighMem() and PageHighMemZone() use case
In reply to [email protected]From: Joonsoo Kim <[email protected]>
Until now, PageHighMem() is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.
Now, we have separate functions, PageHighMem() and PageHighMemZone() for each cases. Use appropriate one.
Note that there are some rules to determine the proper macro.
1. If PageHighMem() is called for checking if the direct mapping exists or not, use PageHighMem().
2. If PageHighMem() is used to predict the previous gfp_flags for this page, use PageHighMemZone(). The zone of the page is related to the gfp_flags.
3. If purpose of calling PageHighMem() is to count highmem page and to interact with the system by using this count, use PageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, use PageHighMemZone(). It's safe since it's implementation is just copy of the previous PageHighMem() implementation and won't be changed.
I apply the rule #3 for this patch.
--- mm/hugetlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/hugetlb.c +1 -1
@@ -2639,7 +2639,7 @@ static void try_to_free_low(struct hstate *h, unsigned long count,list_for_each_entry_safe (page, next, freel, lru) {if (count >= h->nr_huge_pages)return ; -if (PageHighMem (page)) +if (PageHighMemZone (page))continue ;list_del (&page->lru);update_and_free_page (h, page);
Signature · 1 line
2.7.4
Re: [PATCH v2 06/10] mm/hugetlb: separate PageHighMem() and PageHighMemZone() use case
Quoted 25 lines · On Wed, Apr 29, 2020 at 12:26:39PM +0900, [email protected] wrote:
From: Joonsoo Kim <[email protected]>
Until now,
PageHighMem()is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.Now, we have separate functions,
PageHighMem()andPageHighMemZone()for each cases. Use appropriate one.Note that there are some rules to determine the proper macro.
1. If
PageHighMem()is called for checking if the direct mapping exists or not, usePageHighMem().
2. IfPageHighMem()is used to predict the previousgfp_flagsfor this page, usePageHighMemZone(). The zone of the page is related to thegfp_flags.
3. If purpose of callingPageHighMem()is to count highmem page and to interact with the system by using this count, usePageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, usePageHighMemZone(). It's safe since it's implementation is just copy of the previousPageHighMem()implementation and won't be changed.
Quoted 4 lines
I apply the rule #3 for this patch.
Acked-by Roman Gushchin <[email protected]>Signed-off-by Joonsoo Kim <[email protected]>
Why do we care about the zone here? This only cares about having kernel direct mapped pages as far as I can tell.
Re: [PATCH v2 06/10] mm/hugetlb: separate PageHighMem() and PageHighMemZone() use case
2020년 5월 1일 (금) 오후 9:26, Christoph Hellwig <[email protected]>님이 작성:
Quoted 27 lines · On Wed, Apr 29, 2020 at 12:26:39PM +0900, [email protected] wrote:
Quoted 25 lines
From: Joonsoo Kim <[email protected]>
Until now,
PageHighMem()is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.Now, we have separate functions,
PageHighMem()andPageHighMemZone()for each cases. Use appropriate one.Note that there are some rules to determine the proper macro.
1. If
PageHighMem()is called for checking if the direct mapping exists or not, usePageHighMem().
2. IfPageHighMem()is used to predict the previousgfp_flagsfor this page, usePageHighMemZone(). The zone of the page is related to thegfp_flags.
3. If purpose of callingPageHighMem()is to count highmem page and to interact with the system by using this count, usePageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, usePageHighMemZone(). It's safe since it's implementation is just copy of the previousPageHighMem()implementation and won't be changed.
Quoted 7 lines
Quoted 4 lines
I apply the rule #3 for this patch.
Acked-by Roman Gushchin <[email protected]>Signed-off-by Joonsoo Kim <[email protected]>Why do we care about the zone here? This only cares about having kernel direct mapped pages as far as I can tell.
My understaning is that what we want to do here is to first free memory that can be used for kernel allocation. If direct mapped page is on the zone that cannot be used for kernel allocation, there is no meaning to free this page first. So, we need to take a care of the zone here.
Thanks.
PATCH v2 7/10 mm: separate PageHighMem() and PageHighMemZone() use case
In reply to [email protected]From: Joonsoo Kim <[email protected]>
Until now, PageHighMem() is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.
Now, we have separate functions, PageHighMem() and PageHighMemZone() for each cases. Use appropriate one.
Note that there are some rules to determine the proper macro.
1. If PageHighMem() is called for checking if the direct mapping exists or not, use PageHighMem().
2. If PageHighMem() is used to predict the previous gfp_flags for this page, use PageHighMemZone(). The zone of the page is related to the gfp_flags.
3. If purpose of calling PageHighMem() is to count highmem page and to interact with the system by using this count, use PageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, use PageHighMemZone(). It's safe since it's implementation is just copy of the previous PageHighMem() implementation and won't be changed.
I apply the rule #3 for this patch.
--- mm/memory_hotplug.c | 2 +- mm/page_alloc.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
mm/memory_hotplug.c +1 -1
@@ -593,7 +593,7 @@ void generic_online_page(struct page *page, unsigned int order)__free_pages_core (page, order);totalram_pages_add (1UL << order);#ifdef CONFIG_HIGHMEM -if (PageHighMem (page)) +if (PageHighMemZone (page))totalhigh_pages_add (1UL << order);#endif }
mm/page_alloc.c +1 -1
@@ -7444,7 +7444,7 @@ void adjust_managed_page_count(struct page *page, long count)atomic_long_add (count, &page_zone (page)->managed_pages);totalram_pages_add (count);#ifdef CONFIG_HIGHMEM -if (PageHighMem (page)) +if (PageHighMemZone (page))totalhigh_pages_add (count);#endif }
Signature · 1 line
2.7.4
Re: [PATCH v2 07/10] mm: separate PageHighMem() and PageHighMemZone() use case
Quoted 52 lines · On Wed, Apr 29, 2020 at 12:26:40PM +0900, [email protected] wrote:
From: Joonsoo Kim <[email protected]>
Until now,
PageHighMem()is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.Now, we have separate functions,
PageHighMem()andPageHighMemZone()for each cases. Use appropriate one.Note that there are some rules to determine the proper macro.
1. If
PageHighMem()is called for checking if the direct mapping exists or not, usePageHighMem().
2. IfPageHighMem()is used to predict the previousgfp_flagsfor this page, usePageHighMemZone(). The zone of the page is related to thegfp_flags.
3. If purpose of callingPageHighMem()is to count highmem page and to interact with the system by using this count, usePageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, usePageHighMemZone(). It's safe since it's implementation is just copy of the previousPageHighMem()implementation and won't be changed.I apply the rule #3 for this patch.
Acked-by Roman Gushchin <[email protected]>Signed-off-by Joonsoo Kim <[email protected]>--- mm/memory_hotplug.c | 2 +- mm/page_alloc.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)mm/memory_hotplug.c +1
-1@@ -593,7 +593,7 @@ void generic_online_page(struct page *page, unsigned int order)__free_pages_core (page, order);totalram_pages_add (1UL << order);#ifdef CONFIG_HIGHMEM -if (PageHighMem (page)) +if (PageHighMemZone (page))totalhigh_pages_add (1UL << order);#endif }mm/page_alloc.c +0
-0
mm/page_alloc.c void adjust_managed_page_count(struct page *page, long count) · line 7449
Quoted 7 lines
mm/page_alloc.c +1
-1@@ -7444,7 +7444,7 @@ void adjust_managed_page_count(struct page *page, long count)atomic_long_add (count, &page_zone (page)->managed_pages);totalram_pages_add (count);#ifdef CONFIG_HIGHMEM -if (PageHighMem (page)) +if (PageHighMemZone (page))totalhigh_pages_add (count);#endif
This function already uses the page_zone structure above, I think life would be easier of you compare against that, as that makes the code more obvious.
Re: [PATCH v2 07/10] mm: separate PageHighMem() and PageHighMemZone() use case
2020년 5월 1일 (금) 오후 9:30, Christoph Hellwig <[email protected]>님이 작성:
Quoted 57 lines · On Wed, Apr 29, 2020 at 12:26:40PM +0900, [email protected] wrote:
Quoted 55 lines
From: Joonsoo Kim <[email protected]>
Until now,
PageHighMem()is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.Now, we have separate functions,
PageHighMem()andPageHighMemZone()for each cases. Use appropriate one.Note that there are some rules to determine the proper macro.
1. If
PageHighMem()is called for checking if the direct mapping exists or not, usePageHighMem().
2. IfPageHighMem()is used to predict the previousgfp_flagsfor this page, usePageHighMemZone(). The zone of the page is related to thegfp_flags.
3. If purpose of callingPageHighMem()is to count highmem page and to interact with the system by using this count, usePageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, usePageHighMemZone(). It's safe since it's implementation is just copy of the previousPageHighMem()implementation and won't be changed.I apply the rule #3 for this patch.
Acked-by Roman Gushchin <[email protected]>Signed-off-by Joonsoo Kim <[email protected]>--- mm/memory_hotplug.c | 2 +- mm/page_alloc.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)mm/memory_hotplug.c +1
-1@@ -593,7 +593,7 @@ void generic_online_page(struct page *page, unsigned int order)__free_pages_core (page, order);totalram_pages_add (1UL << order);#ifdef CONFIG_HIGHMEM -if (PageHighMem (page)) +if (PageHighMemZone (page))totalhigh_pages_add (1UL << order);#endif }mm/page_alloc.c +0
-0@@ -7444,7 +7444,7 @@ void adjust_managed_page_count(struct page *page, long count)atomic_long_add (count, &page_zone (page)->managed_pages);totalram_pages_add (count);#ifdef CONFIG_HIGHMEM
Quoted 8 lines
mm/page_alloc.c void adjust_managed_page_count(struct page *page, long count) · line 7449
Quoted 4 lines
mm/page_alloc.c +1
-1-if (PageHighMem (page)) +if (PageHighMemZone (page))totalhigh_pages_add (count);#endif This function already uses the
page_zonestructure above, I think life would be easier of you compare against that, as that makes the code more obvious.
If I can kill all the PageHighMemZone() macro, I will use page_zone() above. However, if it's not possible, I will leave it as it is. It would be simpler than
your suggestion.
Thanks.
PATCH v2 8/10 mm/page_alloc: correct the use of is_highmem_idx()
In reply to [email protected]From: Joonsoo Kim <[email protected]>
What we'd like to check here is whether page has direct mapping or not. Use PageHighMem() since it is perfectly matched for this purpose.
--- mm/page_alloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/page_alloc.c +1 -1
@@ -1399,7 +1399,7 @@ static void __meminit __init_single_page(struct page *page, unsigned long pfn,INIT_LIST_HEAD (&page->lru);#ifdef WANT_PAGE_VIRTUAL /* The shift won't overflow because ZONE_NORMAL is below 4G. */ -if (!is_highmem_idx (zone)) +if (!PageHighMem (page))set_page_address (page,__va (pfn <<PAGE_SHIFT ));#endif }
Signature · 1 line
2.7.4
PATCH v2 9/10 mm/migrate: replace PageHighMem() with open-code
In reply to [email protected]From: Joonsoo Kim <[email protected]>
Implementation of PageHighMem() will be changed in following patches. Before that, use open-code to avoid the side effect of implementation change on PageHighMem().
--- include/linux/migrate.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
include/linux/migrate.h +3 -1
@@ -37,6 +37,7 @@ static inline struct page *new_page_nodemask(struct page *page,gfp_t gfp_mask =GFP_USER | __GFP_MOVABLE | __GFP_RETRY_MAYFAIL;unsigned int order =0 ;struct page *new_page =NULL ; +int zidx;if (PageHuge (page)) return alloc_huge_page_nodemask(page_hstate(compound_head(page)),
@@ -47,7 +48,8 @@ static inline struct page *new_page_nodemask(struct page *page, order =HPAGE_PMD_ORDER ; } -if (PageHighMem (page) || (zone_idx (page_zone (page)) ==ZONE_MOVABLE )) + zidx =zone_idx (page_zone (page)); +if (is_highmem_idx (zidx) || zidx ==ZONE_MOVABLE ) gfp_mask |= __GFP_HIGHMEM; new_page = __alloc_pages_nodemask(gfp_mask, order,
Signature · 1 line
2.7.4
PATCH v2 10/10 mm/page-flags: change the implementation of the PageHighMem()
In reply to [email protected]From: Joonsoo Kim <[email protected]>
Until now, PageHighMem() is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.
Previous patches introduce PageHighMemZone() macro and separates both cases strictly. So, now, PageHighMem() is used just for checking if there is a direct mapping for this page or not.
In the following patchset, ZONE_MOVABLE which could be considered as the highmem type zone in some configuration could have both types of pages, direct mapped pages and unmapped pages. So, current implementation of PageHighMem() that checks the zone rather than checks the page in order to check if a direct mapping exists will be invalid. This patch prepares that case by implementing PageHighMem() with the max_low_pfn.
--- include/linux/page-flags.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
include/linux/page-flags.h +3 -1
@@ -375,6 +375,8 @@ PAGEFLAG(Readahead, reclaim, PF_NO_COMPOUND) TESTCLEARFLAG(Readahead, reclaim, PF_NO_COMPOUND)#ifdef CONFIG_HIGHMEM +extern unsigned long max_low_pfn; +/* * Must use a macro here due to header dependency issues. page_zone() is not * available at this point.
@@ -383,7 +385,7 @@ PAGEFLAG(Readahead, reclaim, PF_NO_COMPOUND)* in order to predict previous gfp_flags or to count something for system * memory management. */ -#define PageHighMem (__p) is_highmem_idx(page_zonenum(__p)) +#define PageHighMem (__p) (page_to_pfn(__p) >= max_low_pfn)#define PageHighMemZone (__p) is_highmem_idx(page_zonenum(__p))#else PAGEFLAG_FALSE (HighMem)
Signature · 1 line
2.7.4
Andrew Morton
maintainerQuoted 55 lines · On Wed, 29 Apr 2020 12:26:33 +0900 [email protected] wrote:
From: Joonsoo Kim <[email protected]>
Changes on v2
- add "acked-by", "reviewed-by" tags
- replacePageHighMem()with use open-code, instead of using newPageHighMemZone()macro. Related file is "include/linux/migrate.h"Hello,
This patchset separates two use cases of
PageHighMem()by introducingPageHighMemZone()macro. And, it changes the implementation ofPageHighMem()to reflect the actual meaning of this macro. This patchset is a preparation step for the patchset,
"mm/cma: manage the memory of the CMA area by using theZONE_MOVABLE" [1].
PageHighMem()is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.Until now, both the cases are the perfectly same thing. So, implementation of the
PageHighMem()uses the one case that checks if the zone of the page is the highmem type zone or not."#define
PageHighMem(__p)is_highmem_idx(page_zonenum(__p))"
ZONE_MOVABLEis special. It is considered as normal type zone on !CONFIG_HIGHMEM, but, it is considered as highmem type zone onCONFIG_HIGHMEM. Let's focus on later case. In later case, all pages on theZONE_MOVABLEhas no direct mapping until now.However, following patchset
"mm/cma: manage the memory of the CMA area by using theZONE_MOVABLE" , which is once merged and reverted, will be tried again and will break this assumption that all pages on theZONE_MOVABLEhas no direct mapping. Hence, theZONE_MOVABLEwhich is considered as highmem type zone could have the both types of pages, direct mapped and not. Since theZONE_MOVABLEcould have both type of pages,__GFP_HIGHMEMis still required to allocate the memory from it. And, we conservatively need to consider theZONE_MOVABLEas highmem type zone.Even in this situation,
PageHighMem()for the pages on theZONE_MOVABLEwhen it is called for checking the direct mapping should return correct result. Current implementation ofPageHighMem()just returns TRUE if the zone of the page is on a highmem type zone. So, it could be wrong if the page on theMOVABLE_ZONEis actually direct mapped.To solve this potential problem, this patch introduces a new
PageHighMemZone()macro. In following patches, two use cases ofPageHighMem()are separated by calling proper macro,PageHighMem()andPageHighMemZone(). Then, implementation ofPageHighMem()will be changed as just checking if the direct mapping exists or not, regardless of the zone of the page.Note that there are some rules to determine the proper macro.
Quoted 13 lines
1. If
PageHighMem()is called for checking if the direct mapping exists or not, usePageHighMem().
2. IfPageHighMem()is used to predict the previousgfp_flagsfor this page, usePageHighMemZone(). The zone of the page is related to thegfp_flags.
3. If purpose of callingPageHighMem()is to count highmem page and to interact with the system by using this count, usePageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, usePageHighMemZone(). It's safe since it's implementation is just copy of the previousPageHighMem()implementation and won't be changed.
hm, this won't improve maintainability :(
- Everyone will need to remember when to use PageHighMem() and when to use PageHighMemZone(). If they get it wrong, they're unlikely to notice any problem in their runtime testing, correct?
- New code will pop up which gets it wrong and nobody will notice for a long time.
So I guess we need to be pretty confident that the series "mm/cma:
manage the memory of the CMA area by using the ZONE_MOVABLE" will be useful and merged before proceeding with this, yes?
On the other hand, this whole series is a no-op until [10/10] (correct?) so it can be effectively reverted with a single line change, with later cleanups which revert the other 9 patches.
So I think I'd like to take another look at "mm/cma: manage the memory of the CMA area by using the ZONE_MOVABLE" before figuring out what to do here. Mainly to answer the question "is the new feature valuable enough to justify the maintainability impact". So please do take some care in explaining the end-user benefit when preparing the new version of that patchset.
2020년 4월 30일 (목) 오전 10:47, Andrew Morton <[email protected]>님이 작성:
Quoted 72 lines · On Wed, 29 Apr 2020 12:26:33 +0900 [email protected] wrote:
Quoted 56 lines
From: Joonsoo Kim <[email protected]>
Changes on v2
- add "acked-by", "reviewed-by" tags
- replacePageHighMem()with use open-code, instead of using newPageHighMemZone()macro. Related file is "include/linux/migrate.h"Hello,
This patchset separates two use cases of
PageHighMem()by introducingPageHighMemZone()macro. And, it changes the implementation ofPageHighMem()to reflect the actual meaning of this macro. This patchset is a preparation step for the patchset,
"mm/cma: manage the memory of the CMA area by using theZONE_MOVABLE" [1].
PageHighMem()is used for two different cases. One is to check if there is a direct mapping for this page or not. The other is to check the zone of this page, that is, weather it is the highmem type zone or not.Until now, both the cases are the perfectly same thing. So, implementation of the
PageHighMem()uses the one case that checks if the zone of the page is the highmem type zone or not."#define
PageHighMem(__p)is_highmem_idx(page_zonenum(__p))"
ZONE_MOVABLEis special. It is considered as normal type zone on !CONFIG_HIGHMEM, but, it is considered as highmem type zone onCONFIG_HIGHMEM. Let's focus on later case. In later case, all pages on theZONE_MOVABLEhas no direct mapping until now.However, following patchset
"mm/cma: manage the memory of the CMA area by using theZONE_MOVABLE" , which is once merged and reverted, will be tried again and will break this assumption that all pages on theZONE_MOVABLEhas no direct mapping. Hence, theZONE_MOVABLEwhich is considered as highmem type zone could have the both types of pages, direct mapped and not. Since theZONE_MOVABLEcould have both type of pages,__GFP_HIGHMEMis still required to allocate the memory from it. And, we conservatively need to consider theZONE_MOVABLEas highmem type zone.Even in this situation,
PageHighMem()for the pages on theZONE_MOVABLEwhen it is called for checking the direct mapping should return correct result. Current implementation ofPageHighMem()just returns TRUE if the zone of the page is on a highmem type zone. So, it could be wrong if the page on theMOVABLE_ZONEis actually direct mapped.To solve this potential problem, this patch introduces a new
PageHighMemZone()macro. In following patches, two use cases ofPageHighMem()are separated by calling proper macro,PageHighMem()andPageHighMemZone(). Then, implementation ofPageHighMem()will be changed as just checking if the direct mapping exists or not, regardless of the zone of the page.Note that there are some rules to determine the proper macro.
Quoted 13 lines
1. If
PageHighMem()is called for checking if the direct mapping exists or not, usePageHighMem().
2. IfPageHighMem()is used to predict the previousgfp_flagsfor this page, usePageHighMemZone(). The zone of the page is related to thegfp_flags.
3. If purpose of callingPageHighMem()is to count highmem page and to interact with the system by using this count, usePageHighMemZone(). This counter is usually used to calculate the available memory for an kernel allocation and pages on the highmem zone cannot be available for an kernel allocation.
4. Otherwise, usePageHighMemZone(). It's safe since it's implementation is just copy of the previousPageHighMem()implementation and won't be changed.
Quoted 8 lines
hm, this won't improve maintainability :(
- Everyone will need to remember when to use
PageHighMem()and when to usePageHighMemZone(). If they get it wrong, they're unlikely to notice any problem in their runtime testing, correct?- New code will pop up which gets it wrong and nobody will notice for a long time.
Hmm... I think that it's not that hard to decide correct macro. If we rename PageHighMem() with PageDirectMapped(), they, PageDirectMapped() and PageHighMemZone(), are self-explanation macro. There would be no confusion to use.
Quoted 3 lines
So I guess we need to be pretty confident that the series "mm/cma:
manage the memory of the CMA area by using theZONE_MOVABLE" will be useful and merged before proceeding with this, yes?
Yes and my assumption is that we (MM) have agreed with usefulness of CMA series.
Quoted 2 lines
On the other hand, this whole series is a no-op until [10/10] (correct?) so it can be effectively reverted with a single line change,
Correct!
Quoted 8 lines
with later cleanups which revert the other 9 patches.
So I think I'd like to take another look at "mm/cma: manage the memory of the CMA area by using the
ZONE_MOVABLE" before figuring out what to do here. Mainly to answer the question "is the new feature valuable enough to justify the maintainability impact". So please do take some care in explaining the end-user benefit when preparing the new version of that patchset.
So, do you mean to send the new version of CMA patchset with more explanation before merging this patchset? If yes, I can do. But, I'm not sure that it's worth doing. Problems of CMA are still not solved although the utilization problem will be partially solved by Roman's "mm,page_alloc,cma:
conditionally prefer cma pageblocks for movable allocations" patch in this (v5.7) release. Rationale that we agree with CMA patchset is still remained.
Anyway, if you mean that, I will send the CMA patchset with more explanation.
Thanks.
Quoted 7 lines · Joonsoo Kim
Quoted 2 lines
- New code will pop up which gets it wrong and nobody will notice for a long time.
Hmm... I think that it's not that hard to decide correct macro. If we rename
PageHighMem()withPageDirectMapped(), they,PageDirectMapped()andPageHighMemZone(), are self-explanation macro. There would be no confusion to use.
What confuses me is why we even need PageHighMemZone - mostly code should not care about particular zones. Maybe just open coding PageHighMemZone makes more sense - it is a little more cumersome, but at least it makes it explicit what we check for. I already sent you an incremental diff for one obvious place, but maybe we need to look through the remaining ones if we can kill them or open code them in an obvious way.
2020년 5월 1일 (금) 오후 7:55, Christoph Hellwig <[email protected]>님이 작성:
Quoted 10 lines · Joonsoo Kim
Quoted 7 lines
Quoted 2 lines
- New code will pop up which gets it wrong and nobody will notice for a long time.
Hmm... I think that it's not that hard to decide correct macro. If we rename
PageHighMem()withPageDirectMapped(), they,PageDirectMapped()andPageHighMemZone(), are self-explanation macro. There would be no confusion to use.
Quoted 7 lines
What confuses me is why we even need PageHighMemZone - mostly code should not care about particular zones. Maybe just open coding PageHighMemZone makes more sense - it is a little more cumersome, but at least it makes it explicit what we check for. I already sent you an incremental diff for one obvious place, but maybe we need to look through the remaining ones if we can kill them or open code them in an obvious way.
I think that PageHighMemZone() is long and complicated enough to have a macro.
PageHighMemZone(page) = is_highmem_idx(zone_idx(page_zone(page))
Instead of open-code, how about changing the style of macro like as page_from_highmem()? What PageHighMemZone() represent is derivated attribute from the page so PageXXX() style may not be appropriate.
Thanks.
Quoted 2 lines · Joonsoo Kim
I think that
PageHighMemZone()is long and complicated enough to have a macro.
It is. But then again it also shouldn't really be used by anything but MM internals.
Quoted 6 lines
PageHighMemZone(page) = is_highmem_idx(zone_idx(page_zone(page))
Instead of open-code, how about changing the style of macro like as
page_from_highmem()? WhatPageHighMemZone()represent is derivated attribute from the page soPageXXX()style may not be appropriate.
Maybe page_is_highmem_zone() with a big kerneldoc comment explaining the use case? Bonus points of killing enough users that it can be in mm/internal.h.
2020년 5월 1일 (금) 오후 9:34, Christoph Hellwig <[email protected]>님이 작성:
Quoted 7 lines · Joonsoo Kim
Quoted 2 lines
I think that
PageHighMemZone()is long and complicated enough to have a macro.It is. But then again it also shouldn't really be used by anything but MM internals.
I'm not sure that we can make it MM internal but I will try.
Quoted 10 lines
Quoted 6 lines
PageHighMemZone(page) = is_highmem_idx(zone_idx(page_zone(page))
Instead of open-code, how about changing the style of macro like as
page_from_highmem()? WhatPageHighMemZone()represent is derivated attribute from the page soPageXXX()style may not be appropriate.Maybe
page_is_highmem_zone()with a big kerneldoc comment explaining the use case? Bonus points of killing enough users that it can be inmm/internal.h.
I will try to kill page_is_highmem_zone() as much as possible in next version.
Thanks.