dma-mapping: Improve atomic pool behaviour
Subsystems (5)
Robin Murphy
reviewercover dma-mapping: Improve atomic pool behaviour
Hi all,
These days we have somewhat of a matrix of mm zones that may or may not exist and may or may not be empty, which the per-zone atomic pool logic doesn't actually handle all that well. It's not all that difficult to improve robustness and reduce redundancy, so that's what this series aims for.
I initially wrote this just because I happened to be looking through the boot log of one of my boards and had one of those out-of-the-blue realisations of "hang on, why *is* it allocating pools for zones it doesn't even have memory in?", but coincidentally, it seems that others happened to notice related aspects around the same time[1][2]. This is my attempt to sort it all out properly.
I guess this could go via either the dma-mapping or mm tree, whichever maintainers prefer.
Thanks,
Robin.
[1] https://lore.kernel.org/linux-iommu/[email protected]/
[2] https://lore.kernel.org/linux-iommu/[email protected]/
Robin Murphy (3):
dma/pool: Improve pool lookup
mm_zone: Generalise has_managed_dma()
dma/pool: Avoid allocating redundant pools
include/linux/mmzone.h | 9 +++++---- kernel/dma/pool.c | 27 ++++++++++++++++++--------- mm/page_alloc.c | 8 ++------ 3 files changed, 25 insertions(+), 19 deletions(-)
Signature · 1 line
2.34.1
Robin Murphy
reviewerPATCH 1/3 dma/pool: Improve pool lookup
If CONFIG_ZONE_DMA32 is enabled, but we have not allocated the corresponding atomic_pool_dma32, dma_guess_pool() may return the NULL value of that and fail a GFP_DMA32 allocation without trying to fall back to other pools which may exist. Furthermore, if no GFP_DMA pool exists, it is preferable to try GFP_DMA32 rather than immediately fall back to GFP_KERNEL with even less chance of success. Improve matters by encoding an explicit order of pool preference for each flag.
--- kernel/dma/pool.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/dma/pool.c +4 -4
@@ -224,10 +224,10 @@ postcore_initcall(dma_atomic_pool_init);static inline struct gen_pool *dma_guess_pool (struct gen_pool *prev,gfp_t gfp) {if (prev ==NULL ) { -if (IS_ENABLED (CONFIG_ZONE_DMA32 ) && (gfp &GFP_DMA32 )) -return atomic_pool_dma32; -if (atomic_pool_dma && (gfp &GFP_DMA )) -return atomic_pool_dma; +if (gfp &GFP_DMA ) +return atomic_pool_dma ?: atomic_pool_dma32 ?: atomic_pool_kernel; +if (gfp &GFP_DMA32 ) +return atomic_pool_dma32 ?: atomic_pool_dma ?: atomic_pool_kernel;return atomic_pool_kernel; }if (prev == atomic_pool_kernel)
Signature · 1 line
2.34.1
Robin Murphy
reviewerPATCH 2/3 mm_zone: Generalise has_managed_dma()
In reply to Robin MurphyIt would be useful to be able to check for potential DMA pages beyond just ZONE_DMA - generalise the existing has_managed_dma() function to allow checking other zones too.
--- include/linux/mmzone.h | 9 +++++---- mm/page_alloc.c | 8 ++------ 2 files changed, 7 insertions(+), 10 deletions(-)
include/linux/mmzone.h +5 -4
@@ -1648,14 +1648,15 @@ static inline int is_highmem(const struct zone *zone)return is_highmem_idx (zone_idx (zone)); } -#ifdef CONFIG_ZONE_DMA -bool has_managed_dma (void ); -#else +bool has_managed_zone (enum zone_type zone);static inline bool has_managed_dma (void ) { +#ifdef CONFIG_ZONE_DMA +return has_managed_zone (ZONE_DMA ); +#else return false; -}#endif +}#ifndef CONFIG_NUMA
mm/page_alloc.c +2 -6
@@ -7418,20 +7418,16 @@ bool put_page_back_buddy(struct page *page) }#endif -#ifdef CONFIG_ZONE_DMA -bool has_managed_dma (void ) +bool has_managed_zone (enum zone_type zone) {struct pglist_data *pgdat;for_each_online_pgdat (pgdat) { -struct zone *zone = &pgdat->node_zones[ZONE_DMA ]; - -if (managed_zone (zone)) +if (managed_zone (&pgdat->node_zones[zone]))return true; }return false; } -#endif /* CONFIG_ZONE_DMA */ #ifdef CONFIG_UNACCEPTED_MEMORY
Signature · 1 line
2.34.1
Robin Murphy
reviewerPATCH 3/3 dma/pool: Avoid allocating redundant pools
In reply to Robin MurphyOn smaller systems, e.g. embedded arm64, it is common for all memory to end up in ZONE_DMA32 or even ZONE_DMA. In such cases it is redundant to allocate a nominal pool for an empty higher zone that just ends up coming from a lower zone that should already have its own pool anyway. We already have logic to skip allocating a ZONE_DMA pool when that is empty, so generalise that to save memory in the case of other zones too.
--- kernel/dma/pool.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-)
kernel/dma/pool.c +14 -5
@@ -184,6 +184,12 @@ static __init struct gen_pool *__dma_atomic_pool_init(size_t pool_size,return pool; } +#ifdef CONFIG_ZONE_DMA32 +#define has_managed_dma32 has_managed_zone(ZONE_DMA32) +#else +#define has_managed_dma32 false +#endif +static int __init dma_atomic_pool_init (void ) {int ret =0 ;
@@ -199,17 +205,20 @@ static int __init dma_atomic_pool_init(void) }INIT_WORK (&atomic_pool_work, atomic_pool_work_fn); - atomic_pool_kernel =__dma_atomic_pool_init (atomic_pool_size, +/* All memory might be in the DMA zone(s) to begin with */ +if (has_managed_zone (ZONE_NORMAL )) { + atomic_pool_kernel =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL ); -if (!atomic_pool_kernel) - ret = -ENOMEM ; +if (!atomic_pool_kernel) + ret = -ENOMEM ; + }if (has_managed_dma ()) { atomic_pool_dma =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL |GFP_DMA );if (!atomic_pool_dma) ret = -ENOMEM ; } -if (IS_ENABLED (CONFIG_ZONE_DMA32 )) { +if (has_managed_dma32) { atomic_pool_dma32 =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL |GFP_DMA32 );if (!atomic_pool_dma32)
@@ -228,7 +237,7 @@ static inline struct gen_pool *dma_guess_pool(struct gen_pool *prev, gfp_t gfp)return atomic_pool_dma ?: atomic_pool_dma32 ?: atomic_pool_kernel;if (gfp &GFP_DMA32 )return atomic_pool_dma32 ?: atomic_pool_dma ?: atomic_pool_kernel; -return atomic_pool_kernel; +return atomic_pool_kernel ?: atomic_pool_dma32 ?: atomic_pool_dma; }if (prev == atomic_pool_kernel)return atomic_pool_dma32 ? atomic_pool_dma32 : atomic_pool_dma;
Signature · 1 line
2.34.1
Re: [PATCH 2/3] mm_zone: Generalise has_managed_dma()
In reply to Robin MurphyQuoted 6 lines · Robin Murphy
It would be useful to be able to check for potential DMA pages beyond just
ZONE_DMA- generalise the existinghas_managed_dma()function to allow checking other zones too.Signed-off-by Robin Murphy <[email protected]>---
Signature · 3 lines
Cheers David
Re: [PATCH 3/3] dma/pool: Avoid allocating redundant pools
In reply to Robin MurphyQuoted 2 lines · Robin Murphy
On smaller systems, e.g. embedded arm64, it is common for all memory to end up in
ZONE_DMA32or evenZONE_DMA. In such cases it is redundant
This is true and the whole series looks great to me. Do we need adjust warn_alloc() to handle empty DMA32 zone too like empty DMA zone case?
Quoted 65 lines
to allocate a nominal pool for an empty higher zone that just ends up coming from a lower zone that should already have its own pool anyway. We already have logic to skip allocating a
ZONE_DMApool when that is empty, so generalise that to save memory in the case of other zones too.Signed-off-by Robin Murphy <[email protected]>--- kernel/dma/pool.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-)kernel/dma/pool.c +14
-5@@ -184,6 +184,12 @@ static __init struct gen_pool *__dma_atomic_pool_init(size_t pool_size,return pool; } +#ifdef CONFIG_ZONE_DMA32 +#define has_managed_dma32 has_managed_zone(ZONE_DMA32) +#else +#define has_managed_dma32 false +#endif +static int __init dma_atomic_pool_init (void ) {int ret =0 ;@@ -199,17 +205,20 @@ static int __init dma_atomic_pool_init(void) }INIT_WORK (&atomic_pool_work, atomic_pool_work_fn); - atomic_pool_kernel =__dma_atomic_pool_init (atomic_pool_size, +/* All memory might be in the DMA zone(s) to begin with */ +if (has_managed_zone (ZONE_NORMAL )) { + atomic_pool_kernel =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL ); -if (!atomic_pool_kernel) - ret = -ENOMEM ; +if (!atomic_pool_kernel) + ret = -ENOMEM ; + }if (has_managed_dma ()) { atomic_pool_dma =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL |GFP_DMA );if (!atomic_pool_dma) ret = -ENOMEM ; } -if (IS_ENABLED (CONFIG_ZONE_DMA32 )) { +if (has_managed_dma32) { atomic_pool_dma32 =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL |GFP_DMA32 );if (!atomic_pool_dma32)@@ -228,7 +237,7 @@ static inline struct gen_pool *dma_guess_pool(struct gen_pool *prev, gfp_t gfp)return atomic_pool_dma ?: atomic_pool_dma32 ?: atomic_pool_kernel;if (gfp &GFP_DMA32 )return atomic_pool_dma32 ?: atomic_pool_dma ?: atomic_pool_kernel; -return atomic_pool_kernel; +return atomic_pool_kernel ?: atomic_pool_dma32 ?: atomic_pool_dma; }if (prev == atomic_pool_kernel)return atomic_pool_dma32 ? atomic_pool_dma32 : atomic_pool_dma;Signature · 1 line
2.34.1
Quoted 30 lines
Hi all,
These days we have somewhat of a matrix of mm zones that may or may not exist and may or may not be empty, which the per-zone atomic pool logic doesn't actually handle all that well. It's not all that difficult to improve robustness and reduce redundancy, so that's what this series aims for.
I initially wrote this just because I happened to be looking through the boot log of one of my boards and had one of those out-of-the-blue realisations of "hang on, why *is* it allocating pools for zones it doesn't even have memory in?", but coincidentally, it seems that others happened to notice related aspects around the same time[1][2]. This is my attempt to sort it all out properly.
I guess this could go via either the dma-mapping or mm tree, whichever maintainers prefer.
Thanks,
Robin.[1] Suspicious Link - Removed
[2] Suspicious Link - RemovedRobin Murphy (3):
dma/pool: Improve pool lookup
mm_zone: Generalisehas_managed_dma()
dma/pool: Avoid allocating redundant pools
Quoted 7 lines
include/linux/mmzone.h | 9 +++++---- kernel/dma/pool.c | 27 ++++++++++++++++++--------- mm/page_alloc.c | 8 ++------
3 files changed, 25 insertions(+), 19 deletions(-)--
2.34.1
Tested on platform: upcoming Mobileye eyeq7h - risc-v platform with memory mapped to 32Gbytes so DMA32 zone is empty
Robin Murphy
reviewerRe: [PATCH 3/3] dma/pool: Avoid allocating redundant pools
In reply to Baoquan HeQuoted 6 lines · Baoquan He
Quoted 2 lines · Robin Murphy
On smaller systems, e.g. embedded arm64, it is common for all memory to end up in
ZONE_DMA32or evenZONE_DMA. In such cases it is redundantThis is true and the whole series looks great to me. Do we need adjust
warn_alloc()to handle empty DMA32 zone too like empty DMA zone case?
Hmm, I'd be inclined to think that if nobody's complaining already then we can probably just leave it as-is. A GFP_DMA32 allocation won't OOM unless *both* ZONE_DMA32 and ZONE_DMA are empty, right? At that point I'd imagine it's a bit more significant if someone who wants DMA32 memory can't have any - don't we have a mechanism for reserving some "low" memory for kdump for pretty much this exact reason?
A special case for when ZONE_DMA is tiny such that GFP_DMA can be expected to fail often seems fair, but in general I'd expect that if GFP_DMA32 starts failing then it's more a sign of a genuine mismatch between the kernel's expectations and the system configuration.
Thanks,
Robin.
Quoted 66 lines
Quoted 65 lines
to allocate a nominal pool for an empty higher zone that just ends up coming from a lower zone that should already have its own pool anyway. We already have logic to skip allocating a
ZONE_DMApool when that is empty, so generalise that to save memory in the case of other zones too.Signed-off-by Robin Murphy <[email protected]>--- kernel/dma/pool.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-)kernel/dma/pool.c +14
-5@@ -184,6 +184,12 @@ static __init struct gen_pool *__dma_atomic_pool_init(size_t pool_size,return pool; } +#ifdef CONFIG_ZONE_DMA32 +#define has_managed_dma32 has_managed_zone(ZONE_DMA32) +#else +#define has_managed_dma32 false +#endif +static int __init dma_atomic_pool_init (void ) {int ret =0 ;@@ -199,17 +205,20 @@ static int __init dma_atomic_pool_init(void) }INIT_WORK (&atomic_pool_work, atomic_pool_work_fn); - atomic_pool_kernel =__dma_atomic_pool_init (atomic_pool_size, +/* All memory might be in the DMA zone(s) to begin with */ +if (has_managed_zone (ZONE_NORMAL )) { + atomic_pool_kernel =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL ); -if (!atomic_pool_kernel) - ret = -ENOMEM ; +if (!atomic_pool_kernel) + ret = -ENOMEM ; + }if (has_managed_dma ()) { atomic_pool_dma =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL |GFP_DMA );if (!atomic_pool_dma) ret = -ENOMEM ; } -if (IS_ENABLED (CONFIG_ZONE_DMA32 )) { +if (has_managed_dma32) { atomic_pool_dma32 =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL |GFP_DMA32 );if (!atomic_pool_dma32)@@ -228,7 +237,7 @@ static inline struct gen_pool *dma_guess_pool(struct gen_pool *prev, gfp_t gfp)return atomic_pool_dma ?: atomic_pool_dma32 ?: atomic_pool_kernel;if (gfp &GFP_DMA32 )return atomic_pool_dma32 ?: atomic_pool_dma ?: atomic_pool_kernel; -return atomic_pool_kernel; +return atomic_pool_kernel ?: atomic_pool_dma32 ?: atomic_pool_dma; }if (prev == atomic_pool_kernel)return atomic_pool_dma32 ? atomic_pool_dma32 : atomic_pool_dma;Signature · 1 line
2.34.1
Marek Szyprowski
maintainerQuoted 13 lines · Robin Murphy
These days we have somewhat of a matrix of mm zones that may or may not exist and may or may not be empty, which the per-zone atomic pool logic doesn't actually handle all that well. It's not all that difficult to improve robustness and reduce redundancy, so that's what this series aims for.
I initially wrote this just because I happened to be looking through the boot log of one of my boards and had one of those out-of-the-blue realisations of "hang on, why *is* it allocating pools for zones it doesn't even have memory in?", but coincidentally, it seems that others happened to notice related aspects around the same time[1][2]. This is my attempt to sort it all out properly.
Quoted 2 lines
I guess this could go via either the dma-mapping or mm tree, whichever maintainers prefer.
I will take this patchset to dma-mapping-fixes branch, thanks!
Best regards
Signature · 2 lines
Marek Szyprowski, PhD Samsung R&D Institute Poland
Re: [PATCH 3/3] dma/pool: Avoid allocating redundant pools
In reply to Robin MurphyQuoted 15 lines · Robin Murphy
Quoted 6 lines · Baoquan He
Quoted 2 lines · Robin Murphy
On smaller systems, e.g. embedded arm64, it is common for all memory to end up in
ZONE_DMA32or evenZONE_DMA. In such cases it is redundantThis is true and the whole series looks great to me. Do we need adjust
warn_alloc()to handle empty DMA32 zone too like empty DMA zone case?Hmm, I'd be inclined to think that if nobody's complaining already then we can probably just leave it as-is. A
GFP_DMA32allocation won't OOM unless *both*ZONE_DMA32andZONE_DMAare empty, right? At that point I'd imagine it's a bit more significant if someone who wants DMA32 memory can't have any
- don't we have a mechanism for reserving some "low" memory for kdump for pretty much this exact reason?
Quoted 4 lines
A special case for when
ZONE_DMAis tiny such thatGFP_DMAcan be expected to fail often seems fair, but in general I'd expect that ifGFP_DMA32starts failing then it's more a sign of a genuine mismatch between the kernel's expectations and the system configuration.
Makes sense to me. I remember someone plans to take off both DMA and DMA32 zone in linux kernel, but that might not happen soon.
Quoted 8 lines · Robin Murphy
Hi all,
These days we have somewhat of a matrix of mm zones that may or may not exist and may or may not be empty, which the per-zone atomic pool logic doesn't actually handle all that well. It's not all that difficult to improve robustness and reduce redundancy, so that's what this series aims for.
Quoted 6 lines
I initially wrote this just because I happened to be looking through the boot log of one of my boards and had one of those out-of-the-blue realisations of "hang on, why *is* it allocating pools for zones it doesn't even have memory in?", but coincidentally, it seems that others happened to notice related aspects around the same time[1][2]. This is my attempt to sort it all out properly.
Re: [PATCH 2/3] mm_zone: Generalise has_managed_dma()
In reply to Robin MurphyQuoted 5 lines · Robin Murphy
It would be useful to be able to check for potential DMA pages beyond just
ZONE_DMA- generalise the existinghas_managed_dma()function to allow checking other zones too.Signed-off-by Robin Murphy <[email protected]>
Quoted 59 lines
--- include/linux/mmzone.h | 9 +++++---- mm/page_alloc.c | 8 ++------ 2 files changed, 7 insertions(+), 10 deletions(-)include/linux/mmzone.h +5
-4@@ -1648,14 +1648,15 @@ static inline int is_highmem(const struct zone *zone)return is_highmem_idx (zone_idx (zone)); } -#ifdef CONFIG_ZONE_DMA -bool has_managed_dma (void ); -#else +bool has_managed_zone (enum zone_type zone);static inline bool has_managed_dma (void ) { +#ifdef CONFIG_ZONE_DMA +return has_managed_zone (ZONE_DMA ); +#else return false; -}#endif +}#ifndef CONFIG_NUMA mm/page_alloc.c +2
-6@@ -7418,20 +7418,16 @@ bool put_page_back_buddy(struct page *page) }#endif -#ifdef CONFIG_ZONE_DMA -bool has_managed_dma (void ) +bool has_managed_zone (enum zone_type zone) {struct pglist_data *pgdat;for_each_online_pgdat (pgdat) { -struct zone *zone = &pgdat->node_zones[ZONE_DMA ]; - -if (managed_zone (zone)) +if (managed_zone (&pgdat->node_zones[zone]))return true; }return false; } -#endif /* CONFIG_ZONE_DMA */ #ifdef CONFIG_UNACCEPTED_MEMORY Signature · 1 line
2.34.1
Signature · 2 lines
Sincerely yours, Mike.