dma-mapping: Improve atomic pool behaviour
Subsystems (5)
Robin Murphy
reviewercover dma-mapping: Improve atomic pool behaviour
Hi all,
These days we have somewhat of a matrix of mm zones that may or may not exist and may or may not be empty, which the per-zone atomic pool logic doesn't actually handle all that well. It's not all that difficult to improve robustness and reduce redundancy, so that's what this series aims for.
I initially wrote this just because I happened to be looking through the boot log of one of my boards and had one of those out-of-the-blue realisations of "hang on, why *is* it allocating pools for zones it doesn't even have memory in?", but coincidentally, it seems that others happened to notice related aspects around the same time[1][2]. This is my attempt to sort it all out properly.
I guess this could go via either the dma-mapping or mm tree, whichever maintainers prefer.
Thanks,
Robin.
[1] https://lore.kernel.org/linux-iommu/[email protected]/
[2] https://lore.kernel.org/linux-iommu/[email protected]/
Robin Murphy (3):
dma/pool: Improve pool lookup
mm_zone: Generalise has_managed_dma()
dma/pool: Avoid allocating redundant pools
include/linux/mmzone.h | 9 +++++---- kernel/dma/pool.c | 27 ++++++++++++++++++--------- mm/page_alloc.c | 8 ++------ 3 files changed, 25 insertions(+), 19 deletions(-)
Signature · 1 line
2.34.1
Robin Murphy
reviewerPATCH 1/3 dma/pool: Improve pool lookup
If CONFIG_ZONE_DMA32 is enabled, but we have not allocated the corresponding atomic_pool_dma32, dma_guess_pool() may return the NULL value of that and fail a GFP_DMA32 allocation without trying to fall back to other pools which may exist. Furthermore, if no GFP_DMA pool exists, it is preferable to try GFP_DMA32 rather than immediately fall back to GFP_KERNEL with even less chance of success. Improve matters by encoding an explicit order of pool preference for each flag.
--- kernel/dma/pool.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/dma/pool.c +4 -4
@@ -224,10 +224,10 @@ postcore_initcall(dma_atomic_pool_init);static inline struct gen_pool *dma_guess_pool (struct gen_pool *prev,gfp_t gfp) {if (prev ==NULL ) { -if (IS_ENABLED (CONFIG_ZONE_DMA32 ) && (gfp &GFP_DMA32 )) -return atomic_pool_dma32; -if (atomic_pool_dma && (gfp &GFP_DMA )) -return atomic_pool_dma; +if (gfp &GFP_DMA ) +return atomic_pool_dma ?: atomic_pool_dma32 ?: atomic_pool_kernel; +if (gfp &GFP_DMA32 ) +return atomic_pool_dma32 ?: atomic_pool_dma ?: atomic_pool_kernel;return atomic_pool_kernel; }if (prev == atomic_pool_kernel)
Signature · 1 line
2.34.1
Robin Murphy
reviewerPATCH 2/3 mm_zone: Generalise has_managed_dma()
In reply to Robin MurphyIt would be useful to be able to check for potential DMA pages beyond just ZONE_DMA - generalise the existing has_managed_dma() function to allow checking other zones too.
--- include/linux/mmzone.h | 9 +++++---- mm/page_alloc.c | 8 ++------ 2 files changed, 7 insertions(+), 10 deletions(-)
include/linux/mmzone.h +5 -4
@@ -1648,14 +1648,15 @@ static inline int is_highmem(const struct zone *zone)return is_highmem_idx (zone_idx (zone)); } -#ifdef CONFIG_ZONE_DMA -bool has_managed_dma (void ); -#else +bool has_managed_zone (enum zone_type zone);static inline bool has_managed_dma (void ) { +#ifdef CONFIG_ZONE_DMA +return has_managed_zone (ZONE_DMA ); +#else return false; -}#endif +}#ifndef CONFIG_NUMA
mm/page_alloc.c +2 -6
@@ -7418,20 +7418,16 @@ bool put_page_back_buddy(struct page *page) }#endif -#ifdef CONFIG_ZONE_DMA -bool has_managed_dma (void ) +bool has_managed_zone (enum zone_type zone) {struct pglist_data *pgdat;for_each_online_pgdat (pgdat) { -struct zone *zone = &pgdat->node_zones[ZONE_DMA ]; - -if (managed_zone (zone)) +if (managed_zone (&pgdat->node_zones[zone]))return true; }return false; } -#endif /* CONFIG_ZONE_DMA */ #ifdef CONFIG_UNACCEPTED_MEMORY
Signature · 1 line
2.34.1
Re: [PATCH 2/3] mm_zone: Generalise has_managed_dma()
Quoted 6 lines · Robin Murphy
It would be useful to be able to check for potential DMA pages beyond just
ZONE_DMA- generalise the existinghas_managed_dma()function to allow checking other zones too.Signed-off-by Robin Murphy <[email protected]>---
Signature · 3 lines
Cheers David
Re: [PATCH 2/3] mm_zone: Generalise has_managed_dma()
In reply to Robin MurphyQuoted 5 lines · Robin Murphy
It would be useful to be able to check for potential DMA pages beyond just
ZONE_DMA- generalise the existinghas_managed_dma()function to allow checking other zones too.Signed-off-by Robin Murphy <[email protected]>
Quoted 59 lines
--- include/linux/mmzone.h | 9 +++++---- mm/page_alloc.c | 8 ++------ 2 files changed, 7 insertions(+), 10 deletions(-)include/linux/mmzone.h +5
-4@@ -1648,14 +1648,15 @@ static inline int is_highmem(const struct zone *zone)return is_highmem_idx (zone_idx (zone)); } -#ifdef CONFIG_ZONE_DMA -bool has_managed_dma (void ); -#else +bool has_managed_zone (enum zone_type zone);static inline bool has_managed_dma (void ) { +#ifdef CONFIG_ZONE_DMA +return has_managed_zone (ZONE_DMA ); +#else return false; -}#endif +}#ifndef CONFIG_NUMA mm/page_alloc.c +2
-6@@ -7418,20 +7418,16 @@ bool put_page_back_buddy(struct page *page) }#endif -#ifdef CONFIG_ZONE_DMA -bool has_managed_dma (void ) +bool has_managed_zone (enum zone_type zone) {struct pglist_data *pgdat;for_each_online_pgdat (pgdat) { -struct zone *zone = &pgdat->node_zones[ZONE_DMA ]; - -if (managed_zone (zone)) +if (managed_zone (&pgdat->node_zones[zone]))return true; }return false; } -#endif /* CONFIG_ZONE_DMA */ #ifdef CONFIG_UNACCEPTED_MEMORY Signature · 1 line
2.34.1
Signature · 2 lines
Sincerely yours, Mike.
Robin Murphy
reviewerPATCH 3/3 dma/pool: Avoid allocating redundant pools
In reply to Robin MurphyOn smaller systems, e.g. embedded arm64, it is common for all memory to end up in ZONE_DMA32 or even ZONE_DMA. In such cases it is redundant to allocate a nominal pool for an empty higher zone that just ends up coming from a lower zone that should already have its own pool anyway. We already have logic to skip allocating a ZONE_DMA pool when that is empty, so generalise that to save memory in the case of other zones too.
--- kernel/dma/pool.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-)
kernel/dma/pool.c +14 -5
@@ -184,6 +184,12 @@ static __init struct gen_pool *__dma_atomic_pool_init(size_t pool_size,return pool; } +#ifdef CONFIG_ZONE_DMA32 +#define has_managed_dma32 has_managed_zone(ZONE_DMA32) +#else +#define has_managed_dma32 false +#endif +static int __init dma_atomic_pool_init (void ) {int ret =0 ;
@@ -199,17 +205,20 @@ static int __init dma_atomic_pool_init(void) }INIT_WORK (&atomic_pool_work, atomic_pool_work_fn); - atomic_pool_kernel =__dma_atomic_pool_init (atomic_pool_size, +/* All memory might be in the DMA zone(s) to begin with */ +if (has_managed_zone (ZONE_NORMAL )) { + atomic_pool_kernel =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL ); -if (!atomic_pool_kernel) - ret = -ENOMEM ; +if (!atomic_pool_kernel) + ret = -ENOMEM ; + }if (has_managed_dma ()) { atomic_pool_dma =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL |GFP_DMA );if (!atomic_pool_dma) ret = -ENOMEM ; } -if (IS_ENABLED (CONFIG_ZONE_DMA32 )) { +if (has_managed_dma32) { atomic_pool_dma32 =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL |GFP_DMA32 );if (!atomic_pool_dma32)
@@ -228,7 +237,7 @@ static inline struct gen_pool *dma_guess_pool(struct gen_pool *prev, gfp_t gfp)return atomic_pool_dma ?: atomic_pool_dma32 ?: atomic_pool_kernel;if (gfp &GFP_DMA32 )return atomic_pool_dma32 ?: atomic_pool_dma ?: atomic_pool_kernel; -return atomic_pool_kernel; +return atomic_pool_kernel ?: atomic_pool_dma32 ?: atomic_pool_dma; }if (prev == atomic_pool_kernel)return atomic_pool_dma32 ? atomic_pool_dma32 : atomic_pool_dma;
Signature · 1 line
2.34.1
Re: [PATCH 3/3] dma/pool: Avoid allocating redundant pools
Quoted 2 lines · Robin Murphy
On smaller systems, e.g. embedded arm64, it is common for all memory to end up in
ZONE_DMA32or evenZONE_DMA. In such cases it is redundant
This is true and the whole series looks great to me. Do we need adjust warn_alloc() to handle empty DMA32 zone too like empty DMA zone case?
Quoted 65 lines
to allocate a nominal pool for an empty higher zone that just ends up coming from a lower zone that should already have its own pool anyway. We already have logic to skip allocating a
ZONE_DMApool when that is empty, so generalise that to save memory in the case of other zones too.Signed-off-by Robin Murphy <[email protected]>--- kernel/dma/pool.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-)kernel/dma/pool.c +14
-5@@ -184,6 +184,12 @@ static __init struct gen_pool *__dma_atomic_pool_init(size_t pool_size,return pool; } +#ifdef CONFIG_ZONE_DMA32 +#define has_managed_dma32 has_managed_zone(ZONE_DMA32) +#else +#define has_managed_dma32 false +#endif +static int __init dma_atomic_pool_init (void ) {int ret =0 ;@@ -199,17 +205,20 @@ static int __init dma_atomic_pool_init(void) }INIT_WORK (&atomic_pool_work, atomic_pool_work_fn); - atomic_pool_kernel =__dma_atomic_pool_init (atomic_pool_size, +/* All memory might be in the DMA zone(s) to begin with */ +if (has_managed_zone (ZONE_NORMAL )) { + atomic_pool_kernel =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL ); -if (!atomic_pool_kernel) - ret = -ENOMEM ; +if (!atomic_pool_kernel) + ret = -ENOMEM ; + }if (has_managed_dma ()) { atomic_pool_dma =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL |GFP_DMA );if (!atomic_pool_dma) ret = -ENOMEM ; } -if (IS_ENABLED (CONFIG_ZONE_DMA32 )) { +if (has_managed_dma32) { atomic_pool_dma32 =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL |GFP_DMA32 );if (!atomic_pool_dma32)@@ -228,7 +237,7 @@ static inline struct gen_pool *dma_guess_pool(struct gen_pool *prev, gfp_t gfp)return atomic_pool_dma ?: atomic_pool_dma32 ?: atomic_pool_kernel;if (gfp &GFP_DMA32 )return atomic_pool_dma32 ?: atomic_pool_dma ?: atomic_pool_kernel; -return atomic_pool_kernel; +return atomic_pool_kernel ?: atomic_pool_dma32 ?: atomic_pool_dma; }if (prev == atomic_pool_kernel)return atomic_pool_dma32 ? atomic_pool_dma32 : atomic_pool_dma;Signature · 1 line
2.34.1
Robin Murphy
reviewerRe: [PATCH 3/3] dma/pool: Avoid allocating redundant pools
Quoted 6 lines · Baoquan He
Quoted 2 lines · Robin Murphy
On smaller systems, e.g. embedded arm64, it is common for all memory to end up in
ZONE_DMA32or evenZONE_DMA. In such cases it is redundantThis is true and the whole series looks great to me. Do we need adjust
warn_alloc()to handle empty DMA32 zone too like empty DMA zone case?
Hmm, I'd be inclined to think that if nobody's complaining already then we can probably just leave it as-is. A GFP_DMA32 allocation won't OOM unless *both* ZONE_DMA32 and ZONE_DMA are empty, right? At that point I'd imagine it's a bit more significant if someone who wants DMA32 memory can't have any - don't we have a mechanism for reserving some "low" memory for kdump for pretty much this exact reason?
A special case for when ZONE_DMA is tiny such that GFP_DMA can be expected to fail often seems fair, but in general I'd expect that if GFP_DMA32 starts failing then it's more a sign of a genuine mismatch between the kernel's expectations and the system configuration.
Thanks,
Robin.
Quoted 66 lines
Quoted 65 lines
to allocate a nominal pool for an empty higher zone that just ends up coming from a lower zone that should already have its own pool anyway. We already have logic to skip allocating a
ZONE_DMApool when that is empty, so generalise that to save memory in the case of other zones too.Signed-off-by Robin Murphy <[email protected]>--- kernel/dma/pool.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-)kernel/dma/pool.c +14
-5@@ -184,6 +184,12 @@ static __init struct gen_pool *__dma_atomic_pool_init(size_t pool_size,return pool; } +#ifdef CONFIG_ZONE_DMA32 +#define has_managed_dma32 has_managed_zone(ZONE_DMA32) +#else +#define has_managed_dma32 false +#endif +static int __init dma_atomic_pool_init (void ) {int ret =0 ;@@ -199,17 +205,20 @@ static int __init dma_atomic_pool_init(void) }INIT_WORK (&atomic_pool_work, atomic_pool_work_fn); - atomic_pool_kernel =__dma_atomic_pool_init (atomic_pool_size, +/* All memory might be in the DMA zone(s) to begin with */ +if (has_managed_zone (ZONE_NORMAL )) { + atomic_pool_kernel =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL ); -if (!atomic_pool_kernel) - ret = -ENOMEM ; +if (!atomic_pool_kernel) + ret = -ENOMEM ; + }if (has_managed_dma ()) { atomic_pool_dma =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL |GFP_DMA );if (!atomic_pool_dma) ret = -ENOMEM ; } -if (IS_ENABLED (CONFIG_ZONE_DMA32 )) { +if (has_managed_dma32) { atomic_pool_dma32 =__dma_atomic_pool_init (atomic_pool_size,GFP_KERNEL |GFP_DMA32 );if (!atomic_pool_dma32)@@ -228,7 +237,7 @@ static inline struct gen_pool *dma_guess_pool(struct gen_pool *prev, gfp_t gfp)return atomic_pool_dma ?: atomic_pool_dma32 ?: atomic_pool_kernel;if (gfp &GFP_DMA32 )return atomic_pool_dma32 ?: atomic_pool_dma ?: atomic_pool_kernel; -return atomic_pool_kernel; +return atomic_pool_kernel ?: atomic_pool_dma32 ?: atomic_pool_dma; }if (prev == atomic_pool_kernel)return atomic_pool_dma32 ? atomic_pool_dma32 : atomic_pool_dma;Signature · 1 line
2.34.1
Re: [PATCH 3/3] dma/pool: Avoid allocating redundant pools
Quoted 15 lines · Robin Murphy
Quoted 6 lines · Baoquan He
Quoted 2 lines · Robin Murphy
On smaller systems, e.g. embedded arm64, it is common for all memory to end up in
ZONE_DMA32or evenZONE_DMA. In such cases it is redundantThis is true and the whole series looks great to me. Do we need adjust
warn_alloc()to handle empty DMA32 zone too like empty DMA zone case?Hmm, I'd be inclined to think that if nobody's complaining already then we can probably just leave it as-is. A
GFP_DMA32allocation won't OOM unless *both*ZONE_DMA32andZONE_DMAare empty, right? At that point I'd imagine it's a bit more significant if someone who wants DMA32 memory can't have any
- don't we have a mechanism for reserving some "low" memory for kdump for pretty much this exact reason?
Quoted 4 lines
A special case for when
ZONE_DMAis tiny such thatGFP_DMAcan be expected to fail often seems fair, but in general I'd expect that ifGFP_DMA32starts failing then it's more a sign of a genuine mismatch between the kernel's expectations and the system configuration.
Makes sense to me. I remember someone plans to take off both DMA and DMA32 zone in linux kernel, but that might not happen soon.
Quoted 30 lines
Hi all,
These days we have somewhat of a matrix of mm zones that may or may not exist and may or may not be empty, which the per-zone atomic pool logic doesn't actually handle all that well. It's not all that difficult to improve robustness and reduce redundancy, so that's what this series aims for.
I initially wrote this just because I happened to be looking through the boot log of one of my boards and had one of those out-of-the-blue realisations of "hang on, why *is* it allocating pools for zones it doesn't even have memory in?", but coincidentally, it seems that others happened to notice related aspects around the same time[1][2]. This is my attempt to sort it all out properly.
I guess this could go via either the dma-mapping or mm tree, whichever maintainers prefer.
Thanks,
Robin.[1] Suspicious Link - Removed
[2] Suspicious Link - RemovedRobin Murphy (3):
dma/pool: Improve pool lookup
mm_zone: Generalisehas_managed_dma()
dma/pool: Avoid allocating redundant pools
Quoted 7 lines
include/linux/mmzone.h | 9 +++++---- kernel/dma/pool.c | 27 ++++++++++++++++++--------- mm/page_alloc.c | 8 ++------
3 files changed, 25 insertions(+), 19 deletions(-)--
2.34.1
Tested on platform: upcoming Mobileye eyeq7h - risc-v platform with memory mapped to 32Gbytes so DMA32 zone is empty
Marek Szyprowski
maintainerQuoted 13 lines · Robin Murphy
These days we have somewhat of a matrix of mm zones that may or may not exist and may or may not be empty, which the per-zone atomic pool logic doesn't actually handle all that well. It's not all that difficult to improve robustness and reduce redundancy, so that's what this series aims for.
I initially wrote this just because I happened to be looking through the boot log of one of my boards and had one of those out-of-the-blue realisations of "hang on, why *is* it allocating pools for zones it doesn't even have memory in?", but coincidentally, it seems that others happened to notice related aspects around the same time[1][2]. This is my attempt to sort it all out properly.
Quoted 2 lines
I guess this could go via either the dma-mapping or mm tree, whichever maintainers prefer.
I will take this patchset to dma-mapping-fixes branch, thanks!
Best regards
Signature · 2 lines
Marek Szyprowski, PhD Samsung R&D Institute Poland
Quoted 8 lines · Robin Murphy
Hi all,
These days we have somewhat of a matrix of mm zones that may or may not exist and may or may not be empty, which the per-zone atomic pool logic doesn't actually handle all that well. It's not all that difficult to improve robustness and reduce redundancy, so that's what this series aims for.
Quoted 6 lines
I initially wrote this just because I happened to be looking through the boot log of one of my boards and had one of those out-of-the-blue realisations of "hang on, why *is* it allocating pools for zones it doesn't even have memory in?", but coincidentally, it seems that others happened to notice related aspects around the same time[1][2]. This is my attempt to sort it all out properly.