Skip to contentSkip to conversation

dma-mapping: Improve atomic pool behaviour

PATCH In review Robin Murphy 6 participants 12 messages Jan 12 – Jan 14

Subsystems (5)
lore
Share & tools
lore

Robin Murphy

author reviewer
#0 lore raw

cover dma-mapping: Improve atomic pool behaviour

Hi all,

These days we have somewhat of a matrix of mm zones that may or may not exist and may or may not be empty, which the per-zone atomic pool logic doesn't actually handle all that well. It's not all that difficult to improve robustness and reduce redundancy, so that's what this series aims for.

I initially wrote this just because I happened to be looking through the boot log of one of my boards and had one of those out-of-the-blue realisations of "hang on, why *is* it allocating pools for zones it doesn't even have memory in?", but coincidentally, it seems that others happened to notice related aspects around the same time[1][2]. This is my attempt to sort it all out properly.

I guess this could go via either the dma-mapping or mm tree, whichever maintainers prefer.

Thanks,
Robin.

[1] https://lore.kernel.org/linux-iommu/[email protected]/
[2] https://lore.kernel.org/linux-iommu/[email protected]/

Robin Murphy (3):
dma/pool: Improve pool lookup
mm_zone: Generalise has_managed_dma()
dma/pool: Avoid allocating redundant pools

 include/linux/mmzone.h |  9 +++++----
 kernel/dma/pool.c      | 27 ++++++++++++++++++---------
 mm/page_alloc.c        |  8 ++------
 3 files changed, 25 insertions(+), 19 deletions(-)
Signature · 1 line
2.34.1
author reviewer
#1 lore raw

PATCH 1/3 dma/pool: Improve pool lookup

If CONFIG_ZONE_DMA32 is enabled, but we have not allocated the corresponding atomic_pool_dma32, dma_guess_pool() may return the NULL value of that and fail a GFP_DMA32 allocation without trying to fall back to other pools which may exist. Furthermore, if no GFP_DMA pool exists, it is preferable to try GFP_DMA32 rather than immediately fall back to GFP_KERNEL with even less chance of success. Improve matters by encoding an explicit order of pool preference for each flag.

---
 kernel/dma/pool.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/dma/pool.c +4 -4
index 26392badc36b..2645cfb5718b 100644
@@ -224,10 +224,10 @@ postcore_initcall(dma_atomic_pool_init);
 static inline struct gen_pool *dma_guess_pool(struct gen_pool *prev, gfp_t gfp)
 {
 	if (prev == NULL) {
-		if (IS_ENABLED(CONFIG_ZONE_DMA32) && (gfp & GFP_DMA32))
-			return atomic_pool_dma32;
-		if (atomic_pool_dma && (gfp & GFP_DMA))
-			return atomic_pool_dma;
+		if (gfp & GFP_DMA)
+			return atomic_pool_dma ?: atomic_pool_dma32 ?: atomic_pool_kernel;
+		if (gfp & GFP_DMA32)
+			return atomic_pool_dma32 ?: atomic_pool_dma ?: atomic_pool_kernel;
 		return atomic_pool_kernel;
 	}
 	if (prev == atomic_pool_kernel)
Signature · 1 line
2.34.1
author reviewer
#2 lore raw

PATCH 2/3 mm_zone: Generalise has_managed_dma()

In reply to Robin Murphy

It would be useful to be able to check for potential DMA pages beyond just ZONE_DMA - generalise the existing has_managed_dma() function to allow checking other zones too.

---
 include/linux/mmzone.h | 9 +++++----
 mm/page_alloc.c        | 8 ++------
 2 files changed, 7 insertions(+), 10 deletions(-)
include/linux/mmzone.h +5 -4
index 75ef7c9f9307..fc5d6c88d2f0 100644
@@ -1648,14 +1648,15 @@ static inline int is_highmem(const struct zone *zone)
 	return is_highmem_idx(zone_idx(zone));
 }
 
-#ifdef CONFIG_ZONE_DMA
-bool has_managed_dma(void);
-#else
+bool has_managed_zone(enum zone_type zone);
 static inline bool has_managed_dma(void)
 {
+#ifdef CONFIG_ZONE_DMA
+	return has_managed_zone(ZONE_DMA);
+#else
 	return false;
-}
 #endif
+}
 
 
 #ifndef CONFIG_NUMA
mm/page_alloc.c +2 -6
index 822e05f1a964..36ccc85c5073 100644
@@ -7418,20 +7418,16 @@ bool put_page_back_buddy(struct page *page)
 }
 #endif
 
-#ifdef CONFIG_ZONE_DMA
-bool has_managed_dma(void)
+bool has_managed_zone(enum zone_type zone)
 {
 	struct pglist_data *pgdat;
 
 	for_each_online_pgdat(pgdat) {
-		struct zone *zone = &pgdat->node_zones[ZONE_DMA];
-
-		if (managed_zone(zone))
+		if (managed_zone(&pgdat->node_zones[zone]))
 			return true;
 	}
 	return false;
 }
-#endif /* CONFIG_ZONE_DMA */
 
 #ifdef CONFIG_UNACCEPTED_MEMORY
 
Signature · 1 line
2.34.1
author reviewer
#5 lore raw

PATCH 3/3 dma/pool: Avoid allocating redundant pools

In reply to Robin Murphy

On smaller systems, e.g. embedded arm64, it is common for all memory to end up in ZONE_DMA32 or even ZONE_DMA. In such cases it is redundant to allocate a nominal pool for an empty higher zone that just ends up coming from a lower zone that should already have its own pool anyway. We already have logic to skip allocating a ZONE_DMA pool when that is empty, so generalise that to save memory in the case of other zones too.

---
 kernel/dma/pool.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)
kernel/dma/pool.c +14 -5
index 2645cfb5718b..c5da29ad010c 100644
@@ -184,6 +184,12 @@ static __init struct gen_pool *__dma_atomic_pool_init(size_t pool_size,
 	return pool;
 }
 
+#ifdef CONFIG_ZONE_DMA32
+#define has_managed_dma32 has_managed_zone(ZONE_DMA32)
+#else
+#define has_managed_dma32 false
+#endif
+
 static int __init dma_atomic_pool_init(void)
 {
 	int ret = 0;
@@ -199,17 +205,20 @@ static int __init dma_atomic_pool_init(void)
 	}
 	INIT_WORK(&atomic_pool_work, atomic_pool_work_fn);
 
-	atomic_pool_kernel = __dma_atomic_pool_init(atomic_pool_size,
+	/* All memory might be in the DMA zone(s) to begin with */
+	if (has_managed_zone(ZONE_NORMAL)) {
+		atomic_pool_kernel = __dma_atomic_pool_init(atomic_pool_size,
 						    GFP_KERNEL);
-	if (!atomic_pool_kernel)
-		ret = -ENOMEM;
+		if (!atomic_pool_kernel)
+			ret = -ENOMEM;
+	}
 	if (has_managed_dma()) {
 		atomic_pool_dma = __dma_atomic_pool_init(atomic_pool_size,
 						GFP_KERNEL | GFP_DMA);
 		if (!atomic_pool_dma)
 			ret = -ENOMEM;
 	}
-	if (IS_ENABLED(CONFIG_ZONE_DMA32)) {
+	if (has_managed_dma32) {
 		atomic_pool_dma32 = __dma_atomic_pool_init(atomic_pool_size,
 						GFP_KERNEL | GFP_DMA32);
 		if (!atomic_pool_dma32)
@@ -228,7 +237,7 @@ static inline struct gen_pool *dma_guess_pool(struct gen_pool *prev, gfp_t gfp)
 			return atomic_pool_dma ?: atomic_pool_dma32 ?: atomic_pool_kernel;
 		if (gfp & GFP_DMA32)
 			return atomic_pool_dma32 ?: atomic_pool_dma ?: atomic_pool_kernel;
-		return atomic_pool_kernel;
+		return atomic_pool_kernel ?: atomic_pool_dma32 ?: atomic_pool_dma;
 	}
 	if (prev == atomic_pool_kernel)
 		return atomic_pool_dma32 ? atomic_pool_dma32 : atomic_pool_dma;
Signature · 1 line
2.34.1
maintainer Acked-by on v1 2/3
#3 lore raw

Re: [PATCH 2/3] mm_zone: Generalise has_managed_dma()

In reply to Robin Murphy
Quoted 6 lines · Robin Murphy

It would be useful to be able to check for potential DMA pages beyond just ZONE_DMA - generalise the existing has_managed_dma() function to allow checking other zones too.

---

Signature · 3 lines
Cheers

David
#6 lore raw

Re: [PATCH 3/3] dma/pool: Avoid allocating redundant pools

In reply to Robin Murphy
Quoted 2 lines · Robin Murphy

On smaller systems, e.g. embedded arm64, it is common for all memory to end up in ZONE_DMA32 or even ZONE_DMA. In such cases it is redundant

This is true and the whole series looks great to me. Do we need adjust warn_alloc() to handle empty DMA32 zone too like empty DMA zone case?

Quoted 65 lines

to allocate a nominal pool for an empty higher zone that just ends up coming from a lower zone that should already have its own pool anyway. We already have logic to skip allocating a ZONE_DMA pool when that is empty, so generalise that to save memory in the case of other zones too.

---
 kernel/dma/pool.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)
kernel/dma/pool.c +14 -5
index 2645cfb5718b..c5da29ad010c 100644
@@ -184,6 +184,12 @@ static __init struct gen_pool *__dma_atomic_pool_init(size_t pool_size,
 	return pool;
 }
 
+#ifdef CONFIG_ZONE_DMA32
+#define has_managed_dma32 has_managed_zone(ZONE_DMA32)
+#else
+#define has_managed_dma32 false
+#endif
+
 static int __init dma_atomic_pool_init(void)
 {
 	int ret = 0;
@@ -199,17 +205,20 @@ static int __init dma_atomic_pool_init(void)
 	}
 	INIT_WORK(&atomic_pool_work, atomic_pool_work_fn);
 
-	atomic_pool_kernel = __dma_atomic_pool_init(atomic_pool_size,
+	/* All memory might be in the DMA zone(s) to begin with */
+	if (has_managed_zone(ZONE_NORMAL)) {
+		atomic_pool_kernel = __dma_atomic_pool_init(atomic_pool_size,
 						    GFP_KERNEL);
-	if (!atomic_pool_kernel)
-		ret = -ENOMEM;
+		if (!atomic_pool_kernel)
+			ret = -ENOMEM;
+	}
 	if (has_managed_dma()) {
 		atomic_pool_dma = __dma_atomic_pool_init(atomic_pool_size,
 						GFP_KERNEL | GFP_DMA);
 		if (!atomic_pool_dma)
 			ret = -ENOMEM;
 	}
-	if (IS_ENABLED(CONFIG_ZONE_DMA32)) {
+	if (has_managed_dma32) {
 		atomic_pool_dma32 = __dma_atomic_pool_init(atomic_pool_size,
 						GFP_KERNEL | GFP_DMA32);
 		if (!atomic_pool_dma32)
@@ -228,7 +237,7 @@ static inline struct gen_pool *dma_guess_pool(struct gen_pool *prev, gfp_t gfp)
 			return atomic_pool_dma ?: atomic_pool_dma32 ?: atomic_pool_kernel;
 		if (gfp & GFP_DMA32)
 			return atomic_pool_dma32 ?: atomic_pool_dma ?: atomic_pool_kernel;
-		return atomic_pool_kernel;
+		return atomic_pool_kernel ?: atomic_pool_dma32 ?: atomic_pool_dma;
 	}
 	if (prev == atomic_pool_kernel)
 		return atomic_pool_dma32 ? atomic_pool_dma32 : atomic_pool_dma;
Signature · 1 line
2.34.1
Tested-by on the v1 series
#9 lore raw
In reply to Robin Murphy
Quoted 30 lines

Hi all,

These days we have somewhat of a matrix of mm zones that may or may not exist and may or may not be empty, which the per-zone atomic pool logic doesn't actually handle all that well. It's not all that difficult to improve robustness and reduce redundancy, so that's what this series aims for.

I initially wrote this just because I happened to be looking through the boot log of one of my boards and had one of those out-of-the-blue realisations of "hang on, why *is* it allocating pools for zones it doesn't even have memory in?", but coincidentally, it seems that others happened to notice related aspects around the same time[1][2]. This is my attempt to sort it all out properly.

I guess this could go via either the dma-mapping or mm tree, whichever maintainers prefer.

Thanks,
Robin.

[1] Suspicious Link - Removed
[2] Suspicious Link - Removed

Robin Murphy (3):
  dma/pool: Improve pool lookup
  mm_zone: Generalise has_managed_dma()
  dma/pool: Avoid allocating redundant pools

Quoted 7 lines

 include/linux/mmzone.h |  9 +++++---- kernel/dma/pool.c      | 27 ++++++++++++++++++--------- mm/page_alloc.c        |  8 ++------

 3 files changed, 25 insertions(+), 19 deletions(-)

--
2.34.1

Tested on platform: upcoming Mobileye eyeq7h - risc-v platform with memory mapped to 32Gbytes so DMA32 zone is empty

author reviewer
#7 lore raw

Re: [PATCH 3/3] dma/pool: Avoid allocating redundant pools

In reply to Baoquan He
Quoted 6 lines · Baoquan He
Quoted 2 lines · Robin Murphy

On smaller systems, e.g. embedded arm64, it is common for all memory to end up in ZONE_DMA32 or even ZONE_DMA. In such cases it is redundant

This is true and the whole series looks great to me. Do we need adjust warn_alloc() to handle empty DMA32 zone too like empty DMA zone case?

Hmm, I'd be inclined to think that if nobody's complaining already then we can probably just leave it as-is. A GFP_DMA32 allocation won't OOM unless *both* ZONE_DMA32 and ZONE_DMA are empty, right? At that point I'd imagine it's a bit more significant if someone who wants DMA32 memory can't have any - don't we have a mechanism for reserving some "low" memory for kdump for pretty much this exact reason?

A special case for when ZONE_DMA is tiny such that GFP_DMA can be expected to fail often seems fair, but in general I'd expect that if GFP_DMA32 starts failing then it's more a sign of a genuine mismatch between the kernel's expectations and the system configuration.

Thanks,
Robin.

Quoted 66 lines
Quoted 65 lines

to allocate a nominal pool for an empty higher zone that just ends up coming from a lower zone that should already have its own pool anyway. We already have logic to skip allocating a ZONE_DMA pool when that is empty, so generalise that to save memory in the case of other zones too.

---
  kernel/dma/pool.c | 19 ++++++++++++++-----
  1 file changed, 14 insertions(+), 5 deletions(-)
kernel/dma/pool.c +14 -5
index 2645cfb5718b..c5da29ad010c 100644
@@ -184,6 +184,12 @@ static __init struct gen_pool *__dma_atomic_pool_init(size_t pool_size,
  	return pool;
  }
 
+#ifdef CONFIG_ZONE_DMA32
+#define has_managed_dma32 has_managed_zone(ZONE_DMA32)
+#else
+#define has_managed_dma32 false
+#endif
+
  static int __init dma_atomic_pool_init(void)
  {
  	int ret = 0;
@@ -199,17 +205,20 @@ static int __init dma_atomic_pool_init(void)
  	}
  	INIT_WORK(&atomic_pool_work, atomic_pool_work_fn);
 
-	atomic_pool_kernel = __dma_atomic_pool_init(atomic_pool_size,
+	/* All memory might be in the DMA zone(s) to begin with */
+	if (has_managed_zone(ZONE_NORMAL)) {
+		atomic_pool_kernel = __dma_atomic_pool_init(atomic_pool_size,
  						    GFP_KERNEL);
-	if (!atomic_pool_kernel)
-		ret = -ENOMEM;
+		if (!atomic_pool_kernel)
+			ret = -ENOMEM;
+	}
  	if (has_managed_dma()) {
  		atomic_pool_dma = __dma_atomic_pool_init(atomic_pool_size,
  						GFP_KERNEL | GFP_DMA);
  		if (!atomic_pool_dma)
  			ret = -ENOMEM;
  	}
-	if (IS_ENABLED(CONFIG_ZONE_DMA32)) {
+	if (has_managed_dma32) {
  		atomic_pool_dma32 = __dma_atomic_pool_init(atomic_pool_size,
  						GFP_KERNEL | GFP_DMA32);
  		if (!atomic_pool_dma32)
@@ -228,7 +237,7 @@ static inline struct gen_pool *dma_guess_pool(struct gen_pool *prev, gfp_t gfp)
  			return atomic_pool_dma ?: atomic_pool_dma32 ?: atomic_pool_kernel;
  		if (gfp & GFP_DMA32)
  			return atomic_pool_dma32 ?: atomic_pool_dma ?: atomic_pool_kernel;
-		return atomic_pool_kernel;
+		return atomic_pool_kernel ?: atomic_pool_dma32 ?: atomic_pool_dma;
  	}
  	if (prev == atomic_pool_kernel)
  		return atomic_pool_dma32 ? atomic_pool_dma32 : atomic_pool_dma;
Signature · 1 line
2.34.1
maintainer
#10 lore raw
In reply to Robin Murphy
Quoted 13 lines · Robin Murphy

These days we have somewhat of a matrix of mm zones that may or may not exist and may or may not be empty, which the per-zone atomic pool logic doesn't actually handle all that well. It's not all that difficult to improve robustness and reduce redundancy, so that's what this series aims for.

I initially wrote this just because I happened to be looking through the boot log of one of my boards and had one of those out-of-the-blue realisations of "hang on, why *is* it allocating pools for zones it doesn't even have memory in?", but coincidentally, it seems that others happened to notice related aspects around the same time[1][2]. This is my attempt to sort it all out properly.

Quoted 2 lines

I guess this could go via either the dma-mapping or mm tree, whichever maintainers prefer.

I will take this patchset to dma-mapping-fixes branch, thanks!

Best regards

Signature · 2 lines
Marek Szyprowski, PhD
Samsung R&D Institute Poland
#8 lore raw

Re: [PATCH 3/3] dma/pool: Avoid allocating redundant pools

In reply to Robin Murphy
Quoted 15 lines · Robin Murphy
Quoted 6 lines · Baoquan He
Quoted 2 lines · Robin Murphy

On smaller systems, e.g. embedded arm64, it is common for all memory to end up in ZONE_DMA32 or even ZONE_DMA. In such cases it is redundant

This is true and the whole series looks great to me. Do we need adjust warn_alloc() to handle empty DMA32 zone too like empty DMA zone case?

Hmm, I'd be inclined to think that if nobody's complaining already then we can probably just leave it as-is. A GFP_DMA32 allocation won't OOM unless *both* ZONE_DMA32 and ZONE_DMA are empty, right? At that point I'd imagine it's a bit more significant if someone who wants DMA32 memory can't have any
- don't we have a mechanism for reserving some "low" memory for kdump for pretty much this exact reason?

Quoted 4 lines

A special case for when ZONE_DMA is tiny such that GFP_DMA can be expected to fail often seems fair, but in general I'd expect that if GFP_DMA32 starts failing then it's more a sign of a genuine mismatch between the kernel's expectations and the system configuration.

Makes sense to me. I remember someone plans to take off both DMA and DMA32 zone in linux kernel, but that might not happen soon.

Reviewed-by on the v1 series
#11 lore raw
In reply to Robin Murphy
Quoted 8 lines · Robin Murphy

Hi all,

These days we have somewhat of a matrix of mm zones that may or may not exist and may or may not be empty, which the per-zone atomic pool logic doesn't actually handle all that well. It's not all that difficult to improve robustness and reduce redundancy, so that's what this series aims for.

Quoted 6 lines

I initially wrote this just because I happened to be looking through the boot log of one of my boards and had one of those out-of-the-blue realisations of "hang on, why *is* it allocating pools for zones it doesn't even have memory in?", but coincidentally, it seems that others happened to notice related aspects around the same time[1][2]. This is my attempt to sort it all out properly.

reviewer Acked-by on v1 2/3
#4 lore raw

Re: [PATCH 2/3] mm_zone: Generalise has_managed_dma()

In reply to Robin Murphy
Quoted 5 lines · Robin Murphy

It would be useful to be able to check for potential DMA pages beyond just ZONE_DMA - generalise the existing has_managed_dma() function to allow checking other zones too.

Quoted 59 lines
---
 include/linux/mmzone.h | 9 +++++----
 mm/page_alloc.c        | 8 ++------
 2 files changed, 7 insertions(+), 10 deletions(-)
include/linux/mmzone.h +5 -4
index 75ef7c9f9307..fc5d6c88d2f0 100644
@@ -1648,14 +1648,15 @@ static inline int is_highmem(const struct zone *zone)
 	return is_highmem_idx(zone_idx(zone));
 }
 
-#ifdef CONFIG_ZONE_DMA
-bool has_managed_dma(void);
-#else
+bool has_managed_zone(enum zone_type zone);
 static inline bool has_managed_dma(void)
 {
+#ifdef CONFIG_ZONE_DMA
+	return has_managed_zone(ZONE_DMA);
+#else
 	return false;
-}
 #endif
+}
 
 
 #ifndef CONFIG_NUMA
mm/page_alloc.c +2 -6
index 822e05f1a964..36ccc85c5073 100644
@@ -7418,20 +7418,16 @@ bool put_page_back_buddy(struct page *page)
 }
 #endif
 
-#ifdef CONFIG_ZONE_DMA
-bool has_managed_dma(void)
+bool has_managed_zone(enum zone_type zone)
 {
 	struct pglist_data *pgdat;
 
 	for_each_online_pgdat(pgdat) {
-		struct zone *zone = &pgdat->node_zones[ZONE_DMA];
-
-		if (managed_zone(zone))
+		if (managed_zone(&pgdat->node_zones[zone]))
 			return true;
 	}
 	return false;
 }
-#endif /* CONFIG_ZONE_DMA */
 
 #ifdef CONFIG_UNACCEPTED_MEMORY
 
Signature · 1 line
2.34.1
Signature · 2 lines
Sincerely yours,
Mike.