Skip to contentSkip to conversation

mm/slab_common: move dma-kmalloc caches creation into new_kmalloc_cache()

PATCH In review Ohhoon Kwon 5 participants 5 messages Apr 10, 2022 – Apr 12, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 mm/slab_common: move dma-kmalloc caches creation into new_kmalloc_cache()

There are four types of kmalloc_caches: KMALLOC_NORMAL, KMALLOC_CGROUP, KMALLOC_RECLAIM, and KMALLOC_DMA. While the first three types are created using new_kmalloc_cache(), KMALLOC_DMA caches are created in a separate logic. Let KMALLOC_DMA caches be also created using new_kmalloc_cache(), to enhance readability.

Historically, there were only KMALLOC_NORMAL caches and KMALLOC_DMA caches in the first place, and they were initialized in two separate logics. However, when KMALLOC_RECLAIM was introduced in v4.20 via commit 1291523f2c1d ("mm, slab/slub: introduce kmalloc-reclaimable caches") and KMALLOC_CGROUP was introduced in v5.14 via commit 494c1dfe855e ("mm: memcg/slab: create a new set of kmalloc-cg-<n> caches"), their creations were merged with KMALLOC_NORMAL's only. KMALLOC_DMA creation logic should be merged with them, too.

By merging KMALLOC_DMA initialization with other types, the following two changes might occur:
1. The order dma-kmalloc-<n> caches added in slab_cache list may be sorted by size. i.e. the order they appear in /proc/slabinfo may change as well.
2. slab_state will be set to UP after KMALLOC_DMA is created. In case of slub, freelist randomization is dependent on slab_state>=UP, and therefore KMALLOC_DMA cache's freelist will not be randomized in creation, but will be deferred to init_freelist_randomization().

---
 mm/slab_common.c | 18 +++---------------
 1 file changed, 3 insertions(+), 15 deletions(-)
mm/slab_common.c +3 -15
index 6ee64d6208b3..a959d247c27b 100644
@@ -849,6 +849,8 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags)
 			return;
 		}
 		flags |= SLAB_ACCOUNT;
+	} else if (IS_ENABLED(CONFIG_ZONE_DMA) && (type == KMALLOC_DMA)) {
+		flags |= SLAB_CACHE_DMA;
 	}
 
 	kmalloc_caches[type][idx] = create_kmalloc_cache(
@@ -877,7 +879,7 @@ void __init create_kmalloc_caches(slab_flags_t flags)
 	/*
 	 * Including KMALLOC_CGROUP if CONFIG_MEMCG_KMEM defined
 	 */
-	for (type = KMALLOC_NORMAL; type <= KMALLOC_RECLAIM; type++) {
+	for (type = KMALLOC_NORMAL; type < NR_KMALLOC_TYPES; type++) {
 		for (i = KMALLOC_SHIFT_LOW; i <= KMALLOC_SHIFT_HIGH; i++) {
 			if (!kmalloc_caches[type][i])
 				new_kmalloc_cache(i, type, flags);
@@ -898,20 +900,6 @@ void __init create_kmalloc_caches(slab_flags_t flags)
 
 	/* Kmalloc array is now usable */
 	slab_state = UP;
-
-#ifdef CONFIG_ZONE_DMA
-	for (i = 0; i <= KMALLOC_SHIFT_HIGH; i++) {
-		struct kmem_cache *s = kmalloc_caches[KMALLOC_NORMAL][i];
-
-		if (s) {
-			kmalloc_caches[KMALLOC_DMA][i] = create_kmalloc_cache(
-				kmalloc_info[i].name[KMALLOC_DMA],
-				kmalloc_info[i].size,
-				SLAB_CACHE_DMA | flags, 0,
-				kmalloc_info[i].size);
-		}
-	}
-#endif
 }
 #endif /* !CONFIG_SLOB */
 
Signature · 1 line
2.25.1
Reviewed-by on v2 1/1
#1 lore raw
Quoted 65 lines · Ohhoon Kwon

There are four types of kmalloc_caches: KMALLOC_NORMAL, KMALLOC_CGROUP, KMALLOC_RECLAIM, and KMALLOC_DMA. While the first three types are created using new_kmalloc_cache(), KMALLOC_DMA caches are created in a separate logic. Let KMALLOC_DMA caches be also created using new_kmalloc_cache(), to enhance readability.

Historically, there were only KMALLOC_NORMAL caches and KMALLOC_DMA caches in the first place, and they were initialized in two separate logics. However, when KMALLOC_RECLAIM was introduced in v4.20 via commit 1291523f2c1d ("mm, slab/slub: introduce kmalloc-reclaimable caches") and KMALLOC_CGROUP was introduced in v5.14 via commit 494c1dfe855e ("mm: memcg/slab: create a new set of kmalloc-cg-<n> caches"), their creations were merged with KMALLOC_NORMAL's only. KMALLOC_DMA creation logic should be merged with them, too.

By merging KMALLOC_DMA initialization with other types, the following two changes might occur:
1. The order dma-kmalloc-<n> caches added in slab_cache list may be sorted by size. i.e. the order they appear in /proc/slabinfo may change as well.
2. slab_state will be set to UP after KMALLOC_DMA is created. In case of slub, freelist randomization is dependent on slab_state>=UP, and therefore KMALLOC_DMA cache's freelist will not be randomized in creation, but will be deferred to init_freelist_randomization().

---
 mm/slab_common.c | 18 +++---------------
 1 file changed, 3 insertions(+), 15 deletions(-)
mm/slab_common.c +3 -8
index 6ee64d6208b3..a959d247c27b 100644
@@ -849,6 +849,8 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags)
 			return;
 		}
 		flags |= SLAB_ACCOUNT;
+	} else if (IS_ENABLED(CONFIG_ZONE_DMA) && (type == KMALLOC_DMA)) {
+		flags |= SLAB_CACHE_DMA;
 	}
 
 	kmalloc_caches[type][idx] = create_kmalloc_cache(
@@ -877,7 +879,7 @@ void __init create_kmalloc_caches(slab_flags_t flags)
 	/*
 	 * Including KMALLOC_CGROUP if CONFIG_MEMCG_KMEM defined
 	 */
-	for (type = KMALLOC_NORMAL; type <= KMALLOC_RECLAIM; type++) {
+	for (type = KMALLOC_NORMAL; type < NR_KMALLOC_TYPES; type++) {
 		for (i = KMALLOC_SHIFT_LOW; i <= KMALLOC_SHIFT_HIGH; i++) {
 			if (!kmalloc_caches[type][i])
 				new_kmalloc_cache(i, type, flags);
@@ -898,20 +900,6 @@ void __init create_kmalloc_caches(slab_flags_t flags)
 
 	/* Kmalloc array is now usable */
 	slab_state = UP;
-
-#ifdef CONFIG_ZONE_DMA
-	for (i = 0; i <= KMALLOC_SHIFT_HIGH; i++) {
-		struct kmem_cache *s = kmalloc_caches[KMALLOC_NORMAL][i];
-
-		if (s) {
-			kmalloc_caches[KMALLOC_DMA][i] = create_kmalloc_cache(

mm/slab_common.c void __init create_kmalloc_caches(slab_flags_t flags) · line 905

Quoted 10 lines
mm/slab_common.c +0 -7
-				kmalloc_info[i].name[KMALLOC_DMA],
-				kmalloc_info[i].size,
-				SLAB_CACHE_DMA | flags, 0,
-				kmalloc_info[i].size);
-		}
-	}
-#endif
 }
 #endif /* !CONFIG_SLOB */
 

BTW this patch may conflict with [1] (not merged yet)

[1] https://lore.kernel.org/linux-mm/[email protected]/

Thanks!

Quoted 3 lines
Signature · 1 line
2.25.1
Signature · 2 lines
Thanks,
Hyeonggon
#2 lore raw
Quoted 70 lines · Hyeonggon Yoo
Quoted 69 lines · Ohhoon Kwon

There are four types of kmalloc_caches: KMALLOC_NORMAL, KMALLOC_CGROUP, KMALLOC_RECLAIM, and KMALLOC_DMA. While the first three types are created using new_kmalloc_cache(), KMALLOC_DMA caches are created in a separate logic. Let KMALLOC_DMA caches be also created using new_kmalloc_cache(), to enhance readability.

Historically, there were only KMALLOC_NORMAL caches and KMALLOC_DMA caches in the first place, and they were initialized in two separate logics. However, when KMALLOC_RECLAIM was introduced in v4.20 via commit 1291523f2c1d ("mm, slab/slub: introduce kmalloc-reclaimable caches") and KMALLOC_CGROUP was introduced in v5.14 via commit 494c1dfe855e ("mm: memcg/slab: create a new set of kmalloc-cg-<n> caches"), their creations were merged with KMALLOC_NORMAL's only. KMALLOC_DMA creation logic should be merged with them, too.

By merging KMALLOC_DMA initialization with other types, the following two changes might occur:
1. The order dma-kmalloc-<n> caches added in slab_cache list may be sorted by size. i.e. the order they appear in /proc/slabinfo may change as well.
2. slab_state will be set to UP after KMALLOC_DMA is created. In case of slub, freelist randomization is dependent on slab_state>=UP, and therefore KMALLOC_DMA cache's freelist will not be randomized in creation, but will be deferred to init_freelist_randomization().

---
 mm/slab_common.c | 18 +++---------------
 1 file changed, 3 insertions(+), 15 deletions(-)
mm/slab_common.c +3 -12
index 6ee64d6208b3..a959d247c27b 100644
@@ -849,6 +849,8 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags)
 			return;
 		}
 		flags |= SLAB_ACCOUNT;
+	} else if (IS_ENABLED(CONFIG_ZONE_DMA) && (type == KMALLOC_DMA)) {
+		flags |= SLAB_CACHE_DMA;
 	}
 
 	kmalloc_caches[type][idx] = create_kmalloc_cache(
@@ -877,7 +879,7 @@ void __init create_kmalloc_caches(slab_flags_t flags)
 	/*
 	 * Including KMALLOC_CGROUP if CONFIG_MEMCG_KMEM defined
 	 */
-	for (type = KMALLOC_NORMAL; type <= KMALLOC_RECLAIM; type++) {
+	for (type = KMALLOC_NORMAL; type < NR_KMALLOC_TYPES; type++) {
 		for (i = KMALLOC_SHIFT_LOW; i <= KMALLOC_SHIFT_HIGH; i++) {
 			if (!kmalloc_caches[type][i])
 				new_kmalloc_cache(i, type, flags);
@@ -898,20 +900,6 @@ void __init create_kmalloc_caches(slab_flags_t flags)
 
 	/* Kmalloc array is now usable */
 	slab_state = UP;
-
-#ifdef CONFIG_ZONE_DMA
-	for (i = 0; i <= KMALLOC_SHIFT_HIGH; i++) {
-		struct kmem_cache *s = kmalloc_caches[KMALLOC_NORMAL][i];
-
-		if (s) {
-			kmalloc_caches[KMALLOC_DMA][i] = create_kmalloc_cache(
-				kmalloc_info[i].name[KMALLOC_DMA],
-				kmalloc_info[i].size,
-				SLAB_CACHE_DMA | flags, 0,
-				kmalloc_info[i].size);
Quoted 8 lines

mm/slab_common.c void __init create_kmalloc_caches(slab_flags_t flags) · line 905

Quoted 6 lines
mm/slab_common.c +0 -3
-		}
-	}
-#endif
 }
 #endif /* !CONFIG_SLOB */
 

Thanks.
Added to:
https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab.git/log/?h=for-5.19/refactor

Quoted 3 lines

BTW this patch may conflict with [1] (not merged yet)

[1] https://lore.kernel.org/linux-mm/[email protected]/

FYI Catalin, might want to base v2 on the above once the crypto side is solved. At cursory look it shouldn't be a significant conflict.

Quoted 6 lines

Thanks!

Quoted 3 lines
Signature · 1 line
2.25.1
#3 lore raw
Quoted 29 lines · Vlastimil Babka
Quoted 15 lines · Hyeonggon Yoo
Quoted 14 lines · Ohhoon Kwon
mm/slab_common.c +0 -10
@@ -898,20 +900,6 @@ void __init create_kmalloc_caches(slab_flags_t flags)
 
 	/* Kmalloc array is now usable */
 	slab_state = UP;
-
-#ifdef CONFIG_ZONE_DMA
-	for (i = 0; i <= KMALLOC_SHIFT_HIGH; i++) {
-		struct kmem_cache *s = kmalloc_caches[KMALLOC_NORMAL][i];
-
-		if (s) {
-			kmalloc_caches[KMALLOC_DMA][i] = create_kmalloc_cache(
-				kmalloc_info[i].name[KMALLOC_DMA],
-				kmalloc_info[i].size,
-				SLAB_CACHE_DMA | flags, 0,
Quoted 8 lines

mm/slab_common.c void __init create_kmalloc_caches(slab_flags_t flags) · line 904

Quoted 6 lines
mm/slab_common.c +0 -4
-				kmalloc_info[i].size);
-		}
-	}
-#endif
 }
 #endif /* !CONFIG_SLOB */

Thanks.
Added to:
https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab.git/log/?h=for-5.19/refactor

Quoted 6 lines
Quoted 3 lines

BTW this patch may conflict with [1] (not merged yet)

[1] https://lore.kernel.org/linux-mm/[email protected]/

FYI Catalin, might want to base v2 on the above once the crypto side is solved. At cursory look it shouldn't be a significant conflict.

Thanks for the heads-up. I did wonder why this was a separate loop.

Signature · 1 line
Catalin
reviewer Acked-by on v2 1/1
#4 lore raw
In reply to Ohhoon Kwon
Quoted 25 lines · Ohhoon Kwon

There are four types of kmalloc_caches: KMALLOC_NORMAL, KMALLOC_CGROUP, KMALLOC_RECLAIM, and KMALLOC_DMA. While the first three types are created using new_kmalloc_cache(), KMALLOC_DMA caches are created in a separate logic. Let KMALLOC_DMA caches be also created using new_kmalloc_cache(), to enhance readability.

Historically, there were only KMALLOC_NORMAL caches and KMALLOC_DMA caches in the first place, and they were initialized in two separate logics. However, when KMALLOC_RECLAIM was introduced in v4.20 via commit 1291523f2c1d ("mm, slab/slub: introduce kmalloc-reclaimable caches") and KMALLOC_CGROUP was introduced in v5.14 via commit 494c1dfe855e ("mm: memcg/slab: create a new set of kmalloc-cg-<n> caches"), their creations were merged with KMALLOC_NORMAL's only. KMALLOC_DMA creation logic should be merged with them, too.

By merging KMALLOC_DMA initialization with other types, the following two changes might occur:
1. The order dma-kmalloc-<n> caches added in slab_cache list may be sorted by size. i.e. the order they appear in /proc/slabinfo may change as well.
2. slab_state will be set to UP after KMALLOC_DMA is created. In case of slub, freelist randomization is dependent on slab_state>=UP, and therefore KMALLOC_DMA cache's freelist will not be randomized in creation, but will be deferred to init_freelist_randomization().

Quoted 3 lines