mm/slab_common: move dma-kmalloc caches creation into new_kmalloc_cache()
Subsystems (2)
PATCH v2 1/1 mm/slab_common: move dma-kmalloc caches creation into new_kmalloc_cache()
There are four types of kmalloc_caches: KMALLOC_NORMAL, KMALLOC_CGROUP, KMALLOC_RECLAIM, and KMALLOC_DMA. While the first three types are created using new_kmalloc_cache(), KMALLOC_DMA caches are created in a separate logic. Let KMALLOC_DMA caches be also created using new_kmalloc_cache(), to enhance readability.
Historically, there were only KMALLOC_NORMAL caches and KMALLOC_DMA caches in the first place, and they were initialized in two separate logics. However, when KMALLOC_RECLAIM was introduced in v4.20 via commit 1291523f2c1d ("mm, slab/slub: introduce kmalloc-reclaimable caches") and KMALLOC_CGROUP was introduced in v5.14 via commit 494c1dfe855e ("mm: memcg/slab: create a new set of kmalloc-cg-<n> caches"), their creations were merged with KMALLOC_NORMAL's only. KMALLOC_DMA creation logic should be merged with them, too.
By merging KMALLOC_DMA initialization with other types, the following two changes might occur:
1. The order dma-kmalloc-<n> caches added in slab_cache list may be sorted by size. i.e. the order they appear in /proc/slabinfo may change as well.
2. slab_state will be set to UP after KMALLOC_DMA is created. In case of slub, freelist randomization is dependent on slab_state>=UP, and therefore KMALLOC_DMA cache's freelist will not be randomized in creation, but will be deferred to init_freelist_randomization().
--- mm/slab_common.c | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-)
mm/slab_common.c +3 -15
@@ -849,6 +849,8 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags)return ; } flags |=SLAB_ACCOUNT ; + }else if (IS_ENABLED (CONFIG_ZONE_DMA ) && (type ==KMALLOC_DMA )) { + flags |=SLAB_CACHE_DMA ; } kmalloc_caches[type][idx] = create_kmalloc_cache(
@@ -877,7 +879,7 @@ void __init create_kmalloc_caches(slab_flags_t flags)/* * Including KMALLOC_CGROUP if CONFIG_MEMCG_KMEM defined */ -for (type =KMALLOC_NORMAL ; type <=KMALLOC_RECLAIM ; type++) { +for (type =KMALLOC_NORMAL ; type <NR_KMALLOC_TYPES ; type++) {for (i =KMALLOC_SHIFT_LOW ; i <=KMALLOC_SHIFT_HIGH ; i++) {if (!kmalloc_caches[type][i])new_kmalloc_cache (i, type, flags);
@@ -898,20 +900,6 @@ void __init create_kmalloc_caches(slab_flags_t flags)/* Kmalloc array is now usable */ slab_state = UP; - -#ifdef CONFIG_ZONE_DMA -for (i =0 ; i <=KMALLOC_SHIFT_HIGH ; i++) { -struct kmem_cache *s = kmalloc_caches[KMALLOC_NORMAL ][i]; - -if (s) { - kmalloc_caches[KMALLOC_DMA ][i] =create_kmalloc_cache ( - kmalloc_info[i].name[KMALLOC_DMA ], - kmalloc_info[i].size, -SLAB_CACHE_DMA | flags,0 , - kmalloc_info[i].size); - } - } -#endif }#endif /* !CONFIG_SLOB */
Signature · 1 line
2.25.1
Quoted 65 lines · Ohhoon Kwon
There are four types of
kmalloc_caches:KMALLOC_NORMAL,KMALLOC_CGROUP,KMALLOC_RECLAIM, andKMALLOC_DMA. While the first three types are created usingnew_kmalloc_cache(),KMALLOC_DMAcaches are created in a separate logic. LetKMALLOC_DMAcaches be also created usingnew_kmalloc_cache(), to enhance readability.Historically, there were only
KMALLOC_NORMALcaches andKMALLOC_DMAcaches in the first place, and they were initialized in two separate logics. However, whenKMALLOC_RECLAIMwas introduced in v4.20 via commit1291523f2c1d("mm, slab/slub: introduce kmalloc-reclaimable caches") andKMALLOC_CGROUPwas introduced in v5.14 via commit494c1dfe855e("mm: memcg/slab: create a new set of kmalloc-cg-<n> caches"), their creations were merged with KMALLOC_NORMAL's only.KMALLOC_DMAcreation logic should be merged with them, too.By merging
KMALLOC_DMAinitialization with other types, the following two changes might occur:
1. The order dma-kmalloc-<n> caches added inslab_cachelist may be sorted by size. i.e. the order they appear in/proc/slabinfomay change as well.
2.slab_statewill be set to UP afterKMALLOC_DMAis created. In case of slub, freelist randomization is dependent on slab_state>=UP, and thereforeKMALLOC_DMAcache's freelist will not be randomized in creation, but will be deferred toinit_freelist_randomization().Co-developed-by JaeSang Yoo <[email protected]>Signed-off-by JaeSang Yoo <[email protected]>Signed-off-by Ohhoon Kwon <[email protected]>--- mm/slab_common.c | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-)mm/slab_common.c +3
-8@@ -849,6 +849,8 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags)return ; } flags |=SLAB_ACCOUNT ; + }else if (IS_ENABLED (CONFIG_ZONE_DMA ) && (type ==KMALLOC_DMA )) { + flags |=SLAB_CACHE_DMA ; } kmalloc_caches[type][idx] = create_kmalloc_cache(@@ -877,7 +879,7 @@ void __init create_kmalloc_caches(slab_flags_t flags)/* * Including KMALLOC_CGROUP if CONFIG_MEMCG_KMEM defined */ -for (type =KMALLOC_NORMAL ; type <=KMALLOC_RECLAIM ; type++) { +for (type =KMALLOC_NORMAL ; type <NR_KMALLOC_TYPES ; type++) {for (i =KMALLOC_SHIFT_LOW ; i <=KMALLOC_SHIFT_HIGH ; i++) {if (!kmalloc_caches[type][i])new_kmalloc_cache (i, type, flags);@@ -898,20 +900,6 @@ void __init create_kmalloc_caches(slab_flags_t flags)/* Kmalloc array is now usable */ slab_state =UP ; - -#ifdef CONFIG_ZONE_DMA -for (i =0 ; i <=KMALLOC_SHIFT_HIGH ; i++) { -struct kmem_cache *s = kmalloc_caches[KMALLOC_NORMAL ][i]; - -if (s) { - kmalloc_caches[KMALLOC_DMA][i] = create_kmalloc_cache(
mm/slab_common.c void __init create_kmalloc_caches(slab_flags_t flags) · line 905
Quoted 10 lines
mm/slab_common.c +0
-7- kmalloc_info[i].name[KMALLOC_DMA ], - kmalloc_info[i].size, -SLAB_CACHE_DMA | flags,0 , - kmalloc_info[i].size); - } - } -#endif }#endif /* !CONFIG_SLOB */
BTW this patch may conflict with [1] (not merged yet)
[1] https://lore.kernel.org/linux-mm/[email protected]/
Thanks!
Quoted 3 lines
Signature · 1 line
2.25.1
Signature · 2 lines
Thanks, Hyeonggon
Quoted 70 lines · Hyeonggon Yoo
Quoted 69 lines · Ohhoon Kwon
There are four types of
kmalloc_caches:KMALLOC_NORMAL,KMALLOC_CGROUP,KMALLOC_RECLAIM, andKMALLOC_DMA. While the first three types are created usingnew_kmalloc_cache(),KMALLOC_DMAcaches are created in a separate logic. LetKMALLOC_DMAcaches be also created usingnew_kmalloc_cache(), to enhance readability.Historically, there were only
KMALLOC_NORMALcaches andKMALLOC_DMAcaches in the first place, and they were initialized in two separate logics. However, whenKMALLOC_RECLAIMwas introduced in v4.20 via commit1291523f2c1d("mm, slab/slub: introduce kmalloc-reclaimable caches") andKMALLOC_CGROUPwas introduced in v5.14 via commit494c1dfe855e("mm: memcg/slab: create a new set of kmalloc-cg-<n> caches"), their creations were merged with KMALLOC_NORMAL's only.KMALLOC_DMAcreation logic should be merged with them, too.By merging
KMALLOC_DMAinitialization with other types, the following two changes might occur:
1. The order dma-kmalloc-<n> caches added inslab_cachelist may be sorted by size. i.e. the order they appear in/proc/slabinfomay change as well.
2.slab_statewill be set to UP afterKMALLOC_DMAis created. In case of slub, freelist randomization is dependent on slab_state>=UP, and thereforeKMALLOC_DMAcache's freelist will not be randomized in creation, but will be deferred toinit_freelist_randomization().Co-developed-by JaeSang Yoo <[email protected]>Signed-off-by JaeSang Yoo <[email protected]>Signed-off-by Ohhoon Kwon <[email protected]>--- mm/slab_common.c | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-)mm/slab_common.c +3
-12@@ -849,6 +849,8 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags)return ; } flags |=SLAB_ACCOUNT ; + }else if (IS_ENABLED (CONFIG_ZONE_DMA ) && (type ==KMALLOC_DMA )) { + flags |=SLAB_CACHE_DMA ; } kmalloc_caches[type][idx] = create_kmalloc_cache(@@ -877,7 +879,7 @@ void __init create_kmalloc_caches(slab_flags_t flags)/* * Including KMALLOC_CGROUP if CONFIG_MEMCG_KMEM defined */ -for (type =KMALLOC_NORMAL ; type <=KMALLOC_RECLAIM ; type++) { +for (type =KMALLOC_NORMAL ; type <NR_KMALLOC_TYPES ; type++) {for (i =KMALLOC_SHIFT_LOW ; i <=KMALLOC_SHIFT_HIGH ; i++) {if (!kmalloc_caches[type][i])new_kmalloc_cache (i, type, flags);@@ -898,20 +900,6 @@ void __init create_kmalloc_caches(slab_flags_t flags)/* Kmalloc array is now usable */ slab_state =UP ; - -#ifdef CONFIG_ZONE_DMA -for (i =0 ; i <=KMALLOC_SHIFT_HIGH ; i++) { -struct kmem_cache *s = kmalloc_caches[KMALLOC_NORMAL ][i]; - -if (s) { - kmalloc_caches[KMALLOC_DMA ][i] =create_kmalloc_cache ( - kmalloc_info[i].name[KMALLOC_DMA ], - kmalloc_info[i].size, -SLAB_CACHE_DMA | flags,0 , - kmalloc_info[i].size);
Quoted 8 lines
mm/slab_common.c void __init create_kmalloc_caches(slab_flags_t flags) · line 905
Quoted 6 lines
mm/slab_common.c +0
-3- } - } -#endif }#endif /* !CONFIG_SLOB */ Reviewed-by Hyeonggon Yoo <[email protected]>
Thanks.
Added to:
https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab.git/log/?h=for-5.19/refactor
Quoted 3 lines
BTW this patch may conflict with [1] (not merged yet)
FYI Catalin, might want to base v2 on the above once the crypto side is solved. At cursory look it shouldn't be a significant conflict.
Quoted 6 lines
Thanks!
Quoted 3 lines
Signature · 1 line
2.25.1
Quoted 29 lines · Vlastimil Babka
Quoted 15 lines · Hyeonggon Yoo
Quoted 14 lines · Ohhoon Kwon
mm/slab_common.c +0
-10@@ -898,20 +900,6 @@ void __init create_kmalloc_caches(slab_flags_t flags)/* Kmalloc array is now usable */ slab_state =UP ; - -#ifdef CONFIG_ZONE_DMA -for (i =0 ; i <=KMALLOC_SHIFT_HIGH ; i++) { -struct kmem_cache *s = kmalloc_caches[KMALLOC_NORMAL ][i]; - -if (s) { - kmalloc_caches[KMALLOC_DMA][i] = create_kmalloc_cache( - kmalloc_info[i].name[KMALLOC_DMA], - kmalloc_info[i].size, - SLAB_CACHE_DMA | flags, 0,Quoted 8 lines
mm/slab_common.c void __init create_kmalloc_caches(slab_flags_t flags) · line 904
Quoted 6 lines
mm/slab_common.c +0
-4- kmalloc_info[i].size); - } - } -#endif }#endif /* !CONFIG_SLOB */ Reviewed-by Hyeonggon Yoo <[email protected]>Thanks.
Added to:
https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab.git/log/?h=for-5.19/refactor
Quoted 6 lines
Quoted 3 lines
BTW this patch may conflict with [1] (not merged yet)
FYI Catalin, might want to base v2 on the above once the crypto side is solved. At cursory look it shouldn't be a significant conflict.
Thanks for the heads-up. I did wonder why this was a separate loop.
Signature · 1 line
Catalin
Quoted 25 lines · Ohhoon Kwon
There are four types of
kmalloc_caches:KMALLOC_NORMAL,KMALLOC_CGROUP,KMALLOC_RECLAIM, andKMALLOC_DMA. While the first three types are created usingnew_kmalloc_cache(),KMALLOC_DMAcaches are created in a separate logic. LetKMALLOC_DMAcaches be also created usingnew_kmalloc_cache(), to enhance readability.Historically, there were only
KMALLOC_NORMALcaches andKMALLOC_DMAcaches in the first place, and they were initialized in two separate logics. However, whenKMALLOC_RECLAIMwas introduced in v4.20 via commit1291523f2c1d("mm, slab/slub: introduce kmalloc-reclaimable caches") andKMALLOC_CGROUPwas introduced in v5.14 via commit494c1dfe855e("mm: memcg/slab: create a new set of kmalloc-cg-<n> caches"), their creations were merged with KMALLOC_NORMAL's only.KMALLOC_DMAcreation logic should be merged with them, too.By merging
KMALLOC_DMAinitialization with other types, the following two changes might occur:
1. The order dma-kmalloc-<n> caches added inslab_cachelist may be sorted by size. i.e. the order they appear in/proc/slabinfomay change as well.
2.slab_statewill be set to UP afterKMALLOC_DMAis created. In case of slub, freelist randomization is dependent on slab_state>=UP, and thereforeKMALLOC_DMAcache's freelist will not be randomized in creation, but will be deferred toinit_freelist_randomization().
Quoted 3 lines
Co-developed-by JaeSang Yoo <[email protected]>Signed-off-by JaeSang Yoo <[email protected]>Signed-off-by Ohhoon Kwon <[email protected]>