Skip to contentSkip to conversation

slab: add kmem_cache_setup()

RFC PATCH Superseded Christian Brauner 2 participants 8 messages Sep 2, 2024

Subsystems (6)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Christian Brauner

author maintainer
#0 lore raw

cover slab: add kmem_cache_setup()

Hey,

As discussed last week the various kmem_cache_*() functions should be replaced by a unified function that is based around a struct, with only the basic parameters passed separately.

Vlastimil already hinted that he would like to keep core parameters out of the struct: name, object size, and flags. I personally don't care much and would not object to moving everything into the struct but that's a matter of taste and I yield that decision power to the maintainer.

Here's an RFC built for a kmem_cache_setup() based on struct kmem_cache_args.

The choice of name is somewhat forced as kmem_cache_create() is taken and the only way to reuse it would be to replace all users in one go. Or to do a global sed/kmem_cache_create()/kmem_cache_create2()/g. And then introduce kmem_cache_setup(). That doesn't strike me as a viable option.

If we really cared about the *_create() suffix then an alternative might be to do a sed/kmem_cache_setup()/kmem_cache_create()/g after every user in the kernel is ported. I honestly don't think that's worth it but I wanted to at least mention it to highlight the fact that this might lead to a naming compromise.

From a cursory grep (and not excluding Documentation mentions) we will have to replace 44 kmem_cache_create_usercopy() calls and about 463 kmem_cache_create() calls which makes for a bit above 500 calls to port to kmem_cache_setup(). That'll probably be good work for people getting into kernel development.

Anyway, I wanted to get an RFC out to get some rough agreement on what the struct should look like, get some bikeshedding on the name done, and what else needs to be massaged as part of this. I think that cache_create() is the deepest we should stuff struct kmem_cache_args into the bowels of slab.

---
Christian Brauner (6):
slab: introduce kmem_cache_setup()
slab: port KMEM_CACHE() to kmem_cache_setup()
slab: port KMEM_CACHE_USERCOPY() to kmem_cache_setup()
file: port to kmem_cache_setup()
slab: remove kmem_cache_create_rcu()
io_uring: port to kmem_cache_setup()

 Documentation/core-api/memory-allocation.rst |  10 +-
 fs/file_table.c                              |  12 ++-
 include/linux/slab.h                         |  51 ++++++++---
 io_uring/io_uring.c                          |  15 +--
 mm/slab_common.c                             | 132 ++++++++++++---------------
 5 files changed, 121 insertions(+), 99 deletions(-)

---
base-commit: 6e016babce7c845ed015da25c7a097fa3482d95a
change-id: 20240902-work-kmem_cache_args-e9760972c7d4

author maintainer
#1 lore raw

RFC PATCH 1/6 slab: introduce kmem_cache_setup()

Replace the custom kmem_cache_*() functions with a unified kmem_cache_setup() function that is based on struct kmem_cache_args and will replace kmem_cache_create(), kmem_cache_create_usercopy(), and kmem_cache_create_rcu().

The @name, @object_size, and @flags parameters are passed separately as they are nearly universally used. The rest of the arguments moves into struct kmem_cache_args.

A new "use_freeptr_offset" boolean is added as zero is a valid freelist pointer offset. The boolean allows callers to avoid having to do anything special if they don't care about freelist pointer offsets (most callers don't).

---
 Documentation/core-api/memory-allocation.rst |  10 +--
 include/linux/slab.h                         |  21 ++++++
 mm/slab_common.c                             | 102 +++++++++++++++++----------
 3 files changed, 91 insertions(+), 42 deletions(-)
Documentation/core-api/memory-allocation.rst +5 -5
index 8b84eb4bdae7..83ae3021c89a 100644
@@ -166,11 +166,11 @@ documentation. Note that `kvmalloc` may return memory that is not
 physically contiguous.
 
 If you need to allocate many identical objects you can use the slab
-cache allocator. The cache should be set up with kmem_cache_create() or
-kmem_cache_create_usercopy() before it can be used. The second function
-should be used if a part of the cache might be copied to the userspace.
-After the cache is created kmem_cache_alloc() and its convenience
-wrappers can allocate memory from that cache.
+cache allocator. The cache should be set up with kmem_cache_setup()
+before it can be used. The second function should be used if a part of
+the cache might be copied to the userspace. After the cache is created
+kmem_cache_alloc() and its convenience wrappers can allocate memory from
+that cache.
 
 When the allocated memory is no longer needed it must be freed.
 
include/linux/slab.h +21 -0
index 5b2da2cf31a8..41135ea03299 100644
@@ -240,6 +240,27 @@ struct mem_cgroup;
  */
 bool slab_is_available(void);
 
+/**
+ * @align: The required alignment for the objects.
+ * @useroffset: Usercopy region offset
+ * @usersize: Usercopy region size
+ * @freeptr_offset: Custom offset for the free pointer in RCU caches
+ * @use_freeptr_offset: Whether a @freeptr_offset is used
+ * @ctor: A constructor for the objects.
+ */
+struct kmem_cache_args {
+	unsigned int align;
+	unsigned int useroffset;
+	unsigned int usersize;
+	unsigned int freeptr_offset;
+	bool use_freeptr_offset;
+	void (*ctor)(void *);
+};
+
+struct kmem_cache *kmem_cache_setup(const char *name, unsigned int object_size,
+				    struct kmem_cache_args *args,
+				    slab_flags_t flags);
+
 struct kmem_cache *kmem_cache_create(const char *name, unsigned int size,
 			unsigned int align, slab_flags_t flags,
 			void (*ctor)(void *));
mm/slab_common.c +65 -37

170 lines not on this page · raw message

Signature · 1 line
2.45.2
author maintainer
#2 lore raw

RFC PATCH 2/6 slab: port KMEM_CACHE() to kmem_cache_setup()

In reply to Christian Brauner

Make KMEM_CACHE() use the new kmem_cache_setup() helper.

---
 include/linux/slab.h | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
include/linux/slab.h +7 -3
index 41135ea03299..79f4799b7083 100644
@@ -283,9 +283,13 @@ int kmem_cache_shrink(struct kmem_cache *s);
  * f.e. add ____cacheline_aligned_in_smp to the struct declaration
  * then the objects will be properly aligned in SMP configurations.
  */
-#define KMEM_CACHE(__struct, __flags)					\
-		kmem_cache_create(#__struct, sizeof(struct __struct),	\
-			__alignof__(struct __struct), (__flags), NULL)
+#define KMEM_CACHE(__struct, __flags)                                   \
+	kmem_cache_setup(#__struct, sizeof(struct __struct),            \
+			 &(struct kmem_cache_args){                     \
+				 .align = __alignof__(struct __struct), \
+				 .ctor = NULL,                          \
+			 },                                             \
+			 (__flags))
 
 /*
  * To whitelist a single field for copying to/from usercopy, use this
Signature · 1 line
2.45.2
author maintainer
#3 lore raw

RFC PATCH 3/6 slab: port KMEM_CACHE_USERCOPY() to kmem_cache_setup()

In reply to Christian Brauner

Make KMEM_CACHE_USERCOPY() use kmem_cache_setup().

---
 include/linux/slab.h | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)
include/linux/slab.h +17 -12
index 79f4799b7083..52928474e6a1 100644
@@ -283,24 +283,29 @@ int kmem_cache_shrink(struct kmem_cache *s);
  * f.e. add ____cacheline_aligned_in_smp to the struct declaration
  * then the objects will be properly aligned in SMP configurations.
  */
-#define KMEM_CACHE(__struct, __flags)                                   \
-	kmem_cache_setup(#__struct, sizeof(struct __struct),            \
-			 &(struct kmem_cache_args){                     \
-				 .align = __alignof__(struct __struct), \
-				 .ctor = NULL,                          \
-			 },                                             \
+#define KMEM_CACHE(__struct, __flags)						\
+	kmem_cache_setup(#__struct,						\
+			sizeof(struct __struct),				\
+			 &(struct kmem_cache_args) {				\
+				 .align = __alignof__(struct __struct),		\
+				 .ctor = NULL,					\
+			 },							\
 			 (__flags))
 
 /*
  * To whitelist a single field for copying to/from usercopy, use this
  * macro instead for KMEM_CACHE() above.
  */
-#define KMEM_CACHE_USERCOPY(__struct, __flags, __field)			\
-		kmem_cache_create_usercopy(#__struct,			\
-			sizeof(struct __struct),			\
-			__alignof__(struct __struct), (__flags),	\
-			offsetof(struct __struct, __field),		\
-			sizeof_field(struct __struct, __field), NULL)
+#define KMEM_CACHE_USERCOPY(__struct, __flags, __field)					\
+	kmem_cache_setup(#__struct,							\
+			sizeof(struct __struct),					\
+			&(struct kmem_cache_args) {					\
+				.align = __alignof__(struct __struct),			\
+				.useroffset = offsetof(struct __struct, __field),	\
+				.usersize = sizeof_field(struct __struct, __field),	\
+				.ctor = NULL,						\
+			},								\
+			(__flags))
 
 /*
  * Common kmalloc functions provided by all allocators
Signature · 1 line
2.45.2
author maintainer
#4 lore raw

RFC PATCH 4/6 file: port to kmem_cache_setup()

In reply to Christian Brauner

Replace the custom kmem_cache_create_rcu() method with a regular call to the new kmem_cache_setup() function.

---
 fs/file_table.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
fs/file_table.c +9 -3
index 3ef558f27a1c..c20066b3f677 100644
@@ -511,9 +511,15 @@ EXPORT_SYMBOL(__fput_sync);
 
 void __init files_init(void)
 {
-	filp_cachep = kmem_cache_create_rcu("filp", sizeof(struct file),
-				offsetof(struct file, f_freeptr),
-				SLAB_HWCACHE_ALIGN | SLAB_PANIC | SLAB_ACCOUNT);
+	filp_cachep = kmem_cache_setup("filp", sizeof(struct file),
+		&(struct kmem_cache_args) {
+			.use_freeptr_offset = true,
+			.freeptr_offset = offsetof(struct file, f_freeptr),
+		},
+		SLAB_HWCACHE_ALIGN |
+		SLAB_PANIC |
+		SLAB_ACCOUNT |
+		SLAB_TYPESAFE_BY_RCU);
 	percpu_counter_init(&nr_files, 0, GFP_KERNEL);
 }
 
Signature · 1 line
2.45.2
author maintainer
#5 lore raw

RFC PATCH 5/6 slab: remove kmem_cache_create_rcu()

In reply to Christian Brauner

Since we have kmem_cache_setup() and have ported kmem_cache_create_rcu() users over to it is unused and can be removed.

---
 include/linux/slab.h |  3 ---
 mm/slab_common.c     | 42 ------------------------------------------
 2 files changed, 45 deletions(-)
include/linux/slab.h +0 -3
index 52928474e6a1..7d79410996b8 100644
@@ -269,9 +269,6 @@ struct kmem_cache *kmem_cache_create_usercopy(const char *name,
 			slab_flags_t flags,
 			unsigned int useroffset, unsigned int usersize,
 			void (*ctor)(void *));
-struct kmem_cache *kmem_cache_create_rcu(const char *name, unsigned int size,
-					 unsigned int freeptr_offset,
-					 slab_flags_t flags);
 void kmem_cache_destroy(struct kmem_cache *s);
 int kmem_cache_shrink(struct kmem_cache *s);
 
mm/slab_common.c +0 -42

48 lines not on this page · raw message

Signature · 1 line
2.45.2
author maintainer
#6 lore raw

RFC PATCH 6/6 io_uring: port to kmem_cache_setup()

In reply to Christian Brauner

Replace kmem_cache_create_usercopy() with a call to the new kmem_cache_setup() function.

---
 io_uring/io_uring.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
io_uring/io_uring.c +9 -6
index 3942db160f18..092a43a63236 100644
@@ -3722,12 +3722,15 @@ static int __init io_uring_init(void)
 	 * range, and HARDENED_USERCOPY will complain if we haven't
 	 * correctly annotated this range.
 	 */
-	req_cachep = kmem_cache_create_usercopy("io_kiocb",
-				sizeof(struct io_kiocb), 0,
-				SLAB_HWCACHE_ALIGN | SLAB_PANIC |
-				SLAB_ACCOUNT | SLAB_TYPESAFE_BY_RCU,
-				offsetof(struct io_kiocb, cmd.data),
-				sizeof_field(struct io_kiocb, cmd.data), NULL);
+	req_cachep = kmem_cache_setup("io_kiocb", sizeof(struct io_kiocb),
+		&(struct kmem_cache_args) {
+			.useroffset = offsetof(struct io_kiocb, cmd.data),
+			.usersize = sizeof_field(struct io_kiocb, cmd.data),
+		},
+		SLAB_HWCACHE_ALIGN |
+		SLAB_PANIC |
+		SLAB_ACCOUNT |
+		SLAB_TYPESAFE_BY_RCU);
 	io_buf_cachep = KMEM_CACHE(io_buffer,
 					  SLAB_HWCACHE_ALIGN | SLAB_PANIC | SLAB_ACCOUNT);
 
Signature · 1 line
2.45.2
#7 lore raw
In reply to Christian Brauner
Quoted 11 lines · Christian Brauner

Hey,

As discussed last week the various kmem_cache_*() functions should be replaced by a unified function that is based around a struct, with only the basic parameters passed separately.

Vlastimil already hinted that he would like to keep core parameters out of the struct: name, object size, and flags. I personally don't care much and would not object to moving everything into the struct but that's a matter of taste and I yield that decision power to the maintainer.

Yeah the reason I suggested that is the new struct contains rarely needed arguments, so most usages won't have to instantiate it and thus look a bit nicer.

Quoted 9 lines

Here's an RFC built for a kmem_cache_setup() based on struct kmem_cache_args.

The choice of name is somewhat forced as kmem_cache_create() is taken and the only way to reuse it would be to replace all users in one go. Or to do a global sed/kmem_cache_create()/kmem_cache_create2()/g. And then introduce kmem_cache_setup(). That doesn't strike me as a viable option.

Quoted 5 lines

If we really cared about the *_create() suffix then an alternative might be to do a sed/kmem_cache_setup()/kmem_cache_create()/g after every user in the kernel is ported. I honestly don't think that's worth it but I wanted to at least mention it to highlight the fact that this might lead to a naming compromise.

I think using macros would allow us to have kmem_cache_create() in both the current variant (5 args) and new one (4 args) at the same time? But that's also not ideal so perhaps viable only if we really decided to convert everything sooner than later and drop that temporary compatibility layer.

But perhaps if we're converting, it should be mainly to KMEM_CACHE() as it handles alignment.

Quoted 11 lines

From a cursory grep (and not excluding Documentation mentions) we will have to replace 44 kmem_cache_create_usercopy() calls and about 463 kmem_cache_create() calls which makes for a bit above 500 calls to port to kmem_cache_setup(). That'll probably be good work for people getting into kernel development.

Anyway, I wanted to get an RFC out to get some rough agreement on what the struct should look like, get some bikeshedding on the name done, and what else needs to be massaged as part of this. I think that cache_create() is the deepest we should stuff struct kmem_cache_args into the bowels of slab.

Well, if you wanted to be more adventurous... we could pass it also to __kmem_cache_create(), then remove kmem_cache_open() (move the code to its only caller __kmem_cache_create(), probably another thing not cleaned up after SLAB removal).

And then also pass it to calculate_sizes() and at that point rcu_freeptr_offset can be removed from struct kmem_cache, because we just use the value from kmem_cache_args to calculate s->offset and then we can forget that it was requested specifically.

Quoted 20 lines

---
Christian Brauner (6):
slab: introduce kmem_cache_setup()
slab: port KMEM_CACHE() to kmem_cache_setup()
slab: port KMEM_CACHE_USERCOPY() to kmem_cache_setup()
file: port to kmem_cache_setup()
slab: remove kmem_cache_create_rcu()
io_uring: port to kmem_cache_setup()

 Documentation/core-api/memory-allocation.rst |  10 +-
 fs/file_table.c                              |  12 ++-
 include/linux/slab.h                         |  51 ++++++++---
 io_uring/io_uring.c                          |  15 +--
 mm/slab_common.c                             | 132 ++++++++++++---------------
 5 files changed, 121 insertions(+), 99 deletions(-)

---
base-commit: 6e016babce7c845ed015da25c7a097fa3482d95a
change-id: 20240902-work-kmem_cache_args-e9760972c7d4